kojen 1.2.30

Code generation tools.

Description

Kojen is a protocol (C++), state-machine(C++/C#/Python) and Class (C++/C# from UML) code-generation tool written in Python and existing within the Python Package Index (PyPi). Python >= 3.6.8 is supported. To install run python -m pip install kojen.

The default generated state-machine uses the fast and lightweight single-header-file-only boost sml. It is linked to the repository as a git sub-module for an out-the-box solution. This is suitable for both embedded and PC applications using C++14 and higher.

For state-machines and software-architecture-from-UML-model : user-code is preserved within special comment tags. Should you change the name of tags that had code within them, on the next generation a new text file prepended with .Lostcode will be output next to the original source file with all the code that was lost. The main concept is that code-generation is a manually-initiated process and exists within a hand-crafted source-repository. The intention is to be able to manually program in all auto-generated state-machine / Class Diagram output source files, and preserve hand-crafted code within those files when system requirements change. Its not intended to disassociate the programmer from the code (like most CASE tools). Typically after the Design phase code-generation is done first in the initial implementation phase (and maybe a few more times until things are as they should be) and then development is done the usual way. Naturally in time further changes/additions can be done in the model (Updating the 'diagrams' first) and their changes propagated to code by generation : this also allows for updates to the design (from Model -> Code) in a non-intrusive manner. Protocols and their unit tests are the only files which require no hand-coding.

Unit testing : Tests for protocols are generated, and tests for state-machines too (but those require some hand-coding, code preservation applies). The included testing framework is extremely light-weight and great for embedded (and PC) applications : minunit is used for this and it is also linked to the repository as a git sub-module. The original fork of minunit is a C-based single header file, but some improvements have been made (not possible with C) so it now consists of a single header and a single cpp file. It contains a test-runner which will run all registered test-suites.

It currently supports Windows/Linux/MacOS and FreeRTOS and includes multi-threading components (threads/queues/dispatchers/fixed-block-memory-allocators as well as stl-with-fixed-block-mem-allocator wrappers for no spurious heap usage and much faster STL containers) that works on all platforms by including CMake configuration files in generated outputs.

Message Queues use move-semantics and are thus platform independant and adding a new RTOS will only require adding a 'condition variable' like synchronization mechanism for that RTOS (the provided example of how it is done with FreeRTOS can be followed).

Quickstart

Example folder

The git repository has a fully functional example (which is used in the CI), including CMake integration. An executable is generated and that runs all unit tests. Feel free to go straight there. The example is described below.

Protocol

Input

In a folder where you will execute your project from, create the interface definition file MyInterface.py.

In your interface definition file MyInterface.py create a function CreateInterface() that returns a kojen.Interface object.

Here is an example of what is permissible in an interface. For messages, make sure each has a unique message-id (second parameter after the name in the constructor)

from kojen.kojentypes import *

def CreateInterface():
    sCustomStruct = Struct('sCustomStruct')
    sCustomStruct.AddType('m_Member1','uint16')
    sCustomStruct.AddType('m_Member2','uint16')
    sCustomStruct.AddType('m_Member3','uint32')

    MsgSomeCMD = Message('MsgSomeCMD',0x01)
    MsgSomeCMD.AddType('m_Member1','uint8')

    MsgSomeCMDRSP = Message('MsgSomeCMDRSP',0x02)
    MsgSomeCMDRSP.AddStruct('mStructMember1',sCustomStruct)
    MsgSomeCMDRSP.AddType('m_Member2','uint8')

    MsgSomeREQ = Message('MsgSomeREQ',0x03)
    MsgSomeREQ.AddType('m_Member1','uint8')

    #Array of type : Not supported by ARM due to dynamic memory allocation requirements
    MsgSomeREQRSP = Message('MsgSomeREQRSP',0x04)
    MsgSomeREQRSP.AddStruct('mStructMember1',sCustomStruct)
    MsgSomeREQRSP.AddType('m_Member2','uint8')
    MsgSomeREQRSP.AddArrayOfType('mArrayMember3','double')


    MsgSomeUnsolicitedData = Message('MsgSomeUnsolicitedData',0x05)
    MsgSomeUnsolicitedData.AddArrayOfStruct('mStructArrayMember1',sCustomStruct)

    #
    # Includes : Enums and defines ...
    #
    Type = Enum("Type")
    Type.Add("kOne", 0)
    Type.Add("kTwo", 1)

    Revision = Enum("Revision")
    Revision.Add("kVersion1", 0)
    Revision.Add("kVersion2", 1)
    Revision.Add("kVersion3", 2)

    interface = Interface('IMyInterfaceIO')

    interface.AddEnum(Type)
    interface.AddEnum(Revision)

    interface.AddHashDefine("THREE", 3)
    interface.AddHashDefine("PI", 3.14159265359)

    interface.AddStruct(sCustomStruct)
    interface.AddMessage(MsgSomeCMD)
    interface.AddMessage(MsgSomeCMDRSP)
    interface.AddMessage(MsgSomeREQ)
    interface.AddMessage(MsgSomeREQRSP)
    interface.AddMessage(MsgSomeUnsolicitedData)

    return interface

Then, create a file Generate.py.

import kojen.Generate as Generate
import os

author = "[e-mail]"
group  = "MY_PACKAGE"
brief  = "An example demonstrating protocol code-generation abilities."
namespacename = "ExampleIO"
classname = "CExampleIF"
declspec = "" # your define for symbol export/import in C++ e.g. MY_PROJECT_API
outputdir = os.path.join(os.path.abspath(os.path.dirname(__file__)), os.path.join("autogen","protocol"))
protocolfile = os.path.join(os.path.abspath(os.path.dirname(__file__)), "MyInterface.py")
Generate.Protocol(outputdir, protocolfile, namespacename, classname, declspec, author, group, brief)

Execute Generate.py using python. Your C++ code should be in the folder ./autogen/protocol.

Please note : you may include this code in a library, and thus want to export some of the generated features. Typically somewhere in a common header of your C++ code, you would have declared something like

#ifdef MY_PROJECT_EXPORTS
#	define MY_PROJECT_API __declspec(dllexport)
#else
#	define MY_PROJECT_API __declspec(dllimport)
#endif

You should thus pass MY_PROJECT_API as the declspec argument to the generator.

Output

There will be several files in the output folder.

• allplatforms (folder containing required framework files)
• CExampleIF.h *(contains all factories to create messages which automatically and correctly fill in the headers, as well as serialization to/from bytestreams.)
• CExampleIF.cpp
• CExampleIFDefines.h *(contains all enums and defines)
• CExampleIFReceiver.h *(contains a derivable class that received fully constructed messages. Override receive functions to handle them as per need.)
• CExampleIFReceiver.cpp
• CExampleIFStructs.h *(contains all structs)
• CExampleIFTests.cpp *(contains all unit tests)
• CExampleIFTransmitter.h *(contains all functions to transmit fully constructed messages. Highly advices to use the autogenerated factories.)
• CExampleIFTransmitter.cpp
• MsgHeader.h

The following class diagram explains roughly where focus should be when implementing for actual integration into the real system.

IConnection is the interface to derive from for your actual physical layer for an embedded platform. If you use boost on x86/x64, in the 'allplatforms' directory exist classes for substituting this with a TCPConnection, UDPConnection and SerialPort. Have a good look at the unit tests, this shows roughly how to derive from it and connect everything up, albeit creating a loopback connection, and implementing uncessary routing as the interface for the tests have a special requirement.

A summary to link it up:

• Create a derived IConnection that knows how to send and receive physical layer data.
• Create a generated Transmitter, passing it a pointer to this connection.
• Create a generated Receiver
• Set the Receiver to the derived IConnection. You will then get messages in the message receiving functions...override the correct ones to handle them.
• Use the Transmitter to send.

State Machine in C++

Requrements

• Don't forget to update your git submodules when checking out.

Input

The following diagram defines (roughly) a state machine for a CD player.

This state machine can be represented by the following transition table in python (format : 'startstate', 'event', 'nextstate', 'action', 'guard'):

#
# StartState    Event                     NextState     Action                 Guard
#
 'StateStop',  'EventOpen',              'StateOpen',  'OnOpenDrive',         'None'
 'StateStop',  'EventPlay',              'StatePlay',  'OnPlayTrack',         'GuardCDInside'
 'StateOpen',  'EventOpen',              'StateStop',  'OnCloseDrive',        'None'
 'StatePlay',  'EventPlay',              'StatePause', 'OnPause',             'None'
 'StatePlay',  'EventEndOfTrack',        'None',       'OnPlayNextTrack',     'GuardCDHasMoreTracks'
 'StatePlay',  'EventEndOfTrack',        'StateStop',  'OnStop',              'GuardCDHasNoMoreTracks'
 'StatePlay',  'EventSkipNextTrack',     'None',       'OnPlayNextTrack',     'GuardCDHasMoreTracks'
 'StatePlay',  'EventSkipPreviousTrack', 'None',       'OnPlayPreviousTrack', 'GuardCDHasPreviousTrack'
 'StatePlay',  'EventStop',              'StateStop',  'OnStop',              'None'
 'StatePause', 'EventPlay',              'StatePlay',  'OnPlayTrack',         'None'
 'StatePause', 'EventAfter10Minutes',    'StateStop',  'OnStop',              'None'

Create a file Generate.py (or add your state machine to the one already present in your project) :

from kojen import Generate
from kojen.kojentypes import *

#
# Generate Statemachine (from TransitionTable)
#

# For playing a track the tracknumber should be parameterized.
EventPlay = Struct('EventPlay')
EventPlay.AddType('m_track_no','uint16_t')

eventsinterface = Interface('params')
eventsinterface.AddStruct(EventPlay)

# Optional features
eventsinterface.AddUserTag("StateMachineThread", 1) # 0 = SM runs on caller thread, 1 = SM runs on its own thread. Default is 1.
eventsinterface.AddUserTag("Verbose", 1)            # 0 = No printouts, 1 = printouts. Default is 1.

# Transition Table.
transition_table = []
#                         StartState    Event                     NextState     Action                 Guard
transition_table.append(['StateStop',  'EventOpen',              'StateOpen',  'OnOpenDrive',         'None'                   ])
transition_table.append(['StateStop',  'EventPlay',              'StatePlay',  'OnPlayTrack',         'GuardCDInside'          ]) 
transition_table.append(['StateOpen',  'EventOpen',              'StateStop',  'OnCloseDrive',        'None'                   ]) 
transition_table.append(['StatePlay',  'EventPlay',              'StatePause', 'OnPause',             'None'                   ])
transition_table.append(['StatePlay',  'EventEndOfTrack',        'None',       'OnPlayNextTrack',     'GuardCDHasMoreTracks'   ])
transition_table.append(['StatePlay',  'EventEndOfTrack',        'StateStop',  'OnStop',              'GuardCDHasNoMoreTracks' ])
transition_table.append(['StatePlay',  'EventSkipNextTrack',     'None',       'OnPlayNextTrack',     'GuardCDHasMoreTracks'   ]) 
transition_table.append(['StatePlay',  'EventSkipPreviousTrack', 'None',       'OnPlayPreviousTrack', 'GuardCDHasPreviousTrack'])
transition_table.append(['StatePlay',  'EventStop',              'StateStop',  'OnStop',              'None'                   ])
transition_table.append(['StatePause', 'EventPlay',              'StatePlay',  'OnPlayTrack',         'None'                   ])
transition_table.append(['StatePause', 'EventAfter10Minutes',    'StateStop',  'OnStop',              'None'                   ])

author = "[e-mail]"
group  = "MY_PACKAGE"
brief  = "An example demonstrating state-machine code-generation abilities."

eventsinterface = None
namespacename = "CDPlayerSM"
statemachinenameprefix = "CDPlayer"
declspec = ""
outputdir = "autogen/state_machine"
templatedir = "" # defaults are for 'SML'
Generate.StateMachine(outputdir, transition_table, eventsinterface, namespacename, statemachinenameprefix, declspec, author, group, brief, templatedir)

Execute Generate.py using python. Your C++ code should be generated to the configured folder ./autogen/state_machine.

Please note : you may may wish to parameterize events in your statemachine. An event is of type kojen.Struct in a kojen.Interface object. Simply follow the quickstart for protocols, creating and returning an interface : your Event structs need to have the exact same name as defined in the transition table. You should thus pass your interface object as the eventsinterface argument to the generator instead of None.

Output

There will be several files in the output folder.

• allplatforms (folder containing framework files for standalone solution)
• CDPlayerStateMachine.h
• CDPlayerStateMachineImpl_SML.cpp (all state machine code generated for boost-sml)
• ICDPlayerController.h
• Test.CDPlayerStateMachine.cpp

The following class diagram explains roughly where focus should be when implementing for test-driven development as well as for actual integration into the real system.

White classes are auto-generated, and have code-preservation features. Grey is what you have to manually create, and implement for integration into your real system.

All required functionality for your system should be implemented in the Controller classes, being the base-class ICDPlayerController, the derived C_My_CDPlayerController and the derived test class CTestCDPlayerController.

It is critical that as much of the 'business logic' is put into the base class IPlayerController, and tested using the CTestCDPlayerController in the unit-tests as possible, such that the actual integrated C_My_CDPlayerController uses unit-tested business logic, with a good example of how its intended to work in the unit tests...as well as for regression testing and continuous integration.

As the state-machine-implementation uses the controller base class, the following construction is valid for unit tests and system integration by substituting the correct controller:

  CTestCDPlayerController controller;
  auto* sm = ICDPlayerStateMachine::Create(controller);

All state queries and event triggering are done using the sm instance.

Due to the use of move-semantics, parameterized event structs are unique_ptr's (allocated in fixed-blocks for embedded applications) and thus once an event is triggered those pointers are no longer valid in the calling context.

State Machine in C#

Similar to the above example, the only difference is the Generate call to use.

   ...
   Generate.StateMachine_CSHARP(...)
   ...

State Machine in Python

Similar to the above example, the only difference is the Generate call to use.

    ...
   Generate.StateMachine_PYTHON(...)
   ...