CAmpPython

This is the C code generator for the DTN Management Architecture (DTNMA). It is part of the larger Asynchronous Network Managment System (ANMS) managed for NASA AMMOS.

       (                 ,&&&.
         )                .,.&&
        (  (              \=__/
            )             ,'-'.
      (    (  ,,      _.__|/ /|
       ) /\ -((------((_|___/ |
     (  // | (`'      ((  `'--|
   _ -.;_/ \\--._      \\ \-._/.
  (_;-// | \ \-'.\    <_,\_\`--'|
  ( `.__ _  ___,')      <_,-'__,'
   `'(_ )_)(_)_)'

This tool uses the YANG representation of an Application Data Model (ADM) to generate code for various purposes. CAmp generates:

• C code for usage in a C-implemented DTNMA agent
  • This generation can also carry over custom functions in existing C files for the ADM, if indicated appropriately in the existing code (see the Round-tripping Section).
• SQL code for usage by the ANMS manager
• ACE input files, for usage with the ARI CBOR Encoder (ACE) Tool

Additional generators may be added to account for use cases outside of the reference DTNMA Agent. Please contact the developers for more information or suggestions. The Architecture Section also provides some explanation of the components of CAmp, and how to incorporate additional generators.

NOTE CAmp largely assumes that the ADM YANG input can be trusted (i.e., CAmp does not go to great lengths to fully sanitize all strings found within the ADM). CAmp does properly escape necessary sequences found in the ADMs tested during development (e.g., apostrophes in object descriptions).

Moreover, the files generated by CAmp don't have cross-dependencies on preprocessor definitions or symbols outside of compilation. The output from CAmp can be placed anywhere in the compilation directory tree, because it doesn't depend on the other files generated by CAMP, outside of CACE and the RefDA.

Development

For development purposes, it is advised to operate within a Python virtual environment, ideally Python 3.11, to help prevent dependency errors later on. You can run the following commands to create and activate your venv:

python3.11 -m venv .venv
source .venv/bin/activate

If you wish to deactivate your venv, simply run deactivate.

Because CAmp utilizes ACE to run properly, one must install and set up ACE first before attempting to install and run CAmp. Set-up instructions for ACE can be found in the README here. To install ACE via pip and point to a specific repo and branch (in this case, the main branch) like so:

pip install git+https://github.com/JHUAPL-DTNMA/dtnma-ace.git@main

To install development and test dependencies for this project, run from the root directory (possibly under sudo if installing to the system path):

pip3 install -r <(python3 -m piptools compile --extra test pyproject.toml 2>&1)

If this command fails, you may have to install the pip-tools package first and then run two separate commands like so:

pip3 install pip-tools
python3 -m piptools compile --extra test pyproject.toml
pip3 install -r requirements.txt

If the python3 -m piptools compile --extra test pyproject.toml command fails due to the following error: "Could not find a version that satisfies the requirement dtnma-ace (from versions: none)", then one solution is to remove the dtnma-ace dependency from the pyproject.toml file via a text editor and then try the command again.

To execute all unit tests, importing CAMP from the source tree, run:

PYTHONPATH=src/ python3 -m pytest -v --cov=camp test

To install the project itself from source run:

pip3 install .

If you are still encountering installation errors, you may need to update the submodules:

git submodule update --init --recursive

View Usage Options for CAmp

    camp -h

Basic Usage

The camp tool takes a YANG representation of an ADM for a network protocol as input and calls each of the included generators to generate files for the ADM.

For information on how to format an ADM encoded as a YANG module file, please consult the Application Management Model IETF draft.

Given the YANG representation of the ADM, run camp with:

   camp <adm.yang>

Name Registry

If you're generating files for a new ADM, you may see an error similar to the following:

[Error] this ADM is not present in the name registry. Pass integer value via
command line or set manually in name_registry.cfg

This is because the name of the ADM is not yet present in the camp Name Registry. To solve this, pass the nickname value for the ADM to camp via the -n command line option:

    camp <adm.yang> -n <value>

You can also use the -n option to make camp use a different nickname for an ADM that is present in the camp Name Registry. For example,

    camp bp_agent.yang -n 23

Will generate bp_agent files with a nickname of 23 instead of the registered value of 2. To make these changes permanent (or to add a new ADM to the name registry), pass the -u flag to camp:

    camp <adm.yang> -n <value> -u

Output

During a successful camp execution, output similar to the following will be printed to STDOUT.

INFO:camp.tools.camp:Loading <path_to_yang_file>/<adm.yang>  ...
INFO:camp.tools.camp:Finished loading ADM <adm>
INFO:camp.tools.camp:Generating files under ./
INFO:camp.tools.camp:Generating <adm>.h ...
INFO:camp.tools.camp:done.
INFO:camp.tools.camp:Generating <adm>.c ...
INFO:camp.tools.camp:done.
INFO:camp.tools.camp:Generating <adm>.sql
INFO:camp.tools.camp:done.
[ End of CAmpPython Execution ]

This output shows that camp completed a successful generation of each of the files listed. Use the -oflag with camp to redirect output to a directory different from the current one.

    camp <adm.yang> -o <output_directory>

If the path at <output_directory> does not already exist, camp will create it.

Camp will not delete any existing directory structure, but files present in the output directories with the same name as generated files will be overwritten.

Custom Code and Round-tripping

The adm_<adm>_impl.c and adm_<adm>_impl.h files generated for the DTNMA agent contain functions whose bodies cannot be automatically generated with knowledge of the ADM alone. When generated, these fuctions are marked with tags similar to the following:

    /*
     * +----------------------------------------------------------------------+
     * |START CUSTOM FUNCTION <function_name> BODY
     * +----------------------------------------------------------------------+
     */	
    /*
     * +----------------------------------------------------------------------+
     * |STOP CUSTOM FUNCTION <function_name> BODY	
     * +----------------------------------------------------------------------+
     */	

Additionally, the user may wish to add additional custom functions and/or header files to these generated files. To allow re-generation of camp files with minimal re-work for custom code in these files, camp has a 'roundtripping' feature that allows preservation of these custom additions in subsequent file generations.

The roundtripping feature in camp will save any code in the file that falls between camp custom tags, and will add it to the newly-generated version of the file. Example usage:

    camp <adm.yang> -c <path_to_existing_impl.c> -h <path_to_existing_impl.h>

The resulting generated impl.c and impl.h files will contain the custom code from the impl.c and impl.h files passed to camp.

Current acceptable custom tags are:

• custom function body (example above)
• custom includes (/* [START|STOP] CUSTOM INCLUDES HERE */)
• custom functions (/* [START|STOP] CUSTOM FUNCTIONS HERE */)

For custom function bodies, the <function_name> included in the custom function tag must be the same as the one used in the ADM for the custom function to be copied over to the correct area of the new file.

CAmp Architecture

• src/camp/ - contains all of the source code for camp
  • tools/camp.py - Main script of camp. This script calls all necessary generators and handles user input
  • data/name_registry.cfg - Initial name registry configuration file installed with camp.
  • generators/ - All generator scripts and their utility functions
    • base.py - Generates ACE tool input file
    • create_agent_c.py - Generates agent file (C code) for usage in NASA ION
    • create_gen_h.py - Generates the shared header file needed for NASA ION
    • create_impl_c.py - Generates the implementation file (C code) for usage in DTNMA C agent
    • create_impl_h.py - Generates the header file for the implementation file created by create_impl_c.py
    • create_mgr_c.py - Generates the manager file for usage in DTNMA C agent
    • create_sql.py - Generates an SQL file for usage with DTNMA C agent stored procedures
    • lib/ - Library functions for generating commonly-used patterns and accessing portions of the ADM.
      • campch.py - library functions commonly needed specifically for C code generators.
      • campch_roundtrip.py - round-tripping functions
      • campsettings.py - initializes various global variables for camp (enumerations for portions of the ADM, etc.)
      • camputil.py - utility functions for parsing the YANG input file and creating ARIs. Contains the Retriever class, which is used by all generators to access ADM data

Adding Generators

To add a new generator to camp, create a python script that creates the file and add it to the CAmpPython/generators/ directory. Then, in CAmpPython.py, import the new generator and add it to the file generation code of the main() function (starting at line 105).

All generators should:

• define a create() method as their main function, which takes as its first and second argument:
  1. a Retriever object (pre-populated with the ADM passed to camp)
  2. a string that represents the path to the output directory
• utilize the Retriever object to access fields of the ADM module
• place generated file(s) in the output directory passed as the second argument to the create() function (the generator may choose to make a sub-directory in the output directory)

Contributing

To contribute to this project, through issue reporting or change requests, see the CONTRIBUTING document.