No project description provided
Project description
Annotations
This repository contains a framework for generating annotations for command invocations. It comprises a parser which turns a string into a command invocation data structure. For the time being, there are two sets of annotation generators:
- input-output information which specifies how a command invocation interacts with the files, pipes, stdin, stdout, etc.
- parallelizability information which describes how a command invocation can be parallelized - containing information about how to split inputs, mappers and aggregators, etc.
Command-line tool
main.py
contains a command line tool which, provided a command invocation returns:
- the parsed command invocation data structure
- the input-output information generated
- the parallelizability information generated
Adding an annotation
- Add the command in the dictionary in (https://github.com/binpash/annotations/blob/main/pash_annotations/annotation_generation/AnnotationGeneration.py#L13)
- Add a json file with the command flags in (https://github.com/binpash/annotations/tree/main/pash_annotations/parser/command_flag_option_info/data). This could be used to generate a first version of it: (https://github.com/binpash/annotations/blob/main/pash_annotations/parser/command_flag_option_info/manpage-to-json.sh).
- Add an
InputOutputInfoGeneratorXXX.py
in (https://github.com/binpash/annotations/tree/main/pash_annotations/annotation_generation/annotation_generators) - (Optionally) add a
ParallelizabilityInfoGeneratorXXX.py
in (https://github.com/binpash/annotations/tree/main/pash_annotations/annotation_generation/annotation_generators)
Parser
Use command_flag_option_info JSON files to parse xbd-type terminal commands.
Will split on spaces (" "
) and equal signs ("="
).
Flag and Option Information
The folder command_flag_option_info contains [command_name].json files with list of flags and options for each command.
For arguments that have two options (e.g. -a
and --all
), store them as a pair in the format [short version, long version].
In addition, we store here in which way an argument is accessed if applicable, e.g., if it is a file.
We also have a regex-based script that can be used to generate initial JSON files with parsed arguments. Since there is no standard for man-pages, the quality of results varies but it usually provides a good skeleton and saves quite some time.
Annotation Generation
Currently, annotation generators for input-output information and parallelizability information has been implemented.
Each annotation generator implements a specific generator interface (e.g., InputOutputInfoGenerator_Interface.py
) which specializes a more general generator interface (Generator_Interface.py
).
The general generator interface contains functions that help to check conditions on the command invocation while
the more specific generator interface provides functionality to change the respective information (object) generated.
Terms
- flag = takes no arguments, e.g.
--verbose
- option = takes arguments, e.g.
-n 10
- operand = argument with no flag, e.g.
input.txt
Coding
typing
We strive to use types and typecheck with pyright
(v1.1.232).
This does not only help to catch bugs but shall also help future developers to understand the code more easily.
tests
Use pytest
to run tests.
It will run all tests found (recursively) in the current directory.
imports
For clean imports, we add empty __init__.py
modules in all non-root directories.
Thus, pytest
will add the root directory to sys.path and
we can import modules by prefixing the path from there.
For instance, to import Parallelizer.py
, we use
from annotation_generation.parallelizers.Parallelizer import Parallelizer
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pash-annotations-0.2.2.tar.gz
.
File metadata
- Download URL: pash-annotations-0.2.2.tar.gz
- Upload date:
- Size: 40.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9663892aaf6613f785a61377c9c27ec03d586457eb6b739b26059f095317074 |
|
MD5 | 9f3a9a0b96c78ade4f927b3b76b668c3 |
|
BLAKE2b-256 | 938060a3ef91d92a554bf0d871513932350da65dd34a8bd0140773831e21e239 |
File details
Details for the file pash_annotations-0.2.2-py3-none-any.whl
.
File metadata
- Download URL: pash_annotations-0.2.2-py3-none-any.whl
- Upload date:
- Size: 82.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0f203feefb7f09fbd8d6357ccf3c2557ebb650689db02218d78e5341e0ec8bbe |
|
MD5 | 03264a58ecfa1ce1492efb4c3bab1cda |
|
BLAKE2b-256 | 003392db121cc855d25559695aae329e75b891a18e67ff34c787d7eebe79532a |