atom-tools 0.5.4

Collection of tools for use with AppThreat/atom.

atom-tools

Collection of tools for use with slices generated by AppThreat/atom.

Install atom

This program does not generate slices; its purpose is to manipulate slices generated by atom. The current documentation for atom is housed in the AppThreat/atom GitHub repository.

Atom can easily be installed from a native image or via npm npm install -g @appthreat/atom.

Atom-tools installation

pip install atom-tools

CLI Usage

Atom-tools uses py-poetry/cleo to construct its command-line interface and therefore uses the same sorts of conventions as the Python package management utility poetry.

To access the commands help menu, enter atom-tools list for a list of available commands.

Individual command options can be accessed with atom-tools help and the command name ( e.g. atom-tools help convert).

Atom Tools (version 0.5.0)

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display help for the given command. When no command is given display help for the list command.
  -q, --quiet           Do not output any message.
  -V, --version         Display this application version.
      --ansi            Force ANSI output.
      --no-ansi         Disable ANSI output.
  -n, --no-interaction  Do not ask any interactive question.
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.

Available commands:
  convert         Convert an atom slice to a different format.
  filter          Filter an atom slice based on specified criteria.
  help            Displays help for a command.
  list            Lists commands.
  validate-lines  Check the accuracy of the line numbers in an atom slice.

Features

Convert

The convert command can be used to output an atom slice in a different format. The current capabilities are limited to processing usages in order to generate endpoints for an openapi 3.x paths object. Future iterations will populate the path item objects with more details based on atom slices.

Description:
  Convert an atom slice to a different format

Usage:
  convert [options]

Options:
  -f, --format=FORMAT              Destination format [default: "openapi3.0.1"]
  -i, --input-slice=INPUT-SLICE  Usages slice file
  -t, --type=TYPE                  Origin type of source on which the atom slice was generated. [default: "java"]
  -o, --output-file=OUTPUT-FILE    Output file [default: "openapi_from_slice.json"]
  -s, --server=SERVER              The server url to be included in the server object.
  -h, --help                       Display help for the given command. When no command is given display help for the list command.
  -q, --quiet                      Do not output any message.
  -V, --version                    Display this application version.
      --ansi                       Force ANSI output.
      --no-ansi                    Disable ANSI output.
  -n, --no-interaction             Do not ask any interactive question.
  -v|vv|vvv, --verbose             Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.

Help:
  The convert command converts an atom slice to a different format.
      Currently supports outputting an OpenAPI 3.x document based on a usages
      slice.

Example

atom-tools convert -i usages.slices.json -f openapi3.0.1 -o openapi_usages.json -t java -s https://myserver.com

Filter

The filter command can be run on its own to produce a filtered slice or used before another command to filter a slice before executing another command against the results.

Filters operate on an inclusive-or basis. If you want to operate on an 'and' basis, chain the filter commands.

Mode

The default mode creates a regular expression from the value given. Fuzzy mode is specified using the -f option and a number between 0-100 indicating how close the result must be to be a match. Note that to exactly match the specified input, you need to either include regex anchors at the beginning and end or use -f 100 (to specify a 100% match).

filter -f 100 --criteria filename=path/to/file/server.ts -i usages.json

filter --criteria filename=^path/to/file/server.ts$ -i usages.json

Regex word boundaries can be used if you only want to be exact about the filename.

filter --criteria filename=\bserver.ts$ -i usages.json

This will filter files named server.ts - without the \b, files like ftpserver.ts would also be matched.

Note: You can search for a file name without including the path if needed and fuzzing ratios will be computed based only on the file name.

Chaining filter commands

The filter command can act on itself by specifying an additional filter command as an argument. This may desirable for certain use cases where one wishes some criteria to be required.

Example

atom-tools filter -i slices.json --criteria filename=myfile -e "filter --criteria resolvedMethod=mymethod,resolvedMethod=mymethod2 convert"

This would be equivalent to

if fileName.contains('myfile') and (resolvedMethod.contains('mymethod') or resolvedMethod.contains('mymethod2')):

Available attributes (not case-sensitive):

• callName
• fileName
• fullName
• name
• resolvedMethod
• signature

attribute	locations
callName	objectSlices.usages.argToCalls, objectSlices.usages.invokedCalls, userDefinedTypes.procedures,
fileName	objectSlices, userDefinedTypes
fullName	objectSlices
name	objectSlices.usages.targetObj, objectSlices.usages.definedBy, userDefinedTypes.fields
resolvedMethod	objectSlices.usages.targetObj, objectSlices.usages.definedBy, objectSlices.usages.argToCalls, objectSlices.usages.invokedCalls, userDefinedTypes.procedures
signature	objectSlices

Criteria syntax

Multiple criteria can be given by using a comma as a separator (no space)

--criteria [attribute]=[value],[attribute2]=[value],...

Usage

Description:
  Filter an atom slice based on specified criteria.

Usage:
  filter [options]

Options:
  -i, --input-slice=INPUT-SLICE  Slice file to filter.
  -c, --criteria=CRITERIA        Filter based on an attribute of the slice. May be a Python regular expression. Please see documentation for syntax.
  -o, --outfile=OUTFILE          File to re-export filtered slice to.
  -f, --fuzz=FUZZ                Minimum percentage to match with the given criteria INSTEAD of using a regex. Must be a number between 0 and 100.
  -e, --execute=EXECUTE          Command to execute after filtering. [default: "export"]
  -h, --help                     Display help for the given command. When no command is given display help for the list command.
  -q, --quiet                    Do not output any message.
  -V, --version                  Display this application version.
      --ansi                     Force ANSI output.
      --no-ansi                  Disable ANSI output.
  -n, --no-interaction           Do not ask any interactive question.
  -v|vv|vvv, --verbose           Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.

Examples

Filter a query

The below will produce endpoints from the server.ts file located within the line number range of 50-70.

atom-tools filter -i usages.slices.json --criteria fileName=server.ts -e "query-endpoints -l 50-70"

Filter with the convert command.

atom-tools filter -i usages.slices.json --criteria fileName=server.ts -e "convert -f openapi3.0.1 -o openapi_usages.json -t java"

The above will produce an OpenAPI document based only on slices generated from server.ts.

Filter based on another attribute Create a filtered json that only includes slices where the resolved method equals "validateSignup". Since no command is specified, the filtered slice will only be written to file.

atom-tools -i usages.slices.json filter --criteria resolvedMethod=validateSignup

Filtering can also be used to exclude. The first example could be changed to exclude server.ts with the following:

atom-tools filter --criteria fileName!=server.ts usages.slices.json convert -f openapi3.0.1 -o openapi_usages.json -t java

Multiple filter criteria may be included. The following example will produce a filtered slice based only on server.ts and router.ts slices.

atom-tools filter --criteria fileName=server.ts,callName=router.ts usages.slices.json

Query Endpoints

Query endpoints generates a list of endpoints and returns the output directly to the console.

Note: To suppress logging messages and ONLY output the results, use --quiet/-q

Examples

Query returning all endpoints, including filenames and line numbers

query-endpoints -i usages.slices -t js

Query returning all endpoints without filenames and line numbers

query-endpoints --sparse -i usages.slices -t js

Query filtering by line number or line number range

query-endpoints -i usages.slices -t js -f 50

query-endpoints -i usages.slices -t js -f 50-70

Query using filter command to target by both filename and line number range

filter -i usages.slices -t js -c filename=server.ts -e "query-endpoints -f 50-70"

Validate Lines

The validate-lines command checks the accuracy of the line numbers reported by atom against your source files.

Description:
  Check the accuracy of the line numbers in an atom slice.

Usage:
  validate-lines [options]

Options:
  -i, --input-slice=INPUT-SLICE  Slice file to validate. [default: "slices.json"]
  -t, --type=TYPE                Origin type of source on which the atom slice was generated. [default: "java"]
  -d, --base-path=BASE-PATH      This should be the same path that was used by atom when the slice was generated.
  -l, --interval=INTERVAL        Try matching within a range. Ex. slice has line number 567, with interval of 5, we check lines 562-572. Use 0 for exact matching. [default: 5]
  -r, --report=REPORT            Output summary to file.  [default: "output.txt"]
  -j, --export-json=EXPORT-JSON  JSON report file to store invalid lines. Include valid lines as well using -v flag.
  -h, --help                     Display help for the given command. When no command is given display help for the list command.
  -q, --quiet                    Do not output any message.
  -V, --version                  Display this application version.
      --ansi                     Force ANSI output.
      --no-ansi                  Disable ANSI output.
  -n, --no-interaction           Do not ask any interactive question.
  -v|vv|vvv, --verbose           Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.
  
Help:
  Validate source file line numbers in an atom usages or reachables slice.

Example

atom-tools validate-lines -t java -j project_json_report.json -i usages.slices.json -d /home/my_project_dir