mash-shell 0.2.15

A shell and other utils

Docs: MAIN | Shell | Reference | Library Docs

Mash | My Automation Shell

A shell that can be used to for automation and (REST) resource discovery. It can be used as a cli, repl. It exposes a complete programming language with variables, functions, conditions and pipes. The language can be tailored towards domain-specific applications and it has interoperability with Bash. In addition, this repository contains a library of utilities.

Features:

• A DSL that can interpret user-defined commands: shell.py.
• A file-browser. Query both static datastructures and REST APIs: examples/filesystem.py,
• A REST client to browse APIs with a programmatic yet intuitive interface: examples/discoverable_api.py.
• An object parser which converts JSON data to Python classes: object_parser.py.
• An OAS-generator for Python classes: oas.py.
• A subshell wrapper, to redirect the output of shell scripts: subshell.py.
• A parallelization framework for load testing: parallel.py.

Links

• Mash language reference
• PyPI
• github
• docs

Shell

pip install mash-shell
python -m mash

See src/examples for advances usage examples.

Documentation

| Implementation: Shell

• Language: Reference.

Table of Contents

• Usage
  • CLI
  • Filesystem (CRUD Operations)
  • Usage Examples
• Library & Tools
  • Parallelization Utilities
  • Object Parser

Usage

See reference.

CLI

▶ python -m mash -h
usage: shell.py [-hvsr][-f FILE] [--session SESSION] [cmd [cmd ...]]

If no positional arguments are given then an interactive subshell is started.

positional arguments:
  cmd                   A comma- or newline-separated list of commands

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose
  -s, --safe            Safe-mode. Ask for confirmation before executing commands.
  -f FILE, --file FILE  Read and run FILE as a commands
  -r, --reload          Reload last session
  --session SESSION     Use session SESSION

Filesystem (CRUD Operations)

See examples/filesystem.py and examples/discoverable.py.

Example	Description
cd [DIR]	Change the current working directory. Alias: use
list [DIR]	List the items in a directory. Use the current working directory by default. Alias: l
foreach [DIR]	Iterate over a directory structure. E.g. a resource users/{id}/email.
get NAME	Retrieve a file.
set NAME VALUE	Modify a file.
new NAME [NAME..]	Create new directories.
show [NAME]	Display detailed information about a directory.
cp, mv, rm	Modify files. I.e. copy, move, rename or remove files.

Usage Examples

For real-world examples, see lib.

Example 1

See src/examples/shell_example.py. It shows how to use a user-definnable mapping of custom functions. It uses the library quo to create a user-friendly subshell with autocompletion prompts.

# python3 src/examples/shell_example.py echo 'hello world'
hello world

Example 2: REST Client

Browse through a REST API as if it is mounted as a filesystem.

# python3 src/examples/rest_client_implicit.py

$ cd users # browse some.api.com/v1/users
$ get 2
|       | values             |
|:------|:-------------------|
| email | [e-mail] |
| name  | name_2             |

$ flatten 1 2 >>= get users $ email
[e-mail]
[e-mail]

$ cd -
$ tree
{ 'document': 'https://dummy-api.com/v1',
  'users': { 0: {'email': '[e-mail]', 'name': 'name_0'},
             1: {'email': '[e-mail]', 'name': 'name_1'},
             2: {'email': '[e-mail]', 'name': 'name_2'},
...

Example 3: Commands

Run commands from a file with python src/shell -f FILE or python -m src.shell -f FILE.

# write to file
print A sentence. > out.txt 
!cat out.txt |> export x # save text from file
print $x

y <- shell expr 2 + 2 # store result in variable $y
print "result:" $y # prints "result: 4

shell expr 2 + 2 -> z # store result in variable $z
print "result:" $z # prints "result: 4

Example 4: File System Simulation

Support both static and dynamic data.

With Static Data

See src/examples/filesystem_example.py. This simulates a REST resources with a directory hierarchy. In addition, it provides fuzzy name completion.

$ python src/examples/shell_example.py tree
# example data with dicts and lists
repository = {'worlds': [
    {'name': 'earth',
     'animals': [
         {'name': 'terrestrial',
          'snakes': [{'name': 'python'},
                     {'name': 'cobra'}]},
         {'name': 'aquatic',
          'penquins': [{'name': 'tux'}]}
     ]}]}

# note the autocompletion
$ python src/examples/filesystem.py 'cd world; cd a; cd t; cd snakes; ll'
python
cobra

With Dynamic Data

See examples/discoverable.py.

# list remote/auto-generated data
$ python src/examples/discoverable.py 'ls'
department_805, department_399

# refresh data, then save
$ py src/examples/discoverable.py 'ls ; save'
department_750, department_14

# reload data
$ python src/examples/discoverable.py 'ls'
department_750, department_14

Library & Tools

Repository structure

src
├── examples
├── lib # mash shell scripts
└── mash
    ├── filesystem # Directory-based datastructures: FileSystem, View
    ├── object_parser # Parse JSON objects
    └── shell # Shell, ShellWithFileSystem
test

Setup

Using a Makefile for convenience.

make install
make test

Parallelization Utilities

Some experiments with parallelization, concurrency and asyncio in Python.

Test

Start a dummy server.

python3 src/server.py

Do a simple load test

python3 src/parallel.py -v

Object Parser

• src/object_parser/object_parser.py parses JSON data and instantiate Python objects.
• src/object_parser/oas.py converts domain-models to OAS.

Example

python src/object_parser_example.py

REST API

Server

python src/object_parser_server.py

Client

curl -X 'POST' 'http://localhost:5000/v1/organizations' \
  -H 'Content-Type: application/json' \
  -d '{ "board": [ "string" ], "ceo": "string", "departments": [ { "manager": "string", "teams": [ { "manager": "string", "members": [ "string" ], "team_type": "A", "active": true, "capacity": 0, "value": 0 } ] } ] }'