awk-plus-plus 0.11.0

Declarative Data Orchestration

awk_plus_plus

A domain-specific language designed for data orchestration.

Features

• Fuzzy modern regex engine
• Semantic search
• Orthogonal Persistence based on DuckDB
• Transparent reference with Jsonnet. We plan to execute this future with Dask.
• URL interpreter to manage data sources.

Installation from pip

Install the package with:

pip install awk_plus_plus

CLI Usage

You output your data to JSON with the cti command.

JSONNET support

Hello world

cti i "Hello world" -p -v 4

Jsonnet support

cti i '{"keys":: ["AWK", "SED", "SHELL"], "languages": [std.asciiLower(x) for x in self.keys]}'

URL interpreter

Our step further is the URL interpreter which allows you to manage different data sources with an unique syntax across a set of plugins.

STDIN, STDOUT, STDERR

cti i '{"lines": interpret("stream://stdin?strip=true")}'

Imap

cti i '{"emails": interpret("imap://USER:PASSWORD@HOST:993/INBOX")}'

Keyring

cti i '{"email":: interpret("keyring://backend/awk_plus_plus/email"), "emails": interpret($.email)}'

Files

cti i 'interpret("**/*.csv")'

SQL

cti i 'interpret("sql:SELECT * FROM email")'

Note

This project has been set up using PyScaffold 4.5 and the dsproject extension 0.0.post167+g4386552.