Skip to main content

A collection of command line tools for crate devs

Project description

travis-ci Wheel PyPI Version Python Version

A collection of command line tools for Crate. Most of these tools output JSON. In order to pretty-print the output or filter it using jq is recommended.

Target audience are mostly developers of Crate and not users.

Install 💾

Python >= 3.4 is required to use the command line tools.

Install them using pip:

pip install cr8

Usage 🎠

The main binary is called cr8 which contains a couple of sub-commands.

Use cr8 -h or cr8 <subcommand> -h to get a more detailed usage description.

An example using cr8:

> echo '{"name": "Arthur"}' | cr8 json2insert mytable
('insert into mytable (name) values (?)', ['Arthur'])

The included sub-commands are described in more detail below:

Sub-commands

timeit

A tool that can be used to measure the runtime of a given SQL statement on a cluster:

> cr8 timeit cluster.hostname:4200 --stmt "select name from sys.cluster"

fill_table

A tool that can be used to fill a table with random data. The script will generate the records using faker.

For example given the table as follows:

create table demo (
    name string,
    country string
);

The following command can be used to insert 100k records:

> cr8 fill-table localhost:4200 demo 100000

It will automatically read the schema from the table and map the columns to faker providers and insert the give number of records.

(Currently only top-level string columns are supported)

json2insert

json2insert generates an insert statement from a JSON string:

> echo '{"name": "Arthur"}' | cr8 json2insert mytable
('insert into mytable (name) values (?)', ['Arthur'])

If a Crate host is provided the insert statement will be executed as well.

blobs

A tool to upload a file into a blob table:

> cr8 upload crate.cluster:4200 blobtable /tmp/screenshot.png

bench.sh

A wrapper script that combines timeit with json2insert to measure the runtime of a query and insert the result into the benchmarks table which can be created using the sql/benchmarks_table.sql file and crash:

./bench.sh "select * from rankings limit 100" mycratecluster.hostname:4200 mycratecluster.hostname:4200

Where the first hostname is used to benchmark the query and the second hostname is used to store the results.

(this script also requires jq to be installed)

perf_regressions

A tool which will re-run all queries recorded with the bench.sh script. It will record the runtimes again and output the new runtimes:

> cr8 find-perf-regressions \
        cluster.to.benchmark:4200 \
        cluster.with.log.table:4200

Development 😕

Tests are run using python setup.py test.

To get a sandboxed environment with all dependencies installed one can either use venv or buildout:

venv

Create a new virtualenv using venv and active it:

python -m venv .venv
source .venv/bin/activate

Install the cr8 package using pip:

python -m pip install -e .

Run cr8:

cr8 -h

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cr8-0.3.0.tar.gz (11.5 kB view hashes)

Uploaded Source

Built Distribution

cr8-0.3.0-py3-none-any.whl (12.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page