Skip to main content

Column store implementation for ftm data based on clickhouse

Project description

ftm-columnstore on pypi Python test and package pre-commit Coverage Status GPL-3.0 License

ftm-columnstore

Column-store (based on Clickhouse) implementation for nomenklatura statement-based store for ftm entities.

Minimum Python version: 3.11

It is compatible as a store for ftmq

Usage

Set up a running clickhouse instance (pointed to via DATABASE_URI env var, default: localhost), for developing purposes this could work:

make clickhouse

Then initialize the required table schema:

ftmcs init

Or drop existing data and recreate:

ftmcs init --recreate

When using the make clickhouse command, you can play around with SQL queries in your browser: http://127.0.0.1:8123/play

Command-line usage

# Insert a bunch of FtM entities into a store:
cat ftm-entities.ijson | ftmcs write -d my_dataset
# Re-create the entities in aggregated form:
ftmcs iterate -d my_dataset | alephclient write-entities -f my_dataset

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ftm_columnstore-0.2.0.tar.gz (20.7 kB view hashes)

Uploaded Source

Built Distribution

ftm_columnstore-0.2.0-py3-none-any.whl (22.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page