Skip to main content

Column store implementation for ftm data based on clickhouse

Project description

ftm-columnstore on pypi Python test and package pre-commit Coverage Status GPL-3.0 License

ftm-columnstore

Column-store (based on Clickhouse) implementation for nomenklatura statement-based store for ftm entities.

Minimum Python version: 3.11

It is compatible as a store for ftmq

Usage

Set up a running clickhouse instance (pointed to via DATABASE_URI env var, default: localhost), for developing purposes this could work:

make clickhouse

Then initialize the required table schema:

ftmcs init

Or drop existing data and recreate:

ftmcs init --recreate

When using the make clickhouse command, you can play around with SQL queries in your browser: http://127.0.0.1:8123/play

Command-line usage

# Insert a bunch of FtM entities into a store:
cat ftm-entities.ijson | ftmcs write -d my_dataset
# Re-create the entities in aggregated form:
ftmcs iterate -d my_dataset | alephclient write-entities -f my_dataset

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ftm_columnstore-0.3.2.tar.gz (20.5 kB view details)

Uploaded Source

Built Distribution

ftm_columnstore-0.3.2-py3-none-any.whl (21.9 kB view details)

Uploaded Python 3

File details

Details for the file ftm_columnstore-0.3.2.tar.gz.

File metadata

  • Download URL: ftm_columnstore-0.3.2.tar.gz
  • Upload date:
  • Size: 20.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.12.4 Linux/6.9.10-amd64

File hashes

Hashes for ftm_columnstore-0.3.2.tar.gz
Algorithm Hash digest
SHA256 fccf38958ca10e3a9bd1dc70a6b83f09bb8cd1f1a1b86ea4022160d749437a2e
MD5 db2333d3dc115db4665f8421687a9dd6
BLAKE2b-256 62308c1cb952efd91fbb13ce5bfadde74b9778330baf8c260f0e9bcbf47563a7

See more details on using hashes here.

File details

Details for the file ftm_columnstore-0.3.2-py3-none-any.whl.

File metadata

  • Download URL: ftm_columnstore-0.3.2-py3-none-any.whl
  • Upload date:
  • Size: 21.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.12.4 Linux/6.9.10-amd64

File hashes

Hashes for ftm_columnstore-0.3.2-py3-none-any.whl
Algorithm Hash digest
SHA256 e888b64d5ca43aea842dfddfa6333a84f6e539d523c8d90068140b2bc70ea965
MD5 c553cb72fda8207b03f34c37d98279b2
BLAKE2b-256 f25a054d8f0f4e9593798b4557ba5a8e2ddd06499b37bc00181ea7b530fba0bb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page