Kukur makes time series data and metadata available to the Apache Arrow ecosystem.
Project description
Kukur
Kukur makes time series data and metadata available to the Apache Arrow ecosystem.
WARNING
Kukur is under active development. Breaking changes to the interfaces are possible.
Kukur uses semantic versioning.
While < 1.0.0
, changes to the minor version indicate breaking changes.
Usage
Kukur can be used as a Python library or as a standalone application that exposes an Arrow Flight interface.
Supported Sources
Multiple types of time series sources are supported:
- ADODB connections (including OLEDB)
- Apache Feather files
- Apache Parquet files
- CSV files
- InfluxDB databases
- Other Kukur or Timeseer instances
- ODBC data sources
Check the documentation for more info
Development
Create a virtualenv and install dependencies:
$ python -m venv venv
$ source venv/bin/activate
(venv) $ make deps dev-deps
Launch Kukur using:
(venv) $ python -m kukur.cli
Lint and format:
(venv) $ make lint
Kukur uses black to format all code.
Run unit tests:
(venv) $ make test
Integration Tests
Kukur runs integration tests against real databases where possible.
OS requirements to complete the integration tests are:
- unixodbc
- freetds
Additional Python packages are also required:
(venv) $ pip install -r requirements-python.txt
Integration tests require Kukur to be running with a special configuration. Some time series databases need to be started using docker-compose.
$ docker-compose -f tests/test_data/docker-compose.yml up -d
(venv) $ python -m kukur.cli --config-file tests/test_data/Kukur.toml
Since the location of the freetds libraries varies and this needs to be hardcoded in the unixodbc configuration,
several configuration profiles exist.
These profiles can be selected using the KUKUR_INTEGRATION_TARGET
environment variable:
/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
: the default, noKUKUR_INTEGRATION_TARGET
required/usr/lib/libtdsodbc.so
: useKUKUR_INTEGRATION_TARGET=linux
/usr/local/lib/libtdsodbc.so
: useKUKUR_INTEGRATION_TARGET=local
Then, run the tests:
(venv) $ KUKUR_INTEGRATION_TARGET=linux make integration-test
Stop the databases using:
$ docker-compose -f tests/test_data/docker-compose.yml down
Alternatively, run Kukur in docker-compose as well to have a known stable setup:
$ docker-compose -f tests/test_data/docker-compose.yml -f tests/test_data/docker-compose.container.yml up -d
Run the tests using:
(venv) $ make integration-test
Stop all containers:
$ docker-compose -f tests/test_data/docker-compose.yml -f tests/test_data/docker-compose.container.yml down
License
Copyright 2021 Timeseer.AI
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.