apache-iotdb-nightly 0.11.2.20210408

Apache IoTDB client API

Apache IoTDB

Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. Due to its light-weight architecture, high performance and rich feature set together with its deep integration with Apache Hadoop, Spark and Flink, Apache IoTDB can meet the requirements of massive data storage, high-speed data ingestion and complex data analysis in the IoT industrial fields.

Apache IoTDB Python Client API

Using the package, you can write data to IoTDB, read data from IoTDB and maintain the schema of IoTDB.

Requirements

You have to install thrift (>=0.13) before using the package.

How to use (Example)

First, download the package: pip3 install apache-iotdb

You can get an example of using the package to read and write data at here: Example

(you need to add import iotdb in the head of the file)

Or:

from iotdb.Session import Session

ip = "127.0.0.1"
port_ = "6667"
username_ = 'root'
password_ = 'root'
session = Session(ip, port_, username_, password_)
session.open(False)
zone = session.get_time_zone()
session.close()

IoTDB Testcontainer

The Test Support is based on the lib testcontainers (https://testcontainers-python.readthedocs.io/en/latest/index.html) which you need to install in your project if you want to use the feature.

To start (and stop) an IoTDB Database in a Docker container simply do:

class MyTestCase(unittest.TestCase):

    def test_something(self):
        with IoTDBContainer() as c:
            session = Session('localhost', c.get_exposed_port(6667), 'root', 'root')
            session.open(False)
            result = session.execute_query_statement("SHOW TIMESERIES")
            print(result)
            session.close()

by default it will load the image apache/iotdb:latest, if you want a specific version just pass it like e.g. IoTDBContainer("apache/iotdb:0.10.0") to get version 0.10.0 running.

Pandas Support

To easily transform a query result to a Pandas Dataframe the SessionDataSet has a method .todf() which consumes the dataset and transforms it to a pandas dataframe.

Example:

from iotdb.Session import Session

ip = "127.0.0.1"
port_ = "6667"
username_ = 'root'
password_ = 'root'
session = Session(ip, port_, username_, password_)
session.open(False)
result = session.execute_query_statement("SELECT * FROM root.*")

# Transform to Pandas Dataset
df = result.todf()

session.close()

# Now you can work with the dataframe
df = ...

Developers

Introduction

This is an example of how to connect to IoTDB with python, using the thrift rpc interfaces. Things are almost the same on Windows or Linux, but pay attention to the difference like path separator.

Prerequisites

python3.7 or later is preferred.

You have to install Thrift (0.11.0 or later) to compile our thrift file into python code. Below is the official tutorial of installation, eventually, you should have a thrift executable.

http://thrift.apache.org/docs/install/

Before starting you need to install requirements_dev.txt in your python environment, e.g. by calling

pip install -r requirements_dev.txt

Compile the thrift library and Debug

In the root of IoTDB's source code folder, run mvn clean generate-sources -pl client-py -am.

This will automatically delete and repopulate the folder iotdb/thrift with the generated thrift files. This folder is ignored from git and should never be pushed to git!

Notice Do not upload iotdb/thrift to the git repo.

Session Client & Example

We packed up the Thrift interface in client-py/src/iotdb/Session.py (similar with its Java counterpart), also provided an example file client-py/src/SessionExample.py of how to use the session module. please read it carefully.

Or, another simple example:

from iotdb.Session import Session

ip = "127.0.0.1"
port_ = "6667"
username_ = 'root'
password_ = 'root'
session = Session(ip, port_, username_, password_)
session.open(False)
zone = session.get_time_zone()
session.close()

Tests

Please add your custom tests in tests folder. To run all defined tests just type pytest . in the root folder.

Notice Some tests need docker to be started on your system as a test instance is started in a docker container using testcontainers.

Futher Tools

black and flake8 are installed for autoformatting and linting. Both can be run by black . or flake8 . respectively.

Releasing

To do a release just ensure that you have the right set of generated thrift files. Then run linting and auto-formatting. Then, ensure that all tests work (via pytest .). Then you are good to go to do a release!

Preparing your environment

First, install all necessary dev dependencies via pip install -r requirements_dev.txt.

Doing the Release

There is a convenient script release.sh to do all steps for a release. Namely, these are

• Remove all transient directories from last release (if exists)
• (Re-)generate all generated sources via mvn
• Run Linting (flake8)
• Run Tests via pytest
• Build
• Release to pypi