Python Dataware Toolkit
A Python toolkit for managing, retrieving, and processing data.
Installation
You can install the toolkit with:
$ pip3 install pydtk
If you want to install the toolkit with extra feature (e.g. support for PointCloud and ROS), you can install it with extra dependencies as follows:
$ pip3 install pydtk[pointcloud,ros]
Some PyDTK models require additional packages.
Please refer the following table and install them manually with command pip install ....
| PyDTK model | Required packages |
|---|---|
| rosbag.* | ros_numpy (https://github.com/eric-wieser/ros_numpy.git) |
| pointcloud.PCDModel | pypcd (https://github.com/klintan/pypcd.git) |
Usage
By using Pydtk, you can load a variety of types of data with a unified interface as shown below.
- Load DBHandler for retrieving metadata
from pydtk.db import DBHandler
# Initialize handler (This will read all the metadata from DB on initialization)
handler = DBHandler(
db_class='meta',
db_host='./examples/example_db',
base_dir_path='./test'
)
- Read metadata from db with data selection.
# Select by timestamp
handler.read(pql='start_timestamp > 1420000000 and end_timestamp < 1500000000')
print(handler.data)
# Select by record-id
handler.read(pql='record_id == regex("test.*")')
print(handler.data)
- Load data from files based on metadata.
from pydtk.io import BaseFileReader, NoModelMatchedError
reader = BaseFileReader()
try:
for sample in handler:
print('loading content "{0}" from file "{1}"'.format(sample['contents'], sample['path']))
try:
timestamps, data, columns = reader.read(sample)
assert print(data)
except NoModelMatchedError as e:
print(str(e))
continue
except EOFError:
pass
Documentation
For more information about this toolkit, please refer the document.
Setup for contribution
To improve this toolkit, firstly clone this repository and then run the following command to prepare the environment.
$ git clone git@github.com:dataware-tools/pydtk.git --recurse-submodules
$ poetry install
Make sure that poetry is installed before executing the command.
If you want to install the toolkit with extra feature (e.g. support for ROS),
please specify it with -E option.
Example (installation with pointcloud and ros extras):
$ poetry install -E pointcloud -E ros
1 maintainer
Metadata
1 maintainer
Release files for pydtk 0.2.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| pydtk-0.2.7.tar.gz | 101.5 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pydtk-0.2.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 251.3 kB
Release files / pydtk-0.2.7.tar.gz
| Download URL | pydtk-0.2.7.tar.gz |
|---|---|
| Size | 101.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ac9f723431619cd065fbcbcdad5c5c9048e17f7b97edacbbc59bd7243918f2a4
|
|
BLAKE2b-256 checksum How to use checksums |
de44e4b8617d088b064e35b9cac64055a3d2acdf325b6624a89f09e8aadd71ee
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.1.15 CPython/3.7.15 Linux/5.15.0-1023-azure
|
Release files / pydtk-0.2.7-py3-none-any.whl
| Download URL | pydtk-0.2.7-py3-none-any.whl |
|---|---|
| Size | 149.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1b3f480c0a520e893f226cc97a8b77bb3c1ce21a83217ae2eb2796520887a955
|
|
BLAKE2b-256 checksum How to use checksums |
57016ef801f1aa70d0be1e5cf247321bf4089da2defa4c92d66eb5e0b19b6767
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.1.15 CPython/3.7.15 Linux/5.15.0-1023-azure
|
Security for pydtk 0.2.7
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
