A data tool designed to move data seamlessly between various sources and destinations.
Project description
A data tool designed to move data seamlessly between various sources and destinations.
CLI
Bytebridge aims to have a CLI that can be used to easily transfer data from multiple sources. Some examples are shown below:
Parquet to PostgreSQL
bytebridge transfer \
--connections connections.json \
--source parquet_conn \
--source-object data.parquet \
--target postgresql_conn \
--target-object bytebridge.public.data
PostgreSQL to Parquet
bytebridge transfer \
--connections connections.json \
--source postgresql_conn \
--source-object bytebridge.public.data \
--target postgresql_conn \
--target-object data.parquet
In both cases, the connections metadata are defined in the connections.json
file. An example of the file definition is:
{
"parquet_conn": {
"type": "parquet"
},
"postgresql_conn": {
"type": "postgresql",
"parameters": {
"host": "[hostname_of_the_connection]",
"user": "[postgresql_username_to_be_used]",
"password": "[postgresql_password_to_be_used]",
"port": "[postgresql_port_to_be_used]",
}
}
}
API
Coming soon.
Data Connectors
Currently Supported
Name | Type | Client |
---|---|---|
PostgreSQL | Database | psycopg |
MySQL | Database | mysql-connector-python |
MS SQL Server | Database | pymssql |
Parquet | File | pyarrow |
CSV | File | csv |
Planned
Name | Type |
---|---|
SQLite | Database |
MongoDB | Database |
MariaDB | Database |
Clickhouse | Database |
BigQuery | Database |
Oracle | Database |
Cassandra | Database |
ORC | File |
Avro | File |
Excel (XLSX) | File |
JSON | File |
Contributing
Feel free to contribute to this project. See the contribution guidelines in here.
License
This project is licensed under the terms of the Apache 2.0 license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bytebridge-0.0.3.tar.gz
(9.6 kB
view hashes)
Built Distribution
bytebridge-0.0.3-py3-none-any.whl
(11.4 kB
view hashes)
Close
Hashes for bytebridge-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 524106e72c751420a350f945519c4466934849d39d6fbe8c6109974f57c54a50 |
|
MD5 | 049f3d2774767d8d754b0cd9c3eb251a |
|
BLAKE2b-256 | 90cc093f0a61e1579ebe6b1864979db580b60df37f885f3e9694e6e25410b71b |