Ness
A Python datalake client.
Requirements
Installation
pip install pyarrow ness
Quickstart
import ness
dl = ness.dl(bucket="mybucket", key="mydatalake")
df = dl.read("mytable")
Sync
# Sync all tables
dl.sync()
# Sync a single table
dl.sync("mytable")
# Sync and read a single table
df = dl.read("mytable", sync=True)
Format
Specify the input data source format, the default format is parquet:
import ness
dl = ness.dl(bucket="mybucket", key="mydatalake", format="csv")
AWS Profile
Files are synced using default AWS profile, you can configure another one:
import ness
dl = ness.dl(bucket="mybucket", key="mydatalake", profile="myprofile")
Command Line
Usage: ness sync [OPTIONS] S3_URI
Options:
--format TEXT Data lake source format.
--profile TEXT AWS profile.
--table TEXT Table name to sync.
--help Show this message and exit.
ness sync bucket/key --table mytable
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ness-0.1.5.tar.gz
(4.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
ness-0.1.5-py3-none-any.whl
(5.8 kB
view details)
File details
Details for the file ness-0.1.5.tar.gz.
File metadata
- Download URL: ness-0.1.5.tar.gz
- Upload date:
- Size: 4.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.1.6 CPython/3.10.4 Linux/5.13.0-1017-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
594e51398993d473472ef023b71b893ba54b94d5bc4da77d33669c878a92cdb4
|
|
| MD5 |
caf61435b554edeabcd99df13ddf7cbd
|
|
| BLAKE2b-256 |
5046553614eae9025f85136dfde16ee5d0c8bf8fc2005cb10a7387562df27eed
|
File details
Details for the file ness-0.1.5-py3-none-any.whl.
File metadata
- Download URL: ness-0.1.5-py3-none-any.whl
- Upload date:
- Size: 5.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.1.6 CPython/3.10.4 Linux/5.13.0-1017-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2c408889803f3268974e2d91a04f29cca799812988493ad4c36836c5aff9286e
|
|
| MD5 |
3ce12135fb62dbc499c7510b6deea88a
|
|
| BLAKE2b-256 |
f9776fae072ee9a6adc1ed9e469acd088d4d353951d7734327b768e58f7e5182
|
