Skip to main content

A Python datalake client.

Project description


A Python datalake client.

Test Coverage Package version



pip install pyarrow ness


import ness

dl = ness.dl(bucket="mybucket", key="mydatalake")
df ="mytable")


# Sync all tables

# Sync a single table

# Sync and read a single table
df ="mytable", sync=True)


Specify the input data source format, the default format is parquet:

import ness

dl = ness.dl(bucket="mybucket", key="mydatalake", format="csv")

AWS Profile

Files are synced using default AWS profile, you can configure another one:

import ness

dl = ness.dl(bucket="mybucket", key="mydatalake", profile="myprofile")

Command Line

Usage: ness sync [OPTIONS] S3_URI

  --format TEXT   Data lake source format.
  --profile TEXT  AWS profile.
  --table TEXT    Table name to sync.
  --help          Show this message and exit.
ness sync bucket/key --table mytable

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ness-0.1.5.tar.gz (4.0 kB view hashes)

Uploaded Source

Built Distribution

ness-0.1.5-py3-none-any.whl (5.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page