Skip to main content

Simple tool for data transformation

Project description

data-steps

This projects provides a minmal framework to organize data transformations in pandas.

It is intended to be used in both notebooks and code files.

The main idea is to provide a simple decorator syntax that is easy to maintains when data transfromation steps get changed or added throughout the project. A prime example is data cleaning where only later in the project some required cleaning steps become apparent

Usage

Wrap your data in an instance

from data_steps import DataSteps

my_data = DataSteps(my_pandas_df)

#register transformation steps

@my_data.step
def data_transformation(data):
    #transfromation steps
    ...
    return transformed data

#access original data
my_data.original

#access data after all transformation steps
my_data.transformed

#get an overview of the registered steps
my_data.steps

#only execute some steps to help debugging transformations
my_data.partial_transform(0)

History

0.0.1 (2021-01-31)

  • First release on PyPi.

0.1.0 (2021-02-11)

  • Changed step decorator to work in bare format, i.e. <instance>.step instead of <instance>.step()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

data-steps-0.1.1.tar.gz (3.7 kB view details)

Uploaded Source

File details

Details for the file data-steps-0.1.1.tar.gz.

File metadata

  • Download URL: data-steps-0.1.1.tar.gz
  • Upload date:
  • Size: 3.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/52.0.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.9.1

File hashes

Hashes for data-steps-0.1.1.tar.gz
Algorithm Hash digest
SHA256 d2c4c6387bde01f283550f1f68f43abb2492fb2349a4184cc4bd3125a6182a29
MD5 56511ed60f7b3f3496776f364db6663d
BLAKE2b-256 a7df70433270bbb50088a23dcd999dd272642c7688138746166309b34fafbf58

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page