Skip to main content

resumableds

A Python class that supports Data Science projects.

resumableds supports you in writing data science scripts including save/resume functionality.

Data can be saved and resumed avoiding unnessary retrievals of raw data from data storages.

The data directory structure is inspired by cookiecutter-data-science (https://drivendata.github.io/cookiecutter-data-science/).

The class also supports the statement 'Analysis is a DAG' (https://drivendata.github.io/cookiecutter-data-science/#analysis-is-a-dag).

resumableds is written in pure Python and it is intended to be used within Jupyter notebooks. It however can also be useful in Python scripts or script pipelines.

Example

proj1 = RdsProject('project1') # create object from class (creates the dir if it doesn't exist yet)
proj1.raw.df1 = pd.DataFrame() # create dataframe as attribute of proj1.raw (RdsFs 'raw')
proj1.defs.variable1 = 'foo' # create simple objects as attribute of proj1.defs (RdsFs 'defs')
proj1.save() # saved attributes of all RfdFs in proj1 to disk

This will result in the following directory structure (plus some overhead of internals):

  • <output_dir>/defs/var_variable1.pkl
  • <output_dir>/raw/df1.pkl
  • <output_dir>/raw/df1.csv

Note, pandas dataframes are always dumped as pickle for further processing and as csv for easy exploration. The csv files are never read back anymore.

Later on or in another python session, you can do this:

proj2 = RdsProject('project1') # vars and data are read back to their original names
proj2.defs.variable1 == 'foo' # ==> True
isinstance(proj2.raw.df1, pd.DataFrame) # ==> True

Metadata

Release files for resumableds 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for resumableds 1.0.0
File Size Uploaded Provenance Actions
resumableds-1.0.0.tar.gz 15.9 kB Without provenance Details for resumableds-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for resumableds 1.0.0
File Interpreter ABI Platform Actions
resumableds-1.0.0-py3-none-any.whl Python 3 none any Details for resumableds-1.0.0-py3-none-any.whl

Total release size: 34.9 kB

Release files / resumableds-1.0.0.tar.gz

Download URL resumableds-1.0.0.tar.gz
Size 15.9 kB
Tags Source
SHA-256 checksum
How to use checksums
50fa37de478da40a3429e440f5dd3bc7a58bbef456705ede0904035f60621413
BLAKE2b-256 checksum
How to use checksums
672edff19507eba62e9f48333f49acb5dee536b8e9728e90923e871d2ff71997
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.10.9

Release files / resumableds-1.0.0-py3-none-any.whl

Download URL resumableds-1.0.0-py3-none-any.whl
Size 19.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2915838a4545cebef2eaf4ab8385e9d9db1b84528387a611caea6dac119ce0ee
BLAKE2b-256 checksum
How to use checksums
b84c0d19e94b6a53e55e700dcc4a36f3eb7523e4f51121f93fc79de9245085f1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.10.9

Security for resumableds 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page