Skip to main content

dummio

IO for dummies! A unified save/load interface using the most common and recommendable default options for IO between various object types and file types. For example, instead of

with open(file_path, 'r', encoding='utf-8') as file:
    data = json.load(file)

you can simply

data = dummio.json.load(file_path)

Users may pass additional keyword arguments to the underlying IO methods.

Direct IO vs cloud paths

Most dummio IO calls "just work" against cloud paths like s3://bucket/key, gs://bucket/key, or az://container/key. For example, dummio.json.load("s3://bucket/key") will read a json file from an S3 bucket. Notes:

  • Shout-out: universal-pathlib powers much of the cloud-iteroperability on our backend.
  • Warning: Although we manually run demo/cloud.py to ensure basic functionality, current CI unit testing does not cover cloud interactions.

Standardized IO interface

In some coding applications it is desirable to pass an IO module as an argument to a function. Here it is convenient to pass a dummio submodule, since all dummio submodules have the same save and load interface, having equivalent signatures (except for differences hidden in **kwargs).

Supported object and file types

So far we support:

  • text, pickle, and dill
  • simple dictionaries:
    • json
    • orjson
    • yaml
  • pandas dataframes:
    • csv
    • feather
    • parquet
  • numpy arrays (thin wrapper on numpy.save/load)
  • onnx.ModelProto instances
  • pydantic models (relying on the built-in json serialization methods)
  • mashumaro models inheriting the json or yaml serialization mixins

Filepaths passed to save and load methods can be of type str, pathlib.Path, or universal_pathlib.UPath.

Dependencies

universal-pathlib is our only required dependency.

For other dependencies, such as pandas, calling from dummio.pandas import df_parquet will raise a helpful message to install pandas if you have not already done so.

Examples

Basic IO methods can be accessed directly as dummio.text, dummio.json, etc:.

import dummio

text = "hello world"
data = {"key": text}
path = "io_example_file"

# Text
dummio.text.save(text, path=path)
assert text == dummio.text.load(path)

# YAML
dummio.yaml.save(data)
assert data == dummio.yaml.load(path)

See demo/cloud.py for more many other examples.

Installation

We're on pypi, so pip install dummio.

If working directly on this repo, consider using the simplest-possible virtual environment.

Metadata

Release files for dummio 1.10.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dummio 1.10.2
File Size Uploaded
dummio-1.10.2.tar.gz 10.8 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for dummio 1.10.2
File Interpreter ABI Platform
dummio-1.10.2-py3-none-any.whl Python 3 none any Details

Total release size: 28.9 kB

Release files / dummio-1.10.2.tar.gz

Download URL dummio-1.10.2.tar.gz
Size 10.8 kB
Tags Source
SHA-256 checksum
How to use checksums
c7c3566b0eb5d2686d265417657918a140c72edd053a5173ef63a996ad4ef59f
BLAKE2b-256 checksum
How to use checksums
b10947d71143dbe371bcb007297cf564e8070534331f528124559d3723850eba
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.8

Release files / dummio-1.10.2-py3-none-any.whl

Download URL dummio-1.10.2-py3-none-any.whl
Size 18.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
fc042236e93e7aedec138b46bf1ccd98f914a07fb4c08cc31f8200a0f4fa4648
BLAKE2b-256 checksum
How to use checksums
c6b38bc8cf0766f2bfe3832e3160ad0a696d062062dcb7c59f69eedc7644d7f4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.8

Release history Release notifications | RSS feed

This release

1.10.2 This release

2 release files

1.10.0

2 release files

1.9.1

2 release files

1.9.0

2 release files

1.8.0

2 release files

1.7.0

2 release files

1.6.3

2 release files

1.6.2

2 release files

1.6.1

2 release files

1.6.0

2 release files

1.5.4

2 release files

1.5.0

2 release files

1.4.0

2 release files

1.3.0

2 release files

1.2.0

2 release files

1.1.0

2 release files

1.0.0

2 release files

0.2.0

2 release files

0.1.0

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page