dummio
IO for dummies! A unified save/load interface using the most common and recommendable default options for IO between various object types and file types. For example, instead of
with open(file_path, 'r', encoding='utf-8') as file:
data = json.load(file)
you can simply
data = dummio.json.load(file_path)
Users may pass additional keyword arguments to the underlying IO methods.
Direct IO vs cloud paths
Most dummio IO calls "just work" against cloud paths like s3://bucket/key, gs://bucket/key, or az://container/key. For example, dummio.json.load("s3://bucket/key") will read a json file from an S3 bucket. Notes:
- Shout-out: universal-pathlib powers much of the cloud-iteroperability on our backend.
- Warning: Although we manually run
demo/cloud.pyto ensure basic functionality, current CI unit testing does not cover cloud interactions.
Standardized IO interface
In some coding applications it is desirable to pass an IO module as an argument to a function. Here it is convenient to pass a dummio submodule, since all dummio submodules have the same save and load interface, having equivalent signatures (except for differences hidden in **kwargs).
Supported object and file types
So far we support:
- text, pickle, and dill
- simple dictionaries:
- json
- orjson
- yaml
- pandas dataframes:
- csv
- feather
- parquet
- numpy arrays (thin wrapper on numpy.save/load)
- onnx.ModelProto instances
- pydantic models (relying on the built-in json serialization methods)
- mashumaro models inheriting the json or yaml serialization mixins
Filepaths passed to save and load methods can be of type str, pathlib.Path, or universal_pathlib.UPath.
Dependencies
universal-pathlib is our only required dependency.
For other dependencies, such as pandas, calling from dummio.pandas import df_parquet will raise a helpful message to install pandas if you have not already done so.
Examples
Basic IO methods can be accessed directly as dummio.text, dummio.json, etc:.
import dummio
text = "hello world"
data = {"key": text}
path = "io_example_file"
# Text
dummio.text.save(text, path=path)
assert text == dummio.text.load(path)
# YAML
dummio.yaml.save(data)
assert data == dummio.yaml.load(path)
See demo/cloud.py for more many other examples.
Installation
We're on pypi, so pip install dummio.
If working directly on this repo, consider using the simplest-possible virtual environment.
Metadata
Release files for dummio 1.10.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| dummio-1.10.2.tar.gz | 10.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dummio-1.10.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 28.9 kB
Release files / dummio-1.10.2.tar.gz
| Download URL | dummio-1.10.2.tar.gz |
|---|---|
| Size | 10.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c7c3566b0eb5d2686d265417657918a140c72edd053a5173ef63a996ad4ef59f
|
|
BLAKE2b-256 checksum How to use checksums |
b10947d71143dbe371bcb007297cf564e8070534331f528124559d3723850eba
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.7.8
|
Release files / dummio-1.10.2-py3-none-any.whl
| Download URL | dummio-1.10.2-py3-none-any.whl |
|---|---|
| Size | 18.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fc042236e93e7aedec138b46bf1ccd98f914a07fb4c08cc31f8200a0f4fa4648
|
|
BLAKE2b-256 checksum How to use checksums |
c6b38bc8cf0766f2bfe3832e3160ad0a696d062062dcb7c59f69eedc7644d7f4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.7.8
|