anystore
Store anything anywhere. A wrapper around wrappers to avoid boilerplate code (because we are lazy).
anystore helps you to transfer data from and to a various range of sources (local filesystem, http, s3, redis, sql, ...) with a unified interface. It's main use case is to store data pipeline outcomes in a distributed cache, so that different programs or coworkers can access intermediate results.
Install
pip install anystore
Usage
from anystore import Store
store = Store()
assert store.get("foo/bar.txt", raise_on_nonexist=False) is None
store.set("foo/bar.txt", "Hello world")
assert store.get("foo/bar.txt") == "Hello world"
It comes with a handy decorator:
from anystore import anycache
# use decorator
@anycache(storage="s3://mybucket/cache")
def download_file(url):
# a very time consuming task
return result
# 1. time: slow
res = download_file("https://example.com/foo.txt")
# 2. time: fast, as now cached
res = download_file("https://example.com/foo.txt")
development
This package is using poetry for packaging and dependencies management, so first install it.
Clone this repository to a local destination.
Within the root directory, run
poetry install --with dev
This installs a few development dependencies, including pre-commit which needs to be registered:
poetry run pre-commit install
Before creating a commit, this checks for correct code formatting (isort, black) and some other useful stuff (see: .pre-commit-config.yaml)
test
make test
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file anystore-0.0.1.tar.gz.
File metadata
- Download URL: anystore-0.0.1.tar.gz
- Upload date:
- Size: 18.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.7.1 CPython/3.11.7 Linux/6.5.0-5-amd64
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ab6e944c2ecf108ec9ed6b17c6c20c8c9e88e7cc59fb476093e2689a6b454fc2
|
|
| MD5 |
fc0d6e7b43b53769f655a699aa6f14ee
|
|
| BLAKE2b-256 |
b594071109f5b29914273ae34c6ef3a424c54f7cf91175527871b76a15326be5
|
File details
Details for the file anystore-0.0.1-py3-none-any.whl.
File metadata
- Download URL: anystore-0.0.1-py3-none-any.whl
- Upload date:
- Size: 19.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.7.1 CPython/3.11.7 Linux/6.5.0-5-amd64
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9dfbbdde7da2a6fce9830168f769d10813e6a49fe915665022bcafa4f495f898
|
|
| MD5 |
d0b5bed41e8c4e6760aeccac8f6f54b0
|
|
| BLAKE2b-256 |
dbcf0cf1825fe76f8e9cdacf86f50dcfe93f004a3ca80233a08e6e9808173993
|
