DataCache
Download a dataset once, verify it, and reuse the local file on later runs. DataCache handles streaming downloads, gzip/ZIP decompression, tar-tree installation, offline inspection and SQLite caches built from pandas DataFrames, for Python applications including OpenVax libraries such as pyensembl.
Install
Python 3.9 or newer is required.
python -m pip install datacache
# Optional HTML-table conversion:
python -m pip install "datacache[html]"
Quickstart
Download a file once, then reuse its local path on later calls:
from datacache import Cache
cache = Cache("my-project")
url = "https://raw.githubusercontent.com/openvax/datacache/master/LICENSE"
path = cache.fetch(url, filename="LICENSE", timeout=30)
print(path)
# Reuses the cached file without a network request, even in a later process.
assert cache.fetch(url, filename="LICENSE") == path
Replace the URL and filename with your dataset. Existing files are reused until
you refresh them with force=True, or periodically with expire_after;
DataCache doesn't ask the server whether a file has changed. Add
show_progress=True to display a download bar; no extra installation is needed.
Guides
The guides are also published at https://openvax.github.io/datacache/.
- Working offline example, verification and interface choices
- Downloads, inspection, integrity, freshness, retries and resumption
- Tables, SQLite and transformations
- Shared caches
- Progress and logging
- Pinned bundles, archive trees and the fixed-path file registry
- Interface selection and guarantees
- Derived-artifact materialization: caller-owned builders, outputs published together as one bundle, dependency receipts and resumable inputs.
- Complete API reference
- Individual HTTP ZIP members: bounded range downloads with compressed and expanded checksums.
- Downstream integration: contracts for consuming libraries.
- Release notes and release procedure.
Dependencies and progress display
tqdm is installed by default; existing download APIs enable displays with
show_progress=True, while materialize enables them by default. Quiet use
does not import tqdm or configure your application's logging. The old
datacache[progress] install syntax remains supported as a compatibility alias.
The existing pandas dependency range is unchanged; upgrading DataCache does not introduce a pandas 1.5 requirement. CI covers pandas 1.4.4, 1.5.3, and current releases on supported Python versions.
Development
python -m pip install -e ".[test]"
./lint-and-test.sh
python -m examples.basic_usage
Tests use local files, mocked responses, and local HTTP servers. They do not depend on external dataset servers. See CI for the Python, dependency, and operating-system combinations exercised.
Build the site with python -m pip install -r requirements-docs.txt and ./docs.sh.
License
Apache 2.0; see LICENSE.
5 maintainers
Metadata
5 maintainers
Release files for datacache 1.27.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| datacache-1.27.1.tar.gz | 252.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| datacache-1.27.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 352.2 kB
Release files / datacache-1.27.1.tar.gz
| Download URL | datacache-1.27.1.tar.gz |
|---|---|
| Size | 252.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1d17416caed8bdaaa7271ae9737b8cb8685de4b95a95c3defb98b0e81fb0604f
|
|
BLAKE2b-256 checksum How to use checksums |
10c77f61ea8aca9dce9dd8be491faed74d8e48c38f4a5f90f95c84b4cc8a9a89
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.6
|
Release files / datacache-1.27.1-py3-none-any.whl
| Download URL | datacache-1.27.1-py3-none-any.whl |
|---|---|
| Size | 99.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e38479dd414f95ee4d7e904c7abbf3dc8dcd052ebdda453ff906e1223038a594
|
|
BLAKE2b-256 checksum How to use checksums |
4246e4342f4040fd3e4591e214f90f7ccb0a9f6250f7ef18bbcdf9926532ec12
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.6
|
