Skip to main content
Help the Python Software Foundation raise $60,000 USD by December 31st!  Building the PSF Q4 Fundraiser

A package for caching downloads of specially formatted files.

Project description



The USTDownloadCache is used by the Ubuntu Security Team to improve the runtime of other packages by caching certain JSON files.

Using USTDownloadCache


import logging
from ust_download_cache import USTDownloadCache

logger = logging.getLogger("")
download_cache = USTDownloadCache(logger)

data = download_cache.get_data_from_url(url)
metadata = download_cache.get_cache_metadata_from_url(url) # used by USTDownloadCache

Extracting zipped files

USTDownloadCache has the ability to download, extract, and cache either bz2 or gz archives. These files are extracted and stored uncompressed so that the data can be accessed as quickly as possible.


The USTDownloadCache relies on metadata contained within the file it is downloading/caching. The JSON must supply a "metadata" key, which provides a timestamp (seconds since the Unix epoch) and time to live (ttl) in seconds. The cached version of the file is considered to be expired when timestamp + ttl > now

    "metadata": {
        "timestamp": 1591887905,
        "ttl": 3600,
        "version": "1.0"
    "data": {
        "name1": "value1",
        "name2": "value2"


From Source

To install from source, you can clone this repository and install USTDownloadCache:

$> git clone
$> pip3 install --user ./ust-download-cache/

As a dependency

As USTDownloadCache is not in PyPI at the moment, you must use the github tarball in your

install_requires = [
        "ust-download-cache @",


Installing precommit hooks

To install the precommit hooks, run

pip3 install --user pre-commit
~/.local/bin/pre-commit install

Running the test suite

You can run the automated test suite by running

$> python3 -m pytest

An HTML code coverage report will be generated at ./htmlcov. You can view this with any web browser (e.g. firefox ./htmlcov/index.html).

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for UST-Download-Cache, version 2.1.0
Filename, size File type Python version Upload date Hashes
Filename, size UST_Download_Cache-2.1.0-py3-none-any.whl (17.7 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size UST Download Cache-2.1.0.tar.gz (4.9 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page