Skip to main content

Structured data collected by sparecores-crawler.

Project description

Spare Cores Data

Build Last Run Project Status: Alpha Maintenance Status: Active CC-BY-SA 4.0 License PyPI - Python Version NGI Search Open Call 3 beneficiary

SC Data is a Python package and related tools making use of sparecores-crawler to pull and standardize data on cloud compute resources. This repository actually runs the crawler every 5 minutes to update spot prices, and every hour to update all cloud resources in an internal SCD table and public SQLite snapshot as well.


Stable version from PyPI:

pip install sparecores-data

Most recent version from GitHub:

pip install "sparecores-data @ git+"


For easy access to the SQLite database file, import the db object of the sc_data Python package, which runs an updater thread in the background to keep the SQLite file up-to-date:

from sc_data import db

By default, the SQLite file will be updated every 600 seconds, which can be overwritten by the sc_data_db_refresh_seconds builtins attribute or the SC_DATA_DB_REFRESH_SECONDS environment variable.


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sparecores_data-0.2.1.post1.tar.gz (119.1 kB view hashes)

Uploaded Source

Built Distribution

sparecores_data-0.2.1.post1-py3-none-any.whl (116.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page