sparecores-data 0.5.0

Structured data on cloud servers collected by sparecores-crawler.

Spare Cores Navigator Data

The Spare Cores Navigator Data is a Python package to make it easy to access the latest version of the Spare Cores Navigator data via a SQLite database file. This database file is compiled from an automatically downloaded, compressed SQLite database dump, periodically updated by a background thread.

Installation

Stable version from PyPI:

pip install sparecores-data

Most recent version from GitHub:

pip install "sparecores-data @ git+https://[e-mail]/SpareCores/sc-data.git"

Usage

For easy access to the most recent version of the SQLite database file, import the db object of the sc_data Python package, which runs an updater thread in the background to keep the SQLite file up-to-date:

from sc_data import db
print(db.path)

The database is cached locally in a persistent directory and automatically updated when needed. On import, the package:

1. Checks the local cache for a valid (non-stale) database
2. If cached and fresh, uses it immediately
3. Otherwise, downloads the latest version from our CDN

The cache is stored in a platform-specific location:

• Linux: $XDG_CACHE_HOME/sparecores-data/ or ~/.cache/sparecores-data/
• macOS: ~/Library/Caches/sparecores-data/
• Windows: %LOCALAPPDATA%/sparecores-data/

To enforce waiting for the update to complete, you can use the updated event:

db.updated.wait()

Configuration

The package comes with the following set of default parameters, which can be overridden by builtins or environment variables:

Configuration	Description	Default Value	Builtin Name	Environment Variable
Custom Database Path	Custom file path for the database (bypasses cache)	-	sc_data_db_path	SC_DATA_DB_PATH
Disable Updates	Whether to disable automatic updates	False	sc_data_no_update	SC_DATA_NO_UPDATE
Database URL	The URL of the most recent version of the database file (when unset, the default URL is used)	https://cdn.sparecores.net/sc-data/sc-data-all.sql.xz	sc_data_db_url	SC_DATA_DB_URL
HTTP Timeout	The timeout in seconds for downloading the database file	30	sc_data_http_timeout	SC_DATA_HTTP_TIMEOUT
Refresh Interval	The interval in seconds to check for database updates	600	sc_data_db_refresh_seconds	SC_DATA_DB_REFRESH_SECONDS
Suppress Data License Notice	Suppress the data license notice printed to stderr	-	sc_data_no_license_notice	SC_DATA_NO_LICENSE_NOTICE

Note: Setting SC_DATA_DB_PATH disables caching and uses the specified file directly.

Licensing

The source code of the sparecores-data package is released under the MPL 2.0 license.

The default SQLite database dump file downloaded at runtime is published under the Business Source License 1.1 (BSL 1.1) license.

Details -- including the publisher, timestamped release date, license type, additional usage terms, and a link to the full license text -- is available in the _metadata table of the database, and printed to the stderr by default on every Data instantialization.

References

• Data source
• sparecores.com