Skip to main content

Data package manager library

Project description

Sputnik: a data package manager library

Sputnik is a library for managing data packages for another library, e.g., models for a machine learning library.

It also comes with a command-line interface, run sputnik --help or python -m sputnik --help for assistance.

Sputnik is a pure Python library licensed under MIT, has minimal dependencies (only semver) and is compatible with python >=2.6 and >=3.3 on Linux, OSX and Windows.


Sputnik is available from PyPI via pip:

pip install sputnik

and from spaCy’s Anaconda channel via conda

conda install -c sputnik

Build a package

Add a package.json file with following JSON to a directory sample and add some files in sample/data that you would like to have packaged, e.g., sample/data/model. See a sample layout here.

  "name": "my_model",
  "include": [["data", "*"]],
  "version": "1.0.0"

Note that include’s path components are lists to avoid platform compatibility issues.

Build the package with following code, it should produce a new file and output its path: sample/my_model-1.0.0.sputnik.

import sputnik
archive ='sample')

Install a package

Decide for a location for your installed packages, e.g., packages. Then install the previously built package with following code, it should output the path of the now installed package: packages/my_model-1.0.0

package = sputnik.install(<app_name>, <app_version>, 'sample/my_model-1.0.0.sputnik', data_path='packages')

Replace <app_name> and <app_version> with your app’s name and version. This information is used to check for package compatibility. You can also provide None instead to disable package compatibility checks. Read more about package compatibility under the Compatibility section below.

List installed packages

This should output the package strings for all installed packages, e.g., ['my_model-1.0.0']:

packages = sputnik.find(<app_name>, <app_version>, data_path='packages')
print([p.ident for p in packages])

Access package data

Sputnik makes it easy to access packaged data files without dealing with filesystem paths or archive file formats.

First, get a Sputnik package object with:

package = sputnik.package(<app_name>, <app_version>, 'my_model', data_path='packages')

On the package object you can check for the existence of a file or directory, get it’s path or directly open it. Note that each directory in a path must be provided as separate argument. Do not address paths with slashes or backslashes as this will lead to platform-compatibility issues.

if package.has_path('data', 'model'):
  with'data', 'model'), mode='r', encoding='utf8') as f:
    res =

Alternatively you can use Sputnik’s open() wrapper:

with['data', 'model'], mode='r', encoding='utf8') as f:
  res =

Note that package.file_path() only works on files, not directory. Use package.dir_path() on directories.

If you want to list all file contents of a package use sputnik.files('my_model', data_path='packages').

Remove package

sputnik.remove(<app_name>, <app_version>, 'my_model', data_path='packages')

Purge package pool/cache

sputnik.purge(<app_name>, <app_version>, data_path='packages')


install, find, package, files, search and remove commands accept version constraint strings that follow semantic versioning, e.g.:

sputnik.install(<app_name>, <app_version>, 'my_model ==1.0.0', data_path='packages')
sputnik.find(<app_name>, <app_version>, 'my_model >1.0.0', data_path='packages')
sputnik.package(<app_name>, <app_version>, 'my_model >=1.0.0', data_path='packages')<app_name>, <app_version>, 'my_model <1.0.0', data_path='packages')
sputnik.files(<app_name>, <app_version>, 'my_model <=1.0.0', data_path='packages')
sputnik.remove(<app_name>, <app_version>, 'my_model ==1.0.0', data_path='packages')

Multiple version constraints can be concatenated with commas, e.g., my_model >=1.0.0,<2.0.0. The constraint expression is satisfied if all individual constraints are satisfied.


Sputnik allows to specify compatibility of a package with an app’s name to let an index server provide app-specific views on installable packages. An app in this context is the project that imports Sputnik (e.g., my_library).


  "name": "my_model",
  "description": "this model is awesome",
  "include": ["data/*"],
  "version": "2.0.0",
  "license": "public domain",
  "compatibility": {
    "my_library": null

Currently no compatibility checks are performed within Sputnik code.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sputnik-0.9.3.tar.gz (22.5 kB view hashes)

Uploaded Source

Built Distribution

sputnik-0.9.3-py2.py3-none-any.whl (31.5 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page