pygobbler 0.1.11

Python client for the Gobbler service

Python client for the gobbler service

Pretty much as it says on the tin; provides an Python client for the Gobbler service. It is assumed that the users of this package and the Gobbler service itself are accessing the same shared filesystem; this is typically the case for high-performance computing (HPC) clusters in scientific institutions. To demonstrate, let's spin up a mock Gobbler instance:

import pygobbler as pyg
_, staging, registry, url = pyg.start_gobbler()

Administrators are responsible for creating projects:

pyg.create_project("test", staging=staging, url=url, owners=["akari"])

An authorized user account (in this case, akari) can then upload directory of arbitrary files:

import tempfile
import os
tmp = tempfile.mkdtemp()
with open(os.path.join(tmp, "blah.txt"), "w") as f:
    f.write("BAR")
os.mkdir(os.path.join(tmp, "foo"))
with open(os.path.join(tmp, "foo", "bar.txt"), "w") as f:
    f.write("1 2 3 4 5 6 7 8 9 10")

pyg.upload_directory(
    project="test", 
    asset="foo", 
    version="bar", 
    directory=tmp,
    staging=staging, 
    url=url
)

Anyone can fetch or list the contents, either on the same filesystem or remotely via the REST API.

pyg.list_files("test", "foo", "bar", registry=registry, url=url)
pyg.fetch_manifest("test", "foo", "bar", registry=registry, url=url)
pyg.fetch_summary("test", "foo", "bar", registry=registry, url=url)
pyg.fetch_file("test/foo/bar/blah.txt", registry=registry, url=url)
pyg.version_path("test", "foo", "bar", registry=registry, url=url)

Project owners can set the permissions to allow other users to add new assets or new versions of existing assets:

pyg.set_permissions(
    "test", 
    uploaders=[ { "id": "alicia", "asset": "foo" } ], 
    registry=registry, 
    staging=staging, 
    url=url
)

# And then 'alicia' can do:
pyg.upload_directory(
    project="test", 
    asset="foo", 
    version="bar2", 
    directory=tmp,
    staging=staging, 
    url=url
)

By default, uploaders are untrusted and their uploads will be "probational". Owners can approve/reject probational uploads after review.

pyg.approve_probation("test", "foo", "bar2", staging=staging, url=url)

Finally, administrators can delete projects, though this should be done sparingly.

pyg.remove_project("test", staging=staging, url=url)

Check out the API documentation for more details on each function. For the concepts underlying the Gobbler itself, check out the repository for a detailed explanation.