Skip to main content

Search and cleanup on Docker Registry v2.

Project description

Docker-Registry-Util

Search and cleanup on Docker Registry v2.

Project: https://github.com/merll/docker-registry-util

Overview

This project allows queries on existing repositories and tags of a Docker Registry (v2). It comes with a command line utility dregutil, but can also be used as a lightweight API. Besides gathering information images can also be deleted from a private Docker Registry, allowing for garbage collection.

Background

It is a good idea to clean up a private Docker registry every now and then. For example, images that have only been generated for testing purposes and never been sent to production could be deleted for freeing up storage space.

Since version 2.4.x, the Docker Registry comes with a garbage collector. According to its documentation, it deletes all blobs not referred to by any image manifest, freeing up space.

The storage of the Docker registry itself is similar to the Docker host image collection: The Docker Registry works on content-based storage, with the possibility for adding tags. If we take for example an image that has a certain SHA256 digest sha256:00... and two tags: 1.0.0 and latest. It is possible to address (e.g. fetch) the image using both the digest and either tag. If we upload an image with digest sha256:11... and tags 2.0.0 and latest, the latest tag will be removed from the sha256:00... image.

Other than the Docker API, which tells all the tags of a certain image id, the Registry API only lists repositories and tags and shows the associated digest. Technically an identical digest can even be shared by multiple repositories. If we want to remove an image, we want to make sure that we do not destroy unknown repositories and tags.

Registry configuration

In order for this tool to work, the Docker Registry needs to be configured for allowing deletion. In the configuration file, add the following:

delete:
  enabled: true

Besides that, you need to need to set the environment variable REGISTRY_STORAGE_DELETE_ENABLED to true when creating the container. This is not documented in the Registry Configuration Reference but was required last time I checked and subject to some issue reports (e.g. #989 and #1573).

Installation

This library is implemented in Python 3. After downloading and installing Python, installation is simply done via pip:

pip install docker-registry-util

for installing the latest release or

pip install git+https://github.com/merll/docker-registry-util.git

for installing the latest development version.

Getting started

The library first needs to know how to connect to your registry. The following can be set via the command line or through environment variables:

Environment variable

Command line arg.

Description

REGISTRY

-reg

Registry server to connect to.

REGISTRY_USER

-u

User for basic authentication.

REGISTRY_PASSWORD

-p

Password for basic authentication.

REGISTRY_USE_DIGEST_ AUTH

–digest-auth

Use HTTP Digest Authentication instead of basic auth.

REGISTRY_CLIENT_CERT

-cert

Client certificate (and optionally key) for the registry.

REGISTRY_CLIENT_KEY

-key

Key for the client certificate, if not included in the -cert file.

REQUESTS_CA_BUNDLE

-v

Alternative bundle of certificate authorities for validating the registry.

With this basic configuration, you can query the registry contents via the command line, e.g.

dregutil list-repo-names

lists all the repository names from the registry.

dregutil list-tag-names

lists all available repositories and tags.

On the first start, the tool will query all digests and tags from the registry. In order to speed up operations, this information is stored in a local cache file named according to the registry name. For example, setting the registry to registry.example.com stores the cache in registry_example_com_cache.json. If you have done any upload operations or deletes outside of this tool, you can force a refresh using the --refresh command line argument; you can also relocate the cache file using the -c argument or deactivate the cache entirely setting -c None.

Queries

Queries will list the digests that are used by particular repositories or tags.

Repositories can be queried by exact name:

dregutil query-repos -r my-repo

Tags can be listed using exact names or version selectors.

dregutil query-tags -r my-repo -t latest

Alternatively, you can query versions by prepending an operator <, >, <=, >=, or == to a version number. Make sure to escape the > and < for the shell you are using.

dregutil query-tags -r my-repo -t \<1.4

Queries by regular expressions are also possible.

dregutil query-tags -r my-repo -re 1\\.*

For excluding a tag or version match, use the -x argument. For a RegEx-based exclusion, use -xre.

Deletion

Deleting digests from the registry is possible using the same syntax as for queries, but using remove-repos or remove-tags.

dregutil remove-repos -r my-repo

removes all digests that belong to the repository my-repo from the registry.

dregutil remove-tags -r my-repo -t \<1.4

marks all tags of my-repo as deleted that carry a version number lower than 1.4.

Partial vs. complete match

As image digests may belong to multiple tags, there is a possibility that you might select tags for deletion that are shared with other images, but that you had not intended to remove. Therefore the default behavior is to reverse-check selected digests against your original selection. For example

dregutil remove-tags -r my-repo -t \<1.4

will by erase all digests with a version number lower than 1.4, unless they belong any other tag or repository. If one of the images is also tagged as my-repo/one, it is not removed, unless

dregutil remove-tags -r my-repo -t \<1.4 one

is specified.

This behavior can be changed generally using the --no-match-all-tags command line option. You can also raise an error on the event of unexpected intersections using the --raise-intersecting-tag argument.

Further information

A complete reference to available commands and arguments is available via dregutil --help, or dregutil [command] --help for details about single command.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

docker-registry-util-1.0.1.tar.gz (15.3 kB view details)

Uploaded Source

Built Distribution

docker_registry_util-1.0.1-py3.5.egg (42.5 kB view details)

Uploaded Egg

File details

Details for the file docker-registry-util-1.0.1.tar.gz.

File metadata

File hashes

Hashes for docker-registry-util-1.0.1.tar.gz
Algorithm Hash digest
SHA256 3345d1ee2cc106e990ebe0241b55d54b8ba0e78566b875548407998f1c094806
MD5 ac3d668398ec8a8fb35ea42ba9f73088
BLAKE2b-256 f65dff3af5d3e11aa560c2cefb220b52401e7fd8ffae160c4a80bd226f78c7f1

See more details on using hashes here.

File details

Details for the file docker_registry_util-1.0.1-py3.5.egg.

File metadata

File hashes

Hashes for docker_registry_util-1.0.1-py3.5.egg
Algorithm Hash digest
SHA256 e8f6c1424edefb7f88d4af2ea02bab6c9a9fea253fa6d940298c560a7bc32709
MD5 c325008dbbd0087cd25b8dee9d575fba
BLAKE2b-256 f493fec8a4a19631a032f0f7ef1147a0cd76e16770482d54562dcc3aefd29970

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page