Skip to main content

Minerva DPN Worker, for more information see https://minerva-archive.org

Project description

Minerva Worker
Preserving Myrient's legacy, one file at a time.

License Python version Manager: uv Linter: Ruff Build status


Myrient is shutting down. Minerva is a volunteer-driven effort to archive its entire collection before it goes offline. Run a script, share your bandwidth, help preserve the archive.

Installation

Download and install the Python script from PIP/PyPI:

$ pip install minerva-worker

[!NOTE] If pip gives you a warning about a path not being in your PATH environment variable then promptly add that path then close all open command prompt Windows, or running minerva won't work as it will not be recognized as a program.

You now have the minerva package installed - Voilà 🎉!
Get started by running minerva in your Terminal or Windows Run.
For configuration options, see help and options by running minerva --help.

Alternatively, a Windows EXE is available on the Releases page, simply run it to begin!

Usage

It's very easy to use. Simply run the minerva.exe file, or run minerva in your Terminal/Command Prompt.

You can configure the worker settings by running minerva run --help to see what configuration options you can change. If you have a great computer and network, it's recommended to bump up the -c and -b options.

[!TIP] It's recommended to keep -c smaller or the same as -b.

How it works

The worker script asks the minerva server for jobs to download. The server gives active workers random missing needed file URLs to download. When the worker is given a job, it temporarily downloads the file and uploads it to the minerva file servers. Once the file is downloaded, it is deleted from your machine.

[!TIP] If you also want a copy of the files, use the --keep-files setting.

Jobs are given exclusively to each worker, no two workers download the same file at the same time. However, to verify that uploads aren't corrupted, each job gets given (eventually) to a second worker. Both uploads are then confirmed and if both workers give back the same file to the minerva file server, then the job is marked as complete and verified.

[!NOTE] You may see 409 Conflict errors on upload, this happens when either you or another worker had mismatching files uploaded. Just ignore these error's and let the worker continue. If you suspect every file has this issue, please verify your network connection is stable and verify your downloads arent corrupted.

Discord Authentication

When using the minerva worker, you are prompted upon startup to login and authorize with Discord. This is to authenticate unique users on the minerva server, to know who jobs are given to, and to use your username and avatar in the worker dashboard leaderboards.

This does not give Minerva, this script, or anyone else access to your account, or any permissions.

Docker

You can run the Minerva Worker inside a headless Docker container. The following steps assume some knowledge on git/docker.

To change the settings of the worker, edit the the command section of the docker-compose file. For example, to request 20 jobs at a time, with 10 concurrent downloads, change it to: command: ["uv", "run", "minerva", "run", "-c", "10", "-b", "20"]

1. Download a copy of the repository

  • Clone the Git Repository: git clone https://github.com/rlaphoenix/minerva
  • Enter it: cd minerva

2. Get an Authorization Token

There's two ways to go about this,

  • either run the normal python script to authenticate with it,
  • or, get and save the token to a specific location manually.

To get the token manually, go to https://api.minerva-archive.org/auth/discord/login and you will get a token value once you authorize with Discord. Copy that token value and save it to ~/.minerva-dpn/token on Linux/macOS, or save it to %USERPROFILE%/.minerva-dpn/token on Windows.

The token file must stay there at all times for the Docker container to have it. The location where the token needs to be can be changed, just make sure you change the volume location and environment variable in the docker-compose config.

3. Start the container

Build the Docker image and start the container with docker compose up -d to run it in the background. You can later stop the container with docker compose down.

[!TIP] If you prefer to run it in the foreground (attached to your terminal), simply use docker compose up.

Development

  1. Install uv
  2. uv sync --all-extras --all-groups
  3. .venv\Scripts\activate (or source .venv/bin/activate on macOS and Linux)
  4. uv tool install pre-commit --with pre-commit-uv --force-reinstall
  5. pre-commit install

Now feel free to work on the project however you like, all code will be checked before committing.

Credit

  • bl791 for the original one-file script.
  • Puyodead1 for their improved rich interface and speedups.
  • rlaphoenix for further improvements, bug fixes, support.
  • wikipiti for the aria2 control file parsing code.

Licensing

This software is licensed under the terms of CC0 1.0 Universal. You can find a copy of the license in the LICENSE file in the root folder


© rlaphoenix 2026

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

minerva_worker-1.2.5.tar.gz (29.6 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

minerva_worker-1.2.5-py3-none-any.whl (22.2 kB view details)

Uploaded Python 3

File details

Details for the file minerva_worker-1.2.5.tar.gz.

File metadata

  • Download URL: minerva_worker-1.2.5.tar.gz
  • Upload date:
  • Size: 29.6 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.10.7 {"installer":{"name":"uv","version":"0.10.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for minerva_worker-1.2.5.tar.gz
Algorithm Hash digest
SHA256 96b7b5386888b9e0ce7f7defceffaef82253f0bdb7afd4c78cd3baf7ea289806
MD5 26ec66f14b6e14740c44130058d6a361
BLAKE2b-256 c2b7c7cc15374ff7b544974fd40ab7e6a6b33d3a85fb81fa9dd191b69812a16e

See more details on using hashes here.

File details

Details for the file minerva_worker-1.2.5-py3-none-any.whl.

File metadata

  • Download URL: minerva_worker-1.2.5-py3-none-any.whl
  • Upload date:
  • Size: 22.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.10.7 {"installer":{"name":"uv","version":"0.10.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for minerva_worker-1.2.5-py3-none-any.whl
Algorithm Hash digest
SHA256 adbd182e878857745eea9aca42170847c143e772dab922177bcb4d303dac3fd4
MD5 36f1682a16c3670dde34db693d2ae7d8
BLAKE2b-256 f3f53eb2e3a4aea19f197dc6047ef5b8f9579fa8d653f98ccd626dfdd5fa988f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page