Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

fdroid-dl is a f-droid (offline) mirror generation and update utility

Project description

fdroid-dl

Is a python based f-droid mirror generation and update utility. Point at one or more existing f-droid repositories and the utility will download the metadata (pictures, descriptions,..) for you and place it in your local system. Simply run fdroid-dl && fdroid update and you are set.

Motivation

The idea is to have an no internet local copy of one or more repositories, without the need to compile the thousands of apps on your own build server but rather download them like the android client does. So this tool came into existence to simply download a whole repository and import the apps into your own locally installed one. At the time of writing a full offline copy including assets is ~7.5GB of the official repository of f-droid.org.

Installation

fdroid-dl is available via pip, simply run pip install fdroid-dl and you can use fdroid-dl on your command line. pypi.org - fdroid-dl

Documentation

Can be found at fdroid-dl.readthedocs.io

Command Line Options

Usage: fdroid-dl [OPTIONS] COMMAND [ARGS]...

  Is a python based f-droid mirror generation and update utility. Point at
  one or more existing f-droid repositories and the utility will download
  the metadata (pictures, descriptions,..) for you and place it in your
  local system.

  Simply run "fdroid-dl update && fdroid update" in your folder with repo
  and you are set.

Options:
  -d, --debug               enable debug level logging
  -c, --config FILE         location of your fdroid-dl.json configuration file
                            [default: fdroid-dl.json]
  -r, --repo DIRECTORY      location of your fdroid repository to store the
                            apk files  [default: ./repo]
  -m, --metadata DIRECTORY  location of your fdroid metadata to store the
                            asset files  [default: ./metadata]
  --cache DIRECTORY         location for fdroid-dl to store cached data
                            [default: ./.cache]
  --help                    Show this message and exit.

Commands:
  update  starts updating process
Usage: fdroid-dl update [OPTIONS] COMMAND [ARGS]...

Options:
  --index / --no-index        download repository index files  [default: True]
  --metadata / --no-metadata  download metadata assset files  [default: True]
  --apk / --no-apk            download apk files  [default: True]
  --apk-versions INTEGER      how many versions of apk to download  [default:
                              1]
  --src / --no-src            download src files  [default: True]
  --threads INTEGER           configure number of parallel threads used for
                              download  [default: 10]
  --head-timeout INTEGER      maximum time in seconds a HEAD request is
                              allowed to take  [default: 10]
  --index-timeout INTEGER     maximum time in seconds index file download is
                              allowed to take  [default: 60]
  --download-timeout INTEGER  maximum time in seconds file download is allowed
                              to take  [default: 60]
  --help                      Show this message and exit.

Configuration File

TODO

  • [x] Create backend to crawl existing repos
  • [x] Fetch info directly index.jar and index-v1.jar
  • [x] Compatibility with old and new repo styles
  • [x] Download multi threaded
  • [x] Verify apk checksum
  • [x] Local cache for index files
  • [ ] Source code download not implemented yet
  • [ ] Metadata update to do delta not full download all the time
  • [ ] Cleanup strategy for old apk files (maybe fdroid update does this already?)
  • [x] Create a CLI python click
  • [x] pip package packaging.python.org
  • [x] CI builds for pip package
  • [ ] Documentation ;-)
  • [ ] Writing tests pytest

CHANGELOG

  • WIP: Documentation added
  • WIP: Test added
    • requests-mock ?

Ideas

  • requests-cache ?

Development

Requirements

  • python 2.7.* or 3.5.*
  • pip 3.x
  • virtualenv 3.x

install locally

# git clone https://github.com/t4skforce/fdroid-dl.git
# cd fdroid-dl
# virtualenv .env
# source .env/bin/activate
# python setup.py install
# fdroid-dl --help
# python3 -m fdroid_dl --help

References

While this project was developed the following references where used

F-Droid

Python

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for fdroid-dl, version 0.0.5
Filename, size File type Python version Upload date Hashes
Filename, size fdroid_dl-0.0.5-py3-none-any.whl (29.4 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size fdroid-dl-0.0.5.tar.gz (21.4 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page