getdat 0.2.2

A command line utility for getting resources available online

ATTENTION - Anna's Archive enforces web scraping blocks through Cloudflare. The site is also subject to change. The ebook command may return No Search Results found. A fix is in the works that may involve a new project and archiving this one. A long term solution is hosting an instance of Anna's Archive with community support or possibly creating a cli by forking the main repo and building an api.

GetDat

A command line utility for getting resources available online

GetDat Theme Music

Table of Contents

• Installation Methods
• Commands
• Local Development
• Deployment
  • Publishing Getdat
  • Publishing Docs
• Contributors

Installation Methods

PYPI - Stable Release:

-> pipx install getdat

From Head commit of Repository:

-> pipx install git+https://github.com/Audiosutras/getdat.git

Commands

Job

Launches Braintrust, the job board powered by blockchain, in your default browser

Sport

Launches Totalsportk in your default browser

Cinema

Launches movie-web.app in your default browser

-> getdat cinema

Ebook

Search and download an ebook available through Anna's Archive. You can think of this command as "headless" Anna's Archive

• The demo for this command downloaded an epub format of Robert Louis Stevenson's and N.C. Wyeth's book Treasure Island. This book is in the public domain.
• The demo for this command opens the downloaded ebook using epr, a terminal epub reader to show the contents of the downloaded book. You can expect higher ebook quality by using a desktop e-reader like librum
• Anna's Archive SciDB search is not yet supported.

-> getdat ebook [OPTIONS] [Q]

ARGUMENTS

Name	Help
Q	Search

Example:

-> getdat ebook "Treasure Island Stevenson"

or

-> getdat ebook Treasure Island Stevenson

OPTIONS

-> getdat ebook --help
Usage: getdat ebook [OPTIONS] [Q]...

  Search and download an ebook available through Anna's Archive

  ex: getdat ebook <Search>

Options:
  -o, --output_dir TEXT           Path to ebook's output directory from home
                                  directory. Path must be prefixed by '~' on
                                  Unix or '~user' on Windows. This argument
                                  overrides GETDAT_BOOK_DIR env var if set.
                                  Outputs book to working directory if neither
                                  are set.
  -e, --ext [pdf|epub|mobi|cbr|cbz|fb2|fb2.zip|azw3|djvu]
                                  Preferred ebook extension for ebooks in
                                  search results.
  -l, --lang TEXT                 Preferred language of ebooks in search
                                  results. Provided Language must be ISO 639-1
                                  format. Language region extension supported.
                                  Filtering by multiple languages supported.
                                  Examples: English - en, Spanish - es,
                                  Traditional Chinese - zh-Hant, Multiple
                                  Langauges - en,es,zh-Hant
  -i, --instance [org|gs|se]      The instance of Anna's Archive you would
                                  like to use for your search:  https://annas-
                                  archive.org, https://annas-archive.gs,
                                  https://annas-archive.se - Default: org
  --help                          Show this message and exit.

Example:

-> getdat ebook Treasure Island Stevenson --ext=epub --output_dir=~/books/epub/ --instance=gs --lang=en,es,zh-Hant

or

-> getdat ebook "Treasure Island Stevenson" -e epub -o ~/books/epub -i gs -l es

or

-> getdat ebook "Treasure Island Stevenson"

Environment Variable

• GETDAT_BOOK_DIR - Path from home directory to destination directory. Ignored if --output_dir is specified as an option

Local Development

Python Version: 3.11. To install python on MacOS & Debian-based systems

-> sudo apt install software-properties-common
-> sudo add-apt-repository ppa:deadsnakes/ppa
-> sudo apt update
-> sudo apt install python3.11

This python package uses poetry for dependency management. To install:

# install pipx if you have not already
-> python3 -m pip install --user pipx
-> python3 -m pipx ensurepath
# install poetry
-> pipx install poetry

Assuming that you have forked the repository and have a copy on your local machine. Within the getdat directory, install dependencies and open a virtualenv shell managed by poetry.

-> poetry install
-> poetry shell
(getdat-py3.11) ->

To run/develop the cli program.

(getdat-py3.11) -> getdat
Usage: getdat [OPTIONS] COMMAND [ARGS]...

  A command line utility for getting resources available online

Options:
  --help  Show this message and exit.

Commands:
  cinema  Launches movie-web.app in your default browser
  ebook   Search and download an ebook available through Anna's Archive...

  Check out our docs at https://github.com/Audiosutras/getdat/docs for more
  details

Project uses pytest for unit testing. Test run quickly with the help of pytest-mock

(getdat-py3.11) -> pytest -v --cov=. tests/

Style guide and code check enforced with pre-commit

(getdat-py3.11) -> pre-commit install

Running docs site locally for making changes to github-pages requires ruby to be installed. After installation make sure to install bundler with gem install jekyll bundler as well:

-> cd docs
-> bundler install
-> bundler exec jekyll build
-> bundler exec jekyll serve

Deployment

Publishing Getdat

Workflow: master branch

• In pyproject.toml bump the version number *.*.*

• Create a git tag with the new version number *.*.* you specified in pyproject.toml.

• Push the newly created tag git push origin *.*.* to the repository. This will trigger the pre-release.yml github workflow to publish our package to test.pypi. The pre-release can be seen here for testing. Install with:

-> python3.11 -m pip install --index-url https://test.pypi.org/simple/ getdat --extra-index-url https://pypi.org/simple beautifulsoup4 requests click

- *Note*: `--extra-index-url` option is pulling dependencies from `pypi.org` and not `test.pypi.org` though our package is coming in from `test.pypi.org`. Make sure to add all dependencies from `[tool.poetry.dependencies]` in `pyproject.toml` (except python) before running this command.

• Create a release on github. Make sure to select Tags from the toggle menu. Select the latest tag (highest version number). Name the release Release *.*.*. Make sure the version number in pyproject.toml syncs up with the release version. Click Publish release. This will kick off our release.yml workflow to publish our package to pypi. The release can be seen here and installed for production use. Install with:

-> pipx install getdat

Publishing Docs

Docs are published to the site when a release is created for the python package to pypi.org. This is automated, but can be triggered manually by running the github workflow Deploy Jekyll with GitHub Pages dependencies preinstalled.

---

Contributors