Mindtouch scraper

This scraper transform content on a Mindtouch / Nice CXone Expert instance into a in ZIM file, a clean and user friendly format for storing content for offline usage.

While meant to be generic for any Mindtouch instance, it has so far been tested only on libretexts.org instance, so it is highly expectable some rough edges might need to be polished until it works on all Mindtouch instance.

Installation

There are three main ways to install and use mindtouch2zim from most recommended to least:

Usage

# Get help
docker run -v output:/output ghcr.io/openzim/mindtouch mindtouch2zim --help

# Create a ZIM for https://geo.libretexts.org
docker run -v output:/output ghcr.io/openzim/mindtouch mindtouch2zim --library-url https://geo.libretexts.org --name libretexts.org_en_geo --creator LibreTexts --title "LibreTexts Geosciences" --description "Geosciences courses from libretexts.org"

Developing

Use the commands below to set up the project once:

# Install hatch if it isn't installed already.
❯ pip install hatch

# Local install (in default env) / re-sync packages
❯ hatch run pip list

# Set-up pre-commit
❯ pre-commit install

The following commands can be used to build and test the scraper:

# Show scripts
❯ hatch env show

# linting, testing, coverage, checking
❯ hatch run lint:all
❯ hatch run lint:fixall

# run tests on all matrixed' envs
❯ hatch run test:run

# run tests in a single matrixed' env
❯ hatch env run -e test -i py=3.12 coverage

# run static type checks
❯ hatch env run check:all

# building packages
❯ hatch build

Contributing

This project adheres to openZIM's Contribution Guidelines.

This project has implemented openZIM's Python bootstrap, conventions and policies v1.0.3.

See details for contributions in CONTRIBUTING.md.