download-mediawiki 1.0

OXL Wiki-Download Script for MediaWiki instances

MediaWiki Download Script

This is a very simple script to download the whole contents of MediaWiki instance via its API.

This script should not be used to download huge Wiki's as it will not scale well!

---

Features

• Downloading the content of all pages in their source-format
  • Only downloading the latest revision (rvlimit)
• Optionally converting all source-files to Markdown-format (using pandoc)

We did not yet have the need to also download the images. Feel free to open a PR!

---

Usage

Install:

pip install download-mediawiki

# or clone & run directly
python3 src/download_mediawiki/

Arguments:

download-mediawiki --help
> usage: MediaWiki Download Script (© OXL IT Services, License: MIT) [-h] -u URL [-o OUT_DIR] [-r]
>                                                                    [-m]
> 
> options:
>   -h, --help            show this help message and exit
>   -u URL, --url URL     Base-URL of the MediaWiki instance
>   -o OUT_DIR, --out-dir OUT_DIR
>   -r, --replace         Replace/Update existing pages
>   -m, --convert-to-md   Convert all source-files to Markdown-format (pandoc executable
>                         required!)

---

Result

Overview: dump/overview.json

# namespace-id => page-id => page-title
cat dump/overview.json 
> {
>   "0": {
>     "1": "Page Title"
>   },
>   "15": {}
> }

Files: dump//.mw

tree dump/
> dump/
> ├── 0
> │   ├── 1.mw

Content

head dump/0/1.mw 
> # Main Page
> Welcome to the ''nftables'' HOWTO documentation page. Here you will find documentation on how to build, install, configure and use nftables.
> 
> If you have any suggestion to improve it, please send your comments to Netfilter users mailing list <[e-mail]>.
> 
> 
> = [[News]] =
> 
> 
> = Introduction =