Skip to main content

Scraper for ALLRIS

Project description

Apache License 2.0 Python 3.7 and 3.8 version 0.4.0

This scraper offers both public and private scraping. The latter requires your username and password and performs the following tasks for you:

  • login

  • download of all agendas and motions related to upcoming meetings of committees and plenary sessions

    • Only considers meetings where you already have been invited formally through ALLRIS.

IMPORTANT: All districts are supported but official committee abbreviations will only work for Eimsbüttel as of now.

The public scraper ought to be used with care as it accesses all accessible pages of an entire month. Currently, June 2020 is hardcoded and it can only be used in a GUI environment.

Requirements

Initial setup

  1. Install ALLRIS scraper pip install twomartens.allrisscraper (you need Python 3.7+)

  2. Run ALLRIS scraper a first time tm-allrisscraper (creates config ini in your current working directory)

  3. Fill out the config file with your login credentials and an absolute path on your system to store PDFs of files

Configuration

[Default]
; possible values for district: Altona, Bergedorf, Eimsbüttel, Hamburg-Nord,
; Hamburg-Mitte, Harburg, Wandsbek
district = Eimsbüttel
; if you are not from Eimsbüttel your domain ending will differ
username = max.mustermann@eimsbuettel.de
; password is stored in clear text, therefore ini file should have most
; restrictive read permissions
password = VerySecurePassword
; location for storage of PDFs (trailing slash is IMPORTANT)
pdflocation = /path/to/storage/of/PDFs/
; location of the firefox binary
firefoxBinary = /Pfad/zur/firefox.exe
; location of the geckodriver binary
geckodriver = /Pfad/zum/geckodriver

Usage after initial setup

Run ALLRIS scraper: tm-allrisscraper (takes a few seconds to finish)

In the specified location for download you will find the following structure:

  • YYYY-MM-DD_Abbreviation of committee or plenary session/ (one directory for each meeting)

  • files inside the directory: Einladung.pdf (contains invitation), Mappe.pdf (contains all motions in one document), and Tagesordnung.pdf (agenda)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

twomartens.allrisscraper-0.4.0.tar.gz (20.4 kB view details)

Uploaded Source

Built Distribution

twomartens.allrisscraper-0.4.0-py3-none-any.whl (21.5 kB view details)

Uploaded Python 3

File details

Details for the file twomartens.allrisscraper-0.4.0.tar.gz.

File metadata

  • Download URL: twomartens.allrisscraper-0.4.0.tar.gz
  • Upload date:
  • Size: 20.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.8.0

File hashes

Hashes for twomartens.allrisscraper-0.4.0.tar.gz
Algorithm Hash digest
SHA256 5ca27db23e716347106214db56c7a08d93e7a30c3b05bd8a27e7a2cdfb8978b6
MD5 ced3161f66b9f4a39e49c12dd2cd2a84
BLAKE2b-256 c256a1f0149340ade5a18d95b51e64e67f3b948a27384d66e76a6e4fb88fe6c8

See more details on using hashes here.

File details

Details for the file twomartens.allrisscraper-0.4.0-py3-none-any.whl.

File metadata

  • Download URL: twomartens.allrisscraper-0.4.0-py3-none-any.whl
  • Upload date:
  • Size: 21.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.8.0

File hashes

Hashes for twomartens.allrisscraper-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 79afc51987be46711af6356cd4f6c6f0ef36e3ad9e310e2b072f8f9c9fdc7277
MD5 e3bacaf630213657e8fb5bb4bb2da454
BLAKE2b-256 8fe29e779a278eb0837fe6a09f3570b27bcae54129c254dc9d1af298a9234195

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page