mrkoll.se scraper & RSS generator
Reason this release was yanked:
Breaking bug, use v0.3.2 instead
Project description
mrkoll-scraper
Tool to scrape and generate an RSS feed from the incredibly creepy mrkoll.se Swedish toplist. Each RSS entry represents the list as it looked at a given point in time, plus convenience links to search for the person's name on Duckduckgo, Flashback, and The Facebook.
Installation
pip install mrkoll-scraper
Usage
CLI
mrks --scrape
Scrapes the current list and saves it to the Shelve database mrks.db
in the current working directory. This operation saves the raw scraped HTML as well as a generated feedgen FeedEntry
. The data is saved in a dictionary with the current date as key, so multiple scrapes during the same day doesn't save a new entry, they only update the existing one.
mrks --regenerate
Iterates through the saved lists and re-generates FeedEntry
's from the raw HTML, in case you've made some changes in the HTML template or so, and want them applied retroactively.
WSGI
mrks.wsgi
contains a beautifully simple WSGI application, that simply outputs an RSS feed based on the data currently saved.
Cron
Crontab to run --scrape
every Monday at midnight:
0 0 * * 1 cd /home/robert/mrkoll-scraper && /home/robert/mrkoll-scraper/venv/bin/mrks -s
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mrkoll_scraper-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d662dd7bf23c83d7444b90a6670cad6acec1fa98e7bdb20b1fef3ac6b255b39 |
|
MD5 | 05059b8bc662dcaabfe0a92a9bbe7ce0 |
|
BLAKE2b-256 | 057677ef13e0041923a8eaba8fa002522ece27dac3ec723c29968593b5623968 |