dark-keeper

Dark Keeper

Dark Keeper is open source simple web-parser for podcast-sites.

Goal idea

I like listen IT-podcasts and learn something new. For really good podcasts I want download all episodes. Goal idea is create simple tool for this.

Features

[x] simple web-spider walking on site
[x] cache for all downloaded pages
[x] parse any information from pages
[x] export parsed data to MongoDB

Quick start

$ mkvirtualenv keeper

(keeper)$ pip install dark-keeper

(keeper)$ cat app.py

from collections import OrderedDict

from dark_keeper import DarkKeeper


class PodcastKeeper(DarkKeeper):
    base_url = 'https://radio-t.com/archives/'
    mongo_uri = 'mongodb://localhost/podcasts/radio-t.com'

    def parse_urls(self, content):
        urls = content.parse_urls('#blog-archives h1 a', self.base_url)

        return urls

    def parse_data(self, content):
        data = OrderedDict()
        data['title'] = content.parse_text('.hentry .entry-title')
        data['desc'] = content.parse_text('.hentry .entry-content')
        data['mp3'] = content.parse_attr('.hentry audio', 'src')

        if data['title'] and data['mp3']:
            return data


if __name__ == '__main__':
    pk = PodcastKeeper()
    pk.run()

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dark-keeper-0.2.7.tar.gz (4.6 kB view details)

Uploaded Sep 9, 2017 Source

File details

Details for the file dark-keeper-0.2.7.tar.gz.

File metadata

Download URL: dark-keeper-0.2.7.tar.gz
Upload date: Sep 9, 2017
Size: 4.6 kB
Tags: Source
Uploaded using Trusted Publishing? No

File hashes

Hashes for dark-keeper-0.2.7.tar.gz
Algorithm	Hash digest
SHA256	`bdc3cf438e04b09532837fc8662e8a5f469a20f5d792a2f1e4f0c97fc46b81b1`
MD5	`d5dbe1a464d81ba0e22c0032c6e1aa05`
BLAKE2b-256	`a212975b6f4af983006622e4ecfaf9ecbc7a51712ee1b40ee0fa4ac86c541f76`