Skip to main content

Dark Keeper is open source simple web-parser for podcast-sites

Project description

GitHub Actions CI

Dark Keeper

Dark Keeper is open source simple web-parser for podcast-sites. Also you can use it for any sites.
Goal idea: parsing full information per each podcast episodes like number, description and download link.


  • simple web-spider walking on site
  • cache for all downloaded pages
  • parse any information from pages
  • export parsed data to MongoDB

Quick start

$ mkvirtualenv keeper
(keeper)$ pip install dark-keeper
(keeper)$ cat

from dark_keeper import BaseParser, DarkKeeper
from dark_keeper.exports import ExportMongo
from dark_keeper.http import HttpClient
from dark_keeper.storages import UrlsStorage, DataStorage

class PodcastParser(BaseParser):
    def parse_urls(self, content):
        urls = content.parse_urls('.posts-list > .container-fluid .text-left a')

        return urls

    def parse_data(self, content):
        data = []
        for post_item in content.get_block_items('.posts-list .posts-list-item'):
            post_data = dict(
                mp3=post_item.parse_attr('.post-podcast-content audio', 'src'),

            if post_data['title'] and post_data['mp3']:

        return data

if __name__ == '__main__':
    pk = DarkKeeper(
            user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) '
                       'AppleWebKit/537.36 (KHTML, like Gecko) '
                       'Chrome/81.0.4044.138 Safari/537.36 OPR/68.0.3618.125',

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dark-keeper-0.3.1.tar.gz (6.9 kB view hashes)

Uploaded source

Built Distribution

dark_keeper-0.3.1-py3-none-any.whl (8.3 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page