6 projects
htmldate
Fast and robust extraction of original and updated publication dates from URLs and web pages.
trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML.
simplemma
A lightweight toolkit for multilingual lemmatization and language detection.
courlan
Clean, filter and sample URLs to optimize data collection – includes spam, content type and language filters.
py3langid
Fork of the language identification tool langid.py, featuring a modernized codebase and faster execution times.
shoten
...
