6 projects
simplemma
Fast and zero-dependency lemmatization, tokenization and sentence splitting for 54 languages.
trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML.
htmldate
Fast and robust extraction of original and updated publication dates from URLs and web pages.
courlan
Clean, filter and sample URLs to optimize data collection – includes spam, content type and language filters.
py3langid
Fork of the language identification tool langid.py, featuring a modernized codebase and faster execution times.
shoten
...
