datahtml 0.6.0

A lib to work with html and web data

datahtml

---

datahtml is a library for crawling and extraction of data from html and xml content.

Datahtml lets you:

• Extract ld+json data from html
• Extract frequently used meta tags from html (those that are used for SEO and social media, between others)
• Extract Article data from a html, usually from Newspaper sites
• Parse RSS feeds from sites
• Crawl some specific social media sites like google and youtube

Under the hood datahtml uses libraries like BeautifoulSoup, Newspaper2k, feedparser between others, but datahtml takes an opinionated approach for crawling based on our expriencies doing so.

Quickstart

pip install datahtml

from datahtml import web, crawler

c = crawler.LocalCrawler()
w = web.download("https://www.infobae.com", crawler=c)
w.links()

License

datahtml is distributed under the terms of the MPL-2.0 license.