A lib to work with html and web data
Project description
datahtml
datahtml is a library for crawling and extraction of data from html and xml content.
Datahtml lets you:
- Extract ld+json data from html
- Extract frequently used meta tags from html (those that are used for SEO and social media, between others)
- Extract Article data from a html, usually from Newspaper sites
- Parse RSS feeds from sites
- Crawl some specific social media sites like google and youtube
Under the hood datahtml uses libraries like BeautifoulSoup, Newspaper2k, feedparser between others, but datahtml takes an opinionated approach for crawling based on our expriencies doing so.
Quickstart
pip install datahtml
from datahtml import web, crawler
c = crawler.LocalCrawler()
w = web.download("https://www.infobae.com", crawler=c)
w.links()
License
datahtml
is distributed under the terms of the MPL-2.0 license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
datahtml-0.4.0rc6.tar.gz
(517.6 kB
view hashes)
Built Distribution
Close
Hashes for datahtml-0.4.0rc6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 08506c99e688e2bced4e5b9023f5dc77388c73d3bc9f02532a09a44245f1a0a6 |
|
MD5 | 3f1eb4163bbde33a7fdb62386d1e63c1 |
|
BLAKE2b-256 | 5040883934f87ab01e9bcb6ce77decaa3308b73e67277ed9aec776dd656734d6 |