PyCommonCrawl
A python interface for Common Crawl.
INSTALL
pip3 install pycommoncrawl
USAGE
from pycommoncrawl.common_crawl_data_accessor import CommonCrawlDataAccessor
common_crawl_data_accessor = CommonCrawlDataAccessor()
# Iterate by line
for line in common_crawl_data_accessor.get_raw_resource_data("WAT"):
print(line)
# Iterate by WARC bloc
for warc in common_crawl_data_accessor.get_raw_resource_data_per_warc("WAT"):
print(warc["Content-Length"])
1 maintainer
Release files for pycommoncrawl 0.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pycommoncrawl-0.2-py3-none-any.whl | Python 3 | none | any | Details |
Release files / pycommoncrawl-0.2-py3-none-any.whl
| Download URL | pycommoncrawl-0.2-py3-none-any.whl |
|---|---|
| Size | 5.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a6b9fd48bc7717281f78098ab91831751aa4527cedff093d90234b82574a9552
|
|
BLAKE2b-256 checksum How to use checksums |
c5175362da86620e4a6c832f1614d69f271e4fc80b410706f1a195221e75f683
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.6.8
|
