Skip to main content
https://img.shields.io/badge/License-Apache%202.0-blue.svg https://travis-ci.org/qadium-memex/CommonCrawlJob.svg?branch=master https://badge.fury.io/py/CommonCrawlJob.svg

This work is supported by Qadium Inc as a part of the DARPA Memex Program.

Installation

The easiest way to get started is using pip to install a copy of this library. This will install the stable latest version hosted on PyPI.

$ pip install -e git+https://github.com/qadium-memex/CommonCrawlJob.git#egg=ccjob

Another way is to directly install the code from github to get the bleeding edge version of the code. If that is the case, you can still use pip by pointing it to github and specifying the protocol.

$ pip install CommonCrawlJob

Compatibility

Unfortunately, this code does not yet compatible with Python 3 and Python/PyPy 2.7 are the only current implementations which are tested against. Unfortunately the library for encoding WARC (Web Archive) file formats will need to undergo a rewrite it is possible to have deterministic IO behavior.

Metadata

Release files for CommonCrawlJob 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for CommonCrawlJob 0.1.0
File Size Uploaded Provenance Actions
CommonCrawlJob-0.1.0.tar.gz 312.4 kB Without provenance Details for CommonCrawlJob-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for CommonCrawlJob 0.1.0
File Interpreter ABI Platform Actions
CommonCrawlJob-0.1.0-py2-none-any.whl Python 2 none any Details for CommonCrawlJob-0.1.0-py2-none-any.whl

Total release size: 323.9 kB

Release files / CommonCrawlJob-0.1.0.tar.gz

Download URL CommonCrawlJob-0.1.0.tar.gz
Size 312.4 kB
Tags Source
SHA-256 checksum
How to use checksums
539c6934af1e84459b87a0465e09e8c21e7cf5bccc857cc1bda5db7fea476c7e
BLAKE2b-256 checksum
How to use checksums
6c568085072d352a8875c2c1efdf8ce4074d2b8b9181b6647de9e6c1fcf6a2d2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / CommonCrawlJob-0.1.0-py2-none-any.whl

Download URL CommonCrawlJob-0.1.0-py2-none-any.whl
Size 11.5 kB
Tags Python 2
SHA-256 checksum
How to use checksums
19bff00a963a86956ef886e109244625576afd29df5ee43c27e40d0d33d8a2d2
BLAKE2b-256 checksum
How to use checksums
55caaa387832f21c37e8020dcd414b1c58211d21c3358617d3aca61c67032868
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for CommonCrawlJob 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page