Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.8.0.tar.gz (12.3 kB view details)

Uploaded Source

Built Distribution

w3lib-1.8.0-py2.py3-none-any.whl (14.7 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file w3lib-1.8.0.tar.gz.

File metadata

  • Download URL: w3lib-1.8.0.tar.gz
  • Upload date:
  • Size: 12.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.8.0.tar.gz
Algorithm Hash digest
SHA256 b11a5a2e3c1534ebe4a9674cc66912a4df126b650a8db2af892e8bfaf3fefda2
MD5 4c7d3cf7ba664811ee730738a77fb1e7
BLAKE2b-256 034531aa0a6e50c6fcfd903625e28ca4ab9e4d9960972358d054004d411cc5fc

See more details on using hashes here.

File details

Details for the file w3lib-1.8.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.8.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 62d790bb54dc43421e0e36507b4b1ab5bc44f5987427fdcbbc6c029d4fb9f6ee
MD5 0906c04d2cae8eae06d723526a3f5392
BLAKE2b-256 00e66e9566b8781baa8e9518b937574af777b7555bf0e241e65a3d49b02b409f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page