Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.11.0.tar.gz (12.1 kB view details)

Uploaded Source

Built Distribution

w3lib-1.11.0-py2.py3-none-any.whl (14.8 kB view details)

Uploaded Python 2Python 3

File details

Details for the file w3lib-1.11.0.tar.gz.

File metadata

  • Download URL: w3lib-1.11.0.tar.gz
  • Upload date:
  • Size: 12.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.11.0.tar.gz
Algorithm Hash digest
SHA256 826f6184232c385b7c8038ef7ae5506f1cb3f8fa470ac12ce0fb6fb0a592fb92
MD5 0ecadf121a90f1ac182b739a7d028ae2
BLAKE2b-256 381f3bcd2549e7e8cf26c2574474a8ffc03f91d13a9f9d3fff86e8a47e23e7c7

See more details on using hashes here.

File details

Details for the file w3lib-1.11.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.11.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 d06df19d0a7dfd06fb8d4b7c58e3f3bdc1862a2c92e145bbb7d5526008a54f8a
MD5 e286c4a808d72b7e0def8d27f625ca11
BLAKE2b-256 36e18717334035de69a638b42cbec59736adda4da1cdc67aec6a09075f9c0800

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page