Skip to main content

Library of web-related functions

Project description

https://github.com/scrapy/w3lib/actions/workflows/tests-ubuntu.yml/badge.svg Coverage report

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 3.10+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-2.4.0.tar.gz (48.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

w3lib-2.4.0-py3-none-any.whl (21.6 kB view details)

Uploaded Python 3

File details

Details for the file w3lib-2.4.0.tar.gz.

File metadata

  • Download URL: w3lib-2.4.0.tar.gz
  • Upload date:
  • Size: 48.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for w3lib-2.4.0.tar.gz
Algorithm Hash digest
SHA256 e233ad21649b69d0e047a10f30181ae9677524a29f6f71f6f3c758dc0c8d2648
MD5 1821e838e735e246950fbdc4fcb3d81a
BLAKE2b-256 f191186665abf1a6d16c0c5ea1f0e681d9c852b45c3a750aa8657f8f956690a8

See more details on using hashes here.

Provenance

The following attestation bundles were made for w3lib-2.4.0.tar.gz:

Publisher: publish.yml on scrapy/w3lib

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file w3lib-2.4.0-py3-none-any.whl.

File metadata

  • Download URL: w3lib-2.4.0-py3-none-any.whl
  • Upload date:
  • Size: 21.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for w3lib-2.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 260b5a22aeb86ae73213857f69ed20829a45150f8d5b12050b1f02ada414db79
MD5 0d51e266e8b312ee998d67c0211454b3
BLAKE2b-256 2ef5ce3ab627e0cb51591c9e3dc4b9b173f15d7f2bec1c0010420b15fc442940

See more details on using hashes here.

Provenance

The following attestation bundles were made for w3lib-2.4.0-py3-none-any.whl:

Publisher: publish.yml on scrapy/w3lib

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page