Skip to main content

Library of web-related functions

Project description

https://github.com/scrapy/w3lib/actions/workflows/tests.yml/badge.svg Coverage report

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 3.9+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-2.3.1.tar.gz (49.5 kB view details)

Uploaded Source

Built Distribution

w3lib-2.3.1-py3-none-any.whl (21.8 kB view details)

Uploaded Python 3

File details

Details for the file w3lib-2.3.1.tar.gz.

File metadata

  • Download URL: w3lib-2.3.1.tar.gz
  • Upload date:
  • Size: 49.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.1

File hashes

Hashes for w3lib-2.3.1.tar.gz
Algorithm Hash digest
SHA256 5c8ac02a3027576174c2b61eb9a2170ba1b197cae767080771b6f1febda249a4
MD5 7ef432898835f79a3c11100a992dbd03
BLAKE2b-256 bf7d1172cfaa1e29beb9bf938e484c122b3bdc82e8e37b17a4f753ba6d6e009f

See more details on using hashes here.

File details

Details for the file w3lib-2.3.1-py3-none-any.whl.

File metadata

  • Download URL: w3lib-2.3.1-py3-none-any.whl
  • Upload date:
  • Size: 21.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.1

File hashes

Hashes for w3lib-2.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 9ccd2ae10c8c41c7279cd8ad4fe65f834be894fe7bfdd7304b991fd69325847b
MD5 ac9c4dcca8a30f43a56fc4ad2fb53229
BLAKE2b-256 58dd56f0d8af71e475ed194d702f8b4cf9cea812c95e82ad823d239023c6558c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page