Skip to main content

Library of web-related functions

Project description

https://github.com/scrapy/w3lib/actions/workflows/tests.yml/badge.svg Coverage report

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 3.9+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-2.3.0.tar.gz (49.5 kB view details)

Uploaded Source

Built Distribution

w3lib-2.3.0-py3-none-any.whl (21.8 kB view details)

Uploaded Python 3

File details

Details for the file w3lib-2.3.0.tar.gz.

File metadata

  • Download URL: w3lib-2.3.0.tar.gz
  • Upload date:
  • Size: 49.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.1

File hashes

Hashes for w3lib-2.3.0.tar.gz
Algorithm Hash digest
SHA256 5f45fa29863740bbd45cfd1f9d3e7c30e08b74b0a7f0c09d2b7d4e74253c17b8
MD5 ae07b78d6d5713eec02c49b7bf3e8d5b
BLAKE2b-256 51b637d42e87e58b165f7bde60f779988125241dfc23341d2062002ffbfb3213

See more details on using hashes here.

File details

Details for the file w3lib-2.3.0-py3-none-any.whl.

File metadata

  • Download URL: w3lib-2.3.0-py3-none-any.whl
  • Upload date:
  • Size: 21.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.1

File hashes

Hashes for w3lib-2.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c768e58b57db417612e94b3e5031a9676f619b56b65bb41a2459531160254b7a
MD5 1429e858631995a3d2c67be2bd32513e
BLAKE2b-256 b8532e3ba8c0552c93162c1ab1a588a34f54ffdee24dbfadf9329158a06d3b8b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page