Skip to main content

Python implementation of the WHATWG URL Standard

Project description

urlstd

PyPI PyPI - Python Version PyPI - License CI codecov

urlstd is a Python implementation of the WHATWG URL Standard.

This library provides URL class, URLSearchParams class, and low-level APIs that comply with the URL specification.

Note: The latest release of urlstd is implemented based on the URL specification commit f787850.

Supported APIs

  • URL class

    • class urlstd.parse.URL(url: str, base: Optional[str] = None)
      • href: readonly property href: str
      • origin: readonly property origin: str
      • protocol: property protocol: str
      • username: property username: str
      • password: property password: str
      • host: property host: str
      • hostname: property hostname: str
      • port: property port: str
      • pathname: property pathname: str
      • search: property search: str
      • searchParams: readonly property search_params: URLSearchParams
      • hash: property hash: str
  • URLSearchParams class

    • class urlstd.parse.URLSearchParams(init: Optional[Union[str, Sequence[Sequence[Union[str, int, float]]], Dict[str, Union[str, int, float]], URLRecord, URLSearchParams]] = None)
      • append: append(name: str, value: Union[str, int, float]) -> None
      • delete: delete(name: str) -> None
      • get: get(name: str) -> Optional[str]
      • getAll: get_all(name: str) -> Tuple[str, ...]
      • has: has(name: str) -> bool
      • set: set(name: str, value: Union[str, int, float]) -> None
      • sort: sort() -> None
  • Low-level APIs

  • Compatibility with standard library urllib

    • urlstd.parse.urlparse(urlstring: str, base: str = None, encoding: str = "utf-8", allow_fragments: bool = True) -> urllib.parse.ParseResult

      urlstd.parse.urlparse() ia an alternative to urllib.parse.urlparse(). Parses a string representation of a URL using the basic URL parser, and returns urllib.parse.ParseResult.

Basic Usage

To parse a string into a URL with using a base URL:

from urlstd.parse import URL
url = URL('?ffi&🌈', 'http://example.org')
url  # → URL(href='http://example.org/?%EF%AC%83&%F0%9F%8C%88', origin='http://example.org', protocol='http:', username='', password='', host='example.org', hostname='example.org', port='', pathname='/', search='?%EF%AC%83&%F0%9F%8C%88', hash='')
url.search  # → '?%EF%AC%83&%F0%9F%8C%88'
params = url.search_params
params  # → URLSearchParams([('ffi', ''), ('🌈', '')])
params.sort()
params  # → URLSearchParams([('🌈', ''), ('ffi', '')])
url.search  # → '?%F0%9F%8C%88=&%EF%AC%83='
str(url)  # → 'http://example.org/?%F0%9F%8C%88=&%EF%AC%83='

To parse a string into a urllib.parse.ParseResult with using a base URL:

import html
from urllib.parse import unquote
from urlstd.parse import urlparse
pr = urlparse('?aÿb', 'http://example.org/foo/', encoding='utf-8')
pr  # → ParseResult(scheme='http', netloc='example.org', path='/foo/', params='', query='a%C3%BFb', fragment='')
unquote(pr.query)  # → 'aÿb'
pr = urlparse('?aÿb', 'http://example.org/foo/', encoding='windows-1251')
pr  # → ParseResult(scheme='http', netloc='example.org', path='/foo/', params='', query='a%26%23255%3Bb', fragment='')
unquote(pr.query, encoding='windows-1251')  # → 'aÿb'
html.unescape('aÿb')  # → 'aÿb'
pr = urlparse('?aÿb', 'http://example.org/foo/', encoding='windows-1252')
pr  # → ParseResult(scheme='http', netloc='example.org', path='/foo/', params='', query='a%FFb', fragment='')
unquote(pr.query, encoding='windows-1252')  # → 'aÿb'

Logging

urlstd uses standard library logging for validation error. Change the logger log level of urlstd if needed:

logging.getLogger('urlstd').setLevel(logging.ERROR)

Dependencies

Installation

  1. Configuring environment variables for icupy (ICU):

    • Windows:

      • Set the ICU_ROOT environment variable to the root of the ICU installation (default is C:\icu). For example, if the ICU is located in C:\icu4c:

        set ICU_ROOT=C:\icu4c
        

        or in PowerShell:

        $env:ICU_ROOT = "C:\icu4c"
        
      • To verify settings using icuinfo (64 bit):

        %ICU_ROOT%\bin64\icuinfo
        

        or in PowerShell:

        & $env:ICU_ROOT\bin64\icuinfo
        
    • Linux/POSIX:

      • If the ICU is located in a non-regular place, set the PKG_CONFIG_PATH and LD_LIBRARY_PATH environment variables. For example, if the ICU is located in /usr/local:

        export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
        export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
        
      • To verify settings using pkg-config:

        $ pkg-config --cflags --libs icu-uc
        -I/usr/local/include -L/usr/local/lib -licuuc -licudata
        
  2. Installing from PyPI:

    pip install urlstd
    

Running Tests

Install dependencies:

pip install tox

To run tests and generate a report:

git clone https://github.com/miute/urlstd.git
cd urlstd
tox -e wpt

See result: tests/wpt/report.html

License

MIT License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

urlstd-2021.10.25.post2.tar.gz (57.6 kB view hashes)

Uploaded Source

Built Distribution

urlstd-2021.10.25.post2-py3-none-any.whl (23.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page