Skip to main content

domain_utils

https://img.shields.io/pypi/v/domain_utils.svg CI Documentation Status

A collection of util functions for extracting domains from urls.

Repo: https://github.com/openwpm/domain_utils

Install:

pip install domain-utils

Use:

import domain_utils as du
# Return just the url `my.domain.cloudfront.net/a/path/to/a/file.html`
du.stem_url('https://my.domain.cloudfront.net/a/path/to/a/file.html?a=1')
# Return just the eTLD+1 `domain.cloudfront.net`
du.get_etld1('https://my.domain.cloudfront.net/a/path/to/a/file.html?a=1')
# Get the port `5000`
du.get_port('https://localhost:5000/a/path/to/a/file.html?a=1')
# Get the scheme `wss`
du.get_scheme('wss://somedomain.example.com/a/path/to/a/ws')

This package was originally extracted from openwpm-utils.

Community Participation Guidelines

This project is governed by Mozilla’s code of conduct and etiquette guidelines.

For more details, please read the Mozilla Community Participation Guidelines.

For more information on how to report violations of the Community Participation Guidelines, please read our How to Report page.

History

0.8.0 (2026-09-05)

  • Relax the tldextract pin from ==2.2.2 to >=5.3 (#32). The pinned version cached the public suffix list inside its own installed package directory and refreshed it with an unguarded check-then-act, so any consumer running more than one process could race and lose data. Upstream moved the cache out of the package directory in 3.1.0.

  • Drop support for Python 3.6 to 3.10; the minimum is now 3.11.

  • Ship type annotations. The package now carries a py.typed marker, so type checkers see the annotations, and the whole codebase is checked with pyright in strict mode. Passing extractor=None explicitly is now a type error as well as the ValueError it always raised; leave the argument out to get the automatically created extractor.

  • Replace CircleCI with GitHub Actions (#31). CI now runs a single just ci inside the nix shell, so a red build reproduces locally with the same command and the same tool versions.

  • Replace the Makefile with a justfile. Every target was a .PHONY command alias, which is a command runner’s job rather than make’s.

  • The test suite now fails below 100% statement and branch coverage (#27).

  • Fix the docs build, and point urls at openwpm/domain_utils. Install instructions use the domain-utils project name (#30).

  • Publish to PyPI from a version tag using trusted publishing.

  • Fix url parsing on modern CPython. urlparse no longer refuses to read host:port as a scheme, which meant localhost:8000 and example.com:8080 lost their port, and 127.0.0.1:8080/a?b=1 lost its host entirely.

  • Fix scheme_default=None, which raised AttributeError on Python 3.11 to 3.13 because urlparse gained WHATWG scheme stripping.

  • stem_url now returns the url exactly as it was passed in when return_unparsed is set and the scheme is not parsed, rather than an internally rewritten form.

  • stem_url no longer appends a stray trailing slash to a scheme-less url that already has a path, e.g. 10.0.0.1:80/a/b.html. A scheme-less bare host, e.g. example.com, still gets one.

0.7.1 (2020-04-10)

Fix building on readthedocs.

0.7.0 (2020-04-10)

Thanks to new contributor @yabirgb for two PRs (#20 and #25) in this release.

API changes: #26 renamed get_stripped_url to stem_url, and get_ps_plus_1 to get_etld1. Old method names will continue to work though. #22 updated keyword arguments to get_stripped_url - default behavior is basically the same.

  • API changes (#26 and #22)

  • Support parsing ws/wss urls (#22)

  • Add get_port method (#25)

  • Add get_scheme method (#20)

  • Correct license declaration in setup.py (#24)

0.6.0 (2020-04-06)

  • Use tldextract for parsing domains (#12)

  • Use numpy style docstrings

  • Support case of no scheme and port in URL (#13)

0.5.0 (2020-04-03)

  • Remove support for python 3.5

  • Handle more cases in get_stripped_url and change default behavior:

    • handle a lack of scheme

    • boolean flag to return or not non http urls - default is to return them which is a change of behavior as previously they would not return

    • Use netloc by default instead of hostname with a boolean flag to use hostname.

0.4.0 (2020-03-25)

  • Remove py27 support

0.3.0 (2020-03-25)

  • Restore py27 support.

  • Last version with py27 support.

  • Remove tox

0.2.0 (2020-03-24)

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

domain_utils-0.8.0.tar.gz (29.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

domain_utils-0.8.0-py3-none-any.whl (13.7 kB view details)

Uploaded Python 3

File details

Details for the file domain_utils-0.8.0.tar.gz.

File metadata

  • Download URL: domain_utils-0.8.0.tar.gz
  • Upload date:
  • Size: 29.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for domain_utils-0.8.0.tar.gz
Algorithm Hash digest
SHA256 24500d654585e05b542cdbc53c5a5b017d9bd6b9229632a23104c73a650f2a17
MD5 06c8dfd937ea66bb5a3ccdd79a6ebf7b
BLAKE2b-256 e62430f5513ab2c3322b6e84a879922d5f686c995448d71e7365edc613915a2d

See more details on using hashes here.

Provenance

The following attestation bundles were made for domain_utils-0.8.0.tar.gz:

Publisher: release.yml on openwpm/domain_utils

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file domain_utils-0.8.0-py3-none-any.whl.

File metadata

  • Download URL: domain_utils-0.8.0-py3-none-any.whl
  • Upload date:
  • Size: 13.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for domain_utils-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 094122ca0fd37975ec8a28e740b0bc122f840e219120091789805b0821094e22
MD5 5450d7c6f298cc05ccf8c25f30728fc6
BLAKE2b-256 f042c77f8cf42e708465341747a8c1c40310a92559da07eeb703fd729ad43ad9

See more details on using hashes here.

Provenance

The following attestation bundles were made for domain_utils-0.8.0-py3-none-any.whl:

Publisher: release.yml on openwpm/domain_utils

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.8.0 This release

2 files

0.7.1

2 files

0.7.0

2 files

0.6.0

2 files

0.5.0

2 files

0.4.0

2 files

0.3.0

2 files

0.2.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page