domain_utils
A collection of util functions for extracting domains from urls.
Repo: https://github.com/openwpm/domain_utils
Install:
pip install domain-utils
Use:
import domain_utils as du
# Return just the url `my.domain.cloudfront.net/a/path/to/a/file.html`
du.stem_url('https://my.domain.cloudfront.net/a/path/to/a/file.html?a=1')
# Return just the eTLD+1 `domain.cloudfront.net`
du.get_etld1('https://my.domain.cloudfront.net/a/path/to/a/file.html?a=1')
# Get the port `5000`
du.get_port('https://localhost:5000/a/path/to/a/file.html?a=1')
# Get the scheme `wss`
du.get_scheme('wss://somedomain.example.com/a/path/to/a/ws')
This package was originally extracted from openwpm-utils.
Free software: Mozilla Public License license
Documentation: https://domain-utils.readthedocs.io.
Community Participation Guidelines
This project is governed by Mozilla’s code of conduct and etiquette guidelines.
For more details, please read the Mozilla Community Participation Guidelines.
For more information on how to report violations of the Community Participation Guidelines, please read our How to Report page.
History
0.8.0 (2026-09-05)
Relax the tldextract pin from ==2.2.2 to >=5.3 (#32). The pinned version cached the public suffix list inside its own installed package directory and refreshed it with an unguarded check-then-act, so any consumer running more than one process could race and lose data. Upstream moved the cache out of the package directory in 3.1.0.
Drop support for Python 3.6 to 3.10; the minimum is now 3.11.
Ship type annotations. The package now carries a py.typed marker, so type checkers see the annotations, and the whole codebase is checked with pyright in strict mode. Passing extractor=None explicitly is now a type error as well as the ValueError it always raised; leave the argument out to get the automatically created extractor.
Replace CircleCI with GitHub Actions (#31). CI now runs a single just ci inside the nix shell, so a red build reproduces locally with the same command and the same tool versions.
Replace the Makefile with a justfile. Every target was a .PHONY command alias, which is a command runner’s job rather than make’s.
The test suite now fails below 100% statement and branch coverage (#27).
Fix the docs build, and point urls at openwpm/domain_utils. Install instructions use the domain-utils project name (#30).
Publish to PyPI from a version tag using trusted publishing.
Fix url parsing on modern CPython. urlparse no longer refuses to read host:port as a scheme, which meant localhost:8000 and example.com:8080 lost their port, and 127.0.0.1:8080/a?b=1 lost its host entirely.
Fix scheme_default=None, which raised AttributeError on Python 3.11 to 3.13 because urlparse gained WHATWG scheme stripping.
stem_url now returns the url exactly as it was passed in when return_unparsed is set and the scheme is not parsed, rather than an internally rewritten form.
stem_url no longer appends a stray trailing slash to a scheme-less url that already has a path, e.g. 10.0.0.1:80/a/b.html. A scheme-less bare host, e.g. example.com, still gets one.
0.7.1 (2020-04-10)
Fix building on readthedocs.
0.7.0 (2020-04-10)
Thanks to new contributor @yabirgb for two PRs (#20 and #25) in this release.
API changes: #26 renamed get_stripped_url to stem_url, and get_ps_plus_1 to get_etld1. Old method names will continue to work though. #22 updated keyword arguments to get_stripped_url - default behavior is basically the same.
API changes (#26 and #22)
Support parsing ws/wss urls (#22)
Add get_port method (#25)
Add get_scheme method (#20)
Correct license declaration in setup.py (#24)
0.6.0 (2020-04-06)
Use tldextract for parsing domains (#12)
Use numpy style docstrings
Support case of no scheme and port in URL (#13)
0.5.0 (2020-04-03)
Remove support for python 3.5
Handle more cases in get_stripped_url and change default behavior:
handle a lack of scheme
boolean flag to return or not non http urls - default is to return them which is a change of behavior as previously they would not return
Use netloc by default instead of hostname with a boolean flag to use hostname.
0.4.0 (2020-03-25)
Remove py27 support
0.3.0 (2020-03-25)
Restore py27 support.
Last version with py27 support.
Remove tox
0.2.0 (2020-03-24)
Extracted from https://github.com/mozilla/openwpm-utils/blob/master/openwpm_utils/domain.py
Removed python 2 support and dependencies
Removed broken get_stripped_urls function
First release on PyPI.
Metadata
Release files for domain-utils 0.8.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| domain_utils-0.8.0.tar.gz | 29.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| domain_utils-0.8.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 43.4 kB
Release files / domain_utils-0.8.0.tar.gz
| Download URL | domain_utils-0.8.0.tar.gz |
|---|---|
| Size | 29.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
24500d654585e05b542cdbc53c5a5b017d9bd6b9229632a23104c73a650f2a17
|
|
BLAKE2b-256 checksum How to use checksums |
e62430f5513ab2c3322b6e84a879922d5f686c995448d71e7365edc613915a2d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 5, 2026.
Transparency logRelease files / domain_utils-0.8.0-py3-none-any.whl
| Download URL | domain_utils-0.8.0-py3-none-any.whl |
|---|---|
| Size | 13.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
094122ca0fd37975ec8a28e740b0bc122f840e219120091789805b0821094e22
|
|
BLAKE2b-256 checksum How to use checksums |
f042c77f8cf42e708465341747a8c1c40310a92559da07eeb703fd729ad43ad9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 5, 2026.
Transparency log