domain_utils
A collection of util functions for extracting domains from urls.
Repo: https://github.com/openwpm/domain_utils
Install:
pip install domain-utils
Use:
import domain_utils as du
# Return just the url `my.domain.cloudfront.net/a/path/to/a/file.html`
du.stem_url('https://my.domain.cloudfront.net/a/path/to/a/file.html?a=1')
# Return just the eTLD+1 `domain.cloudfront.net`
du.get_etld1('https://my.domain.cloudfront.net/a/path/to/a/file.html?a=1')
# Get the port `5000`
du.get_port('https://localhost:5000/a/path/to/a/file.html?a=1')
# Get the scheme `wss`
du.get_scheme('wss://somedomain.example.com/a/path/to/a/ws')
This package was originally extracted from openwpm-utils.
Free software: Mozilla Public License license
Documentation: https://domain-utils.readthedocs.io.
Community Participation Guidelines
This project is governed by Mozilla’s code of conduct and etiquette guidelines.
For more details, please read the Mozilla Community Participation Guidelines.
For more information on how to report violations of the Community Participation Guidelines, please read our How to Report page.
History
0.8.0 (2026-09-05)
Relax the tldextract pin from ==2.2.2 to >=5.3 (#32). The pinned version cached the public suffix list inside its own installed package directory and refreshed it with an unguarded check-then-act, so any consumer running more than one process could race and lose data. Upstream moved the cache out of the package directory in 3.1.0.
Drop support for Python 3.6 to 3.10; the minimum is now 3.11.
Ship type annotations. The package now carries a py.typed marker, so type checkers see the annotations, and the whole codebase is checked with pyright in strict mode. Passing extractor=None explicitly is now a type error as well as the ValueError it always raised; leave the argument out to get the automatically created extractor.
Replace CircleCI with GitHub Actions (#31). CI now runs a single just ci inside the nix shell, so a red build reproduces locally with the same command and the same tool versions.
Replace the Makefile with a justfile. Every target was a .PHONY command alias, which is a command runner’s job rather than make’s.
The test suite now fails below 100% statement and branch coverage (#27).
Fix the docs build, and point urls at openwpm/domain_utils. Install instructions use the domain-utils project name (#30).
Publish to PyPI from a version tag using trusted publishing.
Fix url parsing on modern CPython. urlparse no longer refuses to read host:port as a scheme, which meant localhost:8000 and example.com:8080 lost their port, and 127.0.0.1:8080/a?b=1 lost its host entirely.
Fix scheme_default=None, which raised AttributeError on Python 3.11 to 3.13 because urlparse gained WHATWG scheme stripping.
stem_url now returns the url exactly as it was passed in when return_unparsed is set and the scheme is not parsed, rather than an internally rewritten form.
stem_url no longer appends a stray trailing slash to a scheme-less url that already has a path, e.g. 10.0.0.1:80/a/b.html. A scheme-less bare host, e.g. example.com, still gets one.
0.7.1 (2020-04-10)
Fix building on readthedocs.
0.7.0 (2020-04-10)
Thanks to new contributor @yabirgb for two PRs (#20 and #25) in this release.
API changes: #26 renamed get_stripped_url to stem_url, and get_ps_plus_1 to get_etld1. Old method names will continue to work though. #22 updated keyword arguments to get_stripped_url - default behavior is basically the same.
API changes (#26 and #22)
Support parsing ws/wss urls (#22)
Add get_port method (#25)
Add get_scheme method (#20)
Correct license declaration in setup.py (#24)
0.6.0 (2020-04-06)
Use tldextract for parsing domains (#12)
Use numpy style docstrings
Support case of no scheme and port in URL (#13)
0.5.0 (2020-04-03)
Remove support for python 3.5
Handle more cases in get_stripped_url and change default behavior:
handle a lack of scheme
boolean flag to return or not non http urls - default is to return them which is a change of behavior as previously they would not return
Use netloc by default instead of hostname with a boolean flag to use hostname.
0.4.0 (2020-03-25)
Remove py27 support
0.3.0 (2020-03-25)
Restore py27 support.
Last version with py27 support.
Remove tox
0.2.0 (2020-03-24)
Extracted from https://github.com/mozilla/openwpm-utils/blob/master/openwpm_utils/domain.py
Removed python 2 support and dependencies
Removed broken get_stripped_urls function
First release on PyPI.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file domain_utils-0.8.0.tar.gz.
File metadata
- Download URL: domain_utils-0.8.0.tar.gz
- Upload date:
- Size: 29.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
24500d654585e05b542cdbc53c5a5b017d9bd6b9229632a23104c73a650f2a17
|
|
| MD5 |
06c8dfd937ea66bb5a3ccdd79a6ebf7b
|
|
| BLAKE2b-256 |
e62430f5513ab2c3322b6e84a879922d5f686c995448d71e7365edc613915a2d
|
Provenance
The following attestation bundles were made for domain_utils-0.8.0.tar.gz:
Publisher:
release.yml on openwpm/domain_utils
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
domain_utils-0.8.0.tar.gz -
Subject digest:
24500d654585e05b542cdbc53c5a5b017d9bd6b9229632a23104c73a650f2a17 - Sigstore transparency entry: 2731703171
- Sigstore integration time:
-
Permalink:
openwpm/domain_utils@a04ec92999330af7f649ffe9ad993aea5edd2b07 -
Branch / Tag:
refs/tags/v0.8.0 - Owner: https://github.com/openwpm
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@a04ec92999330af7f649ffe9ad993aea5edd2b07 -
Trigger Event:
push
-
Statement type:
File details
Details for the file domain_utils-0.8.0-py3-none-any.whl.
File metadata
- Download URL: domain_utils-0.8.0-py3-none-any.whl
- Upload date:
- Size: 13.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
094122ca0fd37975ec8a28e740b0bc122f840e219120091789805b0821094e22
|
|
| MD5 |
5450d7c6f298cc05ccf8c25f30728fc6
|
|
| BLAKE2b-256 |
f042c77f8cf42e708465341747a8c1c40310a92559da07eeb703fd729ad43ad9
|
Provenance
The following attestation bundles were made for domain_utils-0.8.0-py3-none-any.whl:
Publisher:
release.yml on openwpm/domain_utils
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
domain_utils-0.8.0-py3-none-any.whl -
Subject digest:
094122ca0fd37975ec8a28e740b0bc122f840e219120091789805b0821094e22 - Sigstore transparency entry: 2731703995
- Sigstore integration time:
-
Permalink:
openwpm/domain_utils@a04ec92999330af7f649ffe9ad993aea5edd2b07 -
Branch / Tag:
refs/tags/v0.8.0 - Owner: https://github.com/openwpm
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@a04ec92999330af7f649ffe9ad993aea5edd2b07 -
Trigger Event:
push
-
Statement type: