Skip to main content

url-normalize

tests Coveralls PyPI Python Versions License Ruff

Normalize URL spelling consistently for storage, comparison, and display. Use url_normalize() for normalized URLs and url_humanize() for readable display URLs.

Requires Python 3.11 or newer. Normalization operates on URL text; it does not fetch pages or follow redirects.

Installation

These examples use url-normalize 4.0.0. If you store normalized URLs or use custom parameter filters, read the 4.0.0 upgrade notes before upgrading. The 3.1.0 usage guide covers the previous release.

Install the library and CLI:

python -m pip install "url-normalize>=4.0.0"

Or install the CLI in its own environment with uv:

uv tool install "url-normalize>=4.0.0"

Quick start

from url_normalize import url_normalize

print(url_normalize("HTTP://Example.COM:80/a/../b"))
# Output: http://example.com/b

print(url_normalize("bücher.de/résumé"))
# Output: https://xn--bcher-kva.de/r%C3%A9sum%C3%A9

From the terminal:

url-normalize "HTTP://Example.COM:80/a/../b"
# Output: http://example.com/b

Normalization behavior

Behavior Input Output
Add a missing scheme example.com https://example.com/
Lowercase scheme and host; remove a default port HTTP://Example.COM:80/ http://example.com/
Resolve dot segments https://example.com/a/../b https://example.com/b
Collapse repeated path slashes https://example.com/a//b/// https://example.com/a/b/
Encode Unicode hosts and paths https://bücher.de/résumé https://xn--bcher-kva.de/r%C3%A9sum%C3%A9
Remove one trailing host dot, including Unicode equivalents https://example.com。/ https://example.com/
Preserve encoded path separators; uppercase escape hex https://example.com/a%2fb https://example.com/a%2Fb
Preserve query order and repeated keys https://example.com/?b=2&a=1&a=3 https://example.com/?b=2&a=1&a=3
Preserve explicit empty query values https://example.com/?flag=&flag https://example.com/?flag=&flag
Remove empty query and fragment components https://example.com/?# https://example.com/
Preserve an explicit relative reference ../images/a.png ../images/a.png
Preserve fragment routes https://example.com/#!/users/42 https://example.com/#!/users/42
Preserve text-highlight syntax https://example.com/#:~:text=hello%20world https://example.com/#:~:text=hello%20world

In query parameters, +, %2B, and %20 remain distinct. Parameter filtering is disabled by default. Fragments are preserved and normalized by default, including hashbang routes (#!/route) and text directives (#:~:text=hello). Use strip_fragment=True to remove the entire fragment explicitly.

Fragments keep literal RFC 3986 fragment punctuation and existing ASCII escapes, with uppercase escape hex. This deliberately includes escaped letters, hyphens and tildes: fragment syntax can be interpreted before percent-decoding. For example, #:%7E:text=hello stays encoded instead of becoming an active text directive. Unicode text uses UTF-8 and NFC normalization as usual. Hashbang fragments are no longer converted into _escaped_fragment_ query parameters.

Empty query and fragment delimiters are removed by default. Set keep_empty_components=True in either Python API, or use --keep-empty-components in the CLI, to retain them when your application needs to distinguish an empty component from an absent one. For example, https://example.com/?# stays unchanged with this option.

The option applies to query and fragment delimiters. Empty parameter values such as flag= are preserved with either setting. Empty query fields are still removed: ?a&&b& becomes ?a&b, while ?&& disappears by default or becomes ? with keep_empty_components=True.

Normalization applies consistent spelling rules; it does not establish whether two URLs serve the same content. Applications using normalized URLs as persistent cache keys or identifiers should review the changelog when upgrading.

Python API

Both functions accept a URL as str or None and return str or None. An empty string and None are returned unchanged. Malformed input can raise ValueError, including UnicodeError for invalid hostnames.

Return types follow the input type: a str input gives a str result, None gives None, and a value typed as str | None gives str | None. Type checkers can therefore use the string result directly:

from url_normalize import url_normalize

result: str = url_normalize("HTTP://Example.COM:80/a/../b")
print(result.startswith("http://"))
# Output: True

The same contract applies to url_humanize() and every keyword option. An empty string result is still a string, including when strip_fragment=True removes a fragment-only reference.

The following options are keyword-only and apply to both functions:

Option Default Purpose
default_scheme "https" Scheme to add to bare hosts and network-path references such as //example.com/. Accepts uppercase, surrounding whitespace and one optional trailing colon.
default_domain None Domain to add to an absolute path such as /images/logo.png.
filter_params False Enable built-in domain filtering when no custom allowlist is supplied.
param_allowlist None Keep one name, a sequence or set of names, or names from a domain mapping. Supplying a rule enables filtering, including an empty collection.
param_denylist None Remove names using the same rule forms as the allowlist, without an extra flag. Denied names override allowed names; an empty collection removes nothing.
remove_tracking_params False Remove a built-in set of exact, case-sensitive tracking parameter names in addition to any custom filtering.
sort_query False Sort by normalized, percent-encoded query key after filtering. Preserve the order of repeated values and each field's value separator.
collapse_slashes True Collapse repeated literal path slashes after resolving dot segments, for the schemes described below. Set False to preserve repeated slashes.
remove_trailing_slash False Remove the entire trailing run of literal path slashes, keeping / for the root. Uses the same scheme scope as collapse_slashes.
keep_empty_components False Keep query (?) and fragment (#) delimiters when their content is empty, including a query emptied by parameter filtering.
strip_fragment False Remove the entire fragment and its # delimiter, overriding keep_empty_components for the fragment only.

Unicode URL characters always use UTF-8 percent encoding.

Hostname boundaries

Leading dots, empty interior labels, repeated trailing dots, and dot-only hosts are preserved in both APIs. A single trailing dot is removed when other host characters precede it. ASCII dots and their IDNA equivalents (。, ., 。) follow the same rule:

Input Result
https://example.com./ https://example.com/
https://.example.com/ https://.example.com/
https://a..example.com/ https://a..example.com/
https://example.com../ https://example.com../
https://./ https://./
https://../ https://../
https://a。。example.com/ https://a..example.com/

This hostname policy also applies to default_domain and custom param_allowlist domain keys. Entries for .example.com and example.com remain distinct; an unusual host can have its own explicit allowlist entry. The CLI uses the same normalization policy.

Preservation does not establish DNS validity. The generic registered-name syntax in RFC 3986 section 3.2.2 is broader than DNS hostname syntax. Applications that require DNS-valid hostnames should validate them separately. Empty hosts in local file URLs and path references remain supported.

Literal control characters, bidi controls and Unicode line/paragraph separators remaining in a parsed host, including an IPv6 zone identifier, raise UnicodeError (a ValueError subclass). The CLI exits with status 1 and a diagnostic that does not repeat the hostname. Existing input cleanup still removes tabs, carriage returns and newlines before parsing.

IPv6 addresses

Equivalent bracketed IPv6 addresses use a consistent spelling: lowercase hexadecimal, no leading zeroes in groups, and :: for the longest run of at least two zero groups (the first run wins a tie). These rules follow RFC 5952 section 4.

Input Result
https://[2001:0DB8:0:0:0:0:0:1]/ https://[2001:db8::1]/
https://[FE80:0:0:0:0:0:0:ABCD%25EthA]/ https://[fe80::abcd%25EthA]/
https://[::FFFF:C000:280]/ https://[::ffff:192.0.2.128]/

Zone identifiers, including their case and percent-encoding spelling, are preserved. IPv4-mapped IPv6 addresses use ::ffff: followed by dotted decimal; other IPv6 addresses use hexadecimal groups. IPvFuture literals retain their existing case normalization. These rules apply to both APIs, the CLI, default_domain, and allowlist keys. Expanded and compressed allowlist keys now match the same address; when entries conflict, the canonical key takes precedence.

Port normalization

ASCII decimal ports lose leading zeroes, and scheme defaults are removed: http://example.com:0080/ becomes http://example.com/, while https://example.com:0080/ becomes https://example.com:80/. An empty port and its colon are removed too. Numeric normalization uses string operations, including for heavily padded numbers.

Nonempty parsed ports must contain only ASCII digits (0–9), following RFC 3986 section 3.2.3. Other port text raises ValueError: Port must contain only ASCII digits. For example, http://example.com:abc/, http://example.com:80/, http://example.com:²/, and http://example.com:-1/ are rejected. Percent-encoded digits are rejected too; Unicode digits are not converted to ASCII port numbers.

The check applies to ports in URL authorities, including custom schemes and default_domain. Colons elsewhere, such as mailto:user:abc@example.com and ./example.com:abc/path, retain their existing behavior. Both Python APIs use the same rule. The CLI exits with status 1, writes a diagnostic to stderr and prints no URL when port syntax is invalid.

Normalization does not check the transport's port range: https://example.com:065536/ becomes https://example.com:65536/. Range validation remains the caller's responsibility.

Path slashes and dot segments

Repeated path slashes collapse by default, including trailing runs. Use collapse_slashes=False to preserve empty path segments when they matter to your application:

from url_normalize import url_normalize

url = "https://example.com/a//b///"
print(url_normalize(url))
# Output: https://example.com/a/b/

print(url_normalize(url, collapse_slashes=False))
# Output: https://example.com/a//b///

print(url_normalize(url, remove_trailing_slash=True))
# Output: https://example.com/a/b

print(url_normalize(url, collapse_slashes=False, remove_trailing_slash=True))
# Output: https://example.com/a//b

print(url_normalize("https://example.com/a//../b"))
# Output: https://example.com/a/b

Path operations run in order: resolve dot segments, collapse slash runs unless disabled, then optionally remove trailing slashes. In /a//../b, .. removes the empty segment, leaving /a/b with either collapse setting. Encoded separators such as %2F remain encoded. Query and fragment handling follows the usual normalization rules.

Collapsing retains a single trailing slash. Enable remove_trailing_slash to remove it as well; /a//b/// then becomes /a/b. With collapse_slashes=False, trailing-slash removal preserves internal slash runs, producing /a//b. The root remains /, including when the input path consists entirely of slashes.

These path operations apply to HTTP, HTTPS, WS, WSS, FTP, file URLs and path references. Document-relative paths resolve internal dots while retaining leading ../ segments that need a base URL. Other schemes keep their existing path handling. When collapsing is disabled, a root-relative path that would start with // retains a /. prefix to avoid being interpreted as a hostname; for example, /a/..//b becomes /.//b.

Fragment removal

Set strip_fragment=True in either Python API, or use --strip-fragment in the CLI, to build URLs without document positions. For example, a crawler can use the same fetch key for links to different sections of one article. Fragments are handled separately from retrieval under RFC 3986 section 3.5.

from url_normalize import url_normalize

url = "https://example.com/article?id=1#intro"
print(url_normalize(url))
# Output: https://example.com/article?id=1#intro

print(url_normalize(url, strip_fragment=True))
# Output: https://example.com/article?id=1

print(url_normalize("https://example.com/article?id=1#details", strip_fragment=True))
# Output: https://example.com/article?id=1

Removal is disabled by default. Enabling it removes anchors, fragment routes such as #!/cart, and text highlights such as #:~:text=hello. Keep the default when those destinations matter to your application. The option removes the whole fragment; it does not move fragment content into the query.

Percent-encoded hashes remain part of their original components: https://example.com/a%23b?x=%23keep#section becomes https://example.com/a%23b?x=%23keep. Existing query parameters, including _escaped_fragment_, follow the ordinary query rules. Path cleanup, filtering and sorting remain independently configurable.

strip_fragment=True removes even an empty # when keep_empty_components=True. Empty query preservation still applies: with both options, https://example.com/page?#section becomes https://example.com/page?. A fragment-only reference such as #section becomes an empty reference (""); ?#section becomes "?" when empty components are kept, or "" otherwise. Relative paths keep their existing relative form.

Query escapes

Percent-encoded reserved characters stay encoded in query keys and values, with uppercase hexadecimal digits. A literal / or ; stays distinct from its escaped form because applications can assign it a delimiter role (RFC 3986 section 2.2). The first = separates a key from its value. Additional equals signs in the value normalize to %3D, so ?q=a=b and ?q=a%3Db have the same spelling. Unreserved escapes such as %7E become ~.

Input url_normalize() url_humanize()
https://example.com/?q=a%2fb https://example.com/?q=a%2Fb https://example.com/?q=a%2Fb
https://example.com/?q=a/b https://example.com/?q=a/b https://example.com/?q=a/b
https://example.com/?q=a%3Bb https://example.com/?q=a%3Bb https://example.com/?q=a%3Bb
https://example.com/?q=%C3%A9%2F%C3%B1 https://example.com/?q=%C3%A9%2F%C3%B1 https://example.com/?q=é%2Fñ
https://example.com/?q=%7Ehello https://example.com/?q=~hello https://example.com/?q=~hello

Parameter order is preserved by default. Duplicates and the distinction between flag and flag= are preserved with either query-sorting setting. Literal +, encoded %2B, and %20 remain distinct in normalized output. Unicode still uses NFC normalization and UTF-8 encoding; these guarantees do not imply byte-for-byte preservation for signed URLs. The CLI uses the same rules in both modes.

Query parameter filtering

Supplying param_allowlist keeps only the allowed parameters; no extra flag is needed. Use filter_params=True without a custom allowlist to select the built-in domain rules, which cover Google, Bing, Baidu, and YouTube; see the domain rules for their parameter names.

None means no custom rule. An empty list or dictionary enables filtering and removes all query parameters. An explicit rule takes precedence even if filter_params=False; to turn filtering off, omit the rule or pass param_allowlist=None with filter_params=False.

Filtering that removes every parameter also removes the query delimiter by default. With keep_empty_components=True, the delimiter remains even when filtering empties the query. For example, filtering https://example.com/?drop=1# with no matching allowed name produces https://example.com/ by default or https://example.com/?# with this option.

Unknown domains have an empty default allowlist, so filtering removes all their query parameters unless you supply a custom rule. A custom allowlist replaces the built-in rules.

from url_normalize import url_normalize

# Use the built-in Google rule.
result = url_normalize(
    "www.google.com/search?q=test&utm_source=test", filter_params=True
)
print(result)
# Output: https://www.google.com/search?q=test

# Keep one parameter name on any domain.
result = url_normalize(
    "example.com/?id=123&i=456&d=789",
    param_allowlist="id",
)
print(result)
# Output: https://example.com/?id=123

# Apply a list of allowed names to any domain.
result = url_normalize(
    "example.com/?page=1&id=123&ref=test",
    param_allowlist=["page", "id"],
)
print(result)
# Output: https://example.com/?page=1&id=123

# Use domain-specific rules, including Unicode parameter names.
result = url_normalize(
    "https://example.com/?r%C3%A9sum%C3%A9=1&drop=2",
    param_allowlist={"example.com": "résumé"},
)
print(result)
# Output: https://example.com/?r%C3%A9sum%C3%A9=1

A string is one complete name: "id" means the same as ["id"], and {"example.com": "id"} means the same as {"example.com": ["id"]}. Strings are not split on commas; use ["page", "id"] for two names, while "page,id" selects the literal name page,id. An empty string selects the empty parameter name (?=value); an empty list allows no names. Domain mappings may mix strings, sequences and sets. The CLI retains its comma-separated --param-allowlist format, enables filtering without -f, and treats -p "" as selecting the empty parameter name.

Both filters accept lists, tuples, sets and frozensets of strings, or mappings from domain strings to those forms (including read-only mappings). Rules are reusable and are not mutated. Their order does not change query order. Generators and other one-shot iterators are not supported; materialize them first, for example with tuple(names).

from url_normalize import url_normalize

allowed = ("id", "page")
blocked = frozenset({"ref"})
url = "https://example.com/?page=2&id=123&ref=email"

print(url_normalize(url, param_allowlist=allowed))
# Output: https://example.com/?page=2&id=123

print(url_normalize(url, param_denylist=blocked))
# Output: https://example.com/?page=2&id=123

Allowed names use the same normalization as query keys, including Unicode and spaces. Reserved escapes remain distinct: param_allowlist=["a/b"] matches a/b=1, while param_allowlist=["a%2fb"] matches a%2Fb=1. Domain rules match normalized hostnames and ignore a leading www. and the port. Rules do not automatically apply to other subdomains.

To remove selected names while keeping all other parameters, supply param_denylist. It accepts the same single-name, sequence, set and domain-map forms and uses the same case-sensitive name matching. It does not enable the built-in allowlist. An empty denylist or a domain map without a matching rule removes nothing. Names are literal: "utm_*" is not a wildcard.

from url_normalize import url_normalize

result = url_normalize(
    "https://example.com/products?id=123&utm_source=newsletter&variant=blue",
    param_denylist="utm_source",
)
print(result)
# Output: https://example.com/products?id=123&variant=blue

# Narrow the built-in Google rule without copying its allowed names.
result = url_normalize(
    "https://google.com/search?q=hello&ie=UTF-8&utm_source=newsletter",
    filter_params=True,
    param_denylist="ie",
)
print(result)
# Output: https://google.com/search?q=hello

When both rules apply, a parameter must be allowed and must not be denied. For example, param_allowlist=["id", "tag"] with param_denylist="tag" retains only id. Removal preserves the order, duplicates and value separators of the remaining parameters; ordinary URL normalization and any requested query sorting still apply. Removing the last parameter follows keep_empty_components, just like allowlist filtering. These rules also apply to url_humanize().

Tracking parameter removal

Set remove_tracking_params=True to remove known attribution parameters on any domain while retaining other query names. This works in both Python APIs and does not enable the built-in domain allowlist. It is disabled by default because removing attribution changes campaign measurement.

from url_normalize import url_normalize

url = "https://example.com/product?id=123&ttclid=abc&srsltid=xyz"
print(url_normalize(url))
# Output: https://example.com/product?id=123&ttclid=abc&srsltid=xyz

print(url_normalize(url, remove_tracking_params=True))
# Output: https://example.com/product?id=123

print(
    url_normalize(
        "https://example.com/search?ref=main&source=books&utm_source=email",
        remove_tracking_params=True,
    )
)
# Output: https://example.com/search?ref=main&source=books

The preset removes these exact names, based on provider documentation:

Family Parameter names Sources
UTM utm_id, utm_source, utm_medium, utm_campaign, utm_source_platform, utm_term, utm_content, utm_creative_format, utm_marketing_tactic, utm_referrer Google Analytics, Analytics schema, Yandex
Google attribution gclid, dclid, gbraid, wbraid, gad_source, gad_campaignid, srsltid Analytics identifiers, Google Ads, Merchant Center
Other click identifiers fbclid, msclkid, ttclid, twclid, yclid Meta, Microsoft, TikTok, X, Yandex
Mailchimp mc_cid, mc_eid Mailchimp
Matomo / Piwik mtm_campaign, mtm_keyword, mtm_source, mtm_medium, mtm_content, mtm_cid, mtm_group, mtm_placement, mtm_kwd, matomo_campaign, pk_campaign, piwik_campaign, pk_kwd, piwik_kwd, pk_keyword, matomo_kwd Campaign parameters, Default parameters, Aliases

Matching uses normalized query keys: %75tm_source matches utm_source. Names remain case-sensitive, so UTM_SOURCE is retained. There are no prefix or wildcard rules: utm_custom and gclid_extra remain. Generic names such as ref, source, keyword, and session_id are retained because they can select content or identify a session. The preset only examines query keys; it does not remove matching words from paths, parameter values or fragment routes.

The preset adds removals to param_denylist and takes precedence over param_allowlist. An empty custom denylist does not disable it. To retain a preset name, leave this option off and supply your own param_denylist. Retained parameters keep their order unless sort_query=True is requested. Duplicates and value separators survive either setting, and an emptied query follows keep_empty_components.

Query sorting

Use sort_query=True in either Python API, or --sort-query in the CLI, when your application treats the order of different query keys as irrelevant. This can produce common cache or deduplication keys. Sorting is disabled by default because some applications depend on query order.

from url_normalize import url_normalize

url = "https://example.com/search?tag=red&page=2&tag=blue"
print(url_normalize(url))
# Output: https://example.com/search?tag=red&page=2&tag=blue

print(url_normalize(url, sort_query=True))
# Output: https://example.com/search?page=2&tag=red&tag=blue

Sorting runs after normalization and filtering. It compares the exact normalized, percent-encoded keys in ascending, case-sensitive order. An empty key sorts first; percent escapes sort before letters; uppercase ASCII letters sort before lowercase ones. For example, https://example.com/?z=1&é=2&a=3 becomes https://example.com/?%C3%A9=2&a=3&z=1. url_humanize() displays the same order as https://example.com/?é=2&a=3&z=1.

Equal normalized keys retain their original order, including when normalization makes %61 and a equal. Values are never used as a sorting key. Thus tag=red&tag=blue remains distinct from tag=blue&tag=red. Sorting preserves duplicates, each occurrence of flag versus flag=, and the usual escape distinctions such as +, %2B, and %20. It applies only to query fields; paths and fragment routes follow their existing policies. Allowlist, denylist and tracker removal can all be combined with sorting, and empty queries still follow keep_empty_components.

Default domain and scheme

Use default_domain for paths starting with a single /. URLs starting with // already specify their host. This option does not resolve relative paths such as ../images/logo.png against a page URL.

from url_normalize import url_normalize

result = url_normalize(
    "/images/logo.png", default_domain="example.com", default_scheme="http"
)
print(result)
# Output: http://example.com/images/logo.png

default_scheme accepts a scheme name such as https, HTTP, or git+ssh, optionally followed by one colon. Surrounding whitespace is removed and the name is lowercased. Names follow RFC 3986 section 3.1.

from url_normalize import url_normalize

print(url_normalize("example.com/path", default_scheme=" HTTPS: "))
# Output: https://example.com/path

Malformed values such as https://, https::, or an empty scheme raise ValueError; non-string values raise TypeError. Validation applies to every URL that remains nonempty after input cleanup, including URLs that already have a scheme and relative references. A whitespace-only URL returns "" and None returns None before scheme validation. The CLI applies the same rules and reports configuration errors with exit status 1.

Relative references

Explicit references starting with ./, ../, ?, or # stay relative in both APIs and the CLI. The standalone paths . and .., including query or fragment suffixes, also stay relative. default_domain and default_scheme do not turn these references into absolute URLs.

from url_normalize import url_normalize

print(url_normalize("../images/a.png"))
# Output: ../images/a.png

print(url_normalize("./a//../b"))
# Output: ./a/b

print(url_normalize("../a//../b///"))
# Output: ../a/b/

print(url_normalize("../a//../b///", collapse_slashes=False))
# Output: ../a/b///

print(url_normalize("../a//../b///", remove_trailing_slash=True))
# Output: ../a/b

print(url_normalize("?q=é#section"))
# Output: ?q=%C3%A9#section

Document-relative paths use the same operation order: resolve internal dots, collapse slashes by default, then optionally remove trailing slashes. Unresolved parents stay relative: ../a/../../b becomes ../../b. The ./ prefix keeps an ordinary path from being interpreted as a hostname or scheme; ./a/../b becomes ./b, and ./a/../this:that becomes ./this:that. With collapsing disabled, ./a/..//b becomes .//b, preserving its empty segment without changing it into a root-relative path.

Paths that reduce to the current directory retain a nonempty reference: ./a/.. becomes ./, or . with remove_trailing_slash=True. Similarly, ../a/.. becomes ../, or .. with trailing-slash removal. Percent encoding, Unicode handling, query filtering, and empty-component policy still apply. Use keep_empty_components=True when empty ? or # delimiters matter to later resolution.

To obtain an absolute URL, resolve the reference against a base URL. For example:

from urllib.parse import urljoin

from url_normalize import url_normalize

absolute = urljoin("https://example.com/docs/page.html", "../images/a.png")
print(url_normalize(absolute))
# Output: https://example.com/images/a.png

Bare inputs retain host inference: site/page becomes https://site/page. Use ./site/page to explicitly identify a document-relative path. Network-path references such as //example.com/page receive the selected default scheme.

Human-readable URLs

url_humanize() normalizes its input, then decodes components only when the display URL normalizes back to the same result. Some escapes therefore remain encoded. In fragments, ASCII escapes remain encoded even when a round trip would allow decoding; safe Unicode can still become readable. For example, #:~:text=r%C3%A9sum%C3%A9%2Dnotes becomes #:~:text=résumé%2Dnotes, preserving the encoded hyphen as text rather than directive syntax. Fragment spaces remain %20.

from url_normalize import url_humanize, url_normalize

normalized = "https://xn--bcher-kva.de/r%C3%A9sum%C3%A9"
readable = url_humanize(normalized)
print(readable)
# Output: https://bücher.de/résumé

assert url_normalize(readable) == normalized

Credentials retain reserved escapes, including %40 for an @ inside a username or password. This leaves a single literal @ separating credentials from the host. Safe Unicode around those escapes can still become readable:

from url_normalize import url_humanize

print(url_humanize("https://alice%40example.net:secret@example.com/resource"))
# Output: https://alice%40example.net:secret@example.com/resource

print(url_humanize("https://us%C3%A9r%40example.net:p%C3%A4ss@example.com/"))
# Output: https://usér%40example.net:päss@example.com/

Existing encoded colons, slashes and brackets in credentials remain encoded too. The CLI's --humanize mode follows the same policy.

Command-line usage

The CLI accepts one URL per invocation and prints the result to standard output. Quote URLs containing shell characters such as &.

# Keep selected query parameters.
url-normalize -p page,id "example.com/?page=1&id=123&ref=test"
# Output: https://example.com/?page=1&id=123

# Remove selected parameters while retaining all other names.
url-normalize --param-denylist utm_source,ref "example.com/?id=123&utm_source=newsletter&ref=email"
# Output: https://example.com/?id=123

# Remove the built-in tracker names while retaining other parameters.
url-normalize --remove-tracking-params "https://example.com/product?id=123&ttclid=abc&srsltid=xyz"
# Output: https://example.com/product?id=123

# Sort query keys while preserving the order of repeated values.
url-normalize --sort-query "https://example.com/search?tag=red&page=2&tag=blue"
# Output: https://example.com/search?page=2&tag=red&tag=blue

# Supply a domain and scheme for an absolute path.
url-normalize -d example.com -s http "/images/logo.png"
# Output: http://example.com/images/logo.png

# Collapse repeated slashes in the path (the default).
url-normalize "https://example.com/a//b///"
# Output: https://example.com/a/b/

# Preserve repeated slashes in the path.
url-normalize --no-collapse-slashes "https://example.com/a//b///"
# Output: https://example.com/a//b///

# Remove trailing slashes while preserving internal empty segments.
url-normalize --no-collapse-slashes --remove-trailing-slash "https://example.com/a//b///"
# Output: https://example.com/a//b

# Retain explicitly empty query and fragment components.
url-normalize --keep-empty-components "https://example.com/?#"
# Output: https://example.com/?#

# Remove an anchor, route or text highlight from the URL.
url-normalize --strip-fragment "https://example.com/article?id=1#intro"
# Output: https://example.com/article?id=1

# Display a readable URL.
url-normalize -H "https://xn--bcher-kva.de/r%C3%A9sum%C3%A9"
# Output: https://bücher.de/résumé

Use url-normalize --help for all options and url-normalize --version for the installed version. Normalization errors are written to standard error with exit status 1; invalid command-line arguments use exit status 2.

To run without a persistent tool installation:

uvx url-normalize "example.com"
# Output: https://example.com/

Contributing

After cloning the repository, install the development dependencies and run the checks with uv:

uv sync --locked --extra dev
uv run --locked --extra dev python -m pytest
uv run --locked --extra dev pre-commit run --all-files

The test suite requires 100% statement and branch coverage. Branch coverage checks that tests take both outcomes of measured decisions, such as enabling and disabling query sorting. It does not establish that every input or combination works, so please include exact-output regression tests with bug fixes. CI tests Python 3.11 through 3.15 on Linux, macOS, and Windows.

The twice-monthly scheduled runs also test the newest compatible runtime dependencies on Linux with Python 3.15. This checks dependency releases beyond uv.lock, while regular matrix tests use the locked versions.

Pytest also runs every python code block in this README independently and checks its printed output against the # Output: comments. Include imports in each block and keep those comments accurate. Shell commands and prose tables are not executed by this check.

Type checking uses ty for the package and the public API consumer examples in tests/typing, targeting the minimum Python version from requires-python and all platforms. Those examples use typing.assert_type to check string, None, optional and narrowed inputs through the package exports; they are static checks, separate from pytest. The pre-commit hook uses the version and dependencies in uv.lock. To run type checking alone, use uv run --locked --extra dev ty check.

Alongside exact-output tests, Hypothesis generates 200 HTTP(S) URL and option combinations per property to check normalization idempotence, humanization round trips, and empty-component policy. Another 200 explicit relative references check stable output, slash options, and the absence of invented schemes or authorities. Generation is deterministic for a given Python version, Hypothesis version, and test definition. Add any minimized failures as explicit regression tests.

Changelog and license

See CHANGELOG.md for release history and upgrade notes. Distributed under the MIT License.

Inspired by Sam Ruby's urlnorm.py.

Metadata

Release files for url-normalize 4.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for url-normalize 4.0.0
File Size Uploaded Provenance Actions
url_normalize-4.0.0.tar.gz 90.1 kB Without provenance Details for url_normalize-4.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for url-normalize 4.0.0
File Interpreter ABI Platform Actions
url_normalize-4.0.0-py3-none-any.whl Python 3 none any Details for url_normalize-4.0.0-py3-none-any.whl

Total release size: 125.4 kB

Release files / url_normalize-4.0.0.tar.gz

Download URL url_normalize-4.0.0.tar.gz
Size 90.1 kB
Tags Source
SHA-256 checksum
How to use checksums
52ad9004d6278ea4b581f1cf2f4d016c4873bf3d81fd749b7472ac0206355d05
BLAKE2b-256 checksum
How to use checksums
6c1b0651cd4dc1a2c0b81151a7f0a06ff171e324a2bce47b092121d75fbeb27c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / url_normalize-4.0.0-py3-none-any.whl

Download URL url_normalize-4.0.0-py3-none-any.whl
Size 35.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
dea616fc9c3d4c8bcd46fcaa783bb4e402b4f272789650919e473c5a86a7b3eb
BLAKE2b-256 checksum
How to use checksums
9069b99d4032fd4a2984e7ec0756ef84c031f3cb7b909c2d3b592aa7b0bb7d57
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for url-normalize 4.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page