urlfinderlib

This is a Python (3.10+) library for finding URLs in documents and checking their validity.

Supported Documents

Extracts URLs from the following types of documents:

Binary files (finds URLs within strings)
CSV files
HTML files
iCalendar/vCalendar files
PDF files
Text files (ASCII or UTF-8)
XML files

Every extracted URL is validated such that it contains a domain with a valid TLD (or a valid IP address) and does not contain any invalid characters.

URL Permutations

This was originally written to accommodate finding both valid and obfuscated or slightly malformed URLs used by malicious actors and using them as indicators of compromise (IOCs). As such, the extracted URLs will also include the following permutations:

URL with any Unicode characters in its domain
URL with any Unicode characters converted to its IDNA equivalent

For both domain variations, the following permutations are also returned:

URL with its path %-encoded
URL with its path %-decoded
URL with encoded HTML entities in its path
URL with decoded HTML entities in its path
URL with its path %-decoded and HTML entities decoded

Child URLs

This library also attempts to extract or decode child URLs found in the paths of URLs. The following formats are supported:

Barracuda protected URLs
Base64-encoded URLs found within the URL's path
Google redirect URLs
Mandrill/Mailchimp redirect URLs
Outlook Safe Links URLs
Proofpoint protected URLs
URLs found in the URL's path query parameters

Basic usage

from urlfinderlib import find_urls

with open('/path/to/file', 'rb') as f:
    print(find_urls(f.read())

base_url Parameter

If you are trying to find URLs inside of an HTML file, the paths in the URLs are often relative to their location on the server hosting the HTML. You can use the base_url parameter in this case to extract these "relative" URLs.

from urlfinderlib import find_urls

with open('/path/to/file', 'rb') as f:
    print(find_urls(f.read(), base_url='http://example.com')

Release files for urlfinderlib 0.25.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for urlfinderlib 0.25.5
File	Size	Uploaded
urlfinderlib-0.25.5.tar.gz	16.8 kB	Sep 24, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for urlfinderlib 0.25.5
File	Interpreter	ABI	Platform	Reset
urlfinderlib-0.25.5-py3-none-any.whl 22.9 kB Sep 24, 2026	Python 3	none	any	Details

Total release size: 39.7 kB

Release files / urlfinderlib-0.25.5.tar.gz

Download URL	urlfinderlib-0.25.5.tar.gz
Size	16.8 kB
Tags	Source
SHA-256 checksum How to use checksums	`934f59cc7e67ffdbf5ca04ffd29c70013e4bae2794a37ca1880b1fdcd61689b5`
BLAKE2b-256 checksum How to use checksums	`fd685188b7d425473954eed76e45f876773363df68a731768e359801df536f60`
Upload date	Sep 24, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ACE-Collective/urlfinderlib
Publishing commit: github.com/ACE-Collective/urlfinderlib/tree/e7488a757221bbe052408e03a6c1aa1d1136d5bd

Workflow

Publishing configuration: github.com/ACE-Collective/urlfinderlib/blob/e7488a757221bbe052408e03a6c1aa1d1136d5bd/.github/workflows/pypi.yml
Publishing logs: github.com/ACE-Collective/urlfinderlib/actions/runs/36036779787/attempts/1

Artifact

Subject: urlfinderlib-0.25.5.tar.gz
SHA-256 checksum: 934f59cc7e67ffdbf5ca04ffd29c70013e4bae2794a37ca1880b1fdcd61689b5 Verifying attestations

Release files / urlfinderlib-0.25.5-py3-none-any.whl

Download URL	urlfinderlib-0.25.5-py3-none-any.whl
Size	22.9 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`9a286c02317c00a3441ce6bff01a5d5a45300df2b3e2b1146cb9b200c2afd2bc`
BLAKE2b-256 checksum How to use checksums	`cd6d616c5d2d0674fcdfc1ac5d1787dad322b71cfc73c79769a750df8a8813c1`
Upload date	Sep 24, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ACE-Collective/urlfinderlib
Publishing commit: github.com/ACE-Collective/urlfinderlib/tree/e7488a757221bbe052408e03a6c1aa1d1136d5bd

Workflow

Publishing configuration: github.com/ACE-Collective/urlfinderlib/blob/e7488a757221bbe052408e03a6c1aa1d1136d5bd/.github/workflows/pypi.yml
Publishing logs: github.com/ACE-Collective/urlfinderlib/actions/runs/36036779787/attempts/1

Artifact

Subject: urlfinderlib-0.25.5-py3-none-any.whl
SHA-256 checksum: 9a286c02317c00a3441ce6bff01a5d5a45300df2b3e2b1146cb9b200c2afd2bc Verifying attestations