urlfinderlib

This is a Python (3.10+) library for finding URLs in documents and checking their validity.

Supported Documents

Extracts URLs from the following types of documents:

Binary files (finds URLs within strings)
CSV files
HTML files
iCalendar/vCalendar files
PDF files
Text files (ASCII or UTF-8)
XML files

Every extracted URL is validated such that it contains a domain with a valid TLD (or a valid IP address) and does not contain any invalid characters.

URL Permutations

This was originally written to accommodate finding both valid and obfuscated or slightly malformed URLs used by malicious actors and using them as indicators of compromise (IOCs). As such, the extracted URLs will also include the following permutations:

URL with any Unicode characters in its domain
URL with any Unicode characters converted to its IDNA equivalent

For both domain variations, the following permutations are also returned:

URL with its path %-encoded
URL with its path %-decoded
URL with encoded HTML entities in its path
URL with decoded HTML entities in its path
URL with its path %-decoded and HTML entities decoded

Child URLs

This library also attempts to extract or decode child URLs found in the paths of URLs. The following formats are supported:

Barracuda protected URLs
Base64-encoded URLs found within the URL's path
Google redirect URLs
Mandrill/Mailchimp redirect URLs
Outlook Safe Links URLs
Proofpoint protected URLs
URLs found in the URL's path query parameters

Basic usage

from urlfinderlib import find_urls

with open('/path/to/file', 'rb') as f:
    print(find_urls(f.read())

base_url Parameter

If you are trying to find URLs inside of an HTML file, the paths in the URLs are often relative to their location on the server hosting the HTML. You can use the base_url parameter in this case to extract these "relative" URLs.

from urlfinderlib import find_urls

with open('/path/to/file', 'rb') as f:
    print(find_urls(f.read(), base_url='http://example.com')

Metadata

Release files for urlfinderlib 0.26.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for urlfinderlib 0.26.1
File	Size	Uploaded
urlfinderlib-0.26.1.tar.gz	19.0 kB	Oct 16, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for urlfinderlib 0.26.1
File	Interpreter	ABI	Platform	Reset
urlfinderlib-0.26.1-py3-none-any.whl 24.4 kB Oct 16, 2026	Python 3	none	any	Details

Total release size: 43.4 kB

Release files / urlfinderlib-0.26.1.tar.gz

Download URL	urlfinderlib-0.26.1.tar.gz
Size	19.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`a4ff327dff7a81b37fdc402aed04b7d18b02ac5d1248831cea29204c38493040`
BLAKE2b-256 checksum How to use checksums	`c61bd9a0fb0e057b32a065c41848fd3ca65ae21b4ac4ab6364ca063daab03df6`
Upload date	Oct 16, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ACE-Collective/urlfinderlib
Publishing commit: github.com/ACE-Collective/urlfinderlib/tree/5f8e7799d600f81929f89eb8d84cdbd7384110c3

Workflow

Publishing configuration: github.com/ACE-Collective/urlfinderlib/blob/5f8e7799d600f81929f89eb8d84cdbd7384110c3/.github/workflows/pypi.yml
Publishing logs: github.com/ACE-Collective/urlfinderlib/actions/runs/38897836089/attempts/1

Artifact

Subject: urlfinderlib-0.26.1.tar.gz
SHA-256 checksum: a4ff327dff7a81b37fdc402aed04b7d18b02ac5d1248831cea29204c38493040 Verifying attestations

Release files / urlfinderlib-0.26.1-py3-none-any.whl

Download URL	urlfinderlib-0.26.1-py3-none-any.whl
Size	24.4 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`a0e5271ae7a80d34417621db9d81e473fccdb399b0c997c11f9bf1b3f6d7bef8`
BLAKE2b-256 checksum How to use checksums	`962ab52ffd45830a2a5c2f300338f9958f54a5db2c0decea1425747dd5525688`
Upload date	Oct 16, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ACE-Collective/urlfinderlib
Publishing commit: github.com/ACE-Collective/urlfinderlib/tree/5f8e7799d600f81929f89eb8d84cdbd7384110c3

Workflow

Publishing configuration: github.com/ACE-Collective/urlfinderlib/blob/5f8e7799d600f81929f89eb8d84cdbd7384110c3/.github/workflows/pypi.yml
Publishing logs: github.com/ACE-Collective/urlfinderlib/actions/runs/38897836089/attempts/1

Artifact

Subject: urlfinderlib-0.26.1-py3-none-any.whl
SHA-256 checksum: a0e5271ae7a80d34417621db9d81e473fccdb399b0c997c11f9bf1b3f6d7bef8 Verifying attestations