Check if a URL is reachable
Project description
Reachable checks if a URL exists and is reachable.
Features
- Use
HEAD
request instead ofGET
to save some bandwidth - Follow redirects
- Handle local redirects (without full URL in
location
header) - Record all the URLs of the redirection chain
- Check if redirected URL match the TLD of source URL
- Detect Cloudflare protection
- Avoid basic bot detectors
- Use randome Chrome user agent
- Wait between consecutive requests to the same host
- Include
Host
header
- Use of HTTP/2
Installation
You can install it with pip :
pip install reachable
Or clone this repository and simply run :
cd reachable/
pip install -e .
Usage
Simple URL
from reachable import is_reachable
result = is_reachable("https://google.com")
The output will look like this:
{
"original_url": "https://google.com",
"final_url": "https://www.google.com/",
"response": null,
"status_code": 200,
"success": true,
"error_name": null,
"cloudflare_protection": false,
"redirect": {
"chain": ["https://www.google.com/"],
"final_url": "https://www.google.com/",
"tld_match": true
}
}
Multiple URLs
from reachable import is_reachable
result = is_reachable(["https://google.com", "http://bing.com"])
The output will look like this:
[
{
"original_url": "https://google.com",
"final_url": "https://www.google.com/",
"response": null,
"status_code": 200,
"success": true,
"error_name": null,
"cloudflare_protection": false,
"redirect": {
"chain": ["https://www.google.com/"],
"final_url": "https://www.google.com/",
"tld_match": true
}
},
{
"original_url": "http://bing.com",
"final_url": "https://www.bing.com/?toWww=1&redig=16A78C94",
"response": null,
"status_code": 200,
"success": true,
"error_name": null,
"cloudflare_protection": false,
"redirect": {
"chain": ["https://www.bing.com:443/?toWww=1&redig=16A78C94"],
"final_url": "https://www.bing.com/?toWww=1&redig=16A78C94",
"tld_match": true
}
}
]
Async
import asyncio
from reachable import is_reachable_async
result = asyncio.run(is_reachable_async("https://google.com"))
or
import asyncio
from reachable import is_reachable_async
urls = ["https://google.com", "https://bing.com"]
try:
loop = asyncio.get_running_loop()
except RuntimeError:
# No loop already exists so we crete one
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
result = loop.run_until_complete(asyncio.gather(*[is_reachable_async(url) for url in urls]))
finally:
loop.close()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
reachable-0.3.1.tar.gz
(7.2 kB
view hashes)
Built Distribution
Close
Hashes for reachable-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a662caf91af3d4c8a903e873bb142d2f67cb9cff34e1dfde93354ee0a07e67f |
|
MD5 | 86900aaed1ed29c9e68dbf55bf6a4448 |
|
BLAKE2b-256 | 31433ce2bcfbb46378bcc776224108689fcbeccd814d1b8acd93f3a799f7ae7c |