Check if a URL is reachable
Project description
Reachable checks if a URL exists and is reachable.
Features
- Use
HEAD
request instead ofGET
to save some bandwidth - Follow redirects
- Handle local redirects (without full URL in
location
header) - Record all the URLs of the redirection chain
- Check if redirected URL match the TLD of source URL
- Detect Cloudflare protection
- Avoid basic bot detectors
- Use randome Chrome user agent
- Wait between consecutive requests to the same host
- Include
Host
header
- Use of HTTP/2
Installation
You can install it with pip :
pip install reachable
Or clone this repository and simply run :
cd reachable/
pip install -e .
Usage
Simple URL
from reachable import is_reachable
result = is_reachable("https://google.com")
The output will look like this:
{
"original_url": "https://google.com",
"final_url": "https://www.google.com/",
"response": null,
"status_code": 200,
"success": true,
"error_name": null,
"cloudflare_protection": false,
"redirect": {
"chain": ["https://www.google.com/"],
"final_url": "https://www.google.com/",
"tld_match": true
}
}
Multiple URLs
from reachable import is_reachable
result = is_reachable(["https://google.com", "http://bing.com"])
The output will look like this:
[
{
"original_url": "https://google.com",
"final_url": "https://www.google.com/",
"response": null,
"status_code": 200,
"success": true,
"error_name": null,
"cloudflare_protection": false,
"redirect": {
"chain": ["https://www.google.com/"],
"final_url": "https://www.google.com/",
"tld_match": true
}
},
{
"original_url": "http://bing.com",
"final_url": "https://www.bing.com/?toWww=1&redig=16A78C94",
"response": null,
"status_code": 200,
"success": true,
"error_name": null,
"cloudflare_protection": false,
"redirect": {
"chain": ["https://www.bing.com:443/?toWww=1&redig=16A78C94"],
"final_url": "https://www.bing.com/?toWww=1&redig=16A78C94",
"tld_match": true
}
}
]
Async
import asyncio
from reachable import is_reachable_async
result = asyncio.run(is_reachable_async("https://google.com"))
or
import asyncio
from reachable import is_reachable_async
urls = ["https://google.com", "https://bing.com"]
try:
loop = asyncio.get_running_loop()
except RuntimeError:
# No loop already exists so we crete one
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
result = loop.run_until_complete(asyncio.gather(*[is_reachable_async(url) for url in urls]))
finally:
loop.close()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
reachable-0.4.0.tar.gz
(8.5 kB
view hashes)
Built Distribution
Close
Hashes for reachable-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3706f351258990cbd47cb82e8247e341fbeee2de13147cb0dd9eb1bb06952202 |
|
MD5 | 065d21c69a0089c938cf3d014a449454 |
|
BLAKE2b-256 | 0448954223685625ff786c38d0a7c067f558a53de5f18ef666c80053347103c6 |