Fetch Known Urls
Project description
Unja
Fetch Known Urls
What's Unja?
Unja is a fast & light tool for fetching known URLs from Wayback Machine, Common Crawl, Virus Total & AlienVault's Otx it uses a separate thread for each provider to optimize its speed and use Wayback resumption key to divide scan into multiple parts to handle a large scan & it uses direct filters on API to get only filtered data from API to do less work on your system.
Why Unja?
- Supports
Wayback/Common-Crawl/Virus-Total/Otx
- Automatically handles rate limits and timeouts
- Export results: text or detailed output with status,mime,length in JSON
- MultiThreading: separate thread for each provider to fetch data simultaneously
- Filters: apply filters dirtly on provider to avoid unnecessary data
Installing Unja
You can install Unja
with pip as following:
pip3 install unja
or, by downloading this repository and running
python3 setup.py install
How to use Unja?
A detailed usage guide is available on Usage section of the Wiki.
Optionally, you can use the --help
argument to explore Unja on your own.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
unja-0.0.1.tar.gz
(5.3 kB
view hashes)
Built Distribution
unja-0.0.1-py3-none-any.whl
(18.0 kB
view hashes)