Run a dork on different search sites
Project description
About - v0.1
This is a tool used to run a dork on different search sites. The available sources are currently: DuckDuckGo, Bing, Startpage, Yahoo, Google
WARNING: If you use this tool a lot, then I guess there is the potential to get blocked on these source sites, so use sensibly. Using a VPN will help.
Installation
xnldorker
supports Python 3.
Install xnldorker
in default (global) python environment.
pip install xnldorker
OR
pip install git+https://github.com/xnl-h4ck3r/xnldorker.git -v
You can upgrade with
pip install --upgrade xnldorker
pipx
Quick setup in isolated python environment using pipx
pipx install git+https://github.com/xnl-h4ck3r/xnldorker.git
Usage
Argument | Long Argument | Description |
---|---|---|
-i | --input | A dork to use on the search sources. |
-o | --output | The output file that will contain the results (default: output.txt). If piped to another program, output will be written to STDOUT instead. |
-ow | --output-overwrite | If the output file already exists, it will be overwritten instead of being appended to. |
-os | --output-sources | Show the source of each endpoint in the output. Each endpoint will be prefixed, e.g. [ Bing ] https://example.com . |
-s | --sources | Specific sources to use when searching (e.g. -s duckduckgo,bing ). Use -ls to display all available sources. |
-es | --exclude-sources | Specific sources to exclude searching (-s google,startpage ). Use -ls to display all available sources. |
-cs | --concurrent-sources | The number of sources to search at the same time (default: 2 ). Passing 0 will run ALL specified sources at the same time (this could be very resource intensive and negatively affect results). |
-ls | --list-sources | List all available sources. |
-t | --timeout | How many seconds to wait for the source to respond (default: 30 seconds) |
-sb | --show-browser | View the browser instead of using a headless browser. This has an advantage because if there is a known anti-bot mechanism, then it will pause for a set time (determined by -abt ) so you can manually resolve it before xnldorker continues. |
-abt | --antibot-timeout | How many seconds to wait when the -sb option was used and a known anti-bot mechanism is encountered (default: 30). This is the time you have to manually respond to the anti-bot mechanism before it tries to continue. |
--debug | Save page contents on error. | |
-nb | --no-banner | Hides the tool banner (it is hidden by default if you pipe input to 'xnldorker') output. |
--version | Show current version number. | |
-v | --verbose | Verbose output |
-vv | --vverbose | Increased verbose output |
Examples
Basic use
urless -i target_urls.txt -v
or
echo "site:redbull.com" | xnldorker -v
Capture output
xnldorker -i "site:redbull.com" > redbull_endpoints.txt
or
xnldorker -i "site:redbull.com" -v -o redbull_endpoints.txt
Recommendations
- Using
-v
/--verbose
is always a good idea when you first start using a tool. It will help you understand what the tool is doing and highlight any potential problems too. - If you do mpt need to run silently in the background, I would recommend using the
-sb
/--show-browser
option because you can see whatxnldorker
is doing (and if it seems to be working ok), plus if there is any known ant-bot detection recognised (currently not for all sources) then you will be notified and have the option to resolve this beforexnldorker
continues. - The number of concurrent sources processed defaults to 2. This can be changed with
-cs
/--concurrent-sources
. If you are runningxnldorker
on a low spec VPS, it could be worth setting-cs 1
. The higher the value of-cs
the quicker the tool will be, but may affect the quality and quantity of results. - You may want to run different dorks but write to the same output file. If you use the same output file in
-o
/--output
then any results will be appended to that file automatically (and de-duplicated). But if you want to overwrite it every time, you can use the-ow
/--overwrite-output
argument.
Issues
If you come across any problems at all, or have ideas for improvements, please feel free to raise an issue on Github. If there is a problem, it will be useful if you can provide the exact command you ran and a detailed description of the problem. If possible, run with -v
to reproduce the problem and let me know about any error messages that are given.
Also, if you have problems, it can be useful to use the -sb
/--show-browser
option to see what xnldorker
is doing.
If you use the --debug
option, then xnldorker
will try to write a html file of the contents that it got stuck on, e.g. xnldorker_Google_20240423_133700.html
. It would also be useful to include these in the github issue.
TODO
- Add more sources.
- Identify anti bot mechanism pages on other sources (it's only on a few at the moment) so that
xnldorker
can pause to manually respond if the browser is being viewed. - Maybe allow a file of dorks to be passed as input.
And finally...
Good luck and good hunting! If you really love the tool (or any others), or they helped you find an awesome bounty, consider BUYING ME A COFFEE! ☕ (I could use the caffeine!)
🤘 /XNL-h4ck3r
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.