onionprobe 0.3.3

Test and monitors the status of Tor Onion Services

Onionprobe

Onionprobe is a tool for testing and monitoring the status of Tor Onion Services sites.

It can run a single time or continuously to probe a set of onion services endpoints and paths, optionally exporting to Prometheus.

Requirements

Onionprobe requires the following software:

• Python 3
• Stem Tor Control library
• Prometheus Python client
• PyYAML
• Requests
• PySocks
• Tor daemon

On Debian, they can be installed using

sudo apt install python3 python3-prometheus-client \
                 python3-stem python3-cryptography \
                 python3-yaml python3-requests     \
                 python3-socks tor

Installation

Onionprobe is available on PyPI:

pip install onionprobe

It's also possible to run it directly from the Git repository:

git clone https://gitlab.torproject.org/tpo/onion-services/onionprobe
cd onionprobe

Usage

Simply ask Onionprobe to try an Onion Service site:

onionprobe -e http://2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion

It's possible to supply multiple addresses:

onionprobe -e <onion-address1> <onionaddress2> ...

Onionprobe also accepts a configuration file with a list of .onion endpoints and options. A detailed sample config is provided and can be invoked with:

onionprobe -c configs/tor.yaml

By default, Onionprobe starts it's own Tor daemon instance, so the tor binary must be available in the system.

See the manual page for the complete list of options and available metrics.

Testing

Onionprobe comes with a working test environment with the sample configuration and based on Docker Compose, which can be started using

docker-compose up

Then point your browser to:

• The built-in Prometheus dashboard: https://localhost:9090
• The built-in Onionprobe Prometheus exporter: https://localhost:9091

Compiled configurations

Besides the sample config containing sites listed at https://onion.torproject.org, Onionprobe comes also with other example configs:

1. Real-World Onion Sites .onions at real-world-onion-sites.yaml, generated by the real-world-onion-sites.py script.
2. The SecureDrop API .onions at securedrop.yaml, generated by the securedrop.py script.

You can build your own configuration compiler by using the OnionprobeConfigCompiler class.

Folder structure and files

Relevant folders and files in this repository:

• assets: logos and other stuff.
• configs: miscelaneous configurations.
• containers: container configurations.
• debian: debian packaging.
• docs: documentation.
• packages: python packages codebase.
• scripts: provisioning and other configuration scripts.
• tests: test procedures.
• vendors: other third-party libraries and helpers.
• kvmxfile: please ignore this if you're not a KVMX user.
• .env: should be manually created if you plan to use custom configuration with Docker Composer.
• docker-compose.yml: service container configuration.

Tasks

Check the issue tracker.

Acknowledgements

Thanks to:

• @irl for the idea/specs/tasks.
• @hiro for suggestions.
• @arma and @juga for references.
• @anarcat and @georg for Python and Debian packaging guidance and review.

Alternatives

• OnionScan
• Webmon has support for Onion Services monitoring if used along with Orbot.
• BrassHornCommunications/OnionWatch: A GoLang daemon for notifying Tor Relay and Hidden Service admins of status changes
• systemli/prometheus-onion-service-exporter: Prometheus Exporter for Tor Onion Services
• prometheus/blackbox_exporter: Blackbox prober exporter, which could be configured using proxy_url pointing to a Privoxy instance relaying traffic to tor daemon. See this issue for details.

Known issues

From Stem:

• Python 3.9 warning · Issue #105 · torproject/stem
• noisy log: stem: INFO: Error while receiving a control message (SocketClosed): received exception "peek of closed file" · Issue #112 · torproject/stem · GitHub

References

Related software and libraries with useful routines:

• onbasca
• sbws
• Stem
• txtorcon
• Onionbalance
• hs-health

Relevant issues:

• Write a hidden service hsdir health measurer
• Write tool for onion service health assessment

Research questions:

• When an onion service lookup has failed at the first k HSDirs we tried, what are the chances it will still succeed?
• What's the average number of hsdir fetches before we get the hsdesc?