susbot
The Python package of sus.bot, a robots.txt checker. It tells you which search engines and AI crawlers a site lets in, what is wrong with the file, and which paths it gives away. It ships the Rust engine behind sus.bot as Python bindings and the susbot command.
pip install susbot
Wheels are built for Linux, macOS and Windows on CPython 3.10 and later.
Command line
susbot https://example.com # audit a live site
susbot robots.txt --format markdown --out audit.md
susbot diff old.txt new.txt # what changed, crawler by crawler
python -m susbot --help
It is the same program as cargo install susbot. Exit codes: 0 on success, 1 when a --fail-on or --fail-on-security threshold is met, 2 on errors.
Python
import susbot
text = open("robots.txt", encoding="utf-8").read()
a = susbot.Analysis(text, site_url="https://example.com/")
a.allowed("GPTBot", "/blog/post") # False
a.check_access("Googlebot", "/admin/") # the verdict and the rule that decided it
a.report["summary"] # counts, default policy, AI training, platform
a.issues # lint findings with stable ids
a.security # disallowed paths that look sensitive
a.markdown() # the client audit; also html(), csv_tabs(), tsv()
old = susbot.Analysis(old_text, site_url="https://example.com/")
susbot.diff(old, a) # crawler verdict flips, new sensitive paths, issues
Analysis takes config= for a TOML file merged over susbot.default_config(), and locale= with lang= for a translated report. The report follows the published JSON schema. The package never fetches anything itself; the susbot command does.
Links
- Website: sus.bot
- The same tool elsewhere: susbot on crates.io, @sitefig/susbot on npm
- Source, GitHub Action and issues: github.com/sitefig/robots
Licence
PolyForm Noncommercial 1.0.0: free for noncommercial use. Commercial use needs a licence from Sitefig.
1 maintainer
Release files for susbot 2.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| susbot-2.1.0.tar.gz | 122.1 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| susbot-2.1.0-cp310-abi3-win_amd64.whl | CPython 3.10 | abi3 | Windows x86-64 | Details |
| susbot-2.1.0-cp310-abi3-musllinux_1_2_x86_64.whl | CPython 3.10 | abi3 | Linux musl 1.2+ x86-64 | Details |
| susbot-2.1.0-cp310-abi3-musllinux_1_2_aarch64.whl | CPython 3.10 | abi3 | Linux musl 1.2+ ARM64 | Details |
| susbot-2.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.10 | abi3 | Linux glibc 2.17+ x86-64 | Details |
| susbot-2.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | CPython 3.10 | abi3 | Linux glibc 2.17+ ARM64 | Details |
| susbot-2.1.0-cp310-abi3-macosx_11_0_arm64.whl | CPython 3.10 | abi3 | macOS 11.0+ ARM64 | Details |
| susbot-2.1.0-cp310-abi3-macosx_10_12_x86_64.whl | CPython 3.10 | abi3 | macOS 10.12+ x86-64 | Details |
Total release size: 15.0 MB
Release files / susbot-2.1.0.tar.gz
| Download URL | susbot-2.1.0.tar.gz |
|---|---|
| Size | 122.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
57fdf3372fd91d4b48b6611f0b9fa55d2a7d5201ed34e6384ac09fbffde550e2
|
|
BLAKE2b-256 checksum How to use checksums |
dbac7e4729a6dd6178d33666f73925fd3b18071328484dfce15528e4b003a50f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-win_amd64.whl
| Download URL | susbot-2.1.0-cp310-abi3-win_amd64.whl |
|---|---|
| Size | 2.1 MB |
| Tags | CPython 3.10 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
159d33c08591dbc69afc297a7aa8dd0665d5f86d7d141ec06a30ecb75e45d6a7
|
|
BLAKE2b-256 checksum How to use checksums |
a371156f3cf8571869973c043dc17edfa3eadfcce3bd64ed242df1c4e88c1ae0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-musllinux_1_2_x86_64.whl
| Download URL | susbot-2.1.0-cp310-abi3-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 2.3 MB |
| Tags | CPython 3.10 Linux musl 1.2+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
c32f03ddb66d61944ef5d09847db6bce545f6d48998651167f3a065ec4264a0e
|
|
BLAKE2b-256 checksum How to use checksums |
d4cb39fcd95d4127f1ba17f420cf5c7575cc2ef0e04fe8b8a161ce159f527141
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-musllinux_1_2_aarch64.whl
| Download URL | susbot-2.1.0-cp310-abi3-musllinux_1_2_aarch64.whl |
|---|---|
| Size | 2.2 MB |
| Tags | CPython 3.10 Linux musl 1.2+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
8d3f370cb7e9fe7f69463e359fa789cbce2b3bc11515cefa74e33763c34f2e9f
|
|
BLAKE2b-256 checksum How to use checksums |
cc056ea9959ca911388a5f0ed999f5d8ea848cd58998d72bf3fccffcb3ebe2b6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | susbot-2.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 2.1 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
af85b3aeeaab6f1af675845e46cbd4ccfe5de8d51f989433bca6975cca93d1fe
|
|
BLAKE2b-256 checksum How to use checksums |
1714f2008550c3e4c01de71146f1b1bab355a398be351064fcfbfebd3f05a42f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | susbot-2.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 2.0 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
7c7a901418ad399ee48faf69b0106d2f12bceb9b7f2e0095fdccaa09a5a324ab
|
|
BLAKE2b-256 checksum How to use checksums |
192b558c71a5238f3e695f76994780ac943805eca785c90c92d61e9e0fa143cc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-macosx_11_0_arm64.whl
| Download URL | susbot-2.1.0-cp310-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.0 MB |
| Tags | CPython 3.10 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
2973986c5e0193c0db6e1b5bdfba186adb5a6739be005d53fc36855e8da99ebe
|
|
BLAKE2b-256 checksum How to use checksums |
98ce9354679709eeb7cbdbf763699c64a355a814be8e29a1a0672a7a41ffa0c1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency logRelease files / susbot-2.1.0-cp310-abi3-macosx_10_12_x86_64.whl
| Download URL | susbot-2.1.0-cp310-abi3-macosx_10_12_x86_64.whl |
|---|---|
| Size | 2.1 MB |
| Tags | CPython 3.10 abi3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
611bf19bcb4cc5d2e388ce478c7197e66230e106eaa70a19437b8f12df68d114
|
|
BLAKE2b-256 checksum How to use checksums |
1debea62154ea0d5c6a7a44da2620e9a5640e3c71cd3fd33f54c7685b9e65495
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 20, 2026.
Transparency log
