PTREPO - Exposed repository testing tool
ptrepo is a Penterep tool for testing exposed source-code repositories on web servers. The planned scope covers repository discovery, best-effort repository metadata/content download, commit/revision listing where practical, and native secret scanning of recovered content.
Current MVP status
Current version supports discovery, Git/SVN best-effort content download, Mercurial/Bazaar metadata download, reachable or dangling Git commit listing, observed SVN revision reporting, observed Mercurial/Bazaar count reporting from metadata where possible, and native secret scanning of recovered Git/SVN content, Git history, and recovered Mercurial/Bazaar metadata.
Terminal output shows concise history/count summaries; JSON output keeps the detailed recovery source in historyCoverage.
Extended SCM types such as Darcs, Fossil, CVSweb, and RCS are discovery-level in this MVP. If --download, --commits, or --secrets is requested for a confirmed unsupported SCM type, ptrepo reports an explicit unsupported warning instead of printing a false commit count or ambiguous empty result.
Implemented:
- URL normalization
.git,.svn,_svn,.bzr,.rsc,.hg,.fslckout,_darcs,_fossil_,_FOSSIL_,cgi-bin/cvsweb.cgi, andRCScandidate generation- HTTP probing
- discovery classification
- Git recovery for metadata, refs, reflogs, loose objects, pack files, files recoverable from
.git/index, and files exportable from the reconstructed Git object database - SVN recovery for
entries,text-base,wc.db,pristine, and recovered file contents - Mercurial and Bazaar/Breezy metadata recovery for known metadata paths
- Git validation and history reporting through a defensive low-level
gitbackend - SVN observed revision reporting from recovered
entriesandwc.dbmetadata - Mercurial changeset count reporting from
.hg/store/00changelog.iwhen the recovered revlog index is usable - Bazaar/Breezy revision count reporting from
.bzr/branch/last-revisionwhen available - native Git commit patch/message, recovered Git/SVN file, and recovered Mercurial/Bazaar metadata secret scanning with built-in rules, redaction, fingerprints, and coverage reporting
- human and JSON output
Git download currently saves:
.git/HEAD.git/config.git/index.git/packed-refs.git/info/refs.git/objects/info/packs- validated
.git/shallowboundaries for shallow clones .git/logs/HEADand discovered/common ref logs- branch/tag ref files where discovered
- loose objects discovered from refs, reflogs, commits, trees, and
.git/index - pack files discovered from
.git/objects/info/packs, an accessible.git/objects/pack/directory listing, or standalone/incremental multi-pack-index metadata - locally reconstructed pack indexes where a
.packfile is recovered but the matching.idxfile is unavailable or invalid - recovered blob contents under
git/files/ - files exported from reachable or dangling commit trees when the local Git object database is usable
Fresh clones often store Git objects in packs without an objects/info/packs
list. PTREPO can recover these packs when their names and contents are exposed;
otherwise, full recovery may be unavailable. Pack names cannot be derived from
commit IDs, and multi-pack-index metadata is optional.
When Git objects are unavailable, PTREPO also tries selected files on the same website, including SVN pristine copies, and verifies their contents against the Git index. This can recover current files but not missing history, so coverage remains partial. Findings retain the URL of the recovered source when available.
Indexed versions take precedence over older versions. If Git paths collide on
the local filesystem, PTREPO saves additional files under git/file-collisions/
and reports partial recovery. Shallow clones also have partial history; external
remotes and borrowed object databases are not fetched.
SVN download currently saves:
.svn/entriesor_svn/entries.svn/wc.dbor_svn/wc.db- old working-copy
text-basefiles where discoverable - recursive old working-copy
entries/text-basefiles where subdirectory metadata is exposed - new working-copy
pristinefiles where discoverable fromwc.db - recovered file contents under
svn/files/
Mercurial/Bazaar download currently saves selected metadata and safe metadata-derived files only. It does not reconstruct full working trees or history:
.hg/requires.hg/hgrc.hg/branch.hg/bookmarks.hg/bookmarks.current.hg/dirstate.hg/store/fncache.hg/store/phaseroots.hg/store/obsstore.hg/store/00changelog.i.hg/store/00changelog.d.hg/store/00manifest.i.hg/store/00manifest.d- safe
.hg/store/data/*.i,.hg/store/data/*.d,.hg/store/meta/*.i, and.hg/store/meta/*.dpaths listed in.hg/store/fncache .bzr/branch-format.bzr/branch/format.bzr/branch/last-revision.bzr/branch/branch.conf.bzr/branch/tags.bzr/branch/revision-history.bzr/repository/format.bzr/repository/pack-names.bzr/repository/packs/*.packand common.bzr/repository/indices/*files listed in.bzr/repository/pack-names.bzr/checkout/format.bzr/checkout/dirstate.bzr/checkout/inventory.bzr/checkout/stat-cache
When --download is used, ptrepo also reports available Git commit counts, observed SVN revision counts, and observed Mercurial/Bazaar counts without printing detailed history entries. Use --commits for bounded detailed history output.
These planned options are accepted by the CLI contract but intentionally fail in the current MVP slice:
-r/--redirects-C/--cache
Installation
pip install ptrepo
Adding to PATH
If you're unable to invoke the script from your terminal, it's likely because it's not included in your PATH. You can resolve this issue by executing the following commands, depending on the shell you're using:
For Bash Users
echo "export PATH=\"`python3 -m site --user-base`/bin:\$PATH\"" >> ~/.bashrc
source ~/.bashrc
For ZSH Users
echo "export PATH=\"`python3 -m site --user-base`/bin:\$PATH\"" >> ~/.zshrc
source ~/.zshrc
Usage examples
ptrepo -u https://www.example.com/
ptrepo -u https://www.example.com/plugins/mpdf
ptrepo -u https://www.example.com/ -t git svn bzr hg cvs darcs fossil rcs
ptrepo -f urls.txt -w repository_paths.txt
ptrepo -u https://www.example.com/ --download
ptrepo -u https://www.example.com/ --download ~/Download/repo
ptrepo -u https://www.example.com/ --commits
ptrepo -u https://www.example.com/ --commits --commit-limit 20
ptrepo -u https://www.example.com/ --download --commits
ptrepo -u https://www.example.com/ --secrets
ptrepo -u https://www.example.com/ --secrets --secret-scan-timeout 120 --secret-regex-timeout 0.5
ptrepo -u https://www.example.com/ --threads 5
ptrepo -u https://www.example.com/ --max-response-bytes 32768 -j
ptrepo -u https://www.example.com/ --secrets --max-download-bytes 104857600 --max-secret-file-size 5242880
Options
-u --url <url> Test specified URL
-f --file <file> Load URLs from file
-w --wordlist <file> Load additional supported repository path candidates from file
-t --repo-type <type> Repository type(s) to test: git, svn, bzr, hg, cvs, darcs, fossil, rcs (default: all supported types)
--download [directory] Download recoverable repository content/metadata; defaults to current directory
--commits Temporarily recover metadata and list Git commits or observed SVN/Hg/Bzr counts
--commit-limit <count> Maximum commit/revision entries to print and Git commits to scan; 0 disables both (default: 10)
--secrets Temporarily recover supported repository content/metadata and scan for secrets
--secrets-rules <file> Load additional JSON secret rules
--secrets-baseline <file> Ignore previously reported secret finding fingerprints
--secrets-mode <mode> Secret scan mode: all, files, or history (default: all; history is Git only)
--no-entropy Disable entropy checks for applicable secret rules (enabled by default)
--allowlist <file> Load JSON secret allowlist
--max-secret-file-size <bytes|unlimited> Maximum recovered file size to scan for secrets (default: --max-download-bytes)
--secret-scan-timeout <seconds> Time budget per repository scan phase: files or Git history (default: 30 seconds)
--secret-regex-timeout <seconds> Timeout per secret regex or allowlist operation (default: 0.25 seconds)
-H --headers <header:value> Add custom request header(s) (default: no additional headers)
-T --timeout <timeout> Set timeout (default: 10 seconds)
--threads <count> Maximum concurrent repository discovery candidates (1-64; default: 1)
--max-response-bytes <bytes|unlimited> Maximum bytes to read from each discovery response (default: unlimited)
--max-download-bytes <bytes|unlimited> Maximum bytes per downloaded file or recovered Git blob (default: unlimited)
-a --user-agent <user-agent> Set HTTP User-Agent sent with requests (default: "Penterep Tools")
-c --cookie <cookie=value> Set cookie(s)
-p --proxy <proxy> Set proxy (e.g. http://127.0.0.1:8080)
-v --version Show script version and exit
-h --help Show this help message and exit
-j --json Output JSON only, suppresses banner and human output
Discovery and downloads have no byte limit by default. Use a non-negative byte
count or unlimited for either HTTP size limit. A value of 0 allows no response
bytes; it does not mean unlimited. The download limit applies to each downloaded
file and recovered Git blob, not the total repository size. Oversized or truncated
repository content is reported as incomplete.
When omitted, --max-secret-file-size inherits --max-download-bytes, including
unlimited or 0. Set it explicitly to choose a separate secret scan file limit;
for example, --secrets --max-download-bytes 104857600 --max-secret-file-size 5242880
allows downloads up to 100 MiB and scans recovered files up to 5 MiB. Explicit
--max-secret-file-size unlimited overrides a finite download limit, though it
cannot restore content that recovery skipped. This option requires --secrets.
Without --download, --secrets temporarily recovers content into the system
temporary directory and removes it after analysis. --download [directory]
keeps recovered content in the selected directory. Byte limits apply in both
modes; they do not set a total disk-space budget.
During interactive terminal downloads, progress shows the current URL path and bytes received. A percentage is shown only for an individual response with a usable, unencoded content length; the total repository size is not estimated. Progress is omitted for JSON and redirected output.
With --secrets -j, each target includes scanCoverage with a status
(scanned, partial, or not_scanned), reason codes, and counts for files and
commits. A run can finish with individual targets marked not_scanned, for
example if the server rate-limits discovery. Use the target status to interpret
an empty findings list.
For Git, --download shows the total number of files recovered and how many
files in the Git index were recovered or are still missing. If the index
cannot establish the tracked file counts, the terminal reports this and
fileCoverage.trackedFiles and fileCoverage.missingTrackedFiles are null
in JSON. historyCoverage describes the recovered history separately.
Temporary analysis retains counts after removing local paths from the report.
Terminal and JSON report URLs omit credentials, query strings, and fragments. When a target URL includes credentials, each set of credentials gets its own output directory for that run. A later run creates a fresh directory for the authenticated target.
Planned options
These options are accepted by the CLI contract but intentionally fail in the current MVP slice.
-r --redirects Planned, not implemented in current MVP slice
-C --cache Planned, not implemented in current MVP slice
Secret rule files
--secrets-rules loads additional JSON rules. The file may contain either a list of rules or an object with a rules list.
Custom rules must include at least one keyword so the scanner can skip regex evaluation on unrelated lines.
Custom rule regexes are length-limited, secret_group must reference an existing capture group, and entropy_threshold must be between 0.0 and 8.0.
{
"rules": [
{
"id": "custom-demo-token",
"name": "Custom demo token",
"description": "Project-specific token",
"regex": "(DEMO_[A-Z0-9]{12})",
"secret_group": 1,
"keywords": ["DEMO_"],
"severity": "high",
"confidence": "medium",
"allowlist": {
"patterns": ["DEMO_PUBLIC_FIXTURE"],
"regexes": ["^DEMO_TEST_[A-Z0-9]+$"]
}
}
]
}
--allowlist loads JSON allowlists:
{
"patterns": ["known-fixture-value"],
"regexes": ["^example_[A-Za-z0-9]+$"]
}
Allowlist regexes are length-limited before compilation.
--secrets-baseline loads previously reported fingerprints and suppresses
matching findings. It accepts either a JSON list of fingerprint strings, an
object with a fingerprints list, or a previous PTREPO-style JSON report that
contains nested fingerprint fields. Suppressed findings are counted as
ignored baseline findings in human and JSON output.
When Git history contains more commits than --commit-limit, history-aware
secret scanning reports partial coverage instead of implying that the whole
history was scanned. Setting --commit-limit 0 disables detailed history
listing and Git commit secret scanning; file-mode secret scanning can still run.
Mercurial and Bazaar/Breezy secret scanning is metadata-only in this MVP; it
does not reconstruct or scan each historical changeset/revision.
Built-in secret rules cover common provider and generic credential patterns,
including private key markers, AWS AKIA/ASIA access key IDs, GitHub tokens,
GitLab access/build/deploy/runner/OAuth token prefixes, Slack tokens and
incoming webhooks, Stripe secret/restricted/webhook keys, Google API keys and
OAuth client secrets, npm tokens, PyPI tokens, SendGrid API keys, Discord
webhooks, Azure Storage account keys, Authorization Bearer tokens, Google
service-account JSON, database URLs with credentials, URLs with embedded
credentials, JWT-like tokens, generic password/token/API key assignments,
API signature credential assignments,
username/login assignments, and conservative base64/hex decoded credential
assignments. Git history scanning
checks added and deleted patch lines plus commit message text. Recovered-file
scanning skips oversized files and files that look binary based on NUL bytes or
a high ratio of binary control bytes.
Generic assignment rules recognize credential terms in longer key names (for
example databasePassword or service_api_key), including quoted PHP array
keys. Password values of at least eight characters can be detected even when
their entropy is low. Generic matching is heuristic and cannot reliably infer
that an arbitrary value is a secret without a recognizable key or token format.
PHP array entries and equality comparisons against quoted credential literals
are also scanned. Unquoted comparison operands are treated as code rather than
recovered credential values.
API signature detection recognizes names such as API_SIGNATURE and
apiSignature, including prefixed keys ending in that credential name. Related
metadata keys such as API_SIGNATURE_ALGORITHM are excluded. It requires a
quoted literal in source code or a standalone uppercase environment assignment,
and applies a minimum length and entropy check to reduce unrelated signature
findings.
Username and login literals, including short values such as root or sa, are
reported with low severity. Supported keys include user, username, login,
and configuration keys such as mysqlUser or DB_USER. Quoted literals and
simple unquoted uppercase environment assignments are supported. The rule skips
obvious variable references, interpolation, and function calls. A username alone
does not establish that a password or token was exposed.
Human output separates each finding into its rule and severity/confidence, location and commit, recovered value, and source excerpt. The recovered value text is red and the source excerpt is gray; rule and location text remain neutral. The finding keeps its error icon. Structured JSON findings remain redacted.
Recovered files are read incrementally. Line-based scanning processes up to
1,048,576 characters per line. Longer lines are truncated and reported as partial
coverage. Context parsing also uses bounded buffers and reports partial coverage
when its limits prevent a complete scan. Large service-account JSON documents are
parsed incrementally, retaining credential fields of up to 1,048,576 characters
and JSON nesting of up to 1,024 levels. Literal context allowlist patterns are
checked against the actual file. Whole-file regex allowlists that cannot be
evaluated from bounded input mark context coverage as partial; regex allowlists
for individual lines still run normally. Regex and allowlist
operations have a default 0.25-second time limit. Each repository's recovered-file
scan phase has one shared 30-second budget, and its Git history scan has a separate
30-second budget. Set these independently with
--secret-regex-timeout and --secret-scan-timeout (both require --secrets);
values must be finite positive numbers of seconds, up to 1,000,000,000 seconds
to stay within the supported timer range. A regex operation cannot exceed the
remaining scan budget. The HTTP --timeout controls connection and read waits,
not total runtime, and does not change these scan limits.
Timeouts, match limits, and exhausted budgets are
reported as partial coverage. Long evidence and terminal source lines are
shown as bounded excerpts. These safeguards apply to recovered files, Git
patch lines, and commit messages; they can leave some content unscanned.
The scan budget is checked between operations, not a hard deadline for the
whole process: Git commands, file decoding and other non-regex operations
may finish after it has elapsed. Git commit enumeration runs before the
history-scan budget starts.
Loose Git objects, exported blobs and secret-scan patches use disk-backed streams. This does not impose a process-wide memory limit: some existing SCM metadata parsers still materialize their input, and native Git commands manage their own memory. Unlimited byte limits also do not set a total temporary disk-space budget.
Secret rules use the regex engine in its Python-compatible VERSION0 mode.
Existing Python re expressions generally work, but engine behavior is not
identical for all expressions, including some Unicode case-insensitive matches.
Custom rules and allowlists should be checked against representative inputs.
Dependencies
ptlibs>=1.0.33,<2
requests>=2.31,<3
regex>=2024.11.6,<2027
License
Copyright (c) 2026 Penterep Security s.r.o.
ptrepo is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
ptrepo is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with ptrepo. If not, see https://www.gnu.org/licenses/.
Warning
You are only allowed to run the tool against the websites which you have been given permission to pentest. We do not accept any responsibility for any damage/harm that this application causes to your computer, or your network. Penterep is not responsible for any illegal or malicious use of this code. Be Ethical!
Metadata
Release files for ptrepo 0.0.14
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ptrepo-0.0.14.tar.gz | 189.3 kB | Without provenance | Details for ptrepo-0.0.14.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ptrepo-0.0.14-py3-none-any.whl | Python 3 | none | any | Details for ptrepo-0.0.14-py3-none-any.whl |
Total release size: 299.1 kB
Release files / ptrepo-0.0.14.tar.gz
| Download URL | ptrepo-0.0.14.tar.gz |
|---|---|
| Size | 189.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ac5e663b10a771892b318b7eb26b498220e23e4e4456ecdc681d3091c3ccc6b0
|
|
BLAKE2b-256 checksum How to use checksums |
24cf649a7a20f3c90efca8be78706afda294b7a55d62f50c959393445b354bf6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.18
|
Release files / ptrepo-0.0.14-py3-none-any.whl
| Download URL | ptrepo-0.0.14-py3-none-any.whl |
|---|---|
| Size | 109.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a56e7d94d517e87465ec489fc3613c05ec59be0972d07711169bad071a0aa496
|
|
BLAKE2b-256 checksum How to use checksums |
9e2201d26204408a7ce3eaf77b4f7f5252bed877575937a0662bda420c5647d0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.18
|
Security for ptrepo 0.0.14
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.