detect-secrets-async
Async runtime for Yelp detect-secrets. Wraps the synchronous scanner in a bounded pool of long-lived subprocess workers so hosts can run secret scans with real timeouts, isolated per-request plugin configuration, and cross-request concurrency.
The package is intentionally generic — it has no knowledge of Agent Control, evaluator result mapping, payload normalization, or JSON Pointer metadata.
Install
uv add detect-secrets-async
detect-secrets==1.5.0 is pinned so the default plugin set stays stable across upgrades.
Quick start
import asyncio
from detect_secrets_async import ScanRequest, get_runtime
async def main() -> None:
runtime = get_runtime()
result = await runtime.scan(
ScanRequest(
content="github_token = 'ghp_123456789012345678901234567890123456'",
timeout_ms=5_000,
)
)
for finding in result.findings:
print(finding.type, finding.line_number)
asyncio.run(main())
How it works
One shared runtime per host process. It owns:
- a dedicated background event-loop thread that orchestrates all workers
- up to
pool_sizedetect-secretssubprocesses, each handling one request at a time - a bounded FIFO queue for bursts above
pool_size
Callers from any event loop or thread can await runtime.scan(...) concurrently. The runtime
queues, dispatches, and replaces workers on timeout, crash, protocol failure, or caller
cancellation.
Subprocess isolation is the reason you want this library. Upstream
detect-secretsis synchronous and configures itself through process-global settings, soasyncio.to_threadgives you theatrical timeouts (the scan keeps running) and state bleed between concurrent configs. A serializing lock kills throughput. Subprocesses solve both.
Scope (v1)
Supported
- text/content scanning with per-request plugin selection
- real timeouts across queue wait, worker startup, scan execution, and response read
- automatic worker replacement on timeout, crash, protocol failure, or caller cancellation
- host-level pool size, queue depth, and worker recycling
Not supported
- repository or directory scanning
- baselines or diff mode
- network-backed secret verification
- arbitrary plugin or filter imports
Configuration
Runtime settings are host-level with first-init-wins semantics: one runtime per process, pinned at first use. Configure it through environment variables or an explicit call before the first scan.
| Env var | Default | Field |
|---|---|---|
DETECT_SECRETS_ASYNC_POOL_SIZE |
4 |
pool_size |
DETECT_SECRETS_ASYNC_MAX_QUEUE_DEPTH |
16 |
max_queue_depth |
DETECT_SECRETS_ASYNC_MAX_REQUESTS_PER_WORKER |
100 |
max_requests_per_worker |
from detect_secrets_async import RuntimeConfig, configure_runtime
configure_runtime(RuntimeConfig(pool_size=8, max_queue_depth=32, max_requests_per_worker=200))
Re-initializing with the same config is a no-op. Conflicting config raises
RuntimeConfigConflictError.
API reference
Lifecycle
| Function | Purpose |
|---|---|
get_runtime(config=None, **overrides) |
Return the shared runtime, creating it on first call. |
configure_runtime(config=None, **overrides) |
Pin (or confirm) settings without starting workers. |
shutdown_runtime() (async) |
Tear down workers and the background thread. |
reset_runtime_for_tests() (async) |
Test-fixture alias for shutdown_runtime. |
get_runtime_info() |
Return RuntimeInfo (version + plugin metadata) without running a scan. |
get_runtime and configure_runtime accept either a RuntimeConfig or matching keyword
overrides (pool_size, max_queue_depth, max_requests_per_worker), not both.
Runtime
DetectSecretsRuntime is the object returned by get_runtime().
| Method | Purpose |
|---|---|
scan(request) (async) |
Run a pool-backed scan. Returns ScanResult, raises RuntimeScanError. |
shutdown() (async) |
Instance-level teardown. |
info() |
Same payload as get_runtime_info(). |
Types
class RuntimeConfig:
pool_size: int = 4 # worker subprocesses
max_queue_depth: int = 16 # queued requests when all workers busy (0 disables queueing)
max_requests_per_worker: int = 100 # recycle worker after N requests
class ScanRequest:
content: str # normalized text to scan (≤ ~16 MB protocol limit)
timeout_ms: int # full lifecycle: queue + startup + scan + read
config: ScanConfig = ScanConfig()
class ScanConfig:
enabled_plugins: tuple[str, ...] | None = None # None → upstream default plugins
class ScanResult:
findings: tuple[ScanFinding, ...]
detect_secrets_version: str
# .findings_count -> int (property)
class ScanFinding:
type: str # detect-secrets secret_type, e.g. "GitHub Token"
line_number: int | None # 1-indexed into content; None when unavailable
class RuntimeInfo:
package_version: str
detect_secrets_version: str
available_plugin_names: tuple[str, ...] # class names, e.g. "GitHubTokenDetector"
default_plugin_names: tuple[str, ...] # subset enabled by upstream default_settings()
configured_runtime: RuntimeConfig | None
Plugin naming.
enabled_pluginstakes detect-secrets class names ("GitHubTokenDetector","Base64HighEntropyString"). TheScanFinding.typefield reports the upstreamsecret_typestring instead ("GitHub Token", …). Discover valid class names viaget_runtime_info().available_plugin_names.
Errors
| Class | When |
|---|---|
DetectSecretsAsyncError |
Base class for all package errors. |
RuntimeConfigConflictError |
Conflicting runtime re-initialization. |
RuntimeScanError(code, message) |
Any scan-time failure. Inspect .code. |
ScanFailureCode enumerates the safe codes returned in RuntimeScanError.code:
| Code | Cause |
|---|---|
invalid_config |
e.g. unknown plugin name |
queue_full |
max_queue_depth reached |
queue_timeout |
queue wait exceeded timeout_ms |
worker_startup_error |
subprocess failed to launch or say hello |
worker_timeout |
scan exceeded timeout_ms |
worker_crash |
subprocess exited unexpectedly |
worker_protocol_error |
malformed frame on the subprocess channel |
runtime_error |
other sanitized runtime failure |
Raw exception strings, plaintext secrets, snippets, matching lines, and upstream hashed_secret
values are never surfaced in results or errors.
Operational notes
Cold start. Workers launch lazily, so the first scan in a process pays subprocess startup plus
detect-secrets import. Budget a few hundred milliseconds on the first request, especially in
short-lived SDK processes.
Cancellation. Cancelling an await runtime.scan(...) kills the assigned worker within a
bounded cleanup budget (~2 seconds) and replaces it — no silent background work.
Thread-safety. scan() is safe to call from any event loop or thread. All subprocess I/O
happens on one dedicated background loop inside the runtime, regardless of the caller's loop.
Plugin-name validation. If you validate plugin names before submitting (for example in an
evaluator adapter), detect-secrets must be importable in the validating process — not only in
the worker subprocesses.
Deployment footprint. Each host process gets one background thread plus up to pool_size
detect-secrets subprocesses. A gunicorn/uvicorn deployment with N application workers will
have N × pool_size detect-secrets children in total.
Tuning
| Repeated failure | Try |
|---|---|
queue_full |
raise pool_size, raise max_queue_depth, or smooth caller bursts |
queue_timeout |
raise timeout_ms or add worker/queue capacity |
worker_timeout |
raise timeout_ms, narrow enabled_plugins, or shrink payloads |
worker_startup_error |
verify detect_secrets_async and detect-secrets are installed in the caller's Python environment |
Benchmarking and soak runs
Defaults are conservative. Re-check them on representative hardware before raising them.
make benchmark
make soak
Useful overrides:
make benchmark BENCHMARK_ARGS="--pool-sizes 1 2 4 8 --pool-suite-requests 64"make soak SOAK_ARGS="--duration-seconds 60 --max-requests-per-worker 5 --client-count 12"
make benchmark sweeps pool size and recycle threshold and reports throughput plus p50/p95
latency. make soak runs a sustained non-CI stress pass that forces worker recycling and asserts
request bookkeeping and child processes drain cleanly on shutdown.
Development
make sync # install dev deps
make check # lint + typecheck + test + build
Individual targets: make lint, make typecheck, make test, make build, make benchmark,
make soak.
License
Apache-2.0
Metadata
Release files for detect-secrets-async 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| detect_secrets_async-0.2.0.tar.gz | 19.0 kB | With provenance | Details for detect_secrets_async-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| detect_secrets_async-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for detect_secrets_async-0.2.0-py3-none-any.whl |
Total release size: 40.1 kB
Release files / detect_secrets_async-0.2.0.tar.gz
| Download URL | detect_secrets_async-0.2.0.tar.gz |
|---|---|
| Size | 19.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6037f87b7a8b27c47bbbd9245961d58add22dfdcdefd0278c7133542b880e229
|
|
BLAKE2b-256 checksum How to use checksums |
a02a3ac1474dcce01f4e280e0fbd3f4ce92b71b00c5153aa8cd8d94680333df5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Apr 22, 2026.
Transparency logRelease files / detect_secrets_async-0.2.0-py3-none-any.whl
| Download URL | detect_secrets_async-0.2.0-py3-none-any.whl |
|---|---|
| Size | 21.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c2894074d9e83aaa96762c84a7f13a433c447615189e7b6c5e3b1833d361ea9f
|
|
BLAKE2b-256 checksum How to use checksums |
a4c4facdf7e1f22a10857ff22dbcaf2757fc2fac4453eafedc8ed49e9e04dfaa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Apr 22, 2026.
Transparency logSecurity for detect-secrets-async 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for detect_secrets_async-0.2.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Apr 22, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.