Skip to main content

biston

A structural clone detector for Python code. Written in Rust.

It parses Python files with tree-sitter, normalizes the AST, and finds functions that are structurally similar to each other.

Install

uv add biston

Or build from source:

cargo build --release

Usage

biston <COMMAND>

Commands

biston scan

Scan a directory for code clones.

Usage: biston scan [OPTIONS] [PATH]

Arguments:
  [PATH]  Directory to scan [default: .]

Options:
      --format <FORMAT>        Output format [possible values: text, json, sarif]
      --min-lines <MIN_LINES>  Minimum function length in lines
      --threshold <THRESHOLD>  Similarity threshold (0.0 - 1.0)
      --config <CONFIG>        Config file directory (looks for biston.toml or pyproject.toml)
      --tests-only             Restrict the scan to Python test files (overrides include/exclude)
      --containment            Also report functions already implemented by a run of another
      --suggest                Generate abstraction suggestions for similar pairs
      --files <FILE>           Only emit pairs involving this file (repeat for multiple)
      --files-from <PATH>      Read focus file list from PATH, or `-` for stdin
  -h, --help                   Print help

biston stats

Show statistics about scan findings.

Usage: biston stats [OPTIONS] [PATH]

Arguments:
  [PATH]  Directory to scan [default: .]

Options:
      --format <FORMAT>        Output format [possible values: text, json, sarif]
      --min-lines <MIN_LINES>  Minimum function length in lines
      --threshold <THRESHOLD>  Similarity threshold (0.0 - 1.0)
      --config <CONFIG>        Config file directory (looks for biston.toml or pyproject.toml)
      --tests-only             Restrict the scan to Python test files (overrides include/exclude)
      --containment            Also report functions already implemented by a run of another
      --files <FILE>           Only emit pairs involving this file (repeat for multiple)
      --files-from <PATH>      Read focus file list from PATH, or `-` for stdin
  -h, --help                   Print help
Scanning tests only

Test suites often accumulate duplication (near-identical cases that could be @pytest.mark.parametrize, copy-pasted arrange/act/assert blocks). By default biston excludes test files so production-code findings stay focused. Pass --tests-only to flip the scope and scan only test files:

biston scan --tests-only
biston stats --tests-only

The flag replaces include with common Python test patterns (**/test_*.py, **/*_test.py, **/conftest.py, tests/**/*.py) and clears exclude. Other knobs (min_lines, threshold, normalization) are left untouched — tune them separately in biston.toml if you want different defaults for a test run.

Commit-hook use (focus files)

--files / --files-from let you restrict reporting to pairs involving a specific set of files, while still scanning the whole repo so cross-file clones between those files and the rest of the tree are detected.

For a pre-commit hook, pipe git diff --name-only through --files-from -:

git diff --name-only --diff-filter=ACM -- '*.py' \
  | biston scan --files-from - .

An empty list (no Python files changed) correctly emits no pairs. Prefer --files-from over --files $(git diff --name-only) — the latter expands to an empty flag when nothing changed, which reverts to a full-repo scan.

Configuration

Settings can go in biston.toml or under [tool.biston] in pyproject.toml. If both files exist, biston.toml takes priority. CLI flags override config file settings.

[scan]

Setting Default Description
min_lines 10 Minimum function length in lines
threshold 0.7 Similarity threshold (0.0–1.0)
exclude ["tests/**", "**/conftest.py", "migrations/**"] File patterns to exclude
include ["**/*.py"] File patterns to include

[normalization]

Setting Default Description
anonymize_locals true Replace local variable names
anonymize_literals false Replace literal values
strip_decorators true Remove decorators from AST
strip_type_annotations true Remove type hints
sort_commutative false Sort commutative operations

[output]

Setting Default Description
format "text" Output format (text, json, or sarif)
group_overlapping true Group overlapping clones
max_results 50 Maximum number of results
show_source true Display source code in output
context_lines 3 Number of context lines around clones

[suggest]

Setting Default Description
enabled false Enable suggestion generation
min_quality 0.6 Minimum template coverage score (0.0–1.0)
max_holes 5 Maximum holes before suppressing
render_python true Render templates as Python source

[containment]

Directed detection: one function already implements the leading or trailing run of another's body. Off by default; see Containment.

Setting Default Description
enabled false Enable containment detection (or pass --containment)
min_fragment_lines 15 Minimum executable lines in the matched run
min_ratio 0.30 Contained function size / container size
threshold 0.85 Minimum containment coefficient (0.0–1.0)
size_balance 1.25 Largest tolerated size ratio between the function and the run
max_run_fraction 0.85 Largest share of the container's statements a run may span
max_probes_per_function 12 Cap on candidate-generating probes per function

[suppress]

Setting Default Description
files [] File glob patterns to suppress entirely

Example biston.toml

[scan]
min_lines = 15
threshold = 0.8
exclude = ["vendor/"]
include = ["src/**/*.py"]

[normalization]
anonymize_locals = false
anonymize_literals = true

[output]
format = "json"
max_results = 100

[suggest]
enabled = true
min_quality = 0.8

Inline suppression

You can also suppress findings with Python comments:

  • # biston: ignore-file — suppress the entire file (must appear in the first 5 lines)
  • # biston: ignore — suppress a single function (place in the function body or on the preceding line)

When scan or overview reports clones, the text output ends with a one-line reminder of these options. Run biston usage for the full reference at any time:

biston usage

Documentation

Full docs at https://mojzis.github.io/biston/.

License

MIT

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

biston-0.5.5.tar.gz (922.0 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

biston-0.5.5-py3-none-win_amd64.whl (1.6 MB view details)

Uploaded Python 3Windows x86-64

biston-0.5.5-py3-none-manylinux_2_28_x86_64.whl (1.7 MB view details)

Uploaded Python 3manylinux: glibc 2.28+ x86-64

biston-0.5.5-py3-none-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded Python 3macOS 11.0+ ARM64

File details

Details for the file biston-0.5.5.tar.gz.

File metadata

  • Download URL: biston-0.5.5.tar.gz
  • Upload date:
  • Size: 922.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for biston-0.5.5.tar.gz
Algorithm Hash digest
SHA256 64f27d7e7dd5433a5e5e5aa2e216b15fe0d24c4db8b5b6d19e4f2376211edfc2
MD5 b35dbab62ffce29e4ab5a13f39b66416
BLAKE2b-256 db03e421021bf5ab5bd5386941cf9b27a1ac61393807fdb6aefd0963f18d16c8

See more details on using hashes here.

Provenance

The following attestation bundles were made for biston-0.5.5.tar.gz:

Publisher: release.yml on mojzis/biston

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file biston-0.5.5-py3-none-win_amd64.whl.

File metadata

  • Download URL: biston-0.5.5-py3-none-win_amd64.whl
  • Upload date:
  • Size: 1.6 MB
  • Tags: Python 3, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for biston-0.5.5-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 0c8394646c8aeaacd5c2f313cf015969fce8efa30d90dd069953d30744da7cd4
MD5 87454fc1abdab328223fb4921c6d7635
BLAKE2b-256 6b8a2701ec54ee9bfcc243b92b72b99d9b509eb771b1afd291408f1429b23e39

See more details on using hashes here.

Provenance

The following attestation bundles were made for biston-0.5.5-py3-none-win_amd64.whl:

Publisher: release.yml on mojzis/biston

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file biston-0.5.5-py3-none-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for biston-0.5.5-py3-none-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1b3af74753dc6eac4477a9b896346a8e6cd35a1b495ca640d0884f20f589aebf
MD5 1c27018e88ea8b93d8bb1a623eb2aef3
BLAKE2b-256 2996f0aa7d05c109a4d0dd54e87a3f24f219ed545d8f7ce5d11eebbc035749dc

See more details on using hashes here.

Provenance

The following attestation bundles were made for biston-0.5.5-py3-none-manylinux_2_28_x86_64.whl:

Publisher: release.yml on mojzis/biston

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file biston-0.5.5-py3-none-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for biston-0.5.5-py3-none-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3c6ad3b0b4592a415b7db89fcb0211b8de79556a24e2df1b7f1e767385255001
MD5 4a42d2978bd2ba61d253385404245812
BLAKE2b-256 98ea23e08a6f20eb3fba6fb0722cff0c8b141c0cc0e93db1827b5921af2afb37

See more details on using hashes here.

Provenance

The following attestation bundles were made for biston-0.5.5-py3-none-macosx_11_0_arm64.whl:

Publisher: release.yml on mojzis/biston

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.6.0

4 files

This release

0.5.5 This release

4 files

0.5.4

4 files

0.5.3

4 files

0.5.2

4 files

0.5.1

4 files

0.5.0

4 files

0.4.0

4 files

0.3.0

4 files

0.2.0

4 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page