pysegmenters-blingfire 1.8.58

Segmenter based on BlingFire

pysegmenters_blingfire

Rule based segmenter based on BlingFire

Installation

You can simply pip install pysegmenters_blingfire.

Developing

Pre-requisites

You will need to install uv (for managing the package and running tests):

pip install uv

Clone the repository:

git clone https://github.com/oterrier/pysegmenters_blingfire

Install the package in development mode with test dependencies:

uv sync --extra test

Running the test suite

uv run pytest

Linting

uv run ruff check .
uv run ruff format --check .

Building the documentation

uv run --extra docs sphinx-build docs docs/_build

The built documentation is available at docs/_build/index.html.

SBOM & vulnerability check

Install the SBOM dependencies:

uv sync --extra sbom

Generate a CycloneDX SBOM from the current environment:

uv run cyclonedx-py environment -o sbom.cdx.json --output-format json

Audit dependencies for known vulnerabilities:

uv run pip-audit --format json --output audit-report.json

To fail on any known vulnerability (useful in CI):

uv run pip-audit --strict