pyprocessors-chunk_sentences 0.6.34

Sherpa sentence chunking processor

pyprocessors_chunk_sentences

Create segments from annotations

Installation

You can simply pip install pyprocessors_chunk_sentences.

Developing

Pre-requisites

You will need to install uv for dependency management and building:

pip install uv

Clone the repository:

git clone https://github.com/oterrier/pyprocessors_chunk_sentences

Install the project with test dependencies:

uv sync --extra test

Running the test suite

You can run the full test suite with:

uv run pytest

Linting and formatting

This project uses ruff for linting and formatting:

uv run ruff check .
uv run ruff format --check .

Building the documentation

You can build the HTML documentation with:

uv sync --extra docs
uv run sphinx-build docs docs/_build

The built documentation is available at docs/_build/index.html.

SBOM & vulnerability check

Install the SBOM dependencies:

uv sync --extra sbom

Generate a CycloneDX SBOM from the current environment:

uv run cyclonedx-py environment -o sbom.cdx.json --output-format json

Audit dependencies for known vulnerabilities:

uv run pip-audit --format json --output audit-report.json

To fail on any known vulnerability (useful in CI):

uv run pip-audit --strict