Skip to main content

prolix

Heuristic clarity and vagueness scoring for English prose.

prolix gives a sentence or a whole document a set of scores from 0 to 100, where a higher value always means clearer text. It looks at how many dictionary senses the words have, how precise the vocabulary is, how many vague words appear and how many sentences contain structurally ambiguous patterns. It is most useful for comparing two versions of the same text.

Install

Install the package from PyPI.

pip install prolix

The package needs Python 3.10 or newer.

Download the language data

Download the language data once. The package uses the NLTK tokenizer and tagger and Open English WordNet.

python -m nltk.downloader punkt_tab averaged_perceptron_tagger_eng
python -m wn download oewn:2025

Open English WordNet 2025 is about 11 MB to download and takes about 105 MB on disk. It is stored in the .wn_data directory in your home directory unless the WN_DATA_DIR environment variable names another directory. The NLTK tokenizer and tagger take about 22 MB on disk. They are stored in the nltk_data directory in your home directory unless the NLTK_DATA environment variable names another directory.

Clarity never downloads anything. When the data is missing, creating a Clarity object raises LookupError with the two commands above, and the prolix command prints them and exits with code 2.

The coherence signal of the lower level API is optional and needs an extra.

pip install "prolix[semantic-coherence]"

This extra installs sentence-transformers and PyTorch, and PyTorch is by far the largest part of the install. The all-MiniLM-L6-v2 model adds about 90 MB on top of PyTorch. It is downloaded from the Hugging Face Hub the first time a MiniLMCoherenceScorer is created. On Linux, install the CPU build of PyTorch first if you do not need GPU support.

pip install torch --index-url https://download.pytorch.org/whl/cpu

The all extra installs every optional runtime feature.

pip install "prolix[all]"

Quick start

Score a sentence or a whole document.

from prolix import Clarity

clarity = Clarity()
scores = clarity.score("Replace the leaking kitchen faucet before Friday")

print(scores.clarity)
print(scores.to_dict())

Every score runs from 0 to 100 and a higher value always means clearer text. The clarity field is the headline score. It combines lexical clarity, which is high when words have few dictionary senses, specificity, which is high when the vocabulary is precise and free of vague words, and syntax, which is high when sentences avoid structurally ambiguous patterns. The readability field is the Flesch reading ease score and is reported next to the headline without changing it. The words field counts the words that were scored, so you can weight averages across many texts.

Create one Clarity object and reuse it. It can be shared between threads and passed to worker processes. Threads can share it only when prolix opens WordNet before your application uses the wn package itself, so create the Clarity object first or use processes. Text with no words of two or more letters scores 0 on every field.

Add words that count as vague in your own field.

clarity = Clarity(extra_vague_words=["synergy", "leverage"])

Matching ignores case and extra spaces, and an entry can be a phrase of several words. Entries are matched only against words made of letters, so an entry that contains a hyphen, an apostrophe or a digit, such as cutting-edge, never matches.

What the score means

The clarity score is an uncalibrated heuristic. It has not been validated against human judgments of clarity or against task success. Use it to compare two versions of the same text, not to rank unrelated documents or to grade text against a fixed bar.

vague = clarity.score("We need to handle some stuff with the thing")
concrete = clarity.score("We need to replace the leaking kitchen faucet")
assert vague.clarity < concrete.clarity

The headline is 0.4 times lexical plus 0.4 times specificity plus 0.2 times syntax. Lexical clarity and specificity are averaged over words and syntax is averaged over sentences, so the headline does not depend on the length of the text. A text and the same text repeated ten times get the same clarity.

Most real text scores near the middle of the range rather than near 100, so a middling score is not a failing grade. If you gate a build on the score, choose the threshold from the scores of your own documents. Compare texts of the same kind, sentences with sentences and documents with documents.

The readability field is the Flesch reading ease score limited to the range 0 to 100. It measures sentence and word length, not ambiguity. Plain short sentences often reach the top of its range.

Limitations

The scorer works on English only. Text in other languages gets scores that carry no meaning and are often higher than the scores of comparable English text.

Words that are not in WordNet, such as product names, count as having a single sense. They also count as general rather than specific words. Sense counts ignore context, so common verbs such as run, set and check have many senses and lower lexical clarity, while rare, technical, misspelled or made up words have few senses and raise it. A string of rare nouns or of nonsense words can score higher than a clear instruction.

Text with no words of two or more letters, such as an empty string, punctuation or digits, scores 0 on every field and its words field is 0. Check words to tell such text apart from unclear text.

Markdown markup is not counted as words, but the text of code blocks and YAML front matter is scored like prose. Remove code blocks first if they should not count. The prolix command removes front matter from files and standard input, and Clarity.score does not.

Sentences are counted as runs of periods, question marks and exclamation marks. A list of items without end punctuation counts as one sentence.

Python scores depend on the installed versions of Open English WordNet and the NLTK models.

Command line

Score a file from the command line and fail a build when the score is too low.

prolix SKILL.md --json --fail-under 40

The command removes YAML front matter from files before scoring. It exits with 1 when the score is below the threshold and with 2 when the input cannot be read.

The input is a file, a single hyphen for standard input, or the text to score. python -m prolix runs the same command.

prolix "Click the Save button to store the file."
prolix - < SKILL.md
python -m prolix SKILL.md

The command prints one line for each field of ClarityScores, the field name followed by its value. With --json it prints a JSON object with the same keys as ClarityScores.to_dict, in the same order. --fail-under is compared with the clarity field, with or without --json. Input with no words to score is reported as skipped on standard error and exits with 0, even when --fail-under is set. Without --json it prints no scores.

The exit status is 0 on success, 1 when the clarity score is below --fail-under, and 2 for a usage error, input that cannot be read or decoded as UTF-8, or missing language data.

prolix dir PATH scores every file in a directory that matches --pattern, which defaults to **/SKILL.md, and prints one line per file. --output writes the results to a CSV file, or to a JSON file when the name ends in .json. --readability adds the readability score and --coherence adds the coherence score, which needs the semantic-coherence extra. This subcommand reports the lower level analyze scores described below, not the Clarity scores. It exits with 1 when the path is not a directory, no file matches or a file fails to score. prolix-dir is an alias of prolix dir. To score the word dir as text, pass it after --, as in prolix -- dir.

Use as an evaluation metric

Clarity works as a metric in evaluation frameworks without an adapter. Score the text, compare the clarity field with a threshold and build the reason from the fields of ClarityScores. The package depends on none of the frameworks below, so install the one you use yourself.

Choose the threshold from a baseline of your own texts. One way is to score texts you already accept and set the threshold just below the lowest of their scores. You can also compare the score of a new version of a text with the score of the old version instead of using a fixed bar. Do not divide the score by 100 and gate at 0.5, a common default in evaluation frameworks. Most real text scores near the middle of the range and many clear texts score below 50, so that gate fails texts that are fine.

Output with no words to score gets 0 on every field and fails any threshold above 0. Check the words field if empty output needs its own handling.

Clarity never downloads anything. A grader that runs without network access needs the language data installed ahead of time with the commands in Download the language data, for example when its image is built. Set NLTK_DATA and WN_DATA_DIR when the data is not in the default directories.

The examples share these definitions. Create the Clarity object once and reuse it for every sample.

from prolix import Clarity, ClarityScores

clarity = Clarity()

# Set the threshold from the scores of your own baseline texts.
THRESHOLD = 40.0


def explain(scores: ClarityScores) -> str:
    return ", ".join(f"{name} {value}" for name, value in scores.to_dict().items())

A promptfoo Python assertion returns a pass flag, a score and a reason. promptfoo reports scores from 0 to 1, so the score is divided by 100, while the pass flag compares the clarity field with your threshold. Put the definitions and this function in one file, such as clarity_assert.py.

def get_assert(output, context):
    scores = clarity.score(output)
    return {
        "pass": scores.clarity >= THRESHOLD,
        "score": scores.clarity / 100,
        "reason": explain(scores),
    }

Reference the file from the assertions of a test.

assert:
  - type: python
    value: file://clarity_assert.py

A DeepEval metric sets a score, a success flag and a reason. DeepEval also reports scores from 0 to 1, so the metric reports the clarity field divided by 100 and takes its threshold on the same scale. DeepEval copies a metric by passing its attributes back to the constructor, so the constructor stores the threshold unchanged.

from deepeval.metrics import BaseMetric
from deepeval.test_case import LLMTestCase


class ClarityMetric(BaseMetric):
    def __init__(self, threshold: float = THRESHOLD / 100):
        self.threshold = threshold

    def measure(self, test_case: LLMTestCase) -> float:
        scores = clarity.score(test_case.actual_output)
        self.score = scores.clarity / 100
        self.success = self.score >= self.threshold
        self.reason = explain(scores)
        return self.score

    async def a_measure(self, test_case: LLMTestCase) -> float:
        return self.measure(test_case)

    def is_successful(self) -> bool:
        return self.success

    @property
    def __name__(self):
        return "Clarity"

An Inspect scorer returns the clarity field as the value of each sample, and the mean metric averages it over the samples.

from inspect_ai.scorer import Score, Target, mean, scorer
from inspect_ai.solver import TaskState


@scorer(metrics=[mean()])
def clarity_scorer():
    async def score(state: TaskState, target: Target) -> Score:
        scores = clarity.score(state.output.completion)
        return Score(value=scores.clarity, explanation=explain(scores))

    return score

A pytest gate scores each file as its own test and shows every field when a file falls below the threshold.

from pathlib import Path

import pytest


@pytest.mark.parametrize("path", sorted(Path("prompts").glob("*.txt")), ids=str)
def test_prompt_is_clear(path):
    scores = clarity.score(path.read_text(encoding="utf-8"))
    assert scores.clarity >= THRESHOLD, explain(scores)

Lower level API

The functions below return more detail than Clarity, such as each vague word found, the ambiguous patterns and the WordNet data of each word. They are outside the stability policy and may change in any minor release before 1.0. Their scores differ from Clarity scores by design. analyze uses different default weights and adds a document structure signal for text with more than one line, so its clarity_score is not the Clarity headline.

from prolix import ScoreWeights, analyze

result = analyze("The chef stirred the soup with the spoon.", include_readability=True)
print(result.clarity_score)
print(result.readability_score)
print([hit.phrase for hit in result.vague_hits])
print(result.to_dict())

readability_score maps the Flesch reading ease score onto the range 25 to 100 with a square root, so it is not the same number as the readability field of Clarity. readability_score and context_coherence are reported next to clarity_score and change it only when the readability or coherence weight of ScoreWeights is above zero.

weights = ScoreWeights(readability=0.2)
result = analyze(
    "The chef stirred the soup with the spoon.", include_readability=True, weights=weights
)
print(result.clarity_score)

For the coherence signal, install the semantic-coherence extra and pass a MiniLMCoherenceScorer from prolix.semantic_coherence as coherence_scorer. Unlike Clarity, analyze downloads the NLTK tokenizer and tagger the first time it runs if they are missing. Open English WordNet is never downloaded and must be installed first.

Function Purpose
analyze(text) Detailed analysis of one text, returned as an AnalysisResult
clarify(path) Score one file, every Markdown file in a directory, or the entries of a batch JSONL file
extract_primitives(text) Compute every raw signal once for research on formulas
apply_formula(primitives, weights) Combine raw signals with FormulaWeights
analyze_lexical, analyze_syntax, find_vague_hits, tag_sentence The building blocks of the scores
Class Description
AnalysisResult Result of analyze with every sub-score and report
ScoreWeights Weights of the signals in the analyze formula
ClarityPrimitives Every raw signal of one text
LexicalReport WordNet sense counts and depths of the words
SyntaxReport The structurally ambiguous patterns found
VagueHit One vague word or phrase found in the text

Stability policy

From release 0.3.0 the following parts of the package are stable.

  • The names Clarity, ClarityScores, score, extra_vague_words and to_dict.
  • The six fields of ClarityScores with their names, order and types.
  • The range 0 to 100 of every score, finite values, rounding to two decimals and the rule that a higher value always means clearer text.
  • The meaning of each field, the headline as a weighted average of lexical, specificity and syntax without readability, and the all zero result with words equal to 0 for text with nothing to score.
  • The exception types TypeError and LookupError.
  • Both classes can be pickled, and ClarityScores is immutable and supports equality.
  • The --json and --fail-under options of prolix, the --json keys, which equal the to_dict keys, and the exit codes 0, 1 and 2.

A minor release may add fields at the end of ClarityScores, new optional keyword arguments and new command line options. Read fields by name, do not build or unpack ClarityScores by position, and ignore unknown keys when you read the JSON output.

Score values are not frozen. A minor release may change them when the vague word list, the weights, the constants, the detectors or the bundled data change. The changelog lists every such change under a Scoring changes heading. A patch release never changes a score for the same input and the same data.

Before 1.0, removing or renaming a stable item or changing its type or meaning takes at least one minor release with a DeprecationWarning and a Breaking entry in the changelog. After 1.0 it takes a major release.

The repr text, the wording of error messages and performance are not promised. Everything else the package exports is the lower level API and may change in any minor release before 1.0.

Development

Run the tests from the root of a source checkout.

git clone https://github.com/TrijeetSethi/prolix-py
cd prolix-py
pip install -e ".[dev]"
python -m nltk.downloader punkt_tab averaged_perceptron_tagger_eng
python -m wn download oewn:2025
pytest
ruff check .

The coherence tests are skipped unless sentence-transformers is installed. See CONTRIBUTING.md for the full set of checks and RELEASING.md for how a release reaches PyPI.

Data and attribution

The package bundles no WordNet or NLTK data. At run time it reads Open English WordNet 2025 from https://en-word.net, which you download with python -m wn download oewn:2025. Open English WordNet is Copyright 2019-present The Open English WordNet Team and is licensed under CC BY 4.0, https://creativecommons.org/licenses/by/4.0/. It is derived from Princeton WordNet 3.0, Copyright 2006 by Princeton University. The package also uses the NLTK punkt_tab tokenizer and averaged_perceptron_tagger_eng tagger models from NLTK Data, which are distributed under their own licenses.

License

prolix is licensed under the MIT license. The license text is in LICENSE.md.

Metadata

Release files for prolix 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for prolix 0.3.0
File Size Uploaded Provenance Actions
prolix-0.3.0.tar.gz 91.4 kB With provenance Details for prolix-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for prolix 0.3.0
File Interpreter ABI Platform Actions
prolix-0.3.0-py3-none-any.whl Python 3 none any Details for prolix-0.3.0-py3-none-any.whl

Total release size: 153.6 kB

Release files / prolix-0.3.0.tar.gz

Download URL prolix-0.3.0.tar.gz
Size 91.4 kB
Tags Source
SHA-256 checksum
How to use checksums
a8b66b6b638e2061fa704db0102e63f65c5294aa5dbb4ff799fa7b64cc7f210d
BLAKE2b-256 checksum
How to use checksums
a5eab19fcfd024b0a3744e42afea9993c1d9da3c3b18c34949930101a21be624
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / prolix-0.3.0-py3-none-any.whl

Download URL prolix-0.3.0-py3-none-any.whl
Size 62.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ce8ae8828e0e1eb2f42fc53d11fb9c3bf02b106c0bf21da89bc7f91438f2c7cc
BLAKE2b-256 checksum
How to use checksums
b01e0603bbe0e6db918336fa6d62aa52cfcc899efa031c12087a9ae4bc20cab9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Security for prolix 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for prolix-0.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.3.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page