Swarmauri Tool · Lexical Density
A Swarmauri-compatible NLP utility that measures the lexical density of text—the ratio of content words (nouns, verbs, adjectives, adverbs) versus the total token count. Use it to monitor writing complexity, automate readability checks, or surface style signals inside agent conversations.
- Tokenizes input with NLTK and tags parts of speech to isolate lexical items.
- Returns a percentage score so changes in density are easy to compare between drafts.
- Ships as a Swarmauri tool, ready for registration inside agents or pipelines.
Requirements
- Python 3.10 – 3.13.
nltkwith thepunkt_tabandaveraged_perceptron_tagger_engresources available (downloaded at runtime).textstatfor robust lexicon counting.- Core dependencies (
swarmauri_base,swarmauri_standard,pydantic).
Installation
Pick the installer that matches your project; each command resolves the transitive requirements.
pip
pip install swarmauri_tool_lexicaldensity
Poetry
poetry add swarmauri_tool_lexicaldensity
uv
# Add to the current project and update uv.lock
uv add swarmauri_tool_lexicaldensity
# or install into the running environment without touching pyproject.toml
uv pip install swarmauri_tool_lexicaldensity
Tip: If you deploy in an offline environment, download the required NLTK models during build time (
python -m nltk.downloader punkt_tab averaged_perceptron_tagger_eng).
Quick Start
from swarmauri_tool_lexicaldensity import LexicalDensityTool
text = "This report summarizes quarterly revenue growth across all segments."
lexical_density_tool = LexicalDensityTool()
result = lexical_density_tool(text)
print(result)
# {'lexical_density': 58.333333333333336}
The tool returns a floating-point percentage. Use the same instance to score multiple passages.
Usage Scenarios
Enforce Style Guidelines in Content Pipelines
from swarmauri_tool_lexicaldensity import LexicalDensityTool
product_copy = "Introducing our new AI-powered workstation with modular expansion."
checker = LexicalDensityTool()
score = checker(product_copy)["lexical_density"]
if score < 40:
raise ValueError(f"Copy too simple (density={score:.1f}%). Add more substantive language.")
Gate marketing copy or documentation PRs based on desired complexity thresholds.
Analyze Conversations in a Swarmauri Agent
from swarmauri_core.agent.Agent import Agent
from swarmauri_standard.tools.registry import ToolRegistry
from swarmauri_tool_lexicaldensity import LexicalDensityTool
registry = ToolRegistry()
registry.register(LexicalDensityTool())
agent = Agent(tool_registry=registry)
utterance = "Could you elaborate on the architectural trade-offs in the data plane?"
result = agent.tools["LexicalDensityTool"](utterance)
print(result)
Use lexical density as a signal to adjust agent tone or escalate queries to human operators.
Batch Score Documents and Track Trends
from pathlib import Path
from swarmauri_tool_lexicaldensity import LexicalDensityTool
lexical_density = LexicalDensityTool()
corpus_dir = Path("reports/")
scores = []
for doc in corpus_dir.glob("*.txt"):
text = doc.read_text(encoding="utf-8")
scores.append((doc.name, lexical_density(text)["lexical_density"]))
for name, score in sorted(scores, key=lambda item: item[1], reverse=True):
print(f"{name}: {score:.2f}% lexical words")
Monitor writing complexity across a corpus of articles or support responses.
Troubleshooting
LookupErrorfor NLTK resources – Ensurepunkt_tabandaveraged_perceptron_tagger_engare downloaded prior to calling the tool (seenltk.download(...)).- Low density on short texts – Very short messages yield coarse percentages. Aggregate multiple utterances or relax thresholds for brief content.
- Non-English text – POS tagging models target English. Swap in language-specific models before using the tool with multilingual corpora.
License
swarmauri_tool_lexicaldensity is released under the Apache 2.0 License. See LICENSE for full details.
1 maintainer
Metadata
1 maintainer
Release files for swarmauri_tool_lexicaldensity 0.10.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| swarmauri_tool_lexicaldensity-0.10.0.tar.gz | 8.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| swarmauri_tool_lexicaldensity-0.10.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 18.0 kB
Release files / swarmauri_tool_lexicaldensity-0.10.0.tar.gz
| Download URL | swarmauri_tool_lexicaldensity-0.10.0.tar.gz |
|---|---|
| Size | 8.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4b46a076bb73d3a2855c89426d840d296b10fd1df56df5e50daf0769ec8df169
|
|
BLAKE2b-256 checksum How to use checksums |
567668c84747949c7fc2b1e2497780180212b65b2b2b85655031a909e596628b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.0 {"installer":{"name":"uv","version":"0.11.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / swarmauri_tool_lexicaldensity-0.10.0-py3-none-any.whl
| Download URL | swarmauri_tool_lexicaldensity-0.10.0-py3-none-any.whl |
|---|---|
| Size | 9.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6689e414b815ef13944dc5454a3ae32260e26c62188fcd3b556181ccce20679c
|
|
BLAKE2b-256 checksum How to use checksums |
481cd270afa57e8104ac3e51e51150d97f641025c19941588cb568e2a39c92c5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.0 {"installer":{"name":"uv","version":"0.11.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
