contentintelpy 0.2.0

Production-grade NLP library for unified content intelligence.

contentintelpy

Production-grade NLP library for unified content intelligence.

contentintelpy provides a unified, DAG-based engine for multilingual sentiment analysis, NER, translation, and summarization using real transformer models (RoBERTa, GLiNER, NLLB).

Features

• Real Models: No heuristics. Uses State-of-the-Art Transformers.
  • Sentiment: RoBERTa
  • NER: GLiNER
  • Translation: NLLB (GPU) + ArgosTranslate (Offline CPU)
• Hybrid Execution: Models download on first run (lazy-loaded). Offline fallback available.
• Deterministic Pipelines: DAG-based execution guarantees order.
• Dual API:
  • Pipeline-first for complex workflows.
  • Service-first for quick scripts.
• Production Ready: Thread-safe, standard error handling, sparse outputs.

Installation

Install the base library:

pip install contentintelpy

🧠 Capability Extras (Recommended)

contentintelpy uses optional "extras" to keep the base installation lightweight. Depending on which features you need, use the following commands:

Feature	Target Extras	Install Command
All Features	core,ner,translation,summarization	pip install "contentintelpy[core,ner,translation,summarization]"
Search & Keywords	core	pip install "contentintelpy[core]"
Entity Extraction	ner	pip install "contentintelpy[ner]"
Translation	translation	pip install "contentintelpy[translation]"
Summarization	summarization	pip install "contentintelpy[summarization]"

[!TIP] Minimal Install: If you only need language detection and simple text processing, you only need pip install contentintelpy.

[!IMPORTANT] GPU Support: If you have an NVIDIA GPU, installing torch manually with CUDA support before installing the extras will significantly speed up Translation and Classification.

[!IMPORTANT] spaCy Model Requirement If you use NER or language features, you must install a spaCy model manually:

python -m spacy download en_core_web_sm

---

Quick Start

Ideal for simple tasks in notebooks or scripts.

from contentintelpy import SentimentService, TranslationService

# Sentiment
service = SentimentService()
result = service.analyze("This library is amazing!")
print(result) 
# {'value': 'positive', 'confidence': 0.99, ...}

# Translation
translator = TranslationService()
text = translator.translate("Hola mundo", target="en")
print(text)
# "Hello world"

Production Usage (Pipeline-First)

Recommended for Backends, APIs, and Data Pipelines.

import contentintelpy as ci

# 1. Create the canonical pipeline
pipeline = ci.create_default_pipeline()

# 2. Run it (Thread-safe)
result = pipeline.run({
    "text": "गूगल ने बेंगलुरु में नया कार्यालय खोला"
})

# 3. Access Sparse Output
print(result)

Output Example:

{
  "text": "...",
  "text_translated": "Google opened a new office in Bengaluru",
  "language": "hi",
  "entities": [
    {"text": "Google", "label": "ORG"},
    {"text": "Bengaluru", "label": "LOC"}
  ],
  "sentiment": {
    "value": "neutral",
    "value_en": "neutral",
    "confidence": 0.95
  },
  "summary": "..."
}

Advanced Usage: Custom Pipelines

You are not limited to the default pipeline. You can mix and match specific nodes to create a leaner, faster pipeline tailored to your needs.

1. Build a Custom Pipeline

Import individual nodes and pass them to the Pipeline constructor. The order matters!

from contentintelpy import Pipeline, LanguageDetectionNode, SentimentNode

# A lightweight pipeline that only does Language Detection + Sentiment
# Note: Sentiment often requires translation first if content isn't English, 
# but here we assume English input for speed.
custom_pipeline = Pipeline([
    LanguageDetectionNode(),
    SentimentNode()
])

result = custom_pipeline.run({
    "text": "This is a custom workflow!"
})
print(result)

2. Create Your Own Nodes

You can easily extend the library by creating your own nodes. Inherit from Node and implement process().

from contentintelpy import Node

class ProfanityCheckNode(Node):
    def __init__(self):
        super().__init__("ProfanityCheckNode")
    
    def process(self, context):
        text = context.get("text", "").lower()
        if "badword" in text:
            context.add_error("ProfanityCheckNode", "Content flagged as unsafe.")
        return context

# Add it to a pipeline
pipeline = Pipeline([
    ProfanityCheckNode(),
    SentimentNode()
])

Error Handling

Nodes never crash the pipeline. Errors are collected in errors dict.

{
    "text": "...",
    "errors": {
        "TranslationNode": "Model download failed: Connection error"
    }
}

Architecture

This library is pure logic. It does NOT contain:

• Flask / FastAPI routes
• Database models
• Authentication

It is designed to be consumed by your backend application.