contentintelpy 0.5.0

Production-grade NLP library for unified content intelligence.

contentintelpy

Production-grade NLP library for unified content intelligence.

contentintelpy provides a unified, DAG-based engine for multilingual sentiment analysis, NER, translation, and summarization using real transformer models (RoBERTa, GLiNER, NLLB).

Features

• Real Models: No heuristics. Uses State-of-the-Art Transformers.
  • Sentiment: RoBERTa
  • NER: GLiNER
  • Translation: NLLB (GPU) + ArgosTranslate (Offline CPU)
• Hybrid Execution: Models download on first run (lazy-loaded). Offline fallback available.
• Deterministic Pipelines: DAG-based execution guarantees order.
• Dual API:
  • Pipeline-first for complex workflows.
  • Service-first for quick scripts.
• Production Ready: Thread-safe, standard error handling, sparse outputs.

Installation

pip install contentintelpy

This single command installs everything — torch, transformers, spaCy + its English model, GLiNER, translation, sentiment, keywords, and geocoding.

[!TIP] GPU Support: If you have an NVIDIA GPU, install torch with CUDA support before running the above for faster inference:

pip install torch --index-url https://download.pytorch.org/whl/cu118
pip install contentintelpy

---

Quick Start

Ideal for simple tasks in notebooks or scripts.

from contentintelpy import SentimentService, TranslationService

# Sentiment (Call it directly like a function!)
service = SentimentService()
result = service("This library is amazing!")
print(result) 
# {'value': 'positive', 'confidence': 0.99, ...}

# Translation
translator = TranslationService()
text = translator("Hola mundo", target="en")
print(text)
# "Hello world"

Full List of Standalone Services

Every standalone service can be executed effortlessly by calling it directly like a function.

Service Class	Callable Setup	Example Return
PreprocessingService	service(text)	"Clean text"
LanguageDetectionService	service(text)	{"language": "hi", "confidence": 0.99}
TranslationService	service(text, target)	"Translated text"
NERService	service(text, language)	[{"text": "Apple", "label": "ORG"}]
KeywordExtractionService	service(text, language)	[{"keyword": "AI", "score": 0.05}]
CategoryClassificationService	service(text, language)	{"Technology": 0.90}
SentimentService	service(text, language)	{"value": "positive", "confidence": 0.99}
SummarizationService	service(text, language)	{"summary": "Short version..."}
LocationService	service(text)	[{"city": "Paris", "country": "France"}]

Production Usage (Pipeline-First)

Recommended for Backends, APIs, and Data Pipelines.

import contentintelpy as ci

# 1. Create the canonical pipeline
pipeline = ci.create_default_pipeline()

# 2. Run it (Thread-safe)
result = pipeline.run({
    "text": "गूगल ने बेंगलुरु में नया कार्यालय खोला"
})

# 3. Access Sparse Output
print(result)

Output Example:

{
  "text": "...",
  "text_translated": "Google opened a new office in Bengaluru",
  "language": "hi",
  "entities": [
    {"text": "Google", "label": "ORG"},
    {"text": "Bengaluru", "label": "LOC"}
  ],
  "sentiment": {
    "value": "neutral",
    "value_en": "neutral",
    "confidence": 0.95
  },
  "summary": "..."
}

Advanced Usage: Custom Pipelines

You are not limited to the default pipeline. You can mix and match specific nodes to create a leaner, faster pipeline tailored to your needs.

1. Build a Custom Pipeline

Import individual nodes and pass them to the Pipeline constructor. The order matters!

from contentintelpy import Pipeline, LanguageDetectionNode, SentimentNode

# A lightweight pipeline that only does Language Detection + Sentiment
# Note: Sentiment often requires translation first if content isn't English, 
# but here we assume English input for speed.
custom_pipeline = Pipeline([
    LanguageDetectionNode(),
    SentimentNode()
])

result = custom_pipeline.run({
    "text": "This is a custom workflow!"
})
print(result)

2. Create Your Own Nodes

You can easily extend the library by creating your own nodes. Inherit from Node and implement process().

from contentintelpy import Node

class ProfanityCheckNode(Node):
    def __init__(self):
        super().__init__("ProfanityCheckNode")
    
    def process(self, context):
        text = context.get("text", "").lower()
        if "badword" in text:
            context.add_error("ProfanityCheckNode", "Content flagged as unsafe.")
        return context

# Add it to a pipeline
pipeline = Pipeline([
    ProfanityCheckNode(),
    SentimentNode()
])

Error Handling

Nodes never crash the pipeline. Errors are collected in errors dict.

{
    "text": "...",
    "errors": {
        "TranslationNode": "Model download failed: Connection error"
    }
}

Architecture

This library is pure logic. It does NOT contain:

• Flask / FastAPI routes
• Database models
• Authentication

It is designed to be consumed by your backend application.