indicflow 1.0.0

Analyse documents in Indian and Urdu languages using Gemini AI

📄 indicflow

Analyse documents in Indian and Urdu languages — extract, translate, and understand using Gemini AI

---

Overview

indicflow is a single command-line tool with two modes:

--mode pipeline — Fast batch processing. Translates pages and flags keyword-relevant sections. Good for processing large folders of documents quickly.

--mode deep — In-depth single document analysis. Builds a word frequency table, enriches it with English translations, auto-selects the most meaningful nouns as keywords, and builds rich per-keyword context for every page those words appear on. Good for understanding an unfamiliar document or detecting bias and topic patterns.

Both modes support any document format and any of the supported languages.

---

Supported Languages

Language	Script	Notes
Hindi	Devanagari
Tamil	Tamil
Telugu	Telugu
Kannada	Kannada
Malayalam	Malayalam
Bengali	Bengali
Marathi	Devanagari
Urdu	Nastaliq (Arabic)	RTL script — handled automatically

---

Supported Document Formats

Format	Extension	Notes
PDF	.pdf	Text-based or scanned — auto-detected
Word	.docx	Requires python-docx
PowerPoint	.pptx	One page per slide. Requires python-pptx
Plain text	.txt .md
HTML	.html .htm	Tags stripped automatically
CSV	.csv	Rows treated as plain text
Images	.jpg .jpeg .png .webp .tiff	Gemini Vision reads directly

Scanned PDFs and image files with no selectable text are automatically routed to Gemini Vision — no extra OCR setup needed.

---

Project Structure

indicflow/                     ← root of the GitHub repo
├── pyproject.toml             ← package metadata and dependencies
├── README.md
├── CHANGELOG.md               ← version history
├── CONTRIBUTING.md            ← how to contribute
├── PUBLISHING.md              ← release checklist
├── LICENSE
├── .gitignore
├── run_tests.sh               ← integration test suite (requires API key)
├── tests/                     ← unit tests (no API key needed)
│   ├── conftest.py
│   ├── helpers.py
│   ├── test_extractor.py
│   ├── test_gemini_client.py
│   ├── test_cli.py
│   ├── test_output_schema.py
│   └── fixtures/
│       └── test_outputs/      ← frozen JSON fixtures for schema tests
└── src/
    └── indicflow/             ← the installed Python package
        ├── __init__.py
        ├── __main__.py        ← enables: python -m indicflow
        ├── cli.py             ← entry point for both modes
        ├── extractor.py       ← universal document loader
        ├── gemini_client.py   ← all Gemini API calls
        └── models.py          ← Pydantic output schemas

---

Setup

Install from PyPI

# Core — PDF, TXT, MD, HTML, CSV, images
pip install indicflow

# Everything — adds Word, PowerPoint, and Urdu RTL support
pip install indicflow[all]

# Individual optional extras
pip install indicflow[docx]   # .docx support
pip install indicflow[pptx]   # .pptx support
pip install indicflow[html]   # better HTML extraction
pip install indicflow[urdu]   # Urdu RTL reshaping

Add your API key

Create a .env file in whatever folder you will run indicflow from:

GEMINI_API_KEY=your_key_here

No quotes around the value. Both GEMINI_API_KEY and GOOGLE_API_KEY are accepted. Get a free key at aistudio.google.com/app/apikey

Verify it works

indicflow --help

For development (clone and edit)

git clone https://github.com/kraghavan/indicflow
cd indicflow
pip install -e ".[all]"

---

All Flags

Shared (both modes):
  --input           Path to a file or folder (required)
  --language        Document language e.g. Hindi, Tamil, Urdu (required)
  --mode            pipeline or deep (default: deep)
  --translate       Include English translation per page / snippet
  --pages N M       Page range e.g. --pages 1 10 (PDF and PPTX only)
  --output          Output filename (default: pipeline_output.json / indicflow_output.json / *.csv)
  --output-format   json (default) or csv

Pipeline mode only:
  --keywords        Comma-separated words to scan for (English or native or mixed)
  --threshold       Relevance cutoff 0.0–1.0 (default: 0.15)

Deep mode only:
  --top-x           How many top nouns to use as auto-keywords (default: 10)
  --freq-min        Minimum word frequency to include in table (default: 2)
  --freq-top-n      Rows to show in printed frequency table (default: 50)
  --keyword-source  auto (default) or manual
  --keywords        Used when --keyword-source manual (comma-separated)
  --context-chars   Surrounding text per keyword occurrence (default: 400)

Model override

By default indicflow uses gemini-3-flash-preview. To pin to a specific model, add to your .env:

INDICFLOW_MODEL=gemini-2.0-flash

Useful if the default model is experiencing high demand or you want to control costs.

---

Mode 1 — Pipeline: Examples

The simplest case — translate a single file

indicflow \
  --input court_notice.pdf \
  --language Tamil \
  --mode pipeline \
  --translate

python -m indicflow works identically if you prefer that form.

---

Translate a folder of documents in one run

indicflow \
  --input ./patient_records/ \
  --language Malayalam \
  --mode pipeline \
  --translate \
  --output translated_records.json

---

Scan a document for keywords — same language

indicflow \
  --input property_dispute.pdf \
  --language Hindi \
  --mode pipeline \
  --keywords "जमीन,मालिक,न्याय,किरायेदार,अदालत" \
  --threshold 0.15 \
  --output dispute_scan.json

The --threshold controls sensitivity. 0.05 casts a wide net and flags more pages. 0.30 only flags pages with dense keyword matches.

---

Scan using English keywords on a native-language document

indicflow \
  --input land_records.pdf \
  --language Hindi \
  --mode pipeline \
  --keywords "landlord,tenant,eviction,court order,ownership" \
  --threshold 0.20 \
  --output land_scan.json

---

Mixed English and native keywords in one list

indicflow \
  --input case_file.pdf \
  --language Urdu \
  --mode pipeline \
  --translate \
  --keywords "justice,انصاف,landlord,زمیندار,کسان,evidence" \
  --threshold 0.15 \
  --output case_analysis.json

---

Translate and keyword-scan a specific page range

indicflow \
  --input government_report.pdf \
  --language Telugu \
  --mode pipeline \
  --translate \
  --pages 10 25 \
  --keywords "budget,allocation,expenditure" \
  --output report_section.json

---

Process a scanned Urdu document

indicflow \
  --input scanned_urdu_contract.pdf \
  --language Urdu \
  --mode pipeline \
  --translate \
  --keywords "معاہدہ,شرائط,ادائیگی" \
  --output contract_analysis.json

---

Analyse a Word document

indicflow \
  --input minutes_of_meeting.docx \
  --language Hindi \
  --mode pipeline \
  --translate \
  --keywords "प्रस्ताव,निर्णय,बजट" \
  --output meeting_analysis.json

---

Analyse an image or scanned photo

indicflow \
  --input letter_photo.jpg \
  --language Tamil \
  --mode pipeline \
  --translate \
  --output letter_analysis.json

---

Pipeline output structure

{
  "file": "property_dispute.pdf",
  "file_type": "pdf",
  "language": "Hindi",
  "pages_analysed": "1–12",
  "document_analysis": {
    "doc_type": "Legal Notice",
    "overall_summary": "A notice filed at the district court regarding a property ownership dispute.",
    "key_entities": ["Ram Kumar", "District Court Jodhpur", "Plot No. 44B"],
    "main_topics": ["property dispute", "land ownership", "eviction", "tenant rights"]
  },
  "keyword_summary": {
    "keywords": ["landlord", "tenant", "eviction", "court order"],
    "flagged_pages": [3, 5, 8, 11],
    "threshold": 0.20
  },
  "pages": [
    {
      "page": 3,
      "original_text": "जमींदार ने किरायेदार को...",
      "translated_text": "The landlord issued a notice to the tenant demanding...",
      "page_summary": "Formal eviction notice citing non-payment of rent.",
      "key_points": ["Rent unpaid since April 2023", "Court hearing scheduled for 12 March"],
      "entities": ["Ram Kumar", "12 March 2024", "Plot No. 44B"],
      "keyword_result": {
        "semantic_score": 0.94,
        "matched_keywords": ["landlord → जमींदार", "eviction → बेदखली"],
        "context_note": "Page contains the core eviction demand with specific legal language.",
        "flagged": true
      }
    }
  ]
}

---

Pipeline CSV output

indicflow \
  --input property_dispute.pdf \
  --language Hindi \
  --mode pipeline \
  --translate \
  --keywords "landlord,tenant,eviction" \
  --output-format csv \
  --output dispute_results.csv

Column	Description
file	Source filename
language	Document language
doc_type	Detected document type
overall_summary	Document-level summary
main_topics	Pipe-separated list of topics
page	Page number
original_text	First 200 chars of original text
translated_text	Full English translation (if --translate)
page_summary	1–2 sentence page summary
key_points	Pipe-separated key points
entities	Pipe-separated names, dates, places
kw_flagged	true / false
kw_semantic_score	0.0–1.0 relevance score
kw_matched	Pipe-separated matched keywords
kw_context_note	Why this page was flagged
kw_mode	cross-language / same-script

Written as UTF-8 with BOM so Excel opens Devanagari, Nastaliq, and Tamil script correctly.

---

Mode 2 — Deep: Examples

The simplest case — auto top 10 keywords with translations

indicflow \
  --input folk_tale.pdf \
  --language Hindi \
  --mode deep

The frequency table is printed in the terminal with English translations alongside each word.

---

See more keyword candidates before committing

# Step 1: run with top-x 25 to see what's available
indicflow \
  --input report.pdf \
  --language Urdu \
  --mode deep \
  --top-x 25 \
  --freq-top-n 25

# Step 2: pick from the table and run with manual keywords
indicflow \
  --input report.pdf \
  --language Urdu \
  --mode deep \
  --keyword-source manual \
  --keywords "کسان,زمیندار,انصاف,موہر,پنچایت"

---

Control the frequency table display

indicflow \
  --input annual_report.pdf \
  --language Tamil \
  --mode deep \
  --freq-top-n 75 \
  --freq-min 3 \
  --top-x 12

---

Manual keywords — native script only

indicflow \
  --input court_record.pdf \
  --language Hindi \
  --mode deep \
  --keyword-source manual \
  --keywords "किसान,जमींदार,न्याय,कलश,पंचायत,राजा,लगान"

---

Manual keywords — English only on a native document

indicflow \
  --input case_file.pdf \
  --language Telugu \
  --mode deep \
  --keyword-source manual \
  --keywords "farmer,landlord,justice,ownership,dispute,village council"

---

Manual keywords — mixed English and native

indicflow \
  --input land_case.pdf \
  --language Urdu \
  --mode deep \
  --keyword-source manual \
  --keywords "justice,انصاف,landlord,زمیندار,کسان,power,ظلم,evidence"

---

Include English translation of every snippet

indicflow \
  --input legal_brief.pdf \
  --language Hindi \
  --mode deep \
  --translate \
  --top-x 10 \
  --output brief_deep.json

---

Wider context window per occurrence

indicflow \
  --input dense_report.pdf \
  --language Bengali \
  --mode deep \
  --context-chars 700 \
  --top-x 8 \
  --translate

---

Analyse a specific section of a large document

indicflow \
  --input government_inquiry.pdf \
  --language Hindi \
  --mode deep \
  --pages 40 60 \
  --top-x 15 \
  --translate \
  --output inquiry_section.json

---

Analyse a PowerPoint presentation

indicflow \
  --input budget_presentation.pptx \
  --language Telugu \
  --mode deep \
  --translate \
  --top-x 10 \
  --output budget_analysis.json

---

Analyse a Word document

indicflow \
  --input medical_report.docx \
  --language Malayalam \
  --mode deep \
  --keyword-source manual \
  --keywords "diagnosis,treatment,patient,medication,hospital" \
  --translate \
  --output medical_deep.json

---

Analyse a scanned image

indicflow \
  --input handwritten_letter.jpg \
  --language Urdu \
  --mode deep \
  --translate \
  --top-x 8 \
  --output letter_deep.json

---

Deep mode output structure

{
  "file": "folk_tale.pdf",
  "file_type": "pdf",
  "language": "Urdu",
  "pages_analysed": "1–5",
  "keyword_source": "auto",
  "keywords_used": ["کسان", "زمیندار", "انصاف", "موہر", "پنچایت"],
  "document_analysis": {
    "doc_type": "Folk Tale — Rajasthani",
    "overall_summary": "A Rajasthani folk tale about an honest farmer who discovers seven pots of gold coins buried in his landlord's field.",
    "key_entities": ["Vijaydan Detha", "NCERT", "Rajasthan"],
    "main_topics": ["land ownership", "moral integrity", "power and justice"]
  },
  "word_frequency": [
    {"rank": 1, "word": "کسان", "frequency": 18}
  ],
  "enriched_frequency": [
    {"word": "کسان", "frequency": 18, "english": "farmer", "pos": "noun"}
  ],
  "keyword_contexts": [
    {
      "keyword": "کسان",
      "total_pages_found": 3,
      "total_occurrences": 18,
      "page_contexts": [
        {
          "page": 1,
          "occurrence_count": 6,
          "translated_snippets": ["The farmer had made many requests not to pay rent."],
          "how_keyword_is_used": "Farmer positioned as moral centre but structurally powerless.",
          "sentiment": "mixed",
          "entities_nearby": ["landlord", "gold coins", "village council"],
          "topic_in_context": "Power imbalance between tenant farmer and landowner",
          "bias_indicator": "Farmer consistently described in subordinate terms.",
          "notable_quote": "کسان نے لگان نہ دینے کی خاطر بہت منتیں کیں"
        }
      ]
    }
  ]
}

---

Deep CSV output

indicflow \
  --input folk_tale.pdf \
  --language Urdu \
  --mode deep \
  --top-x 10 \
  --translate \
  --output-format csv \
  --output folk_tale_analysis.csv

Column	Description
file	Source filename
keyword	The keyword in original script
english	English translation of the keyword
pos	Part of speech (noun / proper_noun / adjective / verb)
frequency	Total occurrences in the document
page	Page number of this occurrence
occurrence_count	Times the keyword appears on this page
how_keyword_is_used	Gemini's analysis of the word's role
sentiment	positive / negative / neutral / mixed
topic_in_context	Main theme of this passage
bias_indicator	Any framing or power imbalance detected, or "none"
notable_quote	Most interesting sentence containing this keyword
entities_nearby	Pipe-separated names, places, concepts nearby
translated_snippets	Pipe-separated English translations of surrounding text

Multi-value fields use | as the separator so they stay in a single cell.

---

Cross-Language Keyword Scanning

Both modes support mixing English and native-script keywords freely.

--keywords "gold coins,landlord,justice,किसान,پنچایت,زمیندار"

"gold coins" → matches मोहर, موہر, सोना automatically
"landlord"   → matches जमींदार, زمیندار
"justice"    → matches न्याय, انصاف
"किसान"      → matches "farmer" and کسان semantically

---

Cost Estimate

Uses Gemini 3 Flash by default. Override via INDICFLOW_MODEL in .env if needed.

Mode	Volume	Approx. Cost
pipeline — translate only	100 pages	~$0.08
pipeline — with keywords	100 pages	~$0.10
deep — 10 auto keywords, 5 pages	per run	~$0.04
deep — enrichment + translation	per run	~$0.01

Free tier covers personal and small-scale use comfortably.

---

Troubleshooting

"API key not found" → Check .env exists in the same folder and contains GEMINI_API_KEY=your_key_here with no quotes.

503 UNAVAILABLE — model experiencing high demand → Gemini 3 Flash is new and occasionally hits capacity spikes. The tool retries automatically. If it persists, pin to a stable model: add INDICFLOW_MODEL=gemini-2.0-flash to your .env.

Scanned PDF or image shows no text → Normal — the script switches to Gemini Vision automatically. Just let it run.

Frequency table still showing prepositions after enrichment → Switch to --keyword-source manual and list only the words you want.

--top-x 10 not giving useful keywords → Run with --top-x 25 first to see more candidates, then switch to manual mode.

Keywords not matching across languages → Include at least one keyword in a different script from the document.

python-docx or python-pptx not found → pip install indicflow[docx] or pip install indicflow[pptx].

Very large documents are slow or hit rate limits → Use --pages N M to process a section at a time.

---

Built With

• PyMuPDF — PDF text extraction
• Gemini 3 Flash — translation, enrichment, context analysis, semantic scoring
• google-genai — official Gemini Python SDK
• Pydantic — output schemas
• python-docx — Word document support
• python-pptx — PowerPoint support
• arabic-reshaper + python-bidi — Urdu RTL reshaping

---

Built for analysing Indian and Urdu language documents with care for script accuracy, semantic depth, and context quality.