granule 0.1.11

Granularity-on-demand learning object extractor and composer

Granule (v0.1.1)

Granule ingests blogs/HN/Reddit/YouTube/podcasts/news, atomizes them into SemanticAtoms with citations, and composes MicroLearning/FocusedSessions/DeepMastery units by target duration.

Install

pip install -e .

Minimal Video Insight Card Install

If you only need to generate structured VideoInsightCardPayload objects (YouTube transcript + OpenAI LLM), install the lightweight extra:

pip install granule[videocard]

This pulls only:

• pydantic (core models)
• youtube-transcript-api (transcript fetch)
• openai (LLM generation)
• python-dotenv (optional .env loading)

Example:

from granule.api_simple import video_insight_card
from dotenv import load_dotenv
load_dotenv()  # reads OPENAI_API_KEY
card = video_insight_card("https://www.youtube.com/watch?v=mAClw7r3ETc")
print(card.model_dump_json(indent=2))

If OPENAI_API_KEY is unset, you'll still get a minimal fallback card (header + short TL;DR snippet if transcript exists).

Full Feature Install

For CLI, article ingestion, ML metrics, vector store, API, UI, and advanced agent support:

pip install granule[full]

CLI

granule ingest "Some article text" --kind blog -o doc.json
granule dissect doc.json --max-tokens 60 --only definition,example -o atoms.json
granule expand atoms.json --target 55s -o micro.json
granule stream atoms.json --pace 55s --until 3m -o session.jsonl
granule simple-youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ -o simple.json
granule simple-card https://www.youtube.com/watch?v=dQw4w9WgXcQ -o card.json
granule video-card https://www.youtube.com/watch?v=dQw4w9WgXcQ -o video_card.json
granule video-card https://www.youtube.com/watch?v=dQw4w9WgXcQ --title "Custom Title" -o titled_card.json
granule text-video-card transcript.txt -o text_card.json
granule text-video-card "[00:00] Intro\n[00:05] Point A" --title "Inline Snippet" -o inline_card.json

FastAPI

uvicorn granule.fastapi_app:app --reload --port 8000

Endpoints:

• POST /ingest {source, kind}
• POST /dissect {doc, max_tokens, kinds}
• POST /expand {graph, target_seconds}

YouTube transcript

from granule.ingest.youtube import ingest_youtube
doc = ingest_youtube("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
print(doc.text[:200])

Environment Variables

Granule reads optional environment variables for LLM integration.

1. Copy .env.example to .env (or export variables another way).
2. Add your OpenAI key if you want LLM-powered features (planned / experimental):

cp .env.example .env
echo "OPENAI_API_KEY=sk-..." >> .env

Variables:

• OPENAI_API_KEY – enables future atom enrichment / generation via OpenAI.

• GRANULE_LLM_MODEL – preferred model (e.g. gpt5).

• GRANULE_LLM_PROVIDER – provider alias (openai, azure-openai, etc.).

• GRANULE_SUPPRESS_PROBLEM_QUESTIONS: if set to 1/true/on, skips adding ontology-derived and synthesized integrative Questions Raised (useful when you only want claims & problems without extra exploratory questions).

If python-dotenv is installed, .env will be auto-loaded; otherwise export vars normally.

Streamlit UI (YouTube → Atoms → Unit)

Experimental helper UI.

Install extras:

pip install -e .[ui,llm]

Run:

streamlit run streamlit_app.py

Paste a YouTube URL, adjust parameters, run the pipeline, optionally enrich first atoms (needs OPENAI_API_KEY).

New: Segments, Summaries & Insight Card

The pipeline now also produces:

• segments – improved sentence-cluster segments with token counts.
• segment_summaries – per-segment micro summaries (LLM-backed if key present, heuristic fallback otherwise).
• insight_card – a heuristic high-level Transcript Insight Card (claims, glossary, metrics stubs) serialized for UI consumption.

These appear in the composite JSON returned by process_youtube and in the Streamlit UI (preview table + card header/sections).

Video Insight Card (Structured JSON)

Granule can produce a richer, schema-driven VideoInsightCardPayload either from a YouTube URL (auto transcript + optional title fetch) or any raw transcript text file.

Quickstart:

# (Install)  pip install granule[videocard]
# YouTube → structured video insight card (minimal fallback if no OPENAI_API_KEY set)
granule video-card https://www.youtube.com/watch?v=mAClw7r3ETc -o video_card.json

# Override title (skip auto fetched oEmbed title)
granule video-card https://www.youtube.com/watch?v=dQw4w9WgXcQ --title "My Custom Title" -o custom.json

# Raw transcript file → card
granule text-video-card path/to/transcript.txt --title "Workshop Transcript" -o workshop_card.json

# Inline raw text (small snippets) → card
granule text-video-card "[00:00] Intro to X\n[00:10] Key idea" --title "Snippet" -o snippet_card.json

If you provide an OPENAI_API_KEY, the card is generated via the OpenAI Responses API with strict schema parsing; otherwise a minimal fallback card (header + TL;DR snippet) is produced.

Schema highlights:

• Header (title, subtitle, badges)
• Video meta (url, id)
• Sections (TL;DR, Claims & Evidence, Glossary, Rhetoric, Misconceptions, Questions, Segments, Metrics, etc. – only those with content appear)
• Footer (persuasion modes, devices, timeline events)

Use cases:

• Fast analysis / structuring of transcripts for research
• Feeding downstream UI components or analytics pipelines
• Offline transcript auditing (supply a .txt file without hitting YouTube)

Tip: Pair with simple-card for a lighter heuristic card, or video-card for the full structured extraction.

Extras Overview

Extra	Purpose
videocard	Minimal OpenAI-powered video insight card generation
cli	Typer/Rich command line UX
article	Blog/article/markdown parsing & readability
analysis	Optional numeric/text metrics (numpy, scikit-learn)
vector	Chroma vector store integration
web	FastAPI + Uvicorn API server
ui	Streamlit prototype UI
advanced-llm	pydantic-ai agent experimentation
full	Aggregate of all feature extras

Releasing / Publishing

Helper script publish.sh automates version bump, build, and upload.

Examples:

# Bump patch version, build, upload to PyPI
./publish.sh patch

# Bump minor and upload to TestPyPI
./publish.sh minor --test

# Build only (no version change, no upload)
./publish.sh same --no-upload

# Dry run (show actions without changing files)
./publish.sh patch --dry-run

Set PYPI_TOKEN env var for non-interactive upload (token from PyPI account settings).