seec 0.0.2

Schema-driven LLM extraction pipeline for metadata, classifications, entities, and relations

SEEC

Structured Entity Extraction and Classification — a schema-driven LLM extraction pipeline. Define what to extract in a simple Markdown file, and the pipeline handles metadata, classifications, entities, and relations from any plain-text documents using a local Ollama model.

---

Quick Start

1. Install the package

pip install seec

2. Install Ollama

Download from ollama.ai, then:

ollama serve          # start the server (keep running)
ollama pull qwen3.5:9b   # pull the default model

3. Set up your project

from seec import init_project

init_project("./my_config")
# Creates: my_config/extraction_schema.md  (define what to extract)
#          my_config/prompts/              (LLM prompt templates)

Edit my_config/extraction_schema.md to define the fields, categories, entities, and relations relevant to your domain (see Configuration below).

4. Run extraction

from seec import extract, load_extractions

results = extract(
    input_dir="data/documents",       # folder with .txt or .md files
    output_dir="outputs/extractions",
    model="qwen3.5:9b",
    config_dir="my_config",           # points to your schema + prompts
)

# Load results as DataFrames
docs_df, entities_df, relations_df = load_extractions("outputs/extractions")

Results are saved as one JSON file per document. Re-runs skip passes whose schema section hasn't changed.

---

Configuration {#configuration}

Schema file

The extraction schema (extraction_schema.md) has 4 sections, each a Markdown table:

Section	What it extracts	Example
1. Metadata	Fields from the document header	title, author, year
2. Classifications	Document-level labels	study type, language
3. Entities	Named things in the text	tools, organizations, methods
4. Relations	Directed links between entities	tool used_for task

Each section maps to one extraction pass. Leave a section empty (keep the header, delete data rows) to skip that pass entirely.

Copy-paste examples for each section:

# Metadata
| title | str | Full document title |

# Classifications — plain enum
| language | enum | English; Dutch; French | Language of the document |

# Classifications — enum with descriptions
| study_type | enum | qualitative: Based on interviews; quantitative: Based on numerical data | Study methodology |

# Entities
| organization | Named company or institution | "Acme Corp" |

# Relations
| employs | organization | person | Organization employs this person |

Prompt customization

The prompts/ folder contains system and user prompt files for each pass. Edit the system prompts to adjust the persona for your domain (e.g., change "document" to "legal contract"). Edit the user prompts to change framing, but keep the template variables ({document_text}, {summary_text}, etc.) as-is.

Note: prompt edits do not invalidate the cache. Delete a document's JSON file to force re-extraction.

Model selection

Hardware	Recommended model
CPU only / 6-8 GB VRAM	qwen3.5:4b
8-12 GB VRAM	qwen3.5:9b (default)
16+ GB VRAM	qwen3.5:14b
Apple Silicon, 8-16 GB	qwen3.5:9b
Apple Silicon, 24-36 GB	qwen3.5:14b
Apple Silicon, 48+ GB	qwen3:30b

Apple Silicon uses unified memory — all system RAM is available for inference.

Not sure? Run ollama run qwen3.5:9b "Hello". If it loads, you're good. If it runs out of memory, drop to qwen3.5:4b.

---

API Reference

extract()

Main entry point. Runs the 3-pass pipeline on all .txt/.md files in a directory.

extract(
    input_dir,                          # folder with input documents
    output_dir,                         # where JSON results are saved
    model,                              # Ollama model name (e.g. "qwen3.5:9b")
    config_dir=None,                    # folder from init_project() with schema + prompts
    url="http://localhost:11434/v1",    # Ollama endpoint
    schema_path=None,                   # override: direct path to extraction_schema.md
    prompts_dir=None,                   # override: direct path to prompts/ folder
    summarize=False,                    # summarize long docs before extraction
    summarize_model="qwen3.5:4b",      # model for summarization
    temperature=0.2,                    # LLM temperature (0.0-1.0)
    seed=42,                            # random seed for reproducibility
    timeout=300.0,                      # seconds before an LLM call times out
    num_ctx=32768,                      # context window size in tokens
    think=True,                         # enable model reasoning mode
    sample_size=None,                   # process only first N files (for testing)
    verbose=True,                       # print debug info
)

Returns a dict mapping doc_id to the extraction result.

load_extractions(output_dir)

Loads all JSON results into three pandas DataFrames:

DataFrame	Columns
docs_df	doc_id + all metadata and classification fields
entities_df	doc_id, entity_id, entity_type, value, evidence, section
relations_df	doc_id, relation_type, subject_entity_id, object_entity_id

init_project(target_dir)

Copies the template schema and prompt files into the given directory. Only creates files that don't already exist (safe to re-run).

---

Technical Details

Pipeline architecture

The pipeline runs three sequential passes per document:

1. Pass 1 — Metadata: extracts structured fields from the first 1000 words of the document.
2. Pass 2 — Classifications: assigns document-level labels using the full text (or summary).
3. Pass 3 — Entities & Relations: extracts named entities with evidence quotes, plus directed relations between them.

Each pass uses Instructor to enforce a dynamically generated Pydantic model as the LLM output schema, ensuring type-safe structured extraction.

Caching

Each pass stores a SHA-256 fingerprint of its schema section in the output JSON. On re-run: - Schema unchanged — pass is skipped, cached result is reused. - Schema changed — pass re-runs and overwrites the cached value.

To clear the cache for a document, delete its JSON file. To re-run everything:

rm outputs/extractions/*.json

Summarization

For long documents (>8000 words), enable summarize=True to pre-summarize before passes 2 and 3. This keeps the input within the model's context window and can improve extraction quality. Pass 1 always runs on the raw text (first 1000 words only).

Reproducibility

Pass seed=42 (default) for deterministic output. The same seed + model + schema + document produces the same extraction. Ollama must also be configured with the same num_ctx for full reproducibility.