fin-ai-stats-extract

Overview

fin-ai-stats-extract detects and classifies AI-related discussion in XML earnings-call transcripts and writes the results to CSV.

The tool implements a five-stage methodology via LLM extraction:

1. AI Mention Detection — keyword-based yes/no decision and hit count
2. Representative Sentence Selection — up to 3 most keyword-dense sentences
3. Attitude Classification — excited / concerned / neutral tone label
4. Initiator Attribution — who raised AI first (management, analyst, both, unclear)
5. Confidence Classification — hopeful / confident / transformational / authoritative

The tool is driven by a single TOML config file, config.toml, which defines:

• the base extraction instructions sent to the model
• model and endpoint settings
• the ordered output field list

That same config drives three things at once:

• the final system instructions sent to the model
• the structured JSON schema used for Responses API parsing
• the CSV column order used for output

CLI flags remain available for fast tuning and one-off overrides. When a CLI flag conflicts with a TOML value, the CLI value wins and the tool logs a warning.

Requirements

• Python 3.14+
• uv

Install

For local development in this repository:

uv sync

For one-off usage without creating a local environment:

uvx fin-ai-stats-extract

Configuration

The default config file is ./config.toml.

If you run the tool without --config and config.toml does not exist in the current working directory, the packaged default config is copied there automatically.

The default config includes:

• commented-out optional settings such as temperature, top_p, and reasoning_effort
• the full methodology for AI mention detection, attitude classification, initiator attribution, and confidence classification

Edit config.toml directly to change AI instructions, AI model settings, or output fields.

The output contract now uses a flat ordered list:

[output]
format = [
  { name = "ai_mentioned", description = "Whether at least one core AI keyword appears" },
  { name = "keyword_hit_count", description = "Total count of AI keyword matches" },
]

Descriptions are passed through directly into the rendered LLM instructions. The program does not parse or validate description text beyond requiring it to be non-empty.

Environment

Create an environment file for API-backed runs:

cp .env.example .env

Example values:

OPENAI_API_KEY=sk-your-key-here

By default, config.toml reads the API key from OPENAI_API_KEY via api_key_env = "OPENAI_API_KEY".

Basic Usage

Run using the defaults in config.toml:

uv run fin-ai-stats-extract --input ./data/Current

Run on a single transcript:

uv run fin-ai-stats-extract --input ./data/Current/11473715_T.xml

Write to a different CSV for one run:

uv run fin-ai-stats-extract --output ./sample.csv

Validate XML parsing only, without calling a model:

uv run fin-ai-stats-extract --dry-run

Process only a sample of files:

uv run fin-ai-stats-extract --sample 25

Enable verbose logging:

uv run fin-ai-stats-extract --verbose

Tune common model settings for one run:

uv run fin-ai-stats-extract \
  --temperature 0.2 \
  --top-p 0.9 \
  --max-output-tokens 2500 \
  --reasoning-effort medium \
  --verbosity low

Resume an interrupted run from an existing CSV:

uv run fin-ai-stats-extract --resume

Use a different config file:

uv run fin-ai-stats-extract --config ./custom_config.toml

Open the config editor window instead of supplying every setting on the command line:

uv run fin-ai-stats-extract --gui

The GUI is a pywebview window whose UI is built with React, TypeScript, shadcn/ui, and Tailwind. Its job is to edit the TOML config file live: as you change the instructions, model settings, or output format, each edit is written straight back to the config file (comments and formatting are preserved). Runtime-only options (input/output paths, sample size, concurrency, dry-run/resume/verbose, and a one-off API key) are collected in the Run tab and are not written to the config file. Pressing Run closes the window and executes the pipeline in your terminal, so the progress bar and any interactive prompts behave exactly as a normal run.

Because the GUI edits the file in place, you can point it at any config to tweak it:

uv run fin-ai-stats-extract --config ./custom_config.toml --gui

GUI edits vs. CLI overrides

CLI override flags are applied after the GUI closes, so they always win for the run without being saved to the file. For example:

uv run fin-ai-stats-extract --gui --temperature 0.1

If you change the temperature to 1 in the GUI, the config file is updated to 1, but the run still uses 0.1 (the CLI value). The precedence is: load TOML → apply GUI edits to the TOML file → apply CLI overrides on top at runtime.

Local OpenAI-Compatible Endpoints

You can point the tool at a local or self-hosted OpenAI-compatible server with either:

• base_url in config.toml
• --base-url for a one-off override

Example with LM Studio:

uv run fin-ai-stats-extract \
  --base-url http://127.0.0.1:1234/v1 \
  --model google/gemma-3-4b

You can also pass an API key explicitly:

uv run fin-ai-stats-extract --api-key "$OPENAI_API_KEY"

If base_url is set and no API key is available, the tool uses lm-studio as a fallback key for local endpoints that require a non-empty value.

Command-Line Arguments

• --config: Path to a TOML config file. Defaults to ./config.toml.
• --gui: Open the config-editor window to edit the TOML file live and launch the run. Edits are saved to the config file; CLI override flags still win at runtime without being persisted.
• --input: Required unless --gui is used. Path to one XML file or a folder tree containing XML files.
• --output: Output CSV path. Defaults to output.csv.
• --model: Override the configured model.
• --base-url: Override the configured OpenAI-compatible API base URL.
• --api-key: Override the API key from the configured environment variable.
• --temperature: Override the configured Responses API temperature from 0 to 2.
• --top-p: Override the configured nucleus sampling mass from 0 to 1.
• --max-output-tokens: Override the configured maximum output tokens.
• --reasoning-effort: Override the configured reasoning effort: none, minimal, low, medium, high, xhigh.
• --verbosity: Override the configured output verbosity: low, medium, or high.
• --max-concurrency, --max-async-jobs, --concurrency: Maximum number of concurrent extraction jobs. Defaults to CONCURRENCY_LIMIT or 100.
• --dry-run: Parse XML only. No API calls are made.
• --resume: Resume from an existing output CSV.
• --sample: Randomly process only N files from the input set.
• --verbose: Enable debug logging.
• --yes, -y: Skip cost confirmation.

The most common researcher-facing controls remain temperature, top_p, max_output_tokens, reasoning_effort, and verbosity. In most cases, change temperature or top_p, but not both at the same time.

Output Schema

The [output] section of config.toml is the extraction contract.

Each item in format defines:

• the exact field name
• the output order used in both the JSON contract and CSV columns
• a freeform description that is copied directly into the model instructions

The tool uses that same schema to:

• build the structured output model at runtime
• append an Output Contract section to the final instructions sent to the LLM
• generate CSV headers in the same order

Output

The tool writes one CSV row per transcript.

The CSV always begins with transcript metadata:

• event_id
• company_name
• quarter
• date
• headline
• source_file

Configured extraction fields follow in the order defined in config.toml.

List-valued fields are serialized using a semicolon-space separator.

When the input is a folder, source_file preserves the relative subfolder path, for example subdir/12345_T.xml.

Notes On Local Models

OpenAI-compatible endpoints vary in how well they support strict structured outputs.

In particular:

• some local models may fail on long transcripts because of context-window limits
• some local models may return non-compliant JSON even when a schema is provided

If you use a local endpoint and see parsing or validation failures, try:

• a model with a larger context window
• smaller inputs using --sample or a single file first
• an OpenAI-hosted model for the most reliable structured-output behavior

Developing the GUI

The GUI front-end lives in frontend/ (Vite + React + TypeScript + Tailwind + shadcn/ui). Building it emits static assets into src/fin_ai_stats_extract/resources/webui/, which ship inside the wheel so end users never need Node installed to run --gui.

These built assets are generated artifacts and are git-ignored, not committed. The release workflow (.github/workflows/release.yml) runs pnpm build before uv build, so published wheels always include an up-to-date UI. If you build or install from a source checkout yourself, run pnpm build first — otherwise --gui will report that the assets are missing.

Rebuild the UI after changing anything under frontend/src:

cd frontend
pnpm install
pnpm build   # outputs to ../src/fin_ai_stats_extract/resources/webui

For a fast edit/refresh loop, run the Vite dev server and point the GUI at it:

# terminal 1
cd frontend && pnpm dev

# terminal 2
FIN_AI_GUI_DEV_URL=http://localhost:5173 uv run fin-ai-stats-extract --gui

When FIN_AI_GUI_DEV_URL is unset, the GUI loads the built assets from resources/webui. Run pnpm build before packaging locally; on release, CI builds them for you.