FIXTracer

FIXTracer is a deterministic trade investigation engine for messy FIX logs. It reconstructs order lifecycles, detects anomalies, and produces local-first HTML and JSON artifacts. Current status: Public Technical Preview for FIX-literate engineers. The initial public package offers the Free product only.

Python target: 3.11 and 3.12. Installed-wheel checks cover both in CI.

Technical Preview Status

Current implementation reality:

• fixtracer extract streams one line-oriented FIX log into a direct identifier extract.
• fixtracer report generates a self-contained HTML artifact and optional JSON artifact from one or more FIX logs.
• the current default machine-facing JSON is a compact, versioned packed artifact; legacy expanded {"meta": {...}, "timelines": [...]} artifacts still render
• the current HTML experience is timeline-first
• release posture is Technical Preview, not broad commercial launch

Target product contract:

• FIXTracer converges on a case-first architecture: raw FIX logs become a portable investigation case
• that case should support a lazy viewer, timeline inspection, standalone evidence reports, investigation summary, incident cases, detailed timelines, and provenance
• HTML and JSON should become semantically aligned views or exports from the same case model, with standalone HTML preserved for sharing

Contract status: the current timeline-first ReportArtifact semantic payload is modeled and schema-exported as report_artifact.json; #227 formalizes the stable target Finding contract, #228 formalizes the stable target Ambiguity contract and first-pass packed reconstruction metadata, #239 formalizes the target InvestigationArtifact contract, #240 adds the minimal IncidentCase contract, #241 adds stable timeline_id / subject fields to OrderTimeline plus timeline-ID case references, and #242 defines nullable case_type / owner_domain enum contracts. Full runtime migration to that target envelope is deferred; --json-out now emits a compact packed report artifact for the current timeline-first payload, and render accepts both compact and legacy expanded report JSON.

The timeline reconstruction engine remains the factual backbone and trust surface of the product.

Current implementation status is tracked in docs/status.md. Current validation snapshot and exact test counts are tracked in docs/status.md.

Technical Preview planning: see docs/release_roadmap.md and docs/release_posture.md. Do not treat the current timeline-first artifact as the final incident-case product shape. Known limitations are tracked in docs/known_limitations.md, docs/status.md, and CHANGELOG.md. Pre-1.0 versioning policy is documented in CHANGELOG.md.

Data safety (Technical Preview)

FIXTracer is local-first. After installation, the documented local commands in this README (sample, extract, report, render, and version) run against local files and do not transmit FIX logs or telemetry.

FIX logs can contain sensitive client, order, symbol, routing, account, venue, and execution details. Run FIXTracer on production logs only on approved internal machines, redact data where appropriate, and do not upload production logs to hosted demos, public issues, or third-party analysis surfaces. Sample logs and sample reports use test data. Do not send or attach production logs to support or feedback during this preview. Use descriptions or synthetic reproductions.

During Technical Preview, any public website or demo surface must not accept production FIX log uploads or provide a hosted parser for real/internal logs. Use the local CLI on approved internal machines when evaluating real/internal logs. Public demos and sample reports must use bundled synthetic examples unless a future browser-local demo is explicitly reviewed and approved.

See docs/data_safety.md and docs/known_limitations.md before using the Technical Preview on internal data.

Install (Technical Preview)

Before installing or using FIXTracer, read the Free terms and the bundled LICENSE.txt. The owner-approved terms allow ongoing individual manual diagnostic use, including at work, and authorized internal coworker report sharing. Automated operational processing, team deployment, and external client deliverables are excluded. Existing Free output limits still apply. Third-party notices are in THIRD_PARTY_NOTICES.txt. FIXTracer remains proprietary and free of charge. The release bundle carries the owner-approved agreement and resolved runtime dependency notices. Publication verifies the exact accepted wheel and frozen legal files as described in the legal delivery guide.

Both files are available offline under the installed fixtracer/legal_assets/ directory. Binary archives also place them beside the executable. Reading the files does not activate Pro or create an acceptance record. The existing license.json file serves only the separate Pro activation path.

Version 0.1.0a3 includes a useful default Free sample. The recommended isolated install path for this prerelease is:

pipx install fixtracer==0.1.0a3
fixtracer --version

Equivalent tool installs are:

uv tool install fixtracer==0.1.0a3
pip install fixtracer==0.1.0a3

Installed smoke test using bundled synthetic data:

fixtracer sample --out fixtracer_sample.log
fixtracer report fixtracer_sample.log --out report.html --json-out report.json
fixtracer render report.json --out rerendered.html

The default fill_before_activation sample produces one broad lifecycle finding and four preview events in Free. Use --name simple_happy_path with sample for a clean-order summary instead.

When using 0.1.0a2, select --name fill_before_activation to generate the same sample explicitly.

Planned versioned releases will include SHA256 checksums for downloadable artifacts. See docs/release_verification.md before installing from a direct download. Checksums are provided for integrity verification; stronger artifact signing is future work.

Quickstart (dev)

# use Python 3.11
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
# optional but helpful
pre-commit install
make lint
make test
fixtracer --help
fixtracer version

Synthetic Example

The installed sample command writes a bundled synthetic FIX log or generates a report directly from one. Use synthetic data for public demos, screenshots, and sample reports; do not upload production logs to hosted demos or public feedback channels. The bundled sample data is described in fixtracer/samples/README.md. Named Technical Preview scenario inputs are documented in fixtracer/samples/preview/README.md.

fixtracer sample --report-out report.html
open report.html

Choose different filenames for each selected sample log, HTML, and JSON output. sample checks all selected destinations before writing. It refuses aliases, file/directory conflicts, and effective paths that differ only by case or canonical Unicode normalization, including parent components. Unverifiable paths also fail with exit code 2. Safety diagnostics remain visible with global --quiet. A preflight refusal preserves existing outputs and leaves missing outputs and directories absent.

With --report-out, the default log is unused unless --out or -o selects it explicitly. Thus, report-only output named fixtracer_sample.log is valid. Existing, non-conflicting outputs can still be replaced, and new parent directories can still be created. The check does not protect against concurrent filesystem changes or roll back unrelated failures after writing starts.

Generate a named scenario report with:

fixtracer sample \
  --name invalid_origclordid_reference \
  --report-out report.html \
  --json-out report.json

The generated HTML file is fully self-contained and can be viewed offline. Treat generated HTML/JSON artifacts as potentially sensitive because they may contain FIX fields, order identifiers, account-like values, venue/proprietary tags, and trading activity. Share them only through approved channels after reviewing or redacting their contents. Today it renders the current timeline-first artifact. The target direction is a stronger investigation-artifact shell over the same truth layer. Standalone HTML remains an important derived evidence report/export from the future case model.

Direct Incident Extraction

The approved v0.1.0a2 large-log workflow is:

large line-oriented FIX log
  -> direct identifier extract
  -> FIXTracer report
  -> report.html

The report is self-contained. Version 0.1.0a3 preserves the extraction contract introduced in 0.1.0a2.

Use one or more exact identifiers. Multiple selectors use OR behavior.

fixtracer extract production.log --clordid ABC123 --out incident.log
fixtracer extract production.log --orderid VENUE123 --out incident.log
fixtracer extract production.log --execid EXEC123 --out incident.log
fixtracer report incident.log --out incident.html

--clordid matches 11 (ClOrdID) and 41 (OrigClOrdID). --orderid matches 37 (OrderID). --execid matches 17 (ExecID). Each selector is repeatable.

The input contract is intentionally narrow:

• Each physical line contains zero or one complete FIX message.
• A supported message starts with a supported 8 (BeginString) value after an optional host-log prefix.
• The optional prefix can contain numeric host metadata. It ends immediately before the first supported BeginString candidate.
• One physical line contains the complete message and no second FIX message.
• The delimiter immediately after BeginString selects SOH, literal ^A, pipe, or one ASCII whitespace byte.
• Only the selected delimiter separates fields. Other delimiter-like text can occur in field values.
• The message contains 8, 9 (BodyLength), and 35 (MsgType).
• The final field is 10 (CheckSum), and the selected delimiter follows its value.
• Only the physical line ending can follow the final delimiter.
• The physical-line content is at most 1 MiB, excluding its line ending.
• FIXTracer does not verify numerical BodyLength or CheckSum values during extraction.

A line without a supported BeginString candidate is noise. The command ignores noise. It counts malformed candidates and oversized lines in malformed_skipped.

The command removes the host prefix. It preserves tag order, delimiter form, duplicate messages, and source order. It writes one selected message per output line.

The command writes output atomically. No match exits 1. Invalid or unsafe output exits 2. Existing output remains unchanged after each failure. The fixed match limit is 10,000 messages. The result reports lines_scanned, matched_messages, written_messages, and malformed_skipped.

The command excludes these features:

• wrapped or multiline FIX messages
• multiple FIX messages on one physical line
• mixed delimiters within one message
• host-log suffixes after CheckSum
• relationship expansion, including --follow-chain
• fixtracer scan
• time-window selection
• MsgSeqNum and explicit session selectors
• context-message expansion
• directory and cross-file extraction
• SQLite .fxtcase, viewer, and inspect integration
• Pro licensing behavior

SQLite .fxtcase is in development under separate work. This release makes no delivery-date promise for that path.

Report Input Envelope

The v0.1.0a2 report path is designed for targeted incident extracts. FIXTracer copies each deduplicated resolved input file into a private snapshot. The size limit, source hash, and parser use the same snapshot bytes:

• The recommended maximum is 5,000,000 bytes.
• Input above 5,000,000 bytes produces a memory-risk warning.
• The hard limit is 10,000,000 bytes.
• Input above 10,000,000 bytes exits with code 2 before input hashing or report processing.
• --force-large-input bypasses the refusal and shows a strong memory-risk warning.

The exact 10,000,000-byte boundary is accepted with a warning. A refusal does not create, truncate, or replace HTML or JSON output. Required safety messages remain visible with --quiet and --no-progress.

The private snapshot needs available temporary space equal to the consumed input bytes. Set TMPDIR to select a disk-backed temporary directory. If the selected directory is a tmpfs, the snapshot consumes memory in addition to report processing.

Reduce large input before report generation:

fixtracer extract production.log --clordid ABC123 --out incident.log
fixtracer report incident.log --out incident.html

See the report input benchmark evidence for the measured release envelope.

Outputs

Direct incident extract:

fixtracer extract production.log --clordid ABC123 --out incident.log

HTML report:

fixtracer report path/to/fix.log --out report.html

Structured JSON artifact:

fixtracer report path/to/fix.log --json-out report.json

HTML and JSON in one run:

fixtracer report path/to/fix.log --out report.html --json-out report.json

Render HTML from an existing JSON artifact:

fixtracer render report.json --out report.html

Use --json-out when you want the current structured report artifact for transparency, debugging, triage preview, manually invoked scripting for an individually selected investigation, or a later render step. Technical Preview JSON can change between preview releases. Current implementation: the machine-facing artifact is a compact deterministic JSON envelope containing a manifest and deterministic gzip/base64 packed payload. fixtracer render consumes that artifact to rebuild the HTML report. It also remains compatible with older expanded {"meta": {...}, "timelines": [...]} report JSON files. Target direction: #227/#239/#240/#241/#242 define the stable Finding, explicit InvestigationArtifact, minimal IncidentCase, stable timeline cross-link, and nullable case taxonomy contracts, but this JSON output has not migrated to that envelope yet. Future case-first output should treat standalone HTML as a derived evidence report/export from the portable case model.

Report and render output safety

report and render check all selected input and output paths before processing or writing. Conflicting paths cause exit code 2. The diagnostic remains visible with --quiet and --no-progress. Existing bytes remain unchanged, and missing output files and directories remain absent. Choose different filenames to retry; FIXTracer does not rename an output automatically.

The check rejects aliases through relative paths, .., symbolic links, and hard links. It also rejects complete effective paths that differ only by letter case or canonical Unicode normalization, including parent directory names. This conservative restriction applies on every platform, even when the filesystem could store those names separately. Identical leaf names in clearly different parent directories remain permitted. Safe existing outputs can still be replaced, and safe missing parent directories can still be created. A file destination cannot also be a required output directory. Symbolic-link targets must have traversable directory components, including components before ... Each required directory must exist or be created by the selected writer.

This is a preflight check. It does not protect against concurrent filesystem changes, make combined output publication atomic, or provide crash recovery. The sample command also rejects conflicting selected output destinations. Use distinct paths for its log, HTML, and JSON outputs.

License policy (Technical Preview)

The 0.1.0a3 release retains the Free Technical Preview at no charge. The public release does not offer Pro access, license issuance, pricing, payment, accounts, or commercial activation.

--version, version, sample, and extract are Free. Explicit-file report runs emit a limited Free triage artifact. render remains license-free and follows the artifact profile embedded in the JSON. Directory and recursive inputs are not available in the initial public Free release. Existing Pro-capable code remains dormant and is not a released product surface. See docs/free_pro_policy.md for the longer-term policy.

Run what exists today

• fixtracer extract performs Free direct identifier extraction from one line-oriented FIX log. It does not expand relationships or process multiple files.
• fixtracer report parses FIX logs, builds deterministic timelines, detects implemented anomalies, and renders a self-contained HTML artifact. Public explicit-file reports are Free triage-depth and redact raw FIX and exact rule-code surfaces.
• report accepts explicit file inputs. Directory discovery and --recursive are not available in the initial public Free release.
• fixtracer render rebuilds HTML from the current report JSON artifact without a license check, preserving the embedded output profile.
• Available CLI commands: extract, report, render, sample, version.
• sample writes a bundled synthetic FIX log for installed smoke tests and quick demos.

FIXTracer is CLI-first and produces a single, self-contained artifact per run. A legacy web demo was removed and is no longer supported. There is no hosted production-log parser in the Technical Preview; run fixtracer report locally for real or internal logs.

Feedback

After the public website is available, use the feedback page for bugs, confusing output, documentation gaps, or preview feedback. Do not send or attach production logs to support, feedback, or public issues. Use descriptions or synthetic reproductions.

Product direction

FIXTracer is evolving from timeline/report output toward deterministic, portable investigation cases while preserving whole-log ingestion, evidence-backed timelines, and HTML/JSON/export alignment from one case model.

The direct extraction command is the bounded v0.1.0a2 bridge for supported line-oriented large logs. Whole-log case processing remains the target contract.

Human-facing strategy source of truth: docs/product_direction.md

CLI report progress

fixtracer report uses standard verbosity flags:

• Default: deterministic phase lines ([N/T] <Phase name>).
• --verbose / -v: rich progress (spinners, elapsed time, rates).
• --no-progress: no progress output (warnings/errors still shown).
• --quiet / -q: suppress non-safety output. Required input-size warnings and refusals remain visible.

Browser support for HTML reports

• All interactivity is embedded directly in the generated HTML file.
• Interactive view requires a modern desktop browser with JavaScript and DecompressionStream("gzip") (Chrome, Edge, Firefox, Safari).
• The engaged timeline lane state also relies on CSS :has() support (Chrome 105+, Firefox 121+, Safari 15.4+); older browsers still render the lane, but keep the spine in its neutral tone.
• Mobile/Quick Look viewers fall back to an intentionally minimal static preview. The full artifact is a desktop interactive experience.

Project map

fixtracer/
  cli/        # Typer CLI (extract, report, render, sample, version)
  licensing/  # License verification + deterministic exit policy
  models/     # Canonical Pydantic models + schema export
  parser/     # Deterministic FIX parser + normalization
  rules/      # Deterministic anomaly engine (partial rule set)
  order/      # Deterministic routing + timeline building
  render/     # HTML investigation artifact renderer (current implementation is timeline-first)
docs/         # Human docs (critical path, BMVP overview, epics)
schemas/      # Generated JSON Schemas

Status today

Component	Status	Notes
CLI	Implemented (extract, report, render, sample, version)	extract streams direct matches from one line-oriented log; report runs parse -> route -> timeline -> current artifact; render turns current JSON into HTML
Parser	Implemented	Delimiter/prefix/multiline handling with deterministic normalization
Routing	Implemented	Deterministic order / session / ungrouped / other partitioning
Timeline	Implemented	Union-find grouping + deterministic ordering + event mapping
Anomaly engine	Partial	20 emitted rule codes are implemented today; see docs/status.md for the current rule set and remaining Step 3 gaps
Renderer	Implemented	Self-contained HTML with packed artifact data, a minimal graceful fallback, interactive anomaly/session surfaces, and current JSON round-tripping
Goldens/tests	Implemented (with a small number of legacy markers)	Seed/extended/regression/anomaly/combined suites are present; see docs/status.md for the current validation baseline
Licensing	Implemented verification path	Signature verification, deterministic policy checks, Free triage explicit-file reports, Pro directory/recursive gates, and full-current-artifact profile stamping are tested; broader commercial policy enforcement remains future work
Schemas	Partially implemented	Core model schemas plus the current timeline-first ReportArtifact semantic payload schema export to /schemas; #227/#239/#240/#241/#242 formalize the target Finding, InvestigationArtifact, minimal IncidentCase, stable timeline cross-link, and nullable case taxonomy contracts in code/specs; emitted runtime artifacts now use a compact packed report JSON envelope over that payload while full target-envelope migration remains deferred

Detailed gap analysis and next work: docs/status.md.

Verification status is tracked in docs/status.md (single source of truth for latest counts).

Makefile tasks

• make lint — run ruff lint
• make fmt — run ruff format
• make test — run pytest
• make test-goldens — placeholder target (goldens currently run through normal pytest suites)
• make build-wheel — build Python wheel
• make build-binaries — build PyInstaller binary
• make export-schemas — write JSON Schemas to schemas/

Regenerate schemas

python3 -m fixtracer.models.schemas  # writes to ./schemas

Helpful docs (for humans)

• docs/product_direction.md — strategic direction, terminology, and current-vs-target contract
• docs/json_artifact.md — compact report JSON artifact contract and benchmark entrypoint
• docs/data_safety.md — local-first privacy posture and production-data handling rules
• docs/known_limitations.md — Technical Preview limitations and current non-goals
• docs/release_roadmap.md — release-ring plan for Technical Preview, Investigation Artifact Beta, and future workflow hardening
• docs/release_posture.md — Technical Preview positioning and public-surface guardrails
• docs/release_verification.md — SHA256 checksum verification for release artifacts
• docs/status.md — current implementation status + prioritized remaining work
• docs/critical_path.md — what ships, and in what order
• docs/bmvp_high_level.md — BMVP scope and what is out
• docs/model_relations.md — current message/timeline/anomaly model plus target investigation-artifact direction
• docs/specs/investigation_artifact.md — formalized target Finding, InvestigationArtifact, minimal IncidentCase, timeline cross-link, and nullable case taxonomy contract for #227/#239/#240/#241/#242
• docs/epics/step0_generation/tech_stack.md — locked tech stack choices
• docs/specs/fix_spec.md — FIX rules we use for the BMVP