fwforge 0.1.4

Fixed-width file parser for legacy logistics & supply chain data

FixedWidth Forge

Parse legacy fixed-width files from logistics, warehouses, ERP systems, and mainframes — instantly.

Turn messy carrier reports, shipment manifests, and COBOL exports into clean CSV or JSON with a simple YAML schema.

Why Logistics & Supply Chain?

Legacy formats like fixed-width flat files, EDI, and proprietary mainframes are the glue of the supply chain. fwforge provides the bridge to modern data pipelines without the enterprise bloat.

Features

• Schema-Driven: Define column layouts in YAML. Supports start+length or start+end positions.
• Ultra-Fast: Built on Python principles for rapid parsing of multi-gigabyte flat files.
• Batch Processing: Process entire directories of manifest exports with one command.
• Inference Engine: Use --infer to automatically generate a baseline schema from a sample data file.
• Clean Output: Transform legacy data to clean CSV or JSON with built-in type casting.

Installation (Free / Open Source)

pip install fwforge

Install from source (latest dev):

pip install git+https://github.com/Devotall/fwforge.git

Looking for the commercial Pro binary?
Visit our store — Lemon Squeezy now handles distribution and delivery of the standalone high-performance fwforge-pro binary (multi-threading, audit logs, enterprise features, etc.). One-time purchase or annual license (no PyPI, no subscriptions for the binary itself).

(The fwforge-pro/ directory in this repo contains the source/build configuration for the Pro binary.)

Security

FW Forge is built with a security-first mindset. The codebase has undergone a proactive security review to ensure robustness and data integrity.

• Minimal Dependency Surface Area: We intentionally minimize third-party dependencies to reduce supply chain risk.
• Secure Architecture: The project separates core processing logic from distribution and uses a clean, minimal codebase that is periodically scanned for vulnerabilities.
• Input Sanitization: File processing operations use standard, vetted libraries and avoid insecure shell-based operations.

For any security-related inquiries or to report a potential vulnerability, please reach out via our GitHub Issues page.

1. Infer a schema

Start from scratch with a sample file:

fwforge --infer -i data.txt > my-layout.yaml

2. Parse data

Convert legacy data to CSV (using the included sample):

fwforge -i data.txt -s layout.yaml -f csv -o output.csv
cat output.csv

3. Batch process a folder

fwforge -i ./daily_manifests/ -s manifest.yaml -f json

VS Code Extension (Alpha)

Preview legacy mainframe data as interactive tables directly in VS Code.

• Located in: vscode-extension/
• Supports context-menu "FW Forge: Preview as Table" for .txt and .fwf files (and similar).
• Visualizes raw mainframe output using your custom YAML schemas.
• Requires the fwforge CLI in PATH (configurable via fwforge.cliPath setting).
• Funnel for Pro: Automatically detects fwforge-pro binary (or falls back); shows >50MB capability nudge for free CLI users. Also tracks free single-file previews and offers upgrade after ~20 uses to address batch bypass via 1-at-a-time in the IDE.

Installation

cd fwforge/vscode-extension
npm install -g @vscode/vsce
vsce package
code --install-extension fixedwidth-forge-0.1.0.vsix

The extension runs the CLI to parse and renders a clean HTML table in a webview.

Early Adoption

23 downloads / acquisitions (20 in the last 30 days) 🎉

• Page views in funnel: 12 (100%)
• Acquisition: 23 (191.67% trend)
• Tracked via dashboard analytics (funnel + trend charts)

Thanks to the first 23 users who gave it a shot! This is after the initial PyPI + VS Code extension launch.

Free vs Pro

• Free (PyPI + open source): pip install fwforge. Great for most users. Directory batch processing is limited to 5 files (soft limit). The VS Code extension also has a soft limit on single-file previews (~20) when using the free CLI.
• Pro (commercial binary): Standalone fwforge-pro binary (multi-threading, auto-repair, audit logs, pre-built schemas, unlimited files). Drop it in PATH and the extension picks it up automatically.

Purchase Pro at the Lemon Squeezy store.

This follows an "invisible sales" model: the extension, CLI limits, and docs gently direct power users who hit scale to the paid version.

Invisible Sales / Long-Tail Strategy

• Document specific formats: "How to parse SAP fixed-width to CSV", "AS/400 flat file parsing", etc. (add pages to GitHub Wiki or /docs).
• List supported formats in the VS Code Marketplace description.
• Answer GitHub issues in related logistics/EDI projects with links to fwforge.

Example Schema (layout.yaml)

name: "Freight-Manifest-v1"
columns:
  - name: "carrier_code"
    start: 0
    length: 5
    trim: true
    type: "string"
  - name: "weight"
    start: 20
    length: 10
    trim: true
    type: "float"

Development

git clone https://github.com/Devotall/fwforge.git
cd fwforge
uv sync
uv run pytest -v

# Run the CLI
uv run fwforge --help

Roadmap

• More robust type casting (dates, currency, custom)
• Schema validation + strict mode
• Better inference (header detection, multi-line records)
• Performance / memory improvements for GB+ files
• Standalone binary releases
• Expanded output formats (parquet, etc.)

Changelog

See CHANGELOG.md for release notes.