fletchr-core

Arrow-backed frame and packet containers, transformers, pipeline, and I/O for binary protocol data. The integration layer of the fletchr framework — built on fletchr-uintn for bit-precise storage and fletchr-measurand for bit-fragment extraction and decoding.

Why?

Binary-protocol pipelines repeat the same shape across projects:

• Containers for fixed-width frames (rectangular) and variable-length packets (jagged).
• Transformers to filter, subframe, invert, synchronize, and decode-on-the-fly.
• I/O for the formats raw data arrives in (CSV, packed bits, msgpack, pickle) and the formats it leaves in (Arrow IPC, Parquet).
• A plugin model so domain-specific packet types and transformers plug in cleanly without forking the framework.

fletchr-core provides each of these as one Arrow-native, null-aware unit. Containers are backed by pa.Table with fletchr.uintn(bits=N) columns (no out-of-band schema metadata, nulls propagate end-to-end); the Measurand build chain consumes those tables directly; transformers compose into pipelines with provenance tracking built in.

Features

• Containers — FrameArray (column-major fixed-width frames), MuxFrameArray (multiplexed variants), PacketList (variable-length packets, homogeneous), Packet / GenericPacket / HeaderPacket / define_packet for declaring per-protocol packet types.
• fields declarations — Packet and FrameArray subclasses declare data-derived columns via fields: ClassVar[dict[str, Field]] using the measurand parameter DSL. The framework parses specs at class-definition time, materializes columns at construction, and exposes them via __getattr__. One-line column additions, no codec hooks to write.
• Transformers + Pipeline — FrameFilter, ChecksumFilter, MuxID, MuxCounter, MuxFilter, MuxSelect, SortByTime, FrameSynchronizer, Subframe, Reverse, BitInvert, ByteSwap, BitMask, BitReverse, BitShift, PNXor, PolarityFix, PacketFilter, PacketSelect, PatternPacketExtractor, DelimitedPacketExtractor, DropNull, clock transformers, plus the Pipeline container for composition.
• I/O — read_arrow / write_arrow (Arrow IPC), read_parquet / write_parquet, read_csv / write_csv, read_bits / write_bits, read_txt / write_txt, read_pickle / write_pickle, plus the generic read_file / write_file dispatch and FileReader / FileWriter extension points.
• Polars + pandas adapters — add_columns, merge, stack for interop with the common DataFrame libraries.
• History / provenance — optional processing-history DAG attached to containers; tracks slice / concat / transform operations. Toggle with enable_history / disable_history.
• Plugin registry — downstream packages register custom Packet subclasses, Transformers, and I/O readers/writers via Python entry points. register_plugin_group("my_pkg.plugins") to add a new discovery namespace.
• Custom exception hierarchy — FletchrCoreError root with subsystem branches (ContainerError, TransformError, CodecError, IODispatchError, RegistryError, PluginError, GrammarError). Each leaf also multi-inherits the matching builtin (ValueError / TypeError / KeyError / ImportError) so existing except ValueError: callers keep working.

Install

uv add fletchr-core                # or: pip install fletchr-core
uv add 'fletchr-core[pandas]'      # with pandas adapter (polars is required)

Requires Python 3.9+. Pulls in fletchr-uintn, fletchr-measurand, pyarrow >= 16, numpy >= 2.0, polars >= 0.20, attrs, escapement, ormsgpack, varuintarray.

Quickstart

import numpy as np
from varuintarray import VarUIntArray
from fletchr_core import FrameArray, write_arrow, read_arrow
from fletchr_measurand import Measurand, parameter_parser

# Build a 2-row, 3-word FrameArray with 8-bit words.
time = np.array(["2026-01-01", "2026-01-02"], dtype="datetime64[ns]")
data = VarUIntArray(
    np.array([[10, 20, 30], [40, 50, 60]], dtype=np.uint8),
    word_size=8,
)
frame = FrameArray(time=time, ctime=time, data=data)

frame.shape                  # (2, 3)
frame.table.column_names     # ['time', 'ctime', 'c1', 'c2', 'c3']

# Compute a measurand directly against the FrameArray's Arrow table.
# Parameter "[1+2]" reads word 1 + word 2 as a single 16-bit value.
m = Measurand(parameter=parameter_parser.parse("[1+2]"))
m.build(frame.table).values.to_pylist()
# [2580, 10290]  =  [10*256 + 20, 40*256 + 50]

# Round-trip through Arrow IPC.
write_arrow(frame, "frame.arrow")
back = read_arrow("frame.arrow")
assert back.shape == frame.shape

Declarative fields on a subclass

FrameArray (and Packet) subclasses can declare data-derived columns using the fletchr-measurand parameter DSL. The framework parses specs at class-definition time, materializes the columns at construction, and exposes them as attributes:

from typing import ClassVar
import numpy as np
from varuintarray import VarUIntArray
from fletchr_core import FrameArray, Field

class NumberedFrame(FrameArray):
    fields: ClassVar[dict[str, Field]] = {
        "counter":        Field("[1+2]"),               # raw 16-bit (uintn(bits=16))
        "counter_halved": Field("[1+2];u;EUC[0.5]"),    # decoded float64, scaled ×0.5
    }

time = np.array(["2026-01-01", "2026-01-02"], dtype="datetime64[ns]")
data = VarUIntArray(
    np.array([[0, 1, 100], [1, 0, 200]], dtype=np.uint8),
    word_size=8,
)
fa = NumberedFrame(time=time, ctime=time, data=data)
list(fa.counter)         # [1, 256]
list(fa.counter_halved)  # [0.5, 128.0]

Field specs use the fletchr-measurand grammar — parameter ";" encoding ";" euc ";" sampling-strategy, encoding/euc/ss independently optional. Parameter-only specs like [1+2] extract raw bits (natural output: fletchr.uintn(bits=N)); full measurand specs like [1+2];2c;EUC[0.1] apply Level 1 (encoding) and Level 2 (engineering-unit conversion) at construction time. When dtype= is omitted, the column's type is whatever the measurand chain naturally produces.

Public API

from fletchr_core import (
    # Containers
    FrameArray, MuxFrameArray,
    PacketList, Packet, GenericPacket, HeaderPacket, define_packet,
    # Pipeline
    Transformer, Pipeline, stack,
    # I/O (generic + per-format)
    read_file, write_file, FileReader, FileWriter,
    read_arrow, write_arrow,
    read_parquet, write_parquet,
    read_csv, write_csv,
    read_bits, write_bits,
    read_txt, write_txt,
    read_pickle, write_pickle,
    # DataFrame interop
    add_columns, merge,
    # History
    History, HistoryGraph, HistoryNode,
    enable_history, disable_history, is_history_enabled, reset_history_graph,
    # Plugins
    load_plugins, register_plugin_group,
    # Exceptions
    FletchrCoreError, ContainerError, TransformError,
    HomogeneityError, BitWidthError, SchemaError,
    PipelineError, TransformConfigError,
    CodecError, IODispatchError,
    RegistryError, RegistryCollisionError,
    PluginError, GrammarError,
)

Concrete transformers (FrameFilter, BitInvert, MuxSelect, etc.) live under fletchr_core.transform.

Extensibility

fletchr-core is a framework, not a finished application. Domain packages plug in by registering subclasses through Python entry points:

# in your downstream package's pyproject.toml
[project.entry-points."fletchr_core.plugins"]
my_protocol = "my_pkg.fletchr_plugins"

Modules in that entry-point group are imported lazily on first registry access; any Packet / FrameArray / Transformer / FileReader / FileWriter subclasses they define land in the same global registries as the built-ins. Use register_plugin_group("my_namespace.plugins") to add a new discovery namespace.

Claude Code skills

fletchr-core ships Claude Code skills as bundled package data. Install them into your user Claude skills directory:

python -m fletchr_core.skills list                # show bundled skills
python -m fletchr_core.skills install             # copy to ./.claude/skills/ (project)
python -m fletchr_core.skills install --user      # copy to ~/.claude/skills/

Project-level is the default (skills are typically scoped to the project that needs them); --user elevates to user-level.

The shipped skills travel versioned with the code they describe — when you upgrade fletchr-core, re-run install --force to refresh.

Links

• Source: https://github.com/fletchr-labs/fletchr
• Issues: https://github.com/fletchr-labs/fletchr/issues

License

MIT.