relflow 0.3.2

relflow builds PyTorch/Lightning models from nested records streamed through Apache Arrow. Typed fields represent values; branches combine them into local contexts; decoders learn to predict selected fields from the available context.

Install

Python 3.11 or newer:

uv add relflow

Add relflow[text] for Hugging Face text encoders or relflow[serving] for the HTTP runtime.

Describe One Record

For example, predict whether a payment will be disputed:

payment_id: payment-42
amount: 86.50
merchant: grocery
disputed: false

import relflow as rf

model = rf.Model.xs(
    amount=rf.Number,
    merchant=rf.Category,
    disputed=rf.Boolean(mask=True),
)

Parent keywords name each field. mask=True makes disputed a supervised prediction target whose label is excluded from the inputs. payment_id remains available as application metadata. Add repeated contexts with rf.Branch(...) when a prediction needs transaction history or other related records.

Train And Predict

The application supplies Arrow tables named train_table, validation_table, and request_table. Training and validation include the dispute label; prediction requests omit it.

data = rf.ArrowDataModule(
    model=model,
    train=train_table,
    validate=validation_table,
)
model.fit(data, batch_size=128, learning_rate=1e-4, weight_decay=0.01, max_epochs=30)

predictions = model.predict(request_table, batch_size=256, retain=("payment_id",))
model.save("artifacts/payment.rf")

Fitting includes early stopping and restoration of the best validation model. Configure AdamW directly on fit; each call starts with the current model parameters and fresh optimization state. Hardware, logging, and Lightning options can also be supplied to the model methods. Predictions are an Arrow table: decoded results use schema field names such as predictions.disputed, and repeated branches become lists of records. Requested embeddings appear separately in representations. Retention adds an inputs column. For larger file-backed jobs, use an Arrow data module with rf.Writer to stream predictions to Parquet.

Documentation

• Getting started
• Add transaction history
• Model structure
• Data types
• Arrow data loading
• Preprocessing
• Training and checkpoints
• Batch inference
• Prediction output

Documentation, modeling proofs, and branding sources live in the separate relflow/docs repository. Its website uses static examples and Typst diagrams and builds independently of this package. Run package checks here with uv run pytest; follow the documentation repository's instructions to run learning experiments explicitly. See CONTRIBUTING.md for development conventions.

Community · Apache 2.0 license