Field Set

A lightweight container for named columnar fields with lazy NumExpr expressions, scoped metadata, and pluggable field sources.

Overview

FieldSet holds named, row-aligned columnar fields and computed expressions over them. Each field is obtained through a field source — an eager in-memory numpy array by default, or a lazy provider such as a computed array, a memory-mapped file, or a vcti-datanode DataNode — so large or deferred data is loaded only when it is actually read. Expressions like "strain = stress / youngs_modulus" are parsed and evaluated lazily via NumExpr and cached, avoiding intermediate arrays. Row operations — fs.rows().where(...).sort_by(...).head(...).select() — filter, sort, slice, and group rows lazily into a new FieldSet, folding the chain into one index instead of copying a table per step. Scoped metadata keeps system settings (components) and user attributes (units, labels) in separate namespaces, and an optional [dataframe] extra exports to pandas with MultiIndex columns. The required core stays small (numpy, numexpr, vcti-cache); heavier integrations are optional extras.

Installation

pip install vcti-fieldset>=3.0.0

For pandas DataFrame support:

pip install vcti-fieldset[dataframe]>=3.0.0

For binding vcti-datanode DataNodes as fields:

pip install vcti-fieldset[datanode]>=3.0.0

In pyproject.toml dependencies

dependencies = [
    "vcti-fieldset>=3.0.0",
]

# or, with DataFrame support:
dependencies = [
    "vcti-fieldset[dataframe]>=3.0.0",
]

Quick Start

A FieldSet coordinates three subsystems, each reached as a named part — fs.fields, fs.expressions, fs.properties:

import numpy as np
from vcti.fieldset import FieldSet

# Create from named arrays
fs = FieldSet(
    stress=np.array([100.0, 200.0, 150.0]),
    displacement=np.array([0.1, 0.2, 0.15]),
)

# fs.expressions — lazy computed columns (NumExpr)
fs.expressions.add("strain = stress / 200000")
fs.get_values("strain")  # array([0.0005, 0.001, 0.00075])  (fs[name] resolves field OR expression)

# fs.properties — scoped metadata (user attributes + system settings)
fs.properties.set("units", "MPa", field="stress")
fs.properties.set("units", "mm", field="displacement")

# fs.fields — the stored fields
fs.fields.add_data(temperature=np.array([300.0, 350.0, 325.0]))
fs.fields.names            # ['stress', 'displacement', 'temperature']

# Row operations — filter/sort/slice rows into a new FieldSet
high = fs.rows().where("stress > 120").sort_by("stress", descending=True).select()
high["stress"]  # array([200., 150.])

# Projection — select/rename/cast columns (and freeze expressions) into a new FieldSet
out = fs.project().select("stress", "strain").rename({"stress": "sxx"}).build()
out.names  # ['sxx', 'strain']

Serialization and DataFrame export are free functions (they emit a foreign artifact — a file, a DataFrame — so they live in their modules, not on the class):

from vcti.fieldset import to_npz, from_npz
from vcti.fieldset.dataframe import to_dataframe  # requires the [dataframe] extra

to_npz(fs, "results.npz")
fs = from_npz("results.npz")     # a factory — returns a new FieldSet
df = to_dataframe(fs)             # pandas DataFrame with (field, component) columns

Core API

A FieldSet is a coordinator over three subsystems plus unified access and the two FieldSet→FieldSet builders.

On the FieldSet itself

fs.fields

fs.expressions

fs.properties

Scoped metadata via get / set (and the get_components / set_components shortcuts). System settings use the SystemProperty vocabulary; user attributes are free-form strings; either can be per-field or dataset-wide:

fs.properties.set("units", "MPa", field="stress")   # user attribute, per field
fs.properties.set("units", "MPa")                    # dataset-wide (field=None)

Free functions (import from their module)

Storage is backed by vcti-properties (a scoped PropertyStore); FieldSet does not re-export its key-mapper classes — import them from vcti.properties if you need them directly.

Dependencies

• numpy (>=1.24)
• numexpr (>=2.8)
• vcti-cache (>=1.0.0) — ObjectCache for expression results
• vcti-properties (>=1.0.0) — scoped metadata store

Optional

• pandas (>=2.0) — required for to_dataframe(), install via vcti-fieldset[dataframe]