vcti-array-display 1.1.0

Presentation layer for NumPy structured arrays — column filtering, dtype flattening, enum mapping, array slicing, and adapters

Array Display

Presentation layer for NumPy structured arrays.

ArrayDisplay wraps a structured NumPy array reference (no copy) and provides five capabilities for controlling how the data is presented:

1. Column filtering — hide columns by name, regex, or predicate
2. Dtype flattening — expand vector/matrix fields into scalar columns with custom component names
3. Enum mapping — lazily map integer ID columns to human-readable names
4. Array slicing — create lightweight index arrays for bounded access
5. Adapters — render as text table or pandas DataFrame on bounded slices

Installation

pip install vcti-array-display>=1.1.0

With pandas support (optional):

pip install vcti-array-display[pandas]>=1.1.0

---

1. Column Filtering

Hide unwanted columns using any combination of exact names, regex patterns, and callable predicates.

import re
import numpy as np
from vcti.arraydisplay import ArrayDisplay, FILLER_COLUMNS, LENGTH_COLUMNS, VOID_COLUMNS

dt = np.dtype([
    ('id', 'i4'), ('f0', 'V4'), ('label', 'U20'),
    ('label_len', 'i4'), ('value', 'f8'), ('f3', 'V8'),
])
arr = np.zeros(10, dtype=dt)

# Pre-built patterns for C++ interop noise
view = ArrayDisplay(arr, exclude_columns=[FILLER_COLUMNS, LENGTH_COLUMNS])
view.view_columns  # ['id', 'label', 'value']

# Or exclude by dtype — catches all void padding regardless of name
view = ArrayDisplay(arr, exclude_columns=[VOID_COLUMNS, LENGTH_COLUMNS])

# Mix exact names, regex, and callables
view = ArrayDisplay(arr, exclude_columns=[
    FILLER_COLUMNS,                        # regex: ^f\d+$
    LENGTH_COLUMNS,                        # regex: _len$
    "debug_flag",                          # exact name
    re.compile(r"^tmp_"),                  # custom regex
    lambda name, dtype: dtype.kind == 'V', # by dtype
])

No magic defaults — ArrayDisplay(arr) shows all columns. Pre-built patterns FILLER_COLUMNS, LENGTH_COLUMNS, and VOID_COLUMNS are opt-in. Callables receive (name, dtype) for filtering by type, shape, or both.

2. Dtype Flattening

Expand vector and matrix fields into individual scalar columns, with optional user-defined component names.

dt = np.dtype([('id', 'i4'), ('position', 'f8', (3,))])
arr = np.zeros(5, dtype=dt)

# Default numeric suffixes
view = ArrayDisplay(arr, flatten_dtype=True)
view.view_columns  # ['id', 'position_0', 'position_1', 'position_2']

# Custom component names
view = ArrayDisplay(arr, flatten_dtype=True,
                 component_names={'position': ['x', 'y', 'z']})
view.view_columns  # ['id', 'position_x', 'position_y', 'position_z']

# Component grouping is tracked for consumers (e.g., header spanning)
view.field_components
# {'position': ['position_x', 'position_y', 'position_z']}

# Rename after construction
view.set_component_names('position', ['lat', 'lon', 'alt'])

Note: Default flattened names (e.g., position_0) are generated by vcti-nputils and may truncate long field names. Use component_names to ensure predictable, readable column names regardless of the defaults.

3. Enum Mapping

Map integer ID columns to human-readable names. The mapping is stored as a recipe and resolved lazily — only when presenting a bounded slice.

dt = np.dtype([('id', 'i4'), ('element_type', 'i4'), ('value', 'f8')])
arr = np.array([(1, 1, 10.5), (2, 2, 20.3), (3, 1, 15.0)], dtype=dt)
view = ArrayDisplay(arr)

view.add_name_columns({
    'element_type_name': ('element_type', {1: 'QUAD', 2: 'HEX'}),
})
view.view_columns  # ['id', 'element_type_name', 'value']

# The mapping is NOT materialized yet — it's a recipe:
view.name_columns
# {'element_type_name': ('element_type', {1: 'QUAD', 2: 'HEX'})}

4. Array Slicing

Create lightweight index arrays for bounded access. The index array is tiny regardless of the underlying array size.

idx = view.to_slice(head=20)                       # first 20 rows
idx = view.to_slice(tail=10)                       # last 10 rows
idx = view.to_slice(head=10, tail=5)               # first 10 + last 5
idx = view.to_slice(mask=arr['value'] > 50)        # boolean filter
idx = view.to_slice(indices=np.array([0, 100, 999]))  # explicit

view.array[idx]                                    # sliced data

5. Adapters

Text table (pure numpy — no extra dependencies)

print(view.to_table(head=10, tail=5))
# id | element_type_name | value
# ---+-------------------+------
#  1 | QUAD              |  10.5
#  2 | HEX               |  20.3
# ...
# [1000 rows x 3 columns]

pandas DataFrame (optional dependency)

df = view.to_dataframe()                   # full array
df = view.to_dataframe(head=100)           # first 100 rows
df = view.to_dataframe(resolve_names=False)  # raw ID columns

Enum names are materialized using pd.Categorical for memory efficiency.

Jupyter notebooks

ArrayDisplay provides _repr_html_() — displays a bounded row window with enum names resolved automatically.

---

Performance

Designed for large CAE arrays (millions of rows, GBs of data):

• No array copy — wraps a reference to the original numpy array
• Lazy enum resolution — resolved only on the displayed slice
• Index arrays — to_slice() returns kilobytes regardless of array size
• Bounded presentation — to_table() and to_dataframe(head=N) never touch the full array
• pd.Categorical — ~100x less memory than string columns for enum values

---

API Summary

Method	Pillar	Description
set_view_columns(...)	Filtering	Configure visible columns
include_view_columns(cols)	Filtering	Add columns to the view
exclude_view_columns(cols)	Filtering	Remove columns from the view
replace_view_columns(mapping)	Filtering	Rename columns in the view
set_array(arr, ...)	Flattening	Set array with optional flattening
set_component_names(field, names)	Flattening	Rename flattened components
add_name_columns(mappings)	Enum mapping	Register lazy enum mappings
to_slice(...)	Slicing	Create index array for bounded access
to_table(...)	Adapter	Render text table (pure numpy)
to_dataframe(...)	Adapter	Convert to pandas DataFrame
copy()	Misc	Shallow copy with independent config
append_columns(cols)	Misc	Join additional structured columns

Examples

See examples/full_pipeline.py for a complete end-to-end script demonstrating all five pillars.

Dependencies

• numpy (>=1.24) — required
• vcti-nputils (>=1.0.0) — required
• pandas (>=2.0) — optional, for to_dataframe() and Jupyter display