vcti-nputils 1.2.0

NumPy structured array utilities — joining, flattening, field views, enum mapping, and position arrays

NumPy Utils

NumPy structured array utilities — joining, flattening, field views, enum mapping, and position arrays.

Installation

pip install vcti-nputils>=1.2.0

In pyproject.toml dependencies

dependencies = [
    "vcti-nputils>=1.2.0",
]

---

Quick Start

import numpy as np
from vcti.nputils import (
    join_struct_arrays,
    flatten_record_dtype,
    fields_view,
    drop_fields,
    rename_fields,
    name_array,
    position_array,
    as_ndarray,
    decode_field,
    encode_field,
    check_overflow,
    with_encoding,
)

# Join structured arrays horizontally
dt1 = np.dtype([('id', 'i4'), ('value', 'f8')])
dt2 = np.dtype([('name', 'U10')])
arr1 = np.array([(1, 1.5), (2, 2.5)], dtype=dt1)
arr2 = np.array([('Alice',), ('Bob',)], dtype=dt2)
joined = join_struct_arrays([arr1, arr2])
# dtype: [('id', 'i4'), ('value', 'f8'), ('name', 'U10')]

# Create a zero-copy view with selected fields
view = fields_view(joined, ['id', 'name'])

# Drop fields from a structured array (zero-copy)
clean = drop_fields(joined, ['value'])
# dtype: [('id', 'i4'), ('name', 'U10')]

# Rename fields in a dtype
new_dt = rename_fields(dt1, {'id': 'node_id', 'value': 'temperature'})
# dtype: [('node_id', 'i4'), ('temperature', 'f8')]

# Flatten array fields into individual columns
dt = np.dtype([('id', 'i4'), ('coords', 'f8', (3,))])
flat_dt, cols = flatten_record_dtype(dt)
# cols: ['id', 'coord_0', 'coord_1', 'coord_2']

# Map numeric enum values to names
enum_dict = {1: 'ACTIVE', 2: 'INACTIVE', 3: 'PENDING'}
names = name_array(np.array([1, 2, 1, 3]), enum_dict)

# Convert counts to cumulative offsets
offsets = position_array(np.array([3, 2, 4, 1]))
# array([0, 3, 5, 9, 10])

# Safely coerce inputs to ndarray
arr = as_ndarray([1, 2, 3], dtype=np.float64)
empty = as_ndarray(None)  # array([], dtype=float64)

# Byte <-> string conversion for C++/pybind11 interop
dt = np.dtype([('name', 'S10'), ('name_length', 'i4')])
sa = np.zeros(2, dtype=dt)
encode_field(sa, 'name', ['Alice', 'Bob'], length_field='name_length')
# sa['name'] -> [b'Alice', b'Bob'], sa['name_length'] -> [5, 3]

decoded = decode_field(sa, 'name')
# array(['Alice', 'Bob'], dtype='<U...')

# Detect truncation via paired (bytes, length) fields
overflow = check_overflow(sa, 'name', 'name_length')

# Attach encoding to a dtype so decode_field/encode_field use it automatically
name_dt = with_encoding(np.dtype('S32'), 'latin-1')

---

Functions

Structural array utilities

Function	Purpose
join_struct_arrays(arrays)	Join structured arrays horizontally by combining fields
fields_view(sa, fields)	Zero-copy view with selected fields only
drop_fields(sa, exclude)	Zero-copy view with specified fields excluded
rename_fields(dt, mapping)	Return a new dtype with fields renamed
flatten_record_dtype(dt)	Expand array fields into individual scalar fields
name_array(nparray, enum_dict)	Map numeric values to string names
position_array(counts, dtype)	Convert count array to cumulative offset array
as_ndarray(value, dtype)	Coerce None, list, or ndarray to ndarray

Byte / string conversion (pybind11 interop)

Function	Purpose
string_from_bytes(value, encoding)	Decode a single bytes value, stripping null padding
bytes_from_string(value, length, encoding)	Encode to fixed-length bytes (pad or truncate)
decode_column(byte_array, encoding)	Vectorized decode of a byte column to strings
encode_column(strings, length, encoding)	Vectorized encode to (bytes, lengths)
decode_field(sa, field_name, *, encoding)	Decode a byte field in a structured array
encode_field(sa, field_name, strings, *, length_field, encoding)	Encode strings into a byte field, optionally populating a paired length field
check_overflow(sa, field_name, length_field)	Detect rows where the original string length exceeded the byte field
get_encoding(dtype, default)	Read encoding from dtype.metadata['encoding']
with_encoding(dtype, encoding)	Attach encoding to a scalar dtype via metadata

---

Dependencies

• numpy (>=2.0)