Lookup

Purpose

vcti-lookup provides Lookup, an attribute-based filtering engine for Python sequences. It works at two levels:

Level 1 — Filtering. You have a sequence of items, each with an ID and attributes. Lookup lets you filter by rules and get matching items back. The items can be dicts, dataclasses, Pydantic models, numpy structured-array rows, or any object — pluggable getters control how attributes are extracted.

Level 2 — Data mapping. You have external data (a list of file paths, a numpy array, a tree of nodes) and a separate set of attributes describing each item. You build an attribute set where id = index, filter with Lookup, and use the matching IDs to index back into your data. Lookup doesn't wrap or copy your data — you keep full control.

Level 1 — Filter directly          Level 2 — Map to external data

┌──────────────┐                   ┌───────────────┐
│ items        │                   │ your data     │
│ (id + attrs) │                   │ (list, array) │
└──────┬───────┘                   └───────▲───────┘
       │                                   │ indices
       ▼                                   │
┌──────────────┐  Rules    ┌──────────────┐  Rules
│    Lookup    │◄──────    │    Lookup    │◄──────
└──────┬───────┘           └──────┬───────┘
       │                          │
       ▼                          ▼
  matched items            matching_ids()

Installation

From GitHub (recommended for development)

# Latest main branch
pip install vcti-lookup


### In `requirements.txt`

vcti-lookup>=1.0.2

### In `pyproject.toml` dependencies

```toml
dependencies = [
    "vcti-lookup>=1.0.2",
]

Quick Start

Level 1 — Filter items directly

from vcti.lookup import Lookup, Rule

items = [
    {"id": 1, "type": "pdf", "size": 1000},
    {"id": 2, "type": "txt", "size": 200},
    {"id": 3, "type": "pdf", "size": 1500},
]

lk = Lookup(items)

# Single rule
pdfs = lk.filter([Rule("type", "==", "pdf")])
# [{"id": 1, ...}, {"id": 3, ...}]

# Multiple rules (AND)
large_pdfs = lk.filter([Rule("type", "==", "pdf"), Rule("size", ">", 1200)])
# [{"id": 3, ...}]

# OR logic
result = lk.filter_any([Rule("type", "==", "pdf"), Rule("type", "==", "jpg")])

# Exclusion (complement of filter)
non_pdfs = lk.exclude([Rule("type", "==", "pdf")])

# Lookup by ID
item = lk.get(2)

# Unary operators
lk.filter([Rule("name", "is_empty")])

Level 2 — Map attributes to external data

from vcti.lookup import Lookup, Rule

# Your data — any shape, Lookup doesn't touch it
data = ["report.pdf", "notes.txt", "analysis.pdf"]

# Build attribute set — id = index into your data
attrs = [
    {"id": 0, "type": "pdf", "size": 1000},
    {"id": 1, "type": "txt", "size": 200},
    {"id": 2, "type": "pdf", "size": 1500},
]

lk = Lookup(attrs)
indices = lk.matching_ids([Rule("type", "==", "pdf")])
# [0, 2]

# Map back to your data
results = [data[i] for i in indices]
# ["report.pdf", "analysis.pdf"]

Numpy structured arrays (direct, no copy)

import numpy as np
from vcti.lookup import Lookup, Rule
from vcti.lookup.getter import numpy_getter

dt = np.dtype([("id", "i4"), ("type", "U10"), ("value", "f8")])
arr = np.array([(0, "sensor", 3.14), (1, "actuator", 2.71)], dtype=dt)

# Pass the array directly — no conversion
lk = Lookup(arr, getter=numpy_getter, id_key="id")
indices = lk.matching_ids([Rule("type", "==", "sensor")])
# [0]

# Index back into the array
results = arr[indices]

Dataclasses

from dataclasses import dataclass

@dataclass
class File:
    id: int
    type: str
    size: int

files = [File(1, "pdf", 1000), File(2, "txt", 200), File(3, "pdf", 1500)]
lk = Lookup(files)
result = lk.filter([Rule("type", "==", "pdf")])
# [File(1, "pdf", 1000), File(3, "pdf", 1500)]

Modifiers

Modifiers are keyword arguments forwarded to vcti-predicate. They control comparison behaviour. Pass them as the fourth argument to Rule.

items = [{"id": 1, "name": "Report_Q1"}, {"id": 2, "name": "report_Q2"}]
lk = Lookup(items)

# Default: case-insensitive (both match)
lk.filter([Rule("name", "^=", "report")])

# Case-sensitive via modifier (only item 1)
lk.filter([Rule("name", "^=", "Report", {"case_sensitive": True})])

# Float tolerance
lk.filter([Rule("value", "==", 3.14, {"tolerance": 0.01})])

# Regex with multiline
lk.filter([Rule("content", "~=", r"^error:", {"multiline": True})])

See patterns.md for more modifier examples.

Public API

Lookup methods

Dependencies

• vcti-predicate — condition evaluation engine
• numpy — optional, for numpy_getter

Documentation

• Design — Concepts, architecture, and design decisions
• Patterns — Real-world recipes and usage patterns
• Source Guide — Implementation walkthrough
• Extending — Creating custom value getters
• API Reference — Autodoc for all modules