Fieldwork

Explore unfamiliar data through patterns, source-row evidence, and reproducible investigations. Fieldwork is a Python toolkit for researchers working with pandas dataframes: find availability families, examine dependencies and candidate grains, then choose a useful census path through the data.

Alpha release. Python 3.11–3.14; pandas and NumPy are the only required runtime dependencies.

pip install fieldwork

import pandas as pd
import fieldwork as fw

df = pd.DataFrame({
    "site": ["North", "North", "South", "South"],
    "exam": [1, 1, 2, 2],
    "image": [10, 11, 20, 21],
    "report": ["ok", "ok", "ok", None],
})
overview = fw.explore(df)
availability = fw.missingness(df, entity="exam", min_implication=0.75)
paths = fw.suggest_paths(df, features=["site", "exam"])
tree = paths.best.census(df)
print(tree)

Use result.to_frame() for discovery tables and result.inspect(df, finding_id, exceptions=True) for the saved example rows. Duplicate indexes are supported; inspection checks the ordered source dataset. Use result.select(df, finding_id) to recover the complete matching population as a Scope, then pass that scope to missingness or suggest_paths. paths.best.census(df) preserves the selected population and sentinel conventions. Choose unit="entities" with entity= for equal entity weights, or keep the default unit="rows". Browse connected feature evidence with overview.relationships("image"). Save results with to_dict(), export with render_svg() or render_html(), and reapply a Recipe to later deliveries.

The single-table workflow includes independent levels, contextual pair summaries, joint counts and absence, exact grain graphs, bounded approximate dependency discovery, five census-path objectives, availability signatures and entity summaries, string and numeric patterns, scoped investigations, and delivery comparisons. Topology-only exports retain structure while suppressing quantitative evidence. Automatic related-table discovery is a later extension.

• User documentation source
• Developer documentation: architecture, contracts, algorithms, releases
• Investigation journey and units
• Executable investigation and notebook
• MIT license and extraction provenance

For development: uv sync --locked, uv run pytest, and uv build. Documentation images are generated from executable examples using the public renderers. Run uv run python scripts/generate_assets.py after behavior or styling changes; CI checks concurrence and release builds regenerate the assets.