ZMIP: Zoom-In Pipeline

Refine cell populations within each lineage after multi-sample integration.

Why zoom in? · Get started · Results · Documentation

ZMIP takes an annotated dataset from MSP, analyzes each lineage separately, and refines cell-type labels using marker genes and quality evidence. It writes updated annotations, records removed or reassigned cells, and produces reports for review.

Why zoom in?

Differences between major cell types can dominate a global analysis. Recomputing features, neighbors, and clusters within a lineage helps examine its finer populations. ZMIP uses this local view to refine labels and review remaining quality concerns. When there are too few cells for stable subgroup analysis, small lineages retain their existing annotations.

How it works

An AI assistant groups cells into lineages using the existing labels and embedding. Each selected lineage is re-embedded independently, then reviewed using marker genes, quality measurements, and signals from other lineages. The program checks the submitted decisions before merging the results back into the global dataset.

Click the diagram to open the interactive version, with pan, zoom, search, and guided views.

What you get

The output includes a refined H5AD, a global HTML report linking to detailed lineage reports, and cell-level records of removals and reassignments. Original MSP annotations remain available for comparison; retained cells keep their input expression, counts, and global embedding.

Get started

Start with MSP's annotated.h5ad, including its annotations, counts, batch metadata, graph, and UMAP. Install from PyPI (Python 3.10 or newer; MSP and the agent harness come with it), set your Volcengine Ark API key, then run:

python -m pip install zmip
export ARK_API_KEY="YOUR_ARK_API_KEY"
HARNESS=openai python -m zmip msp_out/annotated.h5ad --outdir zmip_out

Pass --design-context TEXT (same text as MSP's) when sample identity encodes the study design, e.g. one FACS plate = one mouse x one sort gate; every lineage's annotation agent sees it verbatim, so a sample-confined cluster is judged against the design rather than as a batch artefact. Kept in zmip_out/design_context.txt.

Read your results

After a successful run, open zmip_out/report.html in your browser. Check the lineage plan, follow the links to review local labels and supporting genes, then inspect removed and reassigned populations. To share all reports, copy the output directory with its subdirectories so the links still work.

Does ZMIP change my data?

Your input file stays unchanged. Cells removed by local filtering or the annotation agent are excluded from the final H5AD. Reassigned cells receive new labels; they are not re-embedded in the destination lineage during this run. Global plots retain MSP's embedding. See the input and output reference for fields and details.

Can I resume a run?

Repeat the same command to reuse completed, verified stages. Agent settings such as --max-turns or --model may differ between runs; only unfinished work uses the new values. An unfinished lineage reuses its verified integration and restores accepted cluster submissions, including the cell labels of any subclusters. The host revalidates saved decisions before continuing. SIGTERM stops new lineage launches and lets running work reach a safe point; the CLI exits with code 3. Repeat the command to resume.

If you change the input, analysis settings, or runtime, use a new output directory to keep both analyses, or add --force to recompute the plan and all selected lineages. To rebuild only the global report, run:

python -m zmip.report zmip_out

Validation

An isolated installation passed 99 tests, and an OpenAI/Doubao run completed the workflow on 256 Fu2022 cells, including independent data checks and resume without model calls. This checks the workflow at small scale; full-dataset performance and biological accuracy remain unvalidated. See validation records and remaining checks.

Further reading

ZMIP follows OSP for sample-level review and MSP for integration within the ECA-RSI workflow. See inputs and outputs for data conventions, runtime options for installation and configuration, and GitHub issues for questions or problems. ZMIP uses the MIT license.