midas-pipeline

End-to-end MIDAS HEDM orchestrator. FF is the single-scan degeneracy of PF. One package, one CLI, two scan modes.

Status

0.1.0 — end-to-end PF and FF paths live. The scanning indexer matches the C IndexerScanningOMP reference on its 1-voxel C-parity gate (seed identity, solution counts, voxel-center positions exact; orientation matrices within mrad-scale, the refiner closes the gap downstream). Real-data validation: datasetC CP-Ti consolidation_pf reproduces the legacy pf_MIDAS.py grain count (770 == 770, all common). Park22 P5c parity gate now runs in ~6.7s vs the original 790s after the scanning-indexer position-grid fix.

Stages call in-process Python kernels via midas-index / midas-fit-grain / midas-transforms / midas-stress. FF mode shells out to python -m midas_index and python -m midas_fit_grain (same kernels, subprocess for the FF parity-preserving pattern). No CUDA C; GPU is torch-only.

Install

pip install -e packages/midas_pipeline

Notebooks

Worked-example Jupyter notebooks live in notebooks/. They are not shipped with pip install — get them by cloning the MIDAS repository.

CLI

midas-pipeline run --scan-mode {ff,pf,auto} --params Parameters.txt --result rundir/
midas-pipeline status rundir/
midas-pipeline resume rundir/ --from <stage>
midas-pipeline reprocess rundir/
midas-pipeline inspect rundir/LayerNr_1/
midas-pipeline simulate --out simdir/ --params Parameters.txt
midas-pipeline seed --params ... --output UniqueOrientations.csv

When --scan-mode is omitted (default auto), the CLI sniffs the parameter file: nScans > 1 or presence of BeamSize / scanning keys → pf, otherwise ff. For PF mode, --n-scans, --scan-step, --beam-size, and --scan-pos-tol default to values in the params file (CLI flags override).

Indexer backend

midas-pipeline run --indexer-backend {python,c-omp} ...

python (default) — in-process numba/torch indexer. Portable (CPU/CUDA/MPS), differentiable, slower on large PF datasets.

c-omp — bundled unified C binary (midas_indexer) from midas-index. Requires midas-index installed with a working OpenMP toolchain (macOS: brew install libomp). ~290× faster than the Python path on real PF datasets (per the datasetC CP-Ti benchmark in packages/midas_index/dev/). Output is bit-identical to the Python path on the PF parity gate.

Coexistence with midas-ff-pipeline

The legacy midas-ff-pipeline console-script is preserved as an independent FF orchestrator (its own kernels, its own CLI). It is not deprecated by midas-pipeline run --scan-mode ff — both paths invoke the same midas-index / midas-fit-grain kernels under the hood, and both stay green on the FF parity gate. Pick whichever is more convenient for your workflow.

Architecture

  • One orchestrator with a mode-dependent STAGE_ORDER.
  • Shared kernel packages (midas-index, midas-fit-grain, midas-transforms, etc.) extended in place; FF behavior preserved by parity gates.
  • PF-only modules live inside midas_pipeline (find_grains/, sinogen, recon/, fuse, potts, em_refine, seeding/).
  • Differentiability + multi-device mandatory on every new compute path (CPU / CUDA / MPS via torch).

Constraints

  • No CUDA C; GPU support is torch-only.
  • No deletions of legacy code in this effort.
  • midas-process-grains is FF-only; PF consolidation is fresh pure-Python.
  • utils/calcMiso.py is not imported by this package; all orientation math comes from midas-stress.
  • TOMO/midas_tomo_python.py is imported in place, not relocated.

Project links

Data verified by PyPI on Aug 16, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.15.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for marinerhemant from gravatar.com marinerhemant

Credits

Author: Hemant Sharma

GitHub Statistics

Data verified by PyPI on Aug 16, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

dev notebook fast sr

Tags

MIDAS HEDM FF-HEDM PF-HEDM scanning diffraction pipeline PyTorch multi-detector crystallography polycrystal tomography

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

midas_pipeline-0.13.0.tar.gz (214.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

midas_pipeline-0.13.0-py3-none-any.whl (249.5 kB view details)

Uploaded Python 3

File details

Details for the file midas_pipeline-0.13.0.tar.gz.

File metadata

  • Download URL: midas_pipeline-0.13.0.tar.gz
  • Upload date:
  • Size: 214.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for midas_pipeline-0.13.0.tar.gz
Algorithm Hash digest
SHA256 7f022d75b0e415bfc64bcfc84e9cf90f62da8a71d60023985fbcd0330f973e43
MD5 bc2173cc5d01a508a1981759c6e705c1
BLAKE2b-256 296c6ee4bfe67eb6c70d80f286ddde8ebfed91051195de8901c8d152673b2e02

See more details on using hashes here.

Provenance

The following attestation bundles were made for midas_pipeline-0.13.0.tar.gz:

Publisher: python-packages.yml on marinerhemant/MIDAS

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file midas_pipeline-0.13.0-py3-none-any.whl.

File metadata

  • Download URL: midas_pipeline-0.13.0-py3-none-any.whl
  • Upload date:
  • Size: 249.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for midas_pipeline-0.13.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1a0ebf5e41ad1b5a4dc0956859b596f0aa7d5fa897563b8230e7e7fce7905c80
MD5 1141e3d9f5d5a6a725c5251824a37c81
BLAKE2b-256 df2789ad64d748de80a20c1fd04535703caa2092c6365dc25a5c5de1913cc77a

See more details on using hashes here.

Provenance

The following attestation bundles were made for midas_pipeline-0.13.0-py3-none-any.whl:

Publisher: python-packages.yml on marinerhemant/MIDAS

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.15.2

2 files

0.15.1

2 files

0.15.0

2 files

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

This release

0.13.0 This release

2 files

0.12.0

2 files

0.11.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.0

2 files

0.6.1

2 files

0.6.0

2 files

0.5.1

2 files

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.5

2 files

0.4.1

2 files

0.4.0

2 files

0.3.0

2 files

0.2.3

2 files

0.2.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page