midas-pipeline

End-to-end MIDAS HEDM orchestrator. FF is the single-scan degeneracy of PF. One package, one CLI, two scan modes.

Status

0.1.0 — end-to-end PF and FF paths live. The scanning indexer matches the C IndexerScanningOMP reference on its 1-voxel C-parity gate (seed identity, solution counts, voxel-center positions exact; orientation matrices within mrad-scale, the refiner closes the gap downstream). Real-data validation: Wenxi CP-Ti consolidation_pf reproduces the legacy pf_MIDAS.py grain count (770 == 770, all common). Park22 P5c parity gate now runs in ~6.7s vs the original 790s after the scanning-indexer position-grid fix.

Stages call in-process Python kernels via midas-index / midas-fit-grain / midas-transforms / midas-stress. FF mode shells out to python -m midas_index and python -m midas_fit_grain (same kernels, subprocess for the FF parity-preserving pattern). No CUDA C; GPU is torch-only.

Install

pip install -e packages/midas_pipeline

Notebooks

Worked-example Jupyter notebooks live in notebooks/. They are not shipped with pip install — get them by cloning the MIDAS repository.

CLI

midas-pipeline run --scan-mode {ff,pf,auto} --params Parameters.txt --result rundir/
midas-pipeline status rundir/
midas-pipeline resume rundir/ --from <stage>
midas-pipeline reprocess rundir/
midas-pipeline inspect rundir/LayerNr_1/
midas-pipeline simulate --out simdir/ --params Parameters.txt
midas-pipeline seed --params ... --output UniqueOrientations.csv

When --scan-mode is omitted (default auto), the CLI sniffs the parameter file: nScans > 1 or presence of BeamSize / scanning keys → pf, otherwise ff. For PF mode, --n-scans, --scan-step, --beam-size, and --scan-pos-tol default to values in the params file (CLI flags override).

Indexer backend

midas-pipeline run --indexer-backend {python,c-omp} ...

python (default) — in-process numba/torch indexer. Portable (CPU/CUDA/MPS), differentiable, slower on large PF datasets.

c-omp — bundled unified C binary (midas_indexer) from midas-index. Requires midas-index installed with a working OpenMP toolchain (macOS: brew install libomp). ~290× faster than the Python path on real PF datasets (per the Wenxi CP-Ti benchmark in packages/midas_index/dev/). Output is bit-identical to the Python path on the PF parity gate.

Coexistence with midas-ff-pipeline

The legacy midas-ff-pipeline console-script is preserved as an independent FF orchestrator (its own kernels, its own CLI). It is not deprecated by midas-pipeline run --scan-mode ff — both paths invoke the same midas-index / midas-fit-grain kernels under the hood, and both stay green on the FF parity gate. Pick whichever is more convenient for your workflow.

Architecture

  • One orchestrator with a mode-dependent STAGE_ORDER.
  • Shared kernel packages (midas-index, midas-fit-grain, midas-transforms, etc.) extended in place; FF behavior preserved by parity gates.
  • PF-only modules live inside midas_pipeline (find_grains/, sinogen, recon/, fuse, potts, em_refine, seeding/).
  • Differentiability + multi-device mandatory on every new compute path (CPU / CUDA / MPS via torch).

Constraints

  • No CUDA C; GPU support is torch-only.
  • No deletions of legacy code in this effort.
  • midas-process-grains is FF-only; PF consolidation is fresh pure-Python.
  • utils/calcMiso.py is not imported by this package; all orientation math comes from midas-stress.
  • TOMO/midas_tomo_python.py is imported in place, not relocated.

Project links

Data verified by PyPI on Jul 29, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.15.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for marinerhemant from gravatar.com marinerhemant

Credits

Author: Hemant Sharma

GitHub Statistics

Data verified by PyPI on Jul 29, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

dev notebook fast

Tags

MIDAS HEDM FF-HEDM PF-HEDM scanning diffraction pipeline PyTorch multi-detector crystallography polycrystal tomography

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

midas_pipeline-0.6.1.tar.gz (187.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

midas_pipeline-0.6.1-py3-none-any.whl (224.1 kB view details)

Uploaded Python 3

File details

Details for the file midas_pipeline-0.6.1.tar.gz.

File metadata

  • Download URL: midas_pipeline-0.6.1.tar.gz
  • Upload date:
  • Size: 187.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for midas_pipeline-0.6.1.tar.gz
Algorithm Hash digest
SHA256 9a34952a696f3154bf927e73555ecd54e51d7819fd95800b8bbeae57f3545124
MD5 96ed214571f6fa5dbbf2984b4412191e
BLAKE2b-256 999284c64ce124094681c20d13ffdab0cd1f80119cf094104ef71eb61100db40

See more details on using hashes here.

Provenance

The following attestation bundles were made for midas_pipeline-0.6.1.tar.gz:

Publisher: python-packages.yml on marinerhemant/MIDAS

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file midas_pipeline-0.6.1-py3-none-any.whl.

File metadata

  • Download URL: midas_pipeline-0.6.1-py3-none-any.whl
  • Upload date:
  • Size: 224.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for midas_pipeline-0.6.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d4d9b2bfc5fe136bbe1d7ed4ea6731e94caeae41c14c028d32b2f100634a218b
MD5 20468cdc0023fe8e82424af74c706a93
BLAKE2b-256 46f9ac2344a0ea1cfc6ec096e1e01a09f1866cf16881768b71e7bce6c9dd56bb

See more details on using hashes here.

Provenance

The following attestation bundles were made for midas_pipeline-0.6.1-py3-none-any.whl:

Publisher: python-packages.yml on marinerhemant/MIDAS

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.15.2

2 files

0.15.1

2 files

0.15.0

2 files

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.11.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.0

2 files

This release

0.6.1 This release

2 files

0.6.0

2 files

0.5.1

2 files

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.5

2 files

0.4.1

2 files

0.4.0

2 files

0.3.0

2 files

0.2.3

2 files

0.2.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page