midas-pipeline

End-to-end MIDAS HEDM orchestrator. FF is the single-scan degeneracy of PF. One package, one CLI, two scan modes.

Status

0.1.0 — end-to-end PF and FF paths live. The scanning indexer matches the C IndexerScanningOMP reference on its 1-voxel C-parity gate (seed identity, solution counts, voxel-center positions exact; orientation matrices within mrad-scale, the refiner closes the gap downstream). Real-data validation: Wenxi CP-Ti consolidation_pf reproduces the legacy pf_MIDAS.py grain count (770 == 770, all common). Park22 P5c parity gate now runs in ~6.7s vs the original 790s after the scanning-indexer position-grid fix.

Stages call in-process Python kernels via midas-index / midas-fit-grain / midas-transforms / midas-stress. FF mode shells out to python -m midas_index and python -m midas_fit_grain (same kernels, subprocess for the FF parity-preserving pattern). No CUDA C; GPU is torch-only.

Install

pip install -e packages/midas_pipeline

Notebooks

Worked-example Jupyter notebooks live in notebooks/. They are not shipped with pip install — get them by cloning the MIDAS repository.

CLI

midas-pipeline run --scan-mode {ff,pf,auto} --params Parameters.txt --result rundir/
midas-pipeline status rundir/
midas-pipeline resume rundir/ --from <stage>
midas-pipeline reprocess rundir/
midas-pipeline inspect rundir/LayerNr_1/
midas-pipeline simulate --out simdir/ --params Parameters.txt
midas-pipeline seed --params ... --output UniqueOrientations.csv

When --scan-mode is omitted (default auto), the CLI sniffs the parameter file: nScans > 1 or presence of BeamSize / scanning keys → pf, otherwise ff. For PF mode, --n-scans, --scan-step, --beam-size, and --scan-pos-tol default to values in the params file (CLI flags override).

Indexer backend

midas-pipeline run --indexer-backend {python,c-omp} ...

python (default) — in-process numba/torch indexer. Portable (CPU/CUDA/MPS), differentiable, slower on large PF datasets.

c-omp — bundled unified C binary (midas_indexer) from midas-index. Requires midas-index installed with a working OpenMP toolchain (macOS: brew install libomp). ~290× faster than the Python path on real PF datasets (per the Wenxi CP-Ti benchmark in packages/midas_index/dev/). Output is bit-identical to the Python path on the PF parity gate.

Coexistence with midas-ff-pipeline

The legacy midas-ff-pipeline console-script is preserved as an independent FF orchestrator (its own kernels, its own CLI). It is not deprecated by midas-pipeline run --scan-mode ff — both paths invoke the same midas-index / midas-fit-grain kernels under the hood, and both stay green on the FF parity gate. Pick whichever is more convenient for your workflow.

Architecture

  • One orchestrator with a mode-dependent STAGE_ORDER.
  • Shared kernel packages (midas-index, midas-fit-grain, midas-transforms, etc.) extended in place; FF behavior preserved by parity gates.
  • PF-only modules live inside midas_pipeline (find_grains/, sinogen, recon/, fuse, potts, em_refine, seeding/).
  • Differentiability + multi-device mandatory on every new compute path (CPU / CUDA / MPS via torch).

Constraints

  • No CUDA C; GPU support is torch-only.
  • No deletions of legacy code in this effort.
  • midas-process-grains is FF-only; PF consolidation is fresh pure-Python.
  • utils/calcMiso.py is not imported by this package; all orientation math comes from midas-stress.
  • TOMO/midas_tomo_python.py is imported in place, not relocated.

Project links

Data verified by PyPI on Jul 31, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.15.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for marinerhemant from gravatar.com marinerhemant

Credits

Author: Hemant Sharma

GitHub Statistics

Data verified by PyPI on Jul 31, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

dev notebook fast sr

Tags

MIDAS HEDM FF-HEDM PF-HEDM scanning diffraction pipeline PyTorch multi-detector crystallography polycrystal tomography

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

midas_pipeline-0.7.0.tar.gz (193.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

midas_pipeline-0.7.0-py3-none-any.whl (228.4 kB view details)

Uploaded Python 3

File details

Details for the file midas_pipeline-0.7.0.tar.gz.

File metadata

  • Download URL: midas_pipeline-0.7.0.tar.gz
  • Upload date:
  • Size: 193.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for midas_pipeline-0.7.0.tar.gz
Algorithm Hash digest
SHA256 16fa760c32dba9cd819152f132ab5287d686f34e0618235eb08d03f628119dc4
MD5 95a92f7dcd9de8b3d5959bc2326130d7
BLAKE2b-256 1bf3885e27350c53532afd4ab4b4e3144de2ed4635989e1f0a03d99337a75798

See more details on using hashes here.

Provenance

The following attestation bundles were made for midas_pipeline-0.7.0.tar.gz:

Publisher: python-packages.yml on marinerhemant/MIDAS

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file midas_pipeline-0.7.0-py3-none-any.whl.

File metadata

  • Download URL: midas_pipeline-0.7.0-py3-none-any.whl
  • Upload date:
  • Size: 228.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for midas_pipeline-0.7.0-py3-none-any.whl
Algorithm Hash digest
SHA256 6a09daaca938033d9e7c4570ea9dc3a4e30301de79a7b66bdb7732bbc32ab40c
MD5 4811ac10a09beff2460bf4a1a615b2fe
BLAKE2b-256 5ab2b2e8ccdce2698b63e9145b71ce0e02888719e85fa06146c1c8b2ce034208

See more details on using hashes here.

Provenance

The following attestation bundles were made for midas_pipeline-0.7.0-py3-none-any.whl:

Publisher: python-packages.yml on marinerhemant/MIDAS

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.15.2

2 files

0.15.1

2 files

0.15.0

2 files

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.11.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

This release

0.7.0 This release

2 files

0.6.1

2 files

0.6.0

2 files

0.5.1

2 files

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.5

2 files

0.4.1

2 files

0.4.0

2 files

0.3.0

2 files

0.2.3

2 files

0.2.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page