pymdkit

A Python command-line toolkit for high-throughput atomistic workflows in AI-driven materials design. Install pymdkit once and call its tools from anywhere as pmk <command> [options].

Commands expose explicit, documented CLI interfaces designed for reproducible shell, HPC, and AI-agent workflows, and each underlying script is still runnable on its own.

Install

Online installation

When Internet access is available, install from PyPI. This is the recommended method:

pip install pymdkit

This installs the pmk command into the active Python or conda environment, together with its direct dependencies, including ASE, pymatgen, py4vasp-core, h5py, dpdata, matplotlib, UMAP, and tqdm. Some packages also install transitive dependencies.

Complete offline installation

For an HPC system or another network-isolated machine, prepare a complete local package bundle. Transferring only the pymdkit wheel is not sufficient for a fresh offline environment when its required dependencies are not already installed.

On a machine with Internet access, run:

pip download pymdkit==1.7.0 -d pymdkit-offline

This downloads pymdkit and the dependencies resolved by pip into the pymdkit-offline directory. Transfer that entire directory to the offline HPC system. Then, inside the target pymdkit Python or conda environment, run:

pip install --no-index --find-links=pymdkit-offline pymdkit==1.7.0

• --no-index prevents pip from accessing PyPI.
• --find-links=pymdkit-offline tells pip to install packages from the transferred local directory.

Compiled dependencies can have platform- and Python-version-specific wheels. Although the pymdkit wheel may be platform-independent (py3-none-any), packages such as NumPy and SciPy may not be. Prepare the offline bundle for a Python version and platform compatible with the target HPC environment; wheels downloaded for Windows will not necessarily work on Linux. Ideally, prepare the offline bundle on a Linux system with a Python version compatible with the target HPC environment.

Verify:

pmk -version
pmk -help                   # lists every command
pmk <command> -help         # shows that command's flags

Global configuration and logs

On first use, PMK creates .pmkrc.yaml in the active Python environment root. For example, an executable at ~/anaconda3/envs/pymdkit/bin/pmk uses ~/anaconda3/envs/pymdkit/.pmkrc.yaml. View or update configuration with:

pmk config
pmk config silence -on
pmk config silence -off
pmk config pmg-mp-api <MP_API_KEY>
pmk config pmg-vasp-potcar -type <TYPE> -path <PATH>

The two pmg-* forms write their pymatgen keys to both the active PMK .pmkrc.yaml and ~/.config/.pmgrc.yaml, creating the latter and its parent directory when needed. Existing unrelated keys are preserved and matching keys are replaced. pmg-mp-api stores PMG_MAPI_KEY; its value is redacted from terminal configuration listings and command logs.

pmg-vasp-potcar expands and resolves PATH to an absolute PMG_VASP_PSP_DIR and stores TYPE verbatim as PMG_DEFAULT_FUNCTIONAL. The resolved directory must contain <PATH>/POT_GGA_PAW_PBE/. Validation happens before either configuration file is updated.

With silence enabled, successful commands do not print to the terminal. Errors that prevent an operation are still shown. Command details are never discarded: every normal, non-help pmk <command> ... invocation appends its command, timestamp, exit status, and screen output to <command>.log in the directory where PMK was invoked. Repeated runs remain in chronological order in the same file. Values registered by a command as sensitive are shown as <redacted> in the logged command line. The log stays in the invocation directory even when a command changes the working directory or -of selects an output folder.

Command help (pmk <command> -h or -help) creates no execution log and does not run the command. Scientific result files such as rmsd.txt, ehull.txt, d2o.txt, and quantity reports remain separate command-owned artifacts; their contents are never merged into the execution log.

Commands

Commands that transform structures accept a single file (-i/-o), a folder (-if/-of), or a trajectory where applicable. PMK delegates ordinary format recognition to ASE, while CIF uses the specialized pymatgen path needed to preserve occupancy, oxidation, symmetry, and Wyckoff metadata.

Examples

pmk convert -i NaCl.vasp -o NaCl.xyz                    # always extended XYZ
pmk convert -i NaCl.vasp -o NaCl.xyz -config-type Fm-3m
pmk convert -if structures -of structures-pdb -oe pdb
pmk convert -it trajectory.traj -ot trajectory.xyz      # ASE trajectory conversion
pmk convert -i NaCl.vasp -o NaCl-symm.cif -symm
pmk convert -if cifs -of ordered -d2o -supercell-matrix 1x1x1 -ncore 64

pmk supercell -i model.vasp -o model-sc.vasp -matrix 1x1x2
pmk supercell -if structures -of capped -max-abc 40
pmk supercell -if structures -of close -close-abc 50

pmk gpumd-job -if Li3YCl6-all -supercell-close-abc 50 -nep nep.txt -in run.in -temp 300 320 340 360 -group Li Y Cl
pmk gpumd-job -it train.xyz -supercell-matrix 2x2x2 -nep nep.txt -group Li Y Cl
pmk submit -gpumd
pmk submit -gpumd -subscript custom_gpumd.pbs -queue pbs -max-job-num 30

pmk vasp-job -if Li3YCl6-all -incar INCAR
pmk vasp-job -it train.xyz -incar INCAR -supercell-max-abc 40
pmk submit -vasp -subscript sub_vasp -queue slurm -max-job-num 30
pmk vasp-job -status -queue slurm
nohup pmk submit -vasp -queue slurm -max-job-num 30 > submit-terminal.out 2>&1 &

pmk trjconv -i example.xyz -b 10 -e 20 -o frames-10-20.xyz
pmk trjconv -i example.xyz -dt 10 -o every-10-frames.xyz
pmk trjconv -i example.xyz -b 10 -e 20 -timestep 1 -o time-10-20ps.xyz
pmk trjconv -i example.xyz -dt 10 -timestep 1 -o every-10ps.xyz

pmk rd -pca -i descriptor.out -o pca-descriptor.txt -it train.xyz
pmk rd -pca -i descriptor.out -fps 0.01 -it train.xyz
pmk rd -umap -i descriptor.out -o umap-descriptor.txt -fps 0.01 -it train.xyz

pmk mp-chemsys -s Li La Ta Cl
pmk gpumd-out -msd
pmk gpumd-out -thermo
pmk gpumd-out -save-fs -of gpumd-opted
pmk vasp-out -energy -force -stress
pmk vasp-out -born-effective-charge -dielectric-tensor
pmk vasp-out -all
pmk vasp-out -ehull -local Li-La-Ta-Cl-stable-entries-opted
pmk vasp-out -ehull -ref-form-e -1.1
pmk vasp-out -nep-train
pmk vasp-out -nep-train -position-only
pmk vasp-out -save-fs -fs-name CONTCAR -of vasp-opted
pmk dope -i Li3YCl6.cif -se Li -sn 1 -we Na -on 20 -seed 7
pmk ewald -if cifs -ncore 64
pmk realloc-occ -if cifs -of disorder -formula Li3YCl6 -occ 0.25 0.5 0.75 1 -ncore 64
pmk rmsd -if vasp-opted -rm-duplicate -of unique -gpu -ncore 64

File I/O and XYZ

convert uses ASE read, iread/multi-frame indexing, format detection, and write for ordinary formats supported by the installed ASE version. This covers single structures, files in a folder, and trajectory conversion whenever the selected format supports multiple configurations. Run ase info --formats to inspect the exact formats available in that environment. Specialized CIF reading/writing remains pymatgen-based to preserve PMK's disorder and symmetry metadata.

For CIFs with explicit symmetry multiplicities, PMK validates every labeled Wyckoff orbit and the occupied composition after pymatgen expands the asymmetric unit. If finite coordinate noise splits a special position, PMK retries with the smallest bounded site tolerance that restores the declared multiplicities. It never accepts an expansion with different stoichiometry; an irreconcilable CIF stops with an explicit error instead of continuing with the wrong atoms.

Every newly exported .xyz file is written as extended XYZ (extxyz), including single structures and trajectories. Commands that select existing trajectory frames preserve already-extended source blocks byte-for-byte; if the source is plain XYZ, selected frames are upgraded to extxyz. convert -config-type NAME adds or replaces Config_type in every output XYZ frame and supports -i, -if, and -it conversion modes.

Trajectory conversion

Without -timestep, trjconv -b/-e uses one-based, inclusive structure numbers. -dt N retains the first frame and then every Nth frame. With -timestep, every frame must contain a numeric Time= value and physical time is Time * timestep in ps; -b, -e, and -dt are then interpreted in ps. The output format is inferred from -o, and .xyz output is always extxyz.

Dimension reduction

rd requires exactly one of -pca or -umap. Descriptor rows remain aligned with -it trajectory frames. Config_type-specific descriptor files and FPS trajectory/descriptor outputs retain the earlier PCA/UMAP behavior; FPS is run once globally and the selection is then partitioned by Config_type.

GPUMD jobs

gpumd-job accepts exactly one of -i, -if, or -it and optionally one of -supercell-matrix, -supercell-max-abc, or -supercell-close-abc. -nep and -in default to nep.txt and run.in in the invocation directory. Each job receives model.xyz as extxyz, the potential as nep.txt, and the input as run.in; the potential line is synchronized to nep.txt. With -temp, PMK creates <structure>/<T>/, replaces every target_t token with T, applies -group to all model files, and normalizes npt_scr pressure components for the actual orthorhombic or triclinic cell. In -if mode, an existing final job folder (including an empty one) is left untouched while missing siblings are still generated. With -temp, every structure/temperature folder is checked independently.

gpumd-job -status recursively treats every folder containing gpumd.out as a job and prints its path relative to the invocation directory. A minimization is converged only when its completion phrase is present and its final f_max is at or below the parsed force tolerance; a completed minimization above tolerance is not converged, and missing required values or completion is incomplete. A non-minimization is finished only when it has a valid Time used = ... s line. The final valid runtime line is reported. Summary counts are total, finished, converged, not converged, and incomplete; converged and not-converged minimizations both count as finished.

Job submission

pmk submit (-vasp | -gpumd) [-subscript FILE] [-queue slurm|pbs] [-max-job-num INTEGER] [-sleep SECONDS]

Exactly one of -vasp or -gpumd is required. The queue defaults to Slurm, the active-job limit to 30, and the wait between full-queue checks to 60 seconds. VASP uses sub_vasp by default; GPUMD uses sub_gpumd. A custom script is copied into every submitted job folder, and successful scheduler IDs are saved in .pmk-job-id. User-facing job names are relative to the invocation directory, and execution details are recorded in submit.log.

submit -gpumd recursively submits only folders containing model.xyz, run.in, and nep.txt. It applies the common Slurm/PBS queue limiter without checking gpumd-job -status, inspecting convergence, or creating restart artifacts.

GPUMD outputs

gpumd-out requires exactly one operation:

pmk gpumd-out -thermo [-entry-path DIR] [-thermo-file FILE] [-run-input FILE]
pmk gpumd-out -msd [-entry-path DIR] [-diffuse-ion ELEMENT] [-ion-charge INTEGER]
pmk gpumd-out -save-fs [-entry-path DIR] [-fs-name FILENAME] (-of DIR | -output-folder DIR)

-entry-path defaults to .. Thermo mode defaults to thermo.out and run.in, preserving the per-job text tables, thermo.png, and abnormal-time.txt when an abnormal interval is detected. MSD mode defaults to mobile ion Li with charge 1, writes sigma-sdc-summary.txt at the entry root, and preserves each job's msd/group_*_msd.txt, group_*_sdc.txt, and group_*_results.txt. The migration aliases -diffuse_ion and -ion_charge remain accepted. Save-fs mode defaults to relaxed.xyz and gathers converged structures into the explicitly requested output folder. Every executed mode uses gpumd-out.log; scientific artifacts remain separate from that log. The standalone gather-fs command is removed now that GPUMD and VASP gathering are available as gpumd-out -save-fs and vasp-out -save-fs.

The standalone supercell command is now structure-only. Its removed -individual, -temp, -md-if, and -gpumd-group workflows belong to gpumd-job. In v1.7.0, the standalone gpumd-group and gpumd-trjcat commands are removed; grouping needed during job creation remains available as gpumd-job -group.

E_hull reference

vasp-out -ehull -ref-form-e EV_PER_ATOM keeps the normal Materials Project or -local formation-energy calculation, but replaces the phase-diagram hull distance with E_hull = E_form - EV_PER_ATOM. All scanned structures must have the same reduced formula because one scalar reference applies to one composition. The exact difference is retained, including a negative value when a structure lies below the supplied reference.

VASP jobs

pmk vasp-job (-i FILE | -if DIR | -it FILE) [-o DIR] [-incar FILE]
             [-supercell-matrix N... | -supercell-max-abc L | -supercell-close-abc L]
pmk vasp-job -status [-queue slurm|pbs]

The three generation modes create one job folder per structure. PMK uses pymatgen to generate POSCAR, POTCAR, and KPOINTS, then preserves the supplied INCAR file exactly in each job. All three supercell modes are available during generation. In -if mode, an existing final folder (including an empty one) is skipped as a whole, so neither input generation nor the later INCAR copy changes it; other missing targets in the batch are still generated. During vasp-job input generation only, PMK hides pymatgen's BadInputSetWarning that says a POTCAR symbol is not recognized for the selected user_potcar_functional. Other BadInputSetWarning messages and unrelated warnings remain visible. Submission has moved to submit -vasp, which retains the previous convergence-aware Slurm/PBS queue limiter: jobs are discovered recursively, converged jobs are skipped, unconverged relaxations restart from CONTCAR, and the process sleeps while the active queue is full. Use nohup ... & when the submit/check loop must survive logout. The old vasp-job -submit form and its submission-only options are no longer accepted.

vasp-job -status recursively reports each prepared job and summary counts for total, finished, converged, failed, and running. Job identifiers preserve nested paths relative to the invocation directory. Submitted scheduler IDs are stored in each job folder so running jobs can be identified with squeue or qstat. For vaspout.h5, status uses h5py to read only the final energy, final positions, required INCAR scalars, and the final force slice when checking a relaxation. It does not construct a py4vasp Calculation or load complete ionic trajectories. VASP output parsing still follows vaspout.h5 > vasprun.xml > OUTCAR.

VASP outputs

vasp-out has four mutually exclusive operation families:

pmk vasp-out [-entry-path DIR] [-o DIR | -output DIR] (-all | QUANTITY_FLAG...)
pmk vasp-out -ehull [-entry-path DIR] [-mp-api-key KEY] [-local DIR] [-ref-form-e EV_PER_ATOM]
pmk vasp-out -nep-train [-entry-path DIR] [-nep-train-output FILE] [-position-only]
pmk vasp-out -save-fs [-entry-path DIR] [-fs-name FILENAME] (-of DIR | -output-folder DIR) [-max-ehull EV_PER_ATOM] [-ehull-file FILE]

Quantity extraction retains its existing grammar and scans VASP output files in the current path and every descendant folder. Select any combination of quantity flags, or use -all to attempt all 39 quantities listed by py4vasp Calculation, including namespaced quantities such as -electron-phonon-transport and -phonon-mode. Underscore/dotted spellings from the API are also accepted; see pmk vasp-out -h for the exact list.

Reader priority is applied independently for every requested quantity: vaspout.h5, then vasprun.xml with pymatgen, then OUTCAR with pymatgen. For vaspout.h5, final energy is read directly with h5py as one scalar hyperslab; other quantities use py4vasp. The fast path follows VASP's existing dataset and chunk layout and never rewrites or rechunks the file. If a preferred file exists but does not contain that quantity, PMK continues to the next reader. Energy always means the final energy(sigma->0) value (e_0_energy in pymatgen Vasprun data). Older or nonstandard HDF5 layouts fall back to py4vasp.

Quantity results remain one text report per quantity under vasp-out/, such as energy.txt, force.txt, and stress.txt. Numeric values use eight decimal places. Forces include element-number labels; stresses are reported in both GPa and eV/ų using 1 eV/ų = 160.2176621 GPa. Multiple calculations are identified by relative folder path. -all skips unavailable datasets per calculation instead of aborting the scan.

E-hull mode preserves Materials Project/local-reference handling, cache files, deterministic ordering, and writes ehull.txt. A supplied -mp-api-key is redacted in vasp-out.log. NEP-training mode preserves converged VASP structures and extxyz energy/force/stress metadata; its default output is train.xyz, while -position-only omits calculation results. Final-structure mode defaults to CONTCAR, requires an output folder, and optionally applies the strict -max-ehull filter using ehull.txt by default. All modes use vasp-out.log, separate from scientific artifacts. The standalone ehull and vasp2xyz commands are removed.

Doping structures

pmk dope -i FILE -se SELECTOR... [-sn NUMBER...] -we ELEMENT|none... [-wn NUMBER...] [-on NUMBER] [-ref SELECTOR...] [-d DISTANCE...] [-o DIR | -output-folder DIR] [-output-format FORMAT] [-seed INTEGER]

dope replaces substitute and preserves its validation, scientific substitution/removal behavior, and deterministic -seed behavior. Random mode defaults to <input-stem>_dope/; reference-distance mode defaults to <input-stem>_dope.cif. Execution details are written to dope.log. The standalone substitute command is removed.

Breaking changes in v1.7.0

• gpumd-thermo and gpumd-msd are replaced by gpumd-out -thermo and gpumd-out -msd.
• gather-fs is replaced by gpumd-out -save-fs or vasp-out -save-fs.
• ehull and vasp2xyz are replaced by vasp-out -ehull and vasp-out -nep-train.
• substitute is replaced by dope.
• vasp-job -submit is replaced by submit -vasp; GPUMD submission uses submit -gpumd.
• Standalone gpumd-group tagging and gpumd-trjcat are removed without replacement. gpumd-job -group remains available during job creation.

Other behavior

convert -d2o first symmetrizes the disordered structure so each distinct Wyckoff orbit keeps its own occupancy constraint. It then uses pymatgen's branch-and-bound Ewald ordering to find the lowest-energy ordered structure for the fixed supercell matrix. This avoids the previous greedy solution and prevents atoms from being redistributed between unrelated but chemically identical orbits.

There is deliberately no convert -d2o -gpu flag: symmetry grouping and the irregular branch-and-bound search are CPU/control-flow work, so transferring small candidate sets to CUDA would add overhead without accelerating the search.

Multicore CPU

Independent CPU-heavy inputs are processed concurrently by convert -d2o, ewald -if, realloc-occ -if, and exact folder-pair comparisons in rmsd. Use -ncore N to choose the number of worker processes, or -ncore 1 to disable multiprocessing. Without -ncore, PMK uses the single-node CPU allocation reported by Slurm, PBS, or SGE and never creates more workers than there are independent tasks. Outside a scheduler, automatic mode is capped at four workers to avoid unexpectedly occupying an entire workstation. Numerical libraries are limited to one native thread in each worker to prevent nested thread oversubscription.

convert -d2o is more memory-intensive than the other parallel commands because every worker runs a pymatgen branch-and-bound/Ewald search. Its automatic scheduler mode is therefore capped at eight workers. Use -ncore N to override that cap when the node has enough memory. The command prints its selected worker count before starting and reports progress whenever a CIF finishes; final files and reports remain deterministically ordered. rmsd -if folder comparisons show one tqdm bar for the exact pairs scheduled, and convert -d2o shows one bar for all scheduled CIFs. These bars appear only on an interactive terminal: silence mode and redirected/non-TTY execution suppress them, and progress rendering is not stored in command execution logs.

If one CIF remains active for more than 60 seconds, PMK terminates only that CIF's worker, skips its conversion, copies the original input into large-d2o-combination in the invocation directory, and starts the next queued CIF. This applies to both convert -d2o -i and convert -d2o -if; d2o.txt records the timeout, elapsed time, and copied path. The folder is created only when at least one calculation times out.

For one PMK disorder-to-order process using eight CPU cores on one Slurm node:

#!/bin/bash
#SBATCH -J Nwyd-pmk
#SBATCH -p normal
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8

source ~/softwares-env/anaconda.sh
conda activate pymdkit

pmk convert -if Li4YCl6-disorder/ -of Li4YCl6-disorder-order -d2o -ncore 8

Prefer one Slurm task with --cpus-per-task for this local multiprocessing model. The older single-node form -N 1 with -n N is detected through SLURM_NTASKS, but it describes multiple Slurm tasks rather than one multicore PMK process. Keep the job on one node. The project audit intentionally leaves HDF5/VASP scans, GPUMD plotting, trajectory copying, scheduler submission, network/API commands, and NumPy/UMAP reductions outside this process pool: those paths are I/O-bound, own their native parallelism, or write shared state. realloc-occ.txt and every generic <command>.log execution log stay in the invocation directory. Silence mode suppresses successful command output, but pmk <command> -h always displays help and operation-blocking errors are always visible.

Each command's full flag list is available with pmk <command> -h. Automatic scan modes include matching files directly in the current path as well as files in folders, subfolders, and deeper descendants. This rule applies to shared structure-folder processing, VASP calculations, GPUMD outputs, and trajectory gathering; explicit -i and -if modes remain available.

Layout

pymdkit/
|-- pyproject.toml              # package metadata and `pmk` entry point
|-- README.md
`-- src/pymdkit/
    |-- pymdkit_main.py         # command discovery and dispatch
    |-- _config.py              # environment-wide .pmkrc.yaml settings
    |-- _logging.py             # execution-log argument redaction helpers
    `-- commands/
        |-- convert.py          # ASE/pymatgen conversion and CIF workflows
        |-- trjconv.py          # trajectory extraction and stride
        |-- rd.py               # PCA/UMAP dimensional reduction
        |-- gpumd_job.py        # GPUMD job generation
        |-- vasp_job.py         # VASP generation and status
        |-- chemsys_entry.py    # `mp-chemsys` implementation
        |-- ...                 # remaining public commands
        `-- _*.py               # private shared helpers

Modules whose name starts with _ are shared helpers and are skipped by the dispatcher, so they never appear as commands.

Adding a new tool later

Drop a module in src/pymdkit/commands/ that defines four things:

COMMAND = "my-tool"                 # the subcommand name you'll type
HELP = "One-line description."

def add_arguments(parser):          # register flags
    parser.add_argument("-input", required=True)

def run(args):                      # do the work; return an exit code (0 = ok)
    ...
    return 0

if __name__ == "__main__":          # keeps the script runnable on its own
    import argparse
    _p = argparse.ArgumentParser(description=__doc__)
    add_arguments(_p)
    raise SystemExit(run(_p.parse_args()))

It will appear in pmk -help automatically - no central registration needed. Put heavy imports (pymatgen, ase, ...) inside run() where practical; the dispatcher reads each command's name and help without importing it, so pmk -help stays fast and a missing optional dependency only affects the one command that needs it.

Running a script standalone

Every command module still works directly, which is handy for debugging:

python src/pymdkit/commands/supercell.py -i in.cif -close-abc 20 -o sc.vasp