ipmkit

Lossless Mastercard IPM tooling and payment card utilities.

IPM → CSV → IPM gives a file identical to the original, byte for byte. Nothing is converted, trimmed or masked.

Install

pip install ipmkit

The PIN block and key modules (ipmkit.pinblock, ipmkit.key) also need pip install "ipmkit[crypto]".

Convert files

ipmkit ipm2csv T112.ipm -o T112.csv

ipmkit csv2ipm T112.csv -o T112_rebuilt.ipm

ipmkit compare T112.ipm T112_rebuilt.ipm

The encoding is always EBCDIC cp037 unless you pass --encoding (for example --encoding latin_1 for an ASCII file). ipm2csv warns if the first MTI looks like ASCII. 1014 blocking is detected automatically; use --blocked/--unblocked to override it, or --config-file for a custom field config.

ipm2csv then verifies that the CSV rebuilds the identical file. If it doesn't, it fails instead of handing you a lossy CSV.

Speed on large files

Files with 20,000 or more records are split into chunks and converted on several CPU cores, one process per core up to 8. The chunks are joined back in their original order, so records keep their exact order and the CSV is byte-identical to a single-process run. Use --workers N to choose the number of processes (--workers 1 for a single process).

100,002 records (72 MB, cp037, 1014-blocked), on a 10-core Mac:

What's in the CSV

A text value that can't sit safely in a CSV (control characters, or text that itself starts with hex:) is written as hex: followed by its bytes in hex.

Opening the CSV in Excel and saving it drops leading zeros and reformats values. Edit it with a text editor, or import every column as Text.

Editing the CSV

You can change values. Lengths for LLVAR/LLLVAR fields are recalculated and the 1014 blocking is redone. A fixed-length field with the wrong length is rejected, never silently padded or cut. To remove a field, clear its cell and its bit in BITMAP (or clear the BITMAP cell to have it recalculated).

Header and trailer generation

When the CSV has no file header (MTI 1644, DE24 697) or no trailer (MTI 1644, DE24 695), csv2ipm generates them:

• header: DE48 = PDS 0105 File ID + PDS 0122 processing mode (--processing-mode: P production, the default, or T test); DE71 = 00000001
• trailer: DE48 = PDS 0105 + PDS 0301 (sum of all DE4 amounts, 16 digits) + PDS 0306 (message count including header and trailer, 8 digits); DE71 = message count

The File ID is copied from the existing header or trailer if there is one. Otherwise it is built from --file-type, --file-date (YYMMDD, default today), --processor-id and --file-seq. A CSV made from an IPM file that had no header or trailer remembers that, so csv2ipm doesn't add one. --no-header-trailer turns generation off.

A hand-made CSV only needs MTI and DEn columns. It is written as cp037, not blocked, unless you pass --encoding / --blocked. A CSV made by ipm2csv remembers its encoding in the META row, and --encoding overrides that too.

Python API

from ipmkit import ipmcsv

ipmcsv.convert_ipm_to_csv('in.ipm', 'out.csv')          # verified
ipmcsv.convert_csv_to_ipm('out.csv', 'new.ipm')

ipm = ipmcsv.read_ipm(open('in.ipm', 'rb').read())      # IpmFile: records with raw field bytes
ipm.records[1].fields[42]                               # b'MERCHANT0000001'
data = ipmcsv.write_ipm(ipm)

Python modules

Besides ipmkit.ipmcsv, the package has:

• ipmkit.iso8583: loads and dumps for ISO 8583 messages, including PDS and DE43 sub-fields, and DE55 parsed as BER-TLV (multi-byte tags such as DF01, long-form lengths)
• ipmkit.mciipm: IPM file readers and writers (VBS and 1014 blocking), and parameter extracts
• ipmkit.card: Luhn check digits and card number masking
• ipmkit.pinblock and ipmkit.key: PIN blocks (ISO-0 and ISO-4, including 10–12 digit PINs), Visa PVV and key utilities
• ipmkit.config: the field configuration

The library functions (loads, dumps, IpmReader, IpmWriter) default to latin_1. The commands that read or write IPM files (mci_ipm_to_csv, mci_csv_to_ipm, mci_ipm_param_to_csv) default to cp037.

A value too long for its field raises an error. It is never cut off.

License

MIT. See LICENSE.