mat73-reader 0.1.0

Read MATLAB v7.3 HDF5 .mat files that scipy.io.loadmat cannot handle.

mat73-reader

Read MATLAB v7.3 HDF5 .mat files in Python. including MATLAB table objects that other tools cannot decode.

The MATLAB Table Problem

MATLAB v7.3 stores table objects using an undocumented internal system called MCOS (MATLAB Class Object System). Every existing Python tool (scipy.io.loadmat(), mat73, hdf5storage) fails on them:

# scipy can't even open v7.3 files
>>> scipy.io.loadmat("experiment.mat")
NotImplementedError: Please use HDF reader for matlab v7.3 files

# mat73 opens the file but returns None for every table
>>> import mat73
>>> data = mat73.loadmat("experiment.mat")
ERROR: MATLAB type not supported: table, (uint32)   # x 800
>>> data["task"]["gaze"][0]
None

Tables are used extensively in neuroscience, signal processing, cognitive science, biomechanics, and clinical research datasets. If your .mat file contains tables, mat73-reader is currently the only Python tool that can read them.

>>> from mat73_reader import load
>>> data = load("experiment.mat")
>>> data["task"]["gaze"][0]
      gaze_timestamp  world_index  confidence  norm_pos_x  norm_pos_y  ...
0        5410.551714          0.0    0.999499    0.446264    0.846886  ...
1        5410.555834          0.0    0.999653    0.446534    0.847007  ...
2        5410.559773          0.0    0.999648    0.446660    0.846410  ...
...
[8205 rows x 21 columns]

How It Works

When other tools encounter a MATLAB table, they see a (1,6) uint32 header and stop. mat73-reader decodes the MCOS block structure to follow the reference chain to the actual data:

graph TD
    subgraph "What other tools see"
        A["Table Header<br/>(1,6) uint32<br/>0xDD000000 ..."] -->|"???"| B["None"]
    end

    subgraph "What mat73-reader decodes"
        H["Table Header<br/>(1,6) uint32"] -->|"instance index"| M["MCOS Reference Array<br/>#subsystem#/MCOS"]
        M -->|"block offset + 0"| D["Column Data Refs<br/>(ncols, 1) object"]
        M -->|"block offset + 5"| N["Column Name Refs<br/>(ncols, 1) object"]
        D -->|"dereference"| D1["timestamp<br/>float64 (1, N)"]
        D -->|"dereference"| D2["confidence<br/>float64 (1, N)"]
        D -->|"dereference"| D3["...<br/>float64 (1, N)"]
        N -->|"dereference"| N1["'gaze_timestamp'<br/>uint16 chars"]
        N -->|"dereference"| N2["'confidence'<br/>uint16 chars"]
        N -->|"dereference"| N3["'...'<br/>uint16 chars"]
        D1 & D2 & D3 & N1 & N2 & N3 -->|"assemble"| DF["pandas DataFrame"]
    end

    style B fill:#ff6b6b,color:#fff
    style DF fill:#51cf66,color:#fff

Each table instance occupies a fixed block of 7 consecutive entries in the MCOS reference array:

Block layout (7 slots per table):
  +0  (ncols, 1) object refs   --> column data arrays (float64, int, etc.)
  +1  (1, 1) float64           --> ndims
  +2  (1, 1) float64           --> nrows
  +3  (2,) uint64              --> segment info
  +4  (1, 1) float64           --> nvars (number of columns)
  +5  (ncols, 1) object refs   --> column name strings (uint16-encoded)
  +6  Group                    --> table properties (units, descriptions, etc.)

The instance index from the table header maps to a block offset:

block_start = 2 + (instance - 1) * 7

This structure is not documented by MathWorks. It was reverse-engineered by analyzing real-world scientific datasets.

Real-World Validation

mat73-reader has been validated against the COLET dataset (Cognitive workLoad Estimation via Eye-Tracking), a 3.8 GB MATLAB v7.3 file containing:

• 47 subjects, 4 tasks per subject
• 4 data fields per task (gaze, pupil, blinks, annotation)
• 752 MATLAB table objects total
• Over 14,000 individual data arrays

Every table was successfully decoded into a pandas DataFrame with correct column names and data types. Other Python tools return None for all 752 tables.

Installation

pip install mat73-reader

Or install from source:

git clone https://github.com/WilliamGarrow/mat73-reader.git
cd mat73-reader
pip install -e ".[dev]"

Usage

Python API

from mat73_reader import load, inspect

# Inspect file contents without loading data
variables = inspect("experiment.mat")
for var in variables:
    print(var)

# Load everything
data = load("experiment.mat")

# Load a specific top-level variable
results = load("experiment.mat", variable="results")

# Force all compatible arrays to pandas DataFrames
data = load("experiment.mat", as_dataframe=True)

MATLAB tables are always returned as DataFrames automatically, no flags needed.

Command Line

# List variables, types, and shapes
mat73-reader inspect experiment.mat

# Convert to CSV (one file per variable)
mat73-reader convert experiment.mat --format csv --output ./csv_output/

# Convert to JSON
mat73-reader convert experiment.mat --format json --output experiment.json

# Convert a single variable
mat73-reader convert experiment.mat --variable gaze_data --format csv

What It Handles

MATLAB Type	Python Type	Notes
Table objects	pandas.DataFrame	Column names and data types preserved
Numeric arrays	numpy.ndarray	Transposed to row-major order
Structs	dict	Nested to arbitrary depth
Cell arrays	list	HDF5 object references resolved
Char arrays	str	Decoded from uint16
Scalars	Python int/float	Single-element arrays squeezed

When to Use This vs. Other Tools

Scenario	Tool
.mat v5 or earlier (no tables)	scipy.io.loadmat()
.mat v7.3 with arrays and structs only	mat73 or mat73-reader
.mat v7.3 with table objects	mat73-reader (only option in Python)
Not sure what format you have	Try mat73-reader first; it will tell you if it's not v7.3

Development

git clone https://github.com/WilliamGarrow/mat73-reader.git
cd mat73-reader
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pytest

Test Suite

38 tests covering:

• Standard v7.3 reading (arrays, structs, cell arrays, char arrays, scalars)
• MCOS table header detection (positive and negative cases)
• Single and multi-table decoding with synthetic fixtures
• Column name extraction and data value verification
• Edge cases (non-table uint32 arrays, missing variables, invalid files)

License

Apache 2.0. See LICENSE for details.