matrix-toolkit 0.0.4

A Python toolkit for test matrix generation

matrix-toolkit is a Python library for fetching, managing, and converting sparse matrices from the SuiteSparse Matrix Collection, with built-in programmatic generation of structured test matrices (inspired by MATLAB's anymatrix). The project aims to provide a comprehensive set of structured matrices across a wide range of domains and to build an easy-to-use interface for matrix fetching and generation, used for benchmarking and testing numerical linear algebra algorithms and deep learning applications in scientific computing.

Features

SuiteSparse Integration

• Smart Search & Filter: Search matrices by size, sparsity, symmetry, domain, and more
• Multi-Backend Support: Automatic conversion to SciPy, CuPy, JAX, PyTorch formats
• Flexible Storage: Save and load matrix collections with multiple formats (NPZ, HDF5, MAT)
• Dataset Management: Create reproducible matrix datasets with train/val/test splits
• Parallel Processing: Multi-threaded downloading and processing
• CLI Tools: Command-line interface for quick operations

Anymatrix Integration

• Test Matrix Generation: Programmatic generation of 40+ well-defined test matrices
• Property Verification: Automatic checking of mathematical properties
• Comprehensive Testing: Built-in test suite for all generated matrices
• Multiple Groups: Core, gallery, and custom matrix collections
• Unified Interface: Single API for both SuiteSparse and generated matrices

PDE Matrix Generation

• 17 Equation Types: Classical PDEs, fluid mechanics, solid mechanics, electromagnetics, quantum mechanics
• Multi-Dimensional Support: Automatic assembly for 1D, 2D, and 3D problems
• Advanced Discretization: Finite differences with 2nd, 4th, and 6th order accuracy
• Flexible Boundary Conditions: Dirichlet, Neumann, Periodic, Robin, and mixed conditions
• Time Discretization: Forward/Backward Euler, Crank-Nicolson, general $\theta$-method
• Random Parameter Sampling: Uniform, Normal, Log-normal, and Latin Hypercube Sampling
• Batch Generation: Create hundreds of matrices with varying parameters for UQ
• Complex Systems: Saddle-point systems (Stokes, Navier-Stokes), coupled equations (elasticity, Maxwell)
• Automatic Assembly: Kronecker product-based efficient assembly for multi-dimensional problems
• Backend Agnostic: Generate once, use everywhere (CPU, GPU, TPU)

Supported PDE Equations

Category	Equations	Dimensions
Classical	Poisson, Heat, Wave, Convection-Diffusion, Helmholtz, Biharmonic	1D, 2D, 3D
Fluid Mechanics	Stokes, Navier-Stokes, Burgers, Advection-Diffusion	2D, 3D
Solid Mechanics	Linear Elasticity	2D, 3D
Electromagnetics	Maxwell	3D
Reaction-Diffusion	Reaction-Diffusion, Fisher-KPP, Gray-Scott	1D, 2D, 3D
Quantum Mechanics	Schrödinger, Klein-Gordon	1D, 2D, 3D

Installation

matrix-toolkit enables pip manager installation or simply github the repo for local deployment.

Basic Installation

pip install matrix-toolkit

With Optional Dependencies

# For CuPy support
pip install matrix-toolkit[cupy]

# For JAX support
pip install matrix-toolkit[jax]

# For PyTorch support
pip install matrix-toolkit[torch]

# Install all optional dependencies
pip install matrix-toolkit[all]

Development Installation

git clone https://github.com/inEXASCALE/matrix-toolkit.git
cd matrix-toolkit
pip install -e ".[dev]"

Supported Backends

Backend	Formats	Use Cases
SciPy	CSR, CSC, COO, LIL, DOK, DIA, BSR	General sparse operations
NumPy	Dense	Small to medium matrices
CuPy	CSR, CSC, COO, Dense	GPU acceleration
JAX	Dense, BCOO	Auto-diff, JIT compilation
PyTorch	COO, CSR, CSC, Dense	Deep learning integration

Quick Start

matrix-toolkit provides a comprehensive, easy-to-use interface for accessing matrices across diverse domains — ideal for benchmarking and testing numerical linear algebra algorithms and deep learning systems. The simple usage is as follows.

Basic Usage

from matrix_toolkit.core import MatrixFetcher

# Initialize fetcher
fetcher = MatrixFetcher()

# Search for matrices with specific properties
results = fetcher.search(
    rows=(1000, 50000),
    sparsity=(0.8, 0.999),
    symmetry='symmetric',
    is_real=True,
    limit=10
)

print(f"Found {len(results)} matrices")
for m in results[:3]:
    print(f"{m['matrix_id']:30s} {m['rows']:6d}×{m['cols']:6d} sparsity={m['sparsity']:.2%}")

# Fetch a specific matrix
matrix = fetcher.get_matrix('HB/494_bus', backend='scipy', format='csr')
print(f"Shape: {matrix.shape}, NNZ: {matrix.nnz}")

# Random sampling
matrices = fetcher.fetch_random(
    n=10,
    filters={'rows': (1000, 10000), 'is_real': True},
    seed=42
)
print(f"Fetched {len(matrices)} random matrices")

# Fetch a specific matrix
matrix = fetcher.get_matrix(
    'HB/494_bus',
    backend='scipy',
    format='csr'
)

Dataset Creation

from matrix_toolkit.core import MatrixFetcher
from matrix_toolkit.core.dataset_builder import MatrixDatasetBuilder

builder = MatrixDatasetBuilder()

dataset = builder.create_dataset(
    name='my_dataset',
    filters={
        'rows': (5000, 20000),
        'sparsity': (0.9, 0.999),
        'is_real': True
    },
    n_matrices=100, 
    split={'train': 0.7, 'val': 0.15, 'test': 0.15},
    seed=42,
    output_dir='./datasets',  # optional: location
    save_metadata=True  # automatically save metadata
)

print(f"Dataset created at: {dataset['path']}")
print(f"Number of matrices: {dataset['n_matrices']}")
print(f"Splits: {dataset['splits']}")

# Data has been saved to ./datasets/my_dataset/：
#   - matrix_0000.npz, matrix_0001.npz, ... (matrix data document)
#   - metadata.json (metadata)
#   - matrices.csv (matrices list)

# 加载数据集
from scipy import sparse
from pathlib import Path
import json

def load_dataset(dataset_path):
    dataset_dir = Path(dataset_path)
    
    with open(dataset_dir / 'metadata.json', 'r') as f:
        metadata = json.load(f)
    
    matrices = {}
    for split_name, indices in metadata['splits'].items():
        matrices[split_name] = []
        for idx in indices:
            matrix_file = dataset_dir / f"matrix_{idx:04d}.npz"
            matrix = sparse.load_npz(matrix_file)
            matrices[split_name].append(matrix)
    
    return matrices, metadata

matrices, metadata = load_dataset('./datasets/my_dataset')

print(f"Train set: {len(matrices['train'])} matrices")
print(f"Val set: {len(matrices['val'])} matrices")
print(f"Test set: {len(matrices['test'])} matrices")

for i, matrix in enumerate(matrices['train'][:3]):
    print(f"Train matrix {i}: shape={matrix.shape}, nnz={matrix.nnz}")

Batch Generation with Random Parameters

from matrix_toolkit.pde import PDEMatrixGenerator, PDEConfig
from matrix_toolkit.pde.random_params import LatinHypercubeSampler

# Setup parameter sampling
sampler = LatinHypercubeSampler()
sampler.add_parameter('diffusion', low=0.01, high=1.0)
sampler.add_parameter('velocity_x', low=-2.0, high=2.0)
sampler.add_parameter('velocity_y', low=-2.0, high=2.0)

# Generate 100 parameter sets
samples = sampler.sample(n=100, seed=42)

# Generate matrices
matrices = []
for params in samples:
    config = PDEConfig(
        dimension=2,
        mesh_size=32,
        coefficients=params,
        backend='scipy',
        format='csr'
    )
    gen = PDEMatrixGenerator('convection_diffusion', config)
    matrices.append(gen.generate())

print(f"Generated {len(matrices)} matrices")
print(f"Shape: {matrices[0].shape}")
print(f"Sparsity: {100*(1 - matrices[0].nnz/np.prod(matrices[0].shape)):.2f}%")

Storage Management

# Save collection
fetcher.save_collection(
    matrices,
    path='/data/my_matrices',
    format='npz',
    compression=True
)

# Load collection
loaded = fetcher.load_collection('/data/my_matrices')

CLI Usage

# Search matrices
matrix-toolkit search --rows 1000:50000 --sparsity 0.9:0.99

# Fetch a matrix
matrix-toolkit fetch --name HB/494_bus --format csr --backend scipy

# List matrices by domain
matrix-toolkit list --domain physics

# Clear cache
matrix-toolkit cache --clear

# Create dataset
matrix-toolkit dataset create --config dataset.yaml

Anymatrix Test Matrices

from matrix_toolkit.anymatrix import AnyMatrix, MatrixProperties

# Initialize anymatrix
am = AnyMatrix()

# List available matrices
groups = am.groups()  # ['core', 'gallery']
matrices = am.list('core')  # List matrices in core group

# Generate a matrix
beta_matrix = am.generate('core/beta', 10)

# Check properties
assert MatrixProperties.is_symmetric(beta_matrix)
assert MatrixProperties.is_positive_definite(beta_matrix)

# Search for matrices with specific properties
symmetric_matrices = am.search(['symmetric', 'positive definite'])

Unified Interface

from matrix_toolkit import UnifiedMatrixCollection

mc = UnifiedMatrixCollection()

# Get from anymatrix
A = mc.get('anymatrix/core/beta', 10)

# Get from SuiteSparse
B = mc.get('suitesparse/HB/494_bus', backend='scipy')

# Search both collections
results = mc.search(properties=['symmetric'])

# Verify properties automatically
mc.verify_properties('anymatrix/core/beta', 10, verbose=True)

Available Anymatrix Collections

Core Group

• beta - Symmetric positive definite matrix
• fourier - Discrete Fourier transform matrix (unitary)
• nilpot_triang - Nilpotent upper triangular
• nilpot_tridiag - Nilpotent tridiagonal
• vand - Vandermonde matrix
• circul_binom - Circulant with binomial coefficients
• stoch_cesaro - Stochastic Cesaro matrix
• tournament - Random tournament matrix
• perfect_shuffle - Perfect shuffle permutation
• collatz - Collatz conjecture matrix
• And more...

Gallery Group

• lehmer - Lehmer matrix (symmetric positive definite)
• minij - MIN(i,j) matrix
• moler - Moler matrix
• pei - Pei matrix
• clement - Clement tridiagonal
• kms - Kac-Murdock-Szego Toeplitz matrix

Running Tests

Test all anymatrix matrices

python examples/run_anymatrix_tests.py --verbose --report test_report.txt

Test specific groups

python examples/run_anymatrix_tests.py --groups core gallery

Python API

from matrix_toolkit.anymatrix.testing import run_all_tests

results = run_all_tests(verbose=True, generate_report=True)

Contributing

Contributions are welcome! Please read our Contributing Guide for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.