Einsum Benchmark Data

Benchmark data repository accompanying the paper: "Einsum Benchmark: Enabling the Development of Next-Generation Tensor Execution Engines". It includes a package to easily work with the benmark data and all methods we used to convert and create our instances so you can create new ones yourself.

Installation

Install the base package with Python 3.10 through 3.14:

pip install einsum_benchmark

Basic Usage

Now you can load and run an instance like this:

import opt_einsum as oe
import einsum_benchmark

instance = einsum_benchmark.instances["qc_circuit_n49_m14_s9_e6_pEFGH_simplified"]

opt_size_path_meta = instance.paths.opt_size
print("Size optimized path")
print("log10[FLOPS]:", round(opt_size_path_meta.flops, 2))
print("log2[SIZE]:", round(opt_size_path_meta.size, 2))
result = oe.contract(
    instance.format_string, *instance.tensors, optimize=opt_size_path_meta.path
)
print("sum[OUTPUT]:", result.sum(), instance.result_sum)

For more information please read our docs.

The code for our experiments is here: https://github.com/ti2-group/einsum_benchmark_experiments

Acknowledgements

The broader data collection process included contributions from individuals whose data was transformed. We duly acknowledge the following for making their data publicly available:

• Fichte, Johannes; Hecher, Markus; Florim Hamiti: Model Counting Competition 2020
• Fichte, Johannes; Hecher, Markus: Model Counting Competition 2021 2022 2023
• Fichte, Johannes; Hecher, Markus; Woltran, Stefan; Zisser, Markus: A Benchmark Collection of #SAT Instances and Tree Decompositions
• Meel, Kuldeep S.: Model Counting and Uniform Sampling Instances
• Automated Reasoning Group at the University of California, Irvine: UAI Competitions
• Martinis, John M. et al.: Quantum supremacy using a programmable superconducting processor Dataset. Dryad.

Moreover, we thank the following authors of open source software used to generated instances:

• Gray, Johnnie: quimb, cotengra
• Soos, Mate, Meel, Kuldeep S: Arjun
• Stoian, Mihail: Netzwerk
• Liu, Jinguo; Lua, Xiuzhe; Wang, Lei: Yao.jl
• Liu, Jinguo: YaoToEinsum.jl