Exca - ⚔

Execute and cache seamlessly in python.

workflow badge

Quick install

pip install exca

Full documentation

Documentation is available at https://facebookresearch.github.io/exca/

Basic overview

exca provides simple decorators to:

execute a (hierarchy of) computation(s) either locally or on distant nodes,
cache the result.

The problem:

In ML pipelines, the use of a simple python function, such as my_task:

import numpy as np

def my_task(param: int = 12) -> float:
    return param * np.random.rand()

often requires cumbersome overheads to (1) configure the parameters, (2) submit the job on a cluster, (3) cache the results: e.g.

import pickle
from pathlib import Path
import submitit

# Configure
param = 12

# Check task has already been executed
filepath = tmp_path / f'result-{param}.npy'
if not filepath.exists():

    # Submit job on cluster
    executor = submitit.AutoExecutor(cluster=None, folder=tmp_path)
    job = executor.submit(my_task, param)
    result = job.result()

    # Cache result
    with filepath.open("wb") as f:
        pickle.dump(result, f)

These overheads lead to several issues, such as debugging, handling hierarchical execution and properly saving the results (ending in the classic 'result-parm12-v2_final_FIX.npy').

The solution:

exca can be used to decorate the method of a pydantic model so as to seamlessly configure its execution and caching:

import numpy as np
import pydantic
import exca as xk

class MyTask(pydantic.BaseModel):
    param: int = 12
    infra: xk.TaskInfra = xk.TaskInfra()

    @infra.apply
    def process(self) -> float:
        return self.param * np.random.rand()


task = MyTask(param=1, infra={"folder": tmp_path, "cluster": "auto"})
out = task.process()  # runs on slurm if available
# calling process again will load the cache and not a new random number
assert out == task.process()

See the API reference for all the details

Quick comparison

feature \ tool	lru_cache	hydra	submitit	exca
RAM cache	✔			✔
file cache				✔
remote compute		✔	✔	✔
pure python (vs command line)	✔		✔	✔
hierarchical config		✔		✔

Contributing

See the CONTRIBUTING file for how to help out.

Citing

@misc{exca,
    author = {J. Rapin and J.-R. King},
    title = {{Exca - Execution and caching}},
    year = {2024},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/facebookresearch/exca}},
}

License

exca is MIT licensed, as found in the LICENSE file. Also check-out Meta Open Source Terms of Use and Privacy Policy.

Release files for exca 0.5.29

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for exca 0.5.29
File	Size	Uploaded
exca-0.5.29.tar.gz	181.7 kB	Jul 28, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for exca 0.5.29
File	Interpreter	ABI	Platform	Reset
exca-0.5.29-py3-none-any.whl 216.7 kB Jul 28, 2026	Python 3	none	any	Details

Total release size: 398.4 kB

Release files / exca-0.5.29.tar.gz

Download URL	exca-0.5.29.tar.gz
Size	181.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`50e9df684a4f5e759251dfa02429bcb83d4ee22dc1539e4d11211c1e3fe040d1`
BLAKE2b-256 checksum How to use checksums	`2e5aee2c9c0b4a9ed710e20a01f5568a21bd4081c1ca2e9d5e209a36065020a6`
Upload date	Jul 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.12.8`

Release files / exca-0.5.29-py3-none-any.whl

Download URL	exca-0.5.29-py3-none-any.whl
Size	216.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`99ea106814f166a75771e82175baa24490b5cb60c9e0c3871e7e5b8f989ca06f`
BLAKE2b-256 checksum How to use checksums	`eb930c39e8124c0f6cf6da73b218d1db6f9a64bc27a7cbee0fa8ad6079fb35a9`
Upload date	Jul 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.12.8`