Skip to main content

Calkit

Documentation | Tutorials | Discussions

Typical research workflows are horizontally-siloed, i.e., various stages--data collection, analysis, writing--are performed in disconnected systems, turning research into a slow, error-prone, and tedious waterfall process.

Calkit helps you integrate code, data, figures, results, publications, and more into a cohesive, traceable, and portable knowledge creation system, so every output can be traced back to its source (provenance) and reproduced with a single command.

With industry standard tools combined into a unified and simplified experience tailored for research, you can reap the rewards of reproducibility and automation without the cognitive overhead.

pipeline

Features

  • A simplified version control interface that unifies Git and DVC (Data Version Control), so everything can be kept in the same project repository. This way, code doesn't need to be siloed away from other important artifacts like datasets, models, figures, or article PDFs, allowing you to work on all parts of a project without hopping around to different tools.
  • Computational environment management with support for many languages and environment managers: Conda, Docker, uv, Julia, Renv, and more. No need to create and update environments on your own. Calkit will handle them as needed.
  • An environment-aware build system or pipeline with a simple declarative syntax and output caching so you don't need to think about which steps or stages need to be rerun after changing any part of the project. Simply call calkit run. Compose your pipeline from many different kinds of stages, including simple scripts, commands, Jupyter Notebooks, LaTeX, and more.
  • A complementary self-hostable and GitHub-integrated hub to facilitate backup, collaboration, and sharing throughout the entire research lifecycle.
  • Overleaf integration, so analysis, visualization, and writing can all stay in sync (no more manual uploads!)
  • Support for running on high performance computing (HPC) systems that use PBS or SLURM schedulers.
  • Support for automated running with GitHub Actions.
  • Extensions for doing all of the above graphically in JupyterLab and VS Code.
  • A browser extension for collecting references directly to BibTeX (optionally synced with Zotero), viewing DVC-stored files on GitHub, and syncing figures and results with Overleaf directly in Chrome, Microsoft Edge, and more.

Installation

On Linux, macOS, or Windows Git Bash, install Calkit and uv (if not already installed) with:

curl -LsSf install.calkit.org | sh

Or with Windows Command Prompt or PowerShell:

powershell -ExecutionPolicy ByPass -c "irm install-ps1.calkit.org | iex"

If you already have uv installed, install Calkit with:

uv tool install calkit-python

You can also install with your system Python:

pip install calkit-python

To effectively use Calkit, you'll want to ensure Git is installed and properly configured. You may also want to install Docker, since that is the default method by which LaTeX environments are created. If you want to use a Calkit hub for collaboration and backup as a DVC remote, you can connect to the hub with:

calkit hub login

If you use AI agents like Claude, Copilot, or Codex, see AI tools to learn how to install agent skills for working with Calkit.

Use without installing

If you want to use Calkit without installing it, you can use uv's uvx command to run it directly:

uvx calk9 --help

Nix

Calkit ships a flake at the root of its repo, so Nix users can pull the CLI into their environments alongside their other tools.

Run it ad hoc without installing:

nix run github:calkit/calkit -- --help

Drop into a shell that has calkit, git, and uv on PATH:

nix shell github:calkit/calkit

Add it to your own flake.nix as an input:

{
  inputs.calkit.url = "github:calkit/calkit";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

  outputs = { self, nixpkgs, calkit }: {
    devShells.x86_64-linux.default =
      nixpkgs.legacyPackages.x86_64-linux.mkShell {
        packages = [ calkit.packages.x86_64-linux.default ];
      };
  };
}

Then nix develop will give you a shell with the Calkit CLI ready to use. To pin a specific Calkit release inside the shell, set the CALKIT_VERSION environment variable (e.g. CALKIT_VERSION=0.41.0) before invoking calkit.

The flake is currently a thin wrapper around uvx --from calkit-python calkit. It depends on uv from nixpkgs and fetches the published wheel from PyPI on first use. This trades a fully Nix-native build for zero version-drift maintenance, and avoids the macOS docx2pdf / appscript and JupyterLab labextension build issues that block a pure nixpkgs derivation today. If you want a fully nixpkgs-native build, see the community calkit-nix flake.

Nix isn't supported natively on Windows; run Calkit inside WSL2 and use the flake there.

Running against a specific version

If a project requires a Calkit version other than the one you have installed, use the top-level --use-version flag to re-invoke the CLI under that release without changing your installation:

calkit --use-version 0.38 run

This re-execs the CLI via uvx --from calkit-python@<version> calkit, so it requires uv on PATH. You can also declare a minimum version in calkit.yaml; see Pinning the Calkit CLI version.

Calkit Assistant

For Windows users, the Calkit Assistant app is the easiest way to get everything set up and ready to work in VS Code, which can then be used as the primary app for working on all scientific or analytical computing projects.

Calkit Assistant

Quickstart

!!! note ck is an abbreviated alias for the calkit executable. All calkit commands can be run as ck instead, e.g., ck save -am "...".

From an existing project

If you want to use Calkit with an existing project, navigate into its working directory and use the xr command to start executing and recording your scripts, notebooks, LaTeX files, etc., as reproducible pipeline stages. For example:

calkit xr scripts/analyze.py

calkit xr notebooks/plot.ipynb

calkit xr paper/main.tex

Calkit will attempt to detect environments, inputs, and outputs and save them in calkit.yaml. If successful, you'll be able to run the full pipeline with:

calkit run

Next, make a change to e.g., a script and look at the output of calkit status. You'll see that the pipeline has a stage that is out-of-date:

---------------------------- Pipeline ----------------------------
analyze:
        changed deps:
                modified:           scripts/analyze.py

This can be fixed with another call to calkit run.

You can save (add and commit) all changes with:

calkit save -am "Add to pipeline"

Fresh from a Calkit project template

Create a new project from the calkit/example-basic template with:

calkit new project my-research \
    --title "My research" \
    --template calkit/example-basic \
    --hub

Note the --hub flag requires a hub connection to be set up, but can be omitted if the project doesn't need to be backed up to the hub or shared with collaborators. Hub integration can also be set up later.

Next, move into the project folder and run the pipeline, which consists of several stages defined in calkit.yaml:

cd my-research
calkit run

Next, make some edits to a script or LaTeX file and run calkit status to see what stages are out-of-date. For example:

---------------------------- Pipeline ----------------------------
build-paper:
        changed deps:
                modified:           paper/paper.tex

Execute calkit run again to bring everything up-to-date.

To back up or save the project, call:

calkit save -am "Run pipeline"

With an AI coding agent

Simply tell the AI agent:

Turn this folder into a Calkit project

or

Create me a new Calkit project for investigating...

Get involved

We welcome all kinds of contributions! See CONTRIBUTING.md to learn how to get involved.

Acknowledgements

Calkit is supported by the Caltech Schmidt Academy of Software Engineering.

Caltech SASE Schmidt Sciences

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

calkit_python-0.43.0.tar.gz (24.1 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

calkit_python-0.43.0-py3-none-any.whl (705.2 kB view details)

Uploaded Python 3

File details

Details for the file calkit_python-0.43.0.tar.gz.

File metadata

  • Download URL: calkit_python-0.43.0.tar.gz
  • Upload date:
  • Size: 24.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for calkit_python-0.43.0.tar.gz
Algorithm Hash digest
SHA256 453afb9a2fde58d90130d86fe7c056a10f43e7927a8fa8ed1b512e5031e2af6d
MD5 251b1e38b6035dd31c1c05b0f52e286d
BLAKE2b-256 581182d40d3df78b805e61c8ede765eb90ee71efe04b3be7e081b28e595148a1

See more details on using hashes here.

Provenance

The following attestation bundles were made for calkit_python-0.43.0.tar.gz:

Publisher: publish.yml on calkit/calkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file calkit_python-0.43.0-py3-none-any.whl.

File metadata

  • Download URL: calkit_python-0.43.0-py3-none-any.whl
  • Upload date:
  • Size: 705.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for calkit_python-0.43.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2a2f1db15b42ea6308ef313d8538540d21384cd83ba8f431ba662900d5162f67
MD5 ef76b32894804fcaa7d546e28a879862
BLAKE2b-256 9ecaf45b7393a9e67d4291763edfcef38d1332b73b3ed6b2f9633e7a114221a8

See more details on using hashes here.

Provenance

The following attestation bundles were made for calkit_python-0.43.0-py3-none-any.whl:

Publisher: publish.yml on calkit/calkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.47.2

2 files

0.47.1

2 files

0.47.0

2 files

0.46.0

2 files

0.45.3

2 files

0.45.2

2 files

0.45.1

2 files

0.45.0

2 files

0.44.1

2 files

0.44.0

2 files

This release

0.43.0 This release

2 files

0.42.1

2 files

0.42.0

2 files

0.41.25

2 files

0.41.24

2 files

0.41.23

2 files

0.41.22

2 files

0.41.21

2 files

0.41.20

2 files

0.41.19

2 files

0.41.18

2 files

0.41.17

2 files

0.41.16

2 files

0.41.15

2 files

0.41.14

2 files

0.41.13

2 files

0.41.12

2 files

0.41.11

2 files

0.41.10

2 files

0.41.9

2 files

0.41.8

2 files

0.41.7

2 files

0.41.6

2 files

0.41.5

2 files

0.41.4

2 files

0.41.3

2 files

0.41.2

2 files

0.41.1

2 files

0.41.0

2 files

0.40.1

2 files

0.40.0

2 files

0.39.2

2 files

0.39.1

2 files

0.39.0

2 files

0.38.4

2 files

0.38.3

2 files

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.6

2 files

0.37.5

2 files

0.37.4

2 files

0.37.3

2 files

0.37.2

2 files

0.37.1

2 files

0.37.0

2 files

0.36.0

2 files

0.35.7

2 files

0.35.6

2 files

0.35.5

2 files

0.35.4

2 files

0.35.3

2 files

0.35.2

2 files

0.35.1

2 files

0.35.0

2 files

0.34.6

2 files

0.34.5

2 files

0.34.4

2 files

0.34.3

2 files

0.34.2

2 files

0.34.1

2 files

0.34.0

2 files

0.33.5

2 files

0.33.4

2 files

0.33.3

2 files

0.33.1

2 files

0.33.0

2 files

0.32.11

2 files

0.32.10

2 files

0.32.9

2 files

0.32.8

2 files

0.32.7

2 files

0.32.6

2 files

0.32.5

2 files

0.32.4

2 files

0.32.3

2 files

0.32.2

2 files

0.32.1

2 files

0.32.0

2 files

0.31.5

2 files

0.31.4

2 files

0.31.3

2 files

0.31.2

2 files

0.31.1

2 files

0.31.0

2 files

0.30.11

2 files

0.30.10

2 files

0.30.9

2 files

0.30.8

2 files

0.30.7

2 files

0.30.6

2 files

0.30.5

2 files

0.30.4

2 files

0.30.3

2 files

0.30.2

2 files

0.30.1

2 files

0.30.0

2 files

0.29.2

2 files

0.29.0

2 files

0.28.4

2 files

0.28.3

2 files

0.28.2

2 files

0.28.1

2 files

0.28.0

2 files

0.27.3

2 files

0.27.2

2 files

0.27.1

2 files

0.27.0

2 files

0.26.12

2 files

0.26.11

2 files

0.26.10

2 files

0.26.9

2 files

0.26.8

2 files

0.26.7

2 files

0.26.6

2 files

0.26.5

2 files

0.26.4

2 files

0.26.3

2 files

0.26.2

2 files

0.26.1

2 files

0.26.0

2 files

0.25.2

2 files

0.25.1

2 files

0.25.0

2 files

0.24.9

2 files

0.24.8

2 files

0.24.7

2 files

0.24.6

2 files

0.24.5

2 files

0.24.4

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.23.1

2 files

0.23.0

2 files

0.22.3

2 files

0.22.2

2 files

0.22.1

2 files

0.22.0

2 files

0.21.8

2 files

0.21.7

2 files

0.21.6

2 files

0.21.5

2 files

0.21.4

2 files

0.21.3

2 files

0.21.2

2 files

0.21.1

2 files

0.21.0

2 files

0.20.7

2 files

0.20.6

2 files

0.20.5

2 files

0.20.4

2 files

0.20.3

2 files

0.20.2

2 files

0.20.1

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.6

2 files

0.17.5

2 files

0.17.4

2 files

0.17.3

2 files

0.17.2

2 files

0.17.1

2 files

0.17.0

2 files

0.16.3

2 files

0.16.2

2 files

0.16.1

2 files

0.16.0

2 files

0.15.0

2 files

0.14.5

2 files

0.14.4

2 files

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.11.4

2 files

0.11.3

2 files

0.11.2

2 files

0.11.1

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.4

2 files

0.9.3

2 files

0.9.2

2 files

0.9.1

2 files

0.9.0

2 files

0.8.5

2 files

0.8.4

2 files

0.8.3

2 files

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.0

2 files

0.6.1

2 files

0.6.0

2 files

0.5.0

2 files

0.4.0

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

0.0.10

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page