Skip to main content

Calkit

Documentation | Tutorials | Discussions

Typical research workflows are horizontally-siloed, i.e., various stages--data collection, analysis, writing--are performed in disconnected systems, turning research into a slow, error-prone, and tedious waterfall process.

Calkit helps you integrate code, data, figures, results, publications, and more into a cohesive, traceable, and portable knowledge creation system, so every output can be traced back to its source (provenance) and reproduced with a single command.

With industry standard tools combined into a unified and simplified experience tailored for research, you can reap the rewards of reproducibility and automation without the cognitive overhead.

pipeline

Features

  • A simplified version control interface that unifies Git and DVC (Data Version Control), so everything can be kept in the same project repository. This way, code doesn't need to be siloed away from other important artifacts like datasets, models, figures, or article PDFs, allowing you to work on all parts of a project without hopping around to different tools.
  • Computational environment management with support for many languages and environment managers: Conda, Docker, uv, Julia, Renv, and more. No need to create and update environments on your own. Calkit will handle them as needed.
  • An environment-aware build system or pipeline with a simple declarative syntax and output caching so you don't need to think about which steps or stages need to be rerun after changing any part of the project. Simply call calkit run. Compose your pipeline from many different kinds of stages, including simple scripts, commands, Jupyter Notebooks, LaTeX, and more.
  • A complementary self-hostable and GitHub-integrated hub to facilitate backup, collaboration, and sharing throughout the entire research lifecycle.
  • Overleaf integration, so analysis, visualization, and writing can all stay in sync (no more manual uploads!)
  • Support for running on high performance computing (HPC) systems that use PBS or SLURM schedulers.
  • Support for automated running with GitHub Actions.
  • Extensions for doing all of the above graphically in JupyterLab and VS Code.
  • A browser extension for collecting references directly to BibTeX (optionally synced with Zotero), viewing DVC-stored files on GitHub, and syncing figures and results with Overleaf directly in Chrome, Microsoft Edge, and more.

Installation

On Linux, macOS, or Windows Git Bash, install Calkit and uv (if not already installed) with:

curl -LsSf install.calkit.org | sh

Or with Windows Command Prompt or PowerShell:

powershell -ExecutionPolicy ByPass -c "irm install-ps1.calkit.org | iex"

If you already have uv installed, install Calkit with:

uv tool install calkit-python

You can also install with your system Python:

pip install calkit-python

To effectively use Calkit, you'll want to ensure Git is installed and properly configured. You may also want to install Docker, since that is the default method by which LaTeX environments are created. If you want to use a Calkit hub for collaboration and backup as a DVC remote, you can connect to the hub with:

calkit hub login

If you use AI agents like Claude, Copilot, or Codex, see AI tools to learn how to install agent skills for working with Calkit.

Use without installing

If you want to use Calkit without installing it, you can use uv's uvx command to run it directly:

uvx ck9 --help

Nix

Calkit ships a flake at the root of its repo, so Nix users can pull the CLI into their environments alongside their other tools.

Run it ad hoc without installing:

nix run github:calkit/calkit -- --help

Drop into a shell that has calkit, git, and uv on PATH:

nix shell github:calkit/calkit

Add it to your own flake.nix as an input:

{
  inputs.calkit.url = "github:calkit/calkit";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

  outputs = { self, nixpkgs, calkit }: {
    devShells.x86_64-linux.default =
      nixpkgs.legacyPackages.x86_64-linux.mkShell {
        packages = [ calkit.packages.x86_64-linux.default ];
      };
  };
}

Then nix develop will give you a shell with the Calkit CLI ready to use. To pin a specific Calkit release inside the shell, set the CALKIT_VERSION environment variable (e.g. CALKIT_VERSION=0.41.0) before invoking calkit.

The flake is currently a thin wrapper around uvx --from calkit-python calkit. It depends on uv from nixpkgs and fetches the published wheel from PyPI on first use. This trades a fully Nix-native build for zero version-drift maintenance, and avoids the macOS docx2pdf / appscript and JupyterLab labextension build issues that block a pure nixpkgs derivation today. If you want a fully nixpkgs-native build, see the community calkit-nix flake.

Nix isn't supported natively on Windows; run Calkit inside WSL2 and use the flake there.

Running against a specific version

If a project requires a Calkit version other than the one you have installed, use the top-level --use-version flag to re-invoke the CLI under that release without changing your installation:

calkit --use-version 0.38 run

This re-execs the CLI via uvx --from calkit-python@<version> calkit, so it requires uv on PATH. You can also declare a minimum version in calkit.yaml; see Pinning the Calkit CLI version.

Calkit Assistant

For Windows users, the Calkit Assistant app is the easiest way to get everything set up and ready to work in VS Code, which can then be used as the primary app for working on all scientific or analytical computing projects.

Calkit Assistant

Quickstart

!!! note ck is an abbreviated alias for the calkit executable. All calkit commands can be run as ck instead, e.g., ck save -am "...".

From an existing project

If you want to use Calkit with an existing project, navigate into its working directory and use the xr command to start executing and recording your scripts, notebooks, LaTeX files, etc., as reproducible pipeline stages. For example:

calkit xr scripts/analyze.py

calkit xr notebooks/plot.ipynb

calkit xr paper/main.tex

Calkit will attempt to detect environments, inputs, and outputs and save them in calkit.yaml. If successful, you'll be able to run the full pipeline with:

calkit run

Next, make a change to e.g., a script and look at the output of calkit status. You'll see that the pipeline has a stage that is out-of-date:

---------------------------- Pipeline ----------------------------
analyze:
        changed deps:
                modified:           scripts/analyze.py

This can be fixed with another call to calkit run.

You can save (add and commit) all changes with:

calkit save -am "Add to pipeline"

Fresh from a Calkit project template

Create a new project from the calkit/example-basic template with:

calkit new project my-research \
    --title "My research" \
    --template calkit/example-basic \
    --hub

Note the --hub flag requires a hub connection to be set up, but can be omitted if the project doesn't need to be backed up to the hub or shared with collaborators. Hub integration can also be set up later.

Next, move into the project folder and run the pipeline, which consists of several stages defined in calkit.yaml:

cd my-research
calkit run

Next, make some edits to a script or LaTeX file and run calkit status to see what stages are out-of-date. For example:

---------------------------- Pipeline ----------------------------
build-paper:
        changed deps:
                modified:           paper/paper.tex

Execute calkit run again to bring everything up-to-date.

To back up or save the project, call:

calkit save -am "Run pipeline"

With an AI coding agent

Simply tell the AI agent:

Turn this folder into a Calkit project

or

Create me a new Calkit project for investigating...

Get involved

We welcome all kinds of contributions! See CONTRIBUTING.md to learn how to get involved.

Acknowledgements

Calkit is supported by the Caltech Schmidt Academy of Software Engineering.

Caltech SASE Schmidt Sciences

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

calkit_python-0.47.1.tar.gz (25.3 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

calkit_python-0.47.1-py3-none-any.whl (1.0 MB view details)

Uploaded Python 3

File details

Details for the file calkit_python-0.47.1.tar.gz.

File metadata

  • Download URL: calkit_python-0.47.1.tar.gz
  • Upload date:
  • Size: 25.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for calkit_python-0.47.1.tar.gz
Algorithm Hash digest
SHA256 c30d27f29408ffe1728115ed4a9a3689978fcd45a8c1fb1af2f6e1f916c1db7f
MD5 7156acd3e9ac387d258d7bc738d0b80b
BLAKE2b-256 133043bb2d430925d636c96bb2cd2c8cf2c55aa39ce412e37b25b11e44ffad75

See more details on using hashes here.

Provenance

The following attestation bundles were made for calkit_python-0.47.1.tar.gz:

Publisher: publish.yml on calkit/calkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file calkit_python-0.47.1-py3-none-any.whl.

File metadata

  • Download URL: calkit_python-0.47.1-py3-none-any.whl
  • Upload date:
  • Size: 1.0 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for calkit_python-0.47.1-py3-none-any.whl
Algorithm Hash digest
SHA256 f0e977602ed5375aa8d4096a55dcde9e0b0d7014b29bf99240b37df2bf41f01f
MD5 a40abe70f33cb7abc83054ed0030f1b0
BLAKE2b-256 611919fd7d1e7f87f98e448cd66c9361d61cc8ba4e7779044e833ab7e4cca6d3

See more details on using hashes here.

Provenance

The following attestation bundles were made for calkit_python-0.47.1-py3-none-any.whl:

Publisher: publish.yml on calkit/calkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.47.2

2 files

This release

0.47.1 This release

2 files

0.47.0

2 files

0.46.0

2 files

0.45.3

2 files

0.45.2

2 files

0.45.1

2 files

0.45.0

2 files

0.44.1

2 files

0.44.0

2 files

0.43.0

2 files

0.42.1

2 files

0.42.0

2 files

0.41.25

2 files

0.41.24

2 files

0.41.23

2 files

0.41.22

2 files

0.41.21

2 files

0.41.20

2 files

0.41.19

2 files

0.41.18

2 files

0.41.17

2 files

0.41.16

2 files

0.41.15

2 files

0.41.14

2 files

0.41.13

2 files

0.41.12

2 files

0.41.11

2 files

0.41.10

2 files

0.41.9

2 files

0.41.8

2 files

0.41.7

2 files

0.41.6

2 files

0.41.5

2 files

0.41.4

2 files

0.41.3

2 files

0.41.2

2 files

0.41.1

2 files

0.41.0

2 files

0.40.1

2 files

0.40.0

2 files

0.39.2

2 files

0.39.1

2 files

0.39.0

2 files

0.38.4

2 files

0.38.3

2 files

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.6

2 files

0.37.5

2 files

0.37.4

2 files

0.37.3

2 files

0.37.2

2 files

0.37.1

2 files

0.37.0

2 files

0.36.0

2 files

0.35.7

2 files

0.35.6

2 files

0.35.5

2 files

0.35.4

2 files

0.35.3

2 files

0.35.2

2 files

0.35.1

2 files

0.35.0

2 files

0.34.6

2 files

0.34.5

2 files

0.34.4

2 files

0.34.3

2 files

0.34.2

2 files

0.34.1

2 files

0.34.0

2 files

0.33.5

2 files

0.33.4

2 files

0.33.3

2 files

0.33.1

2 files

0.33.0

2 files

0.32.11

2 files

0.32.10

2 files

0.32.9

2 files

0.32.8

2 files

0.32.7

2 files

0.32.6

2 files

0.32.5

2 files

0.32.4

2 files

0.32.3

2 files

0.32.2

2 files

0.32.1

2 files

0.32.0

2 files

0.31.5

2 files

0.31.4

2 files

0.31.3

2 files

0.31.2

2 files

0.31.1

2 files

0.31.0

2 files

0.30.11

2 files

0.30.10

2 files

0.30.9

2 files

0.30.8

2 files

0.30.7

2 files

0.30.6

2 files

0.30.5

2 files

0.30.4

2 files

0.30.3

2 files

0.30.2

2 files

0.30.1

2 files

0.30.0

2 files

0.29.2

2 files

0.29.0

2 files

0.28.4

2 files

0.28.3

2 files

0.28.2

2 files

0.28.1

2 files

0.28.0

2 files

0.27.3

2 files

0.27.2

2 files

0.27.1

2 files

0.27.0

2 files

0.26.12

2 files

0.26.11

2 files

0.26.10

2 files

0.26.9

2 files

0.26.8

2 files

0.26.7

2 files

0.26.6

2 files

0.26.5

2 files

0.26.4

2 files

0.26.3

2 files

0.26.2

2 files

0.26.1

2 files

0.26.0

2 files

0.25.2

2 files

0.25.1

2 files

0.25.0

2 files

0.24.9

2 files

0.24.8

2 files

0.24.7

2 files

0.24.6

2 files

0.24.5

2 files

0.24.4

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.23.1

2 files

0.23.0

2 files

0.22.3

2 files

0.22.2

2 files

0.22.1

2 files

0.22.0

2 files

0.21.8

2 files

0.21.7

2 files

0.21.6

2 files

0.21.5

2 files

0.21.4

2 files

0.21.3

2 files

0.21.2

2 files

0.21.1

2 files

0.21.0

2 files

0.20.7

2 files

0.20.6

2 files

0.20.5

2 files

0.20.4

2 files

0.20.3

2 files

0.20.2

2 files

0.20.1

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.6

2 files

0.17.5

2 files

0.17.4

2 files

0.17.3

2 files

0.17.2

2 files

0.17.1

2 files

0.17.0

2 files

0.16.3

2 files

0.16.2

2 files

0.16.1

2 files

0.16.0

2 files

0.15.0

2 files

0.14.5

2 files

0.14.4

2 files

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.11.4

2 files

0.11.3

2 files

0.11.2

2 files

0.11.1

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.4

2 files

0.9.3

2 files

0.9.2

2 files

0.9.1

2 files

0.9.0

2 files

0.8.5

2 files

0.8.4

2 files

0.8.3

2 files

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.0

2 files

0.6.1

2 files

0.6.0

2 files

0.5.0

2 files

0.4.0

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

0.0.10

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page