elastic-kernel 0.0.29

An IPython Kernel that automatically saves and restores Jupyter Notebook execution states.

ElasticKernel

Never lose your Jupyter variables to a kernel restart again.

ElasticKernel is a custom IPython kernel that automatically checkpoints your notebook's execution state and restores it after a restart or crash — no manual pickle.dump required. Pick up exactly where you left off.

🇯🇵 日本語版は README.ja.md を参照してください。

Demo

The same workflow in both clips: define a variable a = 1, then restart the kernel. The difference is what happens next.

Standard kernel (ipykernel)	ElasticKernel
❌ After the restart, a is gone — %whos reports "Interactive namespace is empty."	✅ After the restart, a is automatically restored — %whos still shows a  int  1.

If the videos don't play inline, click them to view: ipykernel · ElasticKernel.

Why ElasticKernel?

Every Jupyter user has been there: a long computation finishes, then an accidental kernel restart (or an out-of-memory crash) wipes every variable in your session. The usual workaround is scattering pickle.dump / joblib.dump calls everywhere and remembering to reload them by hand.

ElasticKernel removes that chore entirely:

• 🔄 Automatic state recovery — your variables survive kernel restarts and shutdowns, with zero changes to your code.
• 🧠 Dependency-aware — tracks how cells and variables depend on one another to restore a consistent state.
• ⚡ Cost-optimized checkpoints — for each variable it decides whether to serialize it or recompute it on restore, based on serialization size vs. recomputation cost (a min-cut optimization).
• 🪄 Drop-in — just pick the Python 3 (ElasticKernel) kernel; the rest of your workflow is unchanged.

Installation & Usage

Local

1. Install the package:

   $ pip install elastic-kernel
   

2. Install the kernel:

   $ elastic-kernel install
   Elastic Kernel installed from: /path/to/elastic_kernel
   

3. Verify the kernel is installed:

   $ jupyter kernelspec list
   Available kernels:
     elastic_kernel    /path/to/Jupyter/kernels/elastic_kernel
   

4. Launch JupyterLab:

   $ jupyter lab --ip=0.0.0.0
   

5. Open JupyterLab in your browser.

6. Select the Python 3 (ElasticKernel) kernel.

Docker

1. Pull the image:

   docker pull ghcr.io/mryutaro/elastickernel
   

2. Start a container:

   docker run -p 8888:8888 ghcr.io/mryutaro/elastickernel
   

3. Open JupyterLab in your browser.

4. Select the Python 3 (ElasticKernel) kernel.

How It Works

ElasticKernel extends the IPython kernel to observe each cell execution. As you run cells it builds a dependency graph of variables and the cell executions that produce them. When the kernel shuts down or restarts, it profiles serialization speed, runs a cost optimizer to split variables into a migrate set (serialized to disk) and a recompute set (regenerated by re-running cells), and writes a checkpoint. On the next start it loads the checkpoint, injects the migrated variables back into your namespace, and recomputes the rest.

Documentation

• Developer guide: docs/DEVELOPERS.md
• 日本語 README: README.ja.md

Publication

This project was presented in the following paper. If you use ElasticKernel in your research, please cite:

R. Matsumoto, K. Taniguchi, T. Hayami, K. Takahashi, and S. Date. "ElasticHub: A Cost-Efficient JupyterHub Platform via Automated Scaling with Kubernetes on Hybrid Cloud." Proceedings of the 16th International Conference on Cloud Computing and Services Science, pp. 261–268, 2026. DOI: 10.5220/0014840200004039

@inproceedings{matsumoto2026elastichub,
  author    = {Matsumoto, R. and Taniguchi, K. and Hayami, T. and Takahashi, K. and Date, S.},
  title     = {ElasticHub: A Cost-Efficient JupyterHub Platform via Automated Scaling with Kubernetes on Hybrid Cloud},
  booktitle = {Proceedings of the 16th International Conference on Cloud Computing and Services Science},
  year      = {2026},
  pages     = {261--268},
  isbn      = {978-989-758-829-7},
  issn      = {2184-5042},
  doi       = {10.5220/0014840200004039}
}

Acknowledgments

This project includes code from ElasticNotebook, developed at the University of Illinois. ElasticNotebook is licensed under the Apache License 2.0.

Zhaoheng Li, Pranav Gor, Rahul Prabhu, Hui Yu, Yuzhou Mao, Yongjoo Park. "ElasticNotebook: Enabling Live Migration for Computational Notebooks." Proceedings of the VLDB Endowment, Vol. 17, No. 2, pp. 119-133, 2023.

License

Licensed under the Apache License 2.0.