Avatar for Nils Oskar Matteson from gravatar.com

Nils Oskar Matteson

Username    matteso1
Date joined   Joined

2 projects

thaw-vllm

Last released

Fast snapshot/restore for LLM inference. 17x faster cold starts, multi-GPU tensor parallel, KV cache snapshots.

thaw-native

Last released

Rust+CUDA native extension for thaw (pipelined DMA freeze/restore)

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page