substrai-lambdallm 2.0.0

Serverless-native LLM orchestration framework for AWS Lambda

LambdaLLM

Serverless-native LLM orchestration framework for AWS Lambda.

Built by SubstrAI — Open-source GenAI frameworks for serverless infrastructure.

The Problem

Existing LLM frameworks (LangChain, LlamaIndex) assume long-running servers. They break on Lambda:

• Cold starts: 500MB+ dependency trees add seconds
• Stateless: No conversation memory between invocations
• 15-min timeout: Long agent loops crash
• 250MB limit: LangChain alone exceeds this

The Solution

LambdaLLM is purpose-built for Lambda's constraints:

from lambdallm import handler, Prompt, Model

summarize = Prompt(
    template="Summarize in {max_words} words:\n\n{document}",
    output_schema={"summary": str, "key_points": list}
)

@handler(model=Model.CLAUDE_3_HAIKU)
def lambda_handler(event, context):
    return summarize.invoke(
        _context=context,
        document=event["body"]["text"],
        max_words=100
    )

Features

• < 5MB package size (vs 400MB+ for LangChain)
• Cold-start optimized — lazy imports, connection pooling
• DynamoDB-native state — conversation memory that survives stateless execution
• Cost-aware routing — auto-select cheapest model that meets quality threshold
• Multi-step chains — declarative pipelines with checkpoint/resume on timeout
• AI Agents — ReAct-style agents with tool sandboxing and timeout awareness
• One-command deploy — lambdallm deploy generates all AWS infrastructure
• Timeout handling — checkpoint/resume for long chains
• A/B testing — route traffic between prompt versions, compare metrics
• Full observability — X-Ray tracing, CloudWatch metrics, cost tracking built-in

Installation

Python (primary)

pip install substrai-lambdallm

With AWS Bedrock support (recommended):

pip install "substrai-lambdallm[bedrock]"

With all optional dependencies:

pip install "substrai-lambdallm[all]"

npm

npm install substrai-lambdallm

Quick Start

Python (full CLI experience)

# Install
pip install "substrai-lambdallm[bedrock]"

# Scaffold a new project (creates handler, config, tests)
lambdallm init my-project --template basic
cd my-project

# Start local development server
lambdallm dev

# Test your handler
curl -X POST http://localhost:3000 -d '{"text": "Hello world"}'

# Run tests
lambdallm test

# Deploy to AWS
lambdallm deploy --env dev

TypeScript (runtime SDK)

# Install
npm install substrai-lambdallm @aws-sdk/client-bedrock-runtime

Create your handler:

// handler.ts
import { handler, Model } from 'substrai-lambdallm';

export const lambdaHandler = handler(
  { model: Model.CLAUDE_3_HAIKU, maxRetries: 3 },
  async (event, context) => {
    const body = JSON.parse(event.body || \'{}\');
    const result = await context.invoke('Summarize: {text}', { text: body.text });
    return { statusCode: 200, body: { result, cost: context.totalCost } };
  }
);

Deploy with SAM or CDK:

sam build && sam deploy --guided

Key Differences

Capability	Python	TypeScript
CLI (init, dev, deploy)	\u2705 Included	\u274c Use SAM/CDK directly
Project scaffolding	lambdallm init	Manual setup
Local dev server	lambdallm dev	sam local start-api
Runtime SDK	\u2705 Full	\u2705 Full
Chains + Agents	\u2705 Full	\u2705 Full
Observability	\u2705 Full	\u2705 Full

Available Templates

lambdallm init my-app --template basic   # Simple LLM handler
lambdallm init my-app --template chat    # Multi-turn chat with memory
lambdallm init my-app --template agent   # AI agent with tools
lambdallm init my-app --template rag     # Retrieval-augmented generation

Core Concepts

Handlers

from lambdallm import handler, Model

@handler(model=Model.CLAUDE_3_HAIKU, timeout_strategy="checkpoint")
def lambda_handler(event, context):
    result = context.invoke("Summarize: {text}", text=event["body"]["text"])
    return {"statusCode": 200, "body": result}

Chains

from lambdallm import Chain, Step

pipeline = Chain(
    name="analysis",
    steps=[
        Step("extract", prompt="Extract entities from: {input}"),
        Step("classify", prompt="Classify: {extract.output}"),
        Step("summarize", prompt="Summarize: {classify.output}"),
    ],
    timeout_strategy="checkpoint",
)

Agents

from lambdallm.agents import Agent, Tool

@Tool(description="Search the knowledge base")
def search(query: str, max_results: int = 5) -> list:
    # your implementation
    pass

agent = Agent(
    name="researcher",
    system_prompt="You are a research assistant.",
    tools=[search],
    max_iterations=5,
    timeout_buffer=30,
)

CLI Commands

Command	Description
lambdallm init	Scaffold a new project
lambdallm dev	Start local development server
lambdallm deploy	Deploy to AWS (SAM/CDK)
lambdallm test	Run tests
lambdallm cost	Show cost summary and forecast
lambdallm status	Check deployment status
lambdallm rollback	Rollback to previous version
lambdallm eject	Export raw SAM/CDK templates
lambdallm logs	Tail CloudWatch logs
lambdallm metrics	Show key metrics

Documentation

• Full Documentation — Getting started, tutorials, API reference
• Getting Started
• Tutorials
• API Reference
• Architecture Guide
• Migration Guide
• Examples
• Contributing
• Changelog

License

MIT — see LICENSE

Author

Gaurav Kumar Sinha — Founder, SubstrAI

• Email: [e-mail]
• GitHub: @substrai