dspy-lite

DSPy with litellm replaced by litelm and numpy made optional.

What this is

• Same DSPy 3.3.1 -- signatures, modules (Predict, CoT, ReAct/ReActV2, RLM, Flex), optimizers (Bootstrap, MIPRO, SIMBA, GEPA, GRPO)
• litellm (100k+ LOC, 70+ transitive deps) replaced by the thin litelm transport
• numpy moved from core dep to optional [embeddings] extra
• 85% smaller clean install (25 MB vs 171 MB, 34 packages vs 65)
• Unaffected by the litellm supply chain compromise (CVE-2026-33634) — zero litellm code or dependencies

Install

pip install dspy-lite

Supported Python versions: 3.10 through 3.14.

With provider extras:

pip install dspy-lite[anthropic]     # Anthropic
pip install dspy-lite[bedrock]       # AWS Bedrock
pip install dspy-lite[embeddings]    # numpy for embeddings
pip install dspy-lite[deno]          # managed Deno runtime for interpreters
pip install dspy-lite[all]           # everything

Usage

import dspy

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

predict = dspy.Predict("question -> answer")
result = predict(question="What is DSPy?")
print(result.answer)

Everything from DSPy works -- the import is still import dspy.

Native System One decisions (experimental)

Use model_type="decision" explicitly for Microsoft Foundry Decision-1 (azure_ai/<deployment>) or TypeSafe (typesafe/<model>). This is a native probability API, not chat completion or JSON generated by a language model:

class Categorize(dspy.Signature):
    """Classify the synthetic support message."""
    text: str = dspy.InputField()
    category: dspy.Choice[("billing", "Payment question"), ("technical", "Product issue")] = dspy.OutputField(
        desc="Choose the message category."
    )

# AZURE_AI_API_BASE and AZURE_AI_API_KEY, or azure_ad_token_provider=...
lm = dspy.LM("azure_ai/decision-deployment", model_type="decision", num_retries=0, cache=False)
predict = dspy.Predict(Categorize)
result = predict(text="The demo application does not start.", lm=lm)
print(result.category.value, result.category.probabilities, result.category.confidence)
# The same path supports: await predict.acall(text=..., lm=lm)

dspy.Noul retains true-probability; dspy.Choice[...] retains the option distribution; dspy.Score["low", "medium", "high"] returns its probability-weighted mean level index and retains the raw distribution. Native bool and Literal[...] outputs are also supported. Every output needs an explicit question in OutputField(desc=...) or a field override. Rich types are ordinary serializable Pydantic values. result.get_decision_evidence() returns a detached JSON copy of all raw answers, including for native bool/Literal fields; LM history and JSON disk caches also retain that evidence.

Predict's fields mapping supports per-output Noul threshold, Choice weights, and Score cuts, with optional instructions/criteria overrides. These controls are validated, saved with the program, and applied locally to an in-flight snapshot, never sent as generation controls. Weights do not alter the retained probabilities or provider confidence. Noul confidence is distance from its threshold, not calibrated correctness. Decision field overrides require a native decision LM; no generative evidence emulation is performed.

Native calls preserve BaseLM callbacks, usage tracking, history, copy, and JSON program save/load. Credentials and runtime HTTP clients/token callbacks are excluded from JSON LM configuration and native history; reattach runtime authentication after loading. Whole-program pickle saves remain unsafe and may retain runtime state—do not use them for credential-bearing programs. Normal callbacks still receive request/response data: disable them, history, tracing, and caching when handling sensitive inputs. Caches partition endpoints, models, question spaces and API-key identities; token callbacks/custom HTTP clients disable decision caching because their authentication identity is not inspectable. As elsewhere in DSPy, cached calls report no new usage.

Free-form outputs, ChainOfThought, tools, streaming, media, generation settings (including temperature, n and rollout IDs), and non-JSON state are rejected; there is no chat or keyword fallback. Do not assume generation-oriented optimizers work with this mode. Evaluate, labeled demonstrations, and mixed programs with separate generative predictors remain usable. Provider availability and data-residency approval are deployment-specific; mocked HTTP tests do not establish live provider or privacy qualification.

Loading images, audio, and files

Resource validation never reads the filesystem or network implicitly. Load resources explicitly:

image = dspy.Image.from_path("./image.png")
image = dspy.Image.from_url("https://example.com/image.png")  # explicit network request
image_reference = dspy.Image("https://example.com/image.png")  # provider-fetched URL; no local request

audio = dspy.Audio.from_path("./audio.wav")
file = dspy.File.from_path("./document.pdf")

Applications must validate or allowlist untrusted URLs before calling from_url(); explicit downloads do not provide SSRF filtering.

Disk-cache safety

Persistent caches use versioned JSON in stdlib SQLite, not pickle or DiskCache. Built-in provider response types and numeric NumPy arrays round-trip; custom dataclasses and Pydantic data models require dspy.configure_cache(safe_types=[...]). Register only trusted data types: model validation may invoke their validators. Unsupported values remain eligible for memory caching but are not persisted. restrict_pickle=False is rejected.

Storage lives under json-v1/ within the configured cache directory. Legacy pickle directories are left untouched and never read or migrated; old entries become misses. The size limit bounds logical entry costs (serialized values, keys and metadata), not the exact physical size of SQLite files. The latest configuration sets the shared budget for all instances and immediately evicts excess entries. Failed/oversized replacements invalidate the old entry rather than resurrecting stale responses. SQLite transactions coordinate threads and processes; eviction removes least-recently-used entries. Cache files cannot be symlinks/hard links; POSIX storage requires private app-owned paths, including safe parent directories. These protections do not defend against a compromised application account. Corrupt or unknown data become misses without payload logging. Explicit load_memory_cache(..., allow_pickle=True) remains a separate, unsafe opt-in API; persistent-cache reads never fall back to it.

DSPy 3.3 APIs

This release includes DSPy 3.3.1's typed provider-neutral LM boundary, experimental ReActV2 and Flex, structured native tool-call history, interpreter isolation improvements, optional managed Deno runtime, and GEPA 0.1.4 integration. Review the upstream 3.3.0 and 3.3.1 notes for compatibility changes.

What changed from DSPy

Documentation

Full DSPy documentation: dspy.ai

Development disclosure

This project is maintained with AI-assisted coding. Through 2026-05-14, I used Claude Code with Claude Opus 4.6/4.7. As of 2026-05-14, I use pi with GPT-5.5. The human maintainer remains responsible for what is merged and released.

License

MIT (inherited from DSPy)