pico-kafka

Kafka for the pico ecosystem: @kafka_consumer methods and declarative @kafka_producer clients over aiokafka.

Installation

pip install pico-kafka

Quick start

kafka:
  bootstrap_servers: kafka.internal:9092
  group_id: myapp

Consume — a component method per topic, JSON value decoded for you:

from pico_ioc import component
from pico_kafka import kafka_consumer

@component
class OrderProjection:
    @kafka_consumer("orders")
    async def on_order(self, message: dict):
        ...

    @kafka_consumer("orders", group_id="analytics")   # independent fan-out
    def on_order_analytics(self, message: dict):
        ...

Produce — stubs, like a pico-httpx client:

from pico_kafka import kafka_producer, produce

@kafka_producer
class OrderEvents:
    @produce("orders")
    def order_created(self, message): ...

Semantics:

• Consumers and producers run on a dedicated background loop — works in sync scripts, FastAPI apps and workers alike, no lifespan wiring.
• Each record resolves its component through the container (prototype scope = fresh instance per record).
• A record whose handler raises is logged and skipped — offsets advance, a poison record cannot stall the partition.
• Sync produce stubs block until the broker acks; async stubs await it. Everything stops with the container.

Documentation

Full documentation: https://dperezcabrera.github.io/pico-kafka/

Built for AI-assisted development

pico-kafka is part of an ecosystem designed for humans and coding agents building software together. Every package ships AGENTS.md working conventions, an llms.txt machine-readable docs index and documented behaviour pinned by regression tests; pico-testing gives agents a verification loop for their own changes, and releases are gated by the whole ecosystem booting together against real infrastructure. The full story: Built for AI-assisted development.

Install the agent skills for Claude Code or OpenAI Codex:

curl -sL https://raw.githubusercontent.com/dperezcabrera/pico-skills/main/install.sh | bash

The pico-conventions skill teaches the assistant this module's API surface and invariants; /add-component and /add-tests scaffold components and tests that use it.

License

MIT