faststream 0.1.1rc0

FastStream: the simplest way to work with a messaging queues

FastStream

Effortless event stream integration for your services

---

---

Features

FastStream simplifies the process of writing producers and consumers for message queues, handling all the parsing, networking and documentatino generation automatically.

Making streaming microservices has never been easier. Designed with junior developers in mind, FastStream simplifies your work while keeping the door open for more advanced use-cases. Here's a look at the core features that make FastStream a go-to framework for modern, data-centric microservices.

• Multiple Brokers: FastStream provides a unified API to work across multiple message brokers (Kafka, RabbitMQ support)

• Pydantic Validation: Leverage Pydantic's validation capabilities to serialize and validates incoming messages

• Automatic Docs: Stay ahead with automatic AsyncAPI documentation.

• Intuitive: full typed editor support makes your development experience smooth, catching errors before they reach runtime

• Powerful Dependency Injection System: Manage your service dependencies efficiently with FastStream's built-in DI system.

• Testable: supports in-memory tests, making your CI/CD pipeline faster and more reliable

• Extendable: use extensions for lifespans, custom serialization and middlewares

• Integrations: FastStream is fully compatible with any HTTP framework you want (FastAPI especially)

• Built for Automatic Code Generation: FastStream is optimized for automatic code generation using advanced models like GPT and Llama

That's FastStream in a nutshell—easy, efficient, and powerful. Whether you're just starting with streaming microservices or looking to scale, FastStream has got you covered.

---

History

FastStream is a new package based on the ideas and experiences gained from FastKafka and Propan. By joining our forces, we picked up the best from both packages and created the unified way to write services capable of processing streamed data regradless of the underliying protocol. We'll continue to maintain both packages, but new development will be in this project. If you are starting a new service, this package is the recommended way to do it.

---

Install

FastStream works on Linux, macOS, Windows and most Unix-style operating systems. You can install it with pip as usual:

pip install faststream[kafka]
# or
pip install faststream[rabbit]

By default FastStream uses PydanticV2 written in Rust, but you can downgrade it manually, if your platform has no Rust support - FastStream will works with the PydanticV1 correctly as well.

---

Writing app code

FastStream brokers provide convenient function decorators @broker.subscriber and @broker.publisher to allow you to delegate the actual process of

• consuming and producing data to Event queues, and

• decoding and encoding JSON encoded messages

These decorators make it easy to specify the processing logic for your consumers and producers, allowing you to focus on the core business logic of your application without worrying about the underlying integration.

Also, FastStream uses Pydantic to parse input JSON-encoded data into Python objects, making it easy to work with structured data in your applications, so you can serialize you input messages just using type annotations.

Here is an example python app using FastStream that consumes data from an incoming data stream and outputs the data to another one.

from faststream import FastStream
from faststream.kafka import KafkaBroker
# from faststream.rabbit import RabbitBroker

broker = KafkaBroker("localhost:9092")
# broker = RabbitBroker("amqp://guest:guest@localhost:5672/")
app = FastStream(broker)

@broker.subscriber("in")
@broker.publisher("out")
async def handle_msg(user: str, user_id: int) -> str:
    return f"User: {user_id} - {user} registered"

Also, Pydantic’s BaseModel class allows you to define messages using a declarative syntax, making it easy to specify the fields and types of your messages.

from pydantic import BaseModel, Field, PositiveInt
from faststream import FastStream
from faststream.kafka import KafkaBroker

broker = KafkaBroker("localhost:9092")
app = FastStream(broker)

class User(BaseModel):
    user: str = Field(..., examples=["John"])
    user_id: PositiveInt = Field(..., examples=["1"])

@broker.subscriber("in")
@broker.publisher("out")
async def handle_msg(data: User) -> str:
    return f"User: {data.user} - {data.user_id} registered"

---

Testing the service

The service can be tested using the TestBroker context managers which, by default, puts the Broker into "testing mode".

The Tester will redirect your subscriber and publisher decorated functions to the InMemory brokers so that you can quickly test your app without the need for a running broker and all its dependencies.

Using pytest, the test for our service would look like this:

# Code above omitted 👆

import pytest
import pydantic
from faststream.kafka import TestKafkaBroker


@pytest.mark.asyncio
async def test_correct():
    async with TestKafkaBroker(broker) as br:
        await br.publish({
            "user": "John",
            "user_id": 1,
        }, "in")

@pytest.mark.asyncio
async def test_invalid():
    async with TestKafkaBroker(broker) as br:
        with pytest.raises(pydantic.ValidationError):
            await br.publish("wrong message", "in")

Running the application

The application can be started using builtin FastStream CLI command.

To run the service, use the FastStream CLI command and pass the module (in this case, the file where the app implementation is located) and the app simbol to the command.

faststream run basic:app

After running the command you should see the following output:

INFO     - FastStream app starting...
INFO     - input_data |            - `HandleMsg` waiting for messages
INFO     - FastStream app started successfully! To exit press CTRL+C

Also, FastStream provides you a great hotreload feature to improve your Development Experience

faststream run basic:app --reload

And multiprocessing horizontal scaling feature as well

faststream run basic:app --workers 3

You can know more about CLI features here

---

Project Documentation

FastStream automatically generates documentation for your project according to the AsyncAPI specification. You can work with both generated artifacts and place a Web view of your documentation on resources available to related teams.

The availability of such documentation significantly simplifies the integration of services: you can immediately see what channels and message format the application works with. And most importantly, it won't cost anything - FastStream has already created the docs for you!

---

Dependencies

FastStream (thanks to FastDepend) has a dependency management system close to pytest fixtures and FastAPI Depends at the same time. Function arguments declare which dependencies you want are needed, and a special decorator delivers them from the global Context object.

from faststream import Depends, Logger
async def base_dep(user_id: int) -> bool:
    return True

@broker.subscriber("in-test")
async def base_handler(user: str,
                       logger: Logger,
                       dep: bool = Depends(base_dep)):
    assert dep is True
    logger.info(user)

---

HTTP Frameworks integrations

Any Framework

You can use FastStream MQBrokers without FastStream application. Just start and stop them according to your application lifespan.

from aiohttp import web

from faststream.kafka import KafkaBroker

broker = KafkaBroker("localhost:9092")

@broker.subscriber("test")
async def base_handler(body):
    print(body)

async def start_broker(app):
    await broker.start()

async def stop_broker(app):
    await broker.close()

async def hello(request):
    return web.Response(text="Hello, world")

app = web.Application()
app.add_routes([web.get("/", hello)])
app.on_startup.append(start_broker)
app.on_cleanup.append(stop_broker)

if __name__ == "__main__":
    web.run_app(app)

FastAPI Plugin

Also, FastStream can be used as part of FastAPI.

Just import a StreamRouter you need and declare message handler with the same @router.subscriber(...) and @router.publisher(...) decorators.

from fastapi import FastAPI
from pydantic import BaseModel

from faststream.kafka.fastapi import KafkaRouter

router = KafkaRouter("localhost:9092")

class Incoming(BaseModel):
    m: dict

@router.subscriber("test")
@router.publisher("response")
async def hello(m: Incoming):
    return {"response": "Hello, world!"}

app = FastAPI(lifespan=router.lifespan_context)
app.include_router(router)

More integration features can be found here

---

Stay in touch

Please show your support and stay in touch by:

• giving our GitHub repository a star, and

• joining our Discord server

Your support helps us to stay in touch with you and encourages us to continue developing and improving the library. Thank you for your support!

---

Contributors

Thanks for all of these amazing peoples made the project better!