nafka 0.1

A rubbish Kafka broker, for testing simple messaging

A rubbish Kafka broker, for testing simple messaging

Nafka implements just enough of the Kafka broker for simple producer/consumer code to talk to it for testing. It's designed to be convenient to launch, run some tests, and shut down again.

Usage

The package provides a pytest fixture nafka_broker. The broker is running while the fixture is in use, and it has these attributes and methods:

• nb.address: The broker address to connect to, a string like "127.0.0.1:32977"
• nb.topics(): Get a list of topic names for which at least 1 record exists.
• nb.records(topic, start_offset=0, limit=1000): Get a list of records in the given topic.
• nb.next_offset(topic): Get the offset for the next record to be added to the given topic.
• with nb.assert_produces(topic): Check that there are more records in the topic at the end of the with block than at the start. Using with ... as new_recs: also gives an iterable to inspect the new records.
• nb.add_record(topic, value, key=None, timestamp_ms=None): Add a record as if it was sent by a producer. The new record is sent to any waiting consumers.
• nb.consumer_offset(group, topic): Get the committed consumer group offset for the given group in the given topic.

The records are saved in a kafka.sqlite file in a per-test temp directory managed by pytest - this can be useful for investigation.

Limitations

• Standalone single broker, no distributed architecture
  • One partition per topic
• Not optimised for performance
• No log compaction: messages are all stored (until test cleanup removes the entire store)
• Listens only on localhost
• Many Kafka features are not implemented, only what we've needed for tests