Skip to main content

EZPI: a simple library to read and write public-inbox repos

This is a small library for reading and writing public-inbox v2 repositories. Note, that this is not a suitable replacement for public-inbox in itself – the goal is merely to provide a way to produce and consume public-inbox repositories that can be cloned, indexed and served from an actual public-inbox server running elsewhere.

What it does:

  • adds RFC2822 messages to git repositories compatible with public-inbox

  • supports public-inbox v2 format with automatic epoch management

  • reads messages out of a local public-inbox v2 repository using named cursors, with automatic rebase recovery

What it doesn’t do:

  • fetch, clone, or otherwise synchronize with remote repositories (use git, lei, or rsync for that)

  • parse messages (bytes in, bytes out)

  • anything else

For the public-inbox v2 format, see:

Requirements

The git command must exist and be in your path. Reading features that rely on rebase recovery need git 2.40 or newer for the --since-as-filter flag; ordinary reading and all writing work on any recent git.

Installing

For CLI usage, install with pipx:

pipx install ezpi

For library usage, install with pip:

pip install ezpi

Library usage

This is the simplest usage example:

from email.message import EmailMessage
from ezpi import add_rfc822_v2

msg = EmailMessage()
msg.set_content('Hello world!')
# We must have a Subject: and From: headers at least, in order
# to make a useful git commit with that data
msg['Subject'] = 'My excellent subject'
msg['From'] = 'E.X. Ample <example@example.com'>
# The v2 inbox will be created automatically if it doesn't exist
add_rfc822_v2('/path/to/inbox', msg)

You can also pass bytes instead of an EmailMessage object, but we must be able to run message_from_bytes() on it.

Reading a public-inbox repository

EZPI can also read messages out of a local v2 inbox that you’re keeping in sync yourself (via git pull, lei up, etc). Readers identify themselves with a cursor name; EZPI remembers per-cursor where each reader left off.

import ezpi

# First call seeds the cursor at HEAD and yields nothing.
for epoch, commit, msg_bytes in ezpi.iter_new_messages('/path/to/inbox', 'myapp'):
    pass

# After the repo is refreshed (git pull etc), new messages show up here.
# auto_advance=True persists cursor state after each yielded message.
for epoch, commit, msg_bytes in ezpi.iter_new_messages(
    '/path/to/inbox', 'myapp', auto_advance=True
):
    handle(msg_bytes)

# Backfill the whole history instead of starting at HEAD:
for epoch, commit, msg_bytes in ezpi.iter_new_messages(
    '/path/to/inbox', 'myapp', start='beginning'
):
    handle(msg_bytes)

New epochs are detected automatically on every call. If the repository history is rewritten to delete a message, EZPI detects the stale cursor and transparently relocates it by matching on Message-ID, Subject, and commit date.

ezpi command

We also provide an ezpi command. Example:

ezpi --v2-path /path/to/inbox --rfc822 < valid.eml

Run ezpi --help for full usage information.

How to delete messages?

Since every message is a separate commit to the git repository, deleting requires a git history rewrite. You will need to find the commit with the message you want to delete and perform operations directly on the git repository in order to delete that commit and rebase the ones that follow. Once it’s done, you will need to force-push the repository to wherever it is hosted.

Caution: if anyone is replicating your repository without –mirror, the history rewrite will result in an error the next time they do “git remote update”. Needless to say, they will be very interested in finding out what it is you just tried to delete, so use this as the last resort solution.

Support

For support or with any other questions, please email tools@kernel.org, or browse the list archive at https://lore.kernel.org/tools.

Please send patches to tools@kernel.org.

Submissions must be made under the terms of the Linux Foundation certificate of contribution and should include a Signed-off-by: line. Please read the DCO file for full legal definition of what that implies.

Metadata

Release files for ezpi 0.7.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ezpi 0.7.0
File Size Uploaded
ezpi-0.7.0.tar.gz 39.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for ezpi 0.7.0
File Interpreter ABI Platform
ezpi-0.7.0-py3-none-any.whl Python 3 none any Details

Total release size: 61.0 kB

Release files / ezpi-0.7.0.tar.gz

Download URL ezpi-0.7.0.tar.gz
Size 39.4 kB
Tags Source
SHA-256 checksum
How to use checksums
85a123b747c6592e5ef5bb87120de9ead3bd11801412a9dea21aa1edc8ea94df
BLAKE2b-256 checksum
How to use checksums
7a0a9d8770f139902304457fd549eba6f9b704135db233a8a7cc9554f3b089b6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.7

Release files / ezpi-0.7.0-py3-none-any.whl

Download URL ezpi-0.7.0-py3-none-any.whl
Size 21.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
36f029c4ac83404c82a85d2286325a03f1e64538a28b132abc270dd289ed100c
BLAKE2b-256 checksum
How to use checksums
bc81608523b38618f91f57eaf67c0fea3541919bd84e099cf7f7c809bf6d9877
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.7

Release history Release notifications | RSS feed

This release

0.7.0 This release

2 release files

0.6

2 release files

0.5.1

2 release files

0.5.0

2 release files

0.4.0

2 release files

0.3.0

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page