ezpi 0.6

A simple library to write content to public-inbox repositories

This is a small library for reading and writing public-inbox v2 repositories. Note, that this is not a suitable replacement for public-inbox in itself – the goal is merely to provide a way to produce and consume public-inbox repositories that can be cloned, indexed and served from an actual public-inbox server running elsewhere.

What it does:

• adds RFC2822 messages to git repositories compatible with public-inbox

• supports public-inbox v2 format with automatic epoch management

• reads messages out of a local public-inbox v2 repository using named cursors, with automatic rebase recovery

What it doesn’t do:

• fetch, clone, or otherwise synchronize with remote repositories (use git, lei, or rsync for that)

• parse messages (bytes in, bytes out)

• anything else

For the public-inbox v2 format, see:

• https://public-inbox.org/public-inbox-v2-format.html

Requirements

The git command must exist and be in your path. Reading features that rely on rebase recovery need git 2.40 or newer for the --since-as-filter flag; ordinary reading and all writing work on any recent git.

Installing

For CLI usage, install with pipx:

pipx install ezpi

For library usage, install with pip:

pip install ezpi

Library usage

This is the simplest usage example:

from email.message import EmailMessage
from ezpi import add_rfc822_v2

msg = EmailMessage()
msg.set_content('Hello world!')
# We must have a Subject: and From: headers at least, in order
# to make a useful git commit with that data
msg['Subject'] = 'My excellent subject'
msg['From'] = 'E.X. Ample <[e-mail]'>
# The v2 inbox will be created automatically if it doesn't exist
add_rfc822_v2('/path/to/inbox', msg)

You can also pass bytes instead of an EmailMessage object, but we must be able to run message_from_bytes() on it.

Reading a public-inbox repository

EZPI can also read messages out of a local v2 inbox that you’re keeping in sync yourself (via git pull, lei up, etc). Readers identify themselves with a cursor name; EZPI remembers per-cursor where each reader left off.

import ezpi

# First call seeds the cursor at HEAD and yields nothing.
for epoch, commit, msg_bytes in ezpi.iter_new_messages('/path/to/inbox', 'myapp'):
    pass

# After the repo is refreshed (git pull etc), new messages show up here.
# auto_advance=True persists cursor state after each yielded message.
for epoch, commit, msg_bytes in ezpi.iter_new_messages(
    '/path/to/inbox', 'myapp', auto_advance=True
):
    handle(msg_bytes)

# Backfill the whole history instead of starting at HEAD:
for epoch, commit, msg_bytes in ezpi.iter_new_messages(
    '/path/to/inbox', 'myapp', start='beginning'
):
    handle(msg_bytes)

New epochs are detected automatically on every call. If the repository history is rewritten to delete a message, EZPI detects the stale cursor and transparently relocates it by matching on Message-ID, Subject, and commit date.

ezpi command

We also provide an ezpi command. Example:

ezpi --v2-path /path/to/inbox --rfc822 < valid.eml

Run ezpi --help for full usage information.

How to delete messages?

Since every message is a separate commit to the git repository, deleting requires a git history rewrite. You will need to find the commit with the message you want to delete and perform operations directly on the git repository in order to delete that commit and rebase the ones that follow. Once it’s done, you will need to force-push the repository to wherever it is hosted.

Caution: if anyone is replicating your repository without –mirror, the history rewrite will result in an error the next time they do “git remote update”. Needless to say, they will be very interested in finding out what it is you just tried to delete, so use this as the last resort solution.

Support

For support or with any other questions, please email [e-mail], or browse the list archive at https://lore.kernel.org/tools.

Please send patches to [e-mail].

Submissions must be made under the terms of the Linux Foundation certificate of contribution and should include a Signed-off-by: line. Please read the DCO file for full legal definition of what that implies.