EZPI: a simple library to read and write public-inbox repos
===========================================================
This is a small library for reading and writing public-inbox v2
repositories. Note, that this is not a suitable replacement for
public-inbox in itself -- the goal is merely to provide a way to
produce and consume public-inbox repositories that can be cloned,
indexed and served from an actual public-inbox server running elsewhere.

What it does:

- adds RFC2822 messages to git repositories compatible with public-inbox
- supports public-inbox v2 format with automatic epoch management
- reads messages out of a local public-inbox v2 repository using named
  cursors, with automatic rebase recovery

What it doesn't do:

- fetch, clone, or otherwise synchronize with remote repositories
  (use ``git``, ``lei``, or ``rsync`` for that)
- parse messages (bytes in, bytes out)
- anything else

For the public-inbox v2 format, see:

- https://public-inbox.org/public-inbox-v2-format.html

Requirements
------------
The ``git`` command must exist and be in your path. Reading features that
rely on rebase recovery need **git 2.40 or newer** for the
``--since-as-filter`` flag; ordinary reading and all writing work on any
recent git.

Installing
----------
For CLI usage, install with pipx::

  pipx install ezpi

For library usage, install with pip::

  pip install ezpi

Library usage
-------------
This is the simplest usage example:

.. code-block:: python

  from email.message import EmailMessage
  from ezpi import add_rfc822_v2

  msg = EmailMessage()
  msg.set_content('Hello world!')
  # We must have a Subject: and From: headers at least, in order
  # to make a useful git commit with that data
  msg['Subject'] = 'My excellent subject'
  msg['From'] = 'E.X. Ample <example@example.com'>
  # The v2 inbox will be created automatically if it doesn't exist
  add_rfc822_v2('/path/to/inbox', msg)

You can also pass bytes instead of an EmailMessage object, but we must
be able to run message_from_bytes() on it.

Reading a public-inbox repository
---------------------------------
EZPI can also read messages out of a local v2 inbox that you're keeping
in sync yourself (via ``git pull``, ``lei up``, etc). Readers identify
themselves with a **cursor name**; EZPI remembers per-cursor where each
reader left off.

.. code-block:: python

  import ezpi

  # First call seeds the cursor at HEAD and yields nothing.
  for epoch, commit, msg_bytes in ezpi.iter_new_messages('/path/to/inbox', 'myapp'):
      pass

  # After the repo is refreshed (git pull etc), new messages show up here.
  # auto_advance=True persists cursor state after each yielded message.
  for epoch, commit, msg_bytes in ezpi.iter_new_messages(
      '/path/to/inbox', 'myapp', auto_advance=True
  ):
      handle(msg_bytes)

  # Backfill the whole history instead of starting at HEAD:
  for epoch, commit, msg_bytes in ezpi.iter_new_messages(
      '/path/to/inbox', 'myapp', start='beginning'
  ):
      handle(msg_bytes)

New epochs are detected automatically on every call. If the repository
history is rewritten to delete a message, EZPI detects the stale cursor
and transparently relocates it by matching on Message-ID, Subject, and
commit date.

ezpi command
------------
We also provide an ezpi command. Example::

  ezpi --v2-path /path/to/inbox --rfc822 < valid.eml

Run ``ezpi --help`` for full usage information.

How to delete messages?
-----------------------
Since every message is a separate commit to the git repository, deleting
requires a git history rewrite. You will need to find the commit with
the message you want to delete and perform operations directly on the
git repository in order to delete that commit and rebase the ones that
follow. Once it's done, you will need to force-push the repository to
wherever it is hosted.

Caution: if anyone is replicating your repository without --mirror, the
history rewrite will result in an error the next time they do "git
remote update". Needless to say, they will be very interested in finding
out what it is you just tried to delete, so use this as the last resort
solution.

Support
-------
For support or with any other questions, please email tools@kernel.org,
or browse the list archive at https://lore.kernel.org/tools.

Please send patches to tools@kernel.org.

Submissions must be made under the terms of the Linux Foundation
certificate of contribution and should include a Signed-off-by: line.
Please read the DCO file for full legal definition of what that implies.
