Metadata-Version: 2.5
Name: erpdoo
Version: 0.1.0
Summary: Turn Odoo database changes into business events and stream them to Kafka and other sinks
Project-URL: Homepage, https://github.com/theicelotus/erpdoo
Project-URL: Documentation, https://github.com/theicelotus/erpdoo#documentation
Project-URL: Changelog, https://github.com/theicelotus/erpdoo/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/theicelotus/erpdoo/issues
License-Expression: MIT
License-File: LICENSE
Keywords: cdc,change-data-capture,kafka,logical-replication,odoo,postgresql,webhooks
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Office/Business
Requires-Python: >=3.11
Requires-Dist: confluent-kafka>=2.6
Requires-Dist: fastavro>=1.9
Requires-Dist: psycopg2-binary>=2.9
Requires-Dist: pyyaml>=6
Provides-Extra: dev
Requires-Dist: confluent-kafka[avro,json]>=2.6; extra == 'dev'
Requires-Dist: httpx2>=2; extra == 'dev'
Requires-Dist: jsonschema>=4.18; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Provides-Extra: viewer
Requires-Dist: fastapi>=0.110; extra == 'viewer'
Requires-Dist: uvicorn>=0.29; extra == 'viewer'
Description-Content-Type: text/markdown

# erpdoo

An Odoo Kafka connector and webhooks: change data capture that turns Odoo's database changes into clean business events, with nothing installed inside Odoo.

[![CI](https://github.com/theicelotus/erpdoo/actions/workflows/ci.yml/badge.svg)](https://github.com/theicelotus/erpdoo/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/theicelotus/erpdoo/blob/main/LICENSE)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://github.com/theicelotus/erpdoo/blob/main/pyproject.toml)

<!-- ![erpdoo demo: confirming a sale order in Odoo and the event it produces](https://raw.githubusercontent.com/theicelotus/erpdoo/main/docs/demo.gif) -->

## What it does

erpdoo reads Odoo's PostgreSQL logical replication stream directly, with no Debezium and no Kafka Connect, and turns each committed transaction into business events such as `SaleOrderConfirmed` or `InvoicePaid`, with the customer, order lines and products filled in.

Nothing is installed inside Odoo, and no code is needed per event: each event is a short YAML rule, and six ship with erpdoo. Events go to Kafka (JSON Schema or Avro, with schemas registered automatically), a webhook or stdout, and messages from other systems can act on Odoo in the other direction.

Version 0.1: tested end to end; configuration may change before 1.0.

```
Odoo ──▶ PostgreSQL ──(logical replication)──▶ erpdoo ──▶ Kafka / Redpanda / webhook / stdout
  ▲                                              │
  └───────────(Odoo's JSON-RPC API)──────────────┴◀── Kafka (inbound messages)
```

## Features

- **No code per event.** YAML rules on state transitions, field changes, creates and deletes; packs for sales, invoicing, inventory and CRM ship with erpdoo ([events](https://github.com/theicelotus/erpdoo/blob/main/docs/events.md), [rules](https://github.com/theicelotus/erpdoo/blob/main/docs/rules.md)).
- **Complete events.** A transaction's changes are grouped and evaluated at commit, and related records come from a local state store, without querying Odoo ([architecture](https://github.com/theicelotus/erpdoo/blob/main/docs/architecture.md)).
- **Exactly once to Kafka**, with Kafka transactions; at-least-once to webhooks, with deterministic event ids to dedupe on ([delivery](https://github.com/theicelotus/erpdoo/blob/main/docs/sinks.md#delivery)).
- **Schemas registered automatically**, one subject per event type in JSON Schema or Avro, with breaking rule changes refused at startup ([schemas](https://github.com/theicelotus/erpdoo/blob/main/docs/schemas.md)).
- **Backfill** of past events from Odoo's own field tracking, by itself after a lost replication slot or a late app install ([backfill](https://github.com/theicelotus/erpdoo/blob/main/docs/backfill.md)).
- **Inbound rules**: messages from a warehouse or payment provider act on Odoo through its API, exactly once and without event loops ([inbound](https://github.com/theicelotus/erpdoo/blob/main/docs/inbound.md)).
- **Live viewer** that shows each transaction's raw changes next to the events built from it ([viewer](https://github.com/theicelotus/erpdoo/blob/main/docs/architecture.md#live-viewer)).
- **Fast**: 18.4 ms p50 from Postgres commit to Kafka consumer, and 2,500 to 3,000 events a second through a backlog, on one laptop ([performance](https://github.com/theicelotus/erpdoo/blob/main/docs/performance.md)).

## Requirements

- Odoo 18 Community, on PostgreSQL 16: what erpdoo is developed and tested against.
- Python 3.11 or later, or Docker.
- A sink: Kafka or Redpanda (a schema registry is optional), or a webhook endpoint.
- **Replication access to Odoo's Postgres**: `wal_level=logical`, a role with `REPLICATION`, and a database admin who runs a few SQL statements once ([Postgres setup](https://github.com/theicelotus/erpdoo/blob/main/docs/postgres-setup.md)). erpdoo's role stays read-only.
- **Or, for hosted Odoo** without replication access: [polling mode](https://github.com/theicelotus/erpdoo/blob/main/docs/polling.md) reads Odoo's API instead, with real limits.

## Quick start

The dev stack runs Odoo 18 with demo data, Postgres 16, Redpanda with its schema registry, Redpanda Console and erpdoo. It needs Docker with Compose v2.

1. Start it. The first start downloads about 2 GB of images, then installs Odoo with demo data (about a minute).
   ```sh
   git clone https://github.com/theicelotus/erpdoo && cd erpdoo
   docker compose up -d --build
   ```
2. Follow erpdoo's log: `docker compose logs -f erpdoo`.
3. Open Odoo at http://localhost:8069 (login `admin`, password `admin`), go to **Sales**, open a quotation and click **Confirm**.
4. The `SaleOrderConfirmed` event appears in the log, and in Redpanda Console at http://localhost:8080: topic `erpdoo.events` has it as JSON Schema, `erpdoo.events.avro` in Avro, and **Schema Registry** lists each event type's schema. Posting a customer invoice gives an `InvoicePosted` event the same way.
5. Open the live viewer at http://localhost:8000 to see each transaction's raw changes next to its events.

A full example event is in [docs/events.md](https://github.com/theicelotus/erpdoo/blob/main/docs/events.md#an-event).

## Installation

For your own self-hosted Odoo:

1. Have a database admin prepare Postgres once, as described in the [Postgres setup guide](https://github.com/theicelotus/erpdoo/blob/main/docs/postgres-setup.md).
2. Tell erpdoo where events go, in `erpdoo.yml`:
   ```yaml
   sinks:
     - type: kafka
       bootstrap_servers: ${KAFKA_BOOTSTRAP:-kafka:9092}
       topic: erpdoo.events
       schema_registry:
         url: http://schema-registry:8081
   ```
3. Install and run erpdoo in one of these ways:
   - **With pip** (Python 3.11 or later), with the live viewer's extra. Copy the shipped rules from [rules/](https://github.com/theicelotus/erpdoo/blob/main/rules/) into the directory with `erpdoo.yml`, run erpdoo there, and set where Odoo's Postgres is and where the state store's SQLite file goes (on persistent storage):
     ```sh
     pip install 'erpdoo[viewer]'
     export ERPDOO_PG_DSN='host=db dbname=odoo user=erpdoo password=...' ERPDOO_STATE=/var/lib/erpdoo/state.db
     erpdoo
     ```
   - **With Docker**, from the published image, which has the viewer and the shipped rules. Mount your `erpdoo.yml`, and a volume for the state store:
     ```sh
     docker run -d --name erpdoo --restart unless-stopped \
       -e ERPDOO_PG_DSN='host=db dbname=odoo user=erpdoo password=...' \
       -v "$PWD/erpdoo.yml:/app/erpdoo.yml:ro" -v erpdoo-state:/var/lib/erpdoo \
       ghcr.io/theicelotus/erpdoo:0.1.0
     ```
     To use your own rules, mount them at `/app/rules`. For the live viewer, set `viewer: { host: 0.0.0.0 }` in `erpdoo.yml` and add `-p 127.0.0.1:8000:8000`.
   - **From source**, from a clone, with pip or by building the image:
     ```sh
     git clone https://github.com/theicelotus/erpdoo
     pip install './erpdoo[viewer]'          # or: docker build -t erpdoo ./erpdoo
     ```
4. Run it under a supervisor (systemd, or Docker's restart policy). When Postgres or a sink goes away it exits without confirming what is undelivered, and continues from there when restarted.

All settings, `ERPDOO_RULES` and `ERPDOO_CONFIG` among them, are in [Configuration](https://github.com/theicelotus/erpdoo/blob/main/docs/configuration.md).

## Usage

A rule is a model, a condition and the related records to include. This is the shipped `SaleOrderConfirmed`, shortened ([rules/sales.yml](https://github.com/theicelotus/erpdoo/blob/main/rules/sales.yml)):

```yaml
SaleOrderConfirmed:
  model: sale.order
  when:
    state: { to: sale }      # changed to sale in this transaction
  include:
    partner_id: [name, email, phone, vat, is_company]
    order_line: [name, product_uom_qty, price_unit, price_subtotal]
    order_line.product_id: [name, default_code]
```

A consumer on the dev stack, with confluent-kafka (`pip install 'confluent-kafka[json]'`), whose deserializer fetches each message's schema from the registry and checks the event against it:

```python
from confluent_kafka import Consumer
from confluent_kafka.schema_registry import SchemaRegistryClient
from confluent_kafka.schema_registry.json_schema import JSONDeserializer
from confluent_kafka.serialization import MessageField, SerializationContext

registry = SchemaRegistryClient({"url": "http://127.0.0.1:18081"})
deserialize = JSONDeserializer(None, schema_registry_client=registry)  # each message's own schema
consumer = Consumer({"bootstrap.servers": "127.0.0.1:19092", "group.id": "billing",
                     "auto.offset.reset": "earliest"})  # read_committed is librdkafka's default
consumer.subscribe(["erpdoo.events"])
while True:
    msg = consumer.poll(1.0)
    if msg is not None and not msg.error():
        event = deserialize(msg.value(), SerializationContext(msg.topic(), MessageField.VALUE))
        print(event["event_id"], event["event_type"], event["record"]["name"])
```

Commands:

```sh
erpdoo                                    # stream events, by the rules and erpdoo.yml
erpdoo backfill --since 2026-10-01T00:00:00Z --until 2026-10-02T00:00:00Z [--dry-run]
python -m erpdoo.models sale.order        # how an Odoo model maps to its table
docker compose restart erpdoo             # dev stack: after editing rules/ or erpdoo.yml
```

## Documentation

- [Architecture](https://github.com/theicelotus/erpdoo/blob/main/docs/architecture.md): how erpdoo works, the state store and the live viewer
- [Events](https://github.com/theicelotus/erpdoo/blob/main/docs/events.md): the shipped events and the event format
- [Writing rules](https://github.com/theicelotus/erpdoo/blob/main/docs/rules.md): the YAML rule format
- [Sinks and delivery](https://github.com/theicelotus/erpdoo/blob/main/docs/sinks.md): Kafka, webhook, stdout, dead-letter topics, exactly once
- [Schemas](https://github.com/theicelotus/erpdoo/blob/main/docs/schemas.md): JSON Schema and Avro, field types, compatibility, consumers
- [Inbound](https://github.com/theicelotus/erpdoo/blob/main/docs/inbound.md): messages that act on Odoo, the integration user, loop safety
- [Polling mode](https://github.com/theicelotus/erpdoo/blob/main/docs/polling.md): hosted Odoo through its API, and its limits
- [Backfill](https://github.com/theicelotus/erpdoo/blob/main/docs/backfill.md): rebuilding past events, apps installed later
- [Setting up Postgres](https://github.com/theicelotus/erpdoo/blob/main/docs/postgres-setup.md): the role, publication and grants
- [Slot health](https://github.com/theicelotus/erpdoo/blob/main/docs/slot-health.md): replication lag, WAL limits, invalidated slots
- [Performance](https://github.com/theicelotus/erpdoo/blob/main/docs/performance.md): latency, throughput, state store size, WAL cost
- [Configuration](https://github.com/theicelotus/erpdoo/blob/main/docs/configuration.md): environment variables and erpdoo.yml
- [Changelog](https://github.com/theicelotus/erpdoo/blob/main/CHANGELOG.md): what changed in each release

## Contributing

Bug reports and pull requests are welcome. [CONTRIBUTING.md](https://github.com/theicelotus/erpdoo/blob/main/CONTRIBUTING.md) covers the dev stack and the checks every change must pass.

## License

[MIT](https://github.com/theicelotus/erpdoo/blob/main/LICENSE)
