Metadata-Version: 2.4
Name: slate-client
Version: 0.2.0
Summary: A typed Python client for the slate-orm gRPC head node
Author: slate-orm contributors
License-Expression: Apache-2.0
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: grpcio>=1.60
Requires-Dist: protobuf>=5.27
Provides-Extra: dev
Requires-Dist: grpcio-tools==1.83.1; extra == "dev"
Requires-Dist: mypy-protobuf==5.1.0; extra == "dev"
Requires-Dist: ty==0.0.82; extra == "dev"
Requires-Dist: build>=1; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-timeout>=2; extra == "dev"
Requires-Dist: ruff==0.16.7; extra == "dev"

# slate-client

A typed Python client for the `slate-orm` gRPC head node.

```python
from slate import Client, Column, Identity, Query, Table, ValueType, desc, i64, u64

docs = Table(
    "docs",
    [
        Column("id", ValueType.U64),
        Column("kind", ValueType.STR),
        Column("size", ValueType.I64),
        Column("note", ValueType.STR),
    ],
    primary_key=["id"],
)

with Client("127.0.0.1:50051", Identity("u64:1", roles=["app"])) as client:
    client.insert(docs, [(u64(1), "kind-a", i64(5), "first")])

    # Reads the write back — from a replica, without naming a token. The
    # session threaded the sequence the commit returned.
    q = Query(docs)
    for row in client.query(q.where(q.c.size >= 5).sort(desc(q.c.size)).limit(10)):
        print(row.get("kind"), row.get("size"))
```

The most useful thing in this directory is **[`PROTOCOL-FINDINGS.md`](PROTOCOL-FINDINGS.md)**:
fifteen places where the wire is awkward, underspecified or wrong, written from
the position of the first consumer of it outside Rust. The client is the
evidence for that document as much as it is a deliverable.

---

## The four decisions

### A column reference always comes from a builder, never from a table

`ColumnRef` names a producer and an index inside it, and the server does the
ordinal arithmetic — so that a client never adds a table width to anything, and
a column added to an early table cannot silently re-point a predicate over a
later one. A Python SDK can throw that away in one line, by letting a `Table`
produce references: they would have to carry `input=0`, which is right for the
first input of a join and silently wrong for every other.

So a `Table` produces no references at all. Every reference comes from a
`Columns` accessor that was handed its position by the builder that assigned
it:

```python
j = JoinQuery()
authors = j.add(AUTHORS)                                   # position 0
books = j.add(BOOKS, on=[(authors.c.id, "author_id")])     # position 1
books.having(books.c.year > authors.c.born)
```

`authors.c.id` is input 0's column 1; `books.c.year` is input 1's column 4.
There is no `input=` argument anywhere in the API, and no way to get a
reference that is not bound to a position. Adding the same `Table` twice is a
self-join and needs nothing else — an input is a position, which a self-join
distinguishes and a name cannot.

The `own` side of a join condition is a **column name of the table being
added**, not a free reference, so it cannot be pointed at another input. A
reference is accepted there too and is checked.

`tests/test_refs.py` asserts the built protobuf rather than the returned rows,
because the failure this guards against is a request the server *accepts*.

### The session threads the read token; the caller can still see it

Write and then read, and the read sees the write — from a replica, with no
token in the caller's code. `Session` holds a watermark: the highest sequence
it has observed from its own commits and (by default) from the views that
served its reads. Every read carries it as `Freshness.at_least`.

Hiding it completely would have been worse than exposing it. So:

| you want | you write |
|---|---|
| read your own writes | nothing; it is the default |
| a read that may be stale | `freshness=Freshness.any()` |
| the writer's view | `freshness=Freshness.latest()` |
| to know how stale an answer was | `stream.served_by` |
| to hand freshness to another process | `session.watermark` → `other.observe(token)` |
| a scope per end user | `client.session()` |
| to stop paying for monotonic reads | `Client(..., monotonic_reads=False)` |

A `Client` *is* a `Session`, because the common case is one process acting as
one caller. A process serving many users should call `session()` per user, so
that one user's write does not pin another's reads to the writer.

`tests/test_freshness.py` runs against a head node whose pool holds a replica
that is empty and will never catch up, and every test that asserts a token
worked also asserts the control — that the same read without one really does
miss the write. Without the control the whole file would pass against a client
that ignored freshness entirely.

### Errors are a hierarchy about retryability, and there is no string matching

```
SlateError
    Retryable                 ← the same call again may succeed
        Conflict              ABORTED — retry the whole transaction
        Unavailable           UNAVAILABLE — retry, possibly elsewhere
            NotLeader         ... and `.leader` says where
        ResourceLimit         RESOURCE_EXHAUSTED
    InvalidRequest  NotFound  AlreadyExists  PermissionDenied  Unauthenticated
    UnknownOutcome            UNKNOWN — the write may have landed. NOT Retryable.
    DataLoss  DeadlineExceeded  Cancelled  InternalError
```

`UnknownOutcome` not being `Retryable` is the load-bearing line: the server maps
a commit timeout to `UNKNOWN` precisely so a client does not retry an insert
that may already have succeeded, and a hierarchy that put it under `Retryable`
would undo that decision everywhere.

`Session.transact` retries `Conflict` and nothing else — not `Unavailable`,
even though it is `Retryable`, because a fenced writer is permanent for that
node and the retry that helps is against a different one. That is
`RecordStore::transact`'s own rule.

The mapping stops at the status code. The finer distinctions behind
`UNAVAILABLE` and `ALREADY_EXISTS` exist only in the message text, and matching
on prose is not something this client does; finding 6 explains why and what
would fix it.

### The generated stubs are committed

Rather than generated at install time. The argument is `build.rs`'s, one layer
out: a build that depends on an external code generator fails differently on
every machine, and generating at install time makes a wheel's contents depend
on which `grpcio-tools` the installing machine resolved. A committed generated
file also appears in a diff, so a protocol change is reviewed rather than
absorbed.

The cost of committing a generated artifact is drift, so the freshness is
asserted rather than trusted: `tests/test_generated.py` regenerates into a
temporary directory and requires byte equality. Regenerate with
`python scripts/generate_proto.py`.

Which is why the generators are pinned to exact versions in the `dev` extra.
The paragraph above objects to a wheel whose contents depend on which
`grpcio-tools` the installing machine resolved — and with a `>=` pin the
*freshness test* had that same dependency. It went red in CI with nothing
changed in this repository, on a `grpcio-tools` release that emitted a
different `_pb2_grpc.py`. Raising a pin is a deliberate change: bump it,
regenerate, and commit the stubs together, so the diff shows what the new
version did.

Types come from `mypy-protobuf` rather than protoc's own `--pyi_out`, because
protoc types the messages and leaves the *service stub* untyped — and an
untyped stub is exactly the boundary where a wrong field name survives type
checking.

---

## Deadlines

Every call takes the session's deadline, and there is none by default:

```python
client = Client("127.0.0.1:50051", timeout=5.0)   # seconds
slow = client.with_timeout(60.0)                  # a view, for a big scan
rows = list(slow.query(Query(TRIPS)))
```

`with_timeout` returns a *new* session over the same connection and the same
freshness scope — so a write through one is visible to a read through the
other, and a short deadline cannot be left switched on by a caller who forgot
to restore it. A transaction inherits the deadline of the session that began
it.

The deadline covers a whole streaming call rather than each message, which is
what a gRPC deadline means and why this is per call rather than one number for
the connection: a point get and a hundred-thousand-row scan do not want the
same value.

**Seconds here, milliseconds in the TypeScript client** — each follows its own
language, and each names the unit in the parameter.

There is no default, deliberately. Adding one would turn a slow query into a
failure in every caller that upgraded without asking for it. The cost of that
choice is that a head node which accepts a connection and then stops answering
blocks a caller for ever, which `tests/test_deadlines.py` demonstrates.

## Configuring the transport

`Client` takes a built `channel=` and uses it as given:

```python
counter = Counting()
channel = grpc.intercept_channel(grpc.insecure_channel(address), counter)
client = Client(address, APP, channel=channel)
```

Nothing is added to it and nothing is inspected — credentials, keepalive,
message-size limits and interceptors are all the caller's to set, because a
curated subset would need widening every time grpc did.

**Register both interceptor interfaces if you want to see every call.** A read
is server-streaming and a write is unary, so a `UnaryUnaryClientInterceptor`
alone counts writes and reports zero for every query, which reads as "a read is
free" rather than as a hole in the instrument. `tests/test_round_trips.py`
registers both, and the Go client's own counter was found to have exactly this
gap.

## Decimals, and the conditional update

A decimal column holds a count of its own smallest unit. `Units(1250)` in a
column declared `scale=2` is 12.50, and the identical value in a `scale=0`
column is 1250 — the protocol publishes no schema, so nothing on the wire says
which. `Units.to_string_with_scale` therefore takes the scale as an argument,
and the scale itself is declared on the `Column`:

```python
PRICES = Table(
    "prices",
    [Column("id", U64), Column("label", STR), Column("amount", ValueType.DECIMAL, scale=2)],
    primary_key=["id"],
)

client.insert(PRICES, [(u64(1), "coffee", Units(1250))])
row = client.get(PRICES, (u64(1),))
row.get("amount").to_string_with_scale(PRICES.scale_of("amount"))  # "12.50"
```

A `decimal.Decimal` is **refused** rather than scaled: it carries a scale of
its own, there is nothing on the wire to reconcile it against, and a client
that guessed would be off by a factor of ten on a column it guessed wrong
about. A bare `int` against a column declared `DECIMAL` is taken as units,
which is the same rule the `i64`/`u64` hint follows.

`update(..., expected=...)` is optimistic concurrency, per row: one expected
row per updated row, in the same order, each as this caller last read it. The
server compares the *whole stored row* and raises `Aborted` (`ROW_CHANGED`) if
anything about it moved, so a read-modify-write that raced another writer is
reported rather than silently overwriting their edit.

```python
client.update(
    PRICES,
    [(u64(1), "coffee", Units(1400))],
    expected=[(u64(1), "coffee", Units(1250))],
)
```

The whole row rather than a version column, for the reason the kernel gives: a
version column only catches writers who remembered to bump it, which makes it a
convention every call site has to keep rather than a property of the data.

`delete(..., expected=...)` is the same argument for a delete — with one
difference worth knowing. A plain delete reports a key that is not there in
`affected`, because "make sure this is gone" is idempotent; a conditional one
raises `NotFound`, because a caller that said what it expected to find wants to
hear that somebody got there first. `NotFound` and not `Conflict`: a row that
moved can be re-read and the decision remade, and a row that is gone cannot, so
a retry loop on a conflict would spin.

## Windows, arrays, full-text, views and soft delete

Five features shipped after this client did. All five are reachable; none has
a section above, which is why they are listed here.

```python
# A window value, and a sort key naming it. `windowed(i)` is on the query,
# because a window belongs to the query rather than to a table.
q = Query(docs)
ranked = q.window(Window.row_number().over(partition=[q.c.kind], order=[asc(q.c.id)]))
for row in client.query(ranked.sort(desc(q.windowed(0)))):
    rank = row.windowed(0)

# An array column, and a value for one.
Column("tags", ValueType.ARRAY, element=ValueType.STR)
client.insert(posts, [(u64(1), Array(["a", "b"]))])

# Full text, over a column with a text index.
q.where(q.c.title.contains("slate"))

# A view: the same declaration under the catalog's name for it.
docs.as_view("recent")

# Soft delete: reading retired rows, and erasing them.
q.include_deleted()                              # needs `read_deleted`
client.purge_deleted(docs, before=cutoff)        # needs `delete`, `read` and `read_deleted`
```

`include_deleted` without `read_deleted` is **refused**, not ignored: a read that
asked to see retired rows and silently did not is worse than one that failed.

There is no `restore`. A soft delete stamps a column, so undoing one is an
ordinary `update` writing null into that column — which works today, given
`read_deleted`. A dedicated call would be a second way to write one column,
with its own authorization to keep in step.

## Layout

```
src/slate/
    expr.py        ColumnRef, Columns, Expr — the reference model
    scalar.py      computed values
    query.py       Query, JoinQuery, AggregateQuery, Agg
    schema.py      Table — the local copy of the catalog, and its limits
    values.py      Python ↔ Value, and the integer-width problem
    freshness.py   Freshness, ReadToken, ServedBy, Watermark
    rows.py        Row, JoinedRow, Group
    client.py      Client, Session, Transaction, the streams
    errors.py      the exception hierarchy
    _proto/        generated, committed
testserver/        a Rust head node with a `main`, because the crate has none
tests/             pytest, against that head node — no mocks
scripts/
    generate_proto.py
    mutate.py      break each thing; check a named test notices
```

## Running the tests

```
uv pip install -e '.[dev]'  # or pip; uv is what CI uses
pytest                      # builds the head node with cargo and drives it
ty check                    # over src/slate and tests/
ruff check .                # a finding here is a failure, not a suggestion
python scripts/mutate.py    # the mutation table
```

Every test runs against a real head node started as a subprocess. There are no
mocks: a mock is a second statement of what the server does, written by whoever
wrote the client, so it agrees with the client's misunderstandings — and the
whole value of a first client is finding out where those are.

The strongest test is `tests/test_oracle.py`. The test server runs a fixed list
of reads **in process** against `slate-kernel` and dumps the answers; the Python
tests build what should be the same reads with this SDK, ask them over the
socket, and require the two to agree. The two statements are deliberately not
shared — the Rust side is written against the kernel's flat ordinals and the
Python side against the wire's `ColumnRef` — because a helper that built both
would make them agree by construction.

## Not covered

`PROTOCOL-FINDINGS.md` ends with the list. In short: `UNKNOWN` and `DATA_LOSS`
(no way to provoke them with an in-memory store), a real fence, a replica that
genuinely catches up, and every `Value` type the fixture schema has no column
of — vectors, UUIDs, bytes, and the time and distance scalars are built and
type-checked and never executed.
