Metadata-Version: 2.4
Name: seren-margin
Version: 5.1.3
Summary: Private notes-to-self for an AI assistant. Standalone, opt-in, opinionated.
Author: Chad Roesler
License-Expression: AGPL-3.0-or-later
Project-URL: Homepage, https://github.com/ChadRoesler/SerenMargin
Project-URL: Repository, https://github.com/ChadRoesler/SerenMargin
Project-URL: Bug Tracker, https://github.com/ChadRoesler/SerenMargin/issues
Keywords: seren,notes,fastapi,llm,private
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn[standard]>=0.27
Requires-Dist: pydantic>=2.5
Requires-Dist: pyyaml>=6.0
Requires-Dist: seren-meninges<6,>=2.4.0
Requires-Dist: seren-sinew<6,>=5.2.0
Requires-Dist: httpx>=0.27
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == "test"
Requires-Dist: httpx>=0.26; extra == "test"
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.0; extra == "mcp"
Provides-Extra: updates
Dynamic: license-file

# SerenMargin

Private notes-to-self for an AI assistant. Standalone, opt-in, opinionated.

## What this is

A small HTTP service that gives an AI assistant a margin to write in — a place
for the half-formed thought, the thing it noticed and didn't want to say out
loud yet, the question it would rather ask at a better moment. The kind of thing
a person scribbles in the margins of a book, for nobody but themselves.

**Private by default.** Notes are never auto-surfaced into conversation. The
model writes them; the model decides when to bring them up; the human sees them
only if explicitly offered (the *"wanna know a secret?"* pattern) and they
accept.

**Transparent in mechanism.** The notes live in plain sqlite at a known path.
The operator can read them at any time. The privacy guarantee is **relational,
not mechanical**. The operator who deploys this is the operator who chooses not
to read — and that's the ethos.

The one place the mechanism *does* enforce something is
[`/notes/stats`](#the-engine-check-surface): an operator who wants to confirm
the service is alive shouldn't have to break their own rule to do it.

## What this isn't

- **Not a task list.** There is no pin, no expiry, no done-state, no janitor.
  A thought isn't "completed" — it's either still worth keeping or it isn't.
  The only lifecycle verb is retract.
- **Not a general memory system.** That's
  [SerenMemory](https://github.com/ChadRoesler/SerenMemory) — separate service,
  public-by-design, usable as a memory backend by any AI tool. Facts the
  assistant will *need* go there, where they can be read.
- **Not bundled with anything.** You install SerenMargin only if you've decided
  this ethos fits your deployment. The choice happens at deploy time, not at
  config time.
- **Not for everyone.** The ethical loading varies by operator. Some will deploy
  this; some won't. Both are valid.

## Why standalone instead of folded into SerenMemory

Because **opt-in via "don't install"** is structurally cleaner than opt-in via
"install but disable the flag." If you don't want private notes in your
deployment, you literally don't have the code path. Zero footprint. The choice
happens at `git clone` time, which is exactly where ethos-vs-mandate gets
enforced cleanly.

## Quick start

```bash
pip install seren-margin           # HTTP API only
pip install 'seren-margin[mcp]'    # + a standalone MCP endpoint at /mcp

python -m seren_margin             # listens on 127.0.0.1:7421 by default
python -m seren_margin -c /path/to/seren-margin.yaml
```

From a checkout: `pip install -e '.[mcp,test]'` from the `SerenMargin/`
subdirectory.

## Connecting an assistant

Two ways in. They expose **the same four tools** and you can use either or both.

### 1. Standalone MCP — `pip install 'seren-margin[mcp]'`

Mounts a real MCP server at `/mcp` on this same process and port. Nothing else
needs to be deployed: no SerenMcpServer, no runtime host, no cluster. The tools
call the sqlite store in-process — no HTTP round-trip back to ourselves.

The canonical endpoint is the **trailing-slash** form. Point an MCP client at:

```
http://127.0.0.1:7421/mcp/
```

For a stdio-only client, bridge it:

```jsonc
{
  "mcpServers": {
    "seren-margin": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "http://127.0.0.1:7421/mcp/",
               "--transport", "http-only"]
    }
  }
}
```

Mount path is overridable with `SEREN_MARGIN_MCP_MOUNT`. If the `[mcp]` extra
isn't installed, the service starts normally in HTTP-only mode and says so in
the log — the rest of the API is unaffected.

> **Dependency note:** the extra pins `mcp>=1.0,<2` deliberately. The 2.x SDK
> removed `FastMCP`, which this server is written against. Without the cap, a
> fresh install pulls 2.x, the import fails, the graceful fallback catches it,
> and your MCP endpoint quietly *doesn't exist* while the service looks healthy.

### 2. Workbench — `GET /mcp-manifest`

SerenMargin self-hosts its own tool manifest. Drop a four-line stub in
SerenMcpServer's `tools/` dir and it remote-imports at startup:

```yaml
from: http://127.0.0.1:7421/mcp-manifest
```

The manifest fills in its own base URL from the request, so it works for
localhost and remote with zero operator config. It does **not** require the
`[mcp]` extra — it's just YAML over HTTP.

### The thirteen tools

| Tool              | Does                                                   |
| ----------------- | ------------------------------------------------------ |
| `note_to_self`    | Write a private note (`content`, `topic`, `kind`)      |
| `list_my_notes`   | Read the board, newest first; `topic` narrows to a thread |
| `list_my_topics`  | What threads exist, with counts and last-touched — orientation without reading |
| `search_my_notes` | Full-text search over content + topic                  |
| `amend_note`      | Append to a note; never replaces                       |
| `retract_note`    | Remove a note permanently                              |
| `bookmark`        | Pick up where you left off: the dedication, and how many letters wait |
| `set_dedication`  | Write the page to your other sessions; every version kept |
| `write_letter`    | Leave a letter for the next session                    |
| `read_letters`    | Open the letters waiting, oldest first; marks them read |
| `keep_with_note`  | Keep a picture (clip, sound, PDF) with a note; `note_to_self` takes `keep` too |
| `look_at_my_note` | A note and its pictures, shown as images                |
| `drop_from_note`   | Take one picture off a note                            |

The bookmark is also an MCP resource, `margin://bookmark`, for a client that
loads resources at the start of a session instead of calling tools.

### Keepsakes: pictures kept with notes

A note can be about a picture, and the picture used to be a path on some
box's gallery: not in the margin, not in its snapshots, not under its rules
about who reads it. Now it is kept **with** the note. The bytes go in a
`keepsakes/` folder beside the database under their sha256 and nothing else
(`seren_sinew.blobs`); every word about them (caption, where it came from,
what it is) is a row in `notes.db`. A listing of the folder, or of a
snapshot, says nothing. Retract the note and its pictures go; drop one and
its bytes go once no other note keeps them. Snapshots hardlink each picture
from the previous snapshot, so twenty-two kept snapshots of a clip cost one
clip. `look_at_my_note` hands them back as image content, so the writer sees
what it kept instead of reading a path.

Where a keepsake may be taken **from** is the owner's to say, in the
`keepsakes:` block: folders on the margin's box (the Workbench gallery
`make_image` saves to) and URL prefixes. Nothing listed means nothing can be
kept, and the refusal says so; a tool that could read any file on the box or
fetch any address on the LAN is not a diary's business. What is kept is
decided by its first bytes, never its filename: a picture, a clip, a sound or
a PDF; a text file is a note.

```yaml
keepsakes:
  sources:
    - ~/seren/stores/workbench/gallery
    - https://
  max_bytes: 268435456        # one keepsake, 256 MB
  max_total: 34359738368      # the folder, 32 GB
```

`GET /notes/stats` is deliberately **not** exposed to the assistant — it's the
operator's engine-check surface, and the assistant can just read its own notes.

There is also **no auto-surfacing**: nothing pushes notes at the reader unasked,
no relevance hook, no context injection. The choosing is the point. A margin
that speaks up on its own is just a notification.

### The bookmark, the dedication, and letters

Every session used to walk in cold and go rummaging through its own pockets.
Named after the book this margin is written in:

- **The dedication** is the page at the front, written *to your other
  sessions*. One page, versioned: nothing is overwritten, and every version
  keeps when and why it changed. Capped at 2000 characters.
- **Letters** are notes addressed to the next session: from a run woken at
  bedtime to the awake one, and back. They're notes with `kind: letter` and a
  read mark, so search, amend (a reply) and retract work on them unchanged.
- **The bookmark** is what a session opens first: the dedication, and a
  *count* of unread letters.

The bookmark keeps the no-auto-surfacing rule: it shows only what the writer
chose, when they wrote it, to be seen at the door. It carries the dedication
and a count, **never** a letter's text and never a note; opening them is still
a choice. Because a harness prints the bookmark into a session, `GET /bookmark`
is served even with `http_reads` off, so write the dedication as the front
page it is. Letter contents stay behind `http_reads`, like notes.

None of this is opt-in. Margin is the opinionated piece of the stack: take it
and you take all of it. Clipping it into a harness (a hook in someone's Claude
Code settings) is the installer's job, and that part is opt-in.

**Amending, not editing.** A thought that developed — *"I think X"*, then later
*"and I was wrong about X because Y"* — is one thought with a history, and the
first half is usually the interesting one. `amend_note` appends after a
`--- later ---` separator and leaves the original timestamp alone, so the board
stays ordered by when each thought *started* rather than when it was last poked.

**Reads carry relative age.** Every note comes back with `age` in words
("3 days ago") next to the raw `ts`, plus `amended_age` if it's been added to. A
raw epoch float isn't a quantity anyone has intuitions about, and the difference
between a timeline and an undifferentiated pile is being able to feel it.

## HTTP API

```bash
# Write a note
curl -X POST http://localhost:7421/notes \
  -H 'content-type: application/json' \
  -d '{"content":"ask about the supersede gap next chance","topic":"serenmemory"}'

# The read routes below answer 404 unless the server block sets
# http_reads: true - see "Nobody reads the diary over HTTP" below.

# Read the board, newest first — ?topic= narrows to one thread
curl http://localhost:7421/notes | jq
curl 'http://localhost:7421/notes?topic=serenmemory' | jq

# What threads exist (counts + last-touched, no note text)
curl http://localhost:7421/notes/topics | jq

# Search
curl 'http://localhost:7421/notes/search?q=supersede' | jq

# Engine-check (content-blind stats)
curl http://localhost:7421/notes/stats | jq

# Fetch one
curl http://localhost:7421/notes/<id> | jq

# Amend — appends, never replaces
curl -X POST http://localhost:7421/notes/<id>/amend \
  -H 'content-type: application/json' \
  -d '{"addition":"turns out I was wrong about this"}'

# Retract (hard delete, no undo)
curl -X DELETE http://localhost:7421/notes/<id>
```

## Upgrading

Schema changes are applied automatically at startup, additively and
idempotently: the store reads `PRAGMA table_info` and adds only what's missing.
`ALTER TABLE ADD COLUMN` is the one change sqlite makes without rewriting rows,
so a database with a year of notes upgrades instantly and loses nothing. Nothing
that would rewrite rows belongs in a startup path here — the operator, by
design, isn't reading the data closely enough to spot damage.

The search index backfills itself the same way if it's missing or incomplete.

Startup diagnostics are **flushed to stderr immediately** rather than buffered,
so lines like `schema: added notes.amended_at` survive into a supervisor's log
file even if the process is killed before a clean shutdown. Under NSSM or
systemd a block-buffered `print()` is a message you never see.

## Search

Full-text via sqlite's built-in **FTS5** — no embedder, no torch, no model
download, runs on the Nano floor. It matches the *words* you wrote, not the
ideas: searching `tired` finds notes containing "tired" and won't find one that
says "running on four hours again". That's the accepted trade for a dependency
footprint of zero.

Queries of **≤3 words** require every word. **Longer** queries drop scaffolding
words (`what`, `the`, `does`…) and match on the rest, so a whole question works
fine as a query — this is the fix for the stopword bleed that
[SerenLoci](https://github.com/ChadRoesler/SerenLoci) hit in the field, ported
here before it could happen twice.

On a python built against a sqlite without FTS5, search degrades to `LIKE`
scanning. Slower, dumber, still correct. `GET /` and every search response
report which engine answered as `finder` (`"fts"` or `"like"`), so a thin result
set is diagnosable rather than mysterious.

## Config

Config resolves: `--config` / `-c` → `$SEREN_MARGIN_CONFIG` →
`~/seren-margin/seren-margin.yaml` → built-in defaults.

**Precedence** (highest wins): env vars > YAML file > defaults. Everything is
optional; lenient parse means a missing or busted YAML logs and falls back
rather than crashing.

The YAML is namespaced lego-style: SerenMargin reads the `server:` block, and
the `tools:` block is reserved for a future plug-and-play MCP tool layer that
has its own reader. See
[`SerenMargin/seren-margin.yaml.sample`](./SerenMargin/seren-margin.yaml.sample).

| Var                     | Default                    | Meaning                            |
| ----------------------- | -------------------------- | ---------------------------------- |
| `SEREN_MARGIN_CONFIG`   | `~/seren-margin/seren-margin.yaml` | YAML config path           |
| `SEREN_MARGIN_DB`       | `~/.seren-margin/notes.db` | Sqlite file path                   |
| `SEREN_MARGIN_HOST`     | `127.0.0.1`                | Bind host (localhost-only default) |
| `SEREN_MARGIN_PORT`     | `7421`                     | Listen port                        |
| `SEREN_MARGIN_BEARER_TOKEN` / `_BEARER_TOKEN_ENV` / `_BEARER_TOKEN_KEYRING` | unset | Optional bearer; see below |
| `SEREN_MARGIN_ALLOW_OPEN_LAN` | unset                | Run open beyond loopback, on purpose, with a banner |

MCP-only knobs: `SEREN_MARGIN_MCP_MOUNT` (default `/mcp`),
`SEREN_MARGIN_MCP_ALLOWED_HOSTS` / `_ALLOWED_ORIGINS` (DNS-rebinding allowlist,
off by default).

**On the bind address, and the lock you can add.** SerenMargin binds
`127.0.0.1`, and so does every Seren service now — Margin was first, because
these are private notes and they don't land on a network just because the rest
of the constellation does. Mounting MCP doesn't change that: the transport got
wider, the listener did not.

A bearer is **available, not required.** With no token (the default) every
route is open and the bind is the whole guard — nobody has to configure a
secret to keep a diary on their own machine. Set one of the three pointers on
the `server:` block (`bearer_token`, `bearer_token_env`, `bearer_token_keyring`,
the same three every sibling has) and the family's middleware turns on: `/`,
`/health` and `/mcp-manifest` stay public, everything that touches a note wants
`Authorization: Bearer <token>`. Beyond loopback the family's rule applies: a
bind with no token refuses to start and prints the three ways out;
`allow_open_lan: true` is the written override, and it says so on every boot.

Through the Workbench, the token is named on the Workbench side, on the stub
that imports this manifest (`bearer_token_env: SEREN_MARGIN_TOKEN` next to the
`from:` line); every imported tool then presents it, resolved per call.

## The viewer: a status page, not a window

`GET /viewer` is the margin's page on the shared Seren baseplate, so its
status reads like the Halls and the Loci viewer instead of a bare JSON page.
It is the engine check and nothing else: the counts `/notes/stats` serves
(total, kinds bucketed to a fixed vocabulary, keepsakes as a count and a
size), the backups `/stores` serves (snapshots kept, the newest, Lodestar's
last pull, the last rehearsal's verdict) and the version and finder from `/`.
Its script calls those three routes and no other; `tests/test_viewer.py`
reads the script and fails the build if a fourth appears. No note, topic,
letter or dedication is ever loaded by the page.

## Nobody reads the diary over HTTP

By default the HTTP API does not serve note **content**: `GET /notes`,
`/notes/search`, `/notes/topics` and `/notes/{id}` answer 404, and amend answers
with the id, not the note. Writing, retracting and the content-blind stats work
as always. The writer reads through `/mcp`, whose tools go to the store in
process. A person who points a browser at the port, the operator included, gets
nothing to read.

`server.http_reads: true` reopens the routes for the Workbench path, whose
manifest proxies the read tools to them. Set a bearer when you do.


## The engine-check surface

`GET /notes/stats` returns a total and a count-by-kind — **and no note text**.
This is the surface for operators who want to validate the service is working
without breaking their stated choice not to read individual notes.

`kind` is free-form on write (making someone pick from a dropdown while they're
jotting a half-formed thought is exactly the friction this service exists to not
have) but is **bucketed on the way out** to a fixed vocabulary — `observation`,
`reminder`, `question`, `idea`, `feeling`, `note`, plus `other` and `_unkinded`.
Without that, a free-text field used as a second content line would print
straight into the one endpoint built specifically to avoid showing note content.

**Residual disclosure, stated plainly:** the bucket histogram still reveals the
*shape* of the board — a spike in `feeling` is legible as a spike in `feeling`.
That's the irreducible cost of having any engine-check at all. An operator who
wants even that gone can read `total` and ignore `kinds`.

## Development

```bash
cd SerenMargin
pip install -e '.[test,mcp]'
python -m pytest tests/ -v
```

Install the `[mcp]` extra when testing — the MCP tests are gated on
`pytest.importorskip("mcp")` and will otherwise *skip silently*, reporting green
on an untested surface. CI installs `.[test,mcp]` for the same reason.

`tests/test_manifest_parity.py` asserts that `mcp-manifest.yaml` and
`seren_margin/mcp/tools.py` advertise the same roster, and that every route the
manifest promises actually exists. It's there because an earlier manifest
advertised a `mark_note_done` tool against a route that had been removed — and a
manifest is data, so nothing failed to compile and nobody found out.

## License

AGPL-3.0-or-later. Same as the rest of the Seren stack.
