What happens when UltraDict2 runs out of memory

UltraDict2 keeps a dict in POSIX shared memory across processes. Three separate shared segments are involved, and they exhaust in three different ways, with a fourth mode where nothing is exhausted at all but a reader still fails. Knowing which one you are looking at is the difference between raising buffer_size and hunting a leak.

§0The three segments

A dict named N owns a fixed 1000-byte control segment, an update buffer named N_memory, and one or more full dump segments. Writes normally append a serialized frame to the update buffer. When that buffer cannot fit the next frame, the whole dict is serialized into a full dump segment instead and the buffer is reset to empty.

The control segment is the coordination point. Its layout is packed into the first 284 bytes; the remaining 716 bytes are unused today.

§1The update buffer fills By design

This is the common case and it is not an error. append_update() serializes the change, then checks whether the frame fits:

if end_position > self.buffer_size:          # :854
    self.apply_update()
    if not delete:
        self.data.__setitem__(key, item)
    self.dump()
    return

Note what this costs. A write that overflows does not get appended to the stream at all — it reaches the other processes only inside the full dump. So a 20-byte write to a 500 MB dict can trigger a 500 MB serialization. Peers then discover the new full_dump_counter and each reloads the entire dict.

sequenceDiagram
    autonumber
    participant W as Writer
    participant B as Update buffer
    participant C as Control
    participant D as Dump memory
    participant R as Reader

    W->>B: append frame
    W->>C: position += frame size
    R->>C: read position
    R->>B: replay new frames

    Note over W,B: next frame does not fit

    W->>W: apply_update then set locally
    W->>D: serialize the whole dict
    W->>C: publish dump name
    W->>C: full_dump_counter += 1
    W->>C: position = 0

    R->>C: counter changed
    R->>D: load full dump
    R->>R: discard local stream position
    

§2The full dump memory is too small Fails cleanly

Only reachable with a static full_dump_size. That segment is allocated once at construction and is never resized, so once the dict outgrows it, dump() raises:

if length + 6 > full_dump_memory.size:
    self.full_dump_memory_full += 1
    raise Exceptions.FullDumpMemoryFull(...)

The exception propagates out through dump(), append_update() and __setitem__ to your code. What matters is the state it leaves behind, and that used to be bad. The overflow branch in §1 publishes nothing to the stream — the change rides entirely on the dump — so a failed dump once left the writer holding an item that existed in exactly one process and nowhere else, with the buffer still full so every later write raised again. Permanently wedged, silently divergent.

The local mutation now lives inside append_update, which owns the publish. It snapshots what it is replacing, applies the change, and puts it back if the dump does not happen:

had_key  = key in self.data
old_item = self.data.get(key)
...
try:
    self.dump()
except Exception:
    if had_key:
        self.data.__setitem__(key, old_item)
    else:
        self.data.pop(key, None)
    raise

The snapshot is taken after the branch's own apply_update(), so it reflects what peers see rather than a pre-reload guess, and dump() raises before writing a single remote byte. So the failure is now atomic: the write either lands everywhere or nowhere.

It also self-heals. Because dump() serializes self.data after the speculative mutation, a del dumps the reduced dict — so deleting your way out of a full dict works. What does not recover is a dict that exceeds full_dump_size even at its smallest: that keeps raising on every mutation. But the dict stays consistent while stuck, and reads keep working.

flowchart TD
    A["__setitem__"] --> B["append_update serializes frame"]
    B --> C{"frame fits in buffer?"}
    C -->|yes| D["apply locally, write frame, bump position"]
    D --> E["peers see it"]
    C -->|no| F["snapshot old value, apply locally"]
    F --> G["dump serializes whole dict"]
    G --> H{"dict fits in dump memory?"}
    H -->|yes| I["publish dump, reset position"]
    I --> E
    H -->|no| J["restore the snapshot"]
    J --> K["raise FullDumpMemoryFull"]
    K --> L["local copy matches peers"]
    

§3The system runs out of shared memory Typed

With no static full_dump_size, every dump allocates a brand-new segment sized to fit, then unlinks the previous one:

full_dump_memory = self.get_memory(create=True, size=length + 6)   # :693
...
if old and old != full_dump_memory.name and not self.full_dump_size:
    self.unlink_by_name(old)                                       # :740

Steady state is therefore two live dump segments at most. But the allocate-then-unlink order means a process that dies between those two lines leaks the old segment permanently. Nothing reclaims it: /dev/shm entries survive the process that made them. Repeat that under a crash loop and tmpfs fills.

When it does, SharedMemory(create=True, ...) raises a plain OSError. That used to escape unwrapped, making "the machine is out of shared memory" indistinguishable from any other OS error — arriving from an ordinary-looking d[k] = v. It is now caught and re-raised as Exceptions.CannotCreateSharedMemory, with a message naming the actual remedies and the original error kept as __cause__.

Note the filter is the exception type, not the errno. Errno is not dependable here: an exhausted Windows paging file reports EINVAL, not ENOSPC, so an errno allowlist would miss the most likely real failure. Since SharedMemory(create=True) can only fail for host-resource reasons or a name collision, and the collision is handled separately, catching every OSError at that call is the precise filter.

OverflowError is caught alongside it. On a 32-bit build a size beyond ssize_t is rejected by mmap before the OS is ever asked — the segment still cannot be created, and reporting that as an arithmetic error from deep inside mmap tells the caller nothing useful.

flowchart TD
    A["dump needs memory"] --> B["allocate new segment sized length + 6"]
    B --> C["write dump, publish name"]
    C --> D["unlink previous segment"]
    B -.->|"process dies here"| E["previous segment leaked"]
    E --> F["tmpfs usage grows"]
    F --> G["shm_open returns ENOSPC"]
    G --> H["CannotCreateSharedMemory from __setitem__"]
    

§4Readers cannot keep up Bounded, then typed

Nothing is exhausted here. The buffer is being recycled faster than a slow reader can replay it. Readers replay the stream without a lock, so a dump can reset the buffer under a reader mid-replay. Two guards catch it, and both recover by starting over.

Those retries used to be unbounded. A reader that could never catch up recursed until RecursionError — raised from an ordinary d['x'], len(d) or iteration, since all of them enter apply_update(). Both paths now carry a retry count, following the bounded pattern get_full_dump_memory() already used: three lock-free attempts, then a final one under the lock, so the last try is exclusive rather than another dice roll.

if retry < max_retry:
    return self.load(force=True, max_retry=max_retry, retry=retry + 1)
elif retry == max_retry:
    # On the last retry, take the lock so nobody can dump while we read
    with self.lock:
        return self.load(force=True, max_retry=max_retry, retry=retry + 1)
raise Exceptions.FullDumpsTooFast(...)

The recursion is deliberately kept rather than rewritten as a loop: the final apply_update retry runs while holding the lock, and a continue would release it before retrying, quietly deleting that guarantee.

One subtlety worth knowing if you touch this code: FullDumpsTooFast must not subclass AssertionError. CorruptedStream does, precisely so the recovery handlers catch it — so a give-up signal that also inherited from AssertionError would be caught by the very handlers raising it and retried forever. The fix for infinite recursion would itself recurse infinitely.

The warning is still the leading signal: it means you are close to the bound.

sequenceDiagram
    autonumber
    participant W as Writer
    participant C as Control
    participant R as Reader

    R->>C: read position, begin replay without lock
    W->>C: dump, counter += 1, position = 0
    W->>C: dump again, counter += 1
    R->>R: unpickle garbage, CorruptedStream
    R->>C: counter moved, so recover
    R->>R: apply_update again, retry 1 of 3
    Note over R: writer may have dumped again already
    R->>R: final retry runs under the lock
    R->>R: still behind, raise FullDumpsTooFast
    

§5Windows and macOS differ

On Linux a shared segment outlives the process that created it, so the writer can close its handle right after dumping. On Windows the mapping is destroyed when the last handle closes, so the code deliberately holds the handle open instead:

if not self.full_dump_size and sys.platform != 'win32':   # :714
    full_dump_memory.close()

The consequence is that on Windows, a dynamically sized dump lives only as long as the process that wrote it. If that process exits, later readers get CannotAttachSharedMemory. That is why a static full_dump_size is recommended on Windows — and it is exactly the configuration that makes §2 reachable. The two platforms trade one failure mode for the other.

Also note buffer_size and full_dump_size are rounded up to a 4096 multiple on Windows, so buffer_size=10_000 reports as 12288.

macOS is a third case: it is POSIX and shared memory works, but there is no /dev/shm directory to measure, so every shm_* metric is None there, as on Windows.

§6Which metric predicts which failure

All of these come from ultra.get_metrics(). See the Metrics section of the readme for the Prometheus collector.

§7Sizing the buffer, with numbers

tests/performance/buffer_size_sweep.py sweeps buffer_size against a fixed payload and write pattern. Measured on a 100 KB dict in 200 keys, 2000 writes of 50 bytes, one reader following along:

Amplification is bytes moved per byte of real change, counting the writer only — multiply by the number of reader processes. The knee is at 64 KiB for this shape: the undersized buffer moves 16x more bytes than the change is worth and runs at 45% of the throughput.

Read the third column carefully. The worst row reports the buffer as 0.0% used. A buffer too small resets on every dump, so it looks idle exactly when it is thrashing. buffer_used_fraction is only meaningful once the buffer is adequate — alert on buffer_full_forced_dump_total instead.

§8What is still not fixed

Three known gaps remain, each deliberately out of scope so far:

1. The recurse-mode child leak. With recurse=True, assigning a nested dict builds a child UltraDict — creating shared segments and registering its name — before the parent's own write is published. If that write then fails, the child segment and its registry entry persist. Unwinding them would mean unlinking a segment a peer may already have attached to, so doing nothing is the safer bug. The leak is bounded: unlink_recursed() sweeps the register when the parent closes.
2. update() is not atomic across keys. It loops self[k] = v, publishing each key separately, so a failure part-way leaves earlier keys visible to peers. Individual keys are never torn — only the batch. Real batch atomicity needs a multi-key frame format.
3. The name-then-counter publish window in dump(). The new segment's name is written to the control block before the dump counter is incremented, so there is a brief window advertising a new segment under the old generation. The file's own TODO flags it.