Metadata-Version: 2.4
Name: pymembus
Version: 1.0.2
Summary: Python shared memory library
Home-page: https://github.com/wheresjames/pymembus
Author: Robert Umbehant
Author-email: pymembus@wheresjames.com
License: MIT
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: author
Dynamic: author-email
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: summary

# pymembus

Python bindings for [`libmembus`](https://github.com/wheresjames/libmembus), a
small shared-memory IPC library.

`pymembus` is useful when multiple local processes need to exchange data with
low overhead:

- raw shared memory blocks with `memmap`
- broadcast message queues with `memmsg`
- command channels with `memcmd`
- fixed-schema shared key/value state with `memkv`
- video ring buffers with `memvid`
- audio ring buffers with `memaud`
- simple readiness polling with `select()`

All shared-memory names in the examples use POSIX-style names such as
`"/myshare"`. On Linux, stale objects can remain after a crash; each type has a
`remove(name)` helper for cleanup.

## Contents

- <a href="#install">Install</a>
- <a href="#build-from-source">Build From Source</a>
- <a href="#run-tests">Run Tests</a>
- <a href="#quick-start">Quick Start</a>
- <a href="#api-guide">API Guide</a>
- <a href="#diagnostics">Diagnostics</a>
- <a href="#command-line-helpers">Command Line Helpers</a>
- <a href="#troubleshooting">Troubleshooting</a>
- <a href="#development-notes">Development Notes</a>

<a id="install"></a>

## Install

From PyPI:

```bash
python3 -m pip install pymembus
```

If you build from source, install system build dependencies first. On Debian or
Ubuntu:

```bash
sudo apt-get update
sudo apt-get install -y build-essential git cmake libboost-all-dev
sudo apt-get install -y python3 python3-pip
```

Optional tools used by this repository's CMake documentation targets:

```bash
sudo apt-get install -y doxygen graphviz go-md2man
```

<a id="build-from-source"></a>

## Build From Source

Install this checkout into your active Python environment:

```bash
python3 -m pip install .
```

Or build with CMake directly:

```bash
cmake -S . -B ./bld -DCMAKE_BUILD_TYPE=Release
cmake --build ./bld -j
```

The source build fetches pinned third-party dependencies when needed:

- `libmembus` `v1.2.0`
- `pybind11` `v2.13.6`

`libmembus` 1.2.0 requires C++20 and Boost stacktrace support. The Python build
requires CMake 3.30 or newer.

To uninstall a pip install:

```bash
python3 -m pip uninstall -y pymembus
```

To build distribution artifacts:

```bash
python3 setup.py sdist
python3 setup.py bdist_wheel
```

<a id="run-tests"></a>

## Run Tests

The test suite uses pytest. If you built with CMake:

```bash
cmake -S . -B ./bld -DCMAKE_BUILD_TYPE=Release
cmake --build ./bld --target pymembus-test
```

You can also run pytest from the repository root:

```bash
python3 -m pytest -v
```

Wheel builds made through scikit-build do not run the CMake test target during
`install`; run tests explicitly with one of the commands above.

The pytest configuration in `pyproject.toml` limits collection to
`src/pytest/py` and adds `bld/lib` to `PYTHONPATH`, so it will not try to run
vendored tests under `bld/_deps`.

The suite covers maps, messages, commands, key/value stores, video/audio
formats, NumPy buffer sharing, diagnostics, and `select()`. NumPy-specific tests
are skipped when NumPy is not installed.

<a id="quick-start"></a>

## Quick Start

```python
import pymembus

if hasattr(pymembus, "pymembus"):
    pymembus = pymembus.pymembus

name = "/quickstart"
pymembus.memmsg.remove(name)

tx = pymembus.memmsg()
rx = pymembus.memmsg()

assert tx.open(name, 1024, True, True)      # writer, create
assert rx.open(name, 1024, False, False)    # reader, attach

assert tx.write("hello")
message, overrun = rx.read_with_overrun(0)

assert message == "hello"
assert not overrun

rx.close()
tx.close()
pymembus.memmsg.remove(name)
```

<a id="api-guide"></a>

## API Guide

The snippets below assume the normalized import pattern from the quick start:

```python
import pymembus

if hasattr(pymembus, "pymembus"):
    pymembus = pymembus.pymembus
```

### Raw Shared Memory: `memmap`

`memmap` gives direct access to a named shared-memory block.

```python
import pymembus

name = "/my_map"
pymembus.memmap.remove(name)

writer = pymembus.memmap()
reader = pymembus.memmap()

assert writer.open(name, 1024, True, True)
assert writer.write("hello") == 5

assert reader.open(name, 0, False, False, True)  # read-only attach
assert reader.read(5) == "hello"

view = memoryview(writer)
assert view.shape == (1024,)
assert not view.readonly

readonly_view = memoryview(reader)
assert readonly_view.readonly

del readonly_view
del view
reader.close()
writer.close()
pymembus.memmap.remove(name)
```

Parameters for `memmap.open(name, size, create=False, new=False, read_only=False)`:

- `create`: create the object if it does not exist
- `new`: remove any existing object first
- `read_only`: attach without write permissions

### Broadcast Messages: `memmsg`

`memmsg` is a single-writer, multi-reader message queue. Every reader receives
every message independently.

```python
name = "/my_messages"
pymembus.memmsg.remove(name)

tx = pymembus.memmsg()
rx1 = pymembus.memmsg()
rx2 = pymembus.memmsg()

assert tx.open(name, 4096, True, True)
assert rx1.open(name, 4096, False, False)
assert rx2.open(name, 4096, False, False)

assert tx.write("frame-ready")
assert rx1.read_with_overrun(0) == ("frame-ready", False)
assert rx2.read_with_overrun(0) == ("frame-ready", False)

tx.close()
rx1.close()
rx2.close()
```

Use `poll()` for non-blocking readiness checks:

```python
if rx1.poll():
    msg = rx1.read(0)
```

When a reader falls behind far enough that the writer overwrites unread
messages, `read_with_overrun()` returns `("", True)`.

### Command Channels: `memcmd`

`memcmd` is a multi-writer command channel. It is useful for control paths, for
example sending commands from a UI process to a capture process.

```python
name = "/camera_commands"
pymembus.memcmd.remove(name)

receiver = pymembus.memcmd()
sender = pymembus.memcmd()

assert receiver.open(name, 1024, True, True)  # bReader=True, bCreate=True
assert sender.open(name, 1024)

assert sender.write("pan-left")
cmd, overrun = receiver.read_with_overrun(0)
assert cmd == "pan-left"
assert not overrun

sender.close()
receiver.close()
```

### Shared State: `memkv`

`memkv` is a fixed-schema key/value store. The owner creates the store and sets
slot names; any process can then read or write values.

```python
name = "/camera_state"
pymembus.memkv.remove(name)

owner = pymembus.memkv()
assert owner.create(name, 3, 16, 64, True)
assert owner.setName(0, "mode")
assert owner.setName(1, "count")
assert owner.setName(2, "status")

peer = pymembus.memkv()
assert peer.open(name)

assert peer.setValue("mode", "auto")
value, stale = peer.getValue("mode")
assert value == "auto"
assert not stale

epoch = peer.getEpoch()
assert owner.setValue("status", "ready")
changed, epoch = peer.getChanged(epoch)
assert changed == {"status": "ready"}

peer.close()
owner.close()
```

`getValue()` returns `(value, stale)`. `stale` is true if the lock-free read did
not settle before its retry limit.

### Video Ring Buffers: `memvid`

`memvid` stores packed video frames in a shared ring buffer. Use
`video_format` values instead of old numeric bits-per-pixel values.

```python
name = "/video"
pymembus.memvid.remove(name)

video = pymembus.memvid()
assert video.open(
    name,
    True,
    640,
    480,
    pymembus.video_format.rgb24,
    30,
    4,
)

slot = video.getPtr(0)
assert video.setVpts(slot, 123456)
assert video.setApts(slot, 123000)
assert video.next(1) == 1

assert video.getSeq() == 1
assert video.getFrameSeq(slot) == 1
assert video.getFormatName() == "RGB24"

frame = memoryview(video[slot])
assert frame.shape == (480, 640, 3)

del frame
video.close()
pymembus.memvid.remove(name)
```

Supported video formats:

- `gray8`
- `rgb24`
- `bgr24`
- `rgba32`
- `bgra32`
- `yuyv422`
- `uyvy422`

NumPy can view frame buffers without copying:

```python
import numpy as np

frame = np.array(video[slot], copy=False)
frame[10, 10] = [255, 0, 0]

del frame
video.close()
pymembus.memvid.remove(name)
```

### Audio Ring Buffers: `memaud`

`memaud` stores PCM audio buffers in a shared ring buffer. Use `audio_format`
values instead of old numeric bits-per-sample values.

```python
name = "/audio"
pymembus.memaud.remove(name)

audio = pymembus.memaud()
assert audio.open(
    name,
    True,
    2,
    pymembus.audio_format.s16le,
    48000,
    50,
    4,
)

slot = audio.getPtr(0)
assert audio.setPts(slot, 123456)
assert audio.next(1) == 1

assert audio.getChannels() == 2
assert audio.getSampleRate() == 48000
assert audio.getFormatName() == "S16LE"

buf = memoryview(audio[slot])
assert buf.shape == (960, 2)

del buf
audio.close()
pymembus.memaud.remove(name)
```

Supported audio formats:

- `u8`
- `s16le`
- `s24le`
- `s32le`
- `f32le`
- `f64le`

`s24le` is exposed as raw bytes because Python and NumPy do not have a native
24-bit integer scalar type.

### Buffer Lifetime And Read-Only Views

`memmap`, `memvid`, and `memaud` expose Python's buffer protocol. A writer that
created a share exports writable buffers. A reader opened read-only, or with
`open_existing()` for video/audio, exports read-only buffers.

Keep one lifetime rule in mind for video and audio buffers: release
`memoryview` or NumPy arrays before calling `close()` on the owning object.
`pymembus` intentionally raises `RuntimeError` if a video or audio mapping is
closed while exported frame buffers still exist, because those buffers would
otherwise point at unmapped shared memory.

### Waiting On Multiple Sources: `select()`

`select(wait_ms, conditions)` polls a list of Python callables and returns the
zero-based index of the first ready condition, or `-1` on timeout.

```python
idx = pymembus.select(100, [
    lambda: video.getSeq() > last_video_seq,
    lambda: commands.poll(),
])

if idx == 0:
    read_video_frame()
elif idx == 1:
    handle_command()
```

The conditions should be cheap, non-consuming readiness checks.

<a id="diagnostics"></a>

## Diagnostics

Most API calls return `False`, `-1`, or an empty string on failure. Check
`last_error()` or `last_error_message()` immediately after a failed call:

```python
missing = pymembus.memmap()

if not missing.open("/does-not-exist", 0, False):
    assert pymembus.last_error() == pymembus.errc.open_failed
    print(pymembus.last_error_message())
```

Common error codes include:

- `open_failed`
- `create_failed`
- `map_failed`
- `size_mismatch`
- `invalid_layout`
- `not_open`
- `access_denied`
- `message_too_large`
- `lock_timeout`
- `timeout`
- `overrun`

<a id="command-line-helpers"></a>

## Command Line Helpers

After installation, the package may install a `pymembus` helper command on
Linux:

```bash
pymembus help
pymembus files
pymembus info version
sudo pymembus uninstall
```

`pymembus info <variable>` accepts values such as `name`, `description`, `url`,
`version`, `build`, `company`, `author`, `lib`, `include`, `bin`, and `share`.

<a id="troubleshooting"></a>

## Troubleshooting

### `ModuleNotFoundError: No module named 'pymembus'`

Build the extension first:

```bash
cmake -S . -B ./bld -DCMAKE_BUILD_TYPE=Release
cmake --build ./bld -j
```

Then run tests from the repository root:

```bash
python3 -m pytest -v
```

The repository's pytest config adds `bld/lib` to `PYTHONPATH`.

### Pytest tries to run pybind11 tests

This happens when pytest recursively scans build artifacts. The repository
config sets:

```toml
norecursedirs = ["bld", "_skbuild", "dist", "build", ".git", "*.egg-info"]
```

If you use a custom pytest command, prefer:

```bash
python3 -m pytest -v src/pytest/py
```

### A share already exists or has an invalid layout

Remove stale shared-memory objects before creating a fresh one:

```python
pymembus.memmsg.remove("/my_messages")
pymembus.memvid.remove("/video")
pymembus.memaud.remove("/audio")
```

The 1.2.0 `libmembus` wire formats validate shared-memory headers. Old shares
created by earlier library versions may be rejected with `invalid_layout`.

### Doxygen or Graphviz warnings during build

The CMake build may emit documentation warnings from Doxygen or Graphviz. They
do not affect the Python extension or pytest suite.

<a id="development-notes"></a>

## Development Notes

- Project metadata lives in `PROJECT.txt`.
- The Python extension source is in `src/py/cpp/main.cpp`.
- Tests live in `src/pytest/py/test.py`.
- The fallback `libmembus` dependency is configured in `src/libmembus.cmake`.
- See `UPDATE.md` for the libmembus 1.2.0 migration report.

## References

- [Python](https://www.python.org/)
- [CMake](https://cmake.org/)
- [pip](https://pip.pypa.io/en/stable/)
- [git](https://git-scm.com/)
- [Boost](https://www.boost.org/)
- [pybind11](https://github.com/pybind/pybind11)
- [libmembus](https://github.com/wheresjames/libmembus)
