Metadata-Version: 2.4
Name: tau-av
Version: 0.5.0
Summary: Distributed multi-projector rendering: state replication over UDP, OSC parameter sync, per-projection warp/blend, and quad-buffer stereo 3D. For authoring multimedia work in the AlloSphere with no dependency on allolib.
Author-email: Ryan Millett <rmillett@ucsb.edu>
License: MIT
Project-URL: Homepage, https://github.com/kr4g/Tau
Project-URL: Repository, https://github.com/kr4g/Tau
Keywords: allosphere,audiovisual,distributed-rendering,immersive,opengl,osc,simulation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Graphics :: 3D Rendering
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26
Requires-Dist: numba>=0.59
Requires-Dist: moderngl>=5.10
Requires-Dist: glfw>=2.7
Requires-Dist: python-osc>=1.8
Requires-Dist: imgui-bundle>=1.5
Requires-Dist: PyOpenGL>=3.1
Requires-Dist: tomli>=2.0; python_version < "3.11"
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Provides-Extra: publish
Requires-Dist: build>=1.0; extra == "publish"
Requires-Dist: twine>=5.0; extra == "publish"
Dynamic: license-file

# Tau

Tau is a Python runtime for distributed multi-projector rendering in the
AlloSphere, with no dependency on allolib. State replicates over UDP,
parameters sync over OSC, and each projection gets its own warp and blend,
drawn for both eyes.

Content can be an app class, functions registered on a runtime, a loop you
own, or a plain object that the renderers draw. All four run in a window on
one machine. The same file runs on every machine in the AlloSphere, where
one simulates and the renderers draw.

In quantum physics, τ (tau) denotes the *tangle*, a measure of
entanglement. For three or more parts it is what is left once every
pairwise correlation has been accounted for: a property of the whole
system, belonging to no pair within it.

## Install

Use Tau from a clone. The examples, the tests, and the deploy scripts live
in the repository tree.

```bash
git clone https://github.com/kr4g/Tau.git
cd Tau
python3 -m venv .venv
.venv/bin/python -m pip install -e .
```

In a tree of your own, install it as a dependency instead:

```bash
pip install tau-av
```

The package imports as `tau`.

`python -m tau.preflight` checks the machine it runs on: Python version, GL
context, shader compilation, numba JIT, calibration, ports, UDP send and
receive. Run it after installing, and on any machine before it joins a
cluster.

## Run

```bash
.venv/bin/python -m tau.launcher
```

The launcher lists the apps under `examples/` and `apps/` and runs the
selection as a subprocess. When node agents run (see "Running in the
AlloSphere"), a launch also switches what the cluster shows.

A single app runs directly:

```bash
.venv/bin/python -m examples.boids.app --view equirect
```

The views are `pov` (perspective, the default), `cross` (a cubemap box),
`equirect` (a panorama), and `anaglyph` (a red/cyan stereo preview). The
`v` key cycles them while the app runs. The arrow keys look around and WASD
moves the camera. `--help` lists the other controls and flags.

A second instance on the same machine elects as a replica and follows the
first. Every start prints a role banner: host, role, broadcast target,
renderer, calibration.

## Writing content

Apps you write go in `apps/`. Code they import goes in `ext/`. Both ship
empty and git ignores their contents, so a pull never touches them. An app
in `apps/<name>/` runs as `python -m apps.<name>.app`, and the launcher
discovers it there. The launcher lists an app when its `app.py` defines a
top-level `main()`. An `app.py` without one still runs under `python -m`,
but the launcher does not see it, and a cluster launch cannot start it.

### An app class

Subclass `DistributedApp` and override its hooks. A numpy dtype declared as
`state_type` becomes a replicated block: the primary writes it, and every
renderer reads it.

```python
class Cloud(tau.DistributedApp):
    state_type = np.dtype([("pos", np.float32, (256, 3))])
    sync_nav = True                          # replicas follow the primary camera

    def on_animate(self, dt):
        if self.is_primary():
            self.state()["pos"] += drift(dt)  # simulate on one machine

    def on_draw(self, g):
        ...                                   # draw on every machine
```

The other hooks are `on_init`, `on_create` (GL is live), `on_gui(panel)`,
and `on_keys(keys)`. A registered `Parameter` syncs over OSC and appears in
the control panel. Most examples follow one shape: a model module in
vectorized numpy, a `state.py` with the dtype, and an `app.py` whose
`main()` calls `tau.run`.

`examples.collatz` computes a large static layout once and loads it from
the path in the `TAU_CACHE_DIR` environment variable, so every node reads
one copy from a shared mount. This is a convention of the example, not of
the runtime.

### Registered functions

`tau.runtime()` builds a runtime whose hooks are registered functions:

```python
rt = tau.runtime(state_type=my_dtype)

@rt.animate
def animate(dt):
    ...

@rt.draw
def draw(g):
    ...

rt.run()
```

A second registration replaces the first, even while the loop runs. The
loop and the network connections do not stop when the content changes.
With no draw hook the output is black, so a session can start empty and
get content later. The swap is local to one process. Machines that did
not run the registration keep the hooks they have. Changing what the
cluster runs is a relaunch: the launcher stops the current module on
every node and starts the new one.

### Your own loop

`rt.run()` is a plain loop. A program with its own loop makes the same
calls itself:

```python
rt.open()             # election, domains, window
while running:
    rt.step(dt)       # simulate and replicate
    rt.poll(dt)       # domain upkeep
    rt.render()       # draw one frame
rt.shutdown()
```

This suits a notebook or a larger program that uses Tau as a library.

### The Scene protocol

A renderer needs three things from the object it draws: a camera pose, a
lens, and a draw callback. `tau.Scene` names this protocol, and any object
that fills it in will do:

```python
class Wrapped:
    def __init__(self, mesh):        # geometry from any source
        self._nav = tau.Nav()
        self._lens = tau.Lens()
        self._mesh = mesh

    def nav(self):
        return self._nav

    def lens(self):
        return self._lens

    def on_draw(self, g):
        g.clear(0.0)
        g.draw(self._mesh)
```

The class above subclasses nothing and registers nothing. The runtime
takes it whole and renders it through the full capture, warp/blend, and
stereo pipeline:

```python
tau.runtime(scene=Wrapped(mesh)).run()
```

The runtime also calls optional hooks the object defines (`on_animate`,
`on_init`, `on_keys`). Each machine calls `on_draw` on its own schedule,
so content that keeps its own clock drifts across the cluster. Drive
motion from replicated state or from the `dt` passed to `on_animate`.

This is the integration surface for content from other tools: wrap the
content in the three methods, or give its per-frame arrays to the
retained primitives (instanced meshes, points, lines, ribbons). State
replication and parameters work alongside both.

### The upload rule

`on_animate(dt)` runs once per frame. `on_draw(g)` runs once per projector
per eye, so it must only issue draw calls. Build and upload geometry in
`on_animate`. Content that uploads in `on_draw` looks correct in a window
and renders differently on each projector. The checker below catches this.

### Working with a coding agent

[AGENTS.md](AGENTS.md) carries the conventions and the reasons for them,
written for an agent as much as for a person. An agent that works in a
clone reads it from the repository root without being asked. The checker's
`--json` output is for an authoring loop.

## Stereo 3D

A calibrated renderer captures both eyes and presents them through a
quad-buffer framebuffer. If the driver has no stereo framebuffer, the
renderer warns and runs mono. `--mono` disables stereo.

The lens sets the depth. `lens().focal_length(v)` places the convergence
distance. Content at that distance sits at the screen surface. Nearer
content floats inside the sphere, and farther content recedes.
`lens().eye_sep(v)` scales the disparity. Each example places its
convergence where its content lives.

The `/tau/stereo` parameter is a checkbox in the control panel. It switches
the cluster between stereo and mono while the app runs, and mono also
halves the capture cost. A vertex shader gets the displacement when it
calls `stereo_displace(...)`. The runtime inserts the correct variant for
the render path at compile time, and `g.apply_stereo(prog)` sets the
uniforms.

At home, the `anaglyph` view shows the same disparity through red/cyan
glasses. `examples.calibration` draws a depth ladder that reads flat when
stereo is dead, and inverted when the eyes are crossed.

## Control panel

On the simulator, every app gets a second window: a view selector, the
stereo toggle, the pattern selector, the cluster roster, and a widget
for each registered parameter. Overriding `on_gui(panel)` replaces the
parameter widgets with custom UI. The other rows stay. Render nodes
never open one, and `--no-gui` disables it.

## Shaders

The renderer supplies `tau_ModelViewMatrix`, `tau_ProjectionMatrix`, and
`tau_ViewMatrix`. A shader declares whichever it uses. Shaders loaded
through `ShaderManager` reload on file change while the app runs.

## Gaussian splats

`tau.splat` packs a trained Gaussian-splat PLY into a cache under
`TAU_CACHE_DIR`, which the renderers load. `SplatCloud` draws it as one
instanced quad per splat, sorted back to front off the frame loop.

```bash
.venv/bin/python -m tau.splat.prep scene.ply --scene <name>
```

`examples.splat_env` flies through a prepped scene. `scripts/splats.json`
lists free sources it reads, including the World Labs Marble exports and
the official 3DGS pretrained models, and `scripts/fetch_splats.py`
downloads them.

## Checking an app

```bash
.venv/bin/python -m tau.check apps.myapp    # or examples.boids, or a bare name
```

It runs the checks that work at home: shader compilation at the
`#version 410` ceiling, draw purity, uploads misplaced in `on_draw`,
stereo, state size and wire rate, and headless determinism. The checker
phrases each failure as the change that fixes it. `--json` emits the
report for tooling. All it asks of the app is a `smoke()` function in
`app.py`:

```python
def smoke():
    return MyApp(n=64, seed=1, headless=True, fps=0.0)
```

`smoke()` can return a `DistributedApp`, a runtime built by
`tau.runtime` (with hooks or a scene), or a bare `Scene` object. The
checks that need replicated state skip when there is none.

The test suite (`python -m pytest`) checks the core the same way. It
includes purity for every example, a pixel-for-pixel warp oracle, a
two-process rehearsal of election and transport, and the stereo gates.
`bash scripts/test-py310.sh` repeats the suite on the renderers'
interpreter. `python -m tests.bench_sphere` reports timing costs, and
`--save` / `--compare` bracket a change. Subnet broadcast, driver
differences, the spanned X screen, and quad-buffer presentation are
checked on site.

No check covers whether content reads from *inside* the AlloSphere.
Content can pass everything and still be composed for a rectangle. Look at
it in `--view equirect` and `--view pov`.

## Running in the AlloSphere

Stage the work on the shared `/alloshare` mount. Build the venv there
once, from a renderer: `bash deploy/build_venv.sh`. Then start the same
app on every machine by hand:

```bash
.venv/bin/python -m examples.<name>.app     # identical on every machine
```

or start the node agents once and switch content from the simulator:

```bash
bash deploy/launch_sphere.sh start
.venv/bin/python -m tau.launcher
```

Role and renderer resolve from the hostname and the calibration data. The
primary host (`ar01`, or whatever `TAU_PRIMARY_HOST` names) simulates and
sends state. A renderer with a calibration manifest applies warp, blend,
and stereo, and runs fullscreen. `--sim` makes any machine the primary.

State ships as full snapshots over UDP, latest-wins. A change to the
state dtype needs a relaunch on every node: a node running a different
dtype drops the mismatched snapshots and shows `NO STATE` on its output.

Every node sends a heartbeat once a second. The control panel shows the
roster, and `python -m tau.heartbeat` prints the same table. Two
simultaneous primaries appear on both.

The `/tau/pattern` parameter (control panel, or keys `0`–`5` in
`examples.calibration`) switches every renderer into a
projector-identification pattern without stopping the content. `0` returns
to normal. Degraded states, such as missing calibration or state that
stops arriving, appear on the output itself.
