Metadata-Version: 2.5
Name: tensorcamera
Version: 0.2.0
Summary: Point your phone at something and see your own model's prediction, live. Server SDK for the Tensor Camera app.
Project-URL: Homepage, https://github.com/SpunkySarb/tensorcamera
Project-URL: Protocol, https://github.com/SpunkySarb/tensorcamera/blob/main/PROTOCOL.md
Project-URL: Issues, https://github.com/SpunkySarb/tensorcamera/issues
Project-URL: App Store, https://apps.apple.com/ca/app/tensor-camera/id6476598066
Author-email: Sarbjeet Singh <sarbzone@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Sarbjeet Singh
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: camera,computer-vision,inference,machine-learning,socketio
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: numpy>=1.21
Requires-Dist: pillow>=9
Requires-Dist: python-socketio<6,>=5.7
Requires-Dist: qrcode>=7.3
Requires-Dist: simple-websocket>=0.10
Requires-Dist: werkzeug>=2.2
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: python-socketio[client]<6,>=5.7; extra == 'dev'
Provides-Extra: hf
Requires-Dist: torch>=2.0; extra == 'hf'
Requires-Dist: transformers>=4.38; extra == 'hf'
Description-Content-Type: text/markdown

# tensorcamera

Point your phone at something and see your own model's prediction, live.

This is the server half. The other half is the **Tensor Camera** app
([iOS](https://apps.apple.com/ca/app/tensor-camera/id6476598066)), which streams
camera frames to whatever machine your model is on and shows what comes back.

```bash
pip install tensorcamera
```

## Two minutes, no code

```bash
pip install 'tensorcamera[hf]'
tensorcamera serve --model objects
```

That downloads a model from Hugging Face, starts a server, and prints a QR code:

```
  Tensor Camera server on ws://192.168.2.118:8080
  model: google/vit-base-patch16-224

  Scan this with the Tensor Camera app:

   ▄▄▄▄▄▄▄ ▄ ▄▄▄ ▄▄▄▄ ▄▄ ▄▄▄  ▄  ▄▄▄▄▄▄▄
   █ ▄▄▄ █  ▄ ▄ ▀▄█▀█▀▀ █▀▀█  ▄█ █ ▄▄▄ █
   █ ███ █ █▄▀ ▀██▄▀██ ▄ ▄█▀██▄█ █ ███ █
   █▄▄▄▄▄█ █▀█▀█ ▄▀▄▀▄▀█ ▄ ▄▀▄ ▄ █▄▄▄▄▄█
                    ( … )

  Won't scan? Open the image instead:  /tmp/tensorcamera-qr.png
  ...or enter the address by hand:      ws://192.168.2.118:8080

  Phone and computer must be on the same Wi-Fi network.
```

Scan it and you are connected. The code carries the address and, if you set one,
the auth token — so there is no IP address to type on a phone keyboard.

A PNG is always written alongside the terminal version, because block-character
QR codes depend on the font's line spacing and do not survive every terminal or
screenshot. If the printed one will not scan, open the file.

`tensorcamera presets` lists the other built-in models, and any Hugging Face Hub
id works too.

### Ports

The default is **8080**, which is a crowded neighbourhood. If it is taken, the
server steps to the next free port and says what was in the way:

```
  port 8080 is busy (held by node (pid 66527)) — trying 8081
  using port 8081 instead of 8080
```

The port is bound *before* the QR code prints, so the code always encodes a port
that is genuinely being served — it can never advertise a server that failed to
start. Pass `--strict-port` (or `auto_port=False`) to fail instead of stepping,
which is what you want under a process manager.

## Your own model

```python
from tensorcamera import TensorCamera

cam = TensorCamera(model="my-classifier", labels=["dog", "cat"])

@cam.on_frame
def predict(frame):
    probs = my_model(frame.resized(224)[None] / 255.0)[0]
    index = probs.argmax()
    return {"label": cam.labels[index], "confidence": float(probs[index])}

cam.serve()
```

`serve()` prints a QR code and listens on port 8080. Phone and computer need to
be on the same Wi-Fi network.

## The handler

Return whatever is convenient:

| Return | Meaning |
| --- | --- |
| `None` | Skip this frame, send nothing |
| `"Found a dog"` | Display text only |
| `("dog", 0.94)` | Label and confidence |
| `{"label": ..., "confidence": ..., "text": ..., "top": [...]}` | Full control |

**Return a `label` and a `confidence` if you can.** Display text is for humans;
`label` is what lets the app fire rules — play a sound, speak, vibrate, POST a
webhook — when something is detected. A bare string leaves the app display-only.

## The frame

```python
@cam.on_frame
def predict(frame):
    frame.array        # numpy uint8, HWC, RGB — what Keras and torch expect
    frame.float01      # numpy float32, HWC, RGB, 0..1
    frame.resized(224) # numpy uint8, 224x224x3
    frame.jpeg         # raw JPEG bytes, undecoded
    frame.width, frame.height, frame.seq, frame.ts
```

Decoding is lazy. If your handler never touches the pixels, no decode happens.

## Talking back to the phone

```python
cam.send_action({"type": "sound", "asset": "bark.mp3"})
cam.send_action({"type": "speak", "text": "dog detected"})
cam.send_action({"type": "haptic", "style": "success"})
```

The app ignores anything it does not support, and refuses action types it did
not advertise. It also rate-limits inbound actions, so a 10 fps stream cannot
fire a sound ten times a second — but prefer device-side rules for that, since
they keep working when the connection drops.

## Keeping up

If inference is slower than the frame rate, predictions drift behind reality.
Tell the phone how fast you actually are:

```python
cam = TensorCamera(max_fps=3)
```

The app throttles to that budget. The server logs a warning the first time a
handler overruns it.

## Options

```python
TensorCamera(
    model="my-classifier",   # shown in the app's diagnostics
    labels=["dog", "cat"],   # populates the rule editor's label picker (≤40)
    max_fps=10,              # frame budget asked of the phone
    quality=0.4,             # requested JPEG quality
    width=640, height=480,   # requested frame size
    token="shared-secret",   # require this from clients; carried in the QR code
)

cam.serve(
    host="0.0.0.0",
    port=8080,           # steps to the next free port if taken
    show_qr=True,
    name="Workshop MacBook",
    auto_port=True,      # False to fail on a busy port instead
    qr_png="qr.png",     # where to write the scannable image
)
```

## Older app versions

The app on the App Store today predates this protocol. It sends raw float32
tensors on an untyped channel with no handshake, and expects a plain string
back. This server detects that and handles it — including the newer app's
legacy mode, which sends JPEG on the same channel. Both work, no flags.

You lose structured predictions and rules against an old client, because the v0
wire format has nowhere to put a label. Everything else works.

## Protocol

The wire format is specified in [PROTOCOL.md](./PROTOCOL.md). You should not
need it — that is the point of this package — but it is there if you want to
write a server in another language. The app talks to anything that speaks it;
this package is one implementation, not the contract.

## Licence

MIT.
