Metadata-Version: 2.1
Name: sciveo
Version: 0.2.64
Description-Content-Type: text/markdown
Provides-Extra: mon
Provides-Extra: agent
Provides-Extra: agent-local
Provides-Extra: net
Provides-Extra: server
Provides-Extra: storage
Provides-Extra: storage-postgres
Provides-Extra: chat
Provides-Extra: media
Provides-Extra: media-ml
Provides-Extra: sciveyolo
Provides-Extra: sciveyolo-gpu
Provides-Extra: sciveyolo-trt
Provides-Extra: web
Provides-Extra: db
Provides-Extra: power
Provides-Extra: energy
Provides-Extra: all
Provides-Extra: ml

# SCIVEO

Sciveo is a Python toolkit and CLI for ML/AI workflows, engineering automation,
experiment tracking, machine monitoring, local operations, network/industrial
IO, media processing, AI agents, local storage, and production ML inference.

It is meant for practical systems where models, data, devices, services, and
operators meet: train or evaluate ML models, collect measurements, process
media, monitor machines, communicate with edge or plant equipment, run agents,
and keep the operational tooling close to the Python code that uses it.

The package is intentionally broad. The CLI is the easiest entry point for
operations and lab work, while the Python modules expose the same building
blocks for notebooks, services, experiments, and pipelines.

## ML/AI and Engineering Focus

Sciveo is built around the common shape of applied ML, AI, and engineering
work:

- define and run parameterized experiments;
- sample parameter spaces and record configurations;
- attach datasets, metrics, scores, plots, media, and generated artifacts;
- monitor machines that produce data or run services;
- communicate with lab, plant, edge, and networked devices;
- capture and process image/video/audio data;
- run local or remote ML inference close to the data;
- coordinate AI agents and local tools;
- synchronize results to an API-backed project when remote tracking is needed.

The operations modules are part of the same picture. Monitoring, network IO,
local storage, media workers, and admin tooling are there because real ML and
engineering deployments often include cameras, sensors, edge boxes, industrial
controllers, GPU machines, local disks, APIs, and long-running background jobs.

## Main Capabilities

- **Monitoring and watchdogs**: host metrics, GPU/server telemetry where
  available, plant/Modbus monitoring, local watchdog actions, and service
  installation helpers.
- **Experiments and project helpers**: project runs, parameter sampling,
  datasets, scores, plots, metadata, and optional remote synchronization.
- **Network and industrial IO**: network discovery, SSH inventory/execution,
  Modbus TCP/RTU reads and writes, SNMP, MQTT, HTTP helpers, and protocol
  emulators.
- **Admin and diagnostics**: first-boot edge administration, Wi-Fi access point
  provisioning, service checks, doctor reports, and fleet inventory.
- **Media capture and processing**: RTSP/NVR capture, screen/camera capture,
  local media processing, ML media processors, and queue/API-backed media
  workers.
- **AI agent console**: interactive and one-shot agent workflows with hosted
  providers, local tools, image inputs, and optional local Hugging Face/GGUF
  runtimes.
- **Encrypted chat rooms**: shared-token chat rooms for lightweight operator and
  agent coordination.
- **Local S3-compatible storage**: a boto3-compatible object store backed by
  mounted local paths.
- **Certification readiness tests**: local, framework-neutral self-assessment,
  evidence-window scoring, and remediation suggestions, starting with an
  unofficial SOC 2-style security profile.
- **SCIVEYOLO inference**: Sciveo-owned SCIVEYOLO inference artifacts and
  runtime for production object detection without requiring Ultralytics at
  runtime.
- **Supporting API, DB, web, content, and tools modules**: shared building
  blocks used by experiments, services, and pipeline jobs.

## Typical Workflows

Sciveo is useful when a project needs more than one isolated script. The common
pattern is a Python workflow that touches models, measurements, local services,
devices, and result tracking.

### ML/AI Workflow

Use Sciveo to keep model work reproducible and close to the runtime environment:

```python
import sciveo

def train_one_run():
    with sciveo.open() as experiment:
        learning_rate = experiment.config.learning_rate
        batch_size = experiment.config.batch_size

        metrics = {
            "loss": 0.18,
            "accuracy": 0.94,
        }

        experiment.log({"learning_rate": learning_rate, "batch_size": batch_size})
        experiment.eval("loss", metrics["loss"])
        experiment.eval("accuracy", metrics["accuracy"])
        experiment.score(metrics["accuracy"])

sciveo.start(
    project="vision-model-eval",
    function=train_one_run,
    configuration={
        "learning_rate": {"values": [0.001, 0.0005]},
        "batch_size": {"values": [8, 16]},
    },
    remote=False,
    count=4,
)
```

### Engineering and Edge Workflow

Use the CLI to inspect and operate machines that produce data or run inference:

```shell
sciveo doctor --render json
sciveo scan --net 192.168.10.0/24 --health
sciveo fleet --net 192.168.10.0/24 --users operator,admin -i ~/.ssh/id_ed25519 --render json
sciveo monitor --period 60
sciveo watchdog --src memory --threshold 90 --period 10 --execute "systemctl restart app"
```

### Media and Inference Workflow

Use media commands for local processing, and `.sciveyolo` artifacts for
production object detection without requiring Ultralytics at runtime:

```shell
sciveo rtsp --url rtsp://camera/stream --output-path ./clip.mp4
sciveo media-run --input-path ./clip.mp4 --output-path ./media-out --render text
sciveo media-run --mode ml --processor image-object-detection --input-path ./frame.jpg --output-path ./detections
```

```python
from sciveo.ml.images.sciveyolo import SCIVEYOLO

model = SCIVEYOLO("detector.sciveyolo")
results = model.predict("frame.jpg")
```

### Device and Protocol Workflow

Use network and industrial IO helpers directly from the CLI during bring-up,
testing, and diagnostics:

```shell
sciveo read --proto modbus --transport tcp --host 192.168.0.10 --port 502 --id 1 --address 30001 --kind input --type RAW --count 2
sciveo write --proto mqtt --host broker.local --topic plant/cmd --payload '{"limit":80}' --qos 1 --retain
sciveo emulate --server http --host 0.0.0.0 --port 8080 --data-json '{"status":{"ok":true}}'
```

## Installation

Base install:

```shell
pip install sciveo
```

Useful extras:

```shell
pip install "sciveo[mon]"          # host monitoring helpers
pip install "sciveo[net]"          # network, Modbus, SNMP, MQTT helpers
pip install "sciveo[agent]"        # hosted AI agent providers
pip install "sciveo[agent-local]"  # local HF/GGUF runtime support
pip install "sciveo[media]"        # media capture/processing
pip install "sciveo[media-ml]"     # ML model/runtime dependencies
pip install "sciveo[server]"       # API/server helpers
pip install "sciveo[web]"          # Django/web helpers
pip install "sciveo[all]"          # core operations extras
```

`media-ml` intentionally does not install Ultralytics. For image/video ML
workflows, install both media extras so OpenCV and model runtimes are present:

```shell
pip install "sciveo[media,media-ml]"
```

Sciveo SCIVEYOLO runtime loads `.sciveyolo` artifacts with PyTorch or ONNX
Runtime depending on the artifact engine.

## CLI Overview

Show CLI help and version:

```shell
sciveo --help
sciveo --version
sciveo help --json
sciveo help monitor
sciveo help monitor --json
```

Current top-level CLI commands:

```text
init
monitor
watchdog
scan
ssh
read
write
emulate
admin
doctor
fleet
extensions
certification
nvr
rtsp
capture
media-server
media-run
agent
chat
storage
predictors-server
```

The `sciveo help --json` manifest is the machine-readable source for command
summaries, usage forms, options, notes, and examples.

## Certification Readiness Tests

See [`CERTIFICATION.md`](CERTIFICATION.md) for the platform architecture,
verification strategy, implemented scope, and phased roadmap.

Generate an editable SOC 2-style test input, complete its answers and evidence
metadata, then evaluate it locally:

```shell
sciveo certification --action template --framework soc2 --output-path soc2-test.json
sciveo certification --action example --scenario software-company --output-path software-company.json
sciveo certification --action evaluate --framework soc2 --input-path soc2-test.json
sciveo certification --action evaluate --framework soc2 --input-path soc2-test.json --render json --output-path soc2-report.json
```

The initial profile is a Sciveo-authored general security checklist. It scores
the supplied status and evidence metadata, highlights critical gaps, checks a
configurable observation target (90 days by default), and returns concrete
suggestions. Reports also contain structured control-test results, a ranked
remediation plan, evidence-reference classifications, domain scores, and
per-system gap summaries. The summary is an explanatory roll-up; per-check
system reference coverage can cap scores. A reference does not mean the control
is supported, and Sciveo does not open or verify referenced evidence.

This feature is an unofficial readiness self-assessment. It is not a SOC 2
examination, audit, certification, CPA opinion, or guarantee of an audit result;
it does not reproduce the AICPA Trust Services Criteria. An independent licensed
CPA firm determines the actual examination scope, observation period, testing,
exceptions, and report. Use `sciveo certification --help` for the CLI contract.

The generated template is the versioned machine-readable input contract.
Answers use `implemented`, `partial`, `not_implemented`, `not_assessed`, or
`not_applicable`; the last status requires a text rationale, and every proposed
exclusion remains scored until an independent review process exists. Evidence
is a list of non-empty text references or objects containing `name`,
`reference`, `path`, `url`, or `id`, plus optional `source`, `system_ids`, and an
ISO `observed_at` date. `source` alone is not a concrete reference. Dated
evidence outside the configured period is classified and ignored for that
period. Answers may also set `applicable_system_ids`, `gap_system_ids`,
`applicability_rationale`, `remediation_action`, `owner`, and `target_date`.
Omitting a declared system requires a rationale and remains an explicit scope
review item; critical scope exclusions prevent the `strong` readiness band.
Targets must be integer values from 1–366 days.
An explicitly supplied `--observation-days` overrides the input target;
otherwise the input value wins, with 90 days as the fallback.

### Complete Hybrid Software Company Example

Generate the bundled fictional assessment and evaluate it without contacting
any external service:

```shell
sciveo certification --action example \
  --scenario software-company \
  --output-path software-company.json

sciveo certification --action evaluate \
  --input-path software-company.json

sciveo certification --action evaluate \
  --input-path software-company.json \
  --render json \
  --output-path software-company-report.json
```

The example fixes a 90-day window from 2026-01-01 through 2026-03-31 so its
inputs and score remain reproducible. It models the following system boundary:

| System | Synthetic population/sample | Readiness evidence and deliberate gaps |
| --- | --- | --- |
| Git | 18 repositories / 4 sampled | Review rules, approvals, CI results, access reviews |
| AWS EC2 | 24 instances / 6 sampled | Infrastructure changes, vulnerability scans, logging; no account IDs or live calls |
| AWS S3 | 8 logical locations / 4 sampled | Storage configuration and restoration references; no bucket names or credentials |
| Self-hosted servers | 12 servers / 4 sampled | Patch/change/access records with deliberate local-account gaps |
| Edge IoT | 240 gateways / 12 sampled | Inventory and monitoring references with firmware/access lifecycle gaps |
| Edge NVR | 120 machines / 10 sampled | Backup records with deliberate logging and restoration-test gaps |

The synthetic answers produce a stable 60.0% `developing` readiness result,
seven critical gaps, and ten ranked remediation tasks:
Git and AWS practices are comparatively mature, while shared local accounts,
edge firmware governance, centralized edge/NVR logging, NVR restoration tests,
and device-vendor lifecycle reviews create remediation work. This is an example
of how the scoring behaves, not a conclusion about any real company.

The example builder is frozen local data. Shared Sciveo imports may load global
configuration, but certification does not authenticate to services or use,
transmit, or copy credential values. It never invokes Git, connects to AWS,
opens SSH sessions, contacts self-hosted servers, or talks to edge devices,
cameras, or NVRs. Replace the
synthetic inventory, answers, evidence IDs, owners, dates, and gaps with your
own metadata before using the evaluator for internal readiness planning.

## Configuration

Initialize local configuration:

```shell
sciveo init
```

The default local configuration lives under `~/.sciveo/`. API credentials can
also be supplied through environment variables where supported, for example:

```shell
export SCIVEO_SECRET_ACCESS_KEY="..."
```

## Monitoring

Start host monitoring:

```shell
sciveo monitor --period 60
```

Monitor the root filesystem plus additional mounted paths or block devices.
`--paths` accepts mount directories and always includes `/`. `--disks` accepts
filesystem-bearing block devices or partitions and resolves each one to its
current mountpoint:

```shell
sciveo monitor --period 60 \
  --paths '["/mnt/data-1"]' \
  --disks '["/dev/sdb1","/dev/sdc1"]'
```

Root capacity remains under `DISK`; requested mount paths are reported under
`PATHS`, and requested block devices are reported under `DISKS` with their
resolved device, mountpoint, filesystem, capacity, and mount state. Sciveo does
not implicitly expand a whole disk such as `/dev/sdb` to `/dev/sdb1`; name the
filesystem-bearing partition explicitly. Stable `/dev/disk/by-uuid/...` aliases
are preferred when device names may change across boots.

Install monitoring as a service:

```shell
sciveo monitor --period 60 \
  --paths '["/mnt/data-1"]' \
  --disks '["/dev/sdb1"]' \
  --install
```

Run `--install` as the account that should run the monitor. Sciveo asks for
`sudo` only while copying the systemd units and running `systemctl`; the
installed service keeps the invoking account as its runtime user.

Every `--install` rewrites both `sciveo-monitor.service` and
`sciveo-monitor.timer`, reloads systemd, and restarts the service so updated
monitor arguments take effect immediately. The timer starts the continuous
service once after each boot; the service itself continues sampling at
`--period` intervals.

Inspect the installed units and follow monitor logs:

```shell
systemctl status sciveo-monitor.timer sciveo-monitor.service
systemctl list-timers --all sciveo-monitor.timer
journalctl -u sciveo-monitor.service -f
```

Write samples to a local path:

```shell
sciveo monitor --period 120 --output-path ./monitor.json
```

Start non-blocking monitoring from Python:

```python
import sciveo

sciveo.monitor(period=120, block=False)
```

Plant/industrial monitoring uses the `plant` source:

```shell
sciveo monitor --src plant --host 192.168.1.50 --port 502 --period 60 --serial plant-1
```

## Watchdogs

Watchdogs run local checks and either execute a configured command or use the
built-in internet repair path when a condition remains unhealthy.

```shell
sciveo watchdog --src memory --threshold 90 --period 10 --execute "systemctl restart app"
sciveo watchdog --src disk --input-path /data --threshold 80 --period 600 --execute "find /data/tmp -type f -mtime +1 -delete"
sciveo watchdog --src network --targets '["1.1.1.1:443","8.8.8.8:53"]' --threshold 3 --period 30 --execute "echo network outage"
```

The network/internet watchdog performs concurrent deadline-bounded TCP checks,
records resolution/bind/connect results, and applies `at-least-one`, `all`,
`majority`, or `count:N` quorum. With
`--execute`, it runs the configured shell command after `--threshold`
consecutive failures and repeats the command on every failed period until a
healthy check resets the counter. `scan --health` adds read-only gateway/DNS
diagnostics, while monitor HTTP probes record endpoint health without repairs.

Use built-in internet recovery without running the whole watchdog as root:

```shell
sciveo watchdog --net --repair --period 30 --threshold 3
```

On Linux systemd machines, install the same configuration as a supervised
boot service. Run this command from the non-root account that should run the
watchdog; Sciveo asks for `sudo` only while installing root-owned files and
using `systemctl`:

```shell
sciveo watchdog --net --repair --period 30 --threshold 3 --install
sciveo watchdog --net --repair --preflight
systemctl status sciveo-network-watchdog.service
journalctl -u sciveo-network-watchdog.service -f
```

`--install` and `--reinstall` preserve the network repair targets, quorum,
timing, platform, interface, and log-level options in the unit's `ExecStart`.
The service runs as the invoking non-root user with `Restart=always`; it does
not run the watchdog process as root. Every service start first runs a
read-only preflight that validates the unattended helper, exact service-unit
allowlist, detected platform, selected interface, network manager, and that
manager's required core helper actions. Remove all managed watchdog files and
persistent incident state with:

```shell
sciveo watchdog --uninstall
```

Installation places a self-contained root-owned helper at
`/usr/local/libexec/sciveo-network-repair` and grants the service user
passwordless sudo access only to that exact executable. The helper accepts a
fixed action enum, validates physical interface names and canonical
NetworkManager UUIDs, uses fixed absolute system-program paths, and rejects
arbitrary commands. Privileged actions have a fixed 30-second execution
deadline. The optional service allowlist is accepted only as a regular,
root-owned file that is neither group- nor world-writable, and its exact units
are checked by preflight. The helper's DNS configuration repair refuses to
overwrite a regular `/etc/resolv.conf` or a working symlink. Optional VPN/API
service restarts are limited to canonical `.service` units in that allowlist.

Bare `--repair` also implies the network source, but spelling the mode as
`--net --repair` is preferred because it makes the watchdog purpose explicit.

Repair mode defaults to `1.1.1.1:443`, `softel.bg:443`, and `sciveo.com:443`
with majority policy. Additional VPN or API endpoints supplied through
`--targets` are added to those defaults and deduplicated by host and port. Use
`--policy all` when every configured endpoint must participate in the health
decision.

Before changing an interface, repair mode snapshots the owning manager,
link/carrier, usable addresses, default route, gateway-neighbor state, and DNS,
then separately probes numeric underlay anchors (`1.0.0.1:443`, `8.8.8.8:53`,
and `[2606:4700:4700::1111]:443`) through the selected physical
interface/source. The at-least-one default keeps IPv4-only and IPv6-only links
valid while providing dual-stack evidence when both are available. It repairs
the lowest failed layer. If link, address, route, and gateway are healthy while
the public underlay is unavailable, the incident is classified as upstream and
local networking is left intact.

An isolated VPN/API endpoint is never mapped to an arbitrary process. To
restart a known local service after proving that the Internet underlay is
healthy, provide an exact target mapping at installation:

```shell
sciveo watchdog --net --repair \
  --targets "vpn.example.com:443" --policy all \
  --repair-services '{"vpn.example.com:443":["wg-quick@plant.service"]}' \
  --install
```

Only those installed units can pass the privilege helper. Sciveo watchdog,
Sciveo Admin, SSH, hostapd/dnsmasq, and the network-manager/resolver control
units are protected and cannot be mapped. A remote API outage with no explicit
local mapping is reported and safely skipped.

`--platform auto` detects Linux or macOS; on Linux it separately detects
per-interface ownership by NetworkManager, systemd-networkd, or dhcpcd. Explicit
overrides include `linux`, `ubuntu`, `debian`, `raspbian`, `raspberry-pi`, `rpi`,
`macos`, `mac`, and `darwin`, so Intel and Raspberry Pi Ubuntu machines use the
same manager-aware Linux repair backend. At startup, repair mode logs the
requested and detected platform at INFO level. On a multi-homed edge device,
`--interface eth0` is the explicit safe physical-uplink override; otherwise
Sciveo selects the default-route physical interface and avoids tunnel devices.

The network `--threshold`, `--recovery-checks`, and
`--max-disruptive-repairs` must be positive integers. `--period`, `--timeout`,
and `--repair-cooldown` must be finite positive values, and the initial repair
cooldown cannot exceed the 900-second backoff cap. `--underlay-targets` accepts
numeric IP targets only; `--underlay-policy` controls their quorum.

| Stage or condition | Check | Repair action | Verification |
| --- | --- | --- | --- |
| Outage confirmation | Concurrent target quorum fails for the configured consecutive threshold | No mutation before the threshold | Re-evaluate all targets |
| Manager/link | Manager inactive, device down, no carrier, rfkill, or disconnected exact saved profile | Restart a proven inactive manager, bring up the selected interface, unblock/re-enable Wi-Fi, or activate only one unambiguous bound profile | Recheck the full target quorum |
| Address | No usable global address or a lost DHCP lease | Rebind/renew through the detected owning NetworkManager, networkd, or dhcpcd backend | Compare address state and recheck targets |
| Route/gateway | IPv4 and IPv6 default routes are missing, or every known gateway neighbor explicitly fails | Reapply the exact profile, reconfigure the owning backend, then renew its lease | Re-snapshot dual-stack routes/gateways and recheck targets |
| DNS cache/server | Named resolution fails while numeric underlay works | Flush cache and reset learned resolver-server features | Recheck named targets |
| DNS resolver/config | DNS remains broken or the resolved symlink is missing/broken | Restart only the detected resolver; safely restore only a missing/broken resolved symlink | Recheck resolution and configured servers |
| DHCP-provided DNS | Resolver repair did not recover | Renew DNS/lease through the proven interface owner | Recheck DNS and target quorum |
| Upstream provider | Local link, address, route, and gateway are healthy but numeric Internet probes fail | No local mutation; wait with bounded backoff | Re-probe underlay |
| VPN/API endpoint | Underlay is healthy and one configured endpoint fails | Restart only an exact target-mapped, root-allowlisted `.service`; otherwise safely skip | Recheck all configured targets |
| Disruptive guard | Manager/service restart is considered | Block for Admin AP/hostapd, active VPN, or SSH; enforce per-incident budget | Open durable circuit after the budget |
| Recovery | Repairs or external recovery restore quorum | Require stable healthy checks before clearing state | Reset cooldown/circuit after `--recovery-checks` successes |

Built-in repair uses structured commands, never flushes routes, never reboots,
and never guesses which arbitrary VPN/API services to restart. The managed
service sends privileged actions only through its installed strict helper;
operators do not need to grant direct sudo rights to `nmcli`, `networkctl`,
`dhcpcd`, `systemctl`, or a shell. Its incident layer, cooldown, and disruptive
circuit survive process restarts; stable healthy checks clear them.
Watchdog repairs and Sciveo Admin network changes share one mutation lock.
Admin opens a bounded maintenance lease before its first network mutation and
keeps repairs inhibited until confirmation or lease expiry, including after a
partially failed Admin apply.

This recovery path deliberately does not guess new static addressing, unknown
Wi-Fi/VPN credentials, router configuration, or arbitrary services. Physical
NIC/cable/router failures and provider outages require a redundant uplink or
out-of-band management; a local watchdog also cannot prove inbound Internet
reachability without an external heartbeat.

## Diagnostics and Fleet

Local diagnostic report:

```shell
sciveo doctor
sciveo doctor --render json
sciveo doctor --logs --output-path /tmp/sciveo-doctor.json
```

Fleet inventory over SSH:

```shell
sciveo fleet --host operator@edge.local -i ~/.ssh/id_ed25519
sciveo fleet --net 192.168.10.0/24 --users operator,admin -i ~/.ssh/id_ed25519 --render json
```

## Network and Industrial IO

Network scans:

```shell
sciveo scan
sciveo scan --net 192.168.0.0/24 --port 22 --timeout 0.5
sciveo scan --health
sciveo scan --health --l2
sciveo scan --host 192.168.0.10 --health --ports '[22,80,443,502,554,161]'
```

SSH scan and command execution:

```shell
sciveo ssh --net 192.168.0.0/24 --users operator,admin -i ~/.ssh/id_ed25519 --list-shell '["hostname","uptime"]'
```

Modbus reads and writes:

```shell
sciveo read --proto modbus --transport tcp --host 192.168.0.10 --port 502 --id 1 --address 30001 --kind input --type RAW --count 2
sciveo write --proto modbus --transport tcp --host 192.168.0.10 --id 1 --reg '[40010,"U16",1,1]' --value 123
sciveo read --proto modbus --action scan --net 192.168.0.0/24 --render text
```

Serial Modbus:

```shell
sciveo read --proto modbus --transport serial --serial-port /dev/ttyUSB0 --baudrate 9600 --bytesize 8 --parity N --stopbits 1 --id 1 --reg '[5039,"U16",0.1,1]'
```

SNMP, MQTT, and HTTP helpers:

```shell
sciveo read --proto snmp --host 192.168.0.1 --oid 1.3.6.1.2.1.1.1.0
sciveo read --proto snmp --host 192.168.0.1 --action walk --oid 1.3.6.1.2.1.1
sciveo write --proto mqtt --host broker.local --topic plant/cmd --payload '{"limit":80}' --qos 1 --retain
sciveo read --proto http --url http://192.168.0.10/status
sciveo write --proto http --url http://192.168.0.10/api/control --value '{"enabled":true}'
```

Protocol emulators:

```shell
sciveo emulate --server modbus --profile custom --host 0.0.0.0 --port 1502 --data-json '{"device_id":7,"holding":{"40010":123}}'
sciveo emulate --server snmp --host 0.0.0.0 --port 1161 --data-json '{"oids":{"1.3.6.1.2.1.1.1.0":"lab-agent"}}'
sciveo emulate --server mqtt --host 0.0.0.0 --port 1883 --data-json '{"retained":{"plant/power":1234}}'
sciveo emulate --server http --host 0.0.0.0 --port 8080 --data-json '{"status":{"ok":true}}'
```

## Admin UI

The admin command is for first-boot and field administration of an edge machine:

```shell
sciveo admin
sciveo admin --web --wifi-ap --net 10.137.19.0/24 --ap-ssid sciveo-setup --ap-password CHANGE_ME --admin-auth none
sudo sciveo admin --install --web --wifi-ap --net 10.137.19.0/24 --ap-ssid sciveo-setup --ap-password CHANGE_ME --admin-auth none
```

The admin UI covers dashboard diagnostics, Ethernet and Wi-Fi setup, service
state, pending configuration, and installed service management. Pending
configuration is stored under `~/.sciveo/admin/`.

## VS Code Extension

The Python package can install the bundled VS Code extension asset:

```shell
sciveo extensions --install --vscode
sciveo extensions --reinstall --vscode
sciveo extensions --uninstall --vscode
```

If the editor CLI is not on `PATH`, set:

```shell
export SCIVEO_VSCODE_CLI="/path/to/code"
```

Extension config is stored under `~/.sciveo/extensions/vscode/`.

## Encrypted Chat Rooms

Start a room:

```shell
sciveo chat --serve ops-room --host 0.0.0.0 --port 8090 --max-clients 5
```

Join from another terminal:

```shell
sciveo chat --client 'sciveo-chat-v1....' --url ws://HOST:8090/ws/chat --name operator-a
```

Messages use encrypted AES-GCM envelopes, while connection setup uses a
shared-token HMAC proof. Display names are decorative; anyone with the shared
token is trusted as a room participant.

Persist and reload encrypted room history:

```shell
sciveo chat --serve ops-room --output-path ./chat-history.jsonl
sciveo chat --serve ops-room --input-path ./chat-history.jsonl --output-path ./chat-history.jsonl
```

Suppress decrypted message bodies in server logs:

```shell
sciveo chat --serve ops-room --silent
```

TLS:

```shell
sciveo chat --serve ops-room --tls-cert ./chat.crt --tls-key ./chat.key
sciveo chat --client 'sciveo-chat-v1....' --url wss://HOST:8090/ws/chat --name operator-a --tls-no-verify
```

## Agent Console

Interactive agent console:

```shell
sciveo agent --provider auto
sciveo agent --profile coder --provider auto
```

One-shot prompt:

```shell
sciveo agent --provider auto --prompt "List the repo root and summarize the important files"
```

Profiles can be loaded with `--profile NAME` or `--config PATH`. Predefined
profiles include coding, review, testing, research, and operations-oriented
specializations.

Local runtime examples:

```shell
sciveo agent --action pull --model org/model-name --alias local-agent-model
sciveo agent --action run --model local-agent-model --host 127.0.0.1 --port 8910 --device mps --context 8192
sciveo agent --provider hf --prompt "Say hello in one sentence"
```

Agent orchestration uses the chat transport:

```shell
sciveo agent --action orchestrate --serve design-room --prompt "Review this module" --agents researcher,coder,reviewer,tester
sciveo agent --action orchestrate --serve design-room --prompt "Plan the next pass" --agent-write-policy discussion-only
```

## Media Capture and Processing

Capture and stream helpers:

```shell
sciveo nvr --input-path cams.json
sciveo rtsp --url rtsp://camera/stream --output-path ./clip.mp4
sciveo capture --output-path ./screen.mp4 --fps 10
```

Run the queue/API-backed media worker:

```shell
sciveo media-server
sciveo media-run --mode worker
```

Run local/offline media processing:

```shell
sciveo media-run --input-path ./media --output-path ./media-out --render text
sciveo media-run --processor image-resize --input-path ./input.jpg --output-path ./image-out
sciveo media-run --config pipeline.json --input-path ./input.mp4 --output-path ./video-out
```

Run local ML media processing:

```shell
sciveo media-run --mode ml --processor image-to-text --input-path ./input.jpg --output-path ./ml-out
```

Local mode does not require API, queue, S3, or cloud credentials. Worker mode is
for API/queue/storage-backed jobs.

## SCIVEYOLO Object Detection

Sciveo provides a SCIVEYOLO inference runtime in `sciveo.ml.images.sciveyolo`.

Goals:

- load `.sciveyolo` artifacts without Ultralytics installed;
- use a Sciveo-owned PyTorch SCIVEYOLO graph for `engine="pt"`;
- preserve a familiar `SCIVEYOLO(...).predict(...)` style API;
- support SCIVEYOLO model sizes `n`, `s`, `m`, `l`, and `x` through one generic
  scale-parameterized network;
- keep build-time conversion separate from production runtime.

Basic use:

```python
from sciveo.ml.images.sciveyolo import SCIVEYOLO

model = SCIVEYOLO("model.sciveyolo")
results = model.predict("image.jpg", conf=0.25, iou=0.7)
```

Build a Sciveo artifact from a PyTorch checkpoint:

```python
from sciveo.ml.images.sciveyolo import SCIVEYOLO

SCIVEYOLO.build("detector.pt", "detector.sciveyolo")
```

The default build engine is `pt`, which stores a state dict and metadata inside
the `.sciveyolo` file. The artifact metadata includes `engine`, `runtime`,
`architecture`, `scale`, `model_size`, `variant`, class names, image size, and
source hash.

Auto-build sidecar behavior:

```python
model = SCIVEYOLO("detector.pt")
```

When given a `.pt` path, Sciveo first looks for a same-name `.sciveyolo`
sidecar next to it. If present, that artifact is loaded. If absent, Sciveo
attempts to build it with `engine="pt"` and then loads the generated artifact.

Build an ONNX artifact only when explicitly wanted:

```python
SCIVEYOLO.build("detector.pt", "detector-onnx.sciveyolo", engine="onnx")
```

Build a TensorRT-backed artifact when running on NVIDIA CUDA hosts:

```python
SCIVEYOLO.build("detector.pt", "detector-trt.sciveyolo", engine="trt")
```

Build-time conversion may use a separate environment with export tooling. The
production runtime only needs the dependencies required by the artifact engine,
for example PyTorch for `engine="pt"`, ONNX Runtime for `engine="onnx"`, or
ONNX Runtime GPU plus TensorRT for `engine="trt"`.

Install the runtime extra that matches the deployment target:

```shell
pip install "sciveo[sciveyolo]"
pip install "sciveo[sciveyolo-gpu]"
pip install "sciveo[sciveyolo-trt]"
```

Use a fresh pyenv for the GPU/TRT extras when possible, because
`onnxruntime` and `onnxruntime-gpu` provide the same `onnxruntime` Python
module. The TRT extra is intended for CUDA/NVIDIA machines with compatible
drivers and TensorRT libraries.

Runtime options:

```python
model = SCIVEYOLO("model.sciveyolo", device="cpu", nms_method="numpy")
model = SCIVEYOLO("model.sciveyolo", fuse=True)
model = SCIVEYOLO("model.sciveyolo", channels_last=True)
model = SCIVEYOLO("model.sciveyolo", device="cuda:0", engine="auto", auto_engine_source="sample-video.mp4")
```

Native fine-tuning uses UT-style object-detection dataset YAML files:

```python
model = SCIVEYOLO("coco-detector.sciveyolo")
result = model.train(
    data="/datasets/coco128/data.yaml",
    project="/models/object-detection",
    name="coco128-sciveyolo-m",
    epochs=8,
    batch=24,
    augment=False,
    optimizer="SGD",
    lr0=0.005,
    lrf=0.1,
    amp=False,
)
```

The trainer writes `weights/last.pt`, `weights/best.pt`,
`weights/last.sciveyolo`, `weights/best.sciveyolo`, `args.yaml`, and
`results.csv` under `project/name`.

Evaluate an artifact on a dataset split:

```python
model = SCIVEYOLO("coco-detector.sciveyolo")
val_metrics = model.evaluate(data="/datasets/coco128/data.yaml", split="val")
test_metrics = model.test(data="/datasets/coco128/data.yaml")
```

`evaluate()` reports both native validation loss for PyTorch artifacts and
Sciveo object-detection AP/FP metrics. `test()` is the same evaluator with
`split="test"` by default.

`engine="auto"` looks for compatible `.sciveyolo` artifacts near the requested
model, checks what the selected device/runtime can actually load, runs a small
probe when multiple candidates are available, and then sets `model.engine` to
the selected engine.

Pass `auto_engine_source` as an image, video, frame array, or list of sources to
make the probe representative of the deployment workload. Without a source,
Sciveo logs that it is falling back to a synthetic frame.

For CPU deployment, ONNX Runtime can be faster than the PyTorch engine on some
machines. For GPU deployment, `engine="pt"` is the most portable path, while
ONNX/TensorRT performance depends on the installed runtime and GPU generation.

## Predictors API Server

Start the predictors API service:

```shell
sciveo predictors-server --port 8080
```

This command starts the configured Sciveo API predictor server and keeps it
running as a long-lived process.

## Local S3-Compatible Storage

Start a local S3-compatible service:

```shell
sciveo storage --s3 --paths '["/mnt/d1","/mnt/d2"]' --port 9000 --threads 32 --health-check-interval 10800
```

A JSON config can also be supplied:

```json
{
  "paths": [
    {"path": "/mnt/d1", "require_mount": true},
    {"path": "/mnt/d2", "require_mount": true}
  ],
  "credentials": {
    "access_key": "sciveo",
    "secret_key_file": "/etc/sciveo/storage-s3.secret"
  },
  "region": "us-east-1",
  "storage_name": "storage",
  "db_backend": "sqlite",
  "db_path": "~/.sciveo/storage/storage-s3.sqlite3",
  "lock_path": "~/.sciveo/storage/locks",
  "threads": 32,
  "workers": 4,
  "http": {
    "server": "gunicorn",
    "workers": 1,
    "timeout": 3600,
    "keepalive": 65
  },
  "path_health_check_interval": 10800,
  "capacity_check_interval": 5,
  "min_free_ratio": 0.05,
  "min_free_bytes": 1073741824,
  "resume_free_ratio": 0.08,
  "target_free_ratio": 0.15,
  "limits": {
    "max_keys": 1000,
    "max_request_bytes": 10737418240,
    "buckets": {"*": {"max_object_size": 10737418240}}
  },
  "retention": {
    "buckets": {"archive": {"minimum_age": "24h"}}
  }
}
```

`threads` is the request thread count in each storage worker. `workers` is the
number of supervised storage workers: worker `0` listens on `port`, worker `1`
on `port + 1`, and so on. `parallel` remains a backward-compatible alias for
`threads`. When top-level `workers` is greater than one, keep `http.workers` at
one; `http.workers` is the older Gunicorn mode where several processes share a
single port.

The following starts one master supervising ports `3900` through `3903`, with
32 request threads per worker:

```shell
sciveo storage --s3 --config /etc/sciveo/storage-s3.json \
  --port 3900 --workers 4 --threads 32 --master
```

The master role is the default when neither `--master` nor `--slave` is given,
including when `workers` is the default value of one. Keeping `--master` in new
service scripts makes the intended process role explicit. The master does not serve S3 requests. It
starts the children, restarts an unexpectedly failed child with bounded crash
loop protection, and forwards `SIGINT`/`SIGTERM` shutdown. Each child is an
independent server using the same paths, metadata DB, and root-derived lock
directory.

Existing commands without a role option continue to expose the same configured
port, now through a master supervising one child by default. Manual direct
process management remains available through an explicit slave, which serves
only the exact requested port:

```shell
sciveo storage --s3 --config /etc/sciveo/storage-s3.json \
  --slave --port 3902 --threads 32
```

### Multi-Node S3 Cluster

Cluster mode is a separate backend under `sciveo.storage.cluster`; standalone
mode remains the default and keeps its existing file layout and behavior. The
S3 protocol flag does not change:

```shell
sciveo storage --s3 --mode cluster --role controller --config controller.json
sciveo storage --s3 --mode cluster --role node --config node.json
sciveo storage --s3 --mode cluster --role gateway --config gateway.json
```

`--master`, `--slave`, `--workers`, and `--threads` still describe local process
supervision. `--role` describes the network responsibility. A gateway exposes
the boto3-compatible endpoint, a controller owns cluster metadata and placement,
and a node stores immutable replicas on its configured local paths. Object data
flows directly between gateways and nodes or between nodes during repair; it
never flows through the controller.

Controller configuration uses authoritative PostgreSQL metadata in production:

```json
{
  "s3": {
    "mode": "cluster",
    "cluster": {
      "role": "controller",
      "name": "media-cluster",
      "node_id": "controller-1",
      "token": "CHANGE_TO_A_LONG_RANDOM_INTERNAL_TOKEN",
      "replication": {"factor": 3, "write_quorum": 2, "read_quorum": 1},
      "heartbeat_interval": 10,
      "heartbeat_timeout": 30,
      "repair_interval": 30,
      "tls": {
        "ca_file": "/etc/sciveo/cluster-ca.pem",
        "cert_file": "/etc/sciveo/controller.pem",
        "key_file": "/etc/sciveo/controller.key"
      }
    },
    "host": "10.0.0.10",
    "port": 9443,
    "db": {
      "backend": "postgres",
      "url": "postgresql://sciveo:CHANGE_ME@postgres.service/sciveo_cluster"
    },
    "http": {"server": "gunicorn", "workers": 1}
  }
}
```

Every gateway and node receives a redundant controller URL list. A node also
needs a stable identity, an internally reachable advertised URL, mounted paths,
and a failure-domain label:

```json
{
  "s3": {
    "mode": "cluster",
    "cluster": {
      "role": "node",
      "name": "media-cluster",
      "node_id": "storage-1",
      "controllers": [
        "https://controller-1.internal:9443",
        "https://controller-2.internal:9443",
        "https://controller-3.internal:9443"
      ],
      "advertise_url": "https://storage-1.internal:9444",
      "token": "CHANGE_TO_A_LONG_RANDOM_INTERNAL_TOKEN",
      "failure_domain": {"site": "primary", "rack": "rack-1"},
      "replication": {"factor": 3, "write_quorum": 2},
      "tls": {
        "ca_file": "/etc/sciveo/cluster-ca.pem",
        "cert_file": "/etc/sciveo/storage-1.pem",
        "key_file": "/etc/sciveo/storage-1.key"
      }
    },
    "host": "10.0.0.21",
    "port": 9444,
    "paths": [
      {"path": "/srv/storage/d1", "require_mount": true},
      {"path": "/srv/storage/d2", "require_mount": true}
    ],
    "db_path": "~/.sciveo/storage/storage-1-node.sqlite3",
    "workers": 1,
    "http": {"server": "gunicorn", "workers": 1}
  }
}
```

Cluster PUT stages the incoming stream once, calculates CRC32 during that same
pass, writes immutable replicas concurrently, and commits the object generation
only after write quorum. ETags are opaque UUID tokens, never MD5 file hashes.
For replication factor 3, every object is stored on three distinct active nodes:
one primary copy plus two redundant copies. Nodes are selected with
capacity-weighted rendezvous hashing over the bucket, key, and stable node ID,
while preferring distinct failure domains. Placement is deterministic rather
than random, distributes different keys across the full healthy node pool, and
does not reshuffle unrelated objects when membership changes.

Reads try live replicas in turn. Each gateway rotates equivalent read candidates
so concurrent clients do not always select the same physical node; retries still
fall through to another replica. This balancing is local to each gateway process,
not a cluster-global least-in-flight scheduler. Deletes commit a tombstone before
best-effort physical cleanup. Controllers detect live under-replicated
generations and ask a new target node to pull directly from a healthy source.
They also remove live replicas above the configured factor after failed nodes
recover. Active controllers coordinate repair through an expiring lease in the
shared metadata database; each object refreshes replica liveness and retries
another source if a peer becomes unavailable during a long repair pass.

Adding a node does not require gateway changes. Give the machine a new unique
`cluster.node_id`, configure its controller URLs, advertised peer URL, TLS/token,
and mounted paths, then start the node role. It self-registers, heartbeats, and
immediately participates in future placement and repairs. Healthy existing
factor-three objects are not automatically rebalanced merely because capacity
joined the cluster.

Replacing a dead machine may reuse its stable `node_id` after the controller has
classified the old incarnation offline. Every node derives a persistent
incarnation ID from its local metadata database. A replacement with a new
incarnation invalidates the dead machine's controller replica rows, then normal
repair restores factor three from surviving copies. A second incarnation trying
to claim an identity that is still healthy is rejected, preventing accidental
split brain.

Peer URLs require HTTPS unless `cluster.allow_insecure=true` is explicitly set
for isolated loopback development. Protect the shared internal token and use
mTLS certificates on every peer endpoint. Non-loopback controllers should use
PostgreSQL HA; SQLite cluster metadata is intended only for local development.
When Gunicorn terminates peer TLS directly, controller and node roles require a
client certificate whenever `tls.cert_file`, `tls.key_file`, and `tls.ca_file`
are configured. The same requirement can instead be enforced by an internal
reverse proxy.

The initial cluster backend supports bucket lifecycle, PUT/GET/HEAD/DELETE,
range reads, prefix and delimiter pagination, bulk delete, tags, server-side copy,
SigV4 presigned GET/PUT, quorum replication, node failover, and repair. Legacy
Signature V2 remains disabled; set boto3 `Config(signature_version="s3v4")`
when generating presigned URLs for a custom endpoint. Distributed
multipart upload, presigned POST, ACLs, versioning APIs, erasure coding,
multi-site replication, and automatic orphan inventory cleanup remain future
cluster work. Standalone support for those already implemented S3 operations is
unchanged.

```shell
sciveo storage --s3 --config ./storage.json
```

`--paths` are mounted directories, not raw block devices. A path entry may be a
string or an object with `path`, `require_mount`, optional `root_id`, and
optional expected `device`. Sciveo persists an identity marker on every root so
a detached/replaced mount is not silently treated as the original storage.
Object metadata is
stored in `~/.sciveo/storage/<storage-name>-s3.sqlite3` by default with the
SQLite backend. Use `db_path`/`--db-path` for another SQLite file, or
`db_backend=postgres` with `db_url`/`--db-url` for PostgreSQL.
`storage_name`/`--storage-name` selects the default DB filename and observability
label; the persistent root identities and metadata DB, not the display name,
define the actual storage set.

For heavier production metadata concurrency, use PostgreSQL:

```json
{
  "s3": {
    "paths": ["/mnt/d1", "/mnt/d2", "/mnt/d3"],
    "access_key": "sciveo",
    "secret_key": "CHANGE_ME",
    "storage_name": "video-cache",
    "db": {
      "backend": "postgres",
      "url": "postgresql://sciveo:CHANGE_ME@127.0.0.1:5432/sciveo_storage"
    }
  }
}
```

The same settings can also be supplied as flat keys (`db_backend`, `db_url`) or
CLI options (`--db-backend postgres --db-url ...`). Install
`sciveo[storage-postgres]` and verify that `psycopg2` can load the host `libpq`
before selecting PostgreSQL.

Each metadata database has a persistent store UUID and owns one logical set of
storage roots. Starting a service with a completely disjoint path/identity set
against an occupied DB is rejected before metadata can be changed. Temporarily
omitted roots are retained with an inactive index and become visible again when
the same root identity is re-added.

The service does not run a metadata sync pass on normal startup. Run explicit
reconciliation with `--sync` when you want files that exist on disk but are
missing from the SQL metadata DB to be added, metadata rows for files missing
from currently healthy paths to be removed, and pending deletion tombstones to
be completed. Metadata for an unavailable root is retained and hidden from
list/search/GET until that root recovers; sync never converts a detached disk
into permanent object deletion.

```shell
sciveo storage --s3 --sync --config ./storage.json
sciveo storage --s3 --sync --dry-run --config ./storage.json
sciveo storage --s3 --sync --since 10m --config ./storage.json
```

`--since` accepts a Unix timestamp, an ISO timestamp, or a relative
duration such as `10m`, `2h`, or `1d`. It is intended for fast catch-up after a
full index build, for example when preparing a PostgreSQL metadata DB while the
SQLite-backed service is still live. Keep the target PostgreSQL service in sync
mode only during this phase; do not run two metadata backends as active writers.
Both configurations must use the same physical paths and `lock_path`. Stop the
SQLite writers, run one final incremental/full sync with zero inconsistencies,
then cut traffic over to PostgreSQL. Incremental sync only reconciles files with
filesystem mtimes newer than the timestamp and only prunes recently updated
metadata rows; run a full `--sync` when you need a complete consistency audit.
Foreign in-flight mutation journals are skipped and the affected keys return a
retryable unavailable response until their owning metadata DB recovers them.

New objects use capacity-weighted rendezvous hashing by persistent root UUID
across writable paths. Reordering paths or changing a mount point therefore does
not perturb equal-health placement. Paths
above `target_free_ratio` have equal placement weight; paths below that target
receive progressively less new data. A path is excluded from new allocations
when writing an object would violate either the configured `min_free_ratio` or
`min_free_bytes` reserve. It returns to placement only after
crossing `resume_free_ratio`, avoiding rapid full/not-full state changes.

A full path remains healthy and readable: indexed objects on it continue to be
listed and GET/HEAD continue to work. Only new allocations are redirected. A path
is unhealthy only when it is inaccessible, unreadable, detached/unmounted, has a
changed device while active, or reports a real filesystem I/O failure. Metadata
for an inaccessible root remains indexed but is filtered from client results;
when the same root identity returns, its objects become visible again. Configure health checks with
`--health-check-interval`; configure the capacity policy through the generic
`--config` JSON file or `SCIVEO_STORAGE_*` environment variables.

Known object sizes are reserved with filesystem preallocation where supported.
This makes concurrent processes compete for real disk capacity before consuming
the HTTP body and allows a failed candidate to fall through to the next ranked
path safely. Unknown-length streams are bounded while they are consumed: Sciveo
aborts the temporary write before commit if a configured request, object, bucket,
or remaining-capacity limit is crossed.

Object listing uses the bucket/key index with a prefix key range rather than a
table scan. Empty bucket deletion removes the whole bucket tree below each
healthy storage path in one filesystem operation. Unavailable paths are left
untouched, preserving their data for recovery.

Object mutations are coordinated by bucket/key across threads, processes, and
metadata backends on one host. Every backend uses the same root-set-derived
striped POSIX lock directory; `lock_path` or `SCIVEO_STORAGE_LOCK_PATH` can pin
that directory for hardened services. SQLite adds WAL/retry handling, while
PostgreSQL uses transactional/advisory schema locks and a fork-aware connection
pool. Multi-host writers sharing the same files are not supported; use one
storage host behind local Gunicorn workers and an nginx endpoint.
Writes, copies, and local moves keep a temporary destination backup until the
SQL transaction commits. Failed metadata commits restore the previous bytes.
Deletes first create a transactional tombstone, hiding the object immediately;
sync or maintenance can finish physical cleanup after a transient filesystem
failure. Maintenance reconciles only indexed tombstones and does not walk object
files. Schema v5 records a persistent metadata-store UUID in mutation journals
so a migration target cannot roll back a transaction owned by the source DB.

Standalone object ETags are lightweight stat-based metadata tokens; cluster
ETags are opaque UUID tokens. Neither is a content MD5 hash. This avoids hashing
very large video/object streams during upload. Use
an explicit application checksum in object metadata when content verification is
required.

Authentication uses AWS Signature V4 for boto3 requests and presigned
GET/PUT/POST. Header timestamps, credential scope, POST policy conditions, body
SHA-256, and supported streaming checksum trailers are validated. Legacy
Signature V2 is disabled unless `allow_legacy_auth` is explicitly enabled. The
demo `sciveo`/`secret` credentials are rejected by server startup unless
`allow_demo_credentials` is explicitly enabled for a disposable test. Use a
protected secret file and terminate public HTTPS at a hardened reverse proxy.

Gunicorn with `gthread` workers is the Linux production HTTP runtime. Waitress
is an explicitly configured compatibility runtime that buffers request bodies
and is intended for local tests. Sciveo fails fast instead of attempting
Gunicorn prefork on macOS.

Python client example:

```python
import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://127.0.0.1:9000",
    aws_access_key_id="sciveo",
    aws_secret_access_key="CHANGE_ME",
    region_name="us-east-1",
)

s3.upload_file("clip.mp4", "media", "clips/clip.mp4")
s3.download_file("media", "clips/clip.mp4", "clip-copy.mp4")
```

Object tags use the S3 object-tagging model. Add tags during upload with the
normal `Tagging`/`x-amz-tagging` mechanism, update them with
`put_object_tagging`, and search them through the Sciveo metadata search API:

```python
s3.put_object(
    Bucket="media",
    Key="clips/clip.mp4",
    Body=open("clip.mp4", "rb"),
    Tagging="kind=raw&project=demo",
)

s3.put_object_tagging(
    Bucket="media",
    Key="clips/clip.mp4",
    Tagging={"TagSet": [{"Key": "stage", "Value": "ready"}]},
)
```

Sciveo also provides a boto3-like remote client that keeps the normal S3
methods and adds Sciveo-native metadata search endpoints:

```python
import sciveo.storage.s3

s3 = sciveo.storage.s3.client(
    endpoint_url="http://127.0.0.1:9000",
    aws_access_key_id="sciveo",
    aws_secret_access_key="CHANGE_ME",
    region_name="us-east-1",
)

s3.upload_file("clip.mp4", "media", "clips/clip.mp4")
objects = s3.search_objects(Bucket="media", Prefix="clips/", Period=7200)
tagged = s3.search_objects(Bucket="media", Prefix="clips/", Tags={"stage": "ready"})
stats = s3.search_stats(Bucket="media", Prefix="clips/")
metrics = s3.metrics(Bucket="media")
```

The Sciveo-native HTTP surface is intentionally small and separate from AWS S3
compatibility: `/sciveo/storage/search`, `/sciveo/storage/stats`,
`/sciveo/storage/disk-usage`, `/sciveo/storage/health`,
`/sciveo/storage/roots`, `/sciveo/storage/buckets`, and
`/sciveo/storage/metrics`. Search and stats are DB-indexed and support exact
generic tag filters with `tag.NAME=value` or `Tags={"name": "value"}` through
the Python client. `/sciveo/storage/metrics` returns OpenMetrics text for
Prometheus-style scraping.

Storage stats also include rolling file-operation counts for the latest 24 UTC
hours. The profile is calculated with a SQL aggregation over the existing
hourly operation counters, so it adds no counters or writes to the request
path. JSON clients read it from `operations.hourly_files_utc`; terminal users
get the same 24-row table with:

```shell
sciveo storage --s3 --config /etc/sciveo/storage-s3.json --stats --render text
```

Each row includes its UTC hour, total file operations, `files / 60` average per
minute, and write/read/delete counts.

Disk-usage totals count each physical filesystem once even when multiple storage
roots share it. Per-root object counts and bytes remain separate so operators can
still inspect placement. S3 object tags follow AWS request limits: at most 10
tags, with keys up to 128 characters and values up to 256 characters.

Local workers running on the same storage machine can resolve an object to its
committed file path without downloading it through S3:

```python
from sciveo.storage.local import StorageLocalClient

storage = StorageLocalClient.from_config("/etc/sciveo/storage-s3.json")
storage.create_bucket("media")
path = storage.path("media", "clips/clip.mp4")

# Use path directly with local video/image processing code.
storage.copy("media", "clips/clip.mp4", dst_key="processed/clip.mp4")
storage.move("media", "tmp/clip.mp4", dst_key="queue/0/clip.mp4")
storage.delete_objects("media", ["processed/clip.mp4", "queue/0/clip.mp4"])
storage.delete_bucket("media")
```

`StorageLocalClient` reads the same storage config and SQL metadata DB as the
server. It only returns paths for committed objects whose files still exist,
rejects metadata paths outside the configured storage roots, and updates both
object files and metadata for local bucket, copy, move, and delete operations.
`move()` defaults to `no_copy=True`: the file is renamed on its current storage
path and only the bucket/key metadata changes, so changing a key does not trigger
an inter-disk copy when the destination key hashes to another path. Use
`move(..., no_copy=False)` only when the old copy/delete behavior is desired.

Run bounded metadata, staging, and deferred-delete cleanup explicitly. The
`deletions` section reports tombstones seen, reconciled, and deferred:

```shell
sciveo storage --s3 --action maintenance --dry-run --config ./storage.json
sciveo storage --s3 --action maintenance --config ./storage.json
```

Ordinary boto3 clients do not need an update for these server-side changes.
Processes importing `StorageLocalClient` should run the same current Sciveo
release as the servers so they share locking, tombstone, root-identity, and
rollback behavior. Update the Sciveo remote client when using newly added native
search/health/stats methods; normal delegated S3 method signatures remain
boto3-compatible.

When upgrading from a release before metadata schema v5, stop every S3 server
and local writer using these roots, update them together, then restart. Do not
mix old and new lock/journal implementations during a rolling restart. Schema
migration is automatic and does not run `--sync`; run an explicit dry-run/full
sync later as an operational consistency audit.

The repository includes a destructive-only-to-its-marked-directory stress
runner. It prints a parameter table before opening sockets and a result table at
completion:

```shell
PYENV_VERSION=sciveo python test/storage_runtime.py \
  --path /tmp/sciveo-storage-runtime \
  --servers 4 --threads 16 --clients 16 --roots 5 --files 240
```

The runtime mixes boto3, Sciveo remote, and local clients; exercises presigned
requests, transfers, tags, conditions, ranges, copy directives, multipart,
pagination, root loss/recovery, process-crash recovery, root omission/re-add,
disjoint-DB rejection, live sync, and parallel apply-sync while a local cleanup
client deletes the oldest indexed cohort; then verifies reclaimed free space and
exact bytes against SQL metadata. Every parallel sync pass, the final apply sync,
and the final dry-run must report zero inconsistencies.
Every run writes a staged JSON report under `/private/tmp` unless `--report`
overrides it. Use `--db-backend postgres --db-url ...` with a fresh disposable
database to repeat the same gate with PostgreSQL.

## Experiments Client

The experiment helpers expose project runs, parameter sampling, datasets,
scoring, plots, metadata, local execution, and optional remote synchronization.
They are useful for ML, AI, research, and engineering scripts where each run
should have a known configuration and a recorded result.

Minimal pattern:

```python
import sciveo

def run_once():
    with sciveo.open() as experiment:
        experiment.log({"message": "hello experiment"})
        experiment.score(0.0)

sciveo.start(
    project="example-project",
    function=run_once,
    configuration={},
    remote=False,
    count=1,
)
```

Run a project:

```python
import sciveo

def evaluate_model():
    with sciveo.open() as experiment:
        learning_rate = experiment.config.learning_rate
        experiment.log("learning_rate", learning_rate)
        experiment.score(0.94)

sciveo.start(
    project="example-project",
    function=evaluate_model,
    configuration={"learning_rate": {"values": [0.001, 0.01]}},
    remote=False,
    count=2,
)
```

Remote synchronization requires a configured Sciveo API account.

### Experiment Concepts

- **Project**: a named ML, research, or engineering workspace containing
  related runs.
- **Experiment**: one run with a sampled configuration, measurements, outputs,
  and score.
- **Configuration**: parameter values used by the run.
- **Dataset records**: structured references to input data, split definitions,
  or generated data artifacts.
- **Score**: numeric or structured result used for comparison and optimization.
- **Local mode**: runs experiments on the current machine without API
  synchronization.
- **Remote mode**: synchronizes project and experiment data through the Sciveo
  API when configured.

Example with dataset and score metadata:

```python
import sciveo

def evaluate_dataset():
    with sciveo.open() as experiment:
        experiment.dataset({
            "name": "sensor-window-001.csv",
            "split": {"train": 0.8, "test": 0.2},
        })
        experiment.eval("rmse", 0.034)
        experiment.eval("mae", 0.021)
        experiment.score(0.93)

sciveo.start(
    project="sensor-eval",
    function=evaluate_dataset,
    configuration={},
    remote=False,
    count=1,
)
```

Example local parameter sweep:

```python
import sciveo

def calibrate():
    with sciveo.open() as experiment:
        window = experiment.config.window
        threshold = experiment.config.threshold
        experiment.log({"window": window, "threshold": threshold})
        experiment.score(0.8)

sciveo.start(
    project="lab-calibration",
    function=calibrate,
    configuration={
        "window": {"values": [32, 64, 128]},
        "threshold": {"values": [0.1, 0.2, 0.3]},
    },
    remote=False,
    sampler="grid",
)
```

Sciveo does not force a specific analysis or ML stack. Experiments can call
NumPy, SciPy, scikit-learn, PyTorch, TensorFlow, OpenCV, custom sensor clients,
or any other Python code that fits the workflow.

## Python Module Map

Common package areas:

- `sciveo.monitoring`: metrics, watchdogs, monitoring CLI.
- `sciveo.network`: scan, SSH, Modbus, SNMP, MQTT, HTTP, emulators.
- `sciveo.admin`: edge admin web UI and service helpers.
- `sciveo.ops`: doctor and fleet diagnostics.
- `sciveo.agents`: agent console, providers, local runtimes.
- `sciveo.chat`: encrypted chat server/client transport.
- `sciveo.media`: capture, media CLI, pipeline workers.
- `sciveo.ml.images`: image ML helpers, SCIVEYOLO inference, embeddings,
  descriptions, image transforms.
- `sciveo.storage`: local S3-compatible object store.
- `sciveo.certification`: framework profiles, readiness scoring, suggestions,
  rendering, and certification CLI delegation.
- `sciveo.api`: API clients and predictor server.
- `sciveo.db`, `sciveo.web`, `sciveo.content`, `sciveo.tools`: shared support
  modules.

## Development Checks

Useful local checks:

```shell
python -m py_compile sciveo/cli.py sciveo/ml/images/sciveyolo/model.py sciveo/ml/images/sciveyolo/torch_net.py
python -m unittest discover -s test -p "test_*.py" -v
```

For SCIVEYOLO runtime validation, use a UT-capable build environment to create
`.sciveyolo` artifacts, then test production loading in an environment that has
PyTorch/OpenCV/NumPy but no Ultralytics installed.

## Contact

Pavlin Georgiev  
pavlin@softel.bg
