Metadata-Version: 2.1
Name: sciveo
Version: 0.2.33
Description-Content-Type: text/markdown
Provides-Extra: mon
Provides-Extra: agent
Provides-Extra: agent-local
Provides-Extra: net
Provides-Extra: server
Provides-Extra: media
Provides-Extra: media-ml
Provides-Extra: web
Provides-Extra: db
Provides-Extra: power
Provides-Extra: all
Provides-Extra: ml

# SCIVEO - ML/AI and Scientific tools

Sciveo is a practical toolkit for running, observing, and automating scientific, ML, AI, and media systems. The package is intentionally broad, but the main modules are ordered below by day-to-day operational value and how often they are expected to be used.

## Module Positioning

1. **Monitoring and Alerts** - the primary operations module for machine telemetry, GPU/server health, server power usage, alert rules, alert history, and notification delivery.
2. **Agent Console and Local LLM Runtime** - an interactive and one-shot coding/research assistant with local tools, image inputs, hosted providers, and owned local Hugging Face/GGUF runtimes.
3. **Notifications** - reusable Slack and Viber notification providers used by monitoring alerts and available to other Sciveo components.
4. **Network and Industrial IO** - network scanning, Modbus read/write, and Modbus device emulation for lab, edge, and plant-facing integrations.
5. **Media Capture and Processing** - RTSP/NVR capture, screen/camera capture, and media pipeline workers.
6. **Experiments Management Client** - experiment tracking, parameter sampling, scoring, plots, and remote synchronization with sciveo.com.
7. **Supporting API, DB, Web, Content, ML, and Tools packages** - shared client APIs, storage helpers, web/server utilities, content abstractions, ML helpers, and daemon infrastructure used by the modules above.

## CLI Help and Manifest

Every Sciveo CLI command has a focused help page:

```shell
sciveo --help
sciveo watchdog --help
sciveo help watchdog
```

The same command metadata is available as a JSON manifest for agents, scripts, and documentation tooling:

```shell
sciveo help --json
sciveo help watchdog --json
sciveo agent --help --json
```

The JSON output is the machine-readable source for command summaries, usage forms, options, notes, and examples.

## Monitoring and Alerts

The monitoring module is the main Sciveo operations component. It monitors machines ranging from small edge devices to large GPU servers and sends samples to the configured Sciveo API. The local configuration is created with `sciveo init` and normally lives in `~/.sciveo/`.

The monitor currently collects:

- `CPU`: total usage and per-core usage.
- `RAM`: used, total, and free memory.
- `DISK`: read/write counters and speeds.
- `NET`: sent/received counters and speeds.
- `TEMP`: available host temperature sensors.
- `GPU`: GPU telemetry from the GPU monitor, including GPU power draw when available.
- `POWER`: whole-machine/server power, currently stored as `POWER system` in watts.
- `INFO`: human-readable metadata and collector debug information.

Server power is intentionally separate from GPU power. GPU power remains under `GPU`, while `POWER system` means whole-machine power. When the machine exposes a supported server power meter, Sciveo reports it automatically. When power is not available, the metric is simply omitted instead of sending a fake zero value.

Start monitoring the current machine with a 60 second period:

```shell
sciveo monitor --period 60
```

Write samples to a local file as well. When a directory is passed, the default file name is `sciveo_monitor.json`:

```shell
sciveo monitor --period 120 --output-path ./
```

Start monitoring from Python without blocking the rest of the program:

```python
import sciveo

sciveo.monitor(period=120, block=False)
```

Plant/EMS monitoring is separate from server power monitoring and uses the `plant` source:

```shell
sciveo monitor --src plant --host 192.168.1.50 --port 502 --period 60 --serial ems300-1
```

### Alert Rules

The Sciveo web monitoring layer evaluates alert definitions against the latest monitoring samples. Alerts are configured per owner, can be made public by staff/admin users, and can target one or more monitor serials. Public alert definitions are global and visible to all users.

Alert definitions have a type:

- `metric_threshold`: evaluates a numeric metric from monitoring samples.
- `alive`: evaluates how old the latest sample is and detects stale or missing monitors.

Metric alert fields:

- **Metric**: a metric key such as `CPU usage`, `RAM usage`, `POWER system`, `GPU power.draw`, or any metric name that exists in the flattened monitor sample.
- **Window/Fresh time**: how many recent minutes of samples are considered for evaluation.
- **Policy/Aggregation**: `latest`, `max`, `min`, or `avg`.
- **Direction**: `above` when higher values are worse, `below` when lower values are worse.
- **Thresholds**: any number of ordered severity thresholds. Each threshold has a severity name, a color, and a numeric value.

The severity name is the operational label shown in the UI and notification text. The color is presentation metadata used for badges and Slack color bars. The default color labels are only defaults: if the severity name is changed from `caution` to `down`, then `down` is the severity label for that threshold.

Example CPU alert for a service that should keep the CPU busy:

```text
Type: metric_threshold
Metric: CPU usage
Window/Fresh time: 5 min
Policy: max
Direction: below
Thresholds:
  OK       green   50
  idle     yellow  30
  stopped  red     10
```

This evaluates the maximum CPU usage in the last five minutes. A short restart dip below 50% will not trigger if at least one sample in the window is still above 50%. If no samples arrive at all, use a separate `alive` alert to detect that condition.

Example server power alert:

```text
Type: metric_threshold
Metric: POWER system
Window/Fresh time: 5 min
Policy: latest
Direction: above
Thresholds:
  normal    green   1200
  warning   yellow  1800
  critical  red     2200
```

Alive alerts use the synthetic metric `Sample age minutes`. They can also have any number of thresholds:

```text
Type: alive
Metric: Sample age minutes
Policy: latest
Thresholds:
  OK       green   0
  down     yellow  5
  offline  red     15
```

In a Sciveo web deployment, alert evaluation and notification delivery run as background services. Users normally configure alert rules, notification targets, and alert history from the monitoring/admin UI.

### Alert Events, History, and Time

Alert events are created on severity transitions. The current state table shows the latest state per alert and monitor, while the event history keeps transition records. The UI shows today's events first and loads older history by date with normal paginated API responses. Owners are shown where permissions allow, because staff/admin users can see a wider alert set than regular users.

Alert and notification timestamps are stored as UTC and serialized by the API with an explicit UTC timezone. The web UI renders them in the browser/client local time.

### Notifications

Notification targets are standalone objects. Alerts select one or more notification targets, and when an alert event is created the selected targets receive the message. This keeps Slack/Viber configuration reusable by other components instead of hard-coding notification behavior into alert definitions.

The Sciveo notification library currently includes Slack and Viber providers. Slack messages use a short title plus a colored side bar derived from the alert color; the body includes the alert name, severity label, monitor, metric, value, threshold, owner, and evaluation details.

### Watchdog Monitoring With Action

Watchdogs are local process-style guards that execute a command when a simple local condition is exceeded. They are useful for emergency cleanup or local remediation, and are separate from the web alert system.

Memory usage watchdog for 90% max memory, measured every 10 seconds:

```shell
sciveo watchdog --src memory --threshold 90 --period 10 --execute "<some command to handle the high memory event>"
```

Disk usage watchdog for 80% max disk usage on a path, measured every 600 seconds:

```shell
sciveo watchdog --src disk --input-path "/some/path/to/watch" --threshold 80 --period 600 --execute "find /some/path/to/watch -type f -mtime +1"
```

Network/internet connectivity watchdog checks a list of `host:port` targets and executes an action when the selected policy is unhealthy:

```shell
sciveo watchdog \
  --src network \
  --targets "1.1.1.1:443,8.8.8.8:53,google.com:443" \
  --policy at-least-one \
  --threshold 3 \
  --period 30 \
  --timeout 2 \
  --execute "echo internet outage"
```

`--targets` accepts either a comma-separated list or a JSON list:

```shell
sciveo watchdog --src network --targets '["1.1.1.1:443", "8.8.8.8:53"]' --execute "echo outage"
```

Network watchdog policies:

- `at-least-one` or `any`: healthy when at least one target is reachable. This is the usual internet outage policy.
- `all`: healthy only when every target is reachable.
- `majority`: healthy when more than half of the targets are reachable.
- `count:N`: healthy when at least `N` targets are reachable, for example `count:2`.

For the network watchdog, `--threshold` means consecutive failed checks before the action runs. For example, `--threshold 3 --period 30` waits for about 90 seconds of failed checks before triggering.

## Agent Console and Local LLM Runtime

The agent module is the interactive AI workbench in Sciveo. It can run as a REPL or one-shot command, use local workspace tools, attach images, talk to hosted providers, or connect to a Sciveo-owned local Hugging Face/GGUF runtime.

The console includes basic local tools such as `bash`, `read_file`, `write_file`, `list_dir`, and `pwd`. Approval controls allow you to choose how much the agent can do automatically.

Start a local coding-agent console:

```shell
sciveo agent --provider auto
sciveo agent --profile coder --provider auto

OPENAI_API_KEY=<your-api-key> sciveo agent
```

Agent specialization profiles can be loaded explicitly with `--profile NAME` from `.sciveo/agents/NAME.md`, `.sciveo/agents/NAME.json`, or the predefined profiles in `sciveo/agents/configurations/`. Use `--config PATH` to load a specific Markdown or JSON specialization file. Predefined profiles include `coder`, `reviewer`, `tester`, `researcher`, and `ops`.

Run a one-shot prompt instead of opening the REPL:

```shell
sciveo agent --provider auto --prompt "List the repo root and summarize the important files"

OPENAI_API_KEY=<your-api-key> sciveo agent --model gpt-5.4-mini --prompt "Summarize the latest changes in this repo"
```

`openai` is the default agent provider when `OPENAI_API_KEY` is configured, with `gpt-5.4` as the default model. `--provider auto` selects from configured API keys. A remote URL can also be enough to select the runtime:

```shell
sciveo agent --provider http://127.0.0.1:8910 --mode stream
sciveo agent --url http://127.0.0.1:8910 --mode stream
```

When `--provider` itself is a URL, or when `--url`/`--host`/`--port` is provided with `auto`, Sciveo treats it as a remote Hugging Face-compatible runtime endpoint. `--provider local` is an alias for `hf`. `--provider ollama` and `--provider olama` are supported as compatibility providers.

Attach a local image path or base64 image to the next agent prompt. In the REPL, attachments are one-turn only:

```shell
sciveo agent --provider openai --prompt "Describe this image" --input-path ./screenshot.png
sciveo agent --provider hf --prompt "What is visible here?" --data-json '{"images":[{"path":"./screenshot.png"}]}'
sciveo agent --provider gemini --prompt "Read this chart" --data-json '{"images":[{"data":"data:image/png;base64,...","name":"chart.png"}]}'
```

Inside the REPL:

```text
/image ./screenshot.png
/image-data image/png iVBORw0KGgo...
/images
/image-clear
```

Install hosted agent providers with the `agent` extra:

```shell
pip install "sciveo[agent]"
```

Install the owned local Hugging Face runtime with:

```shell
pip install "sciveo[agent-local]"
```

Pull a model snapshot from Hugging Face, run the Sciveo local runtime server, then connect the agent console to it:

```shell
sciveo agent --action pull --model Qwen/Qwen2.5-Coder-3B-Instruct --alias qwen-coder-3b
sciveo agent --action run --model qwen-coder-3b --host 127.0.0.1 --port 8910 --device mps --context 8192
sciveo agent --provider hf --prompt "Say hello in one sentence"
sciveo agent --provider hf --mode batch --prompt "Say hello in one sentence"
```

For GGUF repositories with many quantized files, use `--file` to pull only one model file:

```shell
sciveo agent --action pull --model unsloth/Qwen3.5-4B-GGUF --file Qwen3.5-4B-Q4_K_M.gguf --alias qwen3.5-4b-q4
```

For GGUF vision models, also pull the matching projector file:

```shell
sciveo agent --action pull \
  --model some/vlm-gguf \
  --file model-Q4_K_M.gguf \
  --data-json '{"mmproj_file":"mmproj-model.gguf"}' \
  --alias vlm-q4
```

The HF runtime streams live output by default; use `--mode batch` for request/response style. Use `--device cpu`, `--device mps`, `--device cuda`, or `--device auto` on `agent --action run` to choose where the local model runs. Use `--context` to choose the llama.cpp context size for GGUF models; larger values use more memory. GGUF model snapshots are supported through `llama-cpp-python`; on Apple Silicon, `--device mps` requests llama.cpp Metal acceleration when the installed backend supports it.

## Network and Industrial IO

- **Sciveo Admin Panel** Browser-based edge setup over Ethernet or a temporary Wi-Fi provisioning AP
```shell
sciveo admin

sciveo admin --web --wifi-ap --net 10.137.19.0/24 --ap-ssid sciveo-setup --ap-password CHANGE_ME --admin-auth none

sudo sciveo admin --install --web --wifi-ap --net 10.137.19.0/24 --ap-ssid sciveo-setup --ap-password CHANGE_ME --admin-auth none
```

The admin panel is intended for first-boot and field administration of an edge box. With `--wifi-ap`, `--host` and `--port` are normally not needed: Sciveo binds the web UI to the AP gateway IP on port 80. `--net` supplies the AP network, for example `10.137.19.0/24`, and the AP uses the first host address such as `10.137.19.1/24`. If neither `--net` nor `--ap-cidr` is supplied, Sciveo chooses a non-conflicting private AP CIDR.

The current admin UI includes:

- Dashboard: single-glance readiness, a first-time setup checklist, timestamped status/diagnostic snapshots, CPU, memory, disk, uptime, load, temperature, swap, process count, hardware identity, network summary, services, connectivity, storage, interfaces, traffic, routes, pending configuration details, and hidden administration targets.
- Ethernet: configure wired interfaces immediately and save the same configuration for replay on the next admin start. The selected interface view shows current runtime state, persisted NetworkManager profiles, and matching pending Ethernet changes side by side.
- Wi-Fi Setup: scans on entry, supports re-scan, lets the user select visible SSIDs while still allowing manual SSID entry, and saves Wi-Fi client changes for reboot/admin restart.
- Network: read-only overview of interfaces, AP state, discovery, and routes.
- Diagnostics: rerunnable setup AP enforcement, AP client, AP CIDR overlap, Wi-Fi autoconnect guard, DNS, HTTPS/TLS, clock/time sync, route, internet, ARP, and live DHCPDISCOVER/OFFER checks, with interface selection for Ethernet-focused diagnostics and a visible last-run timestamp.
- Services: service status plus the `sciveo-admin.service` unit configuration, including enabled/load state, unit path, working directory, environment, restart policy, pending admin-service enable/disable change, checked services, and the redacted `ExecStart` command that will run.
- System/Security/Logs: current status and maintenance/security diagnostics.

Pending configuration is stored under `~/.sciveo/admin/config.json`. A pending banner shows saved changes, with Cancel and Reboot actions, while the Dashboard renders the pending network/service changes in a readable table. Applied pending config is archived under `~/.sciveo/admin/executed/`. The admin service can be installed with `sciveo admin --install ...`; the installed systemd unit preserves the CLI arguments used for provisioning and can be disabled through the admin panel when setup is no longer needed.

When started with `--wifi-ap`, the admin process enforces the setup access point: saved Wi-Fi client profiles are prevented from auto-reconnecting and `sciveo-admin-ap` is kept as the active/autoconnecting NetworkManager profile until a pending Wi-Fi client configuration is intentionally applied.

 - **Network Scan** Networks of hosts to scan for a single port
 ```shell
sciveo scan --port 22

sciveo scan

sciveo scan --net 192.168.0.0/24 --port 22 --timeout 0.5
```

- **Network Scan** Single Host list of ports to scan
```shell
sciveo scan --host 192.168.0.10

sciveo scan --host 192.168.0.10 --ports [443,80]
```

- **Network Health Scan** Local configuration, gateway, DNS, internet reachability, duplicate-IP health, LAN discovery, host identity checks, and service-sorted exposure findings. Text mode shows a Softel Labs progress prelude followed by a polished terminal report; `--render json` stays quiet and script-safe.
```shell
sciveo scan --health

sciveo scan --l2

sciveo scan --health --l2

sciveo scan --net 192.168.0.0/24 --health

sciveo scan --net 192.168.0.0/24 --health --mode deep --render json

sciveo scan --host 192.168.0.10 --health --ports '[22,80,443,502,554,161]'
```

Health scan modes and defaults:

- Install network diagnostics with `pip install "sciveo[net]"`; duplicate-IP ARP checks also need raw-socket privileges such as `sudo` on Linux or `/dev/bpf` access on macOS.
- `--mode gentle` keeps the scan light: minimal ports, 1 ARP round, no ICMP sweep, and 1 socket attempt.
- `--mode balanced` is the default field scan: common Ops LAN ports, 4 ARP rounds, ICMP host sweep, SSH identity grouping, and 1 socket attempt.
- `--mode deep` is intentionally more aggressive: broader Ops/industrial/admin/database/camera ports, 10 ARP rounds, ICMP host sweep, SSH identity grouping, and 2 socket attempts per host/port.
- `--parallel 32` runs host/port probes concurrently.
- `--ports '[...]'` overrides the health-mode port list for focused checks.
- In the default interactive terminal, `sciveo scan` without `--net` or `--host` opens a local interface/network selector for the regular single-port scan, with advanced options for port, timeout, and localhost.
- In the default text terminal mode, `sciveo scan --health` without `--net` or `--host` opens a local interface/network selector sorted by interface type such as ethernet, Wi-Fi, tunnel/VPN, and virtual. The selector shows the command being built and includes an Advanced options row for mode, ports, external targets, timeout, and parallel workers. JSON/automation mode should pass `--net` or `--host` explicitly.
- Guided scan menus print the finished `sciveo` command after the menu exits, so the same scan can be copied or reused later.
- Local configuration parsing detects the host OS and supports macOS, Raspberry Pi OS/Debian/Ubuntu, and CentOS-style route/DNS/neighbor command output.
- Local health checks report unresolved hostnames, multiple local target-network IPs on one interface, and wired interfaces that are down or unaddressed while Wi-Fi is carrying the target LAN.
- `sciveo scan --l2` does not require `--net`; it reports linked local interfaces, interfaces with link but no IPv4, ARP/NDP neighbor-table MAC/IP evidence, IPv6 link-local service candidates, and safe TCP probes to those link-local candidates.
- `sciveo scan --health --l2` folds the same L2/link-local evidence into the health report.
- ARP maps IPv4 to MAC, so a true zero-IPv4 Sciveo device cannot be reached through ARP alone. For no-IPv4 communication, prefer IPv6 link-local first; DHCP/LLDP evidence, switch MAC tables, or a purpose-built raw Ethernet protocol can be added later for deeper recovery workflows.
- External targets default to `1.1.1.1:443`, `softel.bg:443`, and `sciveo.com:443`; failed target names are included in findings.
- Duplicate-IP health is red/critical only when the same IP is seen with multiple MACs. One MAC on multiple IPs is yellow/warning because interface aliases can be valid.
- SSH identity grouping warns when multiple IPs present the same host-key fingerprint, which helps reveal aliases, NAT, or an accidental IP takeover that still answers as the same machine.
- RTSP/Modbus/SNMP-style exposure findings are grouped by service/port, for example one RTSP 554 warning listing all camera/NVR hosts.
- ARP duplicate detection may require elevated privileges on macOS because scapy needs `/dev/bpf`.

- **SSH Scan And Execute** Scan a network for SSH, try one or more users with an identity file, and run remote commands
```shell
sciveo ssh \
  --net 10.37.0.0/24 \
  --port 22 \
  -i ~/develop/aws/smiveo/sciveo_stan_ed25519 \
  --users "stan,sgeorgiev,ubuntu,ec2-user" \
  --execute hostname \
  --render online

sciveo ssh \
  --host "ubuntu@10.37.0.18,ec2-user@10.37.0.22" \
  -i ~/.ssh/id_ed25519 \
  --execute '["hostname", "uptime"]' \
  --render text
```

`--host` accepts `host`, `user@host`, comma-separated hosts, or a JSON list.
`--users` accepts a comma-separated list or a JSON list.
`--execute` accepts a single command, newline-separated commands, or a JSON list of commands.
`--render online` prints a compact connected/failed table; `--render json` returns structured results.

- **Modbus Read** Holding or input registers over TCP
```shell
sciveo read

sciveo read --proto modbus --transport tcp --host 192.168.0.10 --port 502 --id 1 --address 30001 --kind input --type RAW --count 2

sciveo read --proto modbus --transport serial --serial-port /dev/ttyUSB0 --baudrate 9600 --bytesize 8 --parity N --stopbits 1 --id 1 --reg '[5039, "U16", 0.1, 1]'
```

- **Modbus Scan** Discover Modbus TCP devices and classify common plant controllers
```shell
sciveo read --proto modbus --action scan --net 10.37.0.0/24 --render text

sciveo read --proto modbus --action scan --host 10.37.0.14 --id 247
```

- **Modbus Write** Holding registers over TCP or serial
```shell
sciveo write

sciveo write --proto modbus --transport tcp --host 192.168.0.10 --id 1 --reg '[40010, "U16", 1, 1]' --value 123

sciveo write --proto modbus --transport serial --serial-port /dev/ttyUSB0 --baudrate 9600 --bytesize 8 --parity N --stopbits 1 --id 1 --address 10 --type RAW --value "[1,2]"

sciveo write --proto modbus --transport serial --serial-port /dev/ttyUSB0 --write-config modbus-write.json
```

`--reg` accepts a compact value spec like `[address, type, factor, count]`.
`--write-config` accepts a JSON list where each item includes a register spec and `value`.
`--proto` defaults to `modbus`, and `read`/`write` also support `snmp`, `mqtt`, `http`, and `https`.
In an interactive terminal, `sciveo read` and `sciveo write` open a guided Modbus menu when required parameters are missing. The menu covers TCP/serial endpoint, register address/kind/type/factor/count, write value/config, serial settings, timing, and zero-based addressing, then prints the finished `sciveo` command.

- **SNMP Read/Write** GET, WALK, and SET for network and industrial devices
```shell
sciveo read --proto snmp --host 192.168.0.1 --oid 1.3.6.1.2.1.1.1.0

sciveo read --proto snmp --host 192.168.0.1 --action walk --oid 1.3.6.1.2.1.1

sciveo write --proto snmp --host 192.168.0.1 --community private --oid 1.3.6.1.4.1.42369.4.2.1 --type integer --value 7500
```

Softel Labs SNMP services use the enterprise OID root `1.3.6.1.4.1.42369`.
Sciveo monitoring and lab SNMP emulators use this namespace for Softel/Sciveo runtime, monitoring, energy, alarm, and configuration values.

| OID | Name | Purpose |
| --- | --- | --- |
| `1.3.6.1.4.1.42369` | Softel Labs root | Enterprise namespace for Softel Labs SNMP objects |
| `1.3.6.1.4.1.42369.3` | `softelSciveo` | Sciveo runtime and monitoring subtree |
| `1.3.6.1.4.1.42369.3.2.1` | `sciveoMonitorCpuUsage` | CPU usage percent, scaled by 100 |
| `1.3.6.1.4.1.42369.3.2.2` | `sciveoMonitorRamUsed` | Used RAM in bytes |
| `1.3.6.1.4.1.42369.3.2.3` | `sciveoMonitorDiskUsage` | Root disk usage percent, scaled by 100 |
| `1.3.6.1.4.1.42369.3.2.4` | `sciveoMonitorLastUploadStatus` | Monitor/status string |
| `1.3.6.1.4.1.42369.4` | `softelEnergy` | Energy and plant telemetry subtree |
| `1.3.6.1.4.1.42369.4.2.1` | `energyDispatchLimit` | Plant dispatch active-power limit percent, scaled by 100 |

- **MQTT Read/Write** Subscribe for one message or publish a payload
```shell
sciveo read --proto mqtt --host broker.local --topic 'site/+/power' --timeout 5

sciveo write --proto mqtt --host broker.local --topic plant/cmd --payload '{"limit":80}' --qos 1 --retain
```

- **HTTP Read/Write** GET/POST device JSON APIs
```shell
sciveo read --proto http --url http://192.168.0.10/status

sciveo write --proto http --url http://192.168.0.10/api/control --value '{"enabled":true}'
```

SNMP and MQTT protocol dependencies are installed with `pip install "sciveo[net]"`. HTTP uses the base `requests` dependency.

- **Modbus Emulate** Run a Modbus TCP emulator with a bundled profile or a custom profile override
```shell
sciveo emulate --server modbus --profile ems300 --host 0.0.0.0 --port 1502

sciveo emulate --server modbus --profile custom --host 0.0.0.0 --port 1502 --data-json '{"device_id": 7, "input": {"30001": [10, 20]}, "holding": {"40010": 123}}'
```

`--profile ems300` starts the bundled EMS300 preset.
Other bundled profiles include `logger1000-sg` for Sungrow Logger1000 and `logger1000-hw` for Huawei SmartLogger1000.
`--host 0.0.0.0` binds the emulator on all interfaces; use `127.0.0.1` for local-only testing.
`--data-json` can supply or override emulator register data using external register numbers.

- **Protocol Emulators** Run SNMP, MQTT, and HTTP test services for lab/device integration checks
```shell
sciveo emulate --server snmp --host 0.0.0.0 --port 1161 --data-json '{"oids": {"1.3.6.1.2.1.1.1.0": "lab-agent"}}'

sciveo emulate --server snmp --mode monitor --host 0.0.0.0 --port 1161

sciveo emulate --server mqtt --host 0.0.0.0 --port 1883 --data-json '{"retained": {"plant/power": 1234}}'

sciveo emulate --server http --host 0.0.0.0 --port 8080 --data-json '{"status": {"ok": true, "service": "lab"}}'
```

Emulator default ports avoid privileged binds where useful: Modbus `1502`, SNMP `1161`, MQTT `1883`, and HTTP `8080` when `--port` is omitted. SNMP supports basic v1/v2c GET, GETNEXT/WALK, and SET; `--mode monitor` refreshes host CPU/RAM/DISK usage OIDs under the Softel Labs Sciveo subtree `1.3.6.1.4.1.42369.3` from the machine where the emulator runs. MQTT is a lightweight MQTT 3.1.1 test broker; HTTP serves JSON GET routes and stores POST/PUT/PATCH payloads by path.

## Media Capture and Processing

### Remote Network Video Recorder

 - **NVR** RTSP cams video recorder
 ```shell
sciveo nvr --input-path cams.json
```

where cams.json has the cam and nvr configuration like
```json
{
  "video_retention_period": 5,
  "max_video_len": 60,
  "transport": "tcp",
  "path": {
    "tmp": "tmp/nvr/tmp",
    "video": "tmp/nvr/video"
  },
  "cam": {
    "cam_livingroom": {
      "url": "rtsp://192.168.1.11/stream1"
    },
    "cam_kitchen": {
      "url": "rtsp://rtsp://192.168.1.12"
    }
  }
}
```


### Media Pipelines Processing Worker

Media pipelines turn uploaded or captured content into derived content: resized images, video frames, albums, extracted audio, waveforms, metadata, embeddings, generated captions, object detections, time-series artifacts, and similar outputs. A pipeline job has input content, a processor graph, and optional postprocessors that store or attach generated outputs.

Start the media pipeline worker:

```shell
sciveo media-server
```

The worker consumes configured media jobs and runs the requested processors. Common processor names include:

- `media-info`: read image/video metadata.
- `image-resize`: create resized image variants.
- `image-histogram`, `image-filters`, `image-watermark`: image analysis and transformation.
- `image-to-text`, `image-object-detection`, `image-segmentation`, `image-depth-estimation`, `image-embedding`: ML-oriented image processors.
- `video-resize`, `video-downsample`, `video-frames-extract`, `video-motion-detector`, `video-album`, `video-to-text`: video processors.
- `audio-extract`: extract audio and create waveform images.
- `archive-zip`: archive generated or selected files.
- `sci-timeseries-trainer`, `sci-timeseries-predictor`, `project-datasets-plots`: scientific/time-series processors.

Run the currently exposed direct helper for audio waveform plotting:

```shell
sciveo media-run --processor audio-plot --width 1280 --height 320 --rate 16000 --input-path ./audio.aac --output-path ./waveform.png
```

Run a local media pipeline without AWS, S3, queue, or Sciveo API credentials:

```shell
sciveo media-run --input-path ./media --output-path ./media-out --render text

sciveo media-run --processor image-resize --input-path ./input.jpg --output-path ./image-out

sciveo media-run --config pipeline.json --input-path ./input.mp4 --output-path ./video-out
```

`media-run` supports explicit execution modes:

- `--mode local`: default local/offline lane for non-ML media processors. No AWS, S3, queue, or Sciveo API credentials are required.
- `--mode ml`: local file processing with model-backed processors enabled, for example `image-to-text`, `image-object-detection`, `image-segmentation`, `image-depth-estimation`, `image-embedding`, or `video-to-text`. This mode may load optional ML dependencies and local model caches.
- `--mode worker`: queue/API/S3-backed worker lane, equivalent to `sciveo media-server`.

Examples:

```shell
sciveo media-run --mode ml --processor image-to-text --input-path ./input.jpg --output-path ./ml-out

sciveo media-run --mode worker
```

When `--input-path` is a local image directory, the default pipeline creates resized image variants. When it is a local video or video directory, the default pipeline extracts representative frames and resizes them. Explicit local configs can be supplied with `--config` or `--data-json`. Remote processors/postprocessors such as S3 download/upload are intentionally not part of the local or ML paths; use `sciveo media-server` or `sciveo media-run --mode worker` for queue/API/S3-backed worker jobs.

Run a small local pipeline from Python:

```python
from sciveo.media.pipelines.pipeline import MediaPipeline

job = {
    "id": "local-image-job",
    "input": [
        {
            "guid": "image-1",
            "content_type": "image",
            "local_path": "./input.jpg",
            "key": "input.jpg",
            "processors": "",
        }
    ],
    "configuration": {
        "force_processing": True,
        "processors": {
            "image-resize": {
                "configuration": {"heights": [720, 480]}
            }
        },
        "postprocessors": {},
    },
}

result = MediaPipeline(job).run()
print(result["output"])
```

Processor graphs can be chained with `next` blocks. For example, extract representative video frames and resize each generated frame:

```python
job["configuration"]["processors"] = {
    "video-frames-extract": {
        "configuration": {"frames-count": 8},
        "next": {
            "image-resize": {
                "configuration": {"heights": [720, 360]}
            }
        },
    }
}
```

Use `pip install "sciveo[media]"` for core media processing. Use `pip install "sciveo[media-ml]"` when the pipeline needs ML processors such as image captioning, segmentation, embeddings, diffusion, or time-series training.


## Experiments Management Client
`sciveo` is a Python library that serves as a client for managing machine learning and scientific experiments on the sciveo.com platform. This library provides a convenient interface to interact with the sciveo.com API, enabling users to organize, track, and analyze their experiments efficiently.
There are few configuration params samplers, which allows easy parameter tuning. The "auto" sampler perhaps is the easiest to use, but also
"random" and "grid" ones are available.

#### Features

- **Experiment Tracking:** Easily log and track your machine learning experiments.
- **Experiment Comparison:** Compare different experiments and their results.
- **Data Visualization:** Visualize experiment metrics and results.
- **Integration with sciveo.com:** Seamlessly connect and synchronize with the sciveo.com platform.
- **Monitoring machines (from HPC to jetson nano):** Visualisation and metrics collection in sciveo platform.


## Installation

Main Sciveo package:

```shell
pip install sciveo
```

Monitoring:

```shell
pip install "sciveo[mon]"
```

Agent hosted providers:

```shell
pip install "sciveo[agent]"
```

Agent owned local Hugging Face/GGUF runtime:

```shell
pip install "sciveo[agent-local]"
```

Network and Modbus tools:

```shell
pip install "sciveo[net]"
```

The `net` extra includes `netifaces`, `scapy`, and `pymodbus`.

Server/web helpers:

```shell
pip install "sciveo[server]"
pip install "sciveo[web]"
```

Full operations install without media/ML support:

```shell
pip install "sciveo[all]"
```

Media and ML processors:

```shell
pip install "sciveo[media]"
pip install "sciveo[media-ml]"
```


## Example usage

There are few public examples in sciveo.com.

The library has local and remote mode. The local one is ready to use, but for the remote one you will need a sciveo.com account.

When have sciveo account:
```shell
sciveo init
```
Where ~/.sciveo/ path and ~/.sciveo/default file will be created. Just need to change the secret_access_key value.

or
```shell
export SCIVEO_SECRET_ACCESS_KEY='my_sciveo_user_auth_token'
```
or create a file like ~/.sciveo/some_file_name where put:
secret_access_key=my_sciveo_user_auth_token

Sciveo Monitoring cli
```shell
sciveo monitor --period 120
```

Monitoring started along with other python code.
```python
import sciveo

# Non blocking monitoring, so continue the code execution after it.
sciveo.monitor(period=120, block=False)

#rest of your python code here

```


Experimental Projects management

```python

# These are experiment specific imports for the demo purposes only.
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score, mean_absolute_percentage_error
from ml.time_series import TimeSeriesTrainer, TimeSeriesPredictor

# This is the only needed import when using sciveo along with the experiment-related imports
import sciveo

def train():
    # sciveo.open() method returns current Experiment object, with its configuration sample
    with sciveo.open() as E:
        # Just an example time series trainer (using TF/Keras simple 1D conv model).
        trainer = TimeSeriesTrainer(
            ds,
            E.config.input_window, # Experiment.config is the configuration, so input_window as hyper parameter.
            E.config.input_window,
            E.config.steps # steps parameter.
        )
        trainer.create()

        history = trainer.train(E.config.max_epochs, E.config.patience)
        trainer_eval = trainer.evaluate()

        model_name, model_path = trainer.save("model-name-path.timeseries")

        # Experiment logging for everything which seems interesting for the experiment.
        E.log({"model_path": model_path})

        E.log({"train history": history.history})
        E.log({"trainer_eval": trainer_eval})

        # Plot data, various input types (dict, list etc.).
        # Showing data as charts (single and combined) and tables.
        # There is a more advanced render option which could be used for tables, charts definition.
        E.plot("train history", history.history)

        predictor = TimeSeriesPredictor(model_path)
        Y_predicted, Y_valid, x = predict_chunk(ds.dataset["test"], predictor)

        # Plot predicted and labeled
        for i, col_name in enumerate(ds.columns):
            y_predicted = Y_predicted[0,:,i].numpy().tolist()
            y_valid     = Y_valid[0,:,i].numpy().tolist()

            # Could provide x column for the plot, there are multiple options like timestamps etc.
            # The "X" is reserved for x column name, if not present default range [1, N]
            E.plot(col_name, { "predicted": y_predicted, "label": y_valid, "X": x })

        mse = mean_squared_error(y_valid, predictions)
        mae = mean_absolute_error(y_valid, predictions)
        rmse = np.sqrt(mean_squared_error(y_valid, predictions))
        r2 = r2_score(y_valid, predictions)
        mape = mean_absolute_percentage_error(y_valid, predictions)

        E.log({"R2": r2})
        E.log(f"RMSE: {rmse}")
        E.log("MAPE", mape)
        E.log("R2", r2, "RMSE", rmse, "MAPE", mape)

        # There is a sorting option for the Project's experiments
        # By default it is "score", so there is a method Experiment.score() which could be used for experiments evaluation.
        E.score(100 - mape)
        # There is explicit Experiment "eval" section where all available evaluation metrics could be logged.
        E.eval("R2", r2)
        E.eval("RMSE", rmse)
        E.eval("MAPE", mape)


# Configuration of the Project's experiments run.
configuration = {
    "input_window": {
        "values": [10, 20, 30, 40, 50, 100, 200] # "values" option provides selection from a list of values.
    },
    "steps": {
        # "min"/"max" is a range of values where sampling will get next value.
        # It is int/float sensitive, so if range is [1, 100], the sampled value will be integer.
        # If range is [1.0, 100.0], sampling float values.
        "min": 1, "max": 100
    },
    "max_epochs": (10, 50), # Same range of values but using a tuple (min, max).
    "patience": (1, 3),
    "idx": {
        "seq": 1 # Sequence sampling, so just increase it on every run, could be used as experiment index.
    }
}

# Dataset info
sciveo.dataset({"name": "EURUSD60.csv", "split": ds.ratios}) # any dict with params.

# sciveo.start() method starts the Project's experiments run.
sciveo.start(
    project="TimeSeriesTrainer param tune", # Project name, could be existing or a new one.
    configuration=configuration, # The hyper param configuration
    function=train, # Function which will be executed on every loop.
    remote=True, # There are 2 modes: local and remote. For remote option there is a need of sciveo.com authentication.
    count=20, # Number of experiments which will be run.
    sampler="random" # Configuration sampling method, options currently are "random" (by default), "grid" and "auto".
)

```




### Who do I talk to? ###

* Pavlin Georgiev
* pavlin@softel.bg
