Metadata-Version: 2.4
Name: visionops
Version: 0.1.1
Summary: CLI application for CV reliability sandbox and Root-Cause-Analysis
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://visionops-ai.web.app/
Keywords: computer-vision,reliability,mlops,rca,cli
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer>=0.12.0
Requires-Dist: pydantic>=2.8.0
Requires-Dist: PyYAML>=6.0.1
Requires-Dist: boto3>=1.34.0
Requires-Dist: requests>=2.32.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: Pillow>=10.0.0
Requires-Dist: opencv-python>=4.9.0.80
Requires-Dist: albumentations<1.4.0,>=1.3.1
Requires-Dist: PyJWT>=2.8.0
Requires-Dist: python-dotenv>=1.0.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Dynamic: license-file

# VisionOps-AI

`visionops` is a CLI-first application for pre-deployment reliability testing and root-cause analysis (RCA) of computer-vision pipelines and repositories.

It is designed to answer not only _whether_ performance drops under corruption, but also _where_ the failure likely starts in the pipeline and what to fix next.

## What it does

- Runs a baseline pipeline in a sandbox-style workspace.
- Generates corruption suites (blur, occlusion, low light, compression, crop truncation, perspective warp, noise).
- Re-runs pipeline and metrics on corrupted datasets.
- Compares baseline vs suite metrics and detects regressions.
- Performs stage localization using runtime evidence (metrics, logs, execution behavior).
- Uses AI agents for repo mapping, RCA, and fix recommendations.
- Writes `summary.json`, `artifact_manifest.json`, `report.md`, and `report.html`.

A key differentiator in Visionops is repository understanding. Before generating RCA hypotheses, the system builds a lightweight structural map of the target codebase (entry points, pipeline stages, likely data flow, and high-signal files) and combines that map with runtime evidence from regressions, logs, and command outcomes. This repo-aware context helps it produce recommendations tied to real files and likely failure boundaries instead of generic “check your model” advice. In practice, that means faster triage, more actionable fixes, and better trust in the output because each hypothesis is grounded in both observed behavior and code structure.

## Install

```bash
python -m pip install visionops
```

## Environment Variables

The CLI loads `.env` automatically at startup (without overriding already-set shell variables).

Optional keys:
- `GEMINI_API_KEY`
- `GITHUB_APP_PRIVATE_KEY`
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_DEFAULT_REGION`

If `GEMINI_API_KEY` is missing, agent behavior falls back to deterministic logic.


## CLI commands

### 1) Create starter config

```bash
visionops init-config --path config.yaml
```
### 2) Autogenerate config

```bash
visionops autoconfig --repo-path /path/to/cv-repo --output config.yaml
```
This scans a repository and infers:

- install command
- baseline run command
- metrics command
- dataset and ground-truth paths
- task type and suite defaults

Review the generated file before running.

### 3) Validate config

```bash
visionops validate-config --config config.yaml
```

To preview without writing and inspect confidence metadata:

```bash
visionops autoconfig --repo-path /path/to/cv-repo --dry-run --explain
```

### 4) Run full local prototype (primary path)

```bash
visionops local-run --repo-path /path/to/cv-repo --config config.yaml
```

This performs:

1. sandbox copy of repo
2. install command
3. baseline run + metrics
4. corruption dataset generation
5. per-suite reruns + metrics
6. regression analysis
7. stage localization
8. repo mapping + RCA + fix suggestions
9. report generation

### 5) Run AWS path (secondary path)

```bash
visionops aws-run --repo owner/repo --config config.yaml
```

Behavior:

- checks AWS credentials/config
- starts CodeBuild build
- polls status until terminal state
- returns build id, status, logs, artifact location
- private repo support uses GitHub App auth (no PAT)

If setup is incomplete, command exits gracefully with actionable guidance.

### 6) Pretty print an existing report

```bash
visionops report --path ./visionguard_output_YYYYMMDDTHHMMSSZ
```

### 7) Environment health check

```bash
visionops doctor --config config.yaml --repo-path /path/to/cv-repo
```

This validates config, checks key environment variables, and runs autoconfig sanity checks.

### 8) Built-in metrics fallback

If a target repo does not provide a metrics script, you can use built-in YOLO-format scoring:

```bash
visionops builtin-metrics --pred /path/to/pred --gt /path/to/gt
```

## AI-assisted RCA approach

Three AI agents:

- `RepoMapperAgent`: scans repo and identifies likely pipeline files/stages.
- `RCAAgent`: combines runtime evidence + repo map into ranked hypotheses.
- `FixAgent`: converts rule-based remediations into concrete fix suggestions.

By default the application uses Gemini 2.5 Pro via OpenAI-compatible API shape. Set `GEMINI_API_KEY` in your environment or in `.env` (see above). If it is missing, all agents use deterministic fallback logic.

## Evidence grounding

RCA is grounded in:

- metric deltas
- per-suite regressions
- logs and return codes
- stage localization heuristics
- repo-aware file mapping

The LLM is never the sole source of truth.

## AWS sandbox expectations

Before `aws-run`:

1. configure AWS credentials (`aws configure` or env vars)
2. create a CodeBuild project named in config (`aws.codebuild_project`)
3. ensure IAM allows CodeBuild + S3 artifact access
4. optionally configure GitHub App values for private repos


## Project Links

- Homepage: https://visionops-ai.web.app/

## License

Proprietary. Copyright (c) 2026 VisionOps AI Lab. All rights reserved.
