Metadata-Version: 2.4
Name: inferyx-monitoring
Version: 1.0.33
Summary: Monitor batch pipelines via API and email alerts — install and deploy on Linux servers from PyPI
Author-email: Inferyx DevOps <devops@inferyx.com>
License: Copyright (c) 2026 Inferyx. All rights reserved.
        
        Proprietary software. Unauthorized copying, distribution, or use is prohibited
        unless agreed in writing with Inferyx.
        
        For open-source release, replace this file with your chosen license (e.g. MIT, Apache-2.0)
        before publishing to public PyPI.
        
Keywords: batch,monitoring,pipeline,email,inferyx
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: System Administrators
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: python-dateutil>=2.8.2
Provides-Extra: admin
Requires-Dist: fastapi>=0.110.0; extra == "admin"
Requires-Dist: uvicorn[standard]>=0.27.0; extra == "admin"
Requires-Dist: pydantic>=2.0.0; extra == "admin"
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: check-manifest; extra == "dev"
Dynamic: license-file

# inferyx-monitoring

Monitor batch jobs from a CSV file, poll the Inferyx API, and send email (and optional Teams / Google Chat) alerts.

**Version:** 1.0.33 · **PyPI:** [inferyx-monitoring](https://pypi.org/project/inferyx-monitoring/) · **CLI:** `inferyx-monitoring`

This page is the full guide: **Install**, **Upgrade**, and **Whats new**. Everything is on this one page — no external doc links.

---

## Contents

- [Install](#install)
- [Upgrade](#upgrade)
- [Admin UI](#admin-ui)
- [Whats new](#whats-new)

---

## Install

First-time setup on a Linux server. Default paths use `/opt/pipeline-monitor` and user `inferyx`.

### Quick install (script)

Copy both scripts to the server (from git repo `server/`), or use them from pip after install:

```bash
sudo bash install_inferyx_monitoring.sh
```

With Admin UI:

```bash
sudo bash install_inferyx_monitoring.sh --admin --admin-ui --install-nginx
```

Pin a version:

```bash
sudo bash ./install_inferyx_monitoring.sh --pin 1.0.32
```

After `pip install`, scripts are also at (run with `bash` or `chmod +x` first):

```text
/opt/pipeline-monitor/.venv/share/inferyx-monitoring/scripts/
```

```bash
sudo bash /opt/pipeline-monitor/.venv/share/inferyx-monitoring/scripts/install_inferyx_monitoring.sh
```

The script creates the user, venv, installs from PyPI, runs `--init-config`, and enables systemd.

### Manual install (step by step)

#### Step 1 — Python and package

```bash
sudo apt update && sudo apt install -y python3 python3-venv python3-pip
id inferyx || sudo useradd --system --home-dir /opt/pipeline-monitor --shell /usr/sbin/nologin inferyx
sudo mkdir -p /opt/pipeline-monitor && sudo chown inferyx:inferyx /opt/pipeline-monitor
sudo -u inferyx python3 -m venv /opt/pipeline-monitor/.venv
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade pip inferyx-monitoring
```

#### Step 2 — Config files

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/inferyx-monitoring --init-config --work-dir /opt/pipeline-monitor
```

Creates `/opt/pipeline-monitor/.env` and `/opt/pipeline-monitor/batch_file.csv` if missing. On every start the monitor adds missing keys only — it does **not** overwrite SMTP/API secrets or batch rows.

#### Step 3 — Edit `.env`

```bash
sudo -u inferyx vi /opt/pipeline-monitor/.env
sudo chmod 600 /opt/pipeline-monitor/.env
```

**Required variables**

| Variable | Description |
|----------|-------------|
| `PIPELINE_SMTP_HOST`, `PIPELINE_SMTP_PORT`, `PIPELINE_SMTP_USERNAME`, `PIPELINE_SMTP_PASSWORD` | SMTP |
| `PIPELINE_FROM_NAME` | Sender name |
| `PIPELINE_MAIL_TO` | Alert recipients |
| `PIPELINE_API_BASE_URL` | API base URL (no `name=` in the URL) |
| `PIPELINE_API_TOKEN`, `PIPELINE_API_TOKEN_HEADER` | API auth |
| `PIPELINE_DEVOPS_EMAIL` | Recipient for `no_data` alerts |

**Recommended scheduling**

```env
PIPELINE_CHECK_MODE=schedule_windows
PIPELINE_CHECK_WINDOW_MINUTES=10
PIPELINE_SCHEDULE_GRACE_MINUTES=5
PIPELINE_API_FILTER_BY_SCHEDULE_DATE=false
```

#### Step 4 — Edit `batch_file.csv`

One batch per row. Last column: `Active` or `Suspended`. Use 24-hour times.

```csv
Name,Frequency,ExpectedStartTime,AvgExecutionTime,ExpectedDayOfMonth,Status
my_batch,Daily,9:00:00,"10 mins",,Active
```

#### Step 5 — Test run

Run as **one line** (do not split with `\`):

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/inferyx-monitoring --once --work-dir /opt/pipeline-monitor --env-file /opt/pipeline-monitor/.env --csv-file /opt/pipeline-monitor/batch_file.csv
```

#### Step 6 — systemd service

```bash
sudo tee /etc/systemd/system/inferyx-monitoring.service <<'EOF'
[Unit]
Description=Inferyx Pipeline Batch Monitor
After=network-online.target

[Service]
Type=simple
User=inferyx
Group=inferyx
WorkingDirectory=/opt/pipeline-monitor
ExecStart=/opt/pipeline-monitor/.venv/bin/inferyx-monitoring --work-dir /opt/pipeline-monitor --env-file /opt/pipeline-monitor/.env --csv-file /opt/pipeline-monitor/batch_file.csv
Restart=always
RestartSec=10
Environment=PYTHONUNBUFFERED=1

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now inferyx-monitoring.service
```

### Paths reference

| Item | Path |
|------|------|
| Install directory | `/opt/pipeline-monitor` |
| Config | `/opt/pipeline-monitor/.env` |
| Batch list | `/opt/pipeline-monitor/batch_file.csv` |
| Log | `/opt/pipeline-monitor/pipeline_script.log` |
| Python venv | `/opt/pipeline-monitor/.venv` |
| Service user | `inferyx` |
| systemd unit | `inferyx-monitoring.service` |

---

## Upgrade

**Latest version:** 1.0.33

### Quick upgrade (script)

```bash
sudo bash ./upgrade_inferyx_monitoring.sh
sudo bash ./upgrade_inferyx_monitoring.sh --mode admin
sudo bash ./upgrade_inferyx_monitoring.sh --mode full
sudo bash ./upgrade_inferyx_monitoring.sh --mode ui
sudo bash ./upgrade_inferyx_monitoring.sh --mode full --pin 1.0.32
```

From an existing install:

```bash
sudo bash /opt/pipeline-monitor/.venv/share/inferyx-monitoring/scripts/upgrade_inferyx_monitoring.sh --mode full
```

| Mode | What it does |
|------|----------------|
| `monitor` | `pip install --upgrade inferyx-monitoring` + restart monitor |
| `admin` | Upgrade `[admin]` + restart monitor and admin API |
| `full` | Upgrade `[admin]` + refresh UI + restart all + reload nginx |
| `ui` | Refresh Admin UI static files only |

Legacy CSV rename (from 1.0.14 or older): add `--legacy-migrate`.

### Manual upgrade

Pick the row that matches what you want to update, run the commands, then restart the service.

### Monitor only

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade inferyx-monitoring
sudo systemctl restart inferyx-monitoring.service
```

### Monitor + Admin API

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade 'inferyx-monitoring[admin]'
sudo systemctl restart inferyx-monitoring.service
sudo systemctl restart inferyx-monitoring-admin.service
```

### Full stack (monitor + admin API + admin UI)

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade 'inferyx-monitoring[admin]'
sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin
sudo systemctl restart inferyx-monitoring.service
sudo systemctl restart inferyx-monitoring-admin.service
sudo systemctl reload nginx
```

### Admin UI files only

Use this when the Python package is already up to date and you only need new static UI files.

**From pip (after upgrading the package):**

```bash
sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin
sudo systemctl reload nginx
```

**From deploy tarball** (copy `inferyx-monitoring-deploy-VERSION.tar.gz` to the server):

```bash
tar -xzf /tmp/inferyx-monitoring-deploy-VERSION.tar.gz -C /tmp
cd /tmp/inferyx-monitoring-deploy-VERSION
sudo ./scripts/install_admin_ui_deploy.sh
sudo systemctl reload nginx
```

### Upgrade from 1.0.14 or older

```bash
sudo -u inferyx mv /opt/pipeline-monitor/jfl_batch.csv /opt/pipeline-monitor/batch_file.csv
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade inferyx-monitoring
sudo systemctl restart inferyx-monitoring.service
```

Add to `.env` if missing:

```env
PIPELINE_CHECK_MODE=schedule_windows
PIPELINE_CHECK_WINDOW_MINUTES=10
```

Update systemd `ExecStart` to use `--csv-file /opt/pipeline-monitor/batch_file.csv` if it still points at `jfl_batch.csv`.

### Common problems after upgrade

| Problem | Fix |
|---------|-----|
| `unrecognized arguments:` | Run CLI commands as one line |
| No email | Check SMTP settings in `.env` |
| API errors | Remove `name=` from API URL; set `PIPELINE_API_FILTER_BY_SCHEDULE_DATE=false` |
| Service crash on 1.0.25 | Upgrade to 1.0.26 or newer |
| Alert crash `NameError: signature` | Upgrade to 1.0.21 or newer |
| `inferyx-monitoring-admin --help` starts server | Upgrade to 1.0.28 or newer |

---

## Admin UI

Optional web UI to edit `.env` and batch CSV. Requires OAuth. **Servers do not need Node.js or npm** — UI files are pre-built in the pip package.

### First-time Admin UI setup

**1. Install admin extra**

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install 'inferyx-monitoring[admin]'
```

**2. Auth policy**

```bash
sudo mkdir -p /etc/pipeline-monitor
sudo cp /opt/pipeline-monitor/.venv/share/inferyx-monitoring/config/auth.policy.example /etc/pipeline-monitor/auth.policy
sudo chmod 600 /etc/pipeline-monitor/auth.policy
sudo vi /etc/pipeline-monitor/auth.policy
```

Set OAuth providers and `ui.enabled: true`.

**3. Install UI static files**

```bash
sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin
```

**4. Admin API service**

```bash
sudo tee /etc/systemd/system/inferyx-monitoring-admin.service <<'EOF'
[Unit]
Description=Inferyx Pipeline Monitor Admin API
After=network-online.target

[Service]
Type=simple
User=inferyx
Group=inferyx
WorkingDirectory=/opt/pipeline-monitor
ExecStart=/opt/pipeline-monitor/.venv/bin/inferyx-monitoring-admin --host 127.0.0.1 --port 8090
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now inferyx-monitoring-admin.service
```

**5. Nginx**

Proxy `/api/` to `127.0.0.1:8090` and serve static files from `/var/www/pipeline-monitor-admin` at `/admin/`.

Example snippet (adjust server name and TLS):

```nginx
location /api/ {
    proxy_pass http://127.0.0.1:8090/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

location /admin/ {
    alias /var/www/pipeline-monitor-admin/;
    try_files $uri $uri/ /admin/index.html;
}
```

Copy the full example:

```bash
sudo cp /opt/pipeline-monitor/.venv/share/inferyx-monitoring/config/nginx-pipeline-monitor-admin.conf.example /etc/nginx/sites-available/pipeline-monitor-admin
sudo ln -sf /etc/nginx/sites-available/pipeline-monitor-admin /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```

### Admin CLI

Use the venv path (`sudo` does not see the venv automatically):

```bash
/opt/pipeline-monitor/.venv/bin/inferyx-monitoring-admin --help
/opt/pipeline-monitor/.venv/bin/inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin
```

---

## Whats new

### 1.0.33

- Fix `inferyx-monitoring-admin-install-ui` — was looking in wrong directory for bundled UI files

**Upgrade:** `pip install --upgrade 'inferyx-monitoring[admin]==1.0.33'` then run install-ui again.

### 1.0.32

- Server install script: `install_inferyx_monitoring.sh`
- Server upgrade script: `upgrade_inferyx_monitoring.sh` (modes: monitor, admin, full, ui)
- Scripts ship in pip package under `share/inferyx-monitoring/scripts/`

**Upgrade:** `sudo ./upgrade_inferyx_monitoring.sh --mode full` or see [Upgrade](#upgrade) above.

### 1.0.31

- Single-page PyPI documentation: install, upgrade, and whats new on this README only
- Pip package no longer bundles separate doc files

**Upgrade:** `pip install --upgrade inferyx-monitoring` then restart services (see [Upgrade](#upgrade) above).

### 1.0.30

- Pre-built Admin UI in pip wheel — no npm on production servers
- `inferyx-monitoring-admin-install-ui` command
- Deploy tarball `inferyx-monitoring-deploy-VERSION.tar.gz` for UI-only updates

**Upgrade:** `pip install --upgrade 'inferyx-monitoring[admin]'` then run `inferyx-monitoring-admin-install-ui` if you use the Admin UI.

### 1.0.29

- Release notes moved to this README (PyPI cannot host separate doc files)

### 1.0.28

- `inferyx-monitoring-admin --help` and `--version` work without starting the server
- Nginx and systemd examples for Admin UI

### 1.0.27

- Admin web UI for `.env` and batch CSV (OAuth, audit log)
- Safer login error messages; plain-text mail status (no icons)

### 1.0.26

- Hotfix for startup crash on 1.0.25 (`CHAT_CONFIG` import error) — upgrade immediately if on 1.0.25

### 1.0.25

- Microsoft Teams and Google Chat webhook alerts
- Test with: `inferyx-monitoring --test-chat-alerts --work-dir /opt/pipeline-monitor`

### 1.0.21

- Fix alert email crash (`NameError: signature`)

### 1.0.20

- Auto-migration on start: adds missing `.env` keys, migrates `jfl_batch.csv` to `batch_file.csv`

### 1.0.15

- Default batch file renamed to `batch_file.csv`
- `--init-config` command
- `PIPELINE_CHECK_MODE=schedule_windows` recommended
