Metadata-Version: 2.4
Name: inferyx-monitoring
Version: 1.0.30
Summary: Monitor batch pipelines via API and email alerts — install and deploy on Linux servers from PyPI
Author-email: Inferyx DevOps <devops@inferyx.com>
License: Copyright (c) 2026 Inferyx. All rights reserved.
        
        Proprietary software. Unauthorized copying, distribution, or use is prohibited
        unless agreed in writing with Inferyx.
        
        For open-source release, replace this file with your chosen license (e.g. MIT, Apache-2.0)
        before publishing to public PyPI.
        
Keywords: batch,monitoring,pipeline,email,inferyx
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: System Administrators
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: python-dateutil>=2.8.2
Provides-Extra: admin
Requires-Dist: fastapi>=0.110.0; extra == "admin"
Requires-Dist: uvicorn[standard]>=0.27.0; extra == "admin"
Requires-Dist: pydantic>=2.0.0; extra == "admin"
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: check-manifest; extra == "dev"
Dynamic: license-file

# inferyx-monitoring — Server deployment

Monitor batch jobs from CSV, poll Inferyx API status, and send email (and optional Teams/Google Chat) alerts.

**Package:** `inferyx-monitoring` · **CLI:** `inferyx-monitoring` · **Version:** `1.0.30` · [PyPI](https://pypi.org/project/inferyx-monitoring/)

> **Reading on PyPI:** PyPI shows this page only — it does not host separate `docs/` files. Use the [table of contents](#table-of-contents) and [release notes](#release-notes) below (in-page links). After `pip install`, extra docs are also installed under `share/doc/inferyx-monitoring/`.

---

## Table of contents

- [Overview](#overview)
- [Paths](#paths)
- [1. Install](#1-install)
- [2. Create config](#2-create-config-files-first-time)
- [3. Configure `.env`](#3-configure-env)
- [4. Configure batch CSV](#4-configure-batch_filecsv)
- [5. Test](#5-test)
- [6. systemd service](#6-start-service-systemd)
- [7. Upgrade](#7-upgrade)
- [Admin UI (optional)](#admin-ui-optional)
- [Troubleshooting](#troubleshooting)
- [Release notes](#release-notes)

---

## Overview

**inferyx-monitoring** watches batch jobs in a CSV file, queries the Inferyx API, and sends alerts when problems are detected.

| Alert type | When it fires |
|------------|---------------|
| **failed** | Batch execution failed |
| **running** | Still running past expected end + grace |
| **missed** | Did not start within schedule window + grace |
| **no_data** | No API record (email to DevOps) |

Recommended check mode: `PIPELINE_CHECK_MODE=schedule_windows` — polls only at start/end windows, not all day.

---

## Paths

| Item | Path |
|------|------|
| Install directory | `/opt/pipeline-monitor` |
| Config | `/opt/pipeline-monitor/.env` |
| Batch list | `/opt/pipeline-monitor/batch_file.csv` |
| Log file | `/opt/pipeline-monitor/pipeline_script.log` |
| Python venv | `/opt/pipeline-monitor/.venv` |
| Service user | `inferyx` |
| systemd unit | `inferyx-monitoring.service` |

---

## 1. Install

```bash
sudo apt update && sudo apt install -y python3 python3-venv python3-pip
id inferyx || sudo useradd --system --home-dir /opt/pipeline-monitor --shell /usr/sbin/nologin inferyx
sudo mkdir -p /opt/pipeline-monitor && sudo chown inferyx:inferyx /opt/pipeline-monitor
sudo -u inferyx python3 -m venv /opt/pipeline-monitor/.venv
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade pip inferyx-monitoring
```

---

## 2. Create config files (first time)

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/inferyx-monitoring --init-config --work-dir /opt/pipeline-monitor
```

Creates `.env` and `batch_file.csv` if missing. Auto-migrates on every start — **never** overwrites SMTP/API secrets or batch rows.

---

## 3. Configure `.env`

```bash
sudo -u inferyx vi /opt/pipeline-monitor/.env && sudo chmod 600 /opt/pipeline-monitor/.env
```

### Required

| Variable | Description |
|----------|-------------|
| `PIPELINE_SMTP_HOST` / `PORT` / `USERNAME` / `PASSWORD` | SMTP settings |
| `PIPELINE_FROM_NAME` | Sender display name |
| `PIPELINE_MAIL_TO` | Alert recipients |
| `PIPELINE_API_BASE_URL` | API base path — **no** `name=` in URL |
| `PIPELINE_API_TOKEN` / `PIPELINE_API_TOKEN_HEADER` | API auth |
| `PIPELINE_DEVOPS_EMAIL` | `no_data` alerts |

### Recommended scheduling

```env
PIPELINE_CHECK_MODE=schedule_windows
PIPELINE_CHECK_WINDOW_MINUTES=10
PIPELINE_SCHEDULE_GRACE_MINUTES=5
PIPELINE_API_FILTER_BY_SCHEDULE_DATE=false
```

### Teams / Google Chat (optional, since 1.0.25)

```env
PIPELINE_TEAMS_ENABLED=true
PIPELINE_TEAMS_WEBHOOK_URL=https://...
PIPELINE_GCHAT_ENABLED=true
PIPELINE_GCHAT_WEBHOOK_URL=https://chat.googleapis.com/...
```

Test: `inferyx-monitoring --test-chat-alerts --work-dir /opt/pipeline-monitor`

Email templates: `PIPELINE_MAIL_GREETING`, `PIPELINE_MAIL_INTRO_*`, `PIPELINE_MAIL_SUMMARY_*`, etc. Omit `PIPELINE_MAIL_BODY_*` for built-in structured layout.

---

## 4. Configure `batch_file.csv`

One batch per line; last column `Active` or `Suspended`. Use 24-hour times.

```csv
Name,Frequency,ExpectedStartTime,AvgExecutionTime,ExpectedDayOfMonth,Status
my_batch,Daily,9:00:00,"10 mins",,Active
```

---

## 5. Test

Run as **one line** (no `\` continuations):

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/inferyx-monitoring --once --work-dir /opt/pipeline-monitor --env-file /opt/pipeline-monitor/.env --csv-file /opt/pipeline-monitor/batch_file.csv
```

---

## 6. Start service (systemd)

```bash
sudo tee /etc/systemd/system/inferyx-monitoring.service <<'EOF'
[Unit]
Description=Inferyx Pipeline Batch Monitor
After=network-online.target

[Service]
Type=simple
User=inferyx
Group=inferyx
WorkingDirectory=/opt/pipeline-monitor
ExecStart=/opt/pipeline-monitor/.venv/bin/inferyx-monitoring --work-dir /opt/pipeline-monitor --env-file /opt/pipeline-monitor/.env --csv-file /opt/pipeline-monitor/batch_file.csv
Restart=always
RestartSec=10
Environment=PYTHONUNBUFFERED=1

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload && sudo systemctl enable --now inferyx-monitoring.service
```

---

## 7. Upgrade

```bash
sudo -u inferyx /opt/pipeline-monitor/.venv/bin/pip install --upgrade inferyx-monitoring
sudo systemctl restart inferyx-monitoring.service
```

See [release notes](#release-notes) for version-specific steps. From **1.0.14 or older**, also migrate `jfl_batch.csv` → `batch_file.csv` ([v1.0.15](#v1015)).

---

## Admin UI (optional)

Requires: `pip install 'inferyx-monitoring[admin]'`

**Angular UI is built by the developer at release time** — production servers do **not** run `npm`.

### Admin CLI

```bash
inferyx-monitoring-admin --help
inferyx-monitoring-admin --version
inferyx-monitoring-admin                    # API on 127.0.0.1:8090
inferyx-monitoring-admin-install-ui       # copy pre-built UI to web root
```

### Server deploy (first time)

1. **PyPI:** `pip install 'inferyx-monitoring[admin]'`
2. **Auth policy:** `/etc/pipeline-monitor/auth.policy` (see `share/inferyx-monitoring/config/auth.policy.example`)
3. **UI static** — choose one:
   - **From pip:** `sudo inferyx-monitoring-admin-install-ui --target /var/www/pipeline-monitor-admin`
   - **From deploy tarball:** `sudo ./scripts/install_admin_ui_deploy.sh` (inside `inferyx-monitoring-deploy-VERSION/`)
4. **systemd** for admin API + **Nginx** (proxy `/api/`, serve `/admin/`)

### Upgrade options

| Upgrade | Command |
|---------|---------|
| Monitor only | `pip install --upgrade inferyx-monitoring` + restart monitor |
| Admin API | `pip install --upgrade 'inferyx-monitoring[admin]'` + restart admin service |
| Admin UI only | New deploy tarball **or** `sudo inferyx-monitoring-admin-install-ui` after pip upgrade |
| Full admin | pip `[admin]` upgrade + `inferyx-monitoring-admin-install-ui` + restart admin service |

### Developer release (build both artifacts)

```bash
./scripts/release.sh
# or: make release
# dist/inferyx_monitoring-VERSION-*.whl     → PyPI
# dist/inferyx-monitoring-deploy-VERSION.tar.gz → server (UI static + install script)
make pip-upload
```

See [DEPLOY.md](docs/DEPLOY.md) in repo or `share/doc/inferyx-monitoring/DEPLOY.md` after install.

---

## Troubleshooting

| Symptom | Fix |
|---------|-----|
| `unrecognized arguments:` | Run commands as one line |
| API errors | Remove `name=` from API URL; `PIPELINE_API_FILTER_BY_SCHEDULE_DATE=false` |
| No email | Check SMTP in `.env` |
| Service crash on 1.0.25 | Upgrade to [v1.0.26+](#v1026) |
| Alert `NameError: signature` | Upgrade to [v1.0.21+](#v1021) |
| `--help` starts admin server | Upgrade to [v1.0.28+](#v1028) |
| PyPI doc links 404 | Use anchors on this page — fixed in [v1.0.29](#v1029) |

---

## Release notes

| Version | Type | Summary |
|---------|------|---------|
| [1.0.30](#v1030) | Feature | Pre-built Admin UI deploy tarball; no npm on server |
| [1.0.29](#v1029) | Docs | PyPI README in-page release notes |
| [1.0.28](#v1028) | Bugfix | Admin CLI `--help`, Nginx docs |
| [1.0.27](#v1027) | Feature | Admin UI, security, mail status |
| [1.0.26](#v1026) | Hotfix | Startup crash (`CHAT_CONFIG`) |
| [1.0.25](#v1025) | Feature | Teams / Google Chat |
| [1.0.24](#v1024) | Docs | PyPI packaging |
| [1.0.23](#v1023) | Patch | Email reliability |
| [1.0.22](#v1022) | Patch | Email layout |
| [1.0.21](#v1021) | Bugfix | Alert email crash fix |
| [1.0.20](#v1020) | Feature | Auto config migration |
| [1.0.19](#v1019) | Feature | Structured alert emails |
| [1.0.18](#v1018) | Feature | Styled HTML emails |
| [1.0.17](#v1017) | Feature | Grace / alert-after in email |
| [1.0.16](#v1016) | Docs | Deployment documentation |
| [1.0.15](#v1015) | Feature | `batch_file.csv`, schedule windows |

---

<a id="v1030"></a>

### Version 1.0.30

**Pre-built Admin UI at release time.** Developers run `./scripts/release.sh` (npm + pip + deploy tarball). Servers install UI without Node.js:

- `sudo inferyx-monitoring-admin-install-ui` (from pip wheel), or
- `inferyx-monitoring-deploy-VERSION.tar.gz` + `install_admin_ui_deploy.sh`

**Upgrade:** see [Admin UI upgrade options](#admin-ui-optional) above.

---

<a id="v1029"></a>

### Version 1.0.29

**Docs fix for PyPI.** Release-note links like `docs/versions/1.0.28.md` do not work on pypi.org (404). All release notes are now **on this README page** with in-page anchors. Documentation is also bundled under `share/doc/inferyx-monitoring/` after install.

**Upgrade:** `pip install --upgrade inferyx-monitoring==1.0.29` + restart service.

---

<a id="v1028"></a>

### Version 1.0.28

**Admin CLI fix** — `inferyx-monitoring-admin --help` and `--version` work (no longer starts the server). Added `--host` / `--port` flags. Nginx + systemd production guide for Admin UI.

```bash
pip install --upgrade 'inferyx-monitoring[admin]==1.0.28'
inferyx-monitoring-admin --help
sudo systemctl restart inferyx-monitoring-admin.service
```

**Nginx:** proxy `/api/` → `127.0.0.1:8090`; serve Angular at `/admin/` with `--base-href /admin/`.

---

<a id="v1027"></a>

### Version 1.0.27

**Admin UI** — web interface for `.env` and batch CSV (OAuth, audit log, view/edit modes). Path fixes (`batch.csv` fallback). Login pages show generic errors only. Mail alert status uses plain text (no icons).

```bash
pip install --upgrade 'inferyx-monitoring[admin]==1.0.27'
```

Configure `/etc/pipeline-monitor/auth.policy`; enable `ui.enabled`.

---

<a id="v1026"></a>

### Version 1.0.26

**Hotfix** — fixes `ImportError: CHAT_CONFIG` startup crash in 1.0.25. Upgrade immediately if on 1.0.25.

```bash
pip install --upgrade inferyx-monitoring==1.0.26
sudo systemctl restart inferyx-monitoring.service
```

---

<a id="v1025"></a>

### Version 1.0.25

**Teams / Google Chat** webhooks for `failed`, `running`, `missed` alerts. `--test-chat-alerts` CLI. New keys: `PIPELINE_TEAMS_*`, `PIPELINE_GCHAT_*`, `PIPELINE_CHAT_GREETING`, `PIPELINE_CHAT_SIGNATURE`.

---

<a id="v1024"></a>

### Version 1.0.24

PyPI README alignment. Package excludes dev/docs/scripts folders from wheel.

---

<a id="v1023"></a>

### Version 1.0.23

Maintenance release — email send reliability (same fixes as 1.0.21 line).

---

<a id="v1022"></a>

### Version 1.0.22

Left-aligned email layout; alert send fixes.

---

<a id="v1021"></a>

### Version 1.0.21

Fixes `NameError: signature` when sending alerts. Left-aligned HTML emails. Safer `.env` migration (duplicate keys, glued comments).

---

<a id="v1020"></a>

### Version 1.0.20

**Auto migration** on start — adds missing `.env` keys only; retires legacy `PIPELINE_MAIL_BODY_*`; migrates `jfl_batch.csv` → `batch_file.csv`; adds `Status` column if missing.

---

<a id="v1019"></a>

### Version 1.0.19

**Structured alert emails** — Batch Details table, Alert Summary, Action Required, Current Status. Configurable via `PIPELINE_MAIL_INTRO_*`, `PIPELINE_MAIL_SUMMARY_*`, etc.

---

<a id="v1018"></a>

### Version 1.0.18

Styled HTML alert emails with tables and formatted status.

---

<a id="v1017"></a>

### Version 1.0.17

Email bodies include Expected Start, Grace Time, Alert After. New: `PIPELINE_MAIL_GREETING`, `{grace_time}`, `{alert_after_time}`.

---

<a id="v1016"></a>

### Version 1.0.16

Documentation release — full deployment guide. No code changes.

---

<a id="v1015"></a>

### Version 1.0.15

**`batch_file.csv`** default filename (replaces `jfl_batch.csv`). **`--init-config`**. **`PIPELINE_CHECK_MODE=schedule_windows`**. **`Status`** column.

**Upgrade from 1.0.14 or older:**

```bash
sudo -u inferyx mv /opt/pipeline-monitor/jfl_batch.csv /opt/pipeline-monitor/batch_file.csv
# Update systemd ExecStart --csv-file if needed
pip install --upgrade inferyx-monitoring
```

Add to `.env`: `PIPELINE_CHECK_MODE=schedule_windows`, `PIPELINE_CHECK_WINDOW_MINUTES=10`.
