Metadata-Version: 2.4
Name: celery-diagnostics
Version: 0.2.0
Summary: Diagnose missing, stuck, retried, and duplicated Celery tasks with a privacy-safe standalone observer.
Author: Celery Diagnostics
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://celerydiagnostics.com/
Project-URL: Documentation, https://celery-diagnostics.readthedocs.io/
Project-URL: Issues, https://github.com/celerydiagnostics/CeleryDiagnosticsObserver/issues
Project-URL: Source, https://github.com/celerydiagnostics/CeleryDiagnosticsObserver
Project-URL: Privacy, https://app.celerydiagnostics.com/dashboard/help/privacy/
Project-URL: Terms, https://app.celerydiagnostics.com/dashboard/help/terms/
Keywords: celery,diagnostics,observer,task-queue,redis
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Celery
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: celery>=5.3
Requires-Dist: cryptography>=42.0
Requires-Dist: redis>=5.0
Requires-Dist: requests>=2.31
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: pytest<9.0,>=8.0; extra == "dev"
Requires-Dist: ruff<1.0,>=0.6; extra == "dev"
Requires-Dist: twine<7.0,>=5.1; extra == "dev"
Dynamic: license-file

# Celery Diagnostics Observer

Standalone observer-first integration for Celery Diagnostics.

The observer runs beside a Celery system. It connects to broker/event sources,
samples safe telemetry, and sends sanitized evidence to Celery Diagnostics. It
does not install code into customer web or worker processes. An optional
scheduler adapter runs only inside Celery Beat when periodic-fire diagnosis is
required.

## What This Package Is

- A CLI package that provides `celery-diagnostics`.
- The primary customer integration path for Celery Diagnostics.
- A passive observer for Redis broker queue depth, Celery task/worker events,
  optional Celery control inspect snapshots, observer health, transport retry,
  and local sanitized spool.
- An executor for bounded, read-only diagnostic checks requested by Celery
  Diagnostics. Checks cover task presence, reservation, scheduling, worker
  capacity and presence, queue consumers, status-only JSON result records in
  Redis, and task-specific presence in configured Redis queues.
- An optional app-aware observer when run with `-A myproject.celery:app`; this
  loads the Celery app inside the observer process to explain routing,
  visibility timeout, `task_track_started`, and beat schedule coverage.
- An optional Celery Beat scheduler adapter that reports bounded schedule
  inventory, due decisions, and publish failures without task payloads.

## What This Package Is Not

- It is not producer, web-process, or worker instrumentation.
- It does not install producer or task instrumentation.
- It does not require changing task definitions.
- It does not require a custom Celery `Task` base.
- It never transmits or persists task args, kwargs, results, raw tracebacks,
  frame locals, or task message bodies.
- It must not claim worker topology or zero consumers from Redis queue depth
  alone.

## Install

After the first public release:

```bash
python -m pip install --upgrade celery-diagnostics
```

Local development from this directory:

```bash
python -m pip install --upgrade -e ".[dev]"
```

If the shell cannot find `celery-diagnostics`, activate the virtual environment
where it was installed or use:

```bash
python -m celery_diagnostics_observer --help
```

## Quick Start

```bash
CD_PROJECT_KEY=cd_xxx \
CELERY_BROKER_URL=redis://YOUR_REDIS_HOST:6379/0 \
celery-diagnostics observe \
  --queues default,emails \
  --ingest-url https://ingest.celerydiagnostics.com
```

`CD_PROJECT_KEY` is intentionally environment-only. The CLI does not support a
`--project-key` option so project keys are less likely to appear in shell
history or process listings.

## Commands

### `observe`

Run the long-lived observer process:

```bash
CD_PROJECT_KEY=cd_xxx \
CELERY_BROKER_URL=redis://localhost:6379/0 \
celery-diagnostics observe --queues default
```

Enhanced observer mode loads your Celery app in the observer process only:

```bash
CD_PROJECT_KEY=cd_xxx \
celery-diagnostics observe \
  --mode project-aware \
  -A myproject.celery:app
```

`observe` prints a startup coverage summary before the runtime loops start. In
dry-run mode, stdout stays machine-readable JSON lines.

Read-only diagnostic checks are enabled by default. Disable them explicitly
when the observer must remain passive:

```bash
celery-diagnostics observe --no-active-probes
```

For Redis message-presence checks, the observer scans only configured queues,
uses the default Kombu priority layout, and applies a bounded scan limit. It
parses message envelopes locally only to match the protocol-v2 task ID; message
bodies are immediately discarded and are never transmitted or persisted. A
missing task is reported only when all
scanned lists were stable and decodable; partial or malformed observations
remain inconclusive.

Result-backend status checks are advertised only for a Redis backend with the
JSON result serializer. A server-side Redis script returns the `status` field
alone, so the Observer does not fetch the task's result value. Other result
backends and serializers remain unsupported rather than silently loading a
private result record through `AsyncResult.state`.

### Periodic schedules (optional Celery Beat adapter)

The standalone Observer can follow a periodic task after publication, but it
cannot know that Celery Beat should have fired an entry and did not. Run Beat
with the package's scheduler wrapper when that distinction matters:

```bash
CD_PROJECT_KEY=cd_xxx \
CD_INGEST_URL=https://ingest.celerydiagnostics.com \
celery -A myproject.celery:app beat \
  --scheduler celery_diagnostics_observer.beat:ObserverPersistentScheduler
```

This remains Celery's `PersistentScheduler`; the wrapper adds sanitized
schedule snapshots and evidence for due and failed publish attempts. It does
not collect task args, kwargs, result values, broker URLs, or credentials.
Schedules above the snapshot cap are reported as an incomplete inventory, so
the Backend will not infer that omitted entries were deleted.

Without the adapter, ordinary task diagnosis continues to work and the
`Periodic schedules` page explicitly reports that Beat evidence is unavailable.

### `doctor`

Explain what Celery Diagnostics can and cannot currently know:

```bash
CELERY_BROKER_URL=redis://localhost:6379/0 \
celery-diagnostics doctor
```

For app-aware coverage:

```bash
celery-diagnostics doctor --mode project-aware -A myproject.celery:app
```

`doctor` does not require `CD_PROJECT_KEY`. It reports telemetry coverage,
safe claims, blocked claims, and next steps. It is a diagnostic coverage
explanation, not a fake health check.

### `status`

Show lightweight local integration status:

```bash
celery-diagnostics status
```

Use `doctor` when you need detailed telemetry coverage.

## Configuration

| Variable | Purpose |
| --- | --- |
| `CD_PROJECT_KEY` | Project key used by `observe` when sending telemetry. Required for non-dry-run `observe`. |
| `CELERY_BROKER_URL` | Celery broker URL. Redis is the current observer target. |
| `CD_QUEUES` | Comma-separated queue names to sample when `--queues` is not provided. |
| `CD_INGEST_URL` | Celery Diagnostics ingest base URL. Defaults to `http://127.0.0.1:8000`. |
| `CD_TELEMETRY_POLICY` | Identity visibility: `readable` or `local-only`. Defaults to `readable`. |
| `CD_IDENTITY_KEY` | Customer-managed identity key. Required only for `local-only`; never sent to the Backend. |
| `CD_OBSERVER_MODE` | `standalone` or `project-aware`. Defaults to `standalone`. |
| `CELERY_APP` | Celery app import path used by project-aware mode, equivalent to `-A`. |
| `CD_SAMPLE_INTERVAL` | Redis queue sample interval in seconds. |
| `CD_INSPECT_INTERVAL` | Celery control inspect interval in seconds. |
| `CD_BATCH_SIZE` | HTTP transport batch size. |
| `CD_FLUSH_INTERVAL` | HTTP transport flush interval in seconds. |
| `CD_SPOOL_PATH` | Optional sanitized JSONL local spool path. |
| `CD_LOG_LEVEL` | Python logging level. |
| `CD_ACTIVE_PROBES` | Enable bounded read-only diagnostic checks. Defaults to `1`. |
| `CD_BROKER_MESSAGE_SCAN_LIMIT` | Maximum Redis messages inspected by one task-presence check. Defaults to `10000`, bounded to `100000`. |
| `CD_BEAT_OBSERVER_ID` | Optional stable source label for the Beat adapter. Defaults to `beat@<hostname>`. |
| `CD_BEAT_SNAPSHOT_INTERVAL` | Beat schedule snapshot interval in seconds. Defaults to `30`, bounded to `5..3600`. |
| `CD_BEAT_SPOOL_PATH` | Optional sanitized JSONL spool used only by the Beat adapter. |

Every CLI option other than the project key can also be passed as an argument.
For example:

```bash
celery-diagnostics observe \
  --broker redis://localhost:6379/0 \
  --queues default \
  --policy readable
```

## Privacy Defaults

The observer sanitizes telemetry before it leaves the customer environment.

`readable` sends operational task, queue, routing, and worker identifiers.
`local-only` sends stable HMAC references plus an authenticated encrypted
identity capsule. The customer-managed identity key never leaves the Observer,
and both modes collect the same lifecycle evidence.

To identify a `local-only` run, execute this inside the customer environment:

```bash
CD_PROJECT_KEY=cf_xxx \
CD_IDENTITY_KEY='customer-managed-secret' \
celery-diagnostics resolve R-XXXXXXXXXXXX
```

Default behavior:

- no task args;
- no task kwargs;
- no task results;
- no task payload body;
- no raw tracebacks;
- no frame locals;
- project keys appear only in the Authorization header and are never written to
  event bodies or the local spool;
- broker and ingest URL credentials are redacted in CLI reports.
- non-loopback ingest endpoints must use HTTPS;
- local spool and replay files are written for the owner only (`0600`).

Redis queue sampling uses safe queue depth checks. Redis-only evidence can show
queue pressure and backlog symptoms, but it cannot prove that no worker is
consuming a queue.

Project-aware mode uses an allowlist of app configuration facts. It does not
dump arbitrary Celery config, task payloads, broker credentials, or exception
messages from failed app imports.

The external Observer cannot directly witness an exception that prevents a
producer from reaching the broker. In that case it reports the boundary of its
evidence instead of inferring a publish failure from absence alone. Progress
events are consumed when they already exist in the Celery event stream; this
package does not add task-side progress instrumentation.

The optional Beat adapter observes only the Beat-to-broker publication
boundary. It does not broaden access to task payloads or application data.

## Local Development

Install editable dependencies:

```bash
python -m pip install --upgrade -e ".[dev]"
```

Run tests:

```bash
python -m pytest tests -q
```

Run syntax checks:

```bash
python -m py_compile celery_diagnostics_observer/*.py
```

Build a wheel:

```bash
python -m build
```

## Repository Layout

```text
celery_diagnostics_observer/
  active_probes.py    # bounded on-demand read-only checks
  capabilities.py     # advertised active-check capabilities
  app_context.py      # allowlisted app-aware config extraction
  beat.py             # optional privacy-safe Celery Beat scheduler adapter
  cli.py              # celery-diagnostics command implementation
  config.py           # env and CLI config normalization
  coverage.py         # doctor/status/startup coverage model
  event_receiver.py   # Celery event stream receiver
  redis_sampler.py    # Redis queue depth sampler
  redis_message_probe.py # bounded task-id presence check for Redis
  inspect_sampler.py  # optional Celery control inspect sampler
  sanitizer.py        # privacy-safe event normalization
  transport.py        # HTTP transport and retry behavior
  spool.py            # sanitized local JSONL spool
tests/
  test_*.py           # observer unit tests
```

## Release Notes

The package name is `celery-diagnostics`; its code imports as
`celery_diagnostics_observer`.

Releases are built in GitHub Actions and published to PyPI with Trusted
Publishing. No long-lived PyPI API token is stored in GitHub.

Before creating a GitHub release, update `pyproject.toml`, `version.py`, and
`CHANGELOG.md` to the same version, then run:

```bash
python -m pytest tests -q
python -m py_compile celery_diagnostics_observer/*.py
python -m build
python -m twine check dist/*
```

Publish by creating a GitHub release tagged `vX.Y.Z`. The protected `pypi`
environment must approve the publish job. For the first release, configure a
pending PyPI Trusted Publisher for repository `celerydiagnostics/CeleryDiagnosticsObserver`,
workflow `release.yml`, environment `pypi`, and project `celery-diagnostics`.

See `CHANGELOG.md` for version notes.
