# Guide for AI Coding Assistants — MCA SDK Instrumentation

**Purpose.** You (an AI coding assistant) are being asked to instrument a
data scientist's model with the MCA SDK. This guide tells you exactly what
to do for each of the four model categories the SDK supports, when to ask
the owner instead of guessing, and how to verify the work.

**Audience.** AI coding assistants. Data scientists trigger you with the
prompt at the bottom of this file (or one like it).

**Location.** Shipped inside the wheel. Resolve with:

    import mca_sdk, os
    guide = os.path.join(os.path.dirname(mca_sdk.__file__), "FOR_AI_ASSISTANTS.md")

---

## 0. Before you touch any code — required declaration

The data scientist MUST tell you (or have written in the repo, e.g. a
README, model card, or `mca.yaml`) two things:

1. **`model_type`** — one of `regression`, `time-series`, `classification`, `generative`, `agentic`.
2. **`model_category`** — one of `internal`, `vendor`.

You also need to infer (or ask):

3. **Deployment shape** — one of `vertex-fastapi`, `cloud-run-fastapi`,
   `script`, `notebook`, `cli-batch`.
4. **LLM access path** (if `generative` or `agentic`) — `litellm`,
   `vertex-gemini-direct`, `openai-direct`, `anthropic-direct`,
   `langchain`, `llamaindex`.

If any of these four are unstated AND not unambiguously derivable from the
code, **stop and ask the owner**. Do not guess. Past field experience
(stories 1-6 / 1-7 / 1-8 in this project) shows guessing produces wrong
metrics that look right on a dashboard until an audit. Ask one short
question that lists the options.

### Quick decision table — code signal -> pattern

| Code signal you see                                              | Pattern to apply                       | Section |
|------------------------------------------------------------------|-----------------------------------------|---------|
| `sklearn`, `xgboost`, `lightgbm` imported AND used at call site  | `@client.predict()` decorator           | §2.1    |
| `joblib.load(...)` / `pickle.load(...)` of an unknown estimator  | `client.trace()` + `record_prediction`  | §2.1    |
| `model.predict(batch)` over a DataFrame, batch >= 2              | Batch trace + `record_prediction(batch_size=N)` | §2.1 |
| `litellm.completion(...)` or `litellm.acompletion(...)`          | `MCALiteLLMCallback`                    | §2.2    |
| `from vertexai.generative_models import GenerativeModel`         | `with client.record_llm_generation(...)`| §2.3    |
| `from openai import OpenAI` / `AsyncOpenAI` direct calls         | `autolog()` (covers OpenAI)             | §2.4    |
| `from anthropic import Anthropic` direct calls                   | `autolog()` (covers Anthropic)          | §2.4    |
| `from langchain...` chains/agents                                | `MCALangChainCallback`                  | §2.5    |
| `from llama_index...` query/chat engines                         | `MCALlamaIndexCallback`                 | §2.5    |
| Multi-step loop: plan -> tool -> observe -> repeat               | Agentic: one trace, per-step `record_prediction` | §2.6 |
| External REST API you do not own, polled                         | `mca_sdk.integrations.bridge` (JSON->OTLP)| §2.7    |
| `requests.post(vendor_api, ...)` inside the prediction path      | `client.trace()` wrapper around the call| §2.7    |

If two signals match (e.g. agentic that also calls LiteLLM), apply the
**higher-level** pattern (agentic) and let the LiteLLM callback layer
underneath it — both can coexist.

---

## 1. Universal startup — pick one

### 1.0 Internal HTTP collector — set `allow_insecure_collector=True`

The fleet collector runs at `http://10.164.76.55:4318` (see
`example_deployed/shared/config.py`). The SDK enforces HTTPS on non-localhost
endpoints by default and will refuse to initialize against a bare `http://…`
URL. Pass `allow_insecure_collector=True` to `MCAClient(...)` whenever
`collector_endpoint` starts with `http://` and isn't `localhost` — every
`example_deployed/*` client does this. Omit the flag only when the endpoint
is `https://…` or `http://localhost:…`.

### 1.1 Vertex AI / Cloud Run FastAPI container (the production case)

Use `lifespan` so startup runs inside the event loop and `asyncio.to_thread`
wraps any blocking I/O. This is what every container in
`example_deployed/` does.

    import os
    from contextlib import asynccontextmanager
    import asyncio
    from fastapi import FastAPI
    from mca_sdk import MCAClient
    from mca_sdk.registry import RegistryClient

    mca_client = None
    registry_client = None

    @asynccontextmanager
    async def lifespan(app: FastAPI):
        global mca_client, registry_client

        model_id = os.environ["MODEL_ID"]
        registry_url = os.environ.get("MCA_REGISTRY_URL")
        collector_endpoint = os.environ.get(
            "MCA_OTEL_ENDPOINT",
            os.environ.get("MCA_COLLECTOR_ENDPOINT", "http://localhost:4318"),
        )

        # Registry is optional - fall back to env if it's unreachable.
        service_name = os.environ.get("MCA_SERVICE_NAME", "unset")
        team_name = os.environ.get("MCA_TEAM_NAME", "unset")
        if registry_url:
            registry_client = RegistryClient(
                url=registry_url,
                use_gcp_auth=True,                       # GCP only
                timeout=10.0,
                refresh_interval_secs=300,
                token_audience=os.environ.get("MCA_REGISTRY_TOKEN_AUDIENCE"),
            )
            try:
                cfg = await asyncio.to_thread(
                    registry_client.fetch_model_config, model_id
                )
                service_name = cfg.service_name
                team_name = cfg.team_name
            except Exception as e:
                # Graceful degradation: SDK requirement.
                app.logger.warning(f"Registry unavailable, using env: {e}")

        mca_client = await asyncio.to_thread(
            MCAClient,
            service_name=service_name,
            model_id=model_id,
            model_type="classification",        # set to the declared type
            model_category="internal",      # set to the declared category
            team_name=team_name,
            llm_provider="google",          # or "openai" | "anthropic" | "litellm"
            llm_model="gemini-2.0-flash-001",  # whatever the model resolves to
            collector_endpoint=collector_endpoint,
            allow_insecure_collector=True,  # required if collector is http://…
            registry_url=registry_url,
            use_gcp_auth=True,
            refresh_interval_secs=300,
            capture_input_data=True,
            capture_output_data=True,
        )

`llm_provider` / `llm_model` dimension GenAI metrics on the collector. Set
them even when your agent talks to the LLM indirectly (LiteLLM proxy,
LangChain wrapper) — otherwise the dashboard groups everything as
`unknown/unknown` and per-model latency/cost splits are gone.

        yield

        if mca_client:
            mca_client.shutdown(timeout_millis=5000)
        if registry_client:
            registry_client.close()

    app = FastAPI(lifespan=lifespan)

### 1.2 Script / notebook / CLI batch

    from mca_sdk import MCAClient
    import atexit

    client = MCAClient(
        service_name="readmission-predictor",
        model_id="mdl-001",
        model_type="classification",
        model_category="internal",
        team_name="clinical-ai",
    )
    atexit.register(client.shutdown)

---

## 2. Pattern catalogue

### 2.1 Predictive ML (sklearn / xgboost / lightgbm / joblib-loaded)

**Single instance**

    @mca_client.predict()
    def score(features: dict) -> dict:
        return {"risk": float(model.predict_proba([features])[0][1])}

**Batch** (this is the production shape — copy from
`example_deployed/internal-model-vertex/main.py`):

    with mca_client.trace("prediction_batch") as span:
        df = pd.DataFrame(batch_data, columns=feature_names)
        preds = model.predict(scaler.transform(df))
        probs = model.predict_proba(scaler.transform(df))

        batch_id = str(uuid.uuid4())
        mca_client.record_prediction(
            latency=time.perf_counter() - start,
            prediction_id=f"batch-{batch_id}",
            batch_size=len(batch_data),
            span_attributes={"ml.batch.id": batch_id, "ml.batch.size": len(batch_data)},
        )
        # Per-instance: LOGS only. Do NOT call record_prediction per row.
        for i, pid in enumerate(prediction_ids):
            mca_client._logger.info(
                "Prediction completed",
                extra={
                    "prediction_id": pid,
                    "risk_score": float(probs[i][1]),
                    "batch_id": batch_id,
                    "association_id": payload.instances[i].association_id,
                },
            )
    mca_client.flush(timeout_millis=2000)

**Why no per-row metric increment:** the metric stream is dimensioned by
labels, not by row count. Calling `record_prediction` once per row inflates
labels and degrades the histogram. Per-row data belongs in logs.

### 2.2 GenAI via LiteLLM

Attach the callback once at startup; do not wrap individual calls.

    import litellm
    from mca_sdk.integrations import MCALiteLLMCallback

    callback = MCALiteLLMCallback(mca_client)
    litellm.callbacks = [callback]

`litellm.completion` / `acompletion` are now fully instrumented (model,
provider, prompt/completion tokens, cost, latency, error class).

### 2.3 GenAI via Vertex AI Gemini direct (no LiteLLM)

When the container imports `vertexai.generative_models` and bypasses
LiteLLM (common on Vertex prediction containers — internal DNS forbids
LiteLLM proxy), use `record_llm_generation`. Pattern is
`example_deployed/internal-genai-vertex/main.py:209-237`.

    from vertexai.generative_models import GenerativeModel, Content, Part

    with mca_client.record_llm_generation(
        model="gemini-2.0-flash-001",
        provider="google",
        operation="chat",
        prompt=messages,
        temperature=0.2,
        max_tokens=512,
    ) as gen:
        resp = model.generate_content(
            [Content(role="user", parts=[Part.from_text(user_text)])],
            generation_config={"temperature": 0.2, "max_output_tokens": 512},
        )
        usage = resp.usage_metadata
        gen.set_completion(resp.text)
        gen.set_usage(
            prompt_tokens=int(usage.prompt_token_count or 0),
            completion_tokens=int(usage.candidates_token_count or 0),
            total_tokens=int(usage.total_token_count or 0),
            response_model="gemini-2.0-flash-001",
        )
        gen.set_cost(estimated_usd)   # owner supplies pricing

### 2.4 Direct OpenAI / Anthropic SDK

If the code uses `openai`, `AsyncOpenAI`, or `anthropic` SDK directly, do
not hand-wrap. Call `autolog()` once and let the patchers handle it.

    from mca_sdk import autolog
    autolog()              # idempotent; patches openai, anthropic, sklearn, xgboost, lightgbm
    # or scope it:
    autolog(providers=["openai", "anthropic"])

### 2.5 LangChain / LlamaIndex

    from mca_sdk.integrations import MCALangChainCallback, MCALlamaIndexCallback

    # LangChain
    chain.invoke(inputs, config={"callbacks": [MCALangChainCallback(mca_client)]})

    # LlamaIndex
    from llama_index.core import Settings
    Settings.callback_manager.add_handler(MCALlamaIndexCallback(mca_client))

**Build the callback in its own try-block.** `MCALangChainCallback` imports
`langchain-core` at construction time. If that extra isn't installed, the
constructor raises — and if you built the client and the callback in the
same `try`, the exception unwinds the whole init, losing the agentic client
that would have worked on its own. Split them:

    client = MCAClient(...)                     # required
    try:
        from mca_sdk.integrations import MCALangChainCallback
        callbacks = [MCALangChainCallback(client)]
    except Exception:
        callbacks = []                          # LLM-call telemetry off;
                                                # agent traces (§2.6) still flow

### 2.6 Agentic (multi-step reasoning)

One outer trace per user request; one `record_prediction` per agent
**step**, not per LLM call. The LiteLLM callback (or `autolog`) handles LLM
metrics underneath. Pattern is
`example_deployed/internal-agentic-vertex/agent_instrumented.py`.

    with mca_client.trace("agent.run", attributes={"agent.goal": goal}) as span:
        for step_idx, step in enumerate(plan):
            with mca_client.trace(f"agent.step.{step.tool}") as step_span:
                step_span.set_attribute("agent.step.index", step_idx)
                step_span.set_attribute("agent.step.tool", step.tool)
                t0 = time.perf_counter()
                result = run_tool(step.tool, step.args)   # LLM calls inside
                mca_client.record_prediction(
                    latency=time.perf_counter() - t0,
                    prediction_id=f"{run_id}.{step_idx}",
                    span_attributes={
                        "agent.step.tool": step.tool,
                        "agent.step.success": bool(result.ok),
                    },
                )

#### 2.6.1 Agentic on LangGraph (`StateGraph.astream`)

When the agent is a LangGraph `StateGraph` (increasingly the default),
you don't have a hand-rolled `for step in plan` loop — you have
`app.astream(state, config)` yielding `{node_name: node_state}` events.
Wrap the streaming loop instead:

    with mca_client.trace("agent.run", attributes={
        "agent.goal": "deploy_application",
        "agent.request_id": request_id,     # also your association_id
    }):
        last_tick = time.perf_counter()
        prev_error_count = 0
        step_idx = 0
        async for event in app.astream(initial_state, config):
            for node_name, node_state in event.items():
                if not isinstance(node_state, dict):
                    continue
                now = time.perf_counter()
                err_n = len(node_state.get("errors", []) or [])
                mca_client.record_prediction(
                    latency=now - last_tick,          # per-step wall time
                    prediction_id=f"{request_id}.{step_idx}",
                    span_attributes={
                        "agent.step.index": step_idx,
                        "agent.step.node": node_name,
                        "agent.step.success": err_n == prev_error_count,
                    },
                )
                prev_error_count = err_n
                last_tick = now
                step_idx += 1
        mca_client.flush(timeout_millis=2000)

Notes:

- Use the `thread_id` you already pass to LangGraph as your
  `association_id` — it's the natural join key for downstream outcomes,
  don't invent a second one.
- Measure per-step latency between yields. LangGraph doesn't expose
  per-node start times in the event stream; `now - last_tick` is the
  right approximation.
- Derive per-step success by comparing `len(errors)` before and after.
  The `add_error()` helpers in most projects append without decrementing,
  so growth ⇒ that node failed.
- The LangChain callback (§2.5) still attaches to your `ChatLiteLLM` /
  `ChatOpenAI` instances underneath and captures per-LLM-call
  telemetry — the agent trace is one layer up, not a replacement.

### 2.7 Vendor model (you do not own the inference code)

Two sub-cases:

**In-process wrapper** (your code calls `requests.post(vendor_api, ...)`):

    with mca_client.trace("vendor.predict", attributes={"vendor.name": "acme"}) as span:
        t0 = time.perf_counter()
        resp = requests.post(VENDOR_URL, json=payload, timeout=10)
        resp.raise_for_status()
        out = resp.json()
        mca_client.record_prediction(
            latency=time.perf_counter() - t0,
            prediction_id=out.get("id"),
            span_attributes={"vendor.http.status": resp.status_code},
        )

**Out-of-process bridge** (vendor exposes a metrics JSON you poll):

    from mca_sdk.integrations.bridge import VendorBridge
    VendorBridge(
        mca_client=mca_client,
        source_url="https://vendor/metrics.json",
        poll_interval_seconds=30,
    ).run_forever()

Use `model_category="vendor"` on the `MCAClient` in both sub-cases.

---

## 3. Hygiene rules — non-negotiable

1. **PHI is the application's responsibility.** The SDK does not auto-mask.
   Strip names, MRNs, DOBs, addresses, free-text notes BEFORE passing
   anything to `record_prediction`, `record_llm_generation`, trace
   attributes, or log `extra` fields. If the owner wants belt-and-braces
   filtering, set `MCA_ENABLE_PHI_FILTER=1` — it is opt-in.

   **Free-form user input is a landmine.** If your model's input is a
   natural-language field (chat prompt, ticket description, deploy
   request, search query, free-text note), do **not** set
   `capture_input_data=True` blindly — it ships the raw text into span
   attributes where a downstream viewer with dashboard access will read
   it. Default to `False`; only enable after you've written an
   application-side scrubber, or after confirming the field truly can't
   hold PHI. The same warning applies to `capture_output_data=True` when
   the model returns generated text.
2. **`association_id` is the join key** between prediction and outcome.
   Forward it from the request payload into per-row logs (see §2.1) so
   ground-truth labels can be joined later. Never use PHI as the
   association key — hash it.
3. **One metric per batch, logs per row.** Inflating metric cardinality
   breaks dashboards.
4. **Flush once per request, not per row.** `mca_client.flush()` is a
   network operation.
5. **HTTPS for all non-localhost endpoints.** The SDK enforces this; do
   not work around it.
6. **Do not log tokens, API keys, or registry secrets.** Ever.

---

## 4. MLOps Pipelines & Dependencies

To ensure telemetry successfully reaches the collector in production, you MUST instrument the build and deployment environment alongside the Python code. Do not skip this step.

### 4.1 Dependency Management
Search the project tree for `requirements.txt` or `pyproject.toml` files located within the serving or inference directories (e.g., `serve_image/requirements.txt`).
Append `mca_sdk` to these files. If the repository has separate training and serving dependencies, ensure it is added to the serving dependencies at a minimum.

### 4.2 Environment Orchestration
The production container needs the telemetry environment variables injected at deployment time.
Search the project for Vertex AI pipeline definitions (e.g., `pipeline.py`, `pipeline.json`), deployment shell scripts (e.g., `bootstrap.sh`), or environment configuration files (e.g., `config.env`).
Ensure that `MCA_OTEL_ENDPOINT`, `MCA_SERVICE_NAME`, `MCA_MODEL_ID`, and `MCA_TEAM_NAME` are explicitly passed into the serving container's configuration (e.g., via `serving_container_environment_variables` in Vertex AI's `ModelDeployOp` or equivalent).

---

## 5. Environment variables (canonical names)

| Var                                | Required? | Default                  | Notes |
|------------------------------------|-----------|--------------------------|-------|
| `MCA_SERVICE_NAME`                 | yes       | —                        | Or fetched from Registry |
| `MCA_MODEL_ID`                     | yes       | —                        | Stable identifier |
| `MCA_TEAM_NAME`                    | yes       | —                        | Or fetched from Registry |
| `MCA_OTEL_ENDPOINT`                | no        | `http://localhost:4318`  | Canonical name |
| `MCA_COLLECTOR_ENDPOINT`           | no        | (read after `MCA_OTEL_ENDPOINT`) | Vertex-deployment override; kept for backwards compat |
| `MCA_OTEL_PROTOCOL`                | no        | `http`                   | `http` or `grpc` |
| `MCA_REGISTRY_URL`                 | no        | —                        | Omit for env-only config |
| `MCA_REGISTRY_TOKEN`               | no        | —                        | Not used with GCP auth |
| `MCA_REGISTRY_TOKEN_AUDIENCE`      | no        | —                        | GCP ID-token audience |
| `MCA_REGISTRY_REFRESH_INTERVAL`    | no        | `600`                    | Seconds |
| `MCA_ENABLE_PHI_FILTER`            | no        | `0`                      | Opt-in PHI scrubbing |
| `MCA_DEBUG_MODE`                   | no        | `false`                  | Verbose logs |
| `MCA_ENV`                          | no        | `production`             | Fleet mock-switch. `dev` / `development` / `mock` / `local` disable real telemetry (init a no-op client); anything else means production. Mirrors `example_deployed/*/main.py`. Also honor `MCA_DISABLED=true` for the same effect. |
| `MCA_LLM_PROVIDER`                 | no        | —                        | GenAI metric dimension. Set to `google` / `openai` / `anthropic` / `litellm` etc. Pass into `MCAClient(llm_provider=...)`. |
| `MCA_LLM_MODEL`                    | no        | —                        | GenAI metric dimension. Pass into `MCAClient(llm_model=...)`. Without both, dashboards group everything as `unknown/unknown`. |

Earlier versions of this guide listed `MCA_COLLECTOR_ENDPOINT` as the
primary endpoint variable. That is wrong. Use `MCA_OTEL_ENDPOINT` in new
code. The deployed examples keep both for compatibility with existing
manifests.

---

## 6. Verification

After instrumenting, do all six:

1. **Import test** — `python -c "from mca_sdk import MCAClient; print('ok')"`.
2. **Dependencies check** — Confirm `mca_sdk` is added to the serving `requirements.txt` or `pyproject.toml`.
3. **Pipeline config check** — Confirm pipeline deployment files (e.g., `pipeline.py`) explicitly inject `MCA_OTEL_ENDPOINT` and registry vars into the serving container.
4. **Local collector** —
       docker-compose up -d otel-collector
       python <their_entrypoint>.py        # or hit the FastAPI endpoint
       docker logs -f otel-collector       # expect "exporting metrics"
5. **Unit smoke** — call the instrumented function once with a tiny
   synthetic input, confirm no exception, confirm collector received a
   span with `service.name`, `model.id`, `model.type`, `model.category`.
6. **Owner sign-off question** — confirm with the data scientist that the
   `model_type` and `model_category` you set match what they declared.

---

## 7. When to refuse / escalate

Stop and ask the owner if any of these are true:

- The four declarations in §0 are missing AND code signals are
  contradictory.
- The repo contains raw PHI in test fixtures (`patient_name`, `MRN`,
  DOB, free-text encounter notes). Refuse to commit instrumentation until
  the fixtures are sanitized.
- The owner wants you to bypass HTTPS, log tokens, or disable validation.
- The owner asks you to auto-mask PHI inside the SDK. That is an
  application responsibility; refuse the scope creep and offer to add an
  application-side scrubber instead.

---

## 8. One-page summary you can paste back to the data scientist

1. Confirm `model_type`, `model_category`, deployment shape, LLM access path.
2. Pick the matching section above (§2.1 – §2.7).
3. Copy the universal startup block (§1.1 for Vertex/Cloud Run, §1.2 for scripts).
4. Apply the pattern. Strip PHI. Use `association_id` for joins.
5. Update MLOps configuration (§4) by injecting `MCA_*` environment variables in pipeline scripts and adding `mca_sdk` to `requirements.txt`.
6. Verify with §6.

That is the entire workflow.

---

## 9. Prompt templates for data scientists

Paste one of the following into your AI coding assistant to start an
instrumentation session.

### Variant A — Short, owner-stated (recommended)

> Please instrument my model with the MCA SDK.
>
> - **model_type:** `<regression | time-series | classification | generative | agentic>`
> - **model_category:** `<internal | vendor>`
> - **Deployment shape:** `<vertex-fastapi | cloud-run-fastapi | script | notebook | cli-batch>`
> - **LLM access path** (only if generative or agentic):
>   `<litellm | vertex-gemini-direct | openai-direct | anthropic-direct | langchain | llamaindex>`
> - **Model ID / service name / team:** `<mdl-..., ..., ...>`
>
> Read `mca_sdk/FOR_AI_ASSISTANTS.md` inside the installed package, follow
> the section that matches the declarations above, and apply that pattern
> to my code. If any of my declarations conflict with what you see in the
> code, **stop and ask me** rather than guessing. Do not auto-mask PHI;
> warn me about PHI fields you find and let me sanitize them.

### Variant B — If the owner refuses to declare (fallback)

> Please instrument my model with the MCA SDK by reading
> `mca_sdk/FOR_AI_ASSISTANTS.md` inside the installed package.
>
> I have not declared the model type. Before writing any instrumentation
> code, do the following in order:
>
> 1. Classify the model by reading my code (use the decision table in §0
>    of the guide).
> 2. State your classification back to me in one sentence with the
>    evidence (which imports, which call sites).
> 3. **Wait for my confirmation.** Do not proceed without it.
> 4. After I confirm, apply the matching pattern from §2 of the guide.
>
> If two patterns match equally well, present both and ask me to pick.
