Metadata-Version: 2.4
Name: brainforge-inference-utils
Version: 0.0.1a1
Summary: Pluggable request/response pipelines for BrainForge LLM inference
Home-page: https://github.com/NeonGeckoCom/brainforge-inference-utils
Author: NeonGecko
Author-email: developers@neon.ai
License: AGPL-3.0-only
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: pydantic~=2.0
Requires-Dist: neon-data-models>=0.0.3a8,~=0.0.2
Requires-Dist: fastmcp~=2.3
Requires-Dist: requests
Provides-Extra: test
Requires-Dist: pytest~=8.0; extra == "test"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: summary

# BrainForge Inference Utilities

Pluggable pipelines for shaping BrainForge LLM requests and responses.

This package defines request/response orchestration that attaches to an
existing inference workflow. Concrete behavior (ambient context, RAG/MCP
injection, request history validation, citations, response sanitizing,
safety guardrails, ...) can be implemented as **steps** that plug into one of
two pipelines:

- **Request pipeline** — runs before generation and shapes the request:
  injecting informational context, RAG retrieval for host-selected
  providers, validating and truncating history, etc.
- **Response pipeline** — runs after generation and shapes the response:
  citation rewrite, sanitization, guardrails, etc.

Primary generation stays with the host, between the two pipelines. The host
keeps the `UserTransaction` from the request pipeline until the response is
ready. A pipeline step may call an inference endpoint to modify the transaction
(for example citation rewrite) without the host owning those requests.

Both pipelines operate on a `UserTransaction`, which carries the request,
response, and two per-side context dicts (`request_step_context` and
`response_step_context`). Requests and responses use the existing
`neon-data-models` models (`LLMRequest` / `LLMResponse`).

## Concepts

| Concept | Purpose |
| --- | --- |
| `UserTransaction` | The request/response pair plus `request_step_context` / `response_step_context` dicts. The host holds this object across generation. |
| `BrainForgeClient` | Builds pipelines from config and exposes `run_request_pipeline` / `run_response_pipeline` / `supported_request_context`. |
| `RequestStep` | Shapes `transaction.request` before generation. Declares supported `request_context` keys/values. |
| `ResponseStep` | Annotates/rewrites `transaction.response` after generation. |
| `RequestContextOption` | One `request_context` key a step reads, with optional allowed values for hosts to present. |
| `RequestPipeline` / `ResponsePipeline` | Run an ordered list of steps over a transaction. |
| `PipelineConfig` / `StepConfig` | Declarative, ordered step lists (see below). |

## Usage sketch

```python
client = BrainForgeClient(pipeline_config)

options = await client.supported_request_context()
# e.g. options["rag_providers"].values, options["conversational"].values

transaction = await client.run_request_pipeline(llm_request)
# host performs generation using transaction.request
transaction = await client.run_response_pipeline(transaction, llm_response)
```

Request steps write `transaction.request_step_context[step.id]`. Response steps write
`transaction.response_step_context[step.id]` and may *read* the request-step context. Those
namespaces are independent, so the same step id may exist on both sides
without colliding. Context is available across generation only because the
same `UserTransaction` is passed back in; this package does not store it
between calls.

## Design notes

- **Steps are independently defensive.** There is no dependency graph between
  steps. A step that needs something it does not find (e.g. a citation step with
  no retrieved sources) reports itself inapplicable via `applicable()` and is
  skipped with a logged reason, rather than failing the pipeline.
- **Ordering is explicit.** Step order is exactly the order declared in
  configuration; the pipeline does not reorder.
- **Steps may be repeated.** The same step id may appear more than once in a
  pipeline, including with different `config`. Each appearance is a separate
  instance and runs in declared order. The pipeline does not merge or
  de-duplicate entries.
- **Steps own their context.** Each side has its own dict keyed by step id.
  A repeated step on the *same* side shares that key; a step id on the request
  side does not clash with the same id on the response side. Each step is
  responsible for how it stores and reads its own data across one or more
  calls (for example appending to a list instead of overwriting it).
  Response steps that consume request-step data should read
  `transaction.request_step_context` and report inapplicable when required 
  context is absent.
- **Format contracts live together.** Steps that share a parse format
  (e.g. a citation formatter and the sanitizer that strips it) belong in the
  same module and should derive from a shared constant so they cannot drift.

## Step plugins

Installed packages advertise steps with setuptools entry points. Request and
response steps use different groups:

| Pipeline | Entry point group |
| --- | --- |
| Request | `brainforge.inference.request_steps` |
| Response | `brainforge.inference.response_steps` |

Both groups use the step id `author.step_name`:

- `author` is the plugin developer's slug (`acme`, `brainforge`).
- `step_name` names that one step (`guardrails`, `rag`).

Each part may contain letters, digits, hyphens, and underscores. The class
`id`, the config `id`, and the context-dict key are that same string.

The same `author.step_name` may be registered once in each group. Each
pipeline reads only its own group, so the request class and the response class
stay distinct. Two packages registering the same name in one group is an
error.

```python
entry_points={
    "brainforge.inference.request_steps": [
        "acme.guardrails=acme_guardrails:GuardrailRequestStep",
    ],
    "brainforge.inference.response_steps": [
        "acme.guardrails=acme_guardrails:GuardrailResponseStep",
    ],
}
```

```yaml
pipeline:
  request:
    - id: acme.guardrails
  response:
    - id: acme.guardrails
```

