Metadata-Version: 2.5
Name: jev-lanes
Version: 0.1.0rc1
Summary: Confidence-based routing and escalation policies for Jev and LangGraph.
Project-URL: Homepage, https://github.com/EricMariano/jev-lanes
Project-URL: Documentation, https://github.com/EricMariano/jev-lanes#readme
Project-URL: Security, https://github.com/EricMariano/jev-lanes/blob/main/SECURITY.md
Project-URL: Repository, https://github.com/EricMariano/jev-lanes
Project-URL: Issues, https://github.com/EricMariano/jev-lanes/issues
Project-URL: Changelog, https://github.com/EricMariano/jev-lanes/blob/main/CHANGELOG.md
Author: Eric Mariano
License-Expression: MIT
License-File: LICENSE
Keywords: human-in-the-loop,jev,langgraph,routing,typesafe
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: langchain-core<2,>=1.6.2
Requires-Dist: langchain-typesafe<0.1,>=0.0.1a3
Requires-Dist: langchain<2,>=1.3.15
Requires-Dist: langgraph<2,>=1.2.12
Description-Content-Type: text/markdown

# jev-lanes

Confidence-based routing and escalation policies for Jev and LangGraph.

**v0.1 release candidate:** the repository implements confidence-band selection, the
serializable `RouteTrace`, the `Classifier`/`FallbackModel` protocols, offline fakes and
`JevRouter`, which wires the three lanes and a resumable human review into a LangGraph
`StateGraph`, plus default adapters for TypeSafe and any LangChain chat model (bring your
own keys). Offline examples and a five-minute notebook demonstrate the full flow.
The [37-case synthetic smoke battery](benchmarks/results/2026-10-20-battery.md)
shows integration behavior; it does not establish decision quality or calibrated
thresholds. A [Banking77 evaluation](benchmarks/results/banking77/report.md) of the
policy on public data found no benefit from the fallback model (see
[Evidence](#evidence)). `langchain-typesafe` itself is an alpha release. The package has not been
published to PyPI; install from this checkout.

## Why

Routing needs an explicit policy for uncertainty: when to accept a classifier's
decision, ask another model, or request human judgment. This project aims to make
that policy testable and auditable, including resuming a graph after human review.

The implemented flow is:

```text
Jev Choice -> confidence band
               auto  -> validated Jev route
               llm   -> fallback model -> validated route or human review
               human -> interrupt -> validated human decision -> route
```

```mermaid
flowchart LR
    A[Classify] --> B{Confidence band}
    B -->|auto| R[Validated route]
    B -->|llm| L[Fallback model]
    B -->|human| H[Human review interrupt]
    L -->|valid route| R
    L -->|abstains or unknown route| H
    H -->|resume with valid route| R
```

The fallback model's benefit must be evaluated; the one evaluation so far did not show one
(see [Evidence](#evidence)). A valid route name alone does not
establish a correct decision. Jev confidence is a distribution statistic, not an
accuracy guarantee; thresholds depend on the task and its consequences.

## Try the current policy locally

Requires Python 3.10+ and [uv](https://docs.astral.sh/uv/getting-started/installation/).
From this checkout:

```bash
uv sync --locked --dev
uv run python examples/policy.py
```

For an application environment, `python -m pip install .` installs the package from
the checkout. No provider keys are needed to import it or run the offline examples.

```python
from jev_lanes import Lanes

# Illustrative thresholds, not calibrated defaults.
lanes = Lanes(auto=0.85, llm=0.60)
assert lanes.lane_for(0.95) == "auto"
assert lanes.lane_for(0.70) == "llm"
assert lanes.lane_for(0.40) == "human"
```

## Try the router offline

`examples/router_offline.py` runs all three lanes and a human review resumed with a
checkpointer, using fakes instead of providers:

```bash
uv run python examples/router_offline.py
```

```python
from typing import Any, TypedDict

from langgraph.graph import END, START, StateGraph

from jev_lanes import Classification, JevRouter, Lanes
from jev_lanes.testing import FakeFallback, FakeJevClassifier


class State(TypedDict, total=False):
    messages: list[str]
    support_route: dict[str, Any]
    handled_by: str


router = JevRouter(
    name="support",
    routes={"orders": "Order status questions", "billing": "Invoices and payments"},
    lanes=Lanes(auto=0.85, llm=0.60),
    classifier=FakeJevClassifier(Classification("orders", 0.95)),
    fallback=FakeFallback(),
)
builder = StateGraph(State)
builder.add_node("intake", lambda state: {})
builder.add_node("orders", lambda state: {"handled_by": "orders"})
builder.add_node("billing", lambda state: {"handled_by": "billing"})
builder.add_edge(START, "intake")
router.attach(builder, after="intake")  # route names are the target node names
builder.add_edge("orders", END)
builder.add_edge("billing", END)
result = builder.compile().invoke({"messages": ["Where is my parcel?"]})
assert result["handled_by"] == "orders"
```

- Declare `router.state_key` (here `support_route`) in your graph state schema; it holds
  the serializable `RouteTrace`. Every invocation starts a new episode.
- By default the state's `messages` are sent to the providers; pass `extract=` to send
  something else or to redact. Case text is never stored in the trace or in the
  review request.
- The fallback model abstaining or naming an unknown route escalates to human review; an
  unknown answer is recorded in the trace as `<invalid route>`, not verbatim.
- Each router attached to a graph needs its own `name`.
  Provider failures raise; they are not converted into a low confidence.
- Human review needs a checkpointer and a stable `thread_id`. Resume with
  `Command(resume="<route>")`; an unknown route is rejected and can be sent again.
  Resuming does not classify again.
- Sync only; requires `langgraph>=1.2.12`.
- The trace replaces its previous episode in `<name>_route` (no list reducer).
  Checkpointers retain state snapshots according to their own retention policy, including
  the case text in `messages`. `InMemorySaver` only persists within the current process.

## Full demos

```bash
uv run --offline --no-sync python examples/atendimento/demo.py
uv run --offline --no-sync python examples/manual_routing.py
```

The [customer-service demo](examples/atendimento/README.md) handles orders, billing and
account questions. It demonstrates fallback abstention, an invalid human response and
successful resumption without reclassifying. Human responses are scripted.

The [manual example](examples/manual_routing.py) expands the edges that `attach()`
adds, using the same public node methods. Behavior comparisons cover all three lanes,
invalid review/retry, provider failures, mapped/shared destinations, `END` and a new
episode. This comparison validates graph wiring, not accuracy or a speed advantage.

Open [the five-minute notebook](examples/quickstart.ipynb) in a notebook editor using
the project interpreter, or execute it from the terminal:

```bash
uv sync --locked --dev --group notebook
uv run --offline --no-sync python examples/execute_notebook.py
```

The executed copy goes to `build/quickstart.executed.ipynb`; source cells stay clean.

## Use real providers (bring your own keys)

```python
from jev_lanes.adapters import JevClassifier, LangChainFallback

router = JevRouter(
    name="support",
    routes={"orders": "...", "billing": "...", "other": "Anything else."},
    lanes=Lanes(auto=0.85, llm=0.60),
    classifier=JevClassifier(),  # reads TYPESAFE_API_KEY
    fallback=LangChainFallback("openai:gpt-5-nano"),
)
```

`JevClassifier` asks TypeSafe a `Choice` whose criteria are your routes (add an `other`
route when they may not cover every input). `LangChainFallback` takes a `BaseChatModel`
or an `init_chat_model` string; install the provider package yourself (for example
`langchain-openai`) and set that provider's key. Keys are never stored or forwarded by
this package. The case text is sent to TypeSafe and, for medium-confidence cases, to the
fallback provider. `examples/byok.py` is a complete example; it calls real services and
is not run in CI.

The fallback model is asked for a route name or `ABSTAIN` in plain text, and the reply
is checked against your routes: anything else escalates to human review.

## Evidence

The policy was evaluated once on [Banking77](https://huggingface.co/datasets/PolyAI/banking77)
(English banking intents grouped into four routes; thresholds chosen on 231 calibration
cases, then applied once to 770 test cases; `gpt-5-nano` as fallback). Full method, tables
and limitations: [report](benchmarks/results/banking77/report.md).

- Jev's confidence is informative: AUROC 0.884 for correct vs wrong routes.
- The 95% automatic-accuracy target was not reached on calibration. At the chosen
  thresholds the three-lane policy automated 77.4% of cases at 90.6% accuracy; Jev plus
  human review (no fallback) automated 57.5% at 94.4%.
- **The fallback did not help.** In the llm lane Jev was right on 25 of 30 in-scope cases
  and the fallback on 23 (0 fixed, 2 broken). Below the auto threshold overall, it fixed 12
  and broke 16 (sign test p = 0.57). Treat the fallback as an unproven option: measure it
  on your own labelled data before relying on it, and prefer sending medium-confidence
  cases to people if reviewers have capacity.
- Out-of-scope messages were still routed automatically when Jev was confident.

One dataset, one run, one fallback model, English only: this is not an accuracy claim for
your domain. Public data are shared under CC BY 4.0; see the
[attribution](benchmarks/banking77/README.md#attribution).

## The lane policy

Thresholds must satisfy `0 <= llm < auto <= 1`. Equality with either threshold
selects the higher band. Invalid confidence raises an error; a provider failure
is not treated as a low-confidence decision. The package itself never calls external
services; only the providers you inject do.

## Development

```bash
uv run --offline --no-sync ruff check .
uv run --offline --no-sync ruff format --check .
uv run --offline --no-sync mypy
uv run --offline --no-sync pytest
uv build
```

Install dependencies with `uv sync --locked --dev` before running offline checks.
The CI workflow targets Python 3.10–3.14. Distribution builds require a cached
build backend or network access.

## Repository

- `src/jev_lanes/`: public policy primitives, with type annotations
  (`lanes`, `trace`, `protocols`, `router`, `adapters`, and `testing` fakes for offline tests).
- `tests/`: deterministic offline tests.
- `examples/`: policy, offline router, customer-service and manual wiring examples,
  a five-minute notebook, and a BYOK example that needs keys.
- `benchmarks/`: synthetic smoke results, the Banking77 evaluation, and the protocol for a
  real-data evaluation in your own domain.
- [Glossary](GLOSSARY.md): domain terminology.
- [Contributing](CONTRIBUTING.md): local workflow and evaluation expectations.
- [Changelog](CHANGELOG.md).

## Naming and related work

[`langgraph-jev-router`](https://pypi.org/project/langgraph-jev-router/) is an existing
third-party project. This is a separate project centered on an explicit three-band
escalation policy. `jev-lanes` returned HTTP 404 from PyPI's public project JSON
endpoint on 2026-10-19; that is not a reservation or publication guarantee.

## References

- [Official LangChain TypeSafe integration](https://docs.langchain.com/oss/python/integrations/providers/typesafe)
- [TypeSafe confidence guidance](https://docs.typesafe.ai/confidence)
- [LangGraph interrupts](https://docs.langchain.com/oss/python/langgraph/interrupts)

MIT licensed. Independent project; not affiliated with TypeSafe or LangChain.
