Metadata-Version: 2.4
Name: pydantic-ai-okf
Version: 0.1.0
Summary: Open Knowledge Format (OKF) plugin for Pydantic AI
Keywords: okf,open-knowledge-format,pydantic-ai,agents,knowledge,llm
Author: SB&O Inc
Author-email: SB&O Inc <contact@sboinc.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Dist: pydantic>=2.10.0
Requires-Dist: pydantic-ai-slim>=2.0,<3
Requires-Dist: pyyaml>=6.0
Requires-Dist: gitpython>=3.1.40 ; extra == 'git'
Requires-Python: >=3.12
Project-URL: Download, https://github.com/sbo-inc/pydantic-ai-okf/releases
Project-URL: Homepage, https://github.com/sbo-inc/pydantic-ai-okf
Project-URL: Issues, https://github.com/sbo-inc/pydantic-ai-okf/issues
Project-URL: Repository, https://github.com/sbo-inc/pydantic-ai-okf
Provides-Extra: git
Description-Content-Type: text/markdown

[![CI](https://github.com/sbo-inc/pydantic-ai-okf/actions/workflows/ci.yaml/badge.svg)](https://github.com/sbo-inc/pydantic-ai-okf/actions/workflows/ci.yaml)

# pydantic-ai-okf

[Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) plugin for [Pydantic AI](https://ai.pydantic.dev).

OKF represents knowledge as a *bundle*: a directory tree of markdown documents
("concepts") with YAML frontmatter, cross-linked with ordinary markdown links.
This package lets Pydantic AI agents consume OKF bundles through **progressive
disclosure** - an overview of each bundle is injected into the system prompt,
and the agent browses, searches, and reads individual concepts on demand
through tools.

- **`OKFToolset`** - a Pydantic AI toolset providing `list_concepts`,
  `read_concept`, and `search_concepts`, with a per-bundle overview injected
  into the system prompt.
- **`OKFCapability`** - the same integration via the Pydantic AI
  `capabilities=[...]` API, with deferred loading for declarative agent specs.
- **`Bundle`** - a standalone API for loading, traversing, searching, and
  conformance-checking OKF bundles (no agent or LLM required).
- **Permissive by design** - per OKF §9, malformed frontmatter, unknown types,
  and broken links never fail a load; problems are reported by
  `Bundle.validate()` instead.
- **Git bundles** - load bundles distributed as git repositories via the
  optional `git` extra.
- Fully typed, high test coverage, no LLM calls in tests.

## Installation

```bash
pip install pydantic-ai-okf
pip install "pydantic-ai-okf[git]"   # to load bundles from git repositories
# or: uv add pydantic-ai-okf
```

## Quick start

### Give an agent a bundle (toolset)

```python
from pydantic_ai import Agent
from pydantic_ai_okf import OKFToolset

agent = Agent(
    'openai:gpt-5.6',
    toolsets=[OKFToolset(bundles=['./knowledge'])],
)

result = agent.run_sync('Which table should I join to get customer revenue?')
print(result.output)
```

That is the whole integration. On every request the agent receives a short
system-prompt block explaining OKF plus each bundle's overview, and it is given
three tools to explore on demand (see [How agents consume a bundle](#how-agents-consume-a-bundle)).

### Give an agent a bundle (capability)

```python
from pydantic_ai import Agent
from pydantic_ai_okf import OKFCapability

agent = Agent('openai:gpt-5.6', capabilities=[OKFCapability(bundles=['./knowledge'])])
```

Set `defer_loading=True` (with a stable `id`) to hide the knowledge tools and
instructions behind the agent's `load_capability` tool until the model loads
them explicitly - useful when an agent has many capabilities and you don't want
every bundle overview in the prompt at once:

```python
agent = Agent(
    'openai:gpt-5.6',
    capabilities=[OKFCapability(id='okf', bundles=['./knowledge'], defer_loading=True)],
)
```

### Multiple bundles

Pass several; names must be unique (they default to the directory name). Use
`Bundle(path, name=...)` to disambiguate, and the model can target one with the
tools' optional `bundle=` argument.

```python
from pydantic_ai_okf import Bundle, OKFToolset

toolset = OKFToolset(bundles=[
    Bundle('./rfc9396', name='rfc9396'),
    Bundle('./internal-apis', name='apis'),
])
```

### Use a bundle without an agent

Everything the tools do is available directly:

```python
from pydantic_ai_okf import Bundle

bundle = Bundle('./knowledge')

bundle.okf_version              # '0.1' (from the bundle-root index.md, if present)
bundle.concept_ids              # ['datasets/sales', 'tables/orders', ...]

concept = bundle.get('tables/orders')   # also accepts '/tables/orders.md'
concept.type                    # 'BigQuery Table'   (the one required field)
concept.description             # one-line summary
concept.tags                    # ['sales', 'orders']
concept.extra                   # {producer-defined frontmatter keys}
concept.body                    # markdown after the frontmatter

for hit in bundle.search('revenue', tags=['sales']):
    print(hit.score, hit.concept.concept_id)

bundle.validate()               # [] for a conformant bundle, else ConformanceIssue list
```

### Load a bundle from git

Requires the `git` extra.

```python
from pydantic_ai_okf import Bundle

bundle = Bundle.from_git(
    'https://github.com/acme/knowledge.git',
    ref='v1.2.0',          # optional branch, tag, or commit
    subdirectory='okf',    # optional bundle root within the repository
)
```

## How agents consume a bundle

When you register an `OKFToolset` / `OKFCapability`, two things happen on each
run:

1. **Instructions are injected.** A system-prompt block explains what OKF is
   (concepts, IDs, cross-links) and how to explore, then lists each bundle's
   `name`, `okf_version`, `concept_count`, and **overview**. The overview is the
   bundle-root `index.md` body, or a synthesized listing if there is none - so a
   good root `index.md` is the single best thing an author can do to help agents
   navigate a bundle.
2. **Three tools are registered:**

   | Tool | What it does |
   |---|---|
   | `list_concepts(directory, bundle)` | List concepts and subdirectories in a directory. |
   | `read_concept(concept_id, bundle)` | Read a full concept document (frontmatter + body). |
   | `search_concepts(query, tags, bundle)` | Keyword/tag search over titles, descriptions, tags, types, IDs, and bodies. |

   The `bundle` argument is optional when a single bundle is configured. Unknown
   IDs, directories, or bundle names raise `ModelRetry` with close-match
   suggestions so the model can self-correct (see `max_retries`).

The intended flow is progressive disclosure: `search_concepts` /
`list_concepts` to find candidates, then `read_concept` only what's needed.

## Customizing the injected instructions

Not every model knows what OKF is, and you may want to add domain framing. There
are three levels of control.

**1. Replace the template** (`instruction_template`, on both `OKFToolset` and
`OKFCapability`). Your template **must** contain the `{bundles_list}`
placeholder, which is where the per-bundle overview is injected. Escape any
literal braces as `{{`/`}}`.

```python
OKFToolset(
    bundles=['./rfc9396'],
    instruction_template=(
        "You are an OAuth compliance assistant. Prefer normative text over "
        "examples and preserve MUST/SHOULD/MAY exactly.\n\n"
        "{bundles_list}\n\n"
        "Cite the concept IDs and RFC sections you used."
    ),
)
```

**2. Layer on agent-level instructions.** Pydantic AI combines the agent's own
`instructions` with the toolset's, so you can keep the OKF block and just add
guidance:

```python
Agent(
    'openai:gpt-5.6',
    toolsets=[OKFToolset(bundles=['./knowledge'])],
    instructions='You are a data catalog expert. Answer only from the bundle.',
)
```

**3. Full control** - subclass and override `build_instructions` (an `async`
method) to change how each bundle block is rendered, drop the overview, or add
navigation hints.

## Authoring / bundle structure (brief)

A bundle is just a directory of markdown files; the only hard requirement (OKF
§9) is that every non-reserved `.md` file has a YAML frontmatter block with a
non-empty `type`:

```markdown
---
type: BigQuery Table          # required
title: Customer Orders        # recommended
description: One row per order.
tags: [sales, orders]
resource: https://…           # optional canonical URI
# any additional producer-defined keys are preserved on Concept.extra
---

# Schema
… body markdown, cross-linking other concepts as [orders](/tables/orders.md) …
```

- `index.md` and `log.md` are **reserved** (directory listing and change log).
  `index.md` files carry no frontmatter - except the bundle-root `index.md`,
  which may declare `okf_version: "0.1"`.
- Loading is **permissive**: nothing above is enforced at load time. Call
  `Bundle.validate()` to get the list of conformance issues.

## Reference

### `OKFToolset` / `OKFCapability` options

| Option | Default | Description |
|---|---|---|
| `bundles` | required | Local directories (`str`/`Path`) and/or pre-loaded `Bundle` instances; names must be unique. |
| `instruction_template` | built-in | Custom system-prompt template; must contain `{bundles_list}`. |
| `exclude_tools` | `None` | Tool names to skip registering (`list_concepts`, `read_concept`, `search_concepts`). |
| `auto_reload` | `False` | Re-scan bundle directories before each agent run. |
| `max_retries` | `3` | Retry budget for `ModelRetry`-raising tool calls, so a model can act on close-match suggestions before failing. |
| `id` | `None` | Stable identifier (required when `defer_loading=True`). |
| `defer_loading` | `False` | *(capability only)* Hide tools/instructions behind `load_capability`. |

### `Bundle` API

| Member | Description |
|---|---|
| `Bundle(path, name=...)` | Load a bundle from a local directory. |
| `Bundle.from_git(url, ref=…, subdirectory=…)` | Clone a git repo and load it as a bundle (`git` extra). |
| `get(concept_id)` | Look up a concept; tolerates `/tables/orders.md` link spellings. Returns `None` if absent. |
| `concepts` / `concept_ids` | All concepts / IDs, ordered by ID. |
| `directories()` / `list_directory(dir)` | Traverse the hierarchy. |
| `index(dir)` / `log(dir)` | Reserved `index.md` / `log.md` content (OKF §6, §7). |
| `overview()` / `synthesize_index(dir)` | Progressive-disclosure listings (what the toolset injects). |
| `search(query, tags=…, limit=…)` | Ranked keyword search returning `SearchResult`s. |
| `validate()` | OKF §9 conformance issues found during the scan (empty if conformant). |
| `reload()` | Re-scan the directory from disk. |

### `Concept` fields

`type` (required), `title`, `description`, `resource`, `tags`, `timestamp`,
`extra` (producer-defined keys), `body`, `text`, `concept_id`, `path`,
`has_frontmatter`, and `display_title` (falls back to the filename).

## Development

```bash
make install   # sync the locked environment
make check     # ruff + mypy
make test      # unit tests
make coverage  # tests with the coverage gate
make build     # build sdist + wheel
```
