Metadata-Version: 2.4
Name: pyxel-config-core
Version: 0.4.1
Summary: Validation and knowledge core for ESA Pyxel configs, callable as a library or an MCP server.
Author: Doby Baxter
License-Expression: MIT
Project-URL: Homepage, https://pyxel-config-lab-ede25c.gitlab.io
Project-URL: Repository, https://gitlab.com/dobybaxter127/pyxel-config-lab
Project-URL: Issues, https://gitlab.com/dobybaxter127/pyxel-config-lab/-/issues
Project-URL: Documentation, https://gitlab.com/dobybaxter127/pyxel-config-lab/-/blob/main/pyxel-config-core/README.md
Project-URL: Pyxel (upstream), https://esa.gitlab.io/pyxel/
Keywords: pyxel,esa,detector-simulation,mcp,model-context-protocol,json-schema,validation,yaml,astronomy
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jsonschema>=4.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: pydantic>=2.0
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == "mcp"
Provides-Extra: rag
Requires-Dist: qdrant-client>=1.7; extra == "rag"
Requires-Dist: fastembed>=0.3; extra == "rag"
Requires-Dist: numpy>=1.24; extra == "rag"
Provides-Extra: checks
Requires-Dist: pyxel-sim<4,>=3.0; extra == "checks"
Requires-Dist: pyyaml>=6.0; extra == "checks"
Requires-Dist: pytest>=7; extra == "checks"
Dynamic: license-file

# pyxel-config-core 🪐🔭

[![PyPI](https://img.shields.io/pypi/v/pyxel-config-core?color=06B6D4&label=PyPI)](https://pypi.org/project/pyxel-config-core/)
[![Python](https://img.shields.io/pypi/pyversions/pyxel-config-core?color=9333EA)](https://pypi.org/project/pyxel-config-core/)
[![License](https://img.shields.io/badge/license-MIT-F5C518)](https://gitlab.com/dobybaxter127/pyxel-config-lab/-/blob/main/pyxel-config-core/LICENSE)

Validation and knowledge core for **ESA Pyxel** configurations — callable as a
plain Python library *or* as an **MCP server** that any LLM assistant can use as
a tool.

It reuses the same source-of-truth artifacts as
[Pyxel Config Lab](https://pyxel-config-lab-ede25c.gitlab.io): the Pyxel JSON
schema, the model-parameter catalogue, and the tutorial library. The browser GUI
and this core stay in lockstep because they read the same bundled contract.

---

## What it exposes

| Capability | Function | MCP tool |
|---|---|---|
| Validate a config against the schema | `validate_config(config)` | `validate_pyxel_config` |
| Friendly diagnostics only | `explain_config(config)` | `explain_pyxel_config` |
| Auto-repair a config (rename / re-stage) | `autofix(config)` | `fix_pyxel_config` |
| List valid models (optionally per stage) | `list_models(stage=None)` | `list_pyxel_models` |
| Fetch a tutorial | `find_tutorial(group, model)` | `get_pyxel_tutorial` |
| Keyword-search tutorials | `search_tutorials(query)` | `search_pyxel_tutorials` |
| Semantic (vector) tutorial search | `semantic_search_with_fallback(query)` | `semantic_search_pyxel_tutorials` |
| Model parameters + defaults | `model_parameters(model)` | `get_pyxel_model_parameters` |

Every input and output is a Pydantic model, so results are structured and
schema-checked on both sides of the boundary.

`validate_config`, `explain_config`, and `autofix` all accept either a config
`dict` or a raw **YAML string**, so an assistant can pass a user's pasted
`.yaml` straight through without parsing it first.

---

## Install

```bash
pip install pyxel-config-core              # library only
pip install "pyxel-config-core[mcp]"       # + MCP server
pip install "pyxel-config-core[rag]"       # + semantic search (Qdrant + fastembed)
pip install "pyxel-config-core[mcp,rag]"   # everything
```

Working on the package itself? Clone the repo and install it editable
instead — `pip install -e ".[mcp,rag]"` from `pyxel-config-core/`.

Requires Python 3.10+.

---

## Library usage

```python
from pyxel_config_core import (
    validate_config, autofix, list_models, find_tutorial, model_parameters,
)

result = validate_config(open("my_config.yaml").read())
for d in result.diagnostics:
    print(d.severity, d.path, d.message)

# Discover what models exist before building a config
for m in list_models("charge_generation").models:
    print(m.func)

# Repair typo'd names and misplaced models in one call
fix = autofix(open("my_config.yaml").read())
for change in fix.changes:
    print(change.action, change.path, change.before, "->", change.after)

tut = find_tutorial("charge_generation", "dark_current")
params = model_parameters("load_image")
```

A HoloViz/Panel assistant (or any Python app) can `import` this directly — no
server, no network, no access to anything private.

---

## Semantic search (RAG)

The tutorial catalogue is split into ~850 heading-level passages, embedded with a
local **fastembed** model (ONNX, CPU, no API keys), and stored in an embedded
**Qdrant** index — a real vector database that runs in-process, with no server to
operate.

Build the index once (downloads the embedding model on first run):

```bash
pip install -e ".[rag]"
pyxel-config-index
```

Then query it:

```python
from pyxel_config_core import semantic_search

for m in semantic_search("how do I add dark current to the detector", limit=5):
    print(f"{m.score:.3f}  {m.group}/{m.model}  §{m.section}")
```

Once the index is built, `search_tutorials(query)` automatically uses it and
falls back to keyword search if the index or the RAG extra is absent. Point the
index elsewhere with `PYXEL_CONFIG_CORE_INDEX=/path/to/dir`.

Scale note: at ~850 passages a brute-force index would also work; Qdrant is used
because it is a standard vector store, scales cleanly, and here runs embedded —
so the retrieval design is production-shaped without a production dependency.

---

## MCP server usage

```bash
pyxel-config-mcp          # runs over stdio
```

Point any MCP-compatible client at that command. The eight tools above become
callable by the model, so an assistant can *check* a config against the real
schema instead of guessing, *repair* it, *discover* the valid models, and
*retrieve* the real tutorial passages to cite.

The server imports cleanly on both the 1.x (`FastMCP`) and 2.x (`MCPServer`)
line of the MCP SDK.

### Building and fixing configs

Two tools turn the server from a read-only checker into an active assistant:

- **`list_pyxel_models(stage=None)`** enumerates every valid model function,
  optionally filtered to one pipeline stage. It reads the same schema the
  validator does, so the list is always the real, current set — an assistant can
  *discover then build* instead of guessing a name and validating it. Each entry
  carries the full `func:` path ready to drop into a config.

- **`fix_pyxel_config(config)`** applies the func diagnostics automatically. It
  renames a typo'd model to its nearest valid match, moves a valid-but-misplaced
  model into its owning stage, and returns the corrected config alongside a
  structured change log (`changes`), anything it could not fix (`unresolved`),
  and the diagnostics that remain after the repair (`remaining_diagnostics`).
  It fixes `func` names only — it never invents required arguments, so a rename
  onto a model with required args will still surface those in
  `remaining_diagnostics`. The input is never mutated; a corrected copy is
  returned.

### Graceful semantic search

`semantic_search_pyxel_tutorials` returns a `TutorialSearchResult` with a `mode`
(`"semantic"` or `"keyword"`), a `degraded` flag, and the `matches`. If the
vector index has not been built or the `[rag]` extra is not installed, it does
not error — it falls back to keyword search and flags the result as degraded, so
a client always gets usable matches with a clear signal about which engine
answered.

---

## The shared contract

Bundled under `pyxel_config_core/data/`:

- `pyxel_schema.json` — the validation contract
- `modelParameters.json` — model parameters and defaults
- `tutorials/` — the tutorial catalogue

These are copied from the Config Lab repo. Keep them refreshed with the same
freshness check the GUI already uses (`check_schema_version.py`,
`buildTutorialManifest.js`) so both consumers track upstream Pyxel together.
Rebuild the vector index (`pyxel-config-index`) whenever the tutorials change.

---

## Func diagnostics

Model-function names in the `pipeline` are checked against the schema itself
(the single source of truth), not a separate list, so every stage is covered.
When a `func` is wrong, `validate_config` now:

- collapses the schema's downstream argument explosion (one bad func could
  otherwise surface hundreds of misleading "X is a required property" errors
  from the branches the validator tried) down to a single diagnostic;
- distinguishes a **typo** from a **valid model in the wrong stage**, and points
  the latter at the stage it belongs to;
- offers a **"did you mean"** suggestion (Levenshtein over the leaf name) and a
  short list of valid models for that stage, carried as structured
  `suggestion` and `valid_options` fields on the `Diagnostic`.

This is the Python port of the browser GUI's `modelDiscriminator.js`, so both
consumers reason about `func` values the same way.

## What validation covers

The core validates exactly what the schema encodes — no more, no less — because
the schema is the single contract shared with the GUI. That already includes a
fair amount of value checking, not just structure:

- **types** and **required arguments** per model;
- **enums** — a bad choice string is rejected against the allowed set
  (e.g. `single` / `exponential` / `binomial` / `sbx`);
- **numeric bounds** (`minimum` / `maximum` / `exclusiveMinimum`) and
  **array-length bounds** (`minItems` / `maxItems`) wherever the schema declares
  them;
- **func discriminators**, with the friendly typo / wrong-stage diagnostics
  described above.

## Deliberately out of scope

Only checks that live *above* the schema are not attempted here — by design, so
there is never a second source of truth to drift from the GUI:

- **Super-schema constraints.** Cross-field consistency (e.g. one argument that
  must stay `<=` another), physical plausibility, and any bound Pyxel enforces
  at *runtime* but that isn't written into the JSON schema. These pass here and
  are caught only when Pyxel actually runs. The fix, if ever wanted, is to
  encode the rule in the schema so both consumers gain it at once — not to add a
  separate checker.
