Metadata-Version: 2.4
Name: isynth-ai-data-config
Version: 0.8.0
Summary: Generate iSynth data_config test-data definitions from a plain-language request, using an LLM that reads the environment's own metadata.
Author-email: Markus Herrmann <markus.herrmann@itopia.ch>
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://isynth.io
Keywords: isynth,mcp,llm,data-config,test-data,synthetic-data
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.13
Description-Content-Type: text/markdown
Requires-Dist: litellm>=1.0
Requires-Dist: fastmcp>=4.0
Requires-Dist: openpyxl>=3.0
Requires-Dist: python-box>=7.0
Requires-Dist: python-dotenv>=1.0
Requires-Dist: httpx>=0.28
Requires-Dist: PyYAML>=6.0
Requires-Dist: starlette>=1.0

# isynth-ai-data-config

Generate iSynth `data_config` files from a request written in plain language, instead
of hand-writing YAML against a data model you have to memorise first.

You describe the test data you want — *"a private client with one account, no
portfolio, no phone number"* — and you get back a ready `data_config` that iSynth can
execute:

```yaml
simple_client:
  based_on: banking.individual_client
  number_of_copies: 1
  parameters:
    number_of_accounts: 1
    number_of_portfolios: 0
    number_of_phone_numbers: 0
    number_of_email_addresses: 0
  composers: []
```

An LLM writes that file, but it does not guess the vocabulary. `banking.individual_client`,
`number_of_accounts` and every other name in the output come from **your** environment's
own metadata, which the model reads through a set of tools while it works. The draft is
then validated against that same metadata, and anything that does not check out goes
back to the model for correction before you ever see it.

## Some iSynth vocabulary first

If you have not worked with iSynth: it is a platform for generating synthetic test
data. A few terms show up throughout this document.

| Term | What it means |
|---|---|
| **env** (environment) | One iSynth project — its own data model, its own directory on disk. This package is installed per env. |
| **object type** | An entity in that model: `NaturalPerson`, `Account`, `PostalAddress`. |
| **constellation template** | A reusable bundle of objects that belong together — a client with their accounts and addresses. It is the `based_on` value of a case. |
| **composer** | A building block that adds objects to a constellation, in a defined order. |
| **data_config** | The YAML file this package produces: one or more named cases, each saying what to build. iSynth executes it to create the actual data. |

Everything above already exists in your env before this package is installed. The
package only reads it.

## How it works

The model is not given a dump of your data model. It gets 14 tools and fetches only
what it needs:

| Tool group | Examples |
|---|---|
| Object types | `list_object_types`, `get_object_type` |
| Composers | `list_composers`, `get_composer`, `get_composer_order` |
| Constellation templates | `list_constellation_templates`, `get_constellation_template` |
| Enums and functions | `list_enums`, `get_enum`, `list_functions`, `get_function` |
| Curated examples | `list_ai_training_examples`, `get_ai_training_example` |
| Validation | `validate_data_config` |

Those tools are plain Python functions in one registry. Generation calls them
**in-process** — no HTTP, no running server. The same registry is also served over MCP
(streamable-http) for external clients such as IDE agents; that is optional and covered
below.

LLM access goes through [LiteLLM](https://www.litellm.ai/), so a local model in LM
Studio and a hosted one are configured the same way. The model must support **native
tool calling**.

## Requirements

- **Python 3.10** or newer.
- **An iSynth env**, with these already in place:

  | Path | Where it comes from |
  |---|---|
  | `__generated__/*.json` | the iSynth action **Initialize environment** (object types, composers, enums, functions) |
  | `constellations/__generated__/**/*.json` | generated from the env's constellation modules |
  | `data_configs/ai_training/` | **must exist**, may be empty — otherwise `list_ai_training_examples` raises `FileNotFoundError` |
  | `__generated__/composer_sequence.txt` | written once by the restart plugin, see *Composer order* below |

  Generation works without training examples, but noticeably worse: they are what the
  model orients itself on.

- **A tool-calling LLM** — local (LM Studio) or hosted. It must handle *native*
  function calling over several turns: choose a tool, read the result, choose the next,
  produce valid JSON arguments, and finally answer with a single JSON object. Prompt-
  engineered tool use is not enough. See *LLM settings* below.

## Install

```
pip install isynth-ai-data-config
```

Or add `isynth-ai-data-config` to your env's `requirements.txt` and install
that as usual.

Installing gives you the library. It does **not** yet put anything in the iSynth UI —
that needs the plugin wrappers from *Full integration* below. If you only want the MCP
tool server, or want to call `generate_data_order()` from your own code, you can stop
after *Configuration*.

### Alternative: copy the folder

The package can also be copied wholesale into an env instead of installed — the way it
was distributed before it was published, still supported for an env that cannot reach
the private index. The steps (copying the folder, including its `requirements.txt` from
the env's own list, the Docker build context and the `.dockerignore` that keeps it
small) are in `install-by-copying.md`, which ships inside the package.

## Configuration

### Model

Defaults ship in the package's own `settings.yml`. To deviate, repeat the same key in
your **env's** `settings.yml` — the env wins:

```yaml
LLM_MODEL: "lm_studio/qwen3.8-27b-gguf"
LLM_API_BASE: "http://192.168.1.100:1234/v1"
```

`${VARIABLE_NAME}` substitution works as in any iSynth `settings.yml`. You never edit a
file inside the package, which is what keeps it upgradable.

Only keys the package declares are read from the env; an `LLM_*`/`AI_*` key with a typo
is reported as a warning naming the key. The ones you are most likely to touch:

| Key | Default | What it does |
|---|---|---|
| `LLM_MODEL` | `openai/gpt-5.6-sol` | any LiteLLM model identifier |
| `LLM_API_BASE` | — | for local or self-hosted endpoints |
| `LLM_REQUEST_TIMEOUT` | `1800` | seconds; local models can be slow |
| `LLM_MAX_OUTPUT_TOKENS` | `10000` | |
| `AI_MAX_TOOL_TURNS` | `20` | budget for the tool-calling loop |
| `AI_MAX_VALIDATION_ROUNDS` | `2` | correction rounds; `0` disables validation |
| `AI_MAX_TRAINING_EXAMPLES` | `3` | how many curated examples may be fetched |

The full list, each with a comment explaining it, is in the package's `settings.yml`.

### LLM settings

The package sends **no sampling parameters** — no temperature, no top_p. Those come
from the defaults of whatever serves the model (LM Studio, vLLM, the hosted API). The
one exception is `LLM_CHAT_TEMPLATE_KWARGS`, which travels with each request. So these
are settings you make on the server, not here. **A model vendor's own recommendation
always takes precedence over the table below.**

| Setting | Suggested | Why |
|---|---|---|
| Reasoning / thinking mode | **off** | Decode dominates the runtime, and thinking tokens are decode. Measured on a local 27B: 85% of all generated tokens went into reasoning. Risk of thinking blocks or empty answers instead of JSON. |
| Tool / function calling | **on, native** | Mandatory — the tools are passed over the API as `tools=[...]`. |
| Temperature | 0.1–0.2 to start | Qwen warns against greedy decoding (`0.0` causes repetition loops, it recommends `0.7`); gpt-oss is specified for `1.0`. Start low, raise if you see loops. |
| Top P | 0.8–0.9 | Keep conservative for structured output. |
| Top K | 20–40 | Qwen recommends 20. |
| Repeat penalty | 1.0–1.05 | Higher values corrupt recurring field names. |
| Context length | 64K | A run including two correction rounds needs ~15–25K tokens. Use 128K only if you need it. |
| Max response tokens | 2048–4096 | Per assistant turn, tool-call arguments included. The final JSON and the largest tool argument stay under 1K. |
| Prompt truncation | **disabled** | Set the overflow policy to stop at the limit. The transcript grows over the correction rounds; "truncate middle" would silently cut away rules or examples. |
| Structured output / JSON schema | use with care | Forcing a schema on every answer can suppress tool calls. `validate_data_config` checks the final JSON anyway. |
| Quantization | 6–8 bit where memory allows | 4-bit produces measurably less precise tool arguments. Drop to 4-bit only as a memory fallback. |
| Seed | fixed | Makes a validation run reproducible. |

Running locally, pick a model by class rather than by name — a specific ranking dates
quickly. With 16 GB expect to be limited to a small MoE or an aggressively quantized
model, and to lose precision in tool arguments; 24–32 GB comfortably runs a mid-size
MoE with few active parameters at 4-bit; from 64 GB a dense model in the 27–35B range
at 6–8 bit becomes practical. Mixture-of-experts models with roughly 3–4B active
parameters give the best interactive latency, because decode speed, not model size, is
what a run waits on.

### API keys

Keys do **not** belong in `settings.yml`. They go in `<env>/.keys`, in dotenv format.
Anything ending in `_API_KEY` or `_API_BASE` is picked up:

```
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
```

A missing file is not an error — an env that sets its keys as real environment
variables does not need one.

### GitHub Copilot

`LLM_MODEL: "github_copilot/<model>"` runs on a Copilot subscription instead of a
provider API key. There is no key to put in `.keys`: litellm authenticates with a
GitHub OAuth token that it obtains once through a device flow and then keeps in a
token directory, as two files.

| File | What it is |
|---|---|
| `access-token` | the long-lived GitHub token, written by the device flow |
| `api-key.json` | the short-lived Copilot key, refreshed automatically |

The default directory is `~/.config/litellm/github_copilot`, which inside a container
is lost on the next rebuild. So set the directory **first**, then log in.

**1. Point the token directory at the env.** In `<env>/.keys`, next to the other
credentials:

```
GITHUB_COPILOT_TOKEN_DIR=/isynth_envs/<env>/.copilot
```

Every `GITHUB_COPILOT_*` variable in `.keys` is exported, the same way as an
`_API_KEY`. Add `.copilot/` to the env's `.gitignore` — it holds a credential.

**2. Log in once, interactively.** The device flow prints a code and waits for
someone at a browser, so do it by hand rather than letting it happen inside a plugin
call. Importing `ai_data_config.llm_helper` is what applies step 1 — with plain
`import litellm` the token would land in the container's home directory instead:

```bash
docker exec -it <appserver> sh -c 'cd /isynth_envs/<env> && python3 -c "
import ai_data_config.llm_helper, litellm
litellm.completion(model=\"github_copilot/<model>\",
                   messages=[{\"role\": \"user\", \"content\": \"hi\"}])
"'
```

It prints `Please visit https://github.com/login/device and enter code XXXX-XXXX`.
Open that page, enter the code, approve. The call then answers normally.

**3. Check that both files are there.**

```bash
docker exec <appserver> ls /isynth_envs/<env>/.copilot
# access-token  api-key.json
```

**4. Set the model.** In the env's `settings.yml`:

```yaml
LLM_MODEL: "github_copilot/<model>"
```

Leave `LLM_API_BASE` empty — litellm resolves `https://api.githubcopilot.com` itself.
Which models you can name is Copilot's list, not litellm's.

**5. Verify**, with the check under *Full integration → Check that it works*. From
here on nothing is Copilot-specific; generation, the MCP tools and validation runs
behave as with any other model.

Steps 1 and 2 are needed once per env. The `api-key.json` expires regularly and is
refreshed without asking; the `access-token` survives until it is revoked on GitHub.

Two notes on the integration. It is litellm's **provider** integration
(`docs.litellm.ai/docs/providers/github_copilot`) — not the *tutorial* page of a
similar name, which points the Copilot IDE extension at a LiteLLM proxy and is the
opposite direction. And tool calling, which this package depends on, works even
though that page does not mention it.

For a non-default setup the remaining variables also go in `.keys`:
`_ACCESS_TOKEN_FILE` and `_API_KEY_FILE` rename the two files; `_API_BASE`,
`_DEVICE_CODE_URL`, `_ACCESS_TOKEN_URL` and `_API_KEY_URL` point at a GitHub
Enterprise endpoint.

## Using it from Python

```python
from ai_data_config.llm_helper import generate_data_order

data_config = generate_data_order("a private client with one account, no portfolio")
```

`generate_data_order(prompt, content=None)` builds the messages, runs the tool-calling
loop, then validates and orders the result. `content` is an existing `data_config` to
modify rather than start from scratch. It returns a dict, ready to be dumped as YAML.

This needs no running MCP server and no iSynth runtime — only the env's generated
metadata on disk, found via `ENV_HOME`.

---

# Full integration

The steps that put the package into the iSynth UI. Skip any you do not need.

## 1. Plugin wrappers

The four files in `plugins/` are only the iSynth UI; the functionality lives in the
package. Copy them into your env's `plugins/` folder.

Each wrapper needs `base.plugin` (the iSynth plugin API), most also `base.logging`.
Beyond that:

| Wrapper | Calls | Additionally needs |
|---|---|---|
| `plugins/ai_data_config.py` | `llm_helper.generate_data_order()` | — |
| `plugins/validate_ai_prompts.py` | `validation.config_validator.prepare_and_execute_validation_run()` | — |
| `plugins/restart_mcp_server.py` | `mcp_server.admin.request_restart()` | `base.composers.Composer`, `settings.BASE_DIR` |
| `plugins/bill_of_materials.py` | `bill_of_materials.calculate_bill_of_materials()` | `base.plugin_pro.ParameterType` |

Afterwards run **"Reload plugins"** from the context menu of the `plugins/` folder in
the UI — the REST API serves new definitions immediately, the UI only after that.

## 2. Composer order (once)

`get_composer_order()` reads `__generated__/composer_sequence.txt`. Without that file it
returns an **empty list**, and the system prompt then fails to tell the model which
order composer entries have to be in.

The restart plugin writes it: run **Restart MCP Server** with the checkbox ticked. It
fetches the order via `Composer.execution_sequence()` and writes the file *before*
restarting the server, so it appears even when no MCP server is running yet (the
restart call then fails, which is fine).

Repeat after any model change that adds, removes or reorders composers. The file is
read at import time, so a running process only picks up a change after a restart.

## 3. MCP server service (optional)

Only needed for external MCP clients (IDE agents) and for the **Restart MCP Server**
plugin. Generation and validation runs do **not** need it — they call the same tools
in-process and keep working with the service stopped.

```yaml
  i-mcpserver:
    build:
      context: ../
      dockerfile: deployment/Dockerfile
    restart: unless-stopped          # safety net, should os.execv fail on restart
    environment:
      - ENV_HOME=/isynth_envs/<env>  # must match the mount target below
      - MCP_TRANSPORT=streamable-http
      - MCP_HOST=0.0.0.0
      - MCP_PORT=8010
      - MCP_PATH=/mcp
      - MCP_WAIT_FOR_GENERATED_TIMEOUT=60
    volumes:
      - "../:/isynth_envs/<env>"
    working_dir: /isynth_envs/<env>  # so "-m ai_data_config.mcp_server" resolves
    networks: [<the appserver's network>]
    ports:
      - "8010:8010"
    command: "python3 -m ai_data_config.mcp_server"
```

**The service name is not free.** `mcp_server/admin.py` has
`DEFAULT_BASE_URL = "http://i-mcpserver:8010"`, which is where the restart plugin looks.
Either the service is called `i-mcpserver`, or the appserver service sets
`MCP_SERVER_URL` to the right address.

Installed from the index, the package also puts an `ai-data-config-mcp` command on
`PATH`, equivalent to `python3 -m ai_data_config.mcp_server`.

## 4. Check that it works

```bash
# package loads, settings arrive, tools answer
docker exec <appserver> sh -c 'cd /isynth_envs/<env> && python3 -c "
from ai_data_config import config, llm_helper
from ai_data_config.mcp_server.tools import TOOLS, call_tool
print(\"ENV_HOME :\", config.ENV_HOME)
print(\"LLM_MODEL:\", llm_helper.DEFAULT_MODEL)
print(\"Tools    :\", len(TOOLS))
print(\"Composer :\", call_tool(\"get_composer_order\", {})[:3], \"...\")
print(\"Examples :\", len(call_tool(\"list_ai_training_examples\", {})))
"'

# MCP server, if you set one up
curl -s http://localhost:8010/health
```

Expected: 14 tools, a **non-empty** composer order, the number of training examples, and
`{"status":"ok",...}`. An empty composer list means step 2 is missing.

# What needs iSynth and what does not

`llm_helper` and `mcp_server` import nothing from `base` — generation and the MCP tools
run in any directory where the dependencies are installed, iSynth or not.

Only `bill_of_materials.py` needs the iSynth runtime (`base.constellation_utils`,
`base.data_config_utils`), and through it the validation run, which compares the bill of
materials of a generated `data_config` against the case's reference. `base` is part of
the iSynth image and cannot be installed from an index, so importing that module
elsewhere raises a `ModuleNotFoundError` saying so.

The MCP server container is needed by none of this except `restart_mcp_server`.

# Troubleshooting

| Symptom | Cause |
|---|---|
| `FileNotFoundError: ai_training_dir source not found` | `data_configs/ai_training/` missing — create it, empty is fine |
| `FileNotFoundError: ... __generated__/...` | **Initialize environment** has not run in the env yet |
| `ModuleNotFoundError: No module named 'box'` or similar | dependencies not installed — see *Install* |
| `ModuleNotFoundError: ... needs the iSynth runtime ('base')` | `bill_of_materials` or the validation run is running outside an iSynth env |
| `MCP server at ... did not answer` | service is not called `i-mcpserver` and `MCP_SERVER_URL` is not set |
| Plugin does not appear in the UI | run "Reload plugins" from the context menu of the `plugins/` folder |
| Env override is ignored | key not declared in the package's `settings.yml` — the log warning names it |
| `get_composer_order()` is empty | `__generated__/composer_sequence.txt` missing — see *Composer order* |
| Logs are empty | not on STDOUT: the package logs with `propagate=False` into `<env>/logs/*.log` |

# Further reading

`install-by-copying.md` ships alongside this file: the alternative to installing,
copying the folder into an env and wiring up its dependencies and the Docker build
yourself.

# Building the package

```bash
python -m build ai_data_config/        # -> ai_data_config/dist/*.whl and *.tar.gz
python -m twine upload ai_data_config/dist/*   # -> pypi.org
```

The build runs from inside the package folder, where `pyproject.toml` sits, because the
folder is meant to travel as one unit. Dependencies are declared only in
`requirements.txt`; `pyproject.toml` reads them from there via
`dynamic = ["dependencies"]`, so there is no second list to drift out of sync. The
package is published on pypi.org; its licence stays proprietary.

In CI (`.gitlab-ci.yml` in the repository root) this runs automatically: every change
under `ai_data_config/` is built and checked, and pushing a tag
`ai_data_config-v<version>` (matching `version` in `pyproject.toml`) publishes the
package to the project's GitLab package registry.
