Metadata-Version: 2.2
Name: ragkit-llmparse
Version: 0.1.1
Summary: Robustly extract, repair, and coerce structured data (JSON) from messy LLM text output.
Author-email: Meet2147 <meetjethwa3@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Meet2147/pythonLibraries/tree/main/llmparse
Project-URL: Repository, https://github.com/Meet2147/pythonLibraries
Project-URL: Issues, https://github.com/Meet2147/pythonLibraries/issues
Keywords: llm,json,parse,repair,extract,schema,genai,structured-output
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Filters
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

<p align="center">
  <img src="https://raw.githubusercontent.com/Meet2147/ragkit-assets/main/llmparse.png" alt="llmparse" width="460">
</p>

<p align="center">
  <a href="https://pypi.org/project/ragkit-llmparse/"><img src="https://img.shields.io/pypi/v/ragkit-llmparse.svg" alt="PyPI"></a>
  <img src="https://img.shields.io/pypi/pyversions/ragkit-llmparse.svg" alt="Python versions">
  <img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License: MIT">
</p>

# llmparse

Robustly extract, repair, and coerce structured data (primarily JSON) out of messy LLM text output.

> Part of the **ragkit** suite. Install with `pip install ragkit-llmparse`, then `import llmparse`.

LLMs love to wrap JSON in markdown fences, add a friendly sentence before and after it, sprinkle in trailing commas, use single quotes, emit Python literals (`True`/`False`/`None`), and forget to quote object keys. `llmparse` cleans all of that up and hands you a real Python object — optionally coerced to a schema you expect.

Pure standard library (`json`, `re`, `ast`). No dependencies. Python 3.8+.

> Note: this is a best-effort, heuristic library. It is designed to recover data from *almost*-JSON. It is not a strict validator and it can be fooled by sufficiently pathological input.

## Install

```bash
pip install ragkit-llmparse
```

Local development (from `llmparse/`):

```bash
pip install -e .
```

## Quick Start

Parse a messy LLM reply — fences plus prose — straight into a dict:

```python
import llmparse

reply = """
Sure! Here is the data you asked for:

```json
{
    "name": "Ada Lovelace",
    "born": 1815,
    "fields": ["math", "computing"]
}
```

Hope this helps!
"""

data = llmparse.loads(reply)
print(data["name"])   # Ada Lovelace
print(data["fields"]) # ['math', 'computing']
```

## Handling each kind of mess

`loads` runs a pipeline: try raw `json.loads`, then extract the first balanced JSON region, then repair it, then fall back to `ast.literal_eval`.

### Trailing commas

```python
llmparse.loads('{"a": 1, "b": 2,}')      # {'a': 1, 'b': 2}
llmparse.loads('[1, 2, 3,]')             # [1, 2, 3]
```

### Single quotes

```python
llmparse.loads("{'name': 'Alice', 'age': 30}")
# {'name': 'Alice', 'age': 30}
```

Apostrophes inside double-quoted strings are left alone:

```python
llmparse.loads('{"msg": "it\'s fine"}')  # {'msg': "it's fine"}
```

### Python literals

```python
llmparse.loads('{"a": True, "b": False, "c": None}')
# {'a': True, 'b': False, 'c': None}
```

### Unquoted keys

```python
llmparse.loads('{name: "Bob", age: 25}')
# {'name': 'Bob', 'age': 25}
```

### Python-dict-style output (ast fallback)

When the text is valid Python but not valid JSON (tuples, etc.), `loads` falls back to `ast.literal_eval`:

```python
llmparse.loads("{'a': (1, 2), 'b': {'nested': True}}")
# {'a': (1, 2), 'b': {'nested': True}}
```

## Extracting without parsing

`extract_json` returns the raw JSON substring(s). The brace scanner respects string literals and escapes, so a `}` inside a string will not cut the object short:

```python
llmparse.extract_json('{"a": "text with } brace"}')
# '{"a": "text with } brace"}'

# All top-level objects/arrays:
llmparse.extract_json('First {"a": 1} then {"b": 2} and [3, 4].', first=False)
# ['{"a": 1}', '{"b": 2}', '[3, 4]']
```

`repair_json` gives you the fixed-up string if you want to inspect it:

```python
llmparse.repair_json("{name: 'Al', active: True, tags: ['x', 'y',],}")
# '{"name": "Al", "active": true, "tags": ["x", "y"]}'
```

Repairing already-valid JSON returns an equivalent, still-parseable string.

## Schema coercion

Describe the shape you expect and let `llmparse` cast values into it. A schema maps each field to either a bare type or a spec dict.

```python
schema = {
    "name":     str,
    "price":    float,
    "in_stock": bool,
    "qty":      int,
    # spec dict form:
    "discount": {"type": float, "required": False, "default": 0.0},
}

obj = {"name": "Widget", "price": "19.99", "in_stock": "true", "qty": "5"}
clean = llmparse.coerce(obj, schema)
# {'name': 'Widget', 'price': 19.99, 'in_stock': True, 'qty': 5, 'discount': 0.0}
```

Coercion rules (when `coerce` is on, which is the default per field):

- `"3"` -> `int` `3`
- `"3.5"` -> `float` `3.5`
- `"true"`, `"false"`, `1`, `0` -> `bool`
- numbers/bools -> `str`

### Spec dict options

| key        | meaning                                              | default |
|------------|------------------------------------------------------|---------|
| `type`     | target type (`int`, `float`, `str`, `bool`, `list`, `dict`) | —       |
| `required` | whether the field must be present                    | `True`  |
| `default`  | value to fill if the field is missing                | —       |
| `coerce`   | cast the value, or require an exact type match       | `True`  |

### Aggregated errors

`coerce` collects *every* problem and raises a single `SchemaError` whose `.errors` list holds them all — so you see all missing/invalid fields at once:

```python
schema = {"a": int, "b": str, "c": float}
try:
    llmparse.coerce({"a": "oops"}, schema)
except llmparse.SchemaError as e:
    for problem in e.errors:
        print(problem)
    # field 'a': cannot coerce 'oops' to int
    # missing required field 'b'
    # missing required field 'c'
```

### Extra fields

Extra fields not mentioned in the schema are **kept** by default. Pass `strict=True` to drop them:

```python
llmparse.coerce({"a": 1, "extra": 2}, {"a": int})                 # {'a': 1, 'extra': 2}
llmparse.coerce({"a": 1, "extra": 2}, {"a": int}, strict=True)    # {'a': 1}
```

## One-shot: parse + coerce

`parse` is `loads` followed by `coerce` when a schema is supplied. Extra kwargs (`repair`, `fallback_ast`) pass through to `loads`.

```python
reply = """Here you go:
```json
{name: 'Widget', 'price': '19.99', in_stock: True, qty: '5',}
```
"""

schema = {"name": str, "price": float, "in_stock": bool, "qty": int}
llmparse.parse(reply, schema)
# {'name': 'Widget', 'price': 19.99, 'in_stock': True, 'qty': 5}
```

## Multiple objects in one blob

`extract_all_json` parses every balanced JSON object/array it can find, skipping the ones that do not parse:

```python
text = 'First user {"id": 1} and second {"id": 2}. Also a list [10, 20].'
llmparse.extract_all_json(text)
# [{'id': 1}, {'id': 2}, [10, 20]]
```

## API summary

- `loads(text, repair=True, fallback_ast=True)` — main entrypoint; returns a Python object or raises `ParseError`.
- `parse(text, schema=None, strict=False, **loads_kwargs)` — `loads` then optional `coerce`.
- `extract_json(text, first=True)` — raw JSON substring(s).
- `extract_all_json(text)` — list of parsed objects.
- `repair_json(s)` — best-effort near-JSON -> JSON string.
- `coerce(obj, schema, strict=False)` — type coercion/validation.
- `ParseError` — has `.snippet`.
- `SchemaError` — has `.errors` (list).

## License

MIT
