Metadata-Version: 2.4
Name: yapyon
Version: 0.2.1
Summary: Python-style typed literals with YAML-like block structure
Author-email: Jei Blanchard <jjb@bmail.club>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/doctorjei/yapyon
Project-URL: Repository, https://github.com/doctorjei/yapyon
Project-URL: Issues, https://github.com/doctorjei/yapyon/issues
Keywords: config,configuration,data format,parser,serialization,yaml
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Markup
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: PyYAML>=6; extra == "dev"
Requires-Dist: kinemata==0.1.1; extra == "dev"
Dynamic: license-file

# yapyon (やぴょん)

> 「YAMLちゃうで。やぴょんやぴょん」
> *It's not YAML. It's yapyon.*

Python-style typed literals with YAML-like block structure. Quoted strings,
`True`/`False`/`None`, indentation blocks, `-` sequences, `+ ` multimaps,
`#` comments — and parse-time splicing that replaces YAML's anchors.

```yapyon
name: "api-gateway"
version: 3.10
port: 8080
debug: False

paths:
  root: "/srv/gateway"
  logs: y"{root}/logs"          # -> "/srv/gateway/logs"
  banner: y"{name} v{version}"  # -> "api-gateway v3.10"   (lexeme, not 3.1)

changes:                        # multimap: keys may repeat, order is data
  + registry: {set: ["Bypass"]}
  + file: {delete: ["Edge"]}
  + registry: {remove: ["Edge Update"]}
```

**The pitch:** the same bytes mean the same data, to every parser, on every
machine, always. No implicit typing (`no` is not `False`; there are no
unquoted scalars at all), no anchors, no tags, no evaluation, one null
spelling, one document per file. A repeated key is an error unless the block
says `+ `, so it can never be mistaken for a typo — or silently collapsed,
the way Python and YAML both drop `{"a": 1, "a": 2}` down to one entry.

## Status

**`0.2.1`.** Lexer, parser, resolver, loader and emitter are done and
tested, including a conformance suite that turns the splice matrix and the
Python divergence table into executable cases, and a portable corpus in
`docs/conformance/` that another implementation can run against itself. The
suite is **779 tests** on the current tree; the number here tracks the tree,
not a release.

This is the **self-contained tier**: every document resolves from its own
bytes. There is no ambient scope, no includes and no environment at the
format level, so a yapyon file cannot reach the filesystem or run
consumer-supplied code — the format's reach is something you can *prove*
rather than audit. Consumer-registered functions and file embedding are a
deliberate later tier, not a gap in this one.

**The normative specification is published, in two documents under
[`docs/`](docs/):**

| | |
|---|---|
| [`docs/GRAMMAR.md`](docs/GRAMMAR.md) | *are these bytes well-formed yapyon?* — self-contained, normative for syntax |
| [`docs/SPEC.md`](docs/SPEC.md) | *what do well-formed bytes mean?* — normative for meaning |
| [`docs/conformance/`](docs/conformance/) | *does an implementation agree?* — 48 portable cases; the directory's own README is the protocol |

The corpus is not a third normative document: it is **derived from** the
other two, and where they disagree the spec wins and the case is wrong. What
it adds is a check rather than a definition — every case is data, not code
in a language, so a second implementation runs the same files against its
own loader and gets a yes or no.

References run one way: the spec cites the grammar, the grammar cites
nothing. A parser can be written from the grammar alone, which is the point of
separating them. This README summarises the format; those documents define it.

**Identifiers are UAX #31, pinned to Unicode 14.0.0, with invisible
characters excluded.** yapyon ships its own table rather than asking the
interpreter, so a key means the same thing on every install and every Python.
Width variants and canonically equivalent spellings are the same name —
`ｎａｍｅ` is `name`, and `é` is `é` however it was composed — while `ﬁ` and
`fi` stay distinct. `yapyon.UNICODE_VERSION` names the pinned version.

```console
$ pip install yapyon
```

```python
>>> import yapyon
>>> yapyon.loads('name: "gw"\nport: 8080\naddr: y"{name}:{port}"\n')
{'name': 'gw', 'port': 8080, 'addr': 'gw:8080'}
```

`loads` and `load` return plain Python. Eight of the ten types are builtins;
a `+ ` block comes back as an `OrderedMultimap`, and a `yt` literal as a
`Template`.

## Templates

`yt` is a y-string that is **not joined**. Its holes resolve against the
document exactly as `y`'s do; the consumer receives the literal text and the
resolved values separately, and renders them itself.

```python
>>> d = yapyon.loads('user: "; rm -rf /"\ncmd: y"echo {user}"\ntmpl: yt"echo {user}"\n')
>>> d["cmd"]
'echo ; rm -rf /'
>>> list(d["tmpl"])
['echo ', Hole('user', '; rm -rf /'), '']
```

`y` : `yt` :: f-string : t-string. A joined string has lost which bytes the
author wrote and which came from the data, and nothing downstream can recover
it; parts keep them apart until a consumer that knows the destination — shell,
SQL, HTML, a registry — decides how each value is made safe.

A hole carries `.value` (typed), `.lexeme` (the document's spelling, so `3.10`
renders back as `3.10` rather than `3.1`) and `.ref` (the reference as
written). Carrying is unconstrained — a hole may hold a list or raw bytes,
because yapyon is not the one rendering. `render()` joins canonically and
reproduces exactly what the equivalent `y` produces; it is an explicit call,
never an implicit `str()`, because an easy implicit join would undo the point.

## Records

A **yapyon record** is the restricted form in which *every leaf element has a
literal value* — no splicing, no templates, nothing deferred. It is a strict
subset: every record is a valid yapyon document meaning the same thing.

```python
>>> yapyon.loads_record('name: "gw"\nport: 8080\n')
{'name': 'gw', 'port': 8080}
>>> yapyon.is_record('addr: "{name}:80"\n')   # braces in a plain string are inert
True
>>> yapyon.loads_record('name: "gw"\naddr: y"{name}:80"\n')
Traceback (most recent call last):
yapyon.lexer.AkanError: akan: line 2, col 6: a record's leaves must all be
literals; a y-string defers its value (load with yapyon.loads for the full form)
```

`loads_record` / `load_record` are the whole pipeline minus the resolver, and
a separate function rather than a keyword argument — the safe path should not
be one typo away from the unsafe one. `is_record` answers the question by
lexing, without loading.

This is why an implementation with no resolver can still be a conforming one:
the safe subset is declared up front rather than carved out afterwards — and
why the spec asks every implementation to say which of the two it provides.
**yapyon implements the full form**, and so also loads records.
*(New in `0.1.0a2`.)*

A record also prints back out as yapyon, normalized:

```console
$ python -m yapyon record examples/registry.ypy
```

Spelling becomes canonical — double quotes, minimal escapes, block containers
— while every value survives exactly: `3.10` stays `3.10` rather than becoming
`3.1`, and `b"..."` is not respelled as base64. It works from the parse tree
for that reason; the source spelling is gone once a value is a Python `float`.
**Comments do not survive**, so this is a normalizer rather than a
round-tripper. `yapyon.emit` is the same thing as a function. A non-record is
refused at the offending prefix, with the record rule as the reason.

## Limits

Resolution has two tiers (SPEC §5.3), and they are different numbers.

| | Threshold | Exceeding it |
|---|---|---|
| Warn tier — fixed by the spec | depth 32, rendered size 1 MiB | `shiran`, and resolution continues |
| Refusal ceiling — this implementation's | depth 128, rendered size 4 MiB | `akan` |

```python
>>> deep = 'a0: "x"\n' + "".join(f'a{i}: y"{{a{i-1}}}"\n' for i in range(1, 41))
>>> yapyon.loads(deep, warn=print)["a40"]
shiran: line 41, col 5: a dependency chain 40 deep passes §5.3's warn threshold
(32); this document is costly to resolve and an implementation may refuse it
-- shorten the chain, or raise max_depth where it is loaded
'x'
```

Move the ceiling with `loads(..., max_depth=…, max_size=…)`. The warn tier is
not an argument: a consumer chooses what it will *refuse*, not what the format
calls suspicious.

The ceiling is not optional and is deliberately finite — chained doubling is
exponential, so a document of a few dozen lines can demand unbounded memory,
and a resolver with no ceiling is a denial-of-service in anything that parses
configuration it did not write. The spec declines to fix one ceiling for every
implementation; it does not suggest having none.

Both tiers are about cost alone. A **cycle or a missing target is always an
akan** at any depth, because it can never resolve.

From a checkout, the stages will also dump what they see:

```console
$ pip install -e ".[dev]" && pytest -q
$ python -m yapyon lexer  examples/gateway.ypy   # token dump
$ python -m yapyon parser examples/gateway.ypy   # AST dump
$ python -m yapyon tier   examples/gateway.ypy   # 2 — this one needs the resolver
$ python -m yapyon expand examples/bridge.ypy    # level 2 in, a record out
```

Every stage writes its output to stdout and its diagnostics to stderr, and
**shows shirans by default** — the library hides them behind `warn=`, but a
person at a terminal asked. A shiran does not change the exit status; an akan
exits 1.

## References

A hole names a value. `a.b` is sugar for `a["b"]` — the bracket is the
general form, and its content yields a key.

```yapyon
protocol: "claude"
mind:
  endpoint: "https://api"
  dialects:
    claude:
      api_uri: y"{endpoint}/v1"        # reaches its uncle by §5.2 scope
    codex:
      api_uri: y"{endpoint}/responses"
  chosen: y"{dialects[protocol].api_uri}"   # -> "https://api/v1"
```

| Spelling | Means |
|---|---|
| `{a.b}` | the key `b` — canonical for identifiers |
| `{a["b.c"]}` | a key a dot cannot reach |
| `{xs[0]}` | a list position; brackets only, since `0` is no identifier |
| `{d[p]}` | the key *stored in* `p`, resolved in the y-string's own scope |
| `{mm.k}` | every value filed under `k` in a multimap, in order, as a list |

`{a["bar"]}` where a dot would do is a `shiran` — legal, but say `{a.bar}`.
Because the enclosing string's delimiter ends it, quote a key with the other
mark: `y"{a['b']}"`.

### Relative references

`__ROOT__` names an absolute position; `__PARENT__` and `__KEY__` name
relative ones, so a block can refer to where it *is*:

```yapyon
store: "/cfg"
personas:
  kimi:
    id:     y"{__PARENT__.__KEY__}"              # -> "kimi"
    prompt: y"{__ROOT__.store}/{__PARENT__.__PARENT__.__KEY__}/{__PARENT__.__KEY__}.md"
```

The same text in every block, each resolving to its own name — without them a
block cannot be renamed or moved without editing its contents. `__PARENT__`
repeats (`{__PARENT__.__PARENT__.x}`) and leads a reference; `__KEY__` ends
one. Bare `{__KEY__}` is the *pair's own* key, which is rarely what you want.

### Serializers

A container cannot be spliced into text — there is no one text form and
picking one would be guessing. Naming the encoding is how it crosses:

```yapyon
store:
  model: "m1"
  host: "https://api.example.com/v1"
body: y"{store.__AS_TOML__()}"
```

```toml
model = "m1"
host = "https://api.example.com/v1"
```

`__AS_JSON__()`, `__AS_TOML__()`, `__AS_YAML__()` and `__AS_YAPYON__()`, all
written without a runtime dependency. Keys stay in document order, nothing is
pruned, and where a format cannot carry a value faithfully the encoding is an
error rather than an approximation — `None` in TOML, bytes in JSON, TOML or
YAML, a multimap's repeated keys. YAML scalars are quoted unless nothing could
misread them, so `"no"` and `"3"` survive as strings.

`__AS_YAPYON__()` writes the block as a canonical yapyon **record**, and it is
the one serializer that carries everything: multimaps, bytes and `None`, with
`3.10` still spelled `3.10`. The only thing it refuses is a `yt`, which has no
record form.

A serializer names the value it encodes, so it cannot name a *bare* anchor:
`{__PARENT__.__AS_JSON__()}` would have to encode the string it is computing,
and is akan. Reach past the anchor — `{__ROOT__.blk.__AS_JSON__()}` — which is
the shape you want anyway.

### Resolving a reference from code

A consumer that assembles a tree from several loaded documents can resolve a
reference against it **by the same rules a hole uses**, without writing its
own copy of the search:

```python
tree = yapyon.loads(base)
tree["harness"] = harness = yapyon.loads(harness_doc)
yapyon.lookup(tree, harness, "mind.dialects[protocol].api_uri")
```

`current` is the node you are holding, and the answer is the one a
`y"{...}"` written inside it would get: its own keys first, then outward,
nearest winning — which is why the tree comes too, since a dict does not
know its parents. The whole hole grammar works, serializers included, and
an indirect key like `[protocol]` resolves from `current`. It is the
resolver's own search, so the two cannot disagree. A refusal is an
`AkanError` naming where `current` sits.

## Prefixes

| Spelling | Name | Spoken | Result |
|---|---|---|---|
| `b`, `b64` | bytes literals | — | bytes |
| `r`, `rb` | raw | — | str, bytes |
| `y` | yapyon string | y-string | str, spliced at parse |
| `yb` | yapyon byte string | yeeb-string | bytes, spliced at parse |
| `yt` | yapyon template string | yeet-string | Template, unjoined |
| `ry` | raw yapyon string | ree-string | str, backslash literal |

`f"..."`, `t"..."`, `u"..."` are errors with hints — their meaning depends on
an enclosing scope, and a data file has none.

## Diagnostics

`shiran:` (warning) → `akan:` (error) → `yakamashiwa:` (internal bug).
Verbose success prints `ええやん`.
