Metadata-Version: 2.4
Name: koine
Version: 0.11.0
Summary: A declarative, data-driven parser generator for creating languages, ASTs, and transpilers.
Author-email: Chris Bates <1314008+chrsbats@users.noreply.github.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/chrsbats/koine
Project-URL: Bug Tracker, https://github.com/chrsbats/koine/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Compilers
Classifier: Topic :: Text Processing :: General
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: parsimonious>=0.10
Requires-Dist: safer-yaml>=0.1.1
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-subtests; extra == "dev"
Dynamic: license-file

# Koine

_A declarative, data-driven parser generator for creating languages, ASTs, and transpilers._

[![PyPI](https://img.shields.io/pypi/v/koine.svg)](https://pypi.org/project/koine/)
[![Python](https://img.shields.io/pypi/pyversions/koine.svg)](https://pypi.org/project/koine/)
[![Build](https://github.com/chrsbats/koine/actions/workflows/ci.yml/badge.svg)](https://github.com/chrsbats/koine/actions/workflows/ci.yml)
[![Coverage](https://img.shields.io/badge/coverage-90%25%2B-brightgreen)](https://github.com/chrsbats/koine/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Koine allows you to define a complete language pipeline—from lexing and validation to Abstract Syntax Tree (AST) generation to final code transpilation—using a simple, human-readable, and JSON-compatible data structure. This means you can write your grammars in YAML, JSON, TOML, or any other format that can be loaded into a nested dictionary structure. The engine consumes these definitions and produces clean, JSON-compatible ASTs.

This approach separates the _what_ (your language definition) from the _how_ (the parsing engine).

### Core Features

- **Declarative:** Define complex grammars entirely in a data format like YAML. No code generation step is required.
- **Pipeline-based:** Use Koine for simple validation, structured AST generation, or full transpilation.
- **Composable Grammars:** Break large languages into modular, independently testable files using `subgrammar` directives. Placeholders allow you to develop each part in isolation before linking them into a complete system.
- **Powerful:** Handles operator precedence, left/right associativity, lookaheads, and features an integrated stateful lexer for indentation-based syntax.
- **Language Agnostic:** One native C++ engine supplies Python and Nim packages through a small C interface. Grammars remain ordinary JSON-compatible data.

## Why Koine?

Koine is designed for people building DSLs, not compilers. If you've struggled with Lark's lookahead limitations or ANTLR's codegen overhead, Koine offers a different model: fully declarative grammar, AST, and transpilation—defined entirely as JSON/YAML. Its modular grammar system allows you to build and test complex languages piece by piece, making it ideal for large, collaborative projects.

It was built for environments where you want to **write, modify, or interpret grammars at runtime**—such as agent scripting, game modding, or interactive systems that evolve on the fly.

➡️ [Read the full rationale](RATIONALE.md)

### Philosophy and When to Use Koine

Koine is designed to be an exceptionally fast tool for **prototyping Domain-Specific Languages (DSLs)** and other custom parsers. Its "configuration-over-code" approach makes it ideal for tasks where clarity, portability of the grammar, and development speed are more important than raw parsing performance.

**Use Koine When:**

- **You are building a DSL:** For query languages, configuration formats, or command languages.
- **You need to parse complex but small-to-medium sized data:** Where the cost of building a traditional parser is too high.
- **You value a declarative, data-driven grammar:** The grammar can be stored as simple data (JSON/YAML) and is portable to other potential Koine engine implementations.
- **You are prototyping language ideas:** The entire lex->parse->transpile pipeline can be defined and modified quickly.

**Consider Other Tools When:**

- **You need extreme performance:** Koine is not designed to compete with high-performance compilers like `gcc` or `ANTLR` for parsing millions of lines of code.
- **You require complex semantic analysis:** If your transpilation logic requires deep, stateful analysis (e.g., advanced type inference, complex symbol tables), a traditional visitor pattern in a general-purpose language may be more suitable.
- **You need sophisticated error recovery:** Koine reports the first syntax error it finds and stops.

---

## Installation

`pip install koine` selects the native package where a platform wheel exists
(currently Linux x86-64, `manylinux_2_27`/`manylinux_2_28`) and otherwise installs
the pure-Python implementation. Force the pure version on any platform with:

```bash
pip install --no-binary=koine koine
```

The pure package depends on `parsimonious` and `safer-yaml`; the native package has
no Python runtime dependencies. Both expose the same `koine` API.

From the checkout, the root project builds the pure package:

```bash
python3 -m pip install .
```

Build the native package from its subproject instead:

```bash
CC=gcc-15 CXX=g++-15 python3 -m pip install ./native
```

Native builds need CMake 3.22+ and a C++17 toolchain; GCC 15 is recommended and
GCC versions before 12 are rejected because of an exception-cleanup compiler bug.
The build downloads its pinned native dependencies and bundles PCRE2 and rapidyaml
into the shared core. A prebuilt platform wheel needs no compiler.

For Nim 2 (default ARC/ORC), install from the repository root with:

```bash
CC=gcc-15 CXX=g++-15 nimble install
```

Both packages use `import koine`, prepare grammars once and manage native handles
automatically. See [native/README.md](native/README.md) for Nim examples, deployment,
development checks and platform coverage. Linux x86-64 and Windows x64 are tested;
platforms without a native wheel use the pure-Python package. Verified Linux wheels
support the `manylinux_2_27` / `manylinux_2_28` x86-64 tags; the Windows wheel is
`py3-none-win_amd64`. Docker is not a build or runtime dependency.

Grammar files accept YAML 1.2 and JSON natively. Unquoted `yes`, `no`, `on`, and
`off` remain strings; `true` and `false` are booleans. The Python behavioral
reference is retained under `reference/koine` and uses
[safer-yaml](https://github.com/chrsbats/safer-yaml).

## Release builds

Build the pure sdist/universal wheel and the portable Linux native wheel, then
include a Windows wheel built on Windows:

```bash
python tools/build.py --windows-wheel /path/to/koine-0.11.0-py3-none-win_amd64.whl
python tools/publish.py --dist dist
```

`tools/build.py` uses the GCC 15/glibc 2.28 SDK at
`build/sdk-alma8-gcc15` by default. Create it with
`native/tools/prepare_linux_sdk.py` if needed, or pass `--sdk`.

After validation, upload to TestPyPI first:

```bash
python tools/publish.py --repository testpypi --upload
```

Once the TestPyPI install has been checked, repeat with
`--repository pypi`. The script uses the matching `[testpypi]` or `[pypi]`
section in `~/.pypirc`; if no section exists it falls back to `TWINE_USERNAME`
and `TWINE_PASSWORD`. It only selects artifacts for the version in
`pyproject.toml`; it never uploads older files left in `dist/`.

---

## Quick Start

Let's build a simple calculator that can parse `2 + 3` and transpile it to `(add 2 3)`.

1.  **Create your parser grammar, `parser_calc.yaml`:** This defines the language syntax and how to build an AST.

    ```yaml
    # A grammar that handles precedence (* before +) and parentheses.
    start_rule: expression
    rules:
      expression:
        ast: { structure: "left_associative_op" }
        sequence:
          - { rule: term }
          - zero_or_more:
              sequence:
                [{ rule: _ }, { rule: add_op }, { rule: _ }, { rule: term }]
      term:
        ast: { structure: "left_associative_op" }
        sequence:
          - { rule: factor }
          - zero_or_more:
              sequence:
                [{ rule: _ }, { rule: mul_op }, { rule: _ }, { rule: factor }]
      factor:
        ast: { promote: true }
        choice:
          - { rule: number }
          - sequence:
              - { literal: "(", ast: { discard: true } }
              - { rule: _ }
              - { rule: expression }
              - { rule: _ }
              - { literal: ")", ast: { discard: true } }
      add_op:
        ast: { leaf: true }
        choice:
          - { literal: "+" }
          - { literal: "-" }
      mul_op:
        ast: { leaf: true }
        choice:
          - { literal: "*" }
          - { literal: "/" }
      number:
        ast: { leaf: true, type: "number" }
        regex: "\\d+"
      _:
        ast: { discard: true }
        regex: "[ \\t]*"
    ```

2.  **Create your transpiler grammar, `transpiler_calc.yaml`:** This defines how to convert the AST into a new string.

    ```yaml
    rules:
      binary_op:
        template: "({op} {left} {right})"
      add_op:
        cases:
          - if: { path: "node.text", equals: "+" }
            then: "add"
          - default: "sub"
      mul_op:
        cases:
          - if: { path: "node.text", equals: "*" }
            then: "mul"
          - default: "div"
      number:
        use: "value"
    ```

3.  **Use the Koine engine in `main.py`:**

    The `parser.parse()` method takes the source text to parse. For testing individual rules, you can also pass an optional `start_rule="<rule_name>"` argument to override the default entry point.

    ```python
    from koine.parser import Parser, Transpiler

    # 1. Load the grammars
    # Use Parser.from_file to correctly set the base path for subgrammars.
    parser = Parser.from_file("parser_calc.yaml")
    transpiler = Transpiler.from_file("transpiler_calc.yaml")

    # 2. Run the pipeline
    source_code = "2 * (3 + 4)"
    parse_result = parser.parse(source_code)

    if parse_result['status'] == 'success':
        # The parser produces a structured AST (as a dictionary)
        ast = parse_result['ast']
        print("Intermediate AST:")
        import json
        print(json.dumps(ast, indent=2))

        # The transpiler consumes the AST to produce the final string
        translation = transpiler.transpile(ast)
        print(f"\nInput: '{source_code}'")
        print(f"Output: {translation}")
    else:
        print(f"Error: {parse_result['message']}")
    ```

4.  **Run it:**

    ```
    Intermediate AST:
    {
      "tag": "binary_op",
      "op": {
        "tag": "mul_op",
        "text": "*",
        "line": 1,
        "col": 3
      },
      "left": {
        "tag": "number",
        "text": "2",
        "line": 1,
        "col": 1,
        "value": 2
      },
      "right": {
        "tag": "binary_op",
        "op": {
          "tag": "add_op",
          "text": "+",
          "line": 1,
          "col": 8
        },
        "left": {
          "tag": "number",
          "text": "3",
          "line": 1,
          "col": 6,
          "value": 3
        },
        "right": {
          "tag": "number",
          "text": "4",
          "line": 1,
          "col": 10,
          "value": 4
        }
      }
    }

    Input: '2 * (3 + 4)'
    Output: (mul 2 (add 3 4))
    ```

---

### Overview

This document describes the data-driven grammar format for the Koine parser. The system is designed as a flexible pipeline that can be used for simple validation, structured data extraction (AST generation), or full-scale language translation (transpilation).

The core philosophy is to separate the _what_ from the _how_. You define _what_ the language looks like and _what_ the output should be, and the Koine engine handles _how_ to parse and transform it.

This guide will walk through five primary use cases, showing how to add complexity at each stage:

1.  **Validation:** Checking if an input string conforms to a grammar.
2.  **AST Generation:** Parsing an input string into a clean, semantic Abstract Syntax Tree.
3.  **Transpilation:** Transforming the AST into a new string format (e.g., infix math to LISP).
4.  **Lexer-Based Parsing:** Handling context-sensitive syntax, like Python's indentation, by defining tokens.
5.  **Indented Output:** Transpiling an AST into an output format that requires proper indentation, like Python.

---

### Use Case 1: Validation

**Goal:** To answer the simple question, "Does this input string conform to my language's syntax?"

At this level, we only care about the structure of the language. We do not need an Abstract Syntax Tree (AST) or a transpiled output. Therefore, we only use the **Grammar Structure Keys**. The `ast` and `transpile` directives are not needed and can be omitted entirely.

#### Full Example: A Validation-Only Calculator Grammar

This grammar can successfully parse valid mathematical expressions, but it produces no useful output beyond "success" or "failure".

**`validation_calculator.yaml`**

```yaml
start_rule: expression

rules:
  expression:
    sequence:
      - { rule: term }
      - zero_or_more:
          sequence: [{ rule: _ }, { rule: add_op }, { rule: _ }, { rule: term }]

  term:
    sequence:
      - { rule: power }
      - zero_or_more:
          sequence:
            [{ rule: _ }, { rule: mul_op }, { rule: _ }, { rule: power }]

  power:
    sequence:
      - { rule: factor }
      - optional:
          sequence:
            [{ rule: _ }, { rule: power_op }, { rule: _ }, { rule: power }]

  factor:
    choice:
      - { rule: number }
      - sequence:
          - { literal: "(" }
          - { rule: _ }
          - { rule: expression }
          - { rule: _ }
          - { literal: ")" }

  add_op:
    choice: [{ literal: "+" }, { literal: "-" }]

  mul_op:
    choice: [{ literal: "*" }, { literal: "/" }]

  power_op:
    literal: "^"

  number:
    regex: "-?\\d+"

  _:
    regex: "[ \\t]*"
```

**Usage:**

```python
# Load the validation-only grammar
validator = Parser.from_file("validation_calculator.yaml")
result = validator.parse("((2 + 3) * 4) ^ 5")

if result['status'] == 'success':
    # The 'ast' key will contain a raw, messy parse tree, which we ignore.
    print("Input string is a valid expression!")
else:
    print(f"Invalid: {result['message']}")
```

---

### Use Case 2: Parsing to a Semantic AST

**Goal:** To validate the input string AND convert it into a clean, structured, and meaningful data representation (an AST).

This is the most common use case for parsing complex data. To achieve this, we build on the validation grammar by adding the **`ast` directive block** to our rules. This block tells the parser how to transform the raw parse tree into a clean AST by discarding whitespace, promoting nodes, and building specific structures. The `transpile` block is still not needed.

#### Full Example: An AST-Generating Calculator Grammar

We now add directives like `discard`, `promote`, `leaf`, and `structure` to produce a useful tree.

**`ast_calculator.yaml`**

```yaml
start_rule: expression

rules:
  expression:
    ast: { structure: "left_associative_op" }
    sequence:
      - { rule: term }
      - zero_or_more:
          sequence: [{ rule: _ }, { rule: add_op }, { rule: _ }, { rule: term }]

  term:
    ast: { structure: "left_associative_op" }
    sequence:
      - { rule: power }
      - zero_or_more:
          sequence:
            [{ rule: _ }, { rule: mul_op }, { rule: _ }, { rule: power }]

  power:
    ast: { structure: "right_associative_op" }
    sequence:
      - { rule: factor }
      - optional:
          sequence:
            [{ rule: _ }, { rule: power_op }, { rule: _ }, { rule: power }]

  factor:
    ast: { promote: true }
    choice:
      - { rule: number }
      - sequence:
          - { literal: "(" }
          - { rule: _ }
          - { rule: expression }
          - { rule: _ }
          - { literal: ")" }

  add_op:
    ast: { leaf: true }
    choice: [{ literal: "+" }, { literal: "-" }]

  mul_op:
    ast: { leaf: true }
    choice: [{ literal: "*" }, { literal: "/" }]

  power_op:
    ast: { leaf: true }
    literal: "^"

  number:
    ast: { leaf: true, type: "number" }
    regex: "-?\\d+"

  _:
    ast: { discard: true }
    regex: "[ \\t]*"
```

**Usage:**

```python
# Load the AST-generating grammar
parser = Parser.from_file("ast_calculator.yaml")
result = parser.parse("((2 + 3) * 4) ^ 5")

if result['status'] == 'success':
    print("Successfully parsed. AST:")
    # The result now contains a valuable 'ast' key with a clean tree
    print(json.dumps(result['ast'], indent=2))
```

**Output AST:**

```json
{
  "tag": "binary_op",
  "op": {
    "tag": "power_op",
    "text": "^",
    "line": 1,
    "col": 16
  },
  "left": {
    "tag": "binary_op",
    "op": {
      "tag": "mul_op",
      "text": "*",
      "line": 1,
      "col": 10
    },
    "left": {
      "tag": "binary_op",
      "op": {
        "tag": "add_op",
        "text": "+",
        "line": 1,
        "col": 5
      },
      "left": {
        "tag": "number",
        "text": "2",
        "line": 1,
        "col": 3,
        "value": 2
      },
      "right": {
        "tag": "number",
        "text": "3",
        "line": 1,
        "col": 7,
        "value": 3
      }
    },
    "right": {
      "tag": "number",
      "text": "4",
      "line": 1,
      "col": 12,
      "value": 4
    }
  },
  "right": {
    "tag": "number",
    "text": "5",
    "line": 1,
    "col": 18,
    "value": 5
  }
}
```

---

### Use Case 3: Full Transpilation

**Goal:** To validate the input, parse it to a clean AST, and then **transform that AST into a different string format** (e.g., from infix math to LISP-style s-expressions).

This is the full power of the pipeline. We use two separate grammar files: one for the `Parser` and one for the `Transpiler`. The `Parser` grammar defines the language and AST structure. The `Transpiler` grammar defines how to convert each AST node into an output string, including conditional logic.

#### Full Example: A Transpiling Calculator Grammar

First, we use the `ast_calculator.yaml` from Use Case 2 to produce the AST. Then, we create a new file to define the LISP transpilation rules.

**`lisp_transpiler.yaml`**

```yaml
rules:
  binary_op:
    template: "({op} {left} {right})"
  add_op:
    cases:
      - if: { path: "node.text", equals: "+" }
        then: "add"
      - default: "sub"
  mul_op:
    cases:
      - if: { path: "node.text", equals: "*" }
        then: "mul"
      - default: "div"
  power_op:
    value: "pow"
  number:
    use: "value"
```

**Usage:**

```python
from koine.parser import Parser, Transpiler

# Load the parser and transpiler grammars
parser = Parser.from_file("ast_calculator.yaml")
transpiler = Transpiler.from_file("lisp_transpiler.yaml")

# Run the pipeline
parse_result = parser.parse("((2 - 3) * 4) ^ 5")

if parse_result['status'] == 'success':
    print("Parse successful. Transpiling AST...")
    translation = transpiler.transpile(parse_result['ast'])
    print(f"Final Output: {translation}")
```

The parser produces the following AST, which becomes the input for the transpiler. Note the `op` node with `"text": "-"` which will be handled by the `cases` directive in the transpiler.

```json
{
  "tag": "binary_op",
  "op": { "tag": "power_op", "text": "^", "line": 1, "col": 17 },
  "left": {
    "tag": "binary_op",
    "op": { "tag": "mul_op", "text": "*", "line": 1, "col": 11 },
    "left": {
      "tag": "binary_op",
      "op": { "tag": "add_op", "text": "-", "line": 1, "col": 5 },
      "left": { "tag": "number", "text": "2", "line": 1, "col": 3, "value": 2 },
      "right": {
        "tag": "number",
        "text": "3",
        "line": 1,
        "col": 7,
        "value": 3
      },
      "line": 1,
      "col": 5
    },
    "right": { "tag": "number", "text": "4", "line": 1, "col": 13, "value": 4 },
    "line": 1,
    "col": 11
  },
  "right": { "tag": "number", "text": "5", "line": 1, "col": 19, "value": 5 },
  "line": 1,
  "col": 17
}
```

**Final Output:**

```
Final Output: (pow (mul (sub 2 3) 4) 5)
```

---

### Use Case 4: Lexer-Based Parsing and Stateful Transpilation

**Goal:** To handle context-sensitive syntax (like Python's indentation) and perform stateful transformations (like emitting `let` only for the first variable assignment).

This requires two new features:

1.  **The `lexer` block:** A top-level key in the parser grammar that defines tokens, offloading work like whitespace and comment handling from the main parser rules. It can also emit special `INDENT`/`DEDENT` tokens.
2.  **Stateful transpiler directives:** `state_set` and conditional `cases` that can check the transpiler's internal state.

#### Example: Transpiling Python to JavaScript

**`py_parser.yaml` (Snippet)**

```yaml
# A top-level key to define tokens
lexer:
  tokens:
    - { regex: "[ \\t]+", action: "skip" }
    - { regex: "\\n[\\t ]*", action: "handle_indent" } # Magic action
    - { regex: "def", token: "DEF" }
    - { regex: "return", token: "RETURN" }
    - { regex: "[a-zA-Z_][a-zA-Z0-9_]*", token: "NAME" }
    - { regex: "=", token: "EQUALS" }

start_rule: function_definition

rules:
  function_definition:
    # Grammar rules now match abstract tokens, not raw text
    sequence: [ { token: "DEF" }, { rule: identifier }, ... ]
  ...
```

Parsing the Python code with `py_parser.yaml` yields the following AST. This structure, with its named children like `iterator`, `limit`, and `body`, is what the transpiler will transform.

```json
{
  "tag": "function_definition",
  "children": {
    "name": { "tag": "NAME", "text": "f", "line": 1, "col": 5 },
    "params": {
      "tag": "parameters",
      "children": [
        { "tag": "NAME", "text": "x", "line": 1, "col": 7 },
        { "tag": "NAME", "text": "y", "line": 1, "col": 10 }
      ]
    },
    "body": [
      {
        "tag": "assignment",
        "children": {
          "target": { "tag": "NAME", "text": "a", "line": 2, "col": 5 },
          "value": { "tag": "NUMBER", "value": 0, "line": 2, "col": 9 }
        }
      },
      {
        "tag": "for_loop",
        "children": {
          "iterator": { "tag": "NAME", "text": "i", "line": 3, "col": 9 },
          "limit": { "tag": "NAME", "text": "y", "line": 3, "col": 21 },
          "body": [
            {
              "tag": "assignment",
              "children": {
                "target": { "tag": "NAME", "text": "a", "line": 4, "col": 9 },
                "value": {
                  "tag": "binary_op",
                  "left": { "tag": "NAME", "text": "a", "line": 4, "col": 13 },
                  "right": { "tag": "NAME", "text": "x", "line": 4, "col": 17 }
                }
              }
            }
          ]
        }
      },
      {
        "tag": "return",
        "children": {
          "value": { "tag": "NAME", "text": "a", "line": 5, "col": 12 }
        }
      }
    ]
  }
}
```

**`py_to_js_transpiler.yaml`**

```yaml
rules:
  function_definition:
    template: "function {name}({params}) {{\n{body}\n}}"
  NAME:
    use: "text"
  NUMBER:
    use: "value"
  parameters:
    join_children_with: ", "
    template: "{children}"
  suite:
    indent: true
    join_children_with: "\n"
    template: "{children}"
  assignment:
    cases:
      # If 'state.vars.{target}' does not exist...
      - if: { path: "state.vars.{target}", negate: true }
        # ...then use the template with 'let'
        then: "let {target} = {value};"
      # Otherwise, use the default template without 'let'
      - default: "{target} = {value};"
    # After transpiling, set a state variable to remember the assignment
    state_set: { "vars.{target}": True }
  for_loop:
    template: "for (let {iterator} = 0; {iterator} < {limit}; {iterator}++) {{\n{body}\n}}"
  return:
    template: "return {value};"
  binary_op:
    template: "{left} + {right}"
```

**Usage:**

```python
# Load the py_parser.yaml and py_to_js_transpiler.yaml grammars
# and instantiate the Parser and Transpiler.
parse_result = parser.parse(python_code)
if parse_result['status'] == 'success':
    translation = transpiler.transpile(parse_result['ast'])
    # The 'translation' variable now holds the transpiled JavaScript code.
```

**Result:** The transpiler correctly handles indentation and uses `let` only for the first assignment to each variable.

**Input Python Code:**

```python
def f(x, y):
    a = 0
    for i in range(y):
        a = a + x
    return a
```

**Output JavaScript Code:**

```javascript
function f(x, y) {
  let a = 0;
  for (let i = 0; i < y; i++) {
    a = a + x;
  }
  return a;
}
```

---

### Use Case 5: Transpiling to Indented Languages

**Goal:** To generate output that requires correct indentation, like Python.

This is accomplished in the transpiler grammar using `indent: true` on rules that represent a new indentation block, and `join_children_with: "\n"` to place child nodes on new lines.

#### Example: Transpiling JavaScript to Python

Parsing the JavaScript code with `js_parser.yaml` yields an AST. The `js_to_py_transpiler.yaml` will use the `indent` directive when processing the `statements` nodes to create a correctly indented Python code block.

```json
{
  "tag": "function_definition",
  "children": {
    "name": { "tag": "NAME", "text": "f" },
    "params": {
      "tag": "parameters",
      "children": [
        { "tag": "NAME", "text": "x" },
        { "tag": "NAME", "text": "y" }
      ]
    },
    "body": {
      "tag": "statements",
      "children": [
        {
          "tag": "assignment",
          "children": {
            "target": { "tag": "NAME", "text": "a" },
            "value": { "tag": "NUMBER", "value": 0 }
          }
        },
        {
          "tag": "for_loop",
          "children": {
            "limit": { "tag": "NAME", "text": "y" },
            "body": {
              "tag": "statements",
              "children": [
                {
                  "tag": "assignment",
                  "children": {
                    "target": { "tag": "NAME", "text": "a" },
                    "value": {
                      "tag": "binary_op",
                      "left": { "tag": "NAME", "text": "a" },
                      "right": { "tag": "NAME", "text": "x" }
                    }
                  }
                }
              ]
            }
          }
        },
        {
          "tag": "return",
          "children": {
            "value": { "tag": "NAME", "text": "a" }
          }
        }
      ]
    }
  }
}
```

**`js_to_py_transpiler.yaml`**

```yaml
rules:
  function_definition:
    template: "def {name}({params}):\n{body}"
  NAME:
    use: "text"
  NUMBER:
    use: "value"
  parameters:
    join_children_with: ", "
    template: "{children}"
  statements:
    indent: true
    join_children_with: "\n"
    template: "{children}"
  assignment:
    template: "{target} = {value}"
  for_loop:
    template: "for i in range({limit}):\n{body}"
  return:
    template: "return {value}"
  binary_op:
    template: "{left} + {right}"
```

**Usage:**

```python
# Load the js_parser.yaml and js_to_py_transpiler.yaml grammars
# and instantiate the Parser and Transpiler.
parse_result = parser.parse(js_code)
if parse_result['status'] == 'success':
    translation = transpiler.transpile(parse_result['ast'])
    # The 'translation' variable now holds the transpiled Python code.
```

**Result:** The transpiler correctly indents the body of the Python function.

**Input JavaScript Code:**

```javascript
function f(x, y) {
  let a = 0;
  for (let i = 0; i < y; i++) {
    a = a + x;
  }
  return a;
}
```

**Output Python Code:**

```python
def f(x, y):
    a = 0
    for i in range(y):
        a = a + x
    return a
```

---

## Documentation

For a complete guide to Koine's features, including detailed explanations and examples for every directive, please see the dedicated documentation files:

- **[PARSING.md](./PARSING.md)**: Covers everything related to parsing text into an AST, from grammar basics to advanced features like lookaheads and lexer-based tokenization.

- **[TRANSPILING.md](./TRANSPILING.md)**: Covers everything related to transforming an AST into text, including conditional logic, state management, and generating indented output.

### Native implementation and verification

A C++ runtime parser and transpiler with a C interface, Python `ctypes` wrapper
and Nim `importc` package are available under **[native/](native/README.md)**. Parser/AST, subgrammar and
placeholder checks pass against Koine and the shared SLIP grammars; native YAML
loading uses rapidyaml.
Native tokenization supports optional lexer blocks and indentation-sensitive
grammars; native transpilation covers templates, conditions, state and output
indentation. Public behavioral tests include complete Python↔JavaScript pipelines.
Nim C/C++ backend ownership checks pass under GCC 15 with leak detection enabled.
Linux and Windows verification is recorded in **[PLAN.md](PLAN.md)**. See
**[SPEC.md](SPEC.md)** for the agreed contracts and **[PLAN.md](PLAN.md)** for
milestones, verification evidence, and remaining work.

## Author

The Koine engine was developed by **Chris Bates**.

- https://github.com/chrsbats

## Acknowledgements

This project was made possible by the foundational ideas and early conceptual prototypes developed by **Adam Griffiths**. Their insights into creating a fully data-driven parsing pipeline were instrumental in shaping the final architecture and philosophy of Koine.

- https://github.com/adamlwgriffiths

## License

This project is licensed under the MIT License. See the `LICENSE` file for details.
