Metadata-Version: 2.4
Name: metac
Version: 0.0.2
Summary: C metaprogramming & code generation package
Author-email: "Oleg A. Khlybov" <fougas@mail.ru>
License-Expression: BSD-3-Clause
Project-URL: Homepage, https://github.com/okhlybov/metac
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: C
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Software Development :: Libraries
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# metac: C metaprogramming & code generation package for Python

[![CI](https://github.com/okhlybov/metac/actions/workflows/ci.yml/badge.svg)](https://github.com/okhlybov/metac/actions/workflows/ci.yml)

<div align="right"><i>/* Automagically generated by metac $your_favorite_version */</i></div><br><br>

`metac` is a general-purpose template engine targeting plain ANSI C — simply put, it is to C what templates are to C++. Generic types are written once in Python against type parameters; the engine specializes each definition for the concrete type arguments a module actually uses and emits ordinary C source files. Using the same monomorphization technique a C++ compiler performs, every (definition, type arguments) pair is stamped out at generation time as a distinct, fully typed implementation. Nothing generic survives into compilation: the C compiler only ever sees plain, C++-safe ANSI C code.

Alongside the template engine, `metac` provides a comprehensive standard library of containers and utility types — a functional analog of the C++ STL. It spans direct-access and linked sequences, strings and string buffers, hashed, intrusive, and ordered sets and maps, bit sets, records, tagged variants, and raw or reference-counted references. Custom types are first-class citizens: implement the value protocol in Python for complex custom types to work seamlessly as elements and fields anywhere in the catalog, intrusive containers included.

What holds everything together is uniform value semantics with the uniform treatment of primitive and composite types. Every type — from int to a nested map of records — carries structured lifetime and maintenance operations, with capabilities derived automatically from its constituent parts and only valid operations emitted. Primitives realize the protocol as inline expressions while composites use generated extern or inline functions, keeping call sites uniform. Owing to free composition of build-in and custom types a list of vectors of records is just another type definition. Ever dreamt of having `std::map<std::string, std::list<int>>` in plain C? So here it is.

## Why metac?

C is notorious for its complete lack of complex data types in the standard library, and the usual workarounds each hurt in a familiar way:

- Macro-based generics (typeof tricks, token pasting) are opaque, hard to debug, and explode into incomprehensible errors.

- `void*`-based generic libraries abandon type safety, incur pointer-chasing overhead, and obscure ownership semantics (who copies? who destroys?).

- Hand-writing one container per type works, but is tedious, error-prone, and never quite finished.

`metac` takes a fourth road: containers are written once against a strict value-semantics protocol, and the generator stamps out the exact C code that would have otherwise written manually (arguably better), with none of the macro or `void*` compromises.

- **Zero dependencies:** generated code targets strict ANSI C and requires nothing beyond the C standard library — just #include the header and compile the generated sources along with consuming code.

- **Build-system native:** provides CMake, SCons, and Meson integrations that automatically re-run the generator whenever definitions change, so specialized code never goes stale.

- **Battle-tested output:** the project validates its generated test suites across MSVC, GCC, Clang, along with a bunch of second tier C compilers.

- **Bolierplate control:** the generator is designed to deal with large amount of generated C code with its ability to distribute the code over multiple translation units to sport parallel compilation and to perform the smart generation to avoid triggering unneccessary recompilations of unaltered code.

__This project is the successor to the [AutoC](https://github.com/okhlybov/autoc) package for Ruby.__

## Quick start

Requires Python ≥ 3.10. `metac` itself has zero dependencies.

**1. Define a module** — e.g. `mymodule.py`:

```python
from metac.module import Module
from metac.vector import Vector
from metac.chained_hash_map import Map
import metac.std as std

with Module("mymodule") as m:
  m.add(Vector("int_vector", "int"))
  m.add(Map("str2double", "const char*", "double"))
```

**2. Generate** — run the script; `metac` writes `mymodule_auto.h` / `mymodule_auto.c` (splitting into several sources when a module grows too large for one translation unit).

**3. Use** — from C:

```c
#include "mymodule_auto.h"

int_vector v;
int_vector_create(&v);
int_vector_set(&v, 0, 42);
assert(int_vector_get(&v, 0) == 42);

/* iterate */
int_vector_range r;
for(r = int_vector_range_new(&v); !int_vector_range_empty(&r);
    int_vector_range_move_front(&r)) {
  printf("%d\n", int_vector_range_front(&r));
}

int_vector_destroy(&v);
```

Every generated type speaks the same vocabulary of value semantics:

```c
void  T_create (T* target);              /* default construction        */
void  T_destroy(T* target);              /* release resources           */
void  T_copy   (T* target, const T* src);
int   T_equal  (const T* left, const T* right);
int   T_compare(const T* left, const T* right);   /* when orderable    */
size_t T_hash  (const T* target);                 /* when hashable     */
```

plus the container-specific operations (`set`/`get`/`view`/`contains` for sequences and sets, `put`/`remove` for sets, `get`/`set`/`view` for maps, `empty`/`size`, forward ranges for iteration, …).

## Tutorial: a complete project, end to end

Let's build a small but real program — a sensor log that accumulates readings into a **list of ints**, aggregates them and drains them — from an empty directory to a running binary.

### 1. Install the generator

From a checkout of this repository (Python ≥ 3.10 required):

```sh
pip install .
```

or as a package from remote PyPi repository:

```sh
pip install metac
```

### 2. Scaffold the project

`metac` provides project scaffolders for CMake, SCons, and Meson:

```sh
python -m metac.cmake readings   # CMake
# or: python -m metac.scons readings   # SCons
# or: python -m metac.meson readings   # Meson
# or: python -m metac.sample readings  # generic, defaults to CMake (-b meson / -b scons)
cd readings
```

This sets up a working project with the generator already wired in. For CMake:

```
readings/
├── CMakeLists.txt            # build + regeneration rules
├── CMakePresets.json         # debug/release presets
├── cmake/
│   └── MetaC.cmake           # add_metac_module() helper
├── readings.py               # type definitions — the generator script
├── readings.c                # your application
├── readings.code-workspace
├── .vscode/launch.json
└── .gitignore
```

For SCons, the scaffolder generates `SConstruct`, `site_scons/site_tools/metac.py`, and `.metac/scons.site`.
For Meson, it generates `meson.build`, `meson.options`, and `.metac/meson.site`.

Re-running a scaffolder into an existing project refuses to overwrite any file (existing ones are listed and left untouched). Passing `--upgrade` refreshes every scaffolding file — including the site-local `.metac/*.site` configuration, so a project relocated to a new checkout or machine gets reseeded — while your own `readings.py` module and `readings.c` main source are never touched:

```sh
python -m metac.cmake readings --upgrade   # refresh support files, keep your code
```

The project name doubles as the module name: `readings.py` will generate `readings_auto.h` / `readings_auto.c`, and CMake will build them as the `readings-auto` library your executable links against.

### 3. Define the container

Replace `readings.py` with:

```python
import metac.core
import metac.module
import metac.cmake
from metac.list import List

# CamelCase naming (IntListPushFront) is the default; switch to the
# snake_case style used throughout the test suite and this document:
metac.core.decorator = metac.core.snake_decorator

with metac.module.Module("readings") as m:
  m.add(List("int_list", "int"))

metac.cmake.CMake(m)   # emit the CMake fragment wiring the generated sources
```

### 4. Use it from C

Replace `readings.c` with:

```c
#include <stdio.h>
#include "readings_auto.h"

int main(void) {
  int_list readings;
  int_list_create(&readings);

  /* readings arrive one at a time, newest first */
  int samples[] = {23, 21, 25, 24, 27, 22, 26};
  for(size_t i = 0; i < sizeof(samples) / sizeof(samples[0]); ++i) {
    int_list_push_front(&readings, samples[i]);
  }

  /* aggregate via the range interface */
  size_t count = int_list_size(&readings);
  long sum = 0;
  int max = 0;
  for(int_list_range r = int_list_range_new(&readings);
      !int_list_range_empty(&r);
      int_list_range_move_front(&r)) {
    int v = int_list_range_front(&r);
    sum += v;
    if(v > max) max = v;
  }
  printf("count=%zu sum=%ld avg=%.1f max=%d\n",
         count, sum, (double)sum / (double)count, max);

  /* membership test */
  printf("seen 25: %s\n", int_list_contains(&readings, 25) ? "yes" : "no");

  /* drain, consuming the stored values one by one */
  while(!int_list_empty(&readings)) {
    printf("consumed %d\n", int_list_pop_front(&readings));
  }

  int_list_destroy(&readings);
  return 0;
}
```

### 5. Build and run

With CMake:

```sh
cmake --preset ninja-Debug
cmake --build --preset ninja-Debug
./build/ninja-Debug/readings
```

The scaffolder offers single-config `ninja-Debug`/`ninja-Release` presets on every platform, `make-Debug`/`make-Release` on POSIX, and `vs2022`/`vs2026` Visual Studio presets on Windows (where the binary is `readings.exe`). The undecorated `ninja` preset is the multi-config Ninja one - all configurations in a single `build/ninja` directory, with Debug as the default and the rest one `--config` away.

With SCons:

```sh
scons
./readings
```

With Meson:

```sh
meson setup build
meson compile -C build
./build/readings
```

Output:

```
count=7 sum=168 avg=24.0 max=27
seen 25: yes
consumed 26
consumed 22
consumed 27
consumed 24
consumed 25
consumed 21
consumed 23
```

(The values come back in reverse order of insertion — a singly-linked list is pushed and popped at the front.)

### What just happened

- Running `readings.py` rendered `readings_auto.h` / `readings_auto.c` — plain C, no runtime — plus `readings.cmake` and `readings.state` (digests of what was generated).
- `add_metac_module()` in `CMakeLists.txt` registered the generator as a proper build step: it re-runs `readings.py` whenever the definitions change, so generated code never goes stale.
- `readings-auto` is an object library; your executable just links it and includes the header.
- The workflow from here: add more types to the module (a `Map` for calibration constants, an intrusive hash set for deduplication, a `Record` for readings with timestamps, …), rerun the build, and use the generated functions — nothing else in your C code changes.

## Principles

**Plain C, zero runtime.** The deliverable is source code you can read, grep, step through in a debugger and ship. No library to link, no ABI to track, no preprocessor machinery to fight. Regenerating is cheap; the code is meant to be regenerated, not edited.

**Type-safe by monomorphization.** Each container is specialized to its element type at generation time. No `void*`, no casts, no element-size parameters — `int_vector_set` takes an `int`, `str2double_set` takes a `const char*` and a `double`. Passing conventions are chosen per element kind: primitives cross API boundaries by value, composites by pointer-to-const.

**One protocol for all types.** Primitives and composites are not two worlds. Every type — from `int` to a nested `Map` of records — carries the same operations (`create`/`destroy`/`copy`/`equal`/`compare`/`hash`), so containers compose freely: a `List` of `Vector`s of records just works. Primitives realize the protocol as inline expressions (`target = source`, `left == right`, a cast for hashing); composites realize it as generated functions. The difference never leaks to the call sites.

**Capabilities are derived, dead code is elided.** Types carry traits — constructible, destructible, copyable, comparable, orderable, hashable. Composites derive them structurally from their fields (a record is destructible if any field is; copyable only if all are), and the generator emits only what the traits permit: a `Vector("int_vector", "int")` contains no element-destruction code at all, and an unordered container never grows a comparison operator.

**Custom types are first-class citizens.** Your own composites join the system by implementing the same protocol in Python — then they work as element or field types everywhere, including inside intrusive containers. Your code is injected as C expressions/functions at well-defined hook points, wrapped so it cannot break operator precedence or naming conventions.

**Intrusive variants trade value space for layout.** The intrusive hash set/map store elements in one flat allocation of exactly the element type — no wrapper entries, no tag arrays — with slot states (empty/deleted) encoded as sentinel *values* you choose. If your element type can reserve two distinguishable states (e.g. `INT_MIN`/`INT_MAX` for an `int`, or low pointer values for handles), you get the most cache-friendly layout possible; if it cannot, use the regular non-intrusive containers.

**Deterministic builds.** Generation is integrated with CMake via `add_metac_module()`, SCons via `env.MetaCModule()`, and Meson via `custom_target`: output digests are tracked, code is regenerated only when definitions actually change, and every generated artifact is declared as a proper build dependency. In the stateful mode the on-disk contents of the generated files are never consulted: post-generation drift (a pretty printer reformatting them, for example) is preserved, and a regeneration pass rewrites a file only when its generated contents themselves change. A deleted generated file is recovered by the next build, which detects the absence and regenerates it; a damaged one, whose contents are disregarded, is recovered by deleting it first.

**Reference-counted memory as an option, not a mandate.** `Counted` references provide automatic reference counting for shared objects — including when used as container elements — while `Raw` references give you plain unmanaged pointers. Both speak the same protocol as every other type.

## Container catalog

| Module | Type | Notes |
|---|---|---|
| `metac.vector` | `Vector` | direct-access sequence, direct-access range |
| `metac.array` | `Array` | fixed-size stack-allocated contiguous C array, direct-access range, zero heap allocation |
| `metac.packet` | `Packet` | fixed-capacity stack-allocated vector-like packet (tuple-record variant storage), direct-access range, zero heap allocations |
| `metac.circular_buffer` | `Static`, `Dynamic` | fixed-capacity stack or dynamic heap circular ring buffer, direct-access range, sortable (opt-out) |
| `metac.tiered_vector` | `Vector` | chunked append-optimized direct-access buffer — amortized O(1) push, stable addresses, direct-access range |
| `metac.list` | `List` | singly-linked sequence, forward range |
| `metac.deque` | `Deque` | doubly-linked sequence, bidirectional range |
| `metac.stack` | `Stack` | LIFO adapter over `List`, forward range |
| `metac.queue` | `Queue` | FIFO adapter over `Deque`, forward range |
| `metac.priority_queue` | `Queue` | binary heap — guaranteed O(log n) push/pop, top = greatest element, duplicate priorities allowed |
| `metac.string` | `String` | string as an index→character map, direct-access range, variadic formatted output |
| `metac.string_buffer` | `Buffer` | append-optimized string buffer with scratch accumulation and lazy joining |
| `metac.chained_hash_set` | `Set` | bucket-chaining hash set — no sentinel values, stable element references, safe default |
| `metac.intrusive_hash_set` | `Set` | flat, sentinel-based open-addressing hash set |
| `metac.treap_set` | `Set` | treap — ordered set, O(log n) expected, iterates in sorted order, bidirectional range |
| `metac.rb_set` | `Set` | red-black tree — ordered set, guaranteed O(log n) height, <= 3 rotations on delete, bidirectional range |
| `metac.avl_set` | `Set` | AVL tree — ordered set, strictly balanced height <= 1.44 log2(n), fastest lookups, bidirectional range |
| `metac.flat_set` | `Set` | contiguous sorted dynamic array with binary search lookup, cache-friendly layout |
| `metac.chained_hash_map` | `Map` | bucket-chaining hash map over internal entry set |
| `metac.intrusive_hash_map` | `Map` | flat, sentinel-based open-addressing hash map over internal entry set |
| `metac.tree_map` | `Map` | generic binary search tree map parameterized by tree set backend, bidirectional range |
| `metac.flat_map` | `Map` | ordered map over a contiguous sorted array of key-value pairs (AoS layout), bidirectional range |
| `metac.flat_multiset` | `Set` | contiguous sorted dynamic array multiset with binary search and duplicates preserved |
| `metac.counter` | `Counter` | multiset frequency counter tracking element multiplicities over an explicit map backend |
| `metac.flat_multimap` | `Map` | ordered multimap over a contiguous sorted array of key-value pairs (AoS layout) with duplicate keys preserved |
| `metac.multimap` | `Map` | generic multimap mapping keys to multiple values, parameterized by a set and collection container |
| `metac.bit_array` | `Array` | fixed-size inline bit array, set algebra, popcount, zero heap allocation |
| `metac.bit_vector` | `Vector` | dynamically resizable packed bit vector, amortized O(1) push/pop, set algebra |
| `metac.record` | `Record` | user-defined field aggregates |
| `metac.variant` | `Variant` | tagged union / sum type over alternative types |
| `metac.reference` | `Raw`, `Counted` | unmanaged / reference-counted handles |

## Build system integration

### CMake

```cmake
include(MetaC)   # from the generated cmake/ directory

add_metac_module(
  mymodule
  DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
  MAIN_DEPENDENCY ${CMAKE_CURRENT_SOURCE_DIR}/mymodule.py
  COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/mymodule.py mymodule
)

target_link_libraries(myapp mymodule-auto)   # header + generated sources
```

The module bootstraps itself on first configure and re-runs the generator only when `mymodule.py` (or its stated dependencies) change. See `test/CMakeLists.txt` for a working example.

### SCons

```python
env = Environment(tools=["default", "metac"], toolpath=["site_scons/site_tools"])

metac_mod = env.MetaCModule("mymodule", directory=".")
env.Program(target="myapp", source=["myapp.c"] + metac_mod.sources)
```

The SCons tool checks file timestamps against `.metac/mymodule.state` to trigger regeneration only when definitions change.

### Meson

```meson
# In meson.build
module_gen = custom_target('mymodule-generate',
  input: 'mymodule.py',
  output: 'mymodule.stamp',
  command: [py, '@INPUT@', 'mymodule', '@OUTPUT@'],
  build_by_default: true,
)
metac_dep = declare_dependency(
  sources: [metac_sources, module_gen],
  include_directories: include_directories('.')
)
executable('myapp', sources: ['myapp.c'], dependencies: [metac_dep])
```

## How it tests itself

The repository's test suite is written the way users write code: Python scripts in `src/metac/test/` define container instances and their unit tests, and `metac` generates a C test suite from them — `test_auto.h` plus `test_auto*.c`, currently 18 translation units and some 62,000 lines — that is compiled and executed as part of the build. The generator is only as good as the C it produces, so that is what gets tested.

A second, programmatic suite (`python -m metac.autotest`) derives its tests from the generator's own type hierarchy over a curated matrix of containers, element kinds, map backends and b-tree orders. [Continuous integration](.github/workflows/ci.yml) regenerates and runs both suites on every push across GCC, Clang, AppleClang and MSVC, exercises the CMake/SCons/Meson project scaffolds, and checks the distribution with `twine`.

## License

BSD 3-Clause — see [LICENSE](LICENSE).

## The end

Happy hacking time and have fun!

Oleg A. Khlybov <fougas@mail.ru>
