Metadata-Version: 2.4
Name: talyn
Version: 0.9.4
Summary: Talyn: A robust, stable, and realistically fast Zig-powered event loop for Python's asyncio.
Author-email: Chaiwat Suttipongsakul <cwt@bashell.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/cwt/talyn
Project-URL: Repository, https://github.com/cwt/talyn
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE.md
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-asyncio; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff; extra == "lint"
Requires-Dist: mypy; extra == "lint"
Provides-Extra: benchmark
Requires-Dist: uvloop; extra == "benchmark"
Requires-Dist: prettytable; extra == "benchmark"
Requires-Dist: matplotlib; extra == "benchmark"
Requires-Dist: pillow; extra == "benchmark"
Dynamic: license-file

# Talyn: Robust, Stable AsyncIO Event Loop for Python

[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE.md)
[![Python Compatibility](https://img.shields.io/badge/python-3.13%20%7C%203.14%20%7C%20Free--Threaded-blue.svg)](#-requirements)
[![Linux Compatibility](https://img.shields.io/badge/linux-7.0+%20%7C%20Fedora%2043--44-orange.svg)](#-requirements)
[![Zig Compatibility](https://img.shields.io/badge/zig-0.16.0-red.svg)](#-requirements)
[![PyPI Version](https://img.shields.io/pypi/v/talyn.svg)](https://pypi.org/project/talyn/)

**Talyn** is a robust, exceptionally stable, and realistically fast `asyncio` event loop drop-in replacement for Python, powered by the asynchronous capabilities of **Zig** and **io_uring**.

Talyn prioritizes **correctness, complete system safety, and high usability** over artificial micro-benchmark superiority. It is fully compatible with CPython's standard single-threaded and free-threaded (GIL-disabled) runtimes.

---

## 🚀 Features

- **Realistic Speed**: Designed to deliver solid and reliable I/O performance on Linux by leveraging `io_uring`'s native kernel-side asynchronous completion queues.
- **Robust & Crash-Resistant**: Meticulously hardened against circular reference memory leaks, stack alignment faults, signal interrupt deadlocks, and use-after-free bugs.
- **Full Asyncio Compatibility**: Passes 100% of standard Python `asyncio`, `subprocess`, `transports`, and connection-lifecycle test suites.
- **Offline AST Bug Hunter & Linter**: Enforces strict invariant safety rules (preventing memory leaks, discarded syscalls, panics, and UAFs) via a built-in, sub-15ms Zig and Python AST static analyzer (`zig build lint`).
- **Modern Packaging**: Fully migrated to PEP 517/518 standard declarative `pyproject.toml` configuration.
- **GIL-Disabled Free-Threading Ready**: Fully compatible with `python3.13t` and `python3.14t` without memory races.

---

## 📜 Requirements

- **Python**: `>= 3.13` (Tested and verified under CPython `3.13`, `3.14`, `3.13t` (free-threaded), and `3.14t` (free-threaded))
- **Linux Kernel**: `>= 7.0` (Verified on Linux Kernel `7.0.x`)
- **Zig Compiler** (for source builds): `0.16.0` (Fedora packages)

> [!NOTE]
> **Tested Platform Verification**:
> Talyn is primarily developed and tested under **Fedora 44** on an **x86_64**
> architecture equipped with an **Intel(R) Core(TM) Ultra 7 265** processor.
> Since v0.8.7 we also build and verify **aarch64** and **riscv64** wheels on
> the same x86_64 host: the native extension is cross-compiled by Zig, and the
> full test suite is executed inside Fedora 44 QEMU VMs (see
> [Linux Development](#-linux-x86_64-development-build-cross-compile--multi-arch-testing)
> below).

---

## 🔧 Installation & Testing

To compile and install Talyn locally on a native Linux machine, run:

```bash
pip install -e .
```

### 🍎 macOS Development & Testing via Podman (Apple Silicon)

Since Talyn is a Linux-only project leveraging `io_uring`, development and testing on macOS require running inside a Linux environment. 

We provide a frictionless, automated setup using **Podman** and a **Fedora 44 (AARCH64)** container:

1. **Install Podman**:
   ```bash
   brew install podman
   ```
2. **Initialize the Podman VM** (only needed once):
   ```bash
   podman machine init
   ```
3. **Run the test suite** (the script automatically starts the VM if it's stopped, builds the image, runs the tests with proper permissions, and stops the VM at the end if it started it):
   ```bash
   ./scripts/macos/run_tests.sh
   ```

You can pass any options supported by `test_all.sh` (e.g., `--verbose` or `--python=3.13`):
```bash
./scripts/macos/run_tests.sh --verbose --python=3.13
```

To force a rebuild of the Fedora testing image:
```bash
./scripts/macos/run_tests.sh --rebuild-image
```

### 📊 Benchmarking

We also provide a wrapper to run the benchmark suite inside the Fedora container using the optimized `ReleaseFast` target:
```bash
./scripts/macos/benchmark.sh --python=python3.13
```

You can target a specific benchmark using `--bench` (note that benchmark names with spaces must be quoted):
```bash
./scripts/macos/benchmark.sh --python=python3.13 --bench="task spawn"
```
The generated comparison plots will be automatically saved to `benchmarks/output/`.

### 📦 Building & Publishing Multi-Architecture Wheels

If you are developing on macOS (Apple Silicon) and need to build and publish wheels for **both `aarch64` and `x86_64`** architectures to PyPI, you can do so in a single command using Podman's emulation:

1. **Build all wheels**:
   This script builds a native `aarch64` container image and an emulated `x86_64` container image, runs `scripts/build.sh` inside both, and collects all 8 wheels (4 Python versions × 2 architectures) in the `./dist/` directory:
   ```bash
   ./scripts/macos/build_all_wheels.sh
   ```

2. **Publish to PyPI**:
   Upload the built distributions in `./dist/` using twine (pre-configured in your repository):
   ```bash
   ./scripts/publish.sh
   ```



## 🖥️ Linux (x86_64) Development: Build, Cross-Compile & Multi-Arch Testing

All of Talyn's development—including **aarch64** and **riscv64** wheels and full test-suite runs—can be done on an **x86_64 (Intel) Linux PC**, with no MacBook required:

- **Building**: Zig is a native cross-compiler, so foreign-architecture wheels and the extension are produced at full host speed — no QEMU involved.
- **Testing**: QEMU user-mode emulation (e.g. `podman run --platform linux/arm64`) **cannot** run Talyn, because `io_uring_setup` returns `ENOSYS` under user-mode emulation and Talyn requires `io_uring` with no fallback. Instead, we boot a **full-system Fedora 44 VM** per architecture, whose real guest kernel passes `io_uring` syscalls through to the host kernel.

### 📋 1. Fedora 44 required packages

Install Zig, all four Python interpreters with development headers, and the stdlib test suites:

```bash
sudo dnf install zig \
    python3.13 python3.13-devel python3.13-freethreading \
    python3.14 python3.14-devel python3.14-freethreading python3.14-freethreading-devel \
    python3.13-test python3.14-test python3.14-freethreading-test
```

Bootstrap pip and the test tooling for each interpreter:

```bash
for p in python3.13 python3.13t python3.14 python3.14t; do
    $p -m ensurepip --upgrade
    $p -m pip install --user pytest pytest-asyncio
done
```

For foreign-architecture VM testing (aarch64 / riscv64), additionally:

```bash
sudo dnf install qemu-system-aarch64 edk2-aarch64 qemu-system-riscv edk2-riscv64 \
    genisoimage openssh-clients
```

> [!NOTE]
> `python3.13-freethreading-test` is not packaged for Fedora 44; the stdlib asyncio modules for the free-threaded 3.13 build are skipped automatically by the test suite.

### 🏗️ 2. Build all wheels natively (x86_64 + aarch64 + riscv64)

```bash
./scripts/linux/build_all_wheels.sh
```

Uses Zig's native cross-compiler — no QEMU needed. Produces **12 wheels** (4 Python variants × 3 architectures) in `./dist/`, ready for `./scripts/publish.sh`.

### 🧪 3. Run the test suite natively (x86_64)

```bash
./scripts/test_all.sh                  # Debug build
./scripts/test_all.sh --starburst      # ReleaseFast build
```

### 🖥️ 4. Test foreign-architecture wheels in VMs

Boots a real Fedora 44 VM per architecture (first run downloads the ~0.5 GB cloud image to `~/.cache/talyn-*-vm/`) and runs the **full pytest suite against each installed wheel**:

```bash
./scripts/linux/run_tests.sh                     # aarch64 wheels
./scripts/linux/run_tests.sh --arch=riscv64      # riscv64 wheels
./scripts/linux/run_tests.sh --smoke             # quick import + event-loop smoke test only
./scripts/linux/run_tests.sh --shutdown          # stop the VM after the run
```

### ⚡ 5. Run the full `test_all.sh` suite in VMs (cross-compiled, fast)

`test_all.sh` normally compiles the extension *inside* the VM, which is slow under QEMU TCG. Instead, cross-compile the four extension variants **natively on the host** (the same build flags the wheels use) and run `test_all.sh --no-build` in the VM:

```bash
./scripts/linux/run_test_all.sh --arch=aarch64 --starburst
./scripts/linux/run_test_all.sh --arch=riscv64 --starburst
```

Under heavy emulation, a stdlib module may exceed its per-module timeout (e.g. `test_subprocess` on riscv64); raise it with `TALYN_STDLIB_TIMEOUT=600` and the memory-safety repro timeout with `TALYN_REPRO_TIMEOUT=900`.

### 🛡️ 6. Run the native Offline AST Linter & Bug Hunter

Talyn includes a zero-dependency static analyzer hooked into `std.zig.Ast` and Python's `ast` module to prevent regressions and enforce architectural safety rules in under 15ms:

```bash
zig build lint
```

For the complete catalog of rules, see [docs/development/ast-linter.md](docs/development/ast-linter.md).

### ⏱️ Measured timings

`scripts/test_all.sh --starburst` (4 Python variants: pytest suite + stdlib asyncio suite):

| Environment | Total time |
|---|---|
| x86_64 (native) | **8m28s** |
| aarch64 VM, in-VM build | 42m33s |
| riscv64 VM, in-VM build | 44m56s |
| aarch64 VM, cross-built (`run_test_all.sh`) | **17m47s** |
| riscv64 VM, cross-built (`run_test_all.sh`) | **24m48s** |

All configurations pass the pytest suite; under QEMU TCG the emulated runs are ~5× slower than native, and the cross-built runs cut the in-VM build phase (~2.4× speedup on aarch64). An occasional stdlib-asyncio module may time out under heavy emulation (see note above about `TALYN_STDLIB_TIMEOUT`).



### ⚡ Optimization & Target Compilation (For Developers & Power Users)

By default, the pre-built wheels generated by `build.sh` are compiled targeting a **generic `x86_64`** CPU architecture baseline to ensure 100% universal compatibility across all 64-bit modern x86 Linux processors (e.g., matching standard PyPI `manylinux` wheel compatibility). 

Based on our benchmarks and comprehensive validation—including 100% passing results in the standard `asyncio` test suite across four distinct Python versions—**we publish the official binary packages (wheels) compiled in Starburst mode (Zig built with `ReleaseFast`)** to deliver peak performance out of the box with proven stability.

If you are compiling from source, you can customize the compilation optimize mode and target CPU architecture to unleash maximum performance:

#### 1. Compile and Optimize for your Native CPU (Highly Recommended)
To compile Talyn so that it takes full advantage of your host's exact CPU instructions (such as AVX2, AVX-512, cache alignment, etc.):
```bash
# Omit TALYN_CPU so Zig defaults to native CPU optimization
TALYN_OPTIMIZE=ReleaseFast pip install .
```

#### 2. Configure Compilation Modes
You can control Zig's optimize mode by setting the `TALYN_OPTIMIZE` environment variable (defaults to `Debug` for developer convenience):
* `TALYN_OPTIMIZE=Debug` (Default): Compiles with heavy runtime assertions and debug symbols.
* `TALYN_OPTIMIZE=ReleaseSafe`: Compiles with full optimizations but keeps safety checks (e.g. out-of-bounds, overflows).
* `TALYN_OPTIMIZE=ReleaseFast`: Compiles with maximum optimizations (Starburst mode), disabling safety checks for peak execution speed.

#### 3. Target a Specific CPU microarchitecture
You can force Zig to compile for a specific target CPU microarchitecture (like `x86_64_v2` or `x86_64_v3`):
```bash
TALYN_OPTIMIZE=ReleaseFast TALYN_CPU=x86_64_v3 pip install .
```

---

## 📦 Usage

### Basic Usage

To run a coroutine directly with the Talyn event loop:

```python
import talyn
import asyncio

async def main():
    print("Hello from Talyn!")
    await asyncio.sleep(1)
    print("Goodbye from Talyn!")

# Run using Talyn event loop
talyn.run(main())
```

### Graceful Fallback & Hardening Check

For production configurations, you may want to support fallback options. If Talyn is not installed, or if the host Linux kernel does not meet Talyn's safety baseline (monitored by the `HARD-01` Kernel Version Guard which requires kernel `>= 6.0`), the code will gracefully fall back to `uvloop` (if available), and finally to standard Python `asyncio`:

```python
import asyncio

# 1. Try to initialize Talyn (performs HARD-01 kernel version check)
try:
    import talyn
    talyn.install()
except (ImportError, RuntimeError):
    # 2. Fallback to uvloop if Talyn is missing or kernel version is too old
    try:
        import uvloop
        uvloop.install()
    except (ImportError, AttributeError):
        # 3. Fallback to standard Python asyncio (do nothing)
        pass

async def main():
    print("Running with the best available event loop!")

asyncio.run(main())
```

---

## 💝 Historical Credits & Origin

Talyn is spun off from **[Leviathan](https://github.com/kython28/leviathan)**, an event loop originally pioneered by **[Enrique Mora](https://github.com/kython28)**. Enrique Mora's creative spark and vision of merging Zig, `io_uring`, and `asyncio` laid the critical foundation and architecture of this project.

As Talyn evolved, the implementation underwent a complete systems-level refactoring to transition from a theoretical prototype to a production-grade, crash-resistant runtime:

- Eliminated multi-crossing Zig/Python vectorcall overhead by implementing a fused scheduler step trampoline in pure Zig.
- Redesigned completion handlers into flat, GC-safe ring buffers.
- Fully audited and resolved all memory-leak reference cycles under concurrent connections.

To honor the project's roots and Enrique's early work:

- The original Leviathan README can be viewed at: [docs/historical/leviathan-readme.md](docs/historical/leviathan-readme.md)
- The original preliminary benchmarks can be viewed at: [docs/historical/leviathan-benchmark.md](docs/historical/leviathan-benchmark.md)

---

## 📖 Project Story

- **[Development Journey](docs/development/development-journey.md)** — The full story: from discovery to challenges, the shift from "ultra-fast" to "realistic fast and stable", and how Talyn was built.
- **[Why Talyn?](docs/development/talyn-naming.md)** — The personal story and meaning behind the new name.

---

## 📄 License

This project is licensed under the MIT License. See [LICENSE.md](LICENSE.md) for details.
