Metadata-Version: 2.4
Name: myel
Version: 0.2.4
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Environment :: GPU :: NVIDIA CUDA
Summary: Myelin — bio-mimetic dynamic layer-masking local LLM engine (Python bindings)
Keywords: llm,llama,inference,cuda,local-ai,layer-masking
Home-Page: https://myelin.firattunaarslan.me
License: Proprietary - All Rights Reserved
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://myelin.firattunaarslan.me

# myelin — Python bindings

First-class [PyO3](https://pyo3.rs) bindings for the Myelin engine: a
bio-mimetic, dynamic layer-masking local LLM runtime. `import myelin` loads a
native extension module — no ctypes, no manual signatures, no server process.

- **Local**: the model runs on your GPU; nothing leaves your machine.
- **Lean**: weights live in VRAM int8-quantized; host RAM stays near the
  embedding table (~1.6 GB for Llama 3.2 1B).
- **Bio-mimetic masking**: per-layer importance is calibrated from the model
  itself at load; under hardware pressure (or an explicit `quality` budget)
  unimportant layers are skipped for speed.

## Install

```bash
pip install myel
```

The PyPI distribution name is `myel`; the importable module is `myelin`
(`import myelin`) — the two differ because `myelin` was already taken.

## Quick start

```python
import myelin

eng = myelin.Engine()                      # returns immediately; the default
                                           # model loads in the background
eng.wait_ready()                           # optional — generate() also waits

print(eng.generate("The capital of France is", max_tokens=32))
print(eng.chat("Explain transformers in one sentence."))

for piece in eng.stream_iter("Write a haiku about GPUs."):
    print(piece, end="", flush=True)
```

First run with no model on disk:

```python
eng = myelin.Engine("llama-3.2-1b-instruct", download=True)   # ~2.5 GB, once
```

## API

### Engine

```python
Engine(model=None, *, download=False, models_dir=None, profile=None)
```

- `model` — catalog id to activate; blocks until loaded. Without it the
  default model auto-loads in the background.
- `download` — with `model`, fetch it first if missing.
- `models_dir` — model library root (default `MYELIN_MODELS_DIR` or `./models`).
- `profile` — `"quality"` (uniform int8, default) or `"speed"` (calibrated
  mixed int8/int4, ~25% faster at a small perplexity cost).

### Generation

```python
eng.generate(prompt, max_tokens=128, *, temperature, top_k, top_p,
             repeat_penalty, seed, quality=1.0) -> str
```

Raw completion (no chat template). Sampling defaults are the Llama-family
chat settings (temperature 0.7, top-k 40, top-p 0.95, repeat penalty 1.1);
pass `temperature=0.0` for greedy decoding, `seed=` for reproducibility.

`quality` is the decision engine's quality floor: `1.0` runs every layer;
lower values let the bio-mimetic mask skip calibrated-unimportant layers.

```python
eng.chat(messages, max_tokens=256, *, system=None, ...) -> str
```

Chat completion through the Llama 3 instruct template. `messages` is a plain
string (single user turn) or a full conversation:

```python
history = [
    {"role": "user", "content": "My name is Ada."},
    {"role": "assistant", "content": "Nice to meet you, Ada!"},
    {"role": "user", "content": "What's my name?"},
]
eng.chat(history, system="Answer briefly.")
```

### Streaming

```python
eng.stream(messages, callback, max_tokens=256, ...) -> str   # callback per fragment
for piece in eng.stream_iter(messages, ...):                 # or iterate
    ...
```

`stream(..., inspect=True, on_activity=fn)` additionally calls
`fn(layer_norms, depth)` per token with the real per-layer activation — the
"neuron liveness" signal (slower; deliberate).

### Introspection

```python
text, info = eng.generate_detailed(prompt, ...)
info["active_layers"]   # per-layer bool mask this generation ran with
info["depth"]           # how many layers were active
info["policy"]          # layer-selection policy (e.g. BIOMIMETIC)

eng.hardware()          # {'vram_used_mb', 'temperature_c', 'utilization_pct', ...}
```

### Model library

```python
eng.list_models()       # curated catalog + download/active/loading status
eng.download_model("llama-3.2-3b-instruct", lambda p: print(p["downloaded_bytes"]))
eng.load_model("llama-3.2-3b-instruct")     # hot-swap; frees old model's VRAM
eng.active_model()      # -> str | None
eng.is_ready()          # -> bool
eng.wait_ready(timeout=None)
```

### Errors

Engine-level failures raise `myelin.MyelinError` (a `RuntimeError` subclass);
argument mistakes raise `TypeError`/`ValueError`.

## Notes

- The GIL is released during Rust-side compute, so generations never block
  other Python threads; callbacks re-acquire it per fragment — keep them light.
- One `Engine` per process is the intended pattern (it owns the GPU state).
  It is thread-safe; generations serialize on the single GPU.
- The catalog is curated (Llama 3.2 1B/3B Instruct) — the runtime implements
  the Llama 3.x architecture only.

