Metadata-Version: 2.4
Name: inferlet
Version: 0.3.0
Summary: Python SDK for writing Pie inferlets
Author: Pie Project
License: Apache-2.0
Keywords: inference,inferlet,llm,pie,wasm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Requires-Dist: jinja2>=3.0
Provides-Extra: build
Requires-Dist: componentize-py>=0.19; extra == 'build'
Provides-Extra: dev
Requires-Dist: componentize-py>=0.19; extra == 'dev'
Requires-Dist: mypy>=1.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1; extra == 'dev'
Description-Content-Type: text/markdown

# inferlet Python SDK

Python API for writing Pie inferlets.

```python
from inferlet import Context, Model, Sampler, runtime

async def main(input: dict) -> str:
    model = Model.load(runtime.models()[0])
    ctx = Context(model)

    ctx.system("You are helpful.").user(input["prompt"])

    return await ctx.generate(
        Sampler.top_p(0.6, 0.95),
        max_tokens=256,
    ).collect_text()
```

## Main pieces

- `Context`: owns KV-cache state and chat/raw token buffers.
- `Forward`: runs one explicit forward pass with samplers, probes, masks, and
  manual page control.
- `Generator`: multi-step generation loop with stop conditions, constraints,
  speculation, adapters, and JSON collection.
- `chat`, `reasoning`, `tools`: optional decoders and helpers for model-native
  formats.
- `runtime`, `session`, `messaging`, `mcp`: host services exposed to inferlets.

## Build notes

Python inferlets are packaged as Wasm components. Pure-Python dependencies can
be bundled; native extensions such as `numpy`, `orjson`, `msgspec`, or
`pydantic_core` cannot be loaded in the Wasm runtime.

Build through Bakery:

```bash
pie build ./my-python-inferlet -o out.wasm
```

For constrained decoding details, see
[`sdk/CONSTRAINED_DECODING.md`](../CONSTRAINED_DECODING.md).
