Metadata-Version: 2.5
Name: baretile
Version: 0.0.3
Summary: A compiler and runtime for Tenstorrent Blackhole, written in Python
Project-URL: Homepage, https://baretile.com
Project-URL: Documentation, https://github.com/baretile/baretile/blob/main/docs/README.md
Project-URL: Source, https://github.com/baretile/baretile
Project-URL: Issues, https://github.com/baretile/baretile/issues
Author: alkinun
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: blackhole,compiler,driver,machine learning,runtime,tensix,tenstorrent
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Compilers
Classifier: Topic :: System :: Hardware :: Hardware Drivers
Requires-Python: >=3.12
Provides-Extra: models
Requires-Dist: numpy; extra == 'models'
Description-Content-Type: text/markdown

# baretile

[![test](https://github.com/baretile/baretile/actions/workflows/test.yml/badge.svg)](https://github.com/baretile/baretile/actions/workflows/test.yml)

baretile is a compiler and runtime for Tenstorrent Blackhole, written in Python.
You write tensor operations, and it compiles them into Tensix instructions. It
also handles the PCIe access needed to load and run those programs on the chip.

It runs on Tenstorrent's ttsim simulator. We haven't tested it on a card yet.

The code in `baretile/` uses only the Python standard library. The driver runs
in userspace, and baretile loads its own firmware onto the workers. It needs
Tenstorrent's board firmware for ARC and Ethernet. There's no dependency on
tt-metal, tt-umd or LLK.

## Try it

You'll need Linux, Python 3.12 or later, and clang with a RISC-V target and lld
(`apt install clang lld` on Debian and Ubuntu). Install baretile, download the
simulator and check the setup:

```sh
python3 -m pip install baretile
python3 -m baretile.chip ttsim
python3 -m baretile.chip check
```

`chip ttsim` downloads the Blackhole library of
[ttsim](https://github.com/tenstorrent/ttsim) v1.11.5, the release baretile is
tested with, into baretile's cache. With no card on the PCI bus, baretile uses it.
`chip check` says what's missing and how to fix it. Then run the first two
commands:

```sh
python3 -m baretile.chip
python3 -m baretile.hello
```

With 120 Tensix tiles, `baretile.hello` prints:

```text
119 Tensix workers computed fib(30) = {832040}, and one tile dispatched the work
```

baretile needs only the standard library. The models and the server also need
NumPy, so install `'baretile[models]'` for those. The ramps and the tests are in
the repository.

To use another ttsim build, such as
[ttsim-chaos](https://github.com/baretile/baretile/blob/main/docs/testing.md#ttsim-chaos),
set `TTSIM=/absolute/path/to/libttsim.so`. It comes before a card and before the
downloaded release. On a machine with a Blackhole and no `TTSIM`, baretile opens
the card, which needs root and takes the card away from its kernel driver. See
[bring-up](https://github.com/baretile/baretile/blob/main/docs/bring-up.md)
before running on a card.

## Tensors

Run this in Python:

```python
from baretile import pci
from baretile.chip import Chip
from baretile.runtime import Runtime
from baretile.tensor import Tensor

rt = Runtime(Chip(pci.open_device()))
x = Tensor.of(rt, [[1.0, 2.0], [3.0, 4.0]])
w = Tensor.of(rt, [[2.0, 0.0], [0.0, 3.0]])
print((x @ w).relu().tolist())
# [[2.0, 6.0], [6.0, 12.0]]
```

The matmul and ReLU run when you call `tolist()`. Until then, they're an
expression waiting to be compiled. They run in one kernel here: the ReLU works on
the matmul's result in the math unit's registers, before it's written out.

Tensors support broadcasting, reductions, views and autograd. Storage formats are
bf16, fp32, int32, bfp8 and bfp4. The
[compiler docs](https://github.com/baretile/baretile/blob/main/docs/compiler.md)
explain the layout and precision limits.

To see the kernels a script launched and what each one computes, run it with
`python3 -m baretile.viz script.py` and open the `viz.html` it writes.

## Run a model

Install baretile with NumPy for the models, then serve a model by its Hugging
Face ID:

```sh
python3 -m pip install 'baretile[models]'
python3 -m baretile.models.serve HuggingFaceTB/SmolLM-135M 8000
```

The server downloads the model the first time, or takes a local folder instead,
puts it on the chip, and says when it's ready. Then ask it for a completion from
another terminal:

```sh
curl http://localhost:8000/v1/completions -H 'Content-Type: application/json' \
  -d '{"prompt": "The meaning of life is", "max_tokens": 4, "temperature": 0}'
```

On ttsim the answer's text is `" a question that has"`, after about half a minute
a token, since the simulator runs every core of the chip. The server speaks
OpenAI's completions and chat completions API, so OpenAI's clients work with it
too.

[Models](https://github.com/baretile/baretile/blob/main/docs/models.md#running-a-model)
lists the families baretile runs and how each one is checked, and
[serving](https://github.com/baretile/baretile/blob/main/docs/models.md#serving)
covers the server's options. To generate from a local checkpoint and compare
each token with NumPy's choice:

```sh
python3 -m baretile.models.llama /path/to/SmolLM-135M "The meaning of life is" 6
```

tinygrad scripts run on baretile through
[baretile/adapters/tinygrad_device.py](https://github.com/baretile/baretile/blob/main/baretile/adapters/tinygrad_device.py),
and PyTorch runs through tinygrad's torch frontend.

## Learn baretile

The [ramps](https://github.com/baretile/baretile/tree/main/ramps) teach baretile
from the top down. Each one is a script that explains its steps and checks
their results. They're in the repository:

```sh
git clone https://github.com/baretile/baretile
cd baretile
python3 ramps/1_tensors.py
python3 ramps/2_runtime.py
python3 ramps/3_tensix.py
```

The first covers tensor operations, fusion, formats, gradients and `jit`. The
second runs C kernels on the workers and moves data over the NoC. The third
programs the unpack, math and pack threads of a Tensix tile by hand.

The [documentation](https://github.com/baretile/baretile/blob/main/docs/README.md)
explains the runtime, the compiler, the models and the hardware, and includes a
map of the code.

## Contributing

The tests run on ttsim. They need NumPy and pytest, and some comparisons also
need PyTorch and transformers, which pytest skips without:

```sh
python3 -m pip install numpy pytest
python3 -m pytest -s test
```

Tests that need checkpoint files are opt-in, and
[testing](https://github.com/baretile/baretile/blob/main/docs/testing.md#optional-checks)
lists them.

[CONTRIBUTING.md](https://github.com/baretile/baretile/blob/main/CONTRIBUTING.md)
explains how to run the checks, what the code should look like and how to
report a bug.

Licensed under [Apache-2.0](https://github.com/baretile/baretile/blob/main/LICENSE).
