Metadata-Version: 2.4
Name: mlsys-lab
Version: 0.1.0
Summary: Auto-graded exercises in low-level ML systems: real C++, real CUDA-C on a software GPU, deterministic metrics
Project-URL: Homepage, https://github.com/mikamika06/mlsys-lab
Project-URL: Issues, https://github.com/mikamika06/mlsys-lab/issues
Author: Oleksandr Savkov
License: MIT License
        
        Copyright (c) 2026 Oleksandr Savkov
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: cpp,cuda,exercises,gpu,llm-inference,performance,quantization,systems-programming
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Programming Language :: C++
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Education
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Description-Content-Type: text/markdown

# mlsys-lab

Auto-graded exercises in low-level ML systems. You write **real C++**, **real
CUDA-C** and Python; a local grader runs it and checks a **measured number**
against declarative gates.

No cloud, no account, no GPU required. Every metric is deterministic, so the same
submission scores identically on any machine.

```
$ mlsys grade gpu-1-padding-to-remove-transpose-bank-conflicts

  max_abs_err        0.0        <= 1e-12   PASS
  smem_wave_ratio    16.5       <= 1.05    FAIL
                     ^ your tile is 32 wide, so every column read hits one bank
```

## Why the numbers are trustworthy

Wall-clock timing is never a gate. Instead the engine models the hardware and
counts what actually happened:

| you write | it runs on | it measures |
|---|---|---|
| `solve.cu` | a software GPU: 32-lane warps, 128-byte transactions, 32 shared-memory banks, barriers, warp shuffles | coalescing transactions, bank-conflict waves, warp divergence, cycles |
| `solve.cpp` | the local `clang++ -O2 -std=c++20`, plus a modelled set-associative cache | cache misses, lines touched, and the program's own output |
| `solve.py` | the interpreter, with an instrumented probe | allocations, copies, numerical error against a computed oracle |

The reference answer is always **computed by the grader**, never hard-coded — so
a task cannot be passed by guessing the expected output.

## Install

```bash
pip install mlsys-lab           # the engine and all 2052 tasks

mlsys list                      # browse the bank
mlsys grade <task-id> --file my_solution.py
```

The bank ships inside the package, so there is nothing to clone and nothing to
download on first run. Your solutions live wherever you put them — `--file` takes
a path, and the bank is never written to.

C++ tasks need a C++20 compiler (`clang++` or `g++`); everything else is Python
plus NumPy. CUDA tasks need **no** NVIDIA hardware — the `.cu` is executed by the
simulator in `src/mlsys/sim/`.

### VS Code

Install **mlsys-lab** from the Marketplace, then run **mlsys-lab: Open Workspace**.
If the package is not installed yet, the extension offers to do it. Solutions go to
`~/mlsys-lab/<task-id>/` (configurable via `mlsys.workDir`).

### Contributing to the bank

```bash
git clone https://github.com/mikamika06/mlsys-lab
cd mlsys-lab && pip install -e .
```

A checkout takes precedence over the installed copy, so the tasks you are editing
are the ones that get graded. `$MLSYS_TASKS` overrides both.

## What is in the bank

Three native tracks plus the systems material:

- **Deep Python** — the data model, descriptors, the GIL, memory layout
- **Deep C++** — ownership, moves, ABI, undefined behaviour you can observe
- **CPU performance** — cache blocking, layout (AoS/SoA), branches, SIMD
- **GPU / CUDA** — coalescing, bank conflicts, divergence, barriers, shuffles
- **Numerics & tensors** — stability, shapes and strides, low-precision formats
- **LLM internals** — attention, RoPE, GQA, the KV cache, sampling
- **LLM systems** — paged KV, continuous batching, parallelism, speculative decoding
- **Applied** — quantization, attention/KV, export & compilation, serving,
  memory & offload, sparsity & distillation

## Concepts, explained with a number

[`docs/concepts/`](docs/concepts/) — one page per concept, each with a measurement produced by
the simulator in this repo and the command that regenerates it, then the graded exercise.
Start with [memory coalescing](docs/concepts/memory-coalescing.md) (128-byte transactions
against read stride, 2 → 33) or [false sharing](docs/concepts/false-sharing.md) (coherence
invalidations against padding, 7,999 → 0).

## What else exists

[`docs/LANDSCAPE.md`](docs/LANDSCAPE.md) surveys the other resources aimed at each of
these areas — 141 of them, every link checked — and marks whether each one actually
grades your work or only shows you code. It says where this bank is *not* your best
option, which is most of the point of having the page.

The short version: GPU/CUDA, algorithms-from-scratch and LLM internals are well served
already (GPU-Puzzles, LeetGPU, Tensara, deep-ml, CS336, LeetCUDA). Six of the fourteen
areas have **no** auto-graded resource anywhere, and four — applied quantization,
compilation & export, memory & offload, and sparsity/pruning/distillation — have nothing
to practise against at all: papers, production libraries and documentation, but no
exercise that tells you that you got it wrong.

## Writing a task

A task is a directory under `tasks/`. See [`docs/TASK_FORMAT.md`](docs/TASK_FORMAT.md).

```
tasks/<id>/
  meta.json     id, title, difficulty, gates       ("native": "cpp" | "cuda")
  task.md       ## Context / ## Task / ## Example / ## What the gate checks
  check.py      grade(sol, fx) -> {metric: value}  — computes its own reference
  starter.py    the empty contract the learner starts from
  solution_ref.py
```

```bash
tools/verify_task.sh   <id>    # python task:  reference passes, starter fails
tools/verify_native.sh <id>    # C++ / CUDA task
```

Both print `TASK_OK` only if the reference passes the gates **and** the shipped
starter fails them — a task that anything can pass is not a task.

## Layout

```
src/mlsys/        the engine
  sim/            software GPU, CUDA-C front end, cache model, ABI model
  runners/        cpp.py (clang++), cuda.py (software GPU)
  scorers.py      the metrics tasks gate on
tasks/            the bank
editor/           VS Code extension
tools/            verify, queue helpers, browser preview
docs/             curriculum and research
```

## License

MIT
