Metadata-Version: 2.3
Name: verifiable-labs-code-humaneval
Version: 0.1.1
Summary: Single-turn procedural code-execution RL environment with sandboxed pytest scoring and conformal coverage
Project-URL: Homepage, https://github.com/stelioszach03/verifiable-labs-envs
Project-URL: Issues, https://github.com/stelioszach03/verifiable-labs-envs/issues
Author: Stelios Zacharioudakis
License: Apache-2.0
Keywords: code-execution,evaluation,humaneval,reinforcement-learning,rlvr,verifiable-labs
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: numpy>=1.26
Requires-Dist: verifiable-labs-envs-core>=0.1.1
Description-Content-Type: text/markdown

# verifiable-labs-code-humaneval

Single-turn procedural code-execution RL environment from the
Verifiable Labs catalogue. Each instance carries a function signature
+ docstring + a small visible test set; the model returns Python
source which the env runs against a hidden test battery inside a
sandboxed subprocess (D5 limits: 512 MB / 30 s wall / 20 s CPU /
``unshare -rn`` network isolation / 16-process fanout cap).

| Component       | Weight | What it rewards                                          |
|-----------------|--------|----------------------------------------------------------|
| `format_valid`  | 0.10   | Output is parseable JSON containing a `code` field       |
| `parse_valid`   | 0.20   | Extracted code compiles via `compile(..., "exec")`       |
| `pass_rate`     | 0.70   | Fraction of (visible ∪ hidden) pytest cases that passed  |

12 procedural templates across lists, strings, dicts, ints, trees,
and graphs — `EFFECTIVE_INSTANCES > 7e23`, well above the
contamination-resistance gate.

## Install

```bash
pip install verifiable-labs-code-humaneval
```

## Use

```python
from verifiable_labs_code_humaneval import load_environment

env = load_environment(calibration_quantile=0.5)
inst = env.generate_instance(seed=42)
print(inst.prompt)
```

Source of truth + full docs:
https://github.com/stelioszach03/verifiable-labs-envs
