Metadata-Version: 2.4
Name: fewlab
Version: 3.0.0
Summary: Pick the fewest items to label for unbiased OLS on shares
Author: Gaurav Sood
Author-email: Gaurav Sood <gsood07@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: numpy>=1.23
Requires-Dist: pandas>=1.5
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/finite-sample/fewlab
Project-URL: Documentation, https://finite-sample.github.io/fewlab/
Project-URL: Repository, https://github.com/finite-sample/fewlab
Description-Content-Type: text/markdown

# fewlab

[![Python application](https://github.com/finite-sample/fewlab/actions/workflows/ci.yml/badge.svg)](https://github.com/finite-sample/fewlab/actions/workflows/ci.yml)
[![Documentation](https://img.shields.io/badge/docs-github.io-blue)](https://finite-sample.github.io/fewlab/)
[![PyPI version](https://img.shields.io/pypi/v/fewlab.svg)](https://pypi.org/project/fewlab/)
[![Downloads](https://pepy.tech/badge/fewlab)](https://pepy.tech/project/fewlab)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python](https://img.shields.io/pypi/pyversions/fewlab.svg)](https://www.python.org/downloads/)

**Buy the labels that buy the smallest standard error.**

On a real browsing panel, labeling 250 of 34,078 domains recovers the same
regression coefficients as labeling all of them, with a ninth of the error that
spending the same 250 labels at random would give you. The estimate stays exactly
unbiased and reports a standard error you can put in a table. At 2,000 domains
the error falls to 11% of the coefficient's own size, from scanning
6% of what was actually scanned.

Those numbers come from [the benchmark](docs/EVALUATION_RESULTS.md), which runs
against a population whose labels are all known, so every design is scored
against the same fixed truth. They are regenerated by
`python benchmarks/make_exhibits.py`, which also writes every figure quoted in
the manuscript; none of them is typed in by hand.

## The problem

You have a usage matrix: rows are people, columns are things they used. Websites,
merchants, products, employers, surnames. You want to regress something about the
column on features of the row, but finding out what the column *is* costs money,
and you cannot afford all of them.

The awkward part is that cost and value sit at different levels. You pay once to
label a website, whether one panelist visited it or a hundred thousand did, and
that single label then enters every one of those people's rows. So what a label
is worth depends both on where the entity appears and on what you intend to
estimate.

That has an exact answer. Write `V` for row shares, so `V_ij` is person `i`'s
share of activity on entity `j`, and `A` for the matrix of entity labels. The
outcome is `Y = VA` and the complete-label regression coefficient is

```
B = (X'X)⁻¹X'VA = HA,        H = (X'X)⁻¹X'V
```

so, writing `h_j` for column `j` of `H`,

```
B = Σⱼ hⱼ aⱼ'
```

exactly, with no error term and no model. Entity `j`'s entire contribution to the
answer is the rank-one matrix `hⱼaⱼ'`, and `hⱼ` is computable from usage counts
and features alone, before you buy a single label.

## Which entities to label

Two goals, and neither is traded for the other. Unbiasedness is a constraint:
every entity that contributes must have a positive chance of being labeled, and
must be weighted by the reciprocal of that chance. Minimum standard error is what
you optimize subject to it.

```python
from fewlab import Study

study = Study.from_long(visits, features, unit="user", entity="domain", count="visits")

design = study.design(budget=250)  # π_j = min(1, λ√(b_j/c_j))
sample = study.draw(design, random_state=0)

labels = your_labeling_pipeline(sample.items)

est = study.estimate(sample, labels)
print(est.summary())
```

Step by step, what that does:

1. Compute `b_j = h_j'h_j`, each entity's contribution to the risk. No labels are
   needed. For a mutually exclusive taxonomy, `a_j'a_j = 1`, so this is exact
   with no pilot.
2. Find the `λ` that exhausts the budget in `π_j = min(1, λ√(b_j/c_j))`.
3. Label with certainty every entity where the cap binds. You do not pick this
   head; the constraint produces it. In a heavy-tailed population it absorbs most
   of the budget, which is why the design pays.
4. Sample the rest at `π_j`, not uniformly. The cube method lands exactly on
   budget and balances on the influence totals.
5. Weight by `1/π_j`. Head entities get weight 1; tail entities get more, standing
   in for the tail you did not label.

Unbiasedness comes from step 4, positive probability everywhere, plus the
weighting in step 5. Minimum standard error comes from the square root rule in
step 2. You need both.

The rule also solves the reverse question. `row_se_min_labels` gives the fewest
labels that hold every row's standard error under a ceiling you name, rather than
the smallest error for a budget you name.

To set a regression-risk ceiling instead, use
`study.design_for_risk(risk_limit=0.01**2, contrast=contrast)` and
`study.draw(design, fixed_size=False)`. This buys the least expected labeling
cost for the chosen contrast's label-sampling variance. Omit `contrast` to bound
the sum of coefficient variances. Unit label norms give exact aggregate risk
for a complete one-hot taxonomy and a conservative bound for a binary indicator.
The ceiling excludes regression uncertainty and realized error; see the
[quickstart](docs/quickstart.rst) for a complete example.

## Two standard errors, and when to stop paying

```
Var(β̂) = σ²(X'X)⁻¹  +  Σⱼ (1−πⱼ)/πⱼ · bⱼ
         your n units      your labeling budget
```

Only the second term moves when you spend more. `Estimate.diagnostics` reports
the share of total variance the budget is responsible for, which is a stopping
rule: once the design term falls well below the regression term, more labels buy
almost nothing.

## Using a cheap classifier without giving up unbiasedness

If you can guess a label cheaply for every entity, a domain name predicts a site
category fairly well, pass it as `proxy` and fewlab uses

```
B̂ = Σⱼ hⱼ μⱼ' + Σⱼ (Rⱼ/πⱼ) hⱼ (aⱼ − μⱼ)'
```

This is exactly unbiased whatever the classifier says, because the proxy term is a
known constant and the sampled term corrects it. A bad proxy costs variance and
never bias, so there is no risk in supplying one. Acquisition then targets
influence times residual label uncertainty, and for calibrated class
probabilities that uncertainty is `1 − ‖μⱼ‖²`, known without labels.

## Buying bias on purpose

Shrinking the weights trades bias for variance, and at small budgets it wins: on
the benchmark, shrinking cuts mean squared error by 46% at 250 labels. Pass
`alpha` to `estimate` and it reports the worst case bias alongside the variance,
so the trade is priced rather than assumed. The default is `alpha=1`, which is
unbiased.

Setting `alpha=0` is what zero-filling unlabeled entities amounts to. On the
benchmark its bias runs from 14% to 53% of the coefficient's own size, and it is
never the best point on the frontier.

## Checking that a design can help you at all

Some populations have nothing to optimize. If every entity influences the
regression in the same direction and differs only in magnitude, every loss
function ranks entities identically and the choice of estimand stops mattering.
`Study.diagnostics["direction_spectrum"]` reports this. Read it against a noise
floor of roughly `1/√(typical count)` rather than a fixed cutoff.

## Installation

```bash
pip install fewlab
```

Requires Python 3.12 or later, numpy and pandas. Counts can be handed over in
long format, one row per unit and entity, which is how panels arrive and what
keeps a 0.3% dense matrix from being materialized.

```bash
pip install -e ".[dev]"   # tests, linting, pre-commit
pip install -e ".[docs]"  # documentation
```

## Upgrading from 2.x

3.0 changes the API throughout and has no compatibility shims. The
[changelog](CHANGELOG.md) lists what was removed, what replaces it, and which 2.x
results should be re-drawn.

## License

MIT
