Metadata-Version: 2.4
Name: fewlab
Version: 4.0.0
Summary: Pick the fewest items to label for unbiased OLS on shares
Author: Gaurav Sood
Author-email: Gaurav Sood <gsood07@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: numpy>=1.23
Requires-Dist: pandas>=1.5
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/finite-sample/fewlab
Project-URL: Documentation, https://finite-sample.github.io/fewlab/
Project-URL: Repository, https://github.com/finite-sample/fewlab
Description-Content-Type: text/markdown

# fewlab

[![Python application](https://github.com/finite-sample/fewlab/actions/workflows/ci.yml/badge.svg)](https://github.com/finite-sample/fewlab/actions/workflows/ci.yml)
[![Documentation](https://img.shields.io/badge/docs-github.io-blue)](https://finite-sample.github.io/fewlab/)
[![PyPI version](https://img.shields.io/pypi/v/fewlab.svg)](https://pypi.org/project/fewlab/)
[![Downloads](https://pepy.tech/badge/fewlab)](https://pepy.tech/project/fewlab)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python](https://img.shields.io/pypi/pyversions/fewlab.svg)](https://www.python.org/downloads/)

**Buy the labels that buy the smallest standard error.**

## The problem

You have a usage matrix: rows are units, usually people, and columns are the
things they use. You want to regress something about the *columns* on features
of the *rows*, but finding out what a column is costs money, and there are too
many columns to label them all.

| Rows | Columns | The label you pay for | A regression you might run |
|---|---|---|---|
| Panelists | Websites they visit | Site category, or what trackers it runs | Do older adults meet more trackers? |
| Cardholders | Merchants they pay | Merchant category, or ownership | Does income predict spending at small businesses? |
| Households | Products they buy | A product attribute coded by hand | Does education predict buying ultra-processed food? |
| Workers | Employers they work for | Employer industry, or remote policy | Did younger workers move to remote employers? |
| Users | Apps they install | App category, or privacy practices | Who installs apps that sell location data? |

The outcome for each row is a usage-weighted average of the column labels: the
share of a person's browsing that goes to news sites, the share of their
spending at small businesses. fewlab says which columns to label, labels as few
as the precision you need allows, and keeps the estimate exactly unbiased with
a standard error you can report.

## Worked example: a browsing panel

On a real browsing panel, labeling 250 of 34,078 domains recovers the same
regression coefficients as labeling all of them, with a ninth of the error that
spending the same 250 labels at random would give you. At 2,000 domains the
error falls to 11% of the coefficient's own size, from scanning 6% of what was
actually scanned.

Those numbers come from [the benchmark](docs/EVALUATION_RESULTS.md), which runs
against a population whose labels are all known, so every design is scored
against the same fixed truth. They are regenerated by
`python benchmarks/make_exhibits.py`, which also writes every figure quoted in
the manuscript; none of them is typed in by hand.

To work through the same decision with a budget slider, without installing
anything, [run the notebook in your browser](https://finite-sample.github.io/fewlab/examples.html#try-it-in-your-browser).

## Why a label's value depends on who used it

Cost and value sit at different levels. You pay once to label a column,
whether one person used it or a hundred thousand did, and that single label then
enters every one of those people's rows. So what a label
is worth depends both on where the entity appears and on what you intend to
estimate.

That has an exact answer. Write `V` for row shares, so `V_ij` is person `i`'s
share of activity on entity `j`, and `A` for the matrix of entity labels. The
outcome is `Y = VA` and the complete-label regression coefficient is

```
B = (X'X)⁻¹X'VA = HA,        H = (X'X)⁻¹X'V
```

so, writing `h_j` for column `j` of `H`,

```
B = Σⱼ hⱼ aⱼ'
```

exactly, with no error term and no model. Entity `j`'s entire contribution to the
answer is the rank-one matrix `hⱼaⱼ'`, and `hⱼ` is computable from usage counts
and features alone, before you buy a single label.

## Which entities to label

Two goals, and neither is traded for the other. Unbiasedness is a constraint:
every entity that contributes must have a positive chance of being labeled, and
must be weighted by the reciprocal of that chance. Minimum standard error is what
you optimize subject to it.

```python
from fewlab import Study

# usage: one row per (person, item) pair with a count; features: one row per person
study = Study.from_long(usage, features, unit="person", entity="item", count="n")

design = study.design(budget=250)  # π_j = min(1, λ√(b_j/c_j))
sample = study.draw(design, random_state=0)
sample.save("label_sample.json")

labels = your_labeling_pipeline(sample.items)

from fewlab import LabelSample

sample = LabelSample.load("label_sample.json")
est = study.estimate(sample, labels)
print(est.summary())
```

Step by step, what that does:

1. Compute `b_j = h_j'h_j`, each entity's contribution to the risk. No labels are
   needed. For a mutually exclusive taxonomy, `a_j'a_j = 1`, so this is exact
   with no pilot.
2. Find the `λ` that exhausts the budget in `π_j = min(1, λ√(b_j/c_j))`.
3. Label with certainty every entity where the cap binds. You do not pick this
   head; the constraint produces it. In a heavy-tailed population it absorbs most
   of the budget, which is why the design pays.
4. Draw each remaining entity independently at `π_j`. The budget is the
   expected number of labels, so the realized count varies. If an exact count
   is required, `study.draw(design, fixed_size=True)` uses the cube method to
   balance on the influence totals.
5. Weight by `1/π_j`. Head entities get weight 1; tail entities get more, standing
   in for the tail you did not label.

Unbiasedness comes from step 4, positive probability everywhere, plus the
weighting in step 5. Minimum standard error comes from the square root rule in
step 2. You need both.

The rule also solves the reverse question. `row_se_min_labels` gives the fewest
labels that hold every row's standard error under a ceiling you name, rather than
the smallest error for a budget you name.

To set a regression-risk ceiling instead, use
`study.design_for_risk(risk_limit=0.01**2, contrast=contrast)` and
`study.draw(design, fixed_size=False)`. This buys the least expected labeling
cost for the chosen contrast's label-sampling variance. Omit `contrast` to bound
the sum of coefficient variances. Unit label norms give exact aggregate risk
for a complete one-hot taxonomy and a conservative bound for a binary indicator.
The ceiling excludes regression uncertainty and realized error; see the
[quickstart](docs/quickstart.rst) for a complete example.

## Inference for the observed frame

`est.design_se` measures uncertainty from labeling only some entities. It
conditions on the observed rows, their features, and the entity frame. The
default `est.summary()` uses this standard error. Independent draws use the
Poisson variance estimator; cube draws use a balanced residual approximation.
The estimator is selected from the recorded draw method. The Poisson risk shown
by a design is exact for independent draws and only a planning criterion for
cube draws.

Save the draw before sending its selected IDs for labeling. The JSON manifest
stores every entity ID and inclusion probability, which IDs were selected, the
draw method, the random seed when one was supplied, and a fingerprint of the
counts and features. Keep the input counts and features too: they define the
finite frame and are needed to resume estimation. `Study.estimate` rejects a
sample loaded against different study data. Labels should be indexed by the
selected entity IDs; their order does not matter. The manifest is an audit
record, not a substitute for the labels or the study data.

`summary()` names the variance method and marks the variance estimator as
`design-unbiased variance`, `approximate`, or `unavailable`. A Poisson draw with the
matching Poisson variance estimator gets the first status. Cube draws use a
balanced approximation.

Entity labels are the sampling units here: a single label can affect every
row's outcome. Applying ordinary survey weights to the regression rows would
describe a different sampling design.

## Row-level model uncertainty and when to stop paying

If the observed rows are themselves viewed as a sample from a larger population,
Pass `row_model_se=True` to `study.estimate` when this model is appropriate.
`regression_se` adds a homoskedastic row-level model assumption. `total_se`
combines that component with label-sampling uncertainty. Neither is part of the
finite-frame design SE.

```
Var(β̂) = σ²(X'X)⁻¹  +  Σⱼ (1−πⱼ)/πⱼ · bⱼ
         your n units      your labeling budget
```

Only the second term moves when you spend more. Under the row-level model,
`Estimate.diagnostics["variance_ratio"]` reports the share of combined variance
owed to labeling; it can guide a budget decision, but it is not a design-based
uncertainty measure for rows that were not sampled by a known design.

## Using a cheap classifier without giving up unbiasedness

If you can guess a label cheaply for every entity, as a domain name predicts a
site category or a merchant name its line of business, pass it as `proxy` and
fewlab uses

```
B̂ = Σⱼ hⱼ μⱼ' + Σⱼ (Rⱼ/πⱼ) hⱼ (aⱼ − μⱼ)'
```

This is exactly unbiased whatever the classifier says, because the proxy term is a
known constant and the sampled term corrects it. A bad proxy costs variance and
never bias, so there is no risk in supplying one. Acquisition then targets
influence times residual label uncertainty, and for calibrated class
probabilities that uncertainty is `1 − ‖μⱼ‖²`, known without labels.

## Buying bias on purpose

Shrinking the weights trades bias for variance, and at small budgets it wins: on
the benchmark, shrinking cuts mean squared error by 46% at 250 labels. Pass
`alpha` to `estimate` and it reports the worst case bias alongside the variance,
so the trade is priced rather than assumed. The default is `alpha=1`, which is
unbiased.

Setting `alpha=0` is what zero-filling unlabeled entities amounts to. On the
benchmark its bias runs from 14% to 53% of the coefficient's own size, and it is
never the best point on the frontier.

## Checking that a design can help you at all

Some populations have nothing to optimize. If every entity influences the
regression in the same direction and differs only in magnitude, every loss
function ranks entities identically and the choice of estimand stops mattering.
`Study.diagnostics["direction_spectrum"]` reports this. Read it against a noise
floor of roughly `1/√(typical count)` rather than a fixed cutoff.

## Installation

```bash
pip install fewlab
```

Requires Python 3.12 or later, numpy and pandas. Counts can be handed over in
long format, one row per unit and entity, which is how usage logs arrive and
what keeps a 0.3% dense matrix from being materialized.

```bash
pip install -e ".[dev]"   # tests, linting, pre-commit
pip install -e ".[docs]"  # documentation
```

## Upgrading from 2.x

3.0 changes the API throughout and has no compatibility shims. The
[changelog](CHANGELOG.md) lists what was removed, what replaces it, and which 2.x
results should be re-drawn.

## License

MIT
