Metadata-Version: 2.4
Name: process-improve
Version: 1.99.0
Summary: Designed Experiments; Latent Variables (PCA, PLS, multivariate methods with missing data); Process Monitoring; Batch data analysis.
Keywords: Designed Experiments,Design of Experiments,DoE,Optimal Design,Response Surface,Latent Variables,PCA,PLS,Multivariate Data Analysis,Chemometrics,Process Monitoring,Batch data analysis
Author: Kevin Dunn
Author-email: Kevin Dunn <kgdunn@gmail.com>
License-Expression: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Dist: numpy>=2.2.6
Requires-Dist: pandas>=2.3.3
Requires-Dist: patsy>=1.0.2
Requires-Dist: pydantic>=2.12.5
Requires-Dist: pyyaml>=6.0
Requires-Dist: scikit-learn>=1.7.2
Requires-Dist: scipy>=1.15
Requires-Dist: statsmodels>=0.14.6
Requires-Dist: threadpoolctl>=3.5.0
Requires-Dist: tqdm>=4.67.1
Requires-Dist: matplotlib>=3.10.8 ; extra == 'all'
Requires-Dist: mcp>=2.0 ; extra == 'all'
Requires-Dist: numba>=0.63.1 ; extra == 'all'
Requires-Dist: openpyxl>=3.1.5 ; extra == 'all'
Requires-Dist: osqp>=1.0 ; extra == 'all'
Requires-Dist: plotly>=6.5.2 ; extra == 'all'
Requires-Dist: ridgeplot>=0.5.0 ; extra == 'all'
Requires-Dist: ruptures>=1.1.9 ; extra == 'all'
Requires-Dist: scikit-image>=0.25.2 ; extra == 'all'
Requires-Dist: seaborn>=0.13.2 ; extra == 'all'
Requires-Dist: openpyxl>=3.1.5 ; extra == 'batch'
Requires-Dist: ruptures>=1.1.9 ; extra == 'batch'
Requires-Dist: scikit-image>=0.25.2 ; extra == 'batch'
Requires-Dist: osqp>=1.0 ; extra == 'control'
Requires-Dist: coverage>=7.13.1 ; extra == 'dev'
Requires-Dist: hypothesis>=6.100 ; extra == 'dev'
Requires-Dist: matplotlib-stubs>=0.3.11 ; extra == 'dev'
Requires-Dist: mypy>=1.19.1,<2.5 ; extra == 'dev'
Requires-Dist: pandas-stubs>=2.3.3.260113 ; extra == 'dev'
Requires-Dist: plotly-stubs>=0.0.6 ; extra == 'dev'
Requires-Dist: pre-commit>=4.5.1 ; extra == 'dev'
Requires-Dist: pytest>=9.0.2 ; extra == 'dev'
Requires-Dist: pytest-benchmark>=4.0.0 ; extra == 'dev'
Requires-Dist: pytest-cov>=7.0.0 ; extra == 'dev'
Requires-Dist: pytest-xdist>=3.8.0 ; extra == 'dev'
Requires-Dist: pydata-sphinx-theme>=0.16.1 ; extra == 'dev'
Requires-Dist: ruff>=0.11.0,<0.17 ; extra == 'dev'
Requires-Dist: sphinx>=8.1.3 ; extra == 'dev'
Requires-Dist: nbsphinx>=0.9.5 ; extra == 'dev'
Requires-Dist: ipykernel>=6.29 ; extra == 'dev'
Requires-Dist: tqdm-stubs>=0.2.1 ; extra == 'dev'
Requires-Dist: watchdog>=6.0.0 ; extra == 'dev'
Requires-Dist: numba>=0.63.1 ; extra == 'fast'
Requires-Dist: mcp>=2.0 ; extra == 'mcp'
Requires-Dist: matplotlib>=3.10.8 ; extra == 'plotting'
Requires-Dist: plotly>=6.5.2 ; extra == 'plotting'
Requires-Dist: ridgeplot>=0.5.0 ; extra == 'plotting'
Requires-Dist: seaborn>=0.13.2 ; extra == 'plotting'
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/kgdunn/process-improve
Project-URL: Documentation, https://kgdunn.github.io/process-improve/
Project-URL: Try it in your browser, https://kgdunn.github.io/process-improve/app/
Project-URL: Textbook, https://learnche.org/pid
Project-URL: Repository, https://github.com/kgdunn/process-improve
Project-URL: Issues, https://github.com/kgdunn/process-improve/issues
Project-URL: Changelog, https://github.com/kgdunn/process-improve/blob/main/CHANGELOG.md
Provides-Extra: all
Provides-Extra: batch
Provides-Extra: control
Provides-Extra: dev
Provides-Extra: fast
Provides-Extra: ilp
Provides-Extra: mcp
Provides-Extra: plotting
Description-Content-Type: text/markdown

# process-improve

**Designed experiments, multivariate analysis and process monitoring for Python.**
Built for the chemometrics, manufacturing and pharma workflows where you need to know
not just *what fits*, but *which factors matter, is this observation normal, which
variable moved, and how sure am I?*

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kgdunn/process-improve/main/docs/_static/readme/banner-dark.png">
  <img alt="Four experimental designs from one generate_design call: a definitive screening design, a central composite design, a constrained I-optimal design and a MaxPro space-filling design" src="https://raw.githubusercontent.com/kgdunn/process-improve/main/docs/_static/readme/banner-light.png">
</picture>

[![PyPI version](https://img.shields.io/pypi/v/process-improve.svg)](https://pypi.org/project/process-improve/)
[![Python versions](https://img.shields.io/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Fkgdunn%2Fprocess-improve%2Fmain%2Fpyproject.toml&label=python)](https://pypi.org/project/process-improve/)
[![Downloads](https://static.pepy.tech/badge/process-improve)](https://pepy.tech/project/process-improve)
[![Downloads per month](https://static.pepy.tech/badge/process-improve/month)](https://pepy.tech/project/process-improve)
[![CI](https://github.com/kgdunn/process-improve/actions/workflows/run-tests.yml/badge.svg?branch=main&event=push)](https://github.com/kgdunn/process-improve/actions/workflows/run-tests.yml?query=branch%3Amain)
[![codecov](https://codecov.io/gh/kgdunn/process-improve/branch/main/graph/badge.svg)](https://codecov.io/gh/kgdunn/process-improve)
[![Docs](https://img.shields.io/badge/docs-kgdunn.github.io-blue.svg)](https://kgdunn.github.io/process-improve/)
[![License](https://img.shields.io/pypi/l/process-improve.svg)](https://github.com/kgdunn/process-improve/blob/main/LICENSE)

**[Try it in your browser](https://factori.al/)** ·
[Documentation](https://kgdunn.github.io/process-improve/) ·
[Free textbook](https://learnche.org/pid) ·
[Design gallery](#designed-experiments-in-one-call)

```bash
pip install process-improve
```

## Try it without installing anything

<a href="https://factori.al/"><img width="600" alt="The designed-experiments app in a browser tab: the yield of a reactor and the factors that may change it, the designs that can learn its curvature compared on nine measures, a Box-Behnken run sheet with results simulated from a known process, and the fitted coefficients beside the true ones" src="https://raw.githubusercontent.com/kgdunn/process-improve/main/docs/_static/readme/app-demo.gif"></a>

[Factorial](https://factori.al/), the designed-experiments app, runs
this package inside your browser tab (Pyodide / WebAssembly). Say what you will measure, with
its units and goal, and what might change it: the app lists the designs that can learn how the
one depends on the other, scored on the nine measures of
[Process Improvement using Data](https://learnche.org/pid) and on their power to find the change
you care about. Run the one you pick, type in the results or practise with results simulated from
a process whose true model is known, and read the analysis, beside the truth when it is known.
Copy the Python behind each step. The analysis runs on your machine, and your data never leave it.

## Learn the methods

- **Free textbook:** [Process Improvement using Data](https://learnche.org/pid), the
  companion to this package, from data visualization to latent-variable models, designed
  experiments and process monitoring.
- **API reference and user guide:** <https://kgdunn.github.io/process-improve/>
- **Applied DoE tutorial (8 modules):** <https://kgdunn.github.io/process-improve/applied_doe/index.html>
- **Every design family and its `design_type`:** <https://kgdunn.github.io/process-improve/user_guide/doe_coverage.html>
- **Fully worked quickstart:** <https://kgdunn.github.io/process-improve/quickstart.html>

## Ask a question, get one line of code and the chapter that explains it

Under each question is the chapter of the free textbook that explains the method.

| You want to know | One call |
| --- | --- |
| **Which design should I run?**<br>[Comparing design families](https://learnche.org/pid/design-analysis-experiments/comparing-design-families) | `recommend_strategy(factors=factors, budget=30)` |
| **Which of my factors matter?**<br>[Definitive screening designs](https://learnche.org/pid/design-analysis-experiments/definitive-screening-designs) | `generate_design(factors, design_type="dsd")` |
| **Which effects are real?**<br>[Significance of effects](https://learnche.org/pid/design-analysis-experiments/full-factorial-designs/assessing-significance-of-main-effects-and-interactions) | `analyze_experiment(design.design, responses)` |
| **Which way is the optimum?**<br>[Response surface methods](https://learnche.org/pid/design-analysis-experiments/response-surface-methods) | `optimize_responses([fit], method="steepest_ascent")` |
| **Is this observation normal?**<br>[Hotelling's T²](https://learnche.org/pid/latent-variable-modelling/principal-component-analysis/hotellings-t2-statistic) | `pca.diagnose(new).spe` |
| **Which variable moved?**<br>[Contribution plots](https://learnche.org/pid/latent-variable-modelling/principal-component-analysis/latent-variable-contribution-plots) | `pca.spe_contributions(new)` |
| **How sure is this prediction?**<br>[Projection to latent structures](https://learnche.org/pid/latent-variable-modelling/projection-to-latent-structures/index) | `pls.prediction_interval(new)` |
| **Which recipe hits my target?**<br>[Using a PLS model backwards](https://learnche.org/pid/latent-variable-modelling/projection-to-latent-structures/pls-model-inversion-and-the-orthogonal-space) | `model.invert(y_desired=20.9)` |
| **Is the process in control?**<br>[Shewhart charts](https://learnche.org/pid/process-monitoring/shewhart-charts) | `ControlChart().calculate_limits(y)` |
| **Is this batch on track?**<br>[Batch process monitoring](https://learnche.org/pid/product-development-product-improvement/batch-process-monitoring) | `BatchMonitor(model).fit(good).monitor(batch)` |

## Designed experiments in one call

```python
from process_improve.experiments import Factor, generate_design

factors = [
    Factor(name="Temperature", low=150, high=200, units="degC"),
    Factor(name="Pressure", low=1, high=5, units="bar"),
    Factor(name="Catalyst", low=0.5, high=2.0, units="g"),
]
design = generate_design(factors, design_type="dsd")  # a definitive screening design
print(design.n_runs)  # 9
run_sheet = design.design_actual  # in your units, ready to run
```

Change `design_type` and the same call builds any of these, each true to its
[published definition](https://kgdunn.github.io/process-improve/user_guide/doe_coverage.html).
Leave `design_type` out and give a `budget`, and one is chosen for you.

<a href="https://kgdunn.github.io/process-improve/user_guide/doe_coverage.html"><picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kgdunn/process-improve/main/docs/_static/readme/design-gallery-dark.png">
  <img width="560" alt="Sixteen designs drawn from generate_design: supersaturated, Plackett-Burman, fractional and full factorial, definitive screening, central composite, Box-Behnken, OMARS, Taguchi, D-optimal, constrained I-optimal, constrained mixture, Latin hypercube, uniform, MaxPro and Sobol" src="https://raw.githubusercontent.com/kgdunn/process-improve/main/docs/_static/readme/design-gallery-light.png">
</picture></a>

`evaluate_design` scores any design (efficiencies, prediction variance, fraction of design
space, aliasing), `analyze_experiment` fits it, `augment_design` extends it, and
`optimize_responses` finds the best settings for several responses at once. Not sure where
to start? Ask for a whole strategy:

```python
from process_improve.experiments import Response
from process_improve.experiments.strategy import recommend_strategy

strategy = recommend_strategy(
    factors=factors,  # the three factors above
    responses=[Response(name="Yield", goal="maximize", units="%")],
    budget=30,
)
print([(stage["design_type"], stage["estimated_runs"]) for stage in strategy["stages"]])  # [('full_factorial', 8), ('ccd', 17), ('replicates_at_optimum', 3)]
```

## Quick start: real data, real answers

The examples below use measurements from a low-density polyethylene reactor: 54 runs,
14 process variables and 5 quality measurements of the polymer. The last four runs come
from a process upset. Paste the blocks in order and you get the numbers in the comments.

### Is this observation normal, and which variable moved?

```python
import pandas as pd
from process_improve.multivariate import PCA, MCUVScaler

ldpe = pd.read_csv("https://openmv.net/file/LDPE.csv", index_col=0)
process, quality = ldpe.loc[:, "Tin":"Press"], ldpe.loc[:, "Conv":"SCB"]

# Learn normal operation from the first 50 runs, then check the last four
scaler = MCUVScaler().fit(process.loc[:50])
pca = PCA(n_components=3).fit(scaler.transform(process.loc[:50]))
new = scaler.transform(process.loc[51:])

# SPE above its limit means "unlike anything seen in normal operation"
print(pca.diagnose(new).spe.round(1).to_list())  # [2.3, 3.7, 5.3, 7.6]
print(round(pca.spe_limit(conf_level=0.95), 1))  # 3.4

# Which variables moved in run 54? All three belong to the second reactor zone
print(pca.spe_contributions(new).loc[54].abs().nlargest(3).round(1).to_dict())  # {'z2': 5.9, 'Fi2': 3.1, 'Tcin2': 1.9}
pca.score_plot()  # interactive Plotly figure
```

### How sure is this prediction?

```python
from process_improve.multivariate import PLS

# Hold out runs 41 to 50 and predict their 5 quality measurements, with 95% intervals
x_scaler, y_scaler = MCUVScaler().fit(process.loc[:40]), MCUVScaler().fit(quality.loc[:40])
n = PLS.select_n_components(process.loc[:40], quality.loc[:40], max_components=6, random_state=0).n_components
pls = PLS(n_components=n).fit(x_scaler.transform(process.loc[:40]), y_scaler.transform(quality.loc[:40]))
interval = pls.prediction_interval(x_scaler.transform(process.loc[41:50]))
low, high = y_scaler.inverse_transform(interval.lower), y_scaler.inverse_transform(interval.upper)

# Cross-validation picks 3 components, and 48 of the 50 held-out measurements land inside
inside = (quality.loc[41:50] >= low) & (quality.loc[41:50] <= high)
print(n, int(inside.to_numpy().sum()))  # 3 48

# The quality is driven by the same reactor zone that moved in the upset
print(pls.vip().nlargest(3).round(2).to_dict())  # {'z2': 1.38, 'Fi2': 1.38, 'Tmax2': 1.37}
```

### Which recipe hits my target?

```python
from process_improve.multivariate import OPLS

# Acetic acid, H2S and lactic acid in 26 cheddar cheeses, and their taste score
cheese = pd.read_csv("https://openmv.net/file/cheddar-cheese.csv").iloc[4:]
X, y = cheese[["Acetic", "H2S", "Lactic"]], cheese[["Taste"]]
model = PLS(n_components=2).fit(X, y)

recipe = model.invert(y_desired=20.9)  # the inputs predicted to give a taste of 20.9
print(recipe.x_new.round(2).to_list())  # [5.52, 5.56, 1.4]
print(round(recipe.hotellings_t2, 2))  # 0.06
print(recipe.null_space_dimension)  # 1

# One free direction: a line of recipes with the same predicted taste
walk = [model.invert(20.9, null_space_coordinates=[s]).x_new.round(2).to_list() for s in (-1.0, 0.0, 1.0)]
print(walk)  # [[4.95, 6.1, 1.33], [5.52, 5.56, 1.4], [6.09, 5.02, 1.46]]

# O-PLS separates that freedom while fitting, so inversion becomes one division
opls = OPLS(n_orthogonal_components=1).fit(X, y)
print(opls.invert(y_desired=20.9).x_new.round(2).to_list())  # [5.46, 5.62, 1.39]
```

The freedom along that line is what you spend on cost, supply or a regulatory window; the
`hotellings_t2` tells you when a recipe has walked past the evidence. The
[user guide](https://kgdunn.github.io/process-improve/user_guide/model_inversion.html) and the
[book chapter](https://learnche.org/pid/latent-variable-modelling/projection-to-latent-structures/pls-model-inversion-and-the-orthogonal-space)
go further.

### Has the process drifted?

```python
from process_improve.multivariate import AdaptivePCA

# Learn from the first 50 runs in their own units (the model centres and scales itself),
# then stream the last four, one at a time
monitor = AdaptivePCA(n_components=3).fit(process.loc[:50])
alarms = [not monitor.update(row.to_numpy()).in_control for _, row in process.loc[51:].iterrows()]
print(alarms)  # [False, True, True, True]
```

`AdaptivePCA` and `AdaptivePLS` keep learning as data arrive, so a model does not go stale
when the process drifts, and they report how far it has moved from where it was trained.

## What's inside

- **Designed experiments**: factorial, screening (Plackett-Burman, DSD, supersaturated),
  response-surface (CCD, Box-Behnken, OMARS), mixture, space-filling and optimal (D, I, A, E,
  G, K) designs, in constrained regions too; design evaluation, analysis (including split
  plots), augmentation, multi-response optimisation and a multi-stage strategy recommender.
- **PCA** with SVD and NIPALS, native missing-value handling by Trimmed Score Regression,
  Hotelling's T² and SPE limits, and score, T² and SPE contributions.
- **PLS** regression with a scikit-learn API, VIP, cross-validated component selection and
  prediction intervals; **PLS-DA** classification with a Bayesian decision rule for
  unbalanced classes and a permutation test that says whether the separation is real.
- **Model inversion**: `PLS.invert()` and `OPLS` solve for the inputs that reach a target
  and return the null space of equally valid designs.
- **Multi-block and T-shaped data**: TPLS (PLS for T-shaped data structures), MBPCA and MBPLS.
- **On-line models**: `AdaptivePCA` and `AdaptivePLS` for monitoring and soft sensing.
- **Process monitoring**: Shewhart, EWMA, CUSUM and Holt-Winters control charts, process capability.
- **Batch data**: alignment, feature extraction, batch PCA and PLS, on-line batch monitoring.
- **Sensory panels**: panel validation, the Mixed Assessor Model, attribute-to-product relations.
- **Robust regression**: repeated-median and Theil-Sen estimators for data with outliers.
- **Interactive Plotly diagnostics** on every fitted model, and `pandas`-native outputs that
  keep your row and column labels.

Release notes are in the [changelog](https://github.com/kgdunn/process-improve/blob/main/CHANGELOG.md).

## Works alongside scikit-learn

`process-improve` sits *next to* scikit-learn, not in place of it. Its estimators follow the
same conventions (`fit`, `predict`, `score`, the `_` suffix on fitted attributes), so they
drop into `Pipeline`, `GridSearchCV` and `cross_val_score`. What it adds is the
process-analytics layer: the diagnostics that tell you whether a new observation is normal,
which variable moved, and how confident a prediction is.

| Capability                                        | scikit-learn | process-improve |
| ------------------------------------------------- | :----------: | :-------------: |
| PCA, PLS with sklearn-style API                   |       ✓      |        ✓        |
| Missing-data fitting (NIPALS / TSR)               |       -      |        ✓        |
| Hotelling's T² + SPE outlier limits               |       -      |        ✓        |
| Variable-level contributions                      |       -      |        ✓        |
| Prediction intervals for PLS                      |       -      |        ✓        |
| Multi-block models (TPLS, MBPLS)                  |       -      |        ✓        |
| Model inversion: design inputs for a target       |       -      |        ✓        |
| On-line / adaptive monitoring (recursive PCA/PLS) |       -      |        ✓        |
| Designed experiments, incl. OMARS & optimal       |       -      |        ✓        |
| Shewhart, EWMA, CUSUM, Holt-Winters charts        |       -      |        ✓        |
| Batch process monitoring                          |       -      |        ✓        |
| Plotly diagnostics built in                       |       -      |        ✓        |
| Labeled `DataFrame` outputs                       |    partial   |        ✓        |

`fit()` returns `self`, fitted attributes end with an underscore (`scores_`, `loadings_`,
`spe_`, `hotellings_t2_`, `r2_cumulative_`, ...), `predict()` returns predictions, and
`diagnose()` returns an `sklearn.utils.Bunch` with named fields such as `spe` and
`hotellings_t2`. Pipelines that mix scaled numeric and one-hot encoded columns, and the
current gaps, are in
[SKLEARN_COMPATIBILITY.md](https://github.com/kgdunn/process-improve/blob/main/SKLEARN_COMPATIBILITY.md).

## Installation

```bash
pip install process-improve                    # core: numpy, pandas, scipy, scikit-learn, statsmodels, ...
pip install 'process-improve[plotting]'        # adds matplotlib, plotly, seaborn, ridgeplot
pip install 'process-improve[batch]'           # adds openpyxl, ruptures, scikit-image (batch data IO, change points)
pip install 'process-improve[control]'         # adds osqp (the batch mid-course correction solver)
pip install 'process-improve[mcp]'             # adds mcp (the MCP server runtime)
pip install 'process-improve[fast]'            # adds numba (JIT speedups for batch alignment)
pip install 'process-improve[all]'             # everything above
```

Requires Python 3.10 or newer. The core install pulls in `numpy`, `pandas`, `scipy`,
`scikit-learn`, `statsmodels`, `patsy`, `pydantic`, `pyyaml`, `threadpoolctl` and `tqdm`.
Heavier optional surfaces (plotting, Taguchi arrays, batch IO, batch control, the MCP server, numba JIT)
live in extras, so a caller who only needs, say, `detect_multivariate_outliers` does not have
to install Plotly or numba.

## Use it from Claude

The designed-experiments tooling ships as a Claude Skill, so you can plan, generate, verify
and analyse experiments in your own Claude account with no server involved:

```
/plugin marketplace add kgdunn/process-improve
/plugin install doe-designer@process-improve
```

The skill's first rule is that a design matrix is never written out by the model: it is
generated from a catalogue and then verified, because a language model asked to produce a
fractional factorial will often return one that looks right and is a lower resolution than
it claims. See [`skills/README.md`](https://github.com/kgdunn/process-improve/blob/main/skills/README.md)
for the other install routes (local folder, claude.ai upload) and for the MCP server, which
exposes the same tool registry without the workflow guidance.

## Citing process-improve

If you use this package in academic work, please cite it. The
[`CITATION.cff`](https://github.com/kgdunn/process-improve/blob/main/CITATION.cff) file carries
the current version and release date, and GitHub renders a *"Cite this repository"* button
in the sidebar with ready-made BibTeX and APA entries:

```bibtex
@software{dunn_process_improve,
  author  = {Dunn, Kevin G.},
  title   = {{process-improve: Multivariate Analysis for Process Improvement}},
  year    = {2026},
  url     = {https://github.com/kgdunn/process-improve}
}
```

Add the `version` field from `CITATION.cff` (or the release tag you installed) when citing a
specific version.

## License

MIT: see [LICENSE](https://github.com/kgdunn/process-improve/blob/main/LICENSE) for details.

## Contributing

Bug reports, feature requests and pull requests are welcome. See
[CONTRIBUTING.md](https://github.com/kgdunn/process-improve/blob/main/CONTRIBUTING.md) for
development setup, testing and code style, and the
[architecture overview](https://kgdunn.github.io/process-improve/architecture.html) for a map
of the codebase. Bugs and feature requests go on the
[issue tracker](https://github.com/kgdunn/process-improve/issues).
