Metadata-Version: 2.4
Name: dtfit
Version: 0.5.0
Summary: Differential-transformation fitting: nonlinear smoothing and forecasting via differential / non-Taylor transformations
Author-email: Oleksandr Tuhanskykh <ringanaro@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/ringavirda/science-nonline
Project-URL: Repository, https://github.com/ringavirda/science-nonline
Project-URL: Issues, https://github.com/ringavirda/science-nonline/issues
Keywords: regression,nonlinear,time-series,differential-transformations,statistical-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.0
Requires-Dist: scipy>=1.10
Requires-Dist: sympy>=1.12
Requires-Dist: scikit-learn>=1.6
Provides-Extra: viz
Requires-Dist: matplotlib>=3.7; extra == "viz"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-cov>=4; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == "pandas"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6; extra == "docs"
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.26; extra == "docs"
Dynamic: license-file

# dtfit

[![CI](https://github.com/ringavirda/science-nonline/actions/workflows/ci.yml/badge.svg)](https://github.com/ringavirda/science-nonline/actions/workflows/ci.yml)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](https://github.com/ringavirda/science-nonline)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://github.com/ringavirda/science-nonline/blob/main/packages/dtfit/LICENSE)
[![Typed](https://img.shields.io/badge/typed-yes-brightgreen)](https://peps.python.org/pep-0561/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

**D**ifferential-**t**ransformation **fit**ting: nonlinear smoothing and
forecasting on time-series / Big Data, using methods built in the scheme of
differential (non-Taylor) transformations. Developed as part of a PhD
dissertation on mathematical models of nonlinear smoothing and prediction.

Models can be a SymPy expression string **or** a plain Python callable
`f(x, *params)`; batch fits accept per-point measurement `sigma` (or sklearn
`sample_weight`); and every fit carries its own quality diagnostics (`R^2`,
`AIC`/`BIC`). All are opt-in - left at their defaults, fits are numerically
identical to before.

## Installation

```bash
pip install dtfit                        # core, from PyPI
pip install 'dtfit[viz]'                 # + matplotlib plotting helpers
```

From a clone of the repository (this package lives at packages/dtfit):

```bash
python -m venv .venv        # create an isolated environment
# Windows:  .venv\Scripts\activate
# Linux/mac: source .venv/bin/activate

# from the repo root:
pip install -e packages/dtfit            # core
pip install -e 'packages/dtfit[viz]'     # + matplotlib plotting helpers
pip install -e 'packages/dtfit[dev]'     # + test/lint tooling

# Optional: the experimental adaptations + experiment suite (a separate
# distribution that depends on dtfit; never shipped in the dtfit wheel):
pip install -e packages/dtfit-experimental             # dtfit_experimental
pip install -e 'packages/dtfit-experimental[bench]'    # + the suite's baselines/plotting
```

A plain `venv` with PyPI wheels is the reference environment and works
identically on Windows and Linux. (Conda is not recommended on Windows: its
MKL-linked numpy only loads its LAPACK DLLs when the env is *activated*, so
tools that call the interpreter directly - VS Code, `pytest` - crash with a
delay-load error.)

## Quick start

```python
import numpy as np
import dtfit as dt

x = np.linspace(0, 10, 400)
y = ...  # your observations

# Batch fit, numeric (no polynomial pre-fit needed):
result = dt.fit("a*atan(w*x)", dt.Original(x, y), "x", basis="block")
print(result.params)

# ...or through the scikit-learn compatible estimator:
from dtfit.sklearn import NonlineRegressor

reg = NonlineRegressor("a0 + a1*x + a2*exp(a3*x)", "x")
reg.fit(x, y)
y_hat = reg.predict(x)
```

### Real-time / streaming

`dt.ImageFilter` tracks parameters online on the sliding window's image;
`ImageFilter(basis="legendre")` and `ImageFilter(basis="block")` fix its basis to Legendre and block:

```python
flt = dt.ImageFilter("A*sin(w*t)", "t", p0=[1.0, 1.0], window_size=50, basis="block")
for t, y in stream:           # bounded-cost per-sample update
    flt.partial_fit(t, y)
print(flt.params_)            # tracks time-varying parameters
print(flt.result().stderr())  # the window as a calibrated batch fit
```

### Picking a model (the model framework)

Most of fitting is *choosing the right structure*. `dtfit.models` is a catalog of
named families that **seed their own `p0`/`bounds` from the data**, compose with
`+`, and can be ranked for you:

```python
from dtfit import models, suggest_models

fit = models.logistic().fit(x, y)            # self-seeded; no p0/bounds to guess
fit = (models.linear() + models.sine()).fit(x, y)   # trend + cycle

for s in suggest_models(x, y)[:3]:           # infer the model from a scored shortlist
    print(s.name, s.r2, s.aic)
```

### Uncertainty & serialization

A `FittingResult` is self-describing - named parameters, uncertainty, and a
JSON-friendly round-trip:

```python
r = dt.fit("a*exp(b*x)", dt.Original(x, y), "x", basis="legendre", p0=[1, 1])
r.params                       # {'a': ..., 'b': ...}
r.stderr(); r.confidence_intervals(0.95)
r.rsquared, r.aic, r.bic       # fit-quality diagnostics
y_hat, y_std = r.predict(x, return_std=True)   # prediction band
dt.FittingResult.from_dict(r.to_dict())        # save / ship a fitted model

# a Python callable model + per-point sigma both work (numerically opt-in):
r2 = dt.fit(lambda x, a, b: a * np.exp(b * x), dt.Original(x, y, sigma=noise_std), basis="block")               # signature-order params, weighted fit
```

### Diagnostics & visualization

`dtfit.diagnostics` is **fit-aware** (it takes a `FittingResult`, not bare
arrays) and does *not* reimplement `sklearn.metrics` - use those / `scipy.stats`
for plain scalar metrics. It adds what's specific to evaluating a DT fit:
information criteria for model comparison and residual-structure tests, plus the
`*Display` plot helpers (which never call `plt.show()`).

```python
from dtfit.diagnostics import fit_report, residual_diagnostics, FitDisplay

print(fit_report(r, x, y))            # n, rmse, r2, aic, bic, durbin_watson, params+-se
print(residual_diagnostics(r, x, y)) # autocorrelation / normality of residuals

FitDisplay.from_estimator(reg, x, y)  # data + fitted curve (needs the viz extra)
```

## Methods

- **LSI** (`fit(basis="legendre")`, `basis="legendre"`) - least-squares integral; numeric
  integral-OLS in the differential-transformation scheme (successor to DSBI).
- **EAC** (`fit(basis="block")`, `basis="block"`) - equal-areas criterion; numeric,
  integration-based and noise-robust (successor to DSBE).
- **ImageFilter** - recursive/online tracker on the window image, with NIS
  drift detection; **ImageFilter(basis="legendre")** and **ImageFilter(basis="block")** fix its basis to Legendre
  and block for real-time tracking.
- **Image analytics** - an `Image` reports its own noise level
  (`noise_sigma`), how many orders carry signal (`effective_order`), how its
  coefficients decay (`decay`), whether it matches another image
  (`test_equal`) or a model (`test_structure`), and draws synthetic records
  from itself (`simulate`).
- **DSB** (`dtfit_legacy.dsb.fit_dsb`, in the `dtfit-legacy` package) - symbolic differential spectra
  balance; kept as the analytical reference, out of `fit` (it takes the
  coefficients of a polynomial pre-fit, not samples).
- **Stochastic** (`fit_stochastic`) - second-order characterization of a
  random series: every gate, estimator and forecaster reads one additive
  image of the record (`SecondOrderImage`), with a block-stream form
  (`SecondOrderStream`) and an online tracker (`StochasticFilter`).

### Scaling out

- **`ImageStream`** accumulates a signal in fixed `O(order)` memory as it
  arrives (the default uniform accumulator; `grid="explicit"` retains
  every position and weight, and block mode grows `coarse_` by one
  block per `fold`); its blocks are retained and `assemble` merges them
  into one `Image`, and a channel axis batches many signals through the
  same accumulator:

  ```python
  s = dtfit.ImageStream("legendre", 6, domain=(0, 10), block=200)
  for x, y in chunks_of_the_stream():
      s.update(x, y)
  img = s.assemble(0, 10)
  ```
- `dtfit.fit_many(problems, n_jobs=-1)` fans many independent fits across cores
  (process or threading backend); the threading backend shares memory and
  avoids pickling.

Adaptations still in trial (further image bases, fits of aggregated data and
of windows aligned to jumps, weak-form ODE identification, the filter bank)
live in the separate **`dtfit-experimental`** package (`dtfit_experimental`);
the notebooks that measure them are indexed in
[dtfit-experimental/experiments/README.md](https://github.com/ringavirda/science-nonline/blob/main/packages/dtfit-experimental/experiments/README.md).

Each method's mathematical grounding (in differential / non-Taylor
transformations), full algorithm, optimizations, guards, applicability, usage
figures and comparison tables are documented in the
[project wiki](https://github.com/ringavirda/science-nonline/wiki/Methods).

Core dependencies: numpy, scipy, sympy, scikit-learn. Plotting helpers require
the optional `viz` extra (matplotlib).
