Metadata-Version: 2.4
Name: tsfresh-rs
Version: 0.1.0
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Dist: numpy>=1.22
Requires-Dist: pandas>=1.3
Requires-Dist: scipy>=1.7
Requires-Dist: scikit-learn>=1.0 ; extra == 'sklearn'
Requires-Dist: pytest ; extra == 'test'
Requires-Dist: tsfresh>=0.21 ; extra == 'test'
Requires-Dist: scipy ; extra == 'test'
Requires-Dist: statsmodels ; extra == 'test'
Requires-Dist: mpmath ; extra == 'test'
Requires-Dist: scikit-learn ; extra == 'test'
Provides-Extra: sklearn
Provides-Extra: test
License-File: LICENSE
Summary: Fast, drop-in time series feature extraction for Python, powered by Rust
Keywords: time series,feature extraction,tsfresh,rust
Author: Fatin Ishraq
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/Fatin-Ishraq/tsfresh-rs/releases
Project-URL: Homepage, https://github.com/Fatin-Ishraq/tsfresh-rs
Project-URL: Issues, https://github.com/Fatin-Ishraq/tsfresh-rs/issues
Project-URL: Repository, https://github.com/Fatin-Ishraq/tsfresh-rs

<p align="center">
  <img src="https://raw.githubusercontent.com/Fatin-Ishraq/tsfresh-rs/main/assets/logo.svg" alt="" width="104" height="104">
</p>

<h1 align="center">tsfresh-rs</h1>

<p align="center">
  <strong>Fast, drop-in time series feature extraction for Python, powered by Rust.</strong>
</p>

<p align="center">
  <a href="https://github.com/Fatin-Ishraq/tsfresh-rs/actions/workflows/ci.yml"><img src="https://github.com/Fatin-Ishraq/tsfresh-rs/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
  <a href="https://pypi.org/project/tsfresh-rs/"><img src="https://img.shields.io/pypi/v/tsfresh-rs.svg" alt="PyPI"></a>
  <a href="https://pypi.org/project/tsfresh-rs/"><img src="https://img.shields.io/badge/python-3.10%20--%203.14-blue.svg" alt="Python 3.10 to 3.14"></a>
  <a href="https://github.com/Fatin-Ishraq/tsfresh-rs/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT"></a>
</p>

```diff
- import tsfresh
+ import tsfresh_rs as tsfresh
```

That is the whole migration. Same functions, same arguments, same 783 feature
columns, same column names in the same order — verified by 478 tests that run
both libraries on the same input and compare every value, including which
inputs each one *refuses*.

```bash
pip install tsfresh-rs
```

Supports **Python 3.10 through 3.14** from a single `abi3` wheel per platform.

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Fatin-Ishraq/tsfresh-rs/main/assets/benchmark-dark.svg">
    <img src="https://raw.githubusercontent.com/Fatin-Ishraq/tsfresh-rs/main/assets/benchmark-light.svg" width="920" alt="End-to-end extraction speedup of tsfresh-rs over tsfresh: 93x to 141x across six workloads, from 2.94 s to 0.028 s on ten 500-point series up to 42.41 s to 0.454 s on twenty 2,000-point series.">
  </picture>
</p>

## Why it is faster

`tsfresh` has two separate performance problems, and fixing only the obvious
one gets you a fraction of the win.

The obvious one is that the calculators are slow. `approximate_entropy`
materialises an `(N−m+1)²×m` boolean array to count Chebyshev neighbours;
`lempel_ziv_complexity` hashes a fresh Python tuple for every substring probe;
`sample_entropy` builds full distance matrices. Rewriting those in Rust is
worth about **17x**.

The less obvious one is everything around them. Per series and per feature, the
reference pays a Python function call, a pandas `groupby`/`apply` step, and the
assembly of a wide DataFrame from hundreds of small pieces. That cost does not
shrink when the calculators get faster — and on a comprehensive extraction it
is most of the wall clock.

So this package does not call calculators one at a time. It flattens every
series into one contiguous `float64` buffer, compiles the whole feature plan
once, and makes a **single** call into Rust that fills the entire output matrix
with the GIL released and Rayon over series.

Owning the loop is what turns 17x into **120x**.

## Benchmarks

AMD Ryzen 5 5600G (6 cores / 12 threads), Python 3.14, `tsfresh` 0.21.2,
pandas 3.0.5. Every row was checked for agreement before it was timed, so a
fast wrong answer cannot appear in these tables. `python bench/bench.py`
reproduces them.

### End to end, `ComprehensiveFCParameters` (783 columns per series)

| workload | `tsfresh` | `tsfresh-rs` | speedup |
|---|---:|---:|---:|
| 10 series × 500 pts | 2.94 s | 0.0278 s | **106x** |
| 20 series × 500 pts | 5.72 s | 0.0464 s | **123x** |
| 50 series × 500 pts | 14.18 s | 0.1007 s | **141x** |
| 100 series × 500 pts | 27.23 s | 0.1951 s | **140x** |
| 20 series × 1,000 pts | 15.09 s | 0.1284 s | **118x** |
| 20 series × 2,000 pts | 42.41 s | 0.4539 s | **93x** |

The reference costs **about 290 ms per series** on the 500-point workload. That is
the number behind [issue #973](https://github.com/blue-yonder/tsfresh/issues/973)
— open since 2022 — where users report 3-hour and 16-hour waits and a
maintainer replies that he lacks the bandwidth to fix it.

### The preset that exists to avoid the cost

`EfficientFCParameters` drops `approximate_entropy` and `sample_entropy`
because they are 62% of a comprehensive run. Here you do not have to choose:

| | time |
|---|---:|
| `tsfresh`, `EfficientFCParameters` (777 columns) | 1.77 s |
| `tsfresh-rs`, `ComprehensiveFCParameters` (783 columns) | 0.0445 s |

**The full feature set here is 40x faster than the reduced one there.**

### Longer series

4 series, `tsfresh` serial against `tsfresh-rs` on all cores — only four series
means threading has little to work with, which is why these ratios are the
lowest in this README rather than the highest:

| points per series | `tsfresh` | `tsfresh-rs` | speedup |
|---|---:|---:|---:|
| 500 | 0.91 s | 0.0197 s | 46x |
| 1,000 | 2.70 s | 0.0506 s | 53x |
| 2,000 | 8.69 s | 0.1836 s | 47x |
| 5,000 | 53.64 s | 1.0416 s | 51x |

### Per calculator

20 series × 500 points, single-threaded, the twelve most expensive in the
reference:

| calculator | cols | `tsfresh` | `tsfresh-rs` | speedup |
|---|---:|---:|---:|---:|
| `approximate_entropy` | 5 | 2.7106 s | 0.17105 s | 16x |
| `change_quantiles` | 60 | 0.9165 s | 0.01441 s | 64x |
| `sample_entropy` | 1 | 0.5419 s | 0.02095 s | 26x |
| `number_cwt_peaks` | 2 | 0.2078 s | 0.00761 s | 27x |
| `augmented_dickey_fuller` | 3 | 0.1581 s | 0.04214 s | 4x |
| `agg_linear_trend` | 48 | 0.1442 s | 0.00374 s | 39x |
| `max_langevin_fixed_point` | 1 | 0.1081 s | 0.00048 s | 223x |
| `friedrich_coefficients` | 4 | 0.0926 s | 0.00073 s | 127x |
| `lempel_ziv_complexity` | 5 | 0.0610 s | 0.00182 s | 34x |
| `permutation_entropy` | 5 | 0.0474 s | 0.00359 s | 13x |
| `fourier_entropy` | 5 | 0.0387 s | 0.00083 s | 46x |
| `ar_coefficient` | 11 | 0.0349 s | 0.00933 s | 4x |
| **all 74** | | **5.2057 s** | **0.30332 s** | **17x** |

All 74 agree with the reference to within 1e-9 relative on this data — the
benchmark checks each one before it times it, so a fast wrong answer cannot
reach this table.

The gap between this table's 17x and the end-to-end 120x is the whole argument
of the section above: most of a comprehensive extraction is not the features.

### `n_jobs`

`tsfresh` parallelises with processes, so every chunk is pickled and every
worker re-imports the library. Below a few dozen series that costs more than it
saves:

| workload | `n_jobs` | `tsfresh` | `tsfresh-rs` |
|---|---|---:|---:|
| 20 × 500 | 1 | 4.34 s | 0.2687 s |
| 20 × 500 | 6 | **5.82 s** | 0.0639 s |
| 20 × 500 | all cores | 4.21 s | 0.0405 s |
| 100 × 500 | 1 | 21.55 s | 1.2553 s |
| 100 × 500 | 6 | 11.43 s | 0.2695 s |
| 100 × 500 | all cores | **24.05 s** | 0.1956 s |

On 20 series, asking `tsfresh` for six processes makes it **slower** than
running serially; on 100 series, asking for every core makes it slower than
asking for six. This package uses threads inside one process: nothing is
pickled, there is no `if __name__ == "__main__"` requirement, threading pays
off at every size, and the output is bit-identical whatever you pass.

One deliberate difference: `n_jobs=0` means *one thread per core* here, where
the reference means *no parallelism at all*. With threads there is nothing to
trade off, so the default should be the fast one. Pass `n_jobs=1` for a
genuinely single-threaded run.

## What it adds: `feature_timings`

`EfficientFCParameters` is a guess — a judgement the maintainers made once, on
their data, about which features are expensive. On a 500-point series those two
features really are 62% of the run. On a 50-point series they are nearly free.
On a 20,000-point series something else dominates entirely.

`feature_timings` measures it on *your* data:

```python
import tsfresh_rs as tsfresh

timings = tsfresh.feature_timings(df, column_id="id", column_sort="time")
print(timings.head(6)[["feature", "columns", "seconds", "pct_of_total"]])
#                 feature  columns   seconds  pct_of_total
#     approximate_entropy        5  0.139937     60.287962
# augmented_dickey_fuller        3  0.032445     13.977916
#          sample_entropy        1  0.018846      8.119415
#        change_quantiles       60  0.007768      3.346718
#        number_cwt_peaks        2  0.007093      3.055826
#          ar_coefficient       11  0.006635      2.858552
```

The shares are exhaustive — all 74 features, summing to 100% — so nothing is
hiding outside the report.

and `drop_slowest` turns that into a feature set:

```python
from tsfresh_rs.profiling import drop_slowest

cheap = drop_slowest(tsfresh.ComprehensiveFCParameters(), timings, budget_pct=90)
X = tsfresh.extract_features(df, default_fc_parameters=cheap, ...)
```

`tsfresh` has no equivalent. The question "which of these 783 columns am I
paying for?" currently has no answer other than deleting features and
re-timing by hand.

## Accuracy

Prefix-summing and compensating is where the speed comes from, and it is also
where a careless port silently disagrees. Three places where this package is
measurably **more accurate than the reference**, each pinned by a test against
exact arithmetic rather than against `tsfresh`:

**Fourier coefficients on offset data.** A signal with a large DC offset makes
every non-DC coefficient a difference of large, nearly-cancelling terms. The
DFT is linear, so this package transforms the mean-centred signal and restores
bin 0 exactly — mathematically identical, far better conditioned. Judged
against a 50-digit exact DFT on a signal offset by 1e6:

| | relative error |
|---|---|
| `numpy.fft.rfft` | 6.6 × 10⁻⁹ |
| `tsfresh-rs` | **4.6 × 10⁻¹⁴** |

**Variance and sums.** Neumaier-compensated accumulation, and a double-double
mean in every regression, so that deviations survive on a series whose spread
is twelve orders of magnitude below its level.

**Histogram binning.** `binned_entropy` on a near-constant series — values
around 5.0 spanning 0.01 — bins correctly here. The naive
`floor((v − lo) / width)` that looks equivalent to NumPy's index computation
loses the whole signal to cancellation; matching NumPy's scale-then-correct
algorithm was worth a 25% difference in the entropy.

**A scale-invariant Dickey-Fuller statistic.** Multiplying a series by a
constant cannot change whether it has a unit root, so the ADF test statistic is
invariant by construction. The reference's is not, because its design matrix
pairs a constant column of 1.0 with level and difference columns that shrink
with the data and statsmodels factorises that without equilibrating. Rescaling
a series to 1e-9 moves its answer:

| | statistic at unit scale | at 1e-9 |
|---|---|---|
| `tsfresh` | −4.673342604953984 | −4.673342580274487 |
| `tsfresh-rs` | −4.673342604953985 | **−4.673342604953988** |

Each column is normalised before the factorisation here, which is the standard
fix and costs nothing.

## Dependency footprint

`tsfresh` imports scipy, statsmodels, scikit-learn, pywt, stumpy, dask,
distributed, tqdm, cloudpickle and matplotlib to compute its features. The
numerics those provide are reimplemented here in Rust — pywt's Mexican-hat
CWT, SciPy's Welch PSD and ridge-line peak finder, statsmodels' `AutoReg`,
`acf`, `pacf` and the ADF test with MacKinnon p-values — so extraction itself
loads **numpy and pandas, and nothing else**:

```python
>>> import tsfresh_rs, sys
>>> tsfresh_rs.extract_features(df, column_id="id", column_sort="time")
>>> [m for m in sys.modules if m.split(".")[0] in ("scipy", "statsmodels",
...                                                "sklearn", "pywt", "stumpy")]
[]
```

`scipy` is still declared as a dependency because `select_features` needs its
hypothesis tests, but it is imported lazily and never touched by extraction.

## Correctness

The package is only worth anything if the answers match, so that is what the
suite tests.

- **478 tests**, almost all differential against `tsfresh` on the same input.
- **250 randomised fuzz cases** over length, scale, offset, tie density and
  degeneracy, comparing every calculator value by value; plus 40 fuzzed
  end-to-end pipelines. This found four real defects during development,
  including a wrong Mexican-hat amplitude that scaled all 60
  `cwt_coefficients` columns by 1.3025 — a uniform error that looks entirely
  plausible in isolation.
- **API coverage is asserted mechanically**: every calculator in
  `dir(tsfresh...feature_calculators)` must exist here with the same
  `fctype`, `minimal`, `high_comp_cost`, `input` and `index_type` attributes,
  because those attributes are what define the presets.
- **All five presets** must select exactly the same features with exactly the
  same parameters.
- **Whole-frame equality** across all four input shapes — long with a kind
  column, long without, wide, and a dict of frames — including column order
  and index.

Reference quirks are reproduced deliberately, not accidentally. `fft_aggregated`'s
kurtosis has a `- 3 * centroid` term where the standardised fourth moment wants
`3 * centroid⁴`; `ComprehensiveFCParameters` builds `mean_n_absolute_max` from a
dict literal with three copies of the same key, so two of its three intended
features have never existed. Both are matched bug-for-bug, because a drop-in
that quietly "fixes" a feature changes numbers people have already trained on.

## Where it deliberately differs

Each documented in `tests/test_equivalence.py::KNOWN_DIVERGENCES` with a test
asserting the divergence is still real.

- **`fft_coefficient(attr="angle")` on a constant series.** Every non-DC
  coefficient is zero, so its argument is undefined. The reference reports the
  argument of its own rounding noise; this reports 0.
- **`fft_coefficient` on a series containing an infinity.** An infinity has no
  Fourier transform. Both libraries propagate it through their own FFT
  decomposition, and which bins emerge as NaN and which as infinite is an
  artefact of the radix each chose — NumPy's pocketfft yields 38 NaN and 3
  infinities where this package yields 34 and 7. Neither pattern carries
  information.
- **`linear_trend` and friends on a constant series.** The correlation of a
  constant is undefined and this returns NaN. The reference returns NaN for a
  constant 2.5 and 0.0 for a constant 0.001 — the difference being whether
  `np.cov`'s mean happens to be exactly representable, which leaves a spurious
  4.7 × 10⁻³⁸ variance in one case and not the other.
- **Conditioning-limited fits** — `friedrich_coefficients`,
  `max_langevin_fixed_point`, `ar_coefficient`, `cwt_coefficients` on a signal
  offset by 1e6. Both implementations solve a numerically rank-deficient
  problem; the coefficients differ in the 4th digit while the residuals they
  describe agree to 10.

Below a relative spread of about 1e-4, standardised statistics are not
reproducible between *any* two implementations: the deviations being
standardised are themselves rounding error. Measured against 60-digit
arithmetic on the cases the fuzzer found, this package is closer to the true
value in 4 of 6 and SciPy in 2 of 6. Neither is authoritative there.

## A note on reference versions

Three `tsfresh` feature values are not decided by `tsfresh`. They are decided
by whichever SciPy, NumPy and PyWavelets are installed next to it, and those
libraries have changed their answers. Running the differential suite across
three operating systems and three Python versions surfaced all three:

| feature | what changed | effect |
|---|---|---|
| `cwt_coefficients` | PyWavelets 1.8's `cwt` has no `precision` parameter and integrates the wavelet on a 1024-point grid; 1.9 added `precision=12`, a 4096-point grid | 4th significant figure |
| `friedrich_coefficients`, `max_langevin_fixed_point` | pandas 3.0 made `qcut` bin on `np.quantile`'s edges; pandas 2.3 disagreed on 3 of 499 boundary points, which moves the bucket means the cubic is fitted to | 4th significant figure |
| `linear_trend`, `agg_linear_trend` on a constant series | SciPy 1.15 returned `rvalue=0.0`, `pvalue=1.0`; SciPy 1.18 returns NaN. The `0.0` came from a spurious ~1e-38 variance left by `np.cov`'s mean rounding | `0.0` vs `NaN` |
| `permutation_entropy` on tied data | `np.argsort` defaults to quicksort, which used to fall back to a stable insertion sort for the short windows involved. NumPy 2's SIMD sort does not, so the ordinal pattern of a tied window now depends on the CPU | 1% of the entropy at dimensions 4–6 |

Each was confirmed by reproducing the older environment and matching its value
exactly — `pywt.cwt(..., precision=10)` returns the old figure to the last
digit, and pandas 2.3.3's `qcut` reproduces the old cubic. The
`permutation_entropy` entry is the starkest of the four: it is not a version
difference at all but a *CPU* difference, and there is no value this package
could pick that would agree with the reference everywhere. It uses the stable
order, which is at least the same everywhere.

This package computes all three in Rust, so it gives one fixed answer wherever
it runs. It targets the current behaviour: PyWavelets ≥ 1.9's precision-12
grid, pandas ≥ 3.0's NumPy-consistent binning, and SciPy ≥ 1.16's NaN for an
undefined correlation. On an older reference those specific columns will differ
— which is a statement about the reference's environment, not about this
package.

The test suite probes the installed reference at runtime and skips exactly
those comparisons when it detects an older behaviour, rather than pinning
dependency versions. Pinning would hide the effect; users do not pin.

## Coverage

Everything in `tsfresh` 0.21.2:

| | |
|---|---|
| **Calculators** | all 76, including `cwt_coefficients` (pywt-exact), `number_cwt_peaks` (SciPy's ridge-line tracker, reproduced), `augmented_dickey_fuller` (with MacKinnon p-values), `query_similarity_count` |
| **Presets** | `Comprehensive`, `Efficient`, `Minimal`, `IndexBased`, `TimeBased` |
| **Extraction** | `extract_features` over long, wide and dict inputs; `kind_to_fc_parameters` |
| **Selection** | `select_features`, `extract_relevant_features`, `calculate_relevance_table` |
| **Utilities** | `impute`, `impute_dataframe_zero`, `impute_dataframe_range`, `roll_time_series`, `make_forecasting_frame`, `add_sub_time_series_index`, `get_ids`, `from_columns` |
| **Compatibility** | `transformers` (the scikit-learn `FeatureAugmenter` family), `utilities.distribution`, `utilities.profiling`, `feature_selection.significance_tests`, `feature_extraction.data`, `examples`, `scripts.run_tsfresh`, and the private windowing helpers (`_roll`, `_into_subchunks`, …) that third-party code imports |
| **New** | `feature_timings`, `drop_slowest` |

## For code you cannot edit

When a dependency deep in the stack imports `tsfresh` by name:

```python
import tsfresh_rs
tsfresh_rs.install()   # before anything imports `tsfresh`

import tsfresh         # this is now tsfresh_rs
```

It refuses rather than half-patching the module graph if the real `tsfresh` has
already been imported.

## Limitations

Stated plainly, because a drop-in that hides its gaps is worse than one that
does not have them.

- **`matrix_profile` is unavailable**, exactly as in the reference. The
  `matrixprofile` package is unmaintained and does not build on current Python,
  so `tsfresh` ships with this feature disabled and excluded from
  `ComprehensiveFCParameters`. This mirrors that, including the `ImportError`.
- **`select_features` is not accelerated.** It is linear in the number of
  features with one cheap hypothesis test each — not the bottleneck this
  package exists to fix — so it runs in Python. `n_jobs` and `chunksize` are
  accepted and ignored there.
- **`chunksize`, `distributor` and the `profile*` arguments** describe
  tsfresh's multiprocessing pipeline, which this implementation does not have.
  They are accepted for signature compatibility and ignored, with a warning.
  The `utilities.distribution` classes exist so that code importing them keeps
  working, but `extract_features` does not route through them.
- **`tsfresh.__version__` reports this package's version after `install()`**,
  not a tsfresh one, so code gating on `tsfresh.__version__ >= "0.20"` will
  take the wrong branch. Claiming to be 0.21.2 would fix that one check and
  lie to every other; the API level being emulated is published separately as
  `tsfresh_rs.__tsfresh_version__`.
- **`profile=True` no longer answers "which calculator is slow".** A cProfile
  trace of an extraction here shows one opaque call into Rust. `feature_timings`
  is the replacement, and it measures what the profiler used to.
- **`augmented_dickey_fuller` is only 4x faster.** Its cost is a lag-selection
  search over many OLS fits, and the reference already spends that time inside
  compiled LAPACK.

## Development

```bash
pip install maturin pytest numpy pandas scipy tsfresh mpmath statsmodels
maturin build --release --out dist && pip install --no-index --find-links dist tsfresh-rs
pytest tests/ -q
python bench/bench.py
```

To reproduce the older-reference environment the version table describes — the
one where three feature columns legitimately differ — and check that the suite
skips exactly those:

```bash
uv venv --python 3.10 /tmp/old && uv pip install --python /tmp/old/bin/python     pytest numpy pandas scipy tsfresh mpmath statsmodels
uv pip install --python /tmp/old/bin/python --no-deps --find-links dist tsfresh-rs
/tmp/old/bin/python -m pytest tests/ -q -rs
```

That resolves to numpy 2.2, pandas 2.3, SciPy 1.15 and PyWavelets 1.8, and
gives `411 passed, 3 skipped`.

## Licence and credit

MIT, the same licence as `tsfresh`.

This is a reimplementation of [`tsfresh`](https://github.com/blue-yonder/tsfresh)
by Maximilian Christ and Blue Yonder GmbH, whose API, feature definitions and
output semantics it deliberately reproduces. If you use this in research, cite
their paper:

> Christ, M., Braun, N., Neuffer, J. and Kempa-Liehr A.W. (2018).
> *Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests
> (tsfresh — A Python package).* Neurocomputing 307 (2018) 72-77.

