Metadata-Version: 2.4
Name: qfeaturelib
Version: 0.5.0
Summary: High-performance feature engineering library for quantitative investment
Project-URL: Homepage, https://github.com/ElenYoung/QFeatureLib
Project-URL: Repository, https://github.com/ElenYoung/QFeatureLib.git
Project-URL: Documentation, https://github.com/ElenYoung/QFeatureLib#readme
Project-URL: Issues, https://github.com/ElenYoung/QFeatureLib/issues
Author-email: ElenYoung <elenyoung@example.com>
Maintainer-email: ElenYoung <elenyoung@example.com>
License-Expression: MIT
License-File: LICENSE
Keywords: feature-engineering,numba,numpy,panel-data,quantitative-finance,time-series
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: Chinese (Simplified)
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Office/Business :: Financial :: Investment
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: numba>=0.58.0
Requires-Dist: numpy>=1.23.0
Requires-Dist: pandas>=1.5.0
Requires-Dist: scikit-learn>=1.3.0
Provides-Extra: all
Requires-Dist: matplotlib>=3.7; extra == 'all'
Requires-Dist: mypy>=1.0.0; extra == 'all'
Requires-Dist: pytest-benchmark>=4.0.0; extra == 'all'
Requires-Dist: pytest-cov>=4.0.0; extra == 'all'
Requires-Dist: pytest>=7.0.0; extra == 'all'
Requires-Dist: ruff>=0.1.0; extra == 'all'
Requires-Dist: torch>=2.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest-benchmark>=4.0.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: gpu
Requires-Dist: torch>=2.0; extra == 'gpu'
Provides-Extra: viz
Requires-Dist: matplotlib>=3.7; extra == 'viz'
Description-Content-Type: text/markdown

# QFeatureLib

[![PyPI version](https://badge.fury.io/py/qfeaturelib.svg)](https://badge.fury.io/py/qfeaturelib)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

[English](README.md) | [中文](README.zh.md)

**QFeatureLib** is a high-performance, production-grade feature engineering library for quantitative investment. It focuses on financial time series processing with strict handling of future function avoidance, computational efficiency, and rigorous sample splitting.

## Key Features

- **Zero Future Function**: All time-series operations use `shift=1` by default to prevent data leakage. The library raises `FutureFunctionError` if you accidentally try to use future information.
- **High Performance**: Pure NumPy implementation with vectorized operations, 10-100x faster than pandas.
- **Memory Efficient**: Uses views instead of copies, supports in-place operations for large-scale panel data.
- **Quantitative Finance Focused**: Specialized for financial scenarios - suspended stock handling, industry neutralization, market cap neutralization, etc.

## Installation

```bash
pip install qfeaturelib
```

For development:

```bash
pip install qfeaturelib[dev]
```

## Quick Start

```python
import numpy as np
from qfeaturelib import PanelData
from qfeaturelib.standardization import rolling_zscore, cs_zscore
from qfeaturelib.splitting import RollingWindowSplitter

# Create panel data (T=100 days, N=50 stocks, F=5 features)
values = np.random.randn(100, 50, 5)
dates = np.arange(100)
tickers = [f'STOCK_{i:02d}' for i in range(50)]

panel = PanelData(values, dates, tickers)

# Time-series standardization (rolling Z-score with shift=1 to prevent leakage)
zscore_values = rolling_zscore(
    panel.values[..., 0],  # First feature
    window=20,
    shift=1,  # Use past 20 days only, excluding current moment
)

# Cross-sectional standardization (Z-score across all stocks each day)
cs_values = cs_zscore(panel.values[..., 0])

# Sample splitting for backtesting
splitter = RollingWindowSplitter(
    n_samples=100,
    train_ratio=0.6,
    val_ratio=0.2,
    test_ratio=0.2,
)

for split in splitter.split():
    train_data = zscore_values[split.train]
    val_data = zscore_values[split.val]
    test_data = zscore_values[split.test]
    # Train your model...
```

## Core Modules

### 1. Time-Series Standardization

Operations along the time dimension with rolling windows:

```python
from qfeaturelib.standardization import (
    rolling_zscore,      # Rolling Z-Score
    rolling_robust_zscore,  # Robust Z-Score using Median/MAD
    rolling_minmax,      # Rolling Min-Max scaling
)

# Parameters explained
result = rolling_zscore(
    data,
    window=20,      # Rolling window size
    shift=1,        # Window end offset (shift=1 excludes current moment)
    outlier_method="squash",  # Outlier handling: 'truncate' or 'squash'
    outlier_bounds=(0.01, 0.99),  # Quantile bounds for outliers
)
```

For long-format pandas frames with multiple groups (e.g. one row per
`(ticker, date)`), the grouped wrappers — added in 0.2 — standardise
each ticker against its own rolling statistics, so a ticker's
outliers cannot leak into another ticker's z-scores:

```python
from qfeaturelib import grouped_rolling_zscore, grouped_rolling_minmax

out = grouped_rolling_zscore(
    df,
    group_col="ticker",
    feature_cols=["ret", "vol"],
    # Optional: per-group feature subset
    # group_feature_map={"AAPL": ["ret"], "MSFT": ["ret", "vol"]},
    window=20,
    shift=1,          # default; shift=0 emits FutureFunctionWarning
)
```

### 2. Cross-Sectional Standardization

Operations across all assets at each time point:

```python
from qfeaturelib.standardization import (
    cs_zscore,           # Cross-sectional Z-Score
    cs_robust_zscore,    # Cross-sectional robust Z-Score
    cs_minmax,           # Cross-sectional Min-Max
    cs_rank,             # Cross-sectional rank (percentile)
)

# Support for group-wise operations
result = cs_zscore(data, groups=industry_labels)
```

### 3. Sample Splitting Engine

Time-series aware train/validation/test splitting:

```python
from qfeaturelib.splitting import (
    RollingWindowSplitter,
    ExpandingWindowSplitter,
    DateAwareRollingSplitter,      # NEW in 0.2 — slice by unique calendar dates
    DateAwareExpandingSplitter,    # NEW in 0.2 — expanding-train variant
)

# Rolling window (fixed training size)
rolling_splitter = RollingWindowSplitter(
    n_samples=1000,
    train_ratio=0.6,
    val_ratio=0.2,
    test_ratio=0.2,
    step=100,  # Roll forward 100 samples each iteration
    gap=0,     # Gap between train/val/test to prevent leakage
)

# Expanding window (growing training size)
expanding_splitter = ExpandingWindowSplitter(
    n_samples=1000,
    train_ratio=0.6,
    val_ratio=0.2,
    test_ratio=0.2,
    step=50,   # Expand by 50 samples each iteration
)

# Use split.apply() to split multiple arrays consistently
for split in rolling_splitter.split():
    (X_train, X_val, X_test), (y_train, y_val, y_test) = split.apply([X, y])

# Multi-asset panels: slice by unique calendar dates so no date is
# split across train / val / test (the row-index splitters above
# silently leak in this case).
date_splitter = DateAwareRollingSplitter(
    dates=df["date"].to_numpy(),
    train_ratio=0.7, val_ratio=0.1, test_ratio=0.2,
)
for split in date_splitter.split():
    (X_tr, X_va, X_te), (y_tr, y_va, y_te) = split.apply([X, y], axis=0)

# Label uses future N periods? Use the per-boundary gaps to embargo
# trading days between consecutive folds and avoid label leakage.
# (Available on all four splitters; ``train_val_gap`` /
# ``val_test_gap`` override ``gap`` independently.)
date_splitter = DateAwareRollingSplitter(
    dates=df["date"].to_numpy(),
    train_ratio=0.7, val_ratio=0.1, test_ratio=0.2,
    train_val_gap=5,   # e.g. label = future 5-day return
    val_test_gap=5,
)

# NEW in 0.5 — size-mode + anchors. Instead of guessing ratios so the
# OOS slice ends up at 2023Q1..2025Q2, you can pin it directly. Window
# sizes use calendar units (D/W/M/Q/Y); gaps stay in trading days.
date_splitter = DateAwareRollingSplitter(
    dates=df["date"].to_numpy(),     # datetime64, int(YYYYMMDD) or str supported
    train_size="3Y", val_size="6M", test_size="2Q", step="1Q",
    test_start_anchor="2023Q1",
    test_end_anchor="2025Q2",
    train_val_gap=5, val_test_gap=5,  # 5 trading-day embargo
)
# → 9 rolling folds whose test windows tile 2023Q1..2025Q2.
```


### 3a. Rolling Prediction Aggregation

When you have one prediction array per rolling fold, stitch them into
a single out-of-sample series:

```python
from qfeaturelib import aggregate_rolling_predictions

oos = aggregate_rolling_predictions(
    predictions=fold_predictions,     # list of (test_pred, ...) per fold
    datetime_labels=fold_datetime,    # list of [train_dates, val_dates, test_dates] per fold
    method="last",                    # 'last' | 'mean' on overlapping dates
)
```

### 3b. 3-D Sliding-Window Patches (sequence models)

Vectorised `(N, n_steps, F)` patch builder for transformer / RNN
inputs, ~3× faster than a hand-rolled Python loop and
cross-asset-safe:

```python
from qfeaturelib import make_sliding_patches

bundle = make_sliding_patches(
    X=features,                  # (T, F)
    y=labels,                    # optional (T,)
    n_steps=20, step_size=1,
    dates=df["date"].to_numpy(),
    assets=df["code"].to_numpy(),  # build patches per ticker; windows
                                   # cannot straddle two tickers
)
# bundle.X: (K, n_steps, F), bundle.y, bundle.dates, bundle.assets, bundle.row_idx
```

### 4. Missing Value Imputation

```python
from qfeaturelib.imputation import (
    ffill,          # Forward fill
    ffill_limit,    # Forward fill with limit (prevents stale data filling)
    cs_median_fill, # Cross-sectional median fill
    cs_mean_fill,   # Cross-sectional mean fill
)

# Forward fill with maximum 5 consecutive fills
result = ffill_limit(data, limit=5)
```

### 5. Feature Neutralization

Remove effects of control factors via regression residuals:

```python
from qfeaturelib.neutralization import (
    neutralize,
    industry_neutralize,
    size_neutralize,
)

# Industry neutralization
neutralized = industry_neutralize(feature, industry_labels)

# Size (market cap) neutralization
neutralized = size_neutralize(feature, log_market_cap)

# Custom control factors
neutralized = neutralize(feature, control_factors, method="ols")
```

### 6. Macro Indicators

Special handling for macro-economic indicators without asset dimension:

```python
from qfeaturelib import (
    macro_rolling_zscore,
    adapt_macro_to_panel,
)

# Direct standardization of 1D macro data
gdp_zscore = macro_rolling_zscore(gdp_growth, window=12, shift=1)

# Broadcast to panel format for combination with asset features
gdp_panel = adapt_macro_to_panel(gdp_growth, n_assets=50)  # (T,) -> (T, N)
```


### 7. Factor / Feature Correlation Analysis

Vectorised correlation computation for large factor panels, with an
optional **PyTorch / GPU** backend, plus generic visualisation that
treats grouping as opt-in:

```python
import numpy as np
from qfeaturelib.correlation import (
    cross_section_corr,
    lagged_autocorr,
    build_sample_keys,
    plot_correlation_heatmap,
    plot_autocorr_heatmap,
)

# X has shape (N, K) — N stacked (date, asset) rows, K factors
X = np.random.randn(100_000, 30).astype("float32")

# Same-period K x K correlation matrix.
# backend="auto" picks torch only on CUDA; otherwise falls back to numpy.
corr = cross_section_corr(X, method="pearson", backend="auto", device="cuda:0")

# Plain heatmap (no grouping)
plot_correlation_heatmap(corr, title="My factors")

# Grouped heatmap (factor family, industry, ...). The library is
# group-agnostic: pass any sequence of labels and you get coloured
# group bands + legend automatically.
groups = ["momentum"] * 10 + ["value"] * 10 + ["growth"] * 10
plot_correlation_heatmap(
    corr,
    groups=groups,
    sort_by_groups=True,
    title="Factors by family",
    group_legend_title="Family",
)

# Multi-lag cross-asset autocorrelation
time_idx = np.repeat(np.arange(200), 500)        # (date_idx, asset_idx)
entity_idx = np.tile(np.arange(500), 200)
keys, base = build_sample_keys(time_idx, entity_idx)
ac = lagged_autocorr(
    X, sample_keys=keys, base=base,
    lags=(1, 5, 10, 20), method="pearson",
    backend="auto", device="cuda:0",
)
plot_autocorr_heatmap(ac, lags=[1, 5, 10, 20])
```

Installation extras:

```bash
pip install qfeaturelib[gpu]   # adds torch for GPU acceleration
pip install qfeaturelib[viz]   # adds matplotlib for the plotting helpers
```

## Performance Benchmarks


On standard test data (T=5000, N=1000, F=50):

| Operation | Pandas | QFeatureLib | Speedup |
|-----------|--------|-------------|---------|
| Rolling Z-Score | ~5s | ~0.1s | 50x |
| Cross-sectional Z-Score | ~2s | ~0.02s | 100x |
| Rolling Rank | ~10s | ~0.5s | 20x |

## Design Principles

1. **Safety First**: Default `shift=1` prevents accidental future function usage
2. **Vectorization**: All core computations use NumPy vectorized operations
3. **Memory Efficiency**: Return views instead of copies, support in-place operations
4. **Type Safety**: Full type annotations, passes mypy strict mode

## Related Projects

- [AssetPanelForest](https://github.com/ElenYoung/AssetPanelForest) - Supervised clustering for panel data
- [MASFactorMiner](https://github.com/ElenYoung/MASFactorMiner) - Factor mining and analysis
- [GeneralBacktest](https://github.com/ElenYoung/GeneralBacktest) - Backtesting framework

## License

MIT License - see [LICENSE](LICENSE) file for details.

## Contributing

Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for version history and changes.

## Support

- GitHub Issues: [https://github.com/ElenYoung/QFeatureLib/issues](https://github.com/ElenYoung/QFeatureLib/issues)
- Documentation: [https://github.com/ElenYoung/QFeatureLib#readme](https://github.com/ElenYoung/QFeatureLib#readme)

---

**Note**: This library is part of a quantitative finance ecosystem. When implementing features, consider compatibility with downstream projects.
