Metadata-Version: 2.5
Name: torchtrade
Version: 0.1.0
Summary: Reinforcement learning environments for trading applications
Project-URL: Homepage, https://github.com/TorchTrade/torchtrade
Project-URL: Bug Reports, https://github.com/TorchTrade/torchtrade/issues
Project-URL: Source, https://github.com/TorchTrade/torchtrade
Author-email: Sebastian Dittert <torchtradecontact@gmail.com>
License: MIT
License-File: LICENSE
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Requires-Dist: alpaca-py
Requires-Dist: ccxt>=4.0.0
Requires-Dist: datasets>=3.0
Requires-Dist: matplotlib
Requires-Dist: numpy>=1.26
Requires-Dist: pandas>=2.1
Requires-Dist: pybit>=5.0.0
Requires-Dist: python-binance
Requires-Dist: python-okx>=0.3.0
Requires-Dist: ta
Requires-Dist: tensordict<0.14,>=0.13
Requires-Dist: torch>=2.1.0
Requires-Dist: torchrl<0.14,>=0.13.2
Requires-Dist: wandb
Provides-Extra: chronos
Requires-Dist: chronos-forecasting; extra == 'chronos'
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material; extra == 'docs'
Provides-Extra: examples
Requires-Dist: hydra-core; extra == 'examples'
Requires-Dist: python-dotenv; extra == 'examples'
Requires-Dist: scikit-learn>=1.6.1; extra == 'examples'
Requires-Dist: tqdm; extra == 'examples'
Provides-Extra: llm
Requires-Dist: accelerate>=0.20.0; extra == 'llm'
Requires-Dist: bitsandbytes>=0.41.0; extra == 'llm'
Requires-Dist: feedparser>=6.0.0; extra == 'llm'
Requires-Dist: openai>=1.0.0; extra == 'llm'
Requires-Dist: peft>=0.10.0; extra == 'llm'
Requires-Dist: transformers>=4.30.0; extra == 'llm'
Requires-Dist: vllm>=0.6.0; extra == 'llm'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/TorchTrade/torchtrade/main/docs/images/torchtrade_white.png" alt="TorchTrade" width="250">
</p>

<p align="center">
  <a href="https://torchtrade.github.io/torchtrade.io/index.html">
    <img src="https://img.shields.io/badge/Website-torchtrade.io-blueviolet.svg" alt="Website">
  </a>

  <img src="https://img.shields.io/badge/python-3.11+-blue.svg" alt="Python 3.11+">
  <img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT">
  <img src="https://img.shields.io/badge/TorchRL-Compatible-green.svg" alt="TorchRL">

<a href="https://github.com/sponsors/TorchTrade">
  <img src="https://img.shields.io/badge/Sponsor-TorchTrade-ea4aaa.svg" alt="Sponsor">
</a>

<a href="https://discord.gg/ptqnufYmaj">
  <img src="https://img.shields.io/badge/Discord-Join%20Chat-5865F2.svg?logo=discord&logoColor=white" alt="Discord">
</a>
</p>

# TorchTrade

**A machine learning framework for algorithmic trading built on [TorchRL](https://github.com/pytorch/rl).**

TorchTrade's goal is to provide accessible deployment of RL methods to trading. The framework supports various RL methodologies including **online RL**, **offline RL**, **model-based RL**, **contrastive learning**, and many more areas of reinforcement learning research. Beyond RL, TorchTrade integrates traditional trading methods such as **rule-based strategies**, as well as modern approaches including **LLMs** (both local models and frontier model integrations) as trading actors.

TorchTrade provides modular environments for both live trading with major exchanges and offline backtesting. The framework supports:
- 🎯 **Multi-Timeframe Observations** - Train on 1m, 5m, 15m, 1h bars simultaneously
- 🤖 **Multiple RL Algorithms** - PPO, DQN, IQL, GRPO, DSAC, CTRL implementations
- 📊 **Feature Engineering** - Add technical indicators and custom features
- 🔴 **Live Trading** - Direct Alpaca, Binance, Bitget, Bybit, and OKX integrations (Polymarket is paper-only)
- 🧠 **LLM Integration** - Use GPT-4o-mini or local LLMs as trading agents
- 🔧 **LLM Tool Use** - Let LLM agents call tools mid-reasoning (e.g. live Google News for sentiment) before choosing an action
- 🎓 **LLM Fine-Tuning** - Train/fine-tune a local LLM actor on your own data with GRPO or [SAO](https://arxiv.org/abs/2607.07508) ([guide](https://github.com/TorchTrade/torchtrade/blob/main/docs/guides/llm-grpo-training.md))
- 📐 **Rule-Based Actors** - Hard-coded strategies for imitation learning and baselines
- 🔮 **Pretrained Encoder Transforms** - Foundation model embeddings for time series
- 📦 **Ready-to-Use Datasets** - Pre-processed OHLCV data at [HuggingFace/Torch-Trade](https://huggingface.co/Torch-Trade)
- 📈 **Research to Production** - Same code for backtesting and live deployment
- 🛠️ **[Claude Code Agents](https://code.claude.com/docs/en/sub-agents)** - Pre-built AI coding agents for developing and extending TorchTrade ([available on our website](https://torchtrade.github.io/torchtrade.io/index.html))
- 📝 **Research Articles** - In-depth articles on RL trading strategies and framework design ([get here](https://torchtrade.github.io/torchtrade.io/index.html))

> **⚠️ Work in Progress:** TorchTrade is under active development. We continuously add new features, improvements, and optimizations. Expect API changes, new environments, and enhanced functionality in future releases.
>
> **Current Scope:** The framework currently focuses on single-asset trading environments (one symbol per environment). Multi-asset portfolio optimization and cross-asset trading environments are planned for future releases.

---

## 📚 Website & Documentation

🌐 **[TorchTrade Website](https://torchtrade.github.io/torchtrade.io/index.html)** — Landing page with overview, features, Claude Code agents, and research articles

📖 **[TorchTrade Documentation](https://torchtrade.github.io/torchtrade/)** — Comprehensive guides, tutorials, and API reference

- **[Getting Started](https://torchtrade.github.io/torchtrade/getting-started/)** - Installation and first environment
- **[Environments](https://torchtrade.github.io/torchtrade/environments/offline/)** - Offline and online trading environments
- **[Examples](https://torchtrade.github.io/torchtrade/examples/)** - Training scripts for PPO, IQL, GRPO, and more
- **[Components](https://torchtrade.github.io/torchtrade/components/losses/)** - Loss functions, transforms, and actors
- **[Advanced Customization](https://torchtrade.github.io/torchtrade/guides/custom-features/)** - Custom features, rewards, and environments

---

## Quick Start

### 1. Installation

```bash
# Install UV (fast Python package installer)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone and install
git clone https://github.com/TorchTrade/torchtrade.git
cd torchtrade
uv sync
source .venv/bin/activate  # On Unix/macOS

# Optional: Install with extra features
uv sync --extra examples         # Deps the scripts under examples/ need
uv sync --extra llm              # LLM actors (OpenAI API + local vLLM/transformers)
uv sync --extra chronos          # Chronos forecasting transforms
uv sync --all-extras             # Install all optional dependencies
```

### 2. Your First Environment

```python
from torchtrade.envs.offline import SequentialTradingEnv, SequentialTradingEnvConfig
import pandas as pd

# Load OHLCV data
df = pd.read_csv("btcusdt_1m.csv")
df['timestamp'] = pd.to_datetime(df['timestamp'])

# Create environment (spot trading = long-only)
config = SequentialTradingEnvConfig(
    leverage=1,             # 1 = spot; >1 for leveraged futures
    action_levels=[0, 1],   # long-only; the default [-1, 0, 1] warns under leverage=1
    time_frames=["1min", "5min", "15min"],
    window_sizes=[12, 8, 8],
    execute_on="5Min",
    initial_cash=1000
)
env = SequentialTradingEnv(df, config)

# Run
tensordict = env.reset()
# reset() carries no action -- step() needs one set on the tensordict it is given.
tensordict["action"] = env.action_spec.rand()
tensordict = env.step(tensordict)
# step() writes the outcome under "next", leaving the input state intact.
print(f"Reward: {tensordict['next']['reward'].item()}")
```

### 3. Train Your First Policy

```bash
# Train PPO with default settings
uv run python examples/online_rl/ppo/train.py

# Customize with Hydra overrides
uv run python examples/online_rl/ppo/train.py \
    env.symbol="BTC/USD" \
    optim.lr=1e-4
```

For detailed tutorials, see **[Getting Started Guide](https://torchtrade.github.io/torchtrade/getting-started/)**.

---

## Live Environments

TorchTrade supports live trading with major exchanges:

| Environment | Exchange | Asset Type | Futures | Leverage | Bracket Orders |
|-------------|----------|------------|---------|----------|----------------|
| **AlpacaTorchTradingEnv** | Alpaca | Crypto | ❌ | ❌ | ❌ |
| **AlpacaSLTPTorchTradingEnv** | Alpaca | Crypto | ❌ | ❌ | ✅ |
| **BinanceFuturesTorchTradingEnv** | Binance | Crypto | ✅ | ✅ (1-125x) | ❌ |
| **BinanceFuturesSLTPTorchTradingEnv** | Binance | Crypto | ✅ | ✅ (1-125x) | ✅ |
| **BitgetFuturesTorchTradingEnv** | Bitget | Crypto | ✅ | ✅ (1-125x) | ❌ |
| **BitgetFuturesSLTPTorchTradingEnv** | Bitget | Crypto | ✅ | ✅ (1-125x) | ✅ |
| **BybitFuturesTorchTradingEnv** | Bybit | Crypto | ✅ | ✅ (1-100x) | ❌ |
| **BybitFuturesSLTPTorchTradingEnv** | Bybit | Crypto | ✅ | ✅ (1-100x) | ✅ |
| **OKXFuturesTorchTradingEnv** | OKX | Crypto | ✅ | ✅ (1-125x) | ❌ |
| **OKXFuturesSLTPTorchTradingEnv** | OKX | Crypto | ✅ | ✅ (1-125x) | ✅ |
| **PolymarketBetEnv** | Polymarket | Prediction markets | ❌ | ❌ | ❌ |

**Need another broker?** Request support for additional platforms (Interactive Brokers, Kraken, etc.) by [creating an issue](https://github.com/TorchTrade/torchtrade/issues/new) or emailing torchtradecontact@gmail.com.

See **[Online Environments Documentation](https://torchtrade.github.io/torchtrade/environments/online/)** for setup guides and examples.

---

## Trading Platforms

Live trading is supported on the platforms below. **Polymarket is the exception: it is
paper-only** — see its entry.

### 🪙 Cryptocurrency Trading

**[Binance](https://accounts.binance.com/register?ref=25015935)** - Leading cryptocurrency exchange
- **Supported by:** `BinanceFuturesTorchTradingEnv`, `BinanceFuturesSLTPTorchTradingEnv`
- **Features:** Spot & futures trading, up to 125x leverage, testnet available
- **Commission:** Maker 0.02% / Taker 0.04% (with BNB discount)
- **Get Started:** [Sign up for Binance](https://accounts.binance.com/register?ref=25015935)

**[Bitget](https://share.bitget.com/u/VGN302X2)** - Fast-growing cryptocurrency exchange
- **Supported by:** `BitgetFuturesTorchTradingEnv`, `BitgetFuturesSLTPTorchTradingEnv`
- **Features:** Futures trading with up to 125x leverage, testnet for safe testing
- **Commission:** Maker 0.02% / Taker 0.06%
- **Get Started:** [Sign up for Bitget](https://share.bitget.com/u/VGN302X2)

**[Bybit](https://www.bybit.eu/invite?ref=MX42GRV)** - Top cryptocurrency derivatives exchange
- **Supported by:** `BybitFuturesTorchTradingEnv`, `BybitFuturesSLTPTorchTradingEnv`
- **Features:** Futures trading with up to 100x leverage, native bracket orders (SL/TP), testnet for safe testing
- **Commission:** Maker 0.02% / Taker 0.055%
- **Get Started:** [Sign up for Bybit](https://www.bybit.eu/invite?ref=MX42GRV)

**[OKX](https://my.okx.com/en-eu/join/52629853)** - Leading global cryptocurrency exchange
- **Supported by:** `OKXFuturesTorchTradingEnv`, `OKXFuturesSLTPTorchTradingEnv`
- **Features:** Futures trading with up to 125x leverage, bracket orders via attachAlgoOrds, demo trading
- **Commission:** Maker 0.02% / Taker 0.05%
- **Get Started:** [Sign up for OKX](https://my.okx.com/en-eu/join/52629853)

### 🔮 Prediction Markets

**[Polymarket](https://polymarket.com/)** - Decentralized prediction market on Polygon
- **Supported by:** `PolymarketBetEnv`
- **Features:** Rolling one-shot bets on short-cadence binary markets (BTC/ETH/SOL up-or-down at 5m / 15m / 1h / 4h / daily cadences), Gamma API market scanner
- **⚠️ Paper trading only — not live:** `dry_run=False` raises. `py-clob-client` is archived/non-functional (Polymarket moved to CLOB V2), and the env holds bets to resolution while Polymarket only releases collateral on an on-chain redeem no client exposes — so a live bot would drain to zero *while winning*. Reviving live needs the V2 port **and** a redemption workflow.
- **Get Started:** [Browse markets at Polymarket](https://polymarket.com/)

### 📈 Crypto Spot API

**[Alpaca](https://alpaca.markets/)** - Commission-free trading API
- **Supported by:** `AlpacaTorchTradingEnv`, `AlpacaSLTPTorchTradingEnv`
- **Features:** Commission-free crypto spot, paper trading, real-time data
- **Best for:** crypto spot, algorithmic trading
- **Get Started:** [Sign up for Alpaca](https://alpaca.markets/signup)

---

## Support TorchTrade Development

- **Buy Me a Coffee**: [buymeacoffee.com/torchtrade](https://buymeacoffee.com/torchtrade)
- ⭐ **Star the repo**: Help others discover TorchTrade on [GitHub](https://github.com/TorchTrade/torchtrade)

Your support helps maintain the project, add new features, and keep documentation up-to-date!

---

<details>
<summary><h2>📦 Offline Environments</h2></summary>

All environments support both spot (`leverage=1`) and futures (`leverage>1`) trading via config.

| Environment | Bracket Orders | One-Step | Best For |
|-------------|----------------|----------|----------|
| **SequentialTradingEnv** | ❌ | ❌ | Standard sequential trading |
| **SequentialTradingEnvSLTP** | ✅ | ❌ | Risk management with SL/TP |
| **OneStepTradingEnv** | ✅ | ✅ | GRPO, contextual bandits |

See **[Offline Environments Documentation](https://torchtrade.github.io/torchtrade/environments/offline/)** for detailed guides.

</details>

<details>
<summary><h2>🚀 Training Algorithms & Examples</h2></summary>

TorchTrade includes implementations of multiple RL algorithms, all usable across any environment via Hydra config switching:

- **PPO** - `examples/online_rl/ppo/`
- **PPO + Chronos** (time series embeddings) - `examples/online_rl/ppo_chronos/`
- **DQN** - `examples/online_rl/dqn/`
- **IQL** - `examples/online_rl/iql/`
- **DSAC** - `examples/online_rl/dsac/`
- **GRPO** - `examples/online_rl/grpo/`
- **CTRL** - Research

### Run Training Examples

```bash
# PPO with default environment (sequential SLTP)
uv run python examples/online_rl/ppo/train.py

# PPO with different environments (switch via command-line)
uv run python examples/online_rl/ppo/train.py env=sequential_futures
uv run python examples/online_rl/ppo/train.py env=onestep_futures
uv run python examples/online_rl/ppo/train.py env=sequential_spot

# GRPO with default (one-step futures)
uv run python examples/online_rl/grpo/train.py

# GRPO with spot trading
uv run python examples/online_rl/grpo/train.py env=onestep_spot

# Customize with Hydra overrides
uv run python examples/online_rl/ppo/train.py \
    env=sequential_futures \
    env.symbol="ETH/USD" \
    env.leverage=10 \
    optim.lr=1e-4 \
    loss.gamma=0.95
```

**Available environment configs** (`env=<name>`):
- `sequential_spot` - Basic spot trading
- `sequential_futures` - Basic futures trading
- `sequential_sltp` - Spot with bracket orders
- `sequential_futures_sltp` - Futures with bracket orders
- `onestep_spot` - Contextual bandit (spot)
- `onestep_futures` - Contextual bandit (futures)

See **[Examples Documentation](https://torchtrade.github.io/torchtrade/examples/)** for all available examples.

</details>

<details>
<summary><h2>🔧 Installation & Setup</h2></summary>

### Prerequisites

- Python 3.11+
- CUDA (optional, for GPU acceleration)
- [UV](https://docs.astral.sh/uv/) - Fast Python package installer

### Full Installation

```bash
# 1. Install UV
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# 2. Clone repository
git clone https://github.com/TorchTrade/torchtrade.git
cd torchtrade

# 3. Install dependencies
uv sync

# Optional: Install with extra features
# uv sync --extra examples         # Deps the scripts under examples/ need
# uv sync --extra llm              # LLM actors (OpenAI API + local vLLM/transformers)
# uv sync --extra chronos          # Chronos forecasting transforms
# uv sync --extra dev              # Development/testing tools
# uv sync --extra docs             # Documentation building
# uv sync --all-extras             # Install all optional dependencies

# 4. Activate virtual environment
source .venv/bin/activate  # Unix/macOS
# .venv\Scripts\activate  # Windows

# 5. For live trading, create .env file
cat > .env << EOF
ALPACA_API_KEY=your_alpaca_api_key
ALPACA_SECRET_KEY=your_alpaca_secret_key
BINANCE_API_KEY=your_binance_api_key
BINANCE_SECRET_KEY=your_binance_secret_key
BYBIT_API_KEY=your_bybit_api_key
BYBIT_API_SECRET=your_bybit_api_secret
OKX_API_KEY=your_okx_api_key
OKX_API_SECRET=your_okx_api_secret
OKX_PASSPHRASE=your_okx_passphrase
EOF

# 6. Verify installation
uv run pytest tests/ -v
```

</details>

<details>
<summary><h2>💡 Common Use Cases</h2></summary>

### Training PPO on Backtesting Data

```python
from torchtrade.envs.offline import SequentialTradingEnv, SequentialTradingEnvConfig
import datasets
import pandas as pd

# Load historical data from HuggingFace
df = datasets.load_dataset("Torch-Trade/btcusdt_spot_1m_03_2023_to_12_2025")
df = df["train"].to_pandas()
df['timestamp'] = pd.to_datetime(df['timestamp'])

# Configure multi-timeframe environment
config = SequentialTradingEnvConfig(
    leverage=1,             # spot
    action_levels=[0, 1],   # long-only; the default [-1, 0, 1] warns under leverage=1
    time_frames=["1min", "5min", "15min", "60min"],
    window_sizes=[12, 8, 8, 24],
    execute_on="5Min",
    initial_cash=[1000, 5000],  # Domain randomization
    transaction_fee=0.0025,
    slippage=0.001
)

env = SequentialTradingEnv(df, config)
# Train with PPO - see examples/online_rl/ppo/train.py
```

### Live Trading with Alpaca

```python
import os

from dotenv import load_dotenv
from torchtrade.envs.live.alpaca import AlpacaTorchTradingEnv, AlpacaTradingEnvConfig
from alpaca.data.timeframe import TimeFrame, TimeFrameUnit

load_dotenv()  # reads the .env written above; os.getenv does not read files

config = AlpacaTradingEnvConfig(
    symbol="BTC/USD",
    time_frames=[
        TimeFrame(1, TimeFrameUnit.Minute),
        TimeFrame(5, TimeFrameUnit.Minute),
    ],
    window_sizes=[12, 8],
    execute_on=TimeFrame(5, TimeFrameUnit.Minute),
    paper=True  # Start with paper trading!
)

env = AlpacaTorchTradingEnv(
    config,
    api_key=os.getenv("ALPACA_API_KEY"),
    api_secret=os.getenv("ALPACA_SECRET_KEY"),
)
# See examples/online_rl/ppo/live.py
```

### LLM-Based Trading

```python
from torchtrade.actor import FrontierLLMActor

# Use GPT as trading policy
policy = FrontierLLMActor(
    model="gpt-4o-mini",
    market_data_keys=env.market_data_keys,
    account_state_labels=env.account_state,  # list of label strings, e.g. ["exposure_pct", ...]
    action_levels=env.action_levels,
    debug=True,
)

tensordict = env.reset()
action = policy(tensordict)
# See examples/llm/frontier/offline.py
```

### Rule-Based Trading Strategies

```python
from torchtrade.actor import MeanReversionActor

# Use as baseline or for imitation learning
actor = MeanReversionActor(
    market_data_keys=["market_data_5Minute_24"],
)
```

### Feature Engineering

```python
import ta

def custom_preprocessing(df):
    """Add technical indicators as features"""
    df["features_open"] = df["open"]
    df["features_close"] = df["close"]
    df["features_rsi_14"] = ta.momentum.RSIIndicator(
        df["close"], window=14
    ).rsi()
    df.fillna(0, inplace=True)
    # timestamp must come back as a COLUMN, not an index, or the sampler raises
    # KeyError: ['timestamp'] not in index
    return df.reset_index()

config = SequentialTradingEnvConfig(
    leverage=1,
    action_levels=[0, 1],   # long-only; the default [-1, 0, 1] warns under leverage=1
    time_frames=["1min", "5min"],
    window_sizes=[12, 8],
)

# feature_preprocessing_fn is a CONSTRUCTOR argument, not a config field.
env = SequentialTradingEnv(df, config, custom_preprocessing)
```

See **[Advanced Customization](https://torchtrade.github.io/torchtrade/guides/custom-features/)** for more examples.

</details>


<details>
<summary><h2>🎯 Key Concepts</h2></summary>

### Multi-Timeframe Observations

```python
config = SequentialTradingEnvConfig(
    leverage=1,
    action_levels=[0, 1],   # long-only; the default [-1, 0, 1] warns under leverage=1
    time_frames=["1min", "5min", "15min", "60min"],
    window_sizes=[12, 8, 8, 24],
    execute_on="5Min"
)

# Results in observations:
# - market_data_1Minute_12: [12, num_features] - Last 12 one-minute bars
# - market_data_5Minute_8: [8, num_features] - Last 40 minutes
# - market_data_15Minute_8: [8, num_features] - Last 120 minutes
# - market_data_60Minute_24: [24, num_features] - Last 24 hours
```

### Observation Structure

```python
observation = {
    "market_data_1Minute_12": tensor([12, num_features]),
    "market_data_5Minute_8": tensor([8, num_features]),
    "account_state": tensor([6]),  # Universal 6-element state
}

# Account state (universal): [exposure_pct, position_direction, unrealized_pnl_pct,
#                             holding_time, leverage, distance_to_liquidation]
# Element definitions:
#   - exposure_pct: position_value / portfolio_value (0-1+ with leverage)
#   - position_direction: sign(position_size) (-1=short, 0=flat, +1=long)
#   - unrealized_pnl_pct: (current_price - entry_price) / entry_price * direction
#   - holding_time: bars the current position has been held (1 on the opening bar, 0 when flat)
#   - leverage: 1.0 for spot, 1-125 for futures
#   - distance_to_liquidation: normalized distance (1.0 for spot/no position)
#
# Spot mode: position_direction in {0, +1}, leverage=1.0, distance_to_liquidation=1.0
# Futures mode: position_direction in {-1, 0, +1}, leverage=1-125, calculated distance
```

### Action Spaces

**Standard (3 actions):**
- Action 0: SELL/SHORT
- Action 1: HOLD
- Action 2: BUY/LONG

**SLTP Combinatorial:**
- Action 0: HOLD
- Actions 1..N: BUY/LONG with (SL, TP) combinations
- Actions N+1..2N: SHORT with (SL, TP) combinations (futures only)

See **[Advanced Customization](https://torchtrade.github.io/torchtrade/guides/custom-features/)** for detailed explanations.

</details>

<details>
<summary><h2>⚙️ Configuration with Hydra</h2></summary>

TorchTrade uses Hydra for configuration management with a defaults list pattern:

```yaml
# examples/online_rl/ppo/config.yaml
defaults:
  - env: sequential_sltp  # Load environment config
  - _self_

collector:
  frames_per_batch: 100000
  total_frames: 100_000_000

optim:
  lr: 2.5e-4
  anneal_lr: true
  max_grad_norm: 0.5

loss:
  gamma: 0.9
  clip_epsilon: 0.1
  entropy_coef: 0.01
```

```yaml
# examples/online_rl/env/sequential_sltp.yaml
env:
  name: SequentialTradingEnvSLTP
  leverage: 1
  symbol: "BTC/USD"
  time_frames: ["5Min", "15Min"]
  window_sizes: [10, 10]
  execute_on: "15Min"
  initial_cash: [1000, 5000]
  transaction_fee: 0.0025
  # ... more env config
```

Override from command line:

```bash
# Switch environment entirely
uv run python examples/online_rl/ppo/train.py env=sequential_futures

# Override specific parameters
uv run python examples/online_rl/ppo/train.py \
    env.symbol="ETH/USD" \
    env.leverage=10 \
    optim.lr=1e-4 \
    loss.gamma=0.95
```

</details>


---

## Contributing

We welcome contributions! To contribute:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Run tests (`pytest tests/ -v`)
5. Commit your changes (`git commit -m 'Add amazing feature'`)
6. Push to the branch (`git push origin feature/amazing-feature`)
7. Open a Pull Request

### Development Setup

```bash
# Install with development dependencies
uv sync --extra dev

# Run tests
uv run pytest tests/ -v

# Run tests with coverage
uv run pytest tests/ -v --cov=torchtrade --cov-report=html

# Build documentation
mkdocs serve
```

### Reporting Issues

Found a bug or have a feature request?

[GitHub Issues](https://github.com/TorchTrade/torchtrade/issues)

---

## License

MIT License - See [LICENSE](https://github.com/TorchTrade/torchtrade/blob/main/LICENSE) file for details.

---

## Support

- 📧 **Email**: torchtradecontact@gmail.com

---

**Built with TorchRL • Designed for Algorithmic Trading • Open Source**
