# PostgreSQL Database Module Usage Guide

The PostgreSQL database module provides production-ready database integration for FastAPI
applications, featuring async/sync SQLAlchemy engines, connection pooling, transaction management,
and comprehensive health checks.

## Overview

This module generates a complete PostgreSQL integration layer that includes:

- **Async & Sync Engines**: SQLAlchemy engines optimized for both async and sync operations
- **Session Management**: FastAPI dependency injection for database sessions
- **Transaction Support**: Context managers for atomic database operations
- **Connection Pooling**: Configurable connection pools with health monitoring
- **Health Checks**: Built-in health endpoints for monitoring database connectivity
- **Test Support**: Separate test database configuration

## Prerequisites

- PostgreSQL 12.0+
- Python 3.10+
- SQLAlchemy 2.0+
- asyncpg (for async operations)
- psycopg (for sync operations)

## Installation

```bash
rapidkit add module db_postgres
```

This generates:

- `src/modules/free/database/db_postgres/postgres.py` - Main database integration
- `src/health/postgres.py` - Health check endpoints
- Integration tests for database functionality

## Configuration

### Environment Variables

Configure your database connection via environment variables:

```bash
# Required
DATABASE_URL=postgresql://user:password@localhost:5432/myapp

# Optional - Connection Pool Settings
DB_POOL_SIZE=10          # Default: 10
DB_MAX_OVERFLOW=20       # Default: 20
DB_POOL_RECYCLE=3600     # Default: 3600 seconds
DB_POOL_TIMEOUT=30       # Default: 30 seconds
DB_ECHO=false           # Default: false (set to true for SQL logging)

# Optional - Test Database
TEST_DATABASE_URL=postgresql://user:password@localhost:5432/myapp_test
```

### Settings Integration

The module integrates with the settings module for configuration:

```python
# src/modules/free/essentials/settings/settings.py (generated by settings module)
DATABASE_URL: str = Field(..., env="DATABASE_URL")
DB_POOL_SIZE: int = Field(default=10, env="DB_POOL_SIZE")
DB_MAX_OVERFLOW: int = Field(default=20, env="DB_MAX_OVERFLOW")
DB_POOL_RECYCLE: int = Field(default=3600, env="DB_POOL_RECYCLE")
DB_POOL_TIMEOUT: int = Field(default=30, env="DB_POOL_TIMEOUT")
DB_ECHO: bool = Field(default=False, env="DB_ECHO")
```

## Basic Usage

### FastAPI Dependency Injection

Use the provided dependencies in your route handlers:

```python
from fastapi import FastAPI, Depends, HTTPException
from sqlalchemy import select
from src.modules.free.database.db_postgres.postgres import (
    get_postgres_db,
    AsyncSession,
)
from my_app.models import User  # Your SQLAlchemy models

app = FastAPI()


@app.get("/users")
async def get_users(db: AsyncSession = Depends(get_postgres_db)):
    """Get all users."""
    result = await db.execute(select(User))
    users = result.scalars().all()
    return {"users": users}


@app.get("/users/{user_id}")
async def get_user(user_id: int, db: AsyncSession = Depends(get_postgres_db)):
    """Get a specific user."""
    result = await db.execute(select(User).where(User.id == user_id))
    user = result.scalar_one_or_none()

    if not user:
        raise HTTPException(status_code=404, detail="User not found")

    return {"user": user}
```

### Creating Records

```python
from pydantic import BaseModel
from src.modules.free.database.db_postgres.postgres import (
    transactional_async,
)


class UserCreate(BaseModel):
    name: str
    email: str


@app.post("/users")
async def create_user(user_data: UserCreate):
    """Create a new user with transaction safety."""

    async with transactional_async() as session:
        user = User(name=user_data.name, email=user_data.email)
        session.add(user)
        await session.flush()  # Get the generated ID
        await session.refresh(user)  # Load any defaults

        return {"user": user, "message": "User created successfully"}
```

### Sync Operations (Legacy Support)

For legacy code or background tasks:

```python
from src.modules.free.database.db_postgres.postgres import (
    get_sync_db,
    Session,
)


@app.get("/legacy-report")
def generate_report(db: Session = Depends(get_sync_db)):
    """Generate report using sync database access."""

    # Raw SQL queries
    result = db.execute(text("""
        SELECT COUNT(*) as user_count,
               AVG(created_at) as avg_age
        FROM users
        WHERE active = true
    """))

    data = result.fetchone()
    return {"user_count": data.user_count, "average_age_days": data.avg_age}
```

## Advanced Features

### Transaction Management

#### Async Transactions

```python
from src.modules.free.database.db_postgres.postgres import (
    transactional_async,
)


@app.post("/transfer-money")
async def transfer_money(from_user_id: int, to_user_id: int, amount: float):
    """Transfer money between users with transaction safety."""

    async with transactional_async() as session:
        # Check sender balance
        sender_result = await session.execute(
            select(User).where(User.id == from_user_id)
        )
        sender = sender_result.scalar_one()

        if sender.balance < amount:
            raise HTTPException(status_code=400, detail="Insufficient funds")

        # Check receiver exists
        receiver_result = await session.execute(
            select(User).where(User.id == to_user_id)
        )
        receiver = receiver_result.scalar_one_or_none()

        if not receiver:
            raise HTTPException(status_code=404, detail="Receiver not found")

        # Perform transfer
        sender.balance -= amount
        receiver.balance += amount

        # Log transaction
        transaction = Transaction(
            from_user_id=from_user_id,
            to_user_id=to_user_id,
            amount=amount,
            timestamp=datetime.utcnow(),
        )
        session.add(transaction)

        return {"message": "Transfer completed successfully"}
```

#### Nested Transactions

```python
async with transactional_async() as session:
    # Outer transaction
    user = User(name="John")
    session.add(user)

    try:
        async with session.begin_nested():  # Savepoint
            # Inner transaction - can be rolled back independently
            profile = UserProfile(user_id=user.id, bio="Test")
            session.add(profile)

            # Simulate error
            if some_condition:
                raise ValueError("Profile creation failed")

    except ValueError:
        # Inner transaction rolled back, but user is still saved
        pass

    return {"user_id": user.id}
```

### Connection Pool Monitoring

Monitor your connection pool status:

```python
from src.modules.free.database.db_postgres.postgres import get_pool_status


@app.get("/debug/pool-status")
async def pool_status():
    """Get current connection pool statistics."""
    status = await get_pool_status()
    return {
        "pool_size": status["pool_size"],
        "connections_checked_in": status["checked_in"],
        "connections_checked_out": status["checked_out"],
        "overflow_connections": status["overflow"],
        "total_connections": status["total_connections"],
    }
```

## Health Checks

### Automatic Health Endpoints

The module provides health check endpoints that are automatically registered:

```python
# Health check endpoint (automatically available)
GET / api / health / module / postgres

# Response:
{
    "status": "healthy",
    "timestamp": "2025-11-01T12:00:00Z",
    "database": {
        "connection": "ok",
        "pool_status": {
            "pool_size": 10,
            "checked_in": 8,
            "checked_out": 2,
            "overflow": 0,
        },
    },
}
```

### Manual Health Checks

Use health check functions directly in your code:

```python
from src.modules.free.database.db_postgres.postgres import (
    check_postgres_connection,
)


@app.get("/custom-health")
async def custom_health_check():
    """Custom health check with additional logic."""
    try:
        await check_postgres_connection()

        # Additional checks
        async with get_postgres_db() as db:
            # Check if critical tables exist
            result = await db.execute(text("SELECT COUNT(*) FROM users"))
            user_count = result.scalar()

        return {"status": "healthy", "database": "connected", "user_count": user_count}

    except Exception as e:
        return {"status": "unhealthy", "error": str(e)}
```

## Testing

### Unit Tests

Test your database operations:

```python
import pytest
from sqlalchemy import select
from src.modules.free.database.db_postgres.postgres import get_postgres_db
from my_app.models import User


@pytest.mark.asyncio
async def test_create_user():
    """Test user creation."""
    async for db in get_postgres_db():
        user = User(name="Test User", email="test@example.com")
        db.add(user)
        await db.commit()
        await db.refresh(user)

        assert user.id is not None
        assert user.name == "Test User"

        # Cleanup
        await db.delete(user)
        await db.commit()
        break
```

### Integration Tests

The module includes comprehensive integration tests:

```bash
# Run database integration tests
pytest tests/modules/integration/database/test_postgres_integration.py
```

### Test Database Configuration

Configure a separate test database:

```python
# In test settings
TEST_DATABASE_URL = "postgresql://test:test@localhost:5432/myapp_test"


# Tests automatically use test database when available
@pytest.fixture
async def test_db():
    """Test database session fixture."""
    async for db in get_postgres_db():
        # This will use TEST_DATABASE_URL if configured
        yield db
```

## Performance Optimization

### Connection Pool Tuning

Optimize pool settings based on your workload:

```python
# High-traffic application
DB_POOL_SIZE = 20
DB_MAX_OVERFLOW = 50
DB_POOL_RECYCLE = 1800  # 30 minutes

# Low-traffic application
DB_POOL_SIZE = 5
DB_MAX_OVERFLOW = 10
DB_POOL_RECYCLE = 3600  # 1 hour
```

### Query Optimization

Use efficient querying patterns:

```python
# ✅ Efficient: Load only needed columns
result = await db.execute(select(User.id, User.name).where(User.active == True))

# ✅ Efficient: Use selectinload for relationships
from sqlalchemy.orm import selectinload

result = await db.execute(
    select(User).options(selectinload(User.posts)).where(User.id == user_id)
)

# ❌ Inefficient: Load all columns when only some are needed
result = await db.execute(select(User).where(User.active == True))
```

### Connection Reuse

Reuse connections efficiently:

```python
# ✅ Good: Reuse session for multiple operations
async with transactional_async() as session:
    # Multiple operations in same transaction
    user = await session.get(User, user_id)
    user.posts = [Post(title="New Post")]
    await session.commit()

# ❌ Bad: Multiple separate sessions
async with get_postgres_db() as db1:
    user = await db1.get(User, user_id)

async with get_postgres_db() as db2:  # New connection
    post = Post(user_id=user.id, title="New Post")
    db2.add(post)
    await db2.commit()
```

## Migration Guide

### From Raw SQLAlchemy

```python
# Before: Manual engine/session management
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

engine = create_engine("postgresql://...")
SessionLocal = sessionmaker(bind=engine)


def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()


# After: Using RapidKit PostgreSQL module
from src.modules.free.database.db_postgres.postgres import get_postgres_db


# Same dependency injection pattern, but with better configuration
async def get_db(db: AsyncSession = Depends(get_postgres_db)):
    return db
```

### From Other ORMs

```python
# From Tortoise ORM
# user = await User.create(name="John")

# To SQLAlchemy with RapidKit
async with transactional_async() as session:
    user = User(name="John")
    session.add(user)
    await session.commit()
```

## Troubleshooting

See the [troubleshooting guide](troubleshooting.md) for common database issues and solutions.

## Examples

- [Basic CRUD Operations](examples/basic_usage.md)
- [Transaction Patterns](examples/transactions.md)
- [Connection Pooling](examples/connection_pooling.md)
