# YouTube URL Handling in Unified Config

## Overview

The unified config system now supports YouTube URLs as a special content type that can **ONLY** be processed by Google Gemini models. All other providers will skip YouTube URLs with a warning message.

## Implementation

### New Content Type: `YouTubeVideoContent`

Located in `ai/unified_config.py`, this class handles YouTube video URLs:

```python
@dataclass
class YouTubeVideoContent:
    type: Literal["youtube_video"] = "youtube_video"
    youtube_url: str = ""
    video_metadata: Optional[Dict[str, Any]] = None
    metadata: Dict[str, Any] = field(default_factory=dict)
```

### Supported YouTube URL Formats

- `https://www.youtube.com/watch?v=VIDEO_ID`
- `https://youtu.be/VIDEO_ID`

## Data Flow

### 1. Input Processing

When a message contains YouTube video content:

```json
{
  "role": "user",
  "content": [
    {
      "type": "youtube_video",
      "youtube_url": "https://www.youtube.com/watch?v=9hE5-98ZeCg"
    },
    {
      "type": "text",
      "text": "Please summarize this video."
    }
  ]
}
```

### 2. Deserialization (`UnifiedMessage.from_dict`)

The `from_dict` method in `UnifiedMessage` recognizes the `youtube_video` type and creates a `YouTubeVideoContent` instance:

```python
elif content_type == "youtube_video":
    parsed_content.append(YouTubeVideoContent(**item))
```

### 3. Provider Translation

#### For Google Gemini (`to_google()`)

YouTube URLs are converted to Google's `fileData` format:

```python
def to_google(self) -> Optional[Dict[str, Any]]:
    if not self.youtube_url:
        return None
    
    # Google accepts YouTube URLs directly in fileUri
    part = {
        "fileData": {
            "fileUri": self.youtube_url,
        }
    }
    
    if self.video_metadata:
        part["videoMetadata"] = self.video_metadata
    
    return part
```

**Result**: The YouTube URL is passed directly to Google's API via `fileData.fileUri`.

#### For OpenAI (`to_openai()`)

```python
def to_openai(self) -> Optional[Dict[str, Any]]:
    from matrx_utils import vcprint
    vcprint(
        f"YouTube URL '{self.youtube_url}' is not supported by OpenAI models and will be skipped.",
        "YouTube URL Warning",
        color="yellow"
    )
    return None
```

**Result**: Returns `None` and prints a warning. The content is excluded from the request.

#### For Anthropic (`to_anthropic()`)

```python
def to_anthropic(self) -> Optional[Dict[str, Any]]:
    from matrx_utils import vcprint
    vcprint(
        f"YouTube URL '{self.youtube_url}' is not supported by Anthropic models and will be skipped.",
        "YouTube URL Warning",
        color="yellow"
    )
    return None
```

**Result**: Returns `None` and prints a warning. The content is excluded from the request.

### 4. Response Processing (from Google)

When Google returns a response with a YouTube video reference, the `from_google` method checks if the `file_uri` is a YouTube URL:

```python
@classmethod
def from_google(cls, part: Any) -> Optional["YouTubeVideoContent"]:
    if not hasattr(part, "file_data") or not part.file_data:
        return None
    
    file_uri = getattr(part.file_data, "file_uri", "")
    
    # Check if it's a YouTube URL
    if not file_uri or not ("youtube.com" in file_uri or "youtu.be" in file_uri):
        return None
    
    # Extract video metadata if present
    video_metadata = None
    if hasattr(part, "video_metadata") and part.video_metadata:
        video_metadata = {
            "start_offset": getattr(part.video_metadata, "start_offset", None),
            "end_offset": getattr(part.video_metadata, "end_offset", None),
            "fps": getattr(part.video_metadata, "fps", None),
        }
    
    return cls(
        youtube_url=file_uri,
        video_metadata=video_metadata,
    )
```

## Complete Data Flow Diagram

```
┌─────────────────────────────────────────────────────────────┐
│ 1. INPUT: User Message with YouTube URL                     │
│    {                                                         │
│      "type": "youtube_video",                               │
│      "youtube_url": "https://www.youtube.com/watch?v=..."   │
│    }                                                         │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│ 2. DESERIALIZATION: UnifiedMessage.from_dict()              │
│    → Creates YouTubeVideoContent instance                   │
│    → Added to UnifiedMessage.content list                   │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│ 3. TRANSLATION: UnifiedMessage.to_google_content()          │
│    → Calls YouTubeVideoContent.to_google()                  │
│    → Converts to Google's fileData format                   │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│ 4. GOOGLE API CALL: GoogleTranslator.to_google()            │
│    {                                                         │
│      "model": "gemini-3-flash-preview",                           │
│      "contents": [                                           │
│        {                                                     │
│          "role": "user",                                     │
│          "parts": [                                          │
│            {                                                 │
│              "fileData": {                                   │
│                "fileUri": "https://www.youtube.com/..."     │
│              }                                               │
│            },                                                │
│            { "text": "Please summarize this video." }       │
│          ]                                                   │
│        }                                                     │
│      ]                                                       │
│    }                                                         │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│ 5. GOOGLE RESPONSE: GoogleTranslator.from_google()          │
│    → Processes response chunks                              │
│    → Handles file_data parts                                │
│    → Checks if file_uri is YouTube URL                      │
│    → Creates YouTubeVideoContent if applicable              │
└─────────────────────────────────────────────────────────────┘
                            ↓
┌─────────────────────────────────────────────────────────────┐
│ 6. OUTPUT: UnifiedResponse with processed content           │
│    → YouTube video content preserved in unified format      │
│    → Can be used in subsequent messages                     │
└─────────────────────────────────────────────────────────────┘
```

## Behavior by Provider

| Provider | Behavior | Warning Message |
|----------|----------|-----------------|
| **Google Gemini** | ✅ Processes YouTube URLs directly via `fileData.fileUri` | None |
| **OpenAI** | ❌ Skips content, returns `None` | "YouTube URL '...' is not supported by OpenAI models and will be skipped." |
| **Anthropic** | ❌ Skips content, returns `None` | "YouTube URL '...' is not supported by Anthropic models and will be skipped." |
| **Together** | ❌ Skips content, returns `None` | "YouTube URL '...' is not supported by Together models and will be skipped." |
| **Groq** | ❌ Skips content, returns `None` | "YouTube URL '...' is not supported by Groq models and will be skipped." |
| **xAI** | ❌ Skips content, returns `None` | "YouTube URL '...' is not supported by xAI models and will be skipped." |
| **Cerebras** | ❌ Skips content, returns `None` | "YouTube URL '...' is not supported by Cerebras models and will be skipped." |

## Example Usage

### Test Case (in `ai/tests/execution_test.py`)

```python
youtube_url_settings = {
    "ai_model_id": google,  # Must be a Google model
    "conversation_id": "12345678-1234-1234-1234-123456789012",
    "user_id": "12345678-1234-1234-1234-123456789012",
    "debug": True,
    "config": {
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "youtube_video",
                        "youtube_url": "https://www.youtube.com/watch?v=9hE5-98ZeCg",
                    },
                    {
                        "type": "text",
                        "text": "Please summarize this video in 3 sentences.",
                    },
                ],
            },
        ],
        "stream": True,
    },
}
```

### Programmatic Usage

```python
from ai.config.unified_config import UnifiedMessage, YouTubeVideoContent, TextContent

# Create a message with YouTube video
message = UnifiedMessage(
    role="user",
    content=[
        YouTubeVideoContent(
            youtube_url="https://www.youtube.com/watch?v=9hE5-98ZeCg"
        ),
        TextContent(
            text="Please summarize this video in 3 sentences."
        )
    ]
)
```

## Integration Points

### 1. Content Type Union

`YouTubeVideoContent` is included in the `UnifiedContent` union type:

```python
UnifiedContent = Union[
    TextContent,
    ImageContent,
    AudioContent,
    VideoContent,
    YouTubeVideoContent,  # ← Added here
    DocumentContent,
    ToolCallContent,
    ToolResultContent,
    ThinkingContent,
    CodeExecutionContent,
    CodeExecutionResultContent,
    WebSearchCallContent,
]
```

### 2. Message Conversion

In `UnifiedMessage.to_google_content()`:

```python
def to_google_content(self) -> Optional[Dict[str, Any]]:
    parts = []
    for content in self.content:
        part = content.to_google()  # Calls YouTubeVideoContent.to_google()
        if part:
            parts.append(part)
    # ... rest of method
```

### 3. Response Processing

In `GoogleTranslator.from_google()`:

```python
# Handle file_data (could be YouTube URL or regular file)
elif part.file_data:
    converted = (
        YouTubeVideoContent.from_google(part)  # ← Checks for YouTube URLs first
        or ImageContent.from_google(part)
        or AudioContent.from_google(part)
        or VideoContent.from_google(part)
        or DocumentContent.from_google(part)
    )
    if converted:
        content.append(converted)
```

## Key Design Decisions

1. **Separate Content Type**: YouTube videos are a distinct content type (`YouTubeVideoContent`) rather than being part of `VideoContent` because:
   - They have fundamentally different handling (URL-only vs. file upload)
   - Only Google supports them
   - Clear separation makes the warning system straightforward

2. **Warning Messages**: Non-Google providers print warnings to `vcprint` (visible in debug logs) rather than raising exceptions, allowing the request to continue with other content.

3. **No MIME Type**: YouTube URLs don't need a `mime_type` field because Google handles them directly without needing format information.

4. **Video Metadata Support**: The class supports optional `video_metadata` (start_offset, end_offset, fps) for advanced use cases where users want to specify a specific portion of the video.

## Testing

To test YouTube URL handling:

1. Use the test case in `ai/tests/execution_test.py`
2. Set `settings_to_use = youtube_url_settings`
3. Ensure `ai_model_id` is set to a Google model
4. Run the test and verify the video is processed correctly

To test warning behavior:

1. Change `ai_model_id` to a non-Google model (e.g., OpenAI, Anthropic)
2. Run the test and verify warning messages appear in the output
3. Confirm the request completes successfully (without the YouTube content)

## Future Enhancements

Potential improvements for future consideration:

1. **Automatic YouTube URL Detection**: Detect YouTube URLs in regular text and suggest converting them to `youtube_video` content type
2. **Thumbnail Extraction**: For non-Google providers, automatically extract and include a thumbnail image
3. **Transcript Fallback**: For non-Google providers, fetch the YouTube transcript and include it as text
4. **Validation**: Add URL validation to ensure the YouTube URL is valid before sending to Google
5. **Playlist Support**: Extend to support YouTube playlist URLs
