Metadata-Version: 2.4
Name: terminatodo
Version: 2.0.0
Summary: Framework unificado de gestión de almacenamiento para Deep Learning con TenMiNaTor
Author-email: Cristian <yoqer@github.com>
License: MIT
Project-URL: Homepage, https://github.com/yoqer/TerminaTodo
Project-URL: Repository, https://github.com/yoqer/TerminaTodo
Project-URL: Issues, https://github.com/yoqer/TerminaTodo/issues
Project-URL: Documentation, https://github.com/yoqer/TerminaTodo#readme
Keywords: deep-learning,storage,cloud,sftp,training,tenminator
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Filesystems
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.110.0
Requires-Dist: uvicorn[standard]>=0.27.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: beautifulsoup4>=4.12.0
Provides-Extra: cloud
Requires-Dist: google-auth>=2.28.0; extra == "cloud"
Requires-Dist: google-auth-oauthlib>=1.2.0; extra == "cloud"
Requires-Dist: google-api-python-client>=2.120.0; extra == "cloud"
Requires-Dist: dropbox>=11.36.0; extra == "cloud"
Requires-Dist: msal>=1.26.0; extra == "cloud"
Provides-Extra: sftp
Requires-Dist: paramiko>=3.4.0; extra == "sftp"
Provides-Extra: s3
Requires-Dist: boto3>=1.34.0; extra == "s3"
Provides-Extra: monitoring
Requires-Dist: psutil>=5.9.0; extra == "monitoring"
Provides-Extra: all
Requires-Dist: terminatodo[cloud,monitoring,s3,sftp]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
Requires-Dist: httpx>=0.27.0; extra == "dev"
Requires-Dist: black>=24.0.0; extra == "dev"
Requires-Dist: ruff>=0.3.0; extra == "dev"
Requires-Dist: mypy>=1.9.0; extra == "dev"
Dynamic: license-file

# TerminaTodo v2.0

**Framework unificado de gestión de almacenamiento para Deep Learning**

[![PyPI version](https://badge.fury.io/py/terminatodo.svg)](https://pypi.org/project/terminatodo/)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![GitHub](https://img.shields.io/badge/github-yoqer%2FTerminaTodo-black)](https://github.com/yoqer/TerminaTodo)

---

## ¿Qué es TerminaTodo?

TerminaTodo conecta **todos tus almacenamientos** en una sola interfaz y los hace disponibles para entrenar modelos con **TenMiNaTor**:

| Tipo | Soportado |
|------|-----------|
| Local (SSD, HDD, USB, RAM) | ✅ |
| Google Drive | ✅ |
| Microsoft OneDrive | ✅ |
| Dropbox | ✅ |
| SFTP | ✅ |
| FTP | ✅ |
| HTTP/S (Apache/Nginx dir listing) | ✅ |
| S3-compatible (MinIO, Wasabi, etc.) | ✅ |

---

## Instalación

```bash
# Instalación básica (API + almacenamiento local)
pip install terminatodo

# Con soporte completo de nube
pip install terminatodo[cloud]

# Con SFTP
pip install terminatodo[sftp]

# Con S3
pip install terminatodo[s3]

# Todo incluido
pip install terminatodo[all]
```

---

## Inicio rápido

### Como librería Python

```python
from terminatodo import TerminaTodo

tt = TerminaTodo()

# Ver todos los almacenamientos
print(tt.list_all())

# Sincronizar datos locales → Google Drive
tt.sync("local:/home/user/datasets", "gdrive:/ML_Datasets")

# Iniciar entrenamiento con TenMiNaTor
from terminatodo.training.training_bridge import TrainingBridge
bridge = TrainingBridge(unified_manager=tt.unified)
session_id = bridge.start_training(
    data_source_uri="local:/data/train.csv",
    model_config={"epochs": 50, "learning_rate": 0.001},
    output_uri="local:/models/",
)
```

### Como API REST

```bash
# Iniciar servidor
terminatodo start --port 8765

# Documentación interactiva en: http://localhost:8765/docs
```

### Como CLI

```bash
# Listar almacenamientos
terminatodo list

# Sincronizar
terminatodo sync local:/data gdrive:/backup

# Entrenamiento
terminatodo training start local:/data/train.csv --epochs 20 --lr 0.001

# Ver sesiones
terminatodo training list
```

---

## Configuración de nube

### Google Drive

```python
import json
tt.cloud.authenticate_google_drive(json.dumps({
    "type": "service_account",
    "project_id": "mi-proyecto",
    "private_key_id": "...",
    "private_key": "-----BEGIN PRIVATE KEY-----\n...",
    "client_email": "mi-cuenta@mi-proyecto.iam.gserviceaccount.com",
    # ... resto del JSON de cuenta de servicio
}))
```

### Dropbox

```python
tt.cloud.authenticate_dropbox("sl.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
```

### OneDrive

```python
tt.cloud.authenticate_onedrive(
    client_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    client_secret="tu_secreto",
    tenant_id="common",  # o tu tenant ID
)
```

---

## Servidores remotos

```python
# SFTP
tt.remote.add_server({
    "server_id": "mi_servidor",
    "name": "Servidor ML",
    "host": "192.168.1.100",
    "port": 22,
    "protocol": "sftp",
    "username": "ubuntu",
    "password": "mi_password",
})

# S3-compatible (MinIO)
tt.remote.add_server({
    "server_id": "minio_local",
    "name": "MinIO Local",
    "host": "localhost",
    "port": 9000,
    "protocol": "s3",
    "username": "minioadmin",      # access_key
    "password": "minioadmin",      # secret_key
    "api_key": "http://localhost:9000",  # endpoint URL
})

# Listar archivos
files = tt.remote.list_files("mi_servidor", "/datasets/")
```

---

## API REST — Endpoints principales

| Método | Endpoint | Descripción |
|--------|----------|-------------|
| POST | `/auth/login` | Obtener JWT |
| GET | `/storage/all` | Ver todos los almacenamientos |
| GET | `/storage/local/devices` | Dispositivos locales |
| GET | `/storage/local/files?path=/data` | Archivos locales |
| GET | `/storage/cloud/services` | Servicios en nube |
| POST | `/storage/cloud/auth/google` | Autenticar Google Drive |
| POST | `/storage/cloud/auth/dropbox` | Autenticar Dropbox |
| GET | `/storage/cloud/{service}/files` | Listar archivos en nube |
| GET | `/storage/remote/servers` | Servidores remotos |
| POST | `/storage/remote/servers` | Añadir servidor |
| GET | `/storage/remote/{id}/files` | Archivos en servidor |
| POST | `/sync` | Sincronizar |
| GET | `/sync/history` | Historial de sync |
| POST | `/training/start` | Iniciar entrenamiento |
| GET | `/training/sessions` | Ver sesiones |

---

## Variables de entorno

| Variable | Descripción | Default |
|----------|-------------|---------|
| `TERMINATODO_SECRET` | Clave secreta JWT | `change-me-in-production-32chars!!` |
| `TERMINATODO_USER` | Usuario admin API | `admin` |
| `TERMINATODO_PASS` | Contraseña admin API | `terminatodo2024` |

**⚠️ Cambiar en producción:**
```bash
export TERMINATODO_SECRET="$(python3 -c 'import secrets; print(secrets.token_hex(32))')"
export TERMINATODO_USER="mi_usuario"
export TERMINATODO_PASS="mi_contraseña_segura"
```

---

## Tests

```bash
pip install terminatodo[dev]
pytest tests/ -v
```

---

## Integración con TenMiNaTor Web App

TerminaTodo se integra directamente con la app web de TenMiNaTor. El endpoint `/training/start` acepta cualquier URI de almacenamiento como fuente de datos.

```bash
# Desde la app web (tenminator-web)
curl -X POST http://localhost:8765/training/start \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "data_source_uri": "gdrive:/datasets/train.csv",
    "model_config": {"epochs": 50, "learning_rate": 0.001, "batch_size": 32},
    "output_uri": "local:/models/"
  }'
```

---

## Estructura del proyecto

```
terminatodo/
├── __init__.py          # Exports principales
├── core.py              # Clase TerminaTodo principal
├── cli.py               # CLI (terminatodo start/list/sync/training)
├── api/
│   └── app.py           # FastAPI con todos los endpoints
├── auth/
│   └── jwt_auth.py      # JWT HS256 + hash de contraseñas
├── db/
│   └── database.py      # SQLite persistencia
├── storage/
│   ├── local_storage.py # SSD/HDD/USB/RAM
│   └── unified_manager.py # Vista unificada
├── cloud/
│   └── cloud_manager.py # Google Drive, Dropbox, OneDrive
├── hosting/
│   └── remote_manager.py # SFTP, FTP, HTTP/S, S3
├── sync/
│   └── sync_manager.py  # Sincronización bidireccional
└── training/
    └── training_bridge.py # Integración TenMiNaTor
tests/
    └── test_terminatodo.py # 25+ tests
```

---

## Licencia

MIT © 2024 Cristian / [yoqer](https://github.com/yoqer)
