Metadata-Version: 2.4
Name: dataforge-dfg
Version: 0.1.0
Summary: A lightweight ELT framework in pure Python — dbt meets dlt
Author-email: Matheus <matheusdigiacomo12@email.com>
License: MIT License
        
        Copyright (c) 2026 Matheus
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/MatheusGiacomo/DataForge-DFG
Project-URL: Documentation, https://github.com/MatheusGiacomo/DataForge-DFG/blob/main/DOCUMENTACAO.md
Project-URL: Repository, https://github.com/MatheusGiacomo/DataForge-DFG
Project-URL: Bug Tracker, https://github.com/MatheusGiacomo/DataForge-DFG/issues
Project-URL: Changelog, https://github.com/MatheusGiacomo/DataForge-DFG/blob/main/CHANGELOG.md
Keywords: elt,etl,data,pipeline,dbt,dlt,sql,analytics
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Classifier: Environment :: Console
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jinja2>=3.1.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: duckdb
Requires-Dist: duckdb>=0.10.0; extra == "duckdb"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"
Provides-Extra: mysql
Requires-Dist: mysql-connector-python>=8.0.0; extra == "mysql"
Provides-Extra: all
Requires-Dist: duckdb>=0.10.0; extra == "all"
Requires-Dist: psycopg2-binary>=2.9.0; extra == "all"
Requires-Dist: mysql-connector-python>=8.0.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Requires-Dist: build>=1.2.0; extra == "dev"
Requires-Dist: twine>=5.0.0; extra == "dev"
Requires-Dist: duckdb>=0.10.0; extra == "dev"
Dynamic: license-file

# 🔥 DataForge (DFG)

> Motor de ELT em Python puro — dbt meets dlt. Simples, rápido, sem dependências pesadas.

[![PyPI version](https://badge.fury.io/py/dataforge-dfg.svg)](https://badge.fury.io/py/dataforge-dfg)
[![Python](https://img.shields.io/badge/Python-3.11%2B-blue)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Status](https://img.shields.io/badge/Status-Beta-orange)]()

O **DataForge** unifica ingestão e transformação de dados em uma única ferramenta de linha de comando. Escreva **modelos Python** para extrair dados de qualquer fonte e **modelos SQL com Jinja2** para transformá-los — tudo no mesmo pipeline, com DAG automático, testes, snapshots SCD2 e documentação visual incluídos.

```
pip install dataforge-dfg[duckdb]
```

---

## ⚡ Início em 60 segundos

```bash
# Instala com DuckDB
pip install dataforge-dfg[duckdb]

# Cria o projeto
mkdir meu_projeto && cd meu_projeto
dfg init

# Executa o pipeline
dfg run

# Visualiza o grafo de dependências
dfg docs --serve
```

---

## ✨ O que o DataForge faz

| Funcionalidade | Comando |
|---|---|
| Ingestão via Python (APIs, arquivos, DBs) | `dfg ingest` |
| Transformação SQL com Jinja2 + DAG automático | `dfg transform` |
| Pipeline completo (ELT) | `dfg run` |
| Testes de qualidade de dados | `dfg test` |
| Snapshots SCD Tipo 2 | `dfg snapshot` |
| Carga de CSVs estáticos | `dfg seed` |
| Documentação HTML com grafo interativo | `dfg docs` |
| Compilação Jinja2 (dry-run) | `dfg compile` |
| Diagnóstico do ambiente | `dfg debug` |
| Busca nos logs | `dfg log` |

---

## 🗄️ Bancos Suportados

```bash
pip install dataforge-dfg[duckdb]    # DuckDB
pip install dataforge-dfg[postgres]  # PostgreSQL
pip install dataforge-dfg[mysql]     # MySQL
pip install dataforge-dfg[all]       # Todos
# SQLite já vem embutido no Python
```

---

## 📝 Exemplo Rápido

**Modelo Python** — Ingere dados de uma API:
```python
# models/ingest_produtos.py
DEPENDENCIES = []

def model(context):
    import urllib.request, json
    with urllib.request.urlopen("https://api.exemplo.com/produtos") as r:
        return json.loads(r.read())["data"]
```

**Modelo SQL** — Transforma os dados ingeridos:
```sql
-- models/stg_produtos.sql
{{ config(materialized='table') }}

SELECT
    id          AS produto_id,
    UPPER(nome) AS nome,
    preco::FLOAT AS preco
FROM {{ ref('ingest_produtos') }}
WHERE ativo = true
```

**Contrato de dados** — Garante qualidade:
```yaml
# models/schema.yml
version: 1
models:
  - name: stg_produtos
    columns:
      - name: produto_id
        tests: [not_null, unique]
```

```bash
dfg run     # executa tudo
dfg test    # valida os contratos
```

---

## 📚 Documentação Completa

Consulte o arquivo [DOCUMENTACAO.md](DOCUMENTACAO.md) para o guia completo com todos os comandos, casos de uso e exemplos.

---

## 🤝 Contribuindo

Contribuições são bem-vindas! Veja [CONTRIBUTING.md](CONTRIBUTING.md) para as diretrizes.

```bash
git clone https://github.com/seu-usuario/dataforge
cd dataforge
python -m venv .venv && source .venv/bin/activate  # ou .venv\Scripts\Activate.ps1
pip install -e ".[dev]"
pytest
```

---

## 📄 Licença

MIT © Matheus — veja [LICENSE](LICENSE).
