Metadata-Version: 2.4
Name: shkit
Version: 1.2.22
Summary: AUTOMODEL — universal data discovery, relationship intelligence, and warehouse architecture (published as the shkit distribution).
Author: AUTOMODEL maintainers
License: Apache-2.0
Keywords: data-warehouse,etl,metadata,profiling,schema-discovery,star-schema
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Provides-Extra: all
Requires-Dist: fastapi>=0.110; extra == 'all'
Requires-Dist: google-cloud-bigquery>=3.0; extra == 'all'
Requires-Dist: openpyxl>=3.1; extra == 'all'
Requires-Dist: oracledb>=2.0; extra == 'all'
Requires-Dist: pandas>=2.0; extra == 'all'
Requires-Dist: psycopg[binary]>=3.1; extra == 'all'
Requires-Dist: pyarrow>=14.0; extra == 'all'
Requires-Dist: pymysql>=1.1; extra == 'all'
Requires-Dist: pyodbc>=5.0; extra == 'all'
Requires-Dist: pyyaml>=6.0; extra == 'all'
Requires-Dist: requests>=2.31; extra == 'all'
Requires-Dist: snowflake-connector-python>=3.0; extra == 'all'
Requires-Dist: uvicorn>=0.29; extra == 'all'
Provides-Extra: api
Requires-Dist: requests>=2.31; extra == 'api'
Provides-Extra: bigquery
Requires-Dist: google-cloud-bigquery>=3.0; extra == 'bigquery'
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.1; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: pyyaml>=6.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: files
Requires-Dist: openpyxl>=3.1; extra == 'files'
Requires-Dist: pandas>=2.0; extra == 'files'
Requires-Dist: pyarrow>=14.0; extra == 'files'
Provides-Extra: mssql
Requires-Dist: pyodbc>=5.0; extra == 'mssql'
Provides-Extra: mysql
Requires-Dist: pymysql>=1.1; extra == 'mysql'
Provides-Extra: oracle
Requires-Dist: oracledb>=2.0; extra == 'oracle'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.1; extra == 'postgres'
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python>=3.0; extra == 'snowflake'
Provides-Extra: ui
Requires-Dist: fastapi>=0.110; extra == 'ui'
Requires-Dist: uvicorn>=0.29; extra == 'ui'
Provides-Extra: yaml
Requires-Dist: pyyaml>=6.0; extra == 'yaml'
Description-Content-Type: text/markdown

# AUTOMODEL

**A universal data discovery, relationship-intelligence, and data-architecture
framework.**

AUTOMODEL inspects a data source, profiles every column, infers primary/foreign
keys and relationships, discovers business **domains** and **metrics**, builds a
**knowledge graph**, assesses **data quality** and **warehouse readiness**, makes
**architecture recommendations**, traces **lineage**, and generates a model in the
**architecture of your choice** — star, snowflake, normalized ER, Data Vault, or a
full dbt project.

The core is **deterministic, rule-based, and statistically driven** — identical
input always yields identical output. Every inferred artifact carries a
**confidence score and structured evidence**, so the platform is explainable and
auditable. (AI is an optional explanation layer, never part of the inference path.)

> Status: **0.2**. Implemented + tested: SQLite/CSV connectors, profiling, PK/FK
> detection, fact/dim classification; pluggable modeling strategies
> (dimensional/snowflake/normalized/data_vault/dbt); domain, metric, knowledge-graph,
> data-quality, readiness, recommendation, and lineage engines; the interactive
> **Architect Explorer** HTML report + named JSON/SVG catalogs; CLI. See
> [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full design and roadmap.

## Choose your architecture

```bash
automodel analyze -s examples/shop.db --strategy data_vault    # or dimensional|snowflake|normalized|dbt
automodel analyze -s examples/shop.db --config examples/automodel.yaml
```

```python
import automodel
bp = automodel.analyze("sqlite:///examples/shop.db", modeling_strategy="snowflake")
print(bp.domains, bp.metrics, bp.readiness.overall)
bp.knowledge_graph.related("customer")   # "show all assets related to Customer"
```

## Install

```bash
pip install -e .            # core (zero third-party deps)
pip install -e ".[dev]"     # + pytest/ruff
pip install -e ".[all]"     # + every connector/UI extra
```

The **core runs on the Python standard library alone**. Heavy backends
(Postgres, Snowflake, BigQuery, Parquet, Excel, …) are opt-in extras.

## Quick start

```bash
# Build the bundled sample retail database
python examples/build_sample_db.py

# Full analysis → JSON + Markdown + HTML + DDL in ./automodel_out
automodel analyze --source examples/shop.db --out automodel_out
```

Or from Python:

```python
import automodel
bp = automodel.analyze("sqlite:///examples/shop.db")
print(bp.model.facts, bp.model.dimensions)
```

## CLI

| Command | Output |
|---|---|
| `automodel analyze` | full pipeline → all reports + catalogs + SVGs + DDL |
| `automodel profile` / `discover` | metadata + profiling (+ PK/FK) JSON |
| `automodel generate` | warehouse model (per `--strategy`) + DDL |
| `automodel domains` / `metrics` | domain / metric catalog (JSON) |
| `automodel quality` / `readiness` / `recommend` | intelligence catalogs (JSON) |
| `automodel graph --query Revenue` | knowledge graph (JSON/SVG) + related-assets query |
| `automodel dbt` | a complete dbt project tree |
| `automodel report --format html\|markdown\|json` | one report |
| `automodel visualize --kind star\|lineage\|graph` | an SVG |

Common flags: `--source/-s`, `--out/-o`, `--strategy`, `--config <yaml>`,
`--sample N` (0 = full scan), `--seed`, `--log-level`.

## Sources

Built-in (core): **SQLite**, **CSV**. Planned extras: PostgreSQL, MySQL, SQL Server,
Oracle, Snowflake, BigQuery, Excel, Parquet, JSON, REST APIs — added via the plugin
system (`automodel.connectors` entry-point group).

## How it works

```
connect → metadata → profile → primary keys → foreign keys → graph →
classify (fact/dim) → entities → warehouse model → blueprint
```

Each stage is an isolated, testable engine. Scoring formulas (PK/FK/fact/dimension)
live in [docs/ARCHITECTURE.md §7](docs/ARCHITECTURE.md#7-algorithms) and every tunable
threshold is in `automodel.core.config.Config` — no magic numbers in engine code.

## Development

```bash
pytest            # run the suite
ruff check .      # lint
```

## License

Apache-2.0
