Metadata-Version: 2.4
Name: nl2sql-engine
Version: 0.2.4
Summary: Ask a database questions in English: the model writes a typed query plan, never SQL, and the plan is checked against the schema and the caller's role before any SQL is generated
License-Expression: MIT
Project-URL: Homepage, https://github.com/nadeem4/nl2sql
Project-URL: Documentation, https://nadeem4.github.io/nl2sql/
Project-URL: Source, https://github.com/nadeem4/nl2sql
Project-URL: Issues, https://github.com/nadeem4/nl2sql/issues
Project-URL: Demo, https://nl2sql.codewithnk.com
Keywords: nl2sql,text-to-sql,natural language,sql,llm,langgraph,sqlglot,database
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Requires-Dist: SQLAlchemy>=2.0
Requires-Dist: PyYAML>=6.0
Requires-Dist: langgraph<2,>=1.2
Requires-Dist: langchain>=0.2.0
Requires-Dist: langchain-core<2,>=1.6
Requires-Dist: langchain-chroma<2,>=1.1
Requires-Dist: langchain-openai<2,>=1.6
Requires-Dist: chromadb<2,>=1.5
Requires-Dist: nl2sql-adapter-sdk~=0.1
Requires-Dist: pydantic>=2.0
Requires-Dist: opentelemetry-api>=1.20.0
Requires-Dist: opentelemetry-sdk>=1.20.0
Requires-Dist: opentelemetry-exporter-otlp>=1.20.0
Requires-Dist: sqlglot>=23.0.0
Requires-Dist: pydantic-settings>=2.0.0
Requires-Dist: pyarrow>=14.0.0
Requires-Dist: pybreaker>=1.3.0
Requires-Dist: polars>=1.44
Requires-Dist: duckdb>=1.5
Requires-Dist: rich>=13.0.0
Requires-Dist: typer>=0.9.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: inquirerpy>=0.3.4
Requires-Dist: typing_extensions>=4.0
Requires-Dist: httpx<1,>=0.23
Provides-Extra: anthropic
Requires-Dist: langchain-anthropic<2,>=1.7; extra == "anthropic"
Provides-Extra: aws
Requires-Dist: boto3; extra == "aws"
Provides-Extra: azure
Requires-Dist: azure-identity; extra == "azure"
Requires-Dist: azure-keyvault-secrets; extra == "azure"
Provides-Extra: demo
Requires-Dist: fastapi>=0.100.0; extra == "demo"
Requires-Dist: uvicorn>=0.20.0; extra == "demo"
Provides-Extra: duckdb
Requires-Dist: duckdb; extra == "duckdb"
Requires-Dist: duckdb-engine; extra == "duckdb"
Provides-Extra: hashicorp
Requires-Dist: hvac; extra == "hashicorp"
Provides-Extra: mssql
Requires-Dist: pyodbc>=4.0.0; extra == "mssql"
Requires-Dist: pymssql>=2.2.0; platform_system == "Linux" and extra == "mssql"
Provides-Extra: mysql
Requires-Dist: pymysql>=1.1.0; extra == "mysql"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9; extra == "postgres"
Provides-Extra: all
Requires-Dist: duckdb; extra == "all"
Requires-Dist: duckdb-engine; extra == "all"
Requires-Dist: pyodbc>=4.0.0; extra == "all"
Requires-Dist: pymssql>=2.2.0; platform_system == "Linux" and extra == "all"
Requires-Dist: pymysql>=1.1.0; extra == "all"
Requires-Dist: psycopg2-binary>=2.9; extra == "all"
Dynamic: license-file

# nl2sql-engine

Ask a database questions in English. The model writes a typed query plan, never
SQL text; the plan is checked against the real schema and the caller's role
before any SQL is generated.

This distribution is the engine: the LangGraph pipeline, the `nl2sql` CLI with
its browser playground, and the database adapters (PostgreSQL, MySQL, SQL
Server, SQLite, DuckDB). The REST server is the separate `nl2sql-api` package.
Full README and docs: <https://github.com/nadeem4/nl2sql>.

## Try it in your browser

**<https://nl2sql.codewithnk.com>** — the playground on three sample
databases, nothing to install.

[![Ask a database in plain English; the model plans, the code writes the SQL](https://raw.githubusercontent.com/nadeem4/nl2sql/main/docs/assets/social-card.png)](https://nl2sql.codewithnk.com)

The server holds no API key: you paste your own into the page, one per
provider, and it stays in that browser tab, is used in memory for the question
that needs it, and is never stored, logged or traced. Settings also puts each
of the five model-using steps on a provider and model of its own.

![The nl2sql playground on its Ask page: the search index over three databases and a switcher for whichever of their schemas to read on the left, the question box and the guided questions on the right](https://raw.githubusercontent.com/nadeem4/nl2sql/main/docs/assets/screenshots/playground-overview.png)

## How it works

- The question is checked for answerability and routed to one of the registered
  databases, then split into sub-queries, and each gets the part of the schema
  it needs.
- The model returns a typed plan (a Pydantic `PlanModel`), never SQL. It may
  only name a function from an allow-list, and date work is written portably
  and rendered by each adapter in its own dialect.
- The logical validator checks every table and column against the schema,
  joins against the declared foreign keys, and every table against the caller's
  role (RBAC); it also builds the query tree the generator would build, so an
  unjoinable plan is caught while the planner can still be asked to fix it. A
  refused plan never becomes SQL.
- Only a plan that passes is rendered to SQL, deterministically, with `sqlglot`,
  then executed and summarised.

The whole pipeline is 13 steps, five of them decided by a model; the
playground's Pipeline page lists them.

## Quickstart

Requires Python 3.12 or newer.

```bash
pip install "nl2sql-engine[demo]"
nl2sql demo
```

That writes a demo project into `./nl2sql-demo`, copies in three SQLite
databases -- [Chinook](https://github.com/nadeem4/nl2sql/blob/main/THIRD_PARTY_NOTICES.md),
`support` and `webanalytics` -- indexes their schemas locally (no key; the
first run downloads a ~79 MB ONNX embedding model) and opens the playground on
<http://127.0.0.1:8765/>.

**Answering a question needs a model:** `--api-key`, or `OPENAI_API_KEY`,
`OPENROUTER_API_KEY` or `ANTHROPIC_API_KEY` in the environment (Claude needs
`pip install "nl2sql-engine[demo,anthropic]"`), a key pasted into the
playground's Settings page, or a reachable Ollama. Without one the demo runs
in replay mode, which has no recorded answers out of the box, so it can show the
schema and the index but answers nothing. `nl2sql demo --record` (with an
OpenAI, OpenRouter or Anthropic key) records the guided questions for later key-free runs.

`nl2sql demo --hosted` runs the public variant above yourself: the server holds
no API key, each visitor pastes their own into the page for that question only
(without one, the guided questions replay recorded runs),
the sample databases are opened read-only, questions are rate limited per
visitor and capped per session, and Settings, Rebuild, ratings and `--record`
are refused. See
[Hosted demo](https://github.com/nadeem4/nl2sql/blob/main/docs/deployment/hosted-demo.md)
in the full docs.

The playground shows each answer's plan, validation checks, SQL, rows and cost,
a per-node Debug view, a Pipeline page listing every step of a run with the
model each used, a Retrieval inspector over the live index, and a right/wrong
rating per answer that `nl2sql feedback stats` reports on.

## CLI

From the demo folder (`cd nl2sql-demo`):

```bash
nl2sql --env demo run "How many customers do we have, by country?"
nl2sql --env demo run --role viewer "Who are the top 5 customers by total spend?"   # refused
nl2sql --env demo run --no-exec "Which artist has the most albums?"                 # plan only
nl2sql --env demo index                       # re-index the schema
nl2sql --env demo doctor                      # check drivers, connectivity, key and index
nl2sql --env demo benchmark --tier 1          # gold plans, no key
nl2sql --env demo benchmark --tier 2 --model gpt-5.4 --max-cost 5
nl2sql --env demo benchmark retrieval         # retrieval recall, no key
nl2sql --env demo feedback stats              # ratings and guardrail rates
```

Every command: [CLI reference](https://github.com/nadeem4/nl2sql#cli-reference).

## Python

```python
from nl2sql import NL2SQL, UserContext

engine = NL2SQL(env="demo")          # loads .env.demo from the working directory
result = engine.run_query(
    "How many customers are there?",
    user_context=UserContext(roles=["admin"]),
)
for sq in result.sub_queries:
    print(sq.sql, [c.name for c in sq.validation if c.passed])
print(result.final_answer["summary"] if result.final_answer else result.errors)
```

Pass a `user_context` with a role: with none, `run_query` currently raises a
validation error, and an unknown role is refused.

## Install extras

```bash
pip install "nl2sql-engine[postgres]"        # or [mysql], [mssql], [duckdb]
pip install "nl2sql-engine[all]"             # every database driver; not [demo] or [anthropic]
```

SQLite needs no extra.

## Security

- The LLM provider receives the question, the schema the planner needs, sample
  column values of tables the role may read, and the result rows (for the
  written answer).
- RBAC is a per-role table allowlist; a plan touching a forbidden table is
  refused. No column masking or row-level security.
- Read-only is not checked by the executor: only SELECTs are generated, and a
  SQLite datasource can be opened read-only with `options.read_only: true` (the
  three sample databases are), but no other dialect is. Give the engine a
  read-only database user.
- No authentication: the caller supplies the role.
- No cross-database questions and no conversation yet: each sub-query is
  planned against one datasource, and each question is answered on its own.

## Status

0.1.x. `nl2sql-adapter-sdk`, `nl2sql-engine` and `nl2sql-api` share one version
and are released together. Known limitations, benchmark results and the
roadmap: <https://github.com/nadeem4/nl2sql#project-status-limitations-and-roadmap>.
MIT licensed.
