Metadata-Version: 2.5
Name: nlqueries-core
Version: 0.3.0
Summary: Open source natural language to SQL (NL2SQL) engine with schema validation and an MCP server for Claude and Cursor.
Project-URL: Homepage, https://nlqueries.com
Project-URL: Documentation, https://nlqueries.com/docs/
Project-URL: Repository, https://github.com/nlqueries/nlqueries
Project-URL: Changelog, https://github.com/nlqueries/nlqueries/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/nlqueries/nlqueries/issues
License: Business Source License 1.1
        
        License text copyright © 2024 MariaDB plc, All Rights Reserved.
        "Business Source License" is a trademark of MariaDB plc.
        
        -----------------------------------------------------------------------------
        
        Licensor:             Theorence Labs Private Limited
        Licensed Work:        NLQueries Core
                              The Licensed Work is © 2026 Theorence Labs Private Limited.
        Change Date:          June 4, 2030
        Change License:       Apache License, Version 2.0
        
        Additional Use Grant: You may make production use of the Licensed Work,
                              provided that such use does not include offering the
                              Licensed Work to third parties on a hosted or managed
                              basis, whereby a third party remotely accesses the
                              functionality of the Licensed Work as a service.
                              Individual and non-commercial self-hosted deployments
                              are permitted without restriction.
        
        -----------------------------------------------------------------------------
        
        Terms
        
        The Licensor hereby grants you the right to copy, modify, create derivative
        works, redistribute, and make non-production use of the Licensed Work. The
        Licensor may make an Additional Use Grant, above, permitting limited
        production use.
        
        Effective on the Change Date, or the fourth anniversary of the first
        publicly available distribution of a specific version of the Licensed Work
        under this License, whichever comes first, the Licensor hereby grants you
        rights under the terms of the Change License, and the rights granted in the
        paragraph above terminate.
        
        If your use of the Licensed Work does not comply with the requirements
        currently in effect as described in this License, you must purchase a
        commercial license from the Licensor, its affiliated entities, or authorized
        resellers, or you must refrain from using the Licensed Work.
        
        All copies of the original and modified Licensed Work, and derivative works
        of the Licensed Work, are subject to this License. This License applies
        separately for each version of the Licensed Work and the Change Date may
        vary for each version of the Licensed Work released by Licensor.
        
        You must conspicuously display this License on each original or modified
        copy of the Licensed Work. If you receive the Licensed Work in original or
        modified form from a third party, the terms and conditions set forth in this
        License apply to your use of that work.
        
        Any use of the Licensed Work in violation of this License will automatically
        terminate your rights under this License for the current and all other
        versions of the Licensed Work.
        
        This License does not grant you any right in any trademark or logo of
        Licensor or its affiliates (provided that you may use a trademark or logo of
        Licensor as expressly required by this License).
        
        TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED
        ON AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND
        CONDITIONS, EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES
        OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT,
        AND TITLE.
        
        MariaDB hereby grants you permission to use this License's text to license
        your works, and to refer to it using the trademark "Business Source
        License", as long as you comply with the Covenants of Licensor below.
        
        -----------------------------------------------------------------------------
        
        Covenants of Licensor
        
        In consideration of the right to use this License's text and the "Business
        Source License" name and trademark, Licensor covenants to MariaDB, and to
        all other recipients of the licensed work to be provided by Licensor:
        
        1. To specify as the Change License the GPL Version 2.0 or any later
           version, or a license that is compatible with GPL Version 2.0 or a later
           version, where "compatible" means that software provided under the Change
           License can be included in a program with software provided under GPL
           Version 2.0 or a later version. Licensor may specify additional Change
           Licenses without limitation.
        
        2. To either:
           (a) specify an additional grant of rights to use that does not impose
               any additional restriction on the right granted in this License, as
               the Additional Use Grant; or
           (b) insert the text "None" to specify a Change Date.
        
        3. Not to modify this License in any other way.
        
        -----------------------------------------------------------------------------
        
        Notice
        
        The Business Source License (this document, or the "License") is not an
        Open Source license. However, the Licensed Work will eventually be made
        available under an Open Source License, as stated in this License.
        
        For more information on the use of the Business Source License generally,
        please visit the Adopting and Developing Business Source License FAQ:
        https://mariadb.com/bsl-faq-adopting/
License-File: LICENSE
Keywords: mcp,natural-language,nlp,query,sql
Classifier: Development Status :: 3 - Alpha
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: <3.15,>=3.11
Requires-Dist: anthropic>=0.28
Requires-Dist: click>=8.1
Requires-Dist: cryptography>=42.0
Requires-Dist: httpx>=0.27
Requires-Dist: keyring>=24.0
Requires-Dist: litellm>=1.0
Requires-Dist: mcp<2,>=1.27
Requires-Dist: opentelemetry-exporter-otlp>=1.24
Requires-Dist: opentelemetry-sdk>=1.24
Requires-Dist: psycopg2-binary>=2.9
Requires-Dist: pydantic>=2.0
Requires-Dist: pyjwt>=2.8
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: qdrant-client>=1.10
Requires-Dist: rich>=13.0
Requires-Dist: sentence-transformers>=3.0
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: sqlglot>=23.0
Provides-Extra: bigquery
Requires-Dist: google-cloud-bigquery>=3.0; extra == 'bigquery'
Provides-Extra: dev
Requires-Dist: duckdb>=1.0; extra == 'dev'
Requires-Dist: google-cloud-bigquery>=3.0; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: snowflake-connector-python>=3.0; extra == 'dev'
Requires-Dist: testcontainers[postgres]>=4.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: openpyxl>=3.1; extra == 'docs'
Requires-Dist: pdfplumber>=0.11; extra == 'docs'
Requires-Dist: python-docx>=1.1; extra == 'docs'
Provides-Extra: duckdb
Requires-Dist: duckdb>=0.10; extra == 'duckdb'
Provides-Extra: mssql
Requires-Dist: pymssql>=2.3; extra == 'mssql'
Provides-Extra: mysql
Requires-Dist: pymysql>=1.1; extra == 'mysql'
Provides-Extra: onnx
Requires-Dist: optimum[onnxruntime]>=1.17; extra == 'onnx'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Provides-Extra: redshift
Requires-Dist: redshift-connector>=2.1; extra == 'redshift'
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python>=3.0; extra == 'snowflake'
Provides-Extra: wiki
Requires-Dist: atlassian-python-api>=3.41; extra == 'wiki'
Requires-Dist: beautifulsoup4>=4.12; extra == 'wiki'
Requires-Dist: notion-client>=2.2; extra == 'wiki'
Description-Content-Type: text/markdown

<p align="center">
  <img src="docs/assets/readme_banner.png" alt="NLQueries" width="1200">
</p>

# nlqueries-core

[![CI](https://github.com/nlqueries/nlqueries/actions/workflows/ci.yml/badge.svg)](https://github.com/nlqueries/nlqueries/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/nlqueries-core)](https://pypi.org/project/nlqueries-core/)
[![Python](https://img.shields.io/pypi/pyversions/nlqueries-core)](https://pypi.org/project/nlqueries-core/)
[![License: BSL 1.1](https://img.shields.io/badge/license-BSL%201.1-blue)](LICENSE)
[![Visitors](https://visitor-badge.laobi.icu/badge?page_id=nlqueries.nlqueries)](https://github.com/nlqueries/nlqueries)

**NLQueries Core** is an open-source natural language to SQL engine — see [nlqueries.com](https://nlqueries.com). It turns plain-English questions into validated SQL, builds a self-updating YAML knowledge base from your schema and query history, and exposes everything as an MCP server your AI assistant can call directly. It also answers questions from your documents (PDF, Word, Excel, Notion, Confluence) and can blend both in a single hybrid answer.

## Links

- **Website** — [nlqueries.com](https://nlqueries.com)
- **Documentation** — [nlqueries.com/docs/](https://nlqueries.com/docs/)
- **Getting started** — [nlqueries.com/docs/getting-started.html](https://nlqueries.com/docs/getting-started.html)
- **Supported databases** — [nlqueries.com/databases/](https://nlqueries.com/databases/)
- **PyPI** — [nlqueries-core](https://pypi.org/project/nlqueries-core/)
- **Docker Hub** — [nlqueries/core](https://hub.docker.com/r/nlqueries/core)

---

## Features

| Capability | Description |
|---|---|
| **Database connectors** | PostgreSQL, MySQL, Snowflake, BigQuery, Redshift, SQL Server / Azure SQL, DuckDB, SQLite — plus a generic SQLAlchemy connector for any other SQLAlchemy-reachable database, driven by a connection URL |
| **Document connectors** | PDF, Word, Excel, Notion, Confluence — ask questions over ingested documents with citations |
| **Query pipeline** | Filter, cluster, and parameterize query history into reusable `QueryCapsule` templates |
| **Knowledge base** | Auto-generated YAML schema + capsule file, with coverage reporting via `kb-stats` |
| **Multi-agent orchestration** | Routes each question to a SQL agent, document agent, or both in parallel (hybrid) |
| **Conversational follow-ups** | Carries context across questions so a follow-up like "and by region?" resolves against the previous query — on by default in `nlqueries query`, reset with `--new-session` |
| **Semantic cache** | Returns previously-answered similar questions in under 50 ms, no LLM or DB round-trip |
| **Embedding daemon** | Keeps the embedding model resident in memory — ~10 ms per call instead of ~9 s |
| **Feedback loop** | Rate an answer with `feedback`, review with `feedback-stats`, and `promote-feedback` the thumbs-up ones — their SQL is re-validated against the current schema and blended into later prompts as verified examples |
| **Regression checks** | `eval` re-asks the agent's own mined capsules — and a golden question set too, with `--golden` — then checks the generated SQL parses and references only tables the agent knows |
| **dbt import** | `import-dbt` merges dbt model and column descriptions into the knowledge base |
| **LLM client** | Anthropic, OpenAI, Amazon Bedrock, or any LiteLLM-supported provider |
| **MCP server** | Query execution and schema/knowledge lookup exposed as MCP tools for Claude, Cursor, etc. |
| **CLI** | `nlqueries` (or the shorter `nlq` alias) — connect, build, query, and inspect from your terminal |

See [docs/architecture.md](docs/architecture.md) (or [read online](https://nlqueries.com/docs/architecture.html)) for how these pieces fit together.

---

## Quickstart

**Prerequisite:** Python 3.11–3.14.

### Option A — Docker (recommended)

Pulls the published [`nlqueries/core`](https://hub.docker.com/r/nlqueries/core) image from Docker Hub — no clone required, just the compose file:

```bash
curl -O https://raw.githubusercontent.com/nlqueries/nlqueries/main/docker-compose.yml
```

Create a `.env` file next to it with one LLM key, a Qdrant key and an MCP
token. All three are required — compose refuses to start without
`QDRANT_API_KEY` or `NLQ_MCP_STATIC_TOKEN`:

```bash
ANTHROPIC_API_KEY=sk-ant-...
# or OPENAI_API_KEY=sk-...
QDRANT_API_KEY=...         # any random secret: openssl rand -hex 32
NLQ_MCP_STATIC_TOKEN=...   # another one: openssl rand -hex 32
```

The MCP server is reachable over the network, so it requires that token:
an MCP client connects to `http://localhost:8080/sse` and sends
`Authorization: Bearer <your NLQ_MCP_STATIC_TOKEN>`. The token has full access.
To give different people different access, see
[docs/mcp-authentication.md](docs/mcp-authentication.md).

Then start the stack:

```bash
docker compose up
```

This pulls `nlqueries/core:latest` and starts it alongside Qdrant (`:6333`), with the MCP server on `:8080`. Run CLI commands against the running stack from a second terminal:

```bash
docker exec -it nlqueries-core nlqueries health
```

### Option B — pip install

```bash
pip install nlqueries-core
export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_API_KEY
nlqueries health
```

Optional extras for specific connectors:

```bash
pip install "nlqueries-core[mysql]"      # MySQL
pip install "nlqueries-core[snowflake]"  # Snowflake
pip install "nlqueries-core[bigquery]"   # Google BigQuery
pip install "nlqueries-core[redshift]"   # Amazon Redshift
pip install "nlqueries-core[mssql]"      # SQL Server / Azure SQL
pip install "nlqueries-core[duckdb]"     # DuckDB
pip install "nlqueries-core[docs]"       # PDF / Word / Excel ingestion
pip install "nlqueries-core[wiki]"       # Notion / Confluence sync
```

PostgreSQL and SQLite need no extra. The generic `sqlalchemy` connector needs
whichever driver its URL names.

### Option C — Clone and install from source

No Docker required — for contributing, or to run against unreleased changes:

```bash
git clone https://github.com/nlqueries/nlqueries.git
cd nlqueries
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\Activate.ps1
pip install -e ".[dev]"
export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_API_KEY
nlqueries health
```

See [CONTRIBUTING.md](CONTRIBUTING.md#development-setup) for linting and test commands.

### First query

```bash
nlqueries connect postgres --host localhost --database mydb --user alice --password secret --alias dev
nlqueries process-history dev --days 30 --annotate
nlqueries export-kb dev
nlqueries query dev "How many orders shipped last month?"
nlqueries query dev "and how many were returned?"   # follow-up — keeps prior context
```

Follow-up context is on by default; pass `--new-session` to start fresh or
`--no-session` to disable it for a one-off question.

Full walkthrough: [docs/getting-started.md](docs/getting-started.md) (or [read online](https://nlqueries.com/docs/getting-started.html)).

---

## Documentation

| Doc | Covers |
|---|---|
| [docs/getting-started.md](docs/getting-started.md) (or [read online](https://nlqueries.com/docs/getting-started.html)) | Step-by-step setup and your first query |
| [docs/cli-reference.md](docs/cli-reference.md) (or [read online](https://nlqueries.com/docs/cli-reference.html)) | Every command and flag |
| [docs/connectors.md](docs/connectors.md) (or [read online](https://nlqueries.com/docs/connectors.html)) | Database and document connector setup, per-connector notes and caveats |
| [docs/configuration.md](docs/configuration.md) (or [read online](https://nlqueries.com/docs/configuration.html)) | Environment variables |
| [docs/troubleshooting.md](docs/troubleshooting.md) (or [read online](https://nlqueries.com/docs/troubleshooting.html)) | Common warnings and errors explained |
| [docs/qdrant-setup.md](docs/qdrant-setup.md) (or [read online](https://nlqueries.com/docs/qdrant-setup.html)) | Setting up Qdrant (required for embeddings, semantic cache, document search) |
| [docs/mcp-authentication.md](docs/mcp-authentication.md) (or [read online](https://nlqueries.com/docs/mcp-authentication.html)) | Authenticating the MCP server — required to serve it over a network |
| [docs/architecture.md](docs/architecture.md) (or [read online](https://nlqueries.com/docs/architecture.html)) | Module layout and request flow |
| [docs/database-hardening.md](docs/database-hardening.md) (or [read online](https://nlqueries.com/docs/database-hardening.html)) | The read-only role to grant per engine, and exactly what each connector does and does not enforce |
| [docs/dependency-locking.md](docs/dependency-locking.md) (or [read online](https://nlqueries.com/docs/dependency-locking.html)) | The hashed lock file the image builds from, the digest-pinned base image, and how to regenerate both |
| [docs/contributing.md](docs/contributing.md) (or [read online](https://nlqueries.com/docs/contributing.html)) | What a new connector has to implement, and what to open an issue about first |

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). All contributors must sign the CLA before a PR can be merged — see [CONTRIBUTOR_LICENSE_AGREEMENT.md](CONTRIBUTOR_LICENSE_AGREEMENT.md).

---

## License

[Business Source License 1.1](LICENSE) — converting to Apache 2.0 no later than **June 4, 2030**. The licence sets that date and also converts each version on the fourth anniversary of its own first publication, whichever comes first, so a version first published before June 4, 2026 converts on its own anniversary, ahead of the fixed date.

Until then the licence grants production use, including commercially and self-hosted, with one carve-out: you may not offer NLQueries Core to third parties as a hosted or managed service. Individual and non-commercial self-hosted deployments are unrestricted. [LICENSE](LICENSE) is the authority; this paragraph is a summary.
