Metadata-Version: 2.4
Name: pyds-ai
Version: 0.2.12
Summary: Data science and LLM work from a notebook, without writing much code. The LLM writes the query and the words; a deterministic engine computes the answer.
Project-URL: Homepage, https://github.com/jsaimanoj/pyds_ai
Project-URL: Documentation, https://github.com/jsaimanoj/pyds_ai#readme
Project-URL: Issues, https://github.com/jsaimanoj/pyds_ai/issues
Author: pyds-ai contributors
License: Apache-2.0
License-File: LICENSE
Keywords: agents,automl,data science,duckdb,llm,notebook,pandas,rag
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Jupyter
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: duckdb>=0.10
Requires-Dist: httpx>=0.27
Requires-Dist: jinja2>=3.1
Requires-Dist: numpy>=1.24
Requires-Dist: openpyxl>=3.1
Requires-Dist: pandas>=2.0
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tomli-w>=1.0
Requires-Dist: tomli>=2.0; python_version < '3.11'
Provides-Extra: all
Requires-Dist: anthropic<2.0,>=0.34; extra == 'all'
Requires-Dist: beautifulsoup4>=4.12; extra == 'all'
Requires-Dist: chromadb>=0.5; extra == 'all'
Requires-Dist: faiss-cpu>=1.8; extra == 'all'
Requires-Dist: gcsfs>=2023.1; extra == 'all'
Requires-Dist: google-genai>=0.3; extra == 'all'
Requires-Dist: joblib>=1.3; extra == 'all'
Requires-Dist: keyring>=24; extra == 'all'
Requires-Dist: markdown-it-py>=3.0; extra == 'all'
Requires-Dist: openai>=1.40; extra == 'all'
Requires-Dist: pdfplumber>=0.11; extra == 'all'
Requires-Dist: pgvector>=0.2; extra == 'all'
Requires-Dist: psycopg[binary]>=3.1; extra == 'all'
Requires-Dist: pypdf>=4.0; extra == 'all'
Requires-Dist: python-docx>=1.1; extra == 'all'
Requires-Dist: python-pptx>=0.6.23; extra == 'all'
Requires-Dist: s3fs>=2023.1; extra == 'all'
Requires-Dist: scikit-learn<2.0,>=1.3; extra == 'all'
Requires-Dist: shap>=0.44; extra == 'all'
Requires-Dist: sqlalchemy>=2.0; extra == 'all'
Requires-Dist: weasyprint>=61; extra == 'all'
Requires-Dist: xlrd>=2.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic<2.0,>=0.34; extra == 'anthropic'
Provides-Extra: db
Requires-Dist: sqlalchemy>=2.0; extra == 'db'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'docs'
Provides-Extra: gcs
Requires-Dist: gcsfs>=2023.1; extra == 'gcs'
Provides-Extra: gemini
Requires-Dist: google-genai>=0.3; extra == 'gemini'
Provides-Extra: keyring
Requires-Dist: keyring>=24; extra == 'keyring'
Provides-Extra: ml
Requires-Dist: joblib>=1.3; extra == 'ml'
Requires-Dist: scikit-learn<2.0,>=1.3; extra == 'ml'
Requires-Dist: shap>=0.44; extra == 'ml'
Provides-Extra: openai
Requires-Dist: openai>=1.40; extra == 'openai'
Provides-Extra: pdf-tables
Requires-Dist: pdfplumber>=0.11; extra == 'pdf-tables'
Provides-Extra: rag
Requires-Dist: beautifulsoup4>=4.12; extra == 'rag'
Requires-Dist: markdown-it-py>=3.0; extra == 'rag'
Requires-Dist: pypdf>=4.0; extra == 'rag'
Requires-Dist: python-docx>=1.1; extra == 'rag'
Provides-Extra: rag-chroma
Requires-Dist: chromadb>=0.5; extra == 'rag-chroma'
Provides-Extra: rag-faiss
Requires-Dist: faiss-cpu>=1.8; extra == 'rag-faiss'
Provides-Extra: rag-pgvector
Requires-Dist: pgvector>=0.2; extra == 'rag-pgvector'
Requires-Dist: psycopg[binary]>=3.1; extra == 'rag-pgvector'
Provides-Extra: report
Requires-Dist: python-docx>=1.1; extra == 'report'
Requires-Dist: python-pptx>=0.6.23; extra == 'report'
Provides-Extra: report-pdf
Requires-Dist: weasyprint>=61; extra == 'report-pdf'
Provides-Extra: s3
Requires-Dist: s3fs>=2023.1; extra == 's3'
Provides-Extra: test
Requires-Dist: ipykernel>=6.29; extra == 'test'
Requires-Dist: nbmake>=1.5; extra == 'test'
Requires-Dist: pytest-cov>=5.0; extra == 'test'
Requires-Dist: pytest>=8.0; extra == 'test'
Requires-Dist: reportlab>=4.0; extra == 'test'
Requires-Dist: xlwt>=1.3; extra == 'test'
Provides-Extra: xls
Requires-Dist: xlrd>=2.0; extra == 'xls'
Description-Content-Type: text/markdown

# pyds-ai

[![CI](https://github.com/jsaimanoj/pyds_ai/actions/workflows/ci.yml/badge.svg)](https://github.com/jsaimanoj/pyds_ai/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/jsaimanoj/pyds_ai/branch/main/graph/badge.svg)](https://codecov.io/gh/jsaimanoj/pyds_ai)
[![PyPI](https://img.shields.io/pypi/v/pyds-ai.svg)](https://pypi.org/project/pyds-ai/)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

Ask a business question in plain English — `d.ask("did churn spike in
EMEA last quarter?")` — and get back a real answer with a chart, the
exact code behind it, and a plain-English caveat if the data can't
really support the question. Built for someone who knows the business,
not someone who knows pandas: no SQL, no coding background — if you can
run a cell in a notebook (Jupyter, or Google Colab with nothing to
install locally), you already have what it takes.

```bash
pip install pyds-ai
```
```python
import pyds_ai as pyds
```
`pyds-ai` is the package name; `pyds_ai` is the import — they don't
match on purpose, because the short names `pyds` and `pyds-cli` are
already taken by unrelated packages on PyPI. Every example on this page
shows both lines together so this is never a guessing game.

**Your data doesn't leave the building by default.** The model gets
column names, types, and summary statistics — never your raw rows —
unless you explicitly turn `send_data_values` on. See
[Privacy default](#privacy-default) below.

Pasting a spreadsheet into a chatbot gets you a confident, plausible
number with no way to check its arithmetic — the same question asked
twice can quietly come back with two different answers, and nobody
notices until it's in front of leadership. **The one rule that decides
everything here: the LLM never produces a number. It writes the query
and it writes the words. A deterministic engine (DuckDB) computes the
answer** — the same way, every time, from the same data. Every result
can show you the exact code that produced it via `result.code()`, so a
number never has to be taken on faith.

<p align="center">
  <img src="https://raw.githubusercontent.com/jsaimanoj/pyds_ai/main/docs/images/result-card.png" alt="A real d.drivers() result card, rendered in a notebook" width="640">
  <br><em>A real result card — the finding, the chart, and (expanded) the data, code, and trace.</em>
</p>
<p align="center">
  <img src="https://raw.githubusercontent.com/jsaimanoj/pyds_ai/main/docs/images/failure-card.png" alt="A real Failure card" width="640">
  <br><em>A failed call renders as a card, not a traceback.</em>
</p>

## Contents

- [Try it with no data and no key](#try-it-with-no-data-and-no-key) — see a real result before installing anything of your own
- [Quickstart](#quickstart) — the five-minute tour: load data, ask, model, chat with docs
- [Real-world files, not clean CSVs](#real-world-files-not-clean-csvs) — messy Excel, CSV, TXT, PDF
- [Writing to a real database](#writing-to-a-real-database) — SQL Server, Oracle, Postgres, MySQL, SQLite
- [Metrics: define it once, everyone gets the same number](#metrics-define-it-once-everyone-gets-the-same-number)
- [Ask the questions leadership actually asks](#ask-the-questions-leadership-actually-asks) — the 8 no-LLM-math verbs
- [Every result is the same shape](#every-result-is-the-same-shape)
- [Learning from a real result](#learning-from-a-real-result) — for the person who wants the SQL and the stats
- [Presentable output](#presentable-output) — decks, branded exports, `.xlsx`/`.html`
- [Failures render as a card, not a traceback](#failures-render-as-a-card-not-a-traceback)
- [Privacy default](#privacy-default)
- [Installing extras](#installing-extras)
- [Command line](#command-line)
- [Plugins](#plugins)
- [One session, one process](#one-session-one-process)
- [Development](#development)

## Try it with no data and no key

```bash
pip install pyds-ai
```
```python
import pyds_ai as pyds

pyds.demo()  # lists the bundled datasets
pyds.demo("product", offline=True)  # a real card, 5 canned answers, no key needed
```

`pyds.demo("product")` / `pyds.demo("revenue")` load the same bundled
datasets as a normal `DataSet` — everything below works on them exactly
as it would on your own data, once you're ready for `pyds.setup()`.

## Quickstart

The tour below is the whole surface area at a glance: load a file, ask a
question, get a chart, train a model, ask a question of a folder of
documents, then combine both into one agent. Each line works on its own
— you don't need the ones before it.

```bash
pip install pyds-ai[anthropic]  # or [openai], [gemini] -- Ollama needs no extra
```
```python
import pyds_ai as pyds

pyds.setup()  # one-time key wizard, writes to ~/.pyds_ai/config.toml

d = pyds.load("sales.csv")
d.suggest()  # ten business-language questions this data can answer — no model call
d.ask("which region grew fastest last quarter?")
d.chart("revenue by month, split by channel")

m = d.model(target="churn", positive_class="yes")  # which value is "churned" — required, not guessed
m.explain()
m.what_if(tenure=12, plan="premium")

k = pyds.knowledge("policies/")
k.ask("what is our refund window for enterprise?")

a = pyds.agent(uses=[d, k])
a.ask("did churn spike in any region where we changed the refund policy?")

e = pyds.evaluate(a, cases="qa_cases.csv")
e.watch()
```

`d.suggest()` generates its ten questions from the shape of the data
alone (dates, categories, missing values) — no model call — and the top
three already show up on `d`'s own card the moment it loads, before you
ask anything.

`d.model()` checks every column against the target for a suspiciously
high correlation (`> 0.95`) — or a name that itself suggests it records
the outcome or its aftermath (`*_status`, `reason`, `cancel_*`) — before
training. Either shape usually means a column is derived from the
target itself (e.g. a status code that's really just churn in disguise,
or a `cancel_reason` that's only ever filled in for people who already
churned) rather than a real predictor of it. A flagged column is
excluded from training by default, not just warned about, so it can't
quietly produce a hollow ~1.0 accuracy/F1/AUC and dominate
`m.explain()`/`m.what_if()` underneath it; the caveat names which column
and why, and `m.why()["excluded_leaky"]` lists them. A column correlated
between `0.7` and `0.95` is kept but still named in a caveat, since
that's also a plausible shape for a subtler leak. Pass
`d.model(target=..., allow_leaky=["that_column"])` if you're sure a
flagged column is legitimate and want it back in.

For a two-value (binary) target, `d.model()` requires `positive_class=`
naming which value is the outcome you're predicting. `predict_proba`'s
column order is just alphabetical — fine for `{0, 1}`, silently backwards
for something like `{"churned", "retained"}` — so every probability,
`roc_auc`, and `m.what_if()` reading would otherwise be relative to
whichever value sorts last, with nothing on the card saying so.

## Real-world files, not clean CSVs

Data reaches this audience as an Excel export from someone else's system
— a merged title row, blank rows, the real header on row 4, six tabs
with the actual data buried on the fifth one, dates that mean something
different depending on where the file came from:

```python
pyds.sheets("book.xlsx")  # lists every sheet's size, no loading
d = pyds.load("book.xlsx")  # finds the real header row automatically,
# loads the sheet that looks like the data (skipping a detected cover page),
# and says exactly what it assumed -- pass sheet="..." to pick one yourself
d2 = pyds.load_clipboard()  # pasted a range from Excel? this reads it directly
d3 = pyds.load("export.txt")  # sniffs the delimiter (comma/tab/pipe/semicolon)
pyds.pdf_tables("report.pdf")  # lists every table pdfplumber finds, before you load one
d4 = pyds.load("report.pdf")  # loads the first table found; table=1 picks another
```
`d.caveats` (shown right on the card) says which row it used as the
header, how many sheets the workbook had and which one it loaded, and —
for a date column like `01/02/2026` that's genuinely ambiguous — that it
assumed month-first (US) and how to say otherwise: `pyds.load("book.xlsx", dayfirst=True)`
for day-first (European) dates. This isn't just a friendlier message: a
date column in a non-ISO format used to come back `NULL` for every row in
every downstream `d.compare()`/`d.trend()`/etc. query, silently — `load()`
now converts it to a real date type before anything else touches it.
A `.pdf`'s tables are detected heuristically (a PDF has no real table
structure the way a spreadsheet does) — every load carries an
unconditional caveat saying so, and `pyds.pdf_tables(path).get(index)`
previews one table's raw extraction before you trust `load()` with it.

## Writing to a real database

The other direction: once a table's loaded, `d.export_to_sql()` writes it
to SQL Server, Oracle, Postgres, MySQL, or SQLite, via SQLAlchemy —
needs `pip install pyds-ai[db]` plus the driver for your target database
(e.g. `pyodbc` for SQL Server, `oracledb` for Oracle). Nothing about this
is reachable from `d.ask()` — it's a completely separate, explicitly-
invoked path, unlike everything else in this README.

```python
d = pyds.load("export.csv")
r = d.export_to_sql(
    "mssql+pyodbc://user:pass@myserver/mydb?driver=ODBC+Driver+17+for+SQL+Server",
    "sales_data",
    if_exists="replace",  # pandas' own vocabulary: "fail" (the default), "replace", or "append"
)
print(r.headline)
# "Wrote 12,483 row(s) to 'sales_data' (replace) at mssql+pyodbc://user:***@myserver/mydb..."
r.teach()  # a walkthrough of the dtype mapping, batching, and why it was written this way
```
The connection string never appears anywhere with credentials — every
headline, caveat, and trace entry shows only the redacted form, and even
a failed connection's own driver exception gets the literal password
scrubbed out before it reaches you. `if_exists` defaults to `"fail"` so a
repeat call never silently overwrites anything.

## Metrics: define it once, everyone gets the same number

Without a definition, "activation rate" gets re-guessed by the model on
every call — two people asking the same question can get two different
numbers. `pyds.define()` fixes a metric's formula once and caches it next
to your data, in a plain YAML file a human can open and edit:

```python
pyds.define(
    "activation rate",
    "users who completed onboarding divided by signups",
    grain="user",
    owner="you@company.com",
)
pyds.metrics()  # lists what's defined for this dataset
pyds.undefine("activation rate")
```

This writes (and reads) `metrics.yaml` next to the dataset's source
file — the handoff artifact between the data person and the product
owner. It ships with a starter template of ~15 common metrics (activation,
retention, churn, conversion, MAU/WAU/DAU, ARPU, CAC, NRR, funnel-step
conversion, LTV, and more) with descriptions filled in but no formula
yet; `pyds.define(name, description)` resolves each one for your real
data. Resolving a definition is the *only* place a model touches a
metric's arithmetic — one call, cached forever after. Every `ask()` about
a defined metric compiles that cached formula, and the card names which
definition it used, in the interpretation line and in `result.why()`. If
a question mentions something close to (but not exactly) a defined name,
you get offered the defined metric instead of a fresh guess:

```python
d.ask("what's our activation this quarter?")
# -> a Clarify card: "Did you mean the defined metric 'activation_rate'?"
```

## Ask the questions leadership actually asks

```python
d.compare("activation_rate", period="quarter", vs="target")
d.trend("mau", period="week", window=12)
d.drivers("revenue", period="quarter")  # "why did it fall" -> a ranked, waterfall answer
d.segment("conversion_rate")  # "which groups are different"
d.funnel(steps=["signed_up", "completed_onboarding", "made_purchase"])  # "where do people drop off"
d.cohort(by="signup_month")  # "is the March cohort better than January"
d.before_after("2026-03-01", "revenue", window="4 weeks")  # "did the launch work"
d.experiment("variant", "converted")  # "did the test win"
```

- **`d.compare(metric, period="quarter", vs=None)`** — current period, prior period, absolute and percentage change, direction. `vs="target"` compares against a `target` column or a fixed target set on the metric.
- **`d.trend(metric, period="week", window=12)`** — a plain-English read on whether the latest move is normal or worth a look ("down 4 percent, which is within the normal week-to-week swing for this metric"), never "standard deviation" or "confidence interval."
- **`d.drivers(metric, period="quarter")`** — decomposes a change across every categorical column, ranks the dimensions and categories that actually explain it, and renders a waterfall chart. An additive metric (a sum or count) gets an exact category-by-category contribution; a defined ratio metric gets a mix-vs-rate split (did the *mix* of segments shift, or did a segment's own rate change).
- **`d.segment(metric, dimensions=None)`** — which groups depart most from the overall average, ranked, with the sample size next to each so nobody acts on a segment of four users.
- **`d.funnel(steps, event_col=None, id_col=None, split_by=None)`** — step-by-step conversion, counting distinct entities at each named step, with per-step and overall conversion and the largest gap between consecutive steps (by count, not percentage) called out in the headline. Each step is a presence count (who/what ever had that event) rather than a verified chronological path, and the card says so — "drop-off" implies a sequential loss this counting method can't actually confirm. The headline says "people" only when the id column actually looks like one (`user_id`, `customer_id`, ...) — a funnel over `order_id`/`session_id` reads as orders or sessions, not people. Renders as a horizontal funnel chart; `split_by` breaks the same funnel out by a dimension (channel, plan, ...).
- **`d.cohort(by="signup_month", metric="retention")`** — the standard cohort retention table (cohorts down the side, periods since signup across the top), rendered as a heatmap, with a plain read on whether recent cohorts retain better or worse than older ones. `by` names a date column plus a period grain (`"signup_week"`, `"signup_quarter"`, ...).
- **`d.before_after(date, metric, window="4 weeks")`** — the window before a date vs. the window after it, both averages and the change — with an explicit, unconditional caveat that this is a before-and-after comparison, not a controlled test.
- **`d.experiment(group_col, metric, control=None)`** — an A/B read for someone who's never run one: both group sizes, both metric values, the lift, and a plain-English statement of whether the difference is unlikely to be due to chance (backed by a two-proportion z-test or Welch's t-test, every piece of statistics vocabulary kept out of the headline — it's still in `result.why()`). That's a statement about chance, not a recommendation: it never says a difference is "large enough to act on," since whether to act needs a cost/benefit call this package can't make, and it never estimates "how many more users you'd need" from the observed effect (a post-hoc power calculation, which isn't statistically valid). `control="..."` names which group is the baseline — without it, the two groups are ordered alphabetically and the card says so, since the lift's sign otherwise carries no guarantee about which one you actually consider the starting point. Every result also carries standing caveats about unit of randomization, peeking, and multiple comparisons, and flags heavy imbalance as a possible sample-ratio mismatch. Refuses, with a clear reason, when the groups are too small or too imbalanced to read.

`metric` accepts a defined metric name or a plain column name either way.

`d.compare()`, `d.drivers()`, `d.segment()`, and `d.before_after()` also
take `filter={"column": ..., "op": ..., "value": ...}` (or a list of
those) to scope the whole call to a segment — `d.drivers("revenue",
filter={"column": "region", "op": "=", "value": "EMEA"})` — without
pre-filtering in pandas and reloading, or reaching for `d.ask()` (which
needs a model and a key). The card names exactly what was filtered to,
so a scoped result never looks identical to an unscoped one.

**Accuracy:** **186/203 (91.6%)**, run 2026-08-23 against `claude-sonnet-4-6`
— see [`benchmark/`](benchmark/) to run the 200+-case golden set yourself,
and its README for a full breakdown of what the remaining gap actually is
(entirely ambiguous wording in a handful of the golden set's own
questions at this point — not silently excluded, read before you trust
this number). Scored by execution match (does the
generated SQL return the same data as a known-correct query), not by
comparing SQL text.

## Every result is the same shape

Every call returns a `Result` — never a bare DataFrame, never a dict:

```python
result.explain()  # what was done and why, in sentences
result.interpretation()  # the one-line "read as: ..." restatement of the question
result.code()  # the exact SQL/code that produced it
result.data()  # the underlying DataFrame, full precision — never reformatted
result.next()  # up to three suggested follow-up questions
result.why()  # trace: prompt version, model, cost, row counts
result.summary()  # three to five plain-English sentences for an email or speaker notes
result.teach()  # a walkthrough of the SQL and the stats behind this result, for learning
result.save(path)  # .html, .csv, .docx, .pptx, .xlsx, .pdf
```

An `Answer`/`Chart` card also opens with a one-sentence restatement of
how the question was read — which metric (naming the defined metric used,
if any), which filter, which grouping, and which time window — built from
the plan structure, not a second model call. It's how you catch a wrong
reading before it lands in a deck. A question that asks for a
comparison ("...vs last quarter", "...compared to target") gets a real
`baseline_value` computed alongside the answer, not just the current
number with the comparison silently dropped. Every result derived from a dataset —
not just `ask()`, also `.model()`, `.profile()`, `.compare()`, `.trend()`,
and the rest — shows the data's as-of date (`result.as_of`) on its card
and in every export, so a number repeated a week later still says which
snapshot it came from. Numbers on the card, in chart labels, and in
every export are formatted for presentation (thousands separators, `$`
when a column looks like money, `%` when it looks like a rate) — none of
that touches `result.data()` or a CSV export, which stay full precision.

## Learning from a real result

Every card deliberately keeps SQL and statistics out of the headline —
but they're never hidden, and `result.teach()` is built for the person
who actually wants them:

```python
r = d.experiment("variant", "converted", control="A")
print(r.headline)  # "A (control) vs B (treatment): converted is 19.6% vs 23.2% ... unlikely to be due to random chance."

r.teach()
# Names the actual technique behind the headline (a two-proportion
# z-test here, because the outcome is yes/no) and why it fits, walks
# through the SQL clause by clause, connects the headline's numbers
# back to result.why()'s z-score and p-value, and ends with one
# concrete "try it yourself" question tied to a real number already on
# the card (e.g. "what happens to required_n_per_group with half the
# sample size?").
```

Grounded in the exact same `result.code()`/`result.why()` any result
already carries — nothing new is computed, and it never describes a
technique that isn't actually reflected in the SQL or the trace. A
paid call like `.summary()`, cached separately after the first one;
`Failure.teach()` never calls the model at all.

## Presentable output

Anything this audience has to edit before showing it is a failure, so the
output is built to go straight into an email, a deck, or a spreadsheet:

```python
result.summary()  # "Revenue grew 12% this quarter, driven mostly by Enterprise/AMER. ..."

pyds.deck(
    [d.compare("revenue"), d.drivers("revenue"), d.segment("revenue")],
    title="Q2 Business Review",
    path="q2_review.pptx",
)

pyds.brand(logo="logo.png", colors=["#8e44ad"], template="corp.potx")

result.save(
    "report.xlsx"
)  # Answer (headline + a live chart), Data (autofiltered), How this was calculated
```

- **`result.summary()`** — three to five plain sentences: the finding, its size, and the one caveat that matters, written by the model from the numbers already on the card — it never computes one itself. A paid call, so it's cached after the first one; `Failure.summary()` never calls the model at all.
- **`pyds.deck(results, title=, path="deck.pptx")`** — a title slide, one large-chart slide per result, a closing summary slide, and an appendix with the full tables, the metric definitions used, and the SQL. Charts are native, editable PowerPoint chart objects built straight from each result's own data — not a picture of the notebook chart — so a `d.cohort()` result becomes a real colored table instead of a chart that doesn't apply to it.
- **`pyds.brand(logo=, colors=, template=)`** — configure once, applied everywhere: every chart's colors, the notebook/HTML card's accent and logo, and the logo/colors on every `.docx`/`.pptx`/`.xlsx` export. `template` is your own `.potx`/`.pptx` file, used as the base for every pptx export and deck instead of the stock layout. A missing logo or template file is skipped quietly — a decorative asset should never block a report.
- **`.xlsx` export** — "Answer" (headline, interpretation, as-of date, caveats, a live chart referencing the Data sheet), "Data" (the full table, real numeric cells with a formatted number style, autofiltered), "How this was calculated" (the metric definition, the SQL, the model call and prompt version — or an honest "no model call, computed deterministically" for the eight verbs that never touch the LLM at all).
- **Shareable `.html`** — already a single, genuinely self-contained file (the chart is inline SVG, there's no external stylesheet, font, or image — a configured logo is embedded as a base64 data URI, never a file path). Includes a print stylesheet, so printing or saving to PDF from a browser shows the full table/code/trace instead of a collapsed disclosure triangle.

## Failures render as a card, not a traceback

A call that hits a real problem — no API key configured, a bad file
path, an unknown target column — returns a `Failure` (also a `Result`)
instead of raising. `Failure` is falsy, so `if not result:` is the check:

```python
d = pyds.load("sales.csv")
if not d:
    print(d.explain())  # what went wrong, and what to try next
```

For a script where nobody is there to read a card, turn this off and get
the old raise-through behavior back:

```python
pyds.config.set_strict(True)
```

## Privacy default

`send_data_values = false` by default: the model gets column names, types
and summary statistics, not your raw rows. Your data does not leave the
building unless you explicitly opt in. A date column's min/max (used to
resolve a relative phrase like "last quarter" against your data's own
timeline, not today's real-world date) is one such summary statistic and
is sent either way; per-row sample values are the part this flag gates.
Set it via:

```bash
pip install pyds-ai
```
```python
import pyds_ai as pyds

cfg = pyds.config.load_config()
cfg["privacy"]["send_data_values"] = True
pyds.config.save_config(cfg)
```

Your API key is stored in your OS credential store automatically when
`pip install pyds-ai[keyring]` is installed. Without it, the key falls
back to `~/.pyds_ai/config.toml`, restricted to your account with
owner-only file permissions on macOS/Linux — on Windows, `os.chmod`
can't restrict an NTFS ACL, so the file stays plain text there and
`pyds.setup()` says so plainly rather than claiming a protection it
can't actually provide; install `pyds-ai[keyring]` for real protection
on Windows. Either way, `pyds.setup()` never prints the key back.

## Installing extras

```bash
pip install pyds-ai[ml]          # AutoML: scikit-learn (+ joblib, shap)
pip install pyds-ai[rag]         # RAG: pdf/docx/html parsing
pip install pyds-ai[rag-chroma]  # persistent vector store (Chroma)
pip install pyds-ai[rag-faiss]   # in-process ANN vector store (FAISS)
pip install pyds-ai[rag-pgvector]# Postgres + pgvector vector store
pip install pyds-ai[report]      # result.save(): .pptx, .docx; pyds.deck() (.xlsx and Excel loading need no extra — openpyxl is core)
pip install pyds-ai[report-pdf]  # result.save(...".pdf"); needs Pango/cairo/GDK-pixbuf too (see below), not just pip install
pip install pyds-ai[anthropic]   # or [openai], [gemini] — Ollama needs no extra
pip install pyds-ai[keyring]     # store your API key in the OS credential store
pip install pyds-ai[db]          # d.export_to_sql(): SQLAlchemy (+ your own DB driver, e.g. pyodbc/oracledb)
pip install pyds-ai[pdf-tables]  # pyds.pdf_tables() / loading tables out of a .pdf
pip install pyds-ai[all]         # everything above, all three vector stores included
```

Importing a missing extra returns a friendly message with the exact pip
command, never a bare `ImportError`. The default `pyds.knowledge()` vector
store is a zero-dependency in-memory hashing embedder — RAG works the
moment `pyds-ai[rag]` is installed, no embedding API call required.

`report-pdf` is split out from `report` on purpose: `pip install` always
succeeds for weasyprint itself, but actually rendering a PDF needs
Pango/cairo/GDK-pixbuf — native libraries pip can't install (on Windows,
a separate GTK3 runtime). `result.save("x.pptx")`/`.docx` never need any
of that; a `.pdf` save without those native libraries fails with a message
telling you what to install, instead of every other format failing along
with it.

## Command line

```
$ pyds doctor
[X]  Config file
      Fix: run `pyds setup`.
[X]  API key
      Fix: run `pyds setup` (or set the anthropic provider's API key environment variable).
[OK] DuckDB

Run the fix command(s) above, then run `pyds doctor` again.
```

`pyds doctor --json` prints the same checks as raw JSON for scripts.
`pyds setup` runs the same interactive key wizard as `pyds.setup()`, from
a terminal instead of a notebook. `pyds report <data> "<question>" --out
<path>` and `pyds eval <cases.csv> --docs <folder>` run the `ask` and
`evaluate` pipelines end to end without opening Python; add `--watch` to
`pyds eval` to record the run and flag a regression against the last one.

## Plugins

Internal teams can register a company database reader or a private model
provider without forking, via entry points:

```toml
[project.entry-points."pyds.providers"]
internal = "mycompany.pyds_plugin:InternalLLM"
```

Anything registered under the `pyds.providers` group is picked up by
`pyds.setup(provider="internal")` / `pyds.config.get_llm_client()` the
same way the four built-in providers are.

## One session, one process

pyds-ai keeps one process-wide session — one DuckDB connection shared by
every dataset you load. That fits its intended use (a single notebook
kernel, one person at a time) but it is **not** a concurrency-safe
building block: a Streamlit app or web API serving multiple users on
different threads would have them all silently sharing the same
connection, loaded datasets, and defined metrics, with no isolation
between requests. If you're wiring pyds-ai into something like that,
give each request/user its own process (or otherwise don't share one
`import pyds_ai` across them) rather than assuming this is handled for
you.

## Development

```bash
git clone <this repo> && cd pyds-ai
python -m venv .venv && .venv/Scripts/activate   # or source .venv/bin/activate
pip install -e ".[all,test]"
pytest
```

The test suite runs fully offline: a fake LLM client (`tests/support.py`)
stands in for every provider, so no API key or network access is needed
to validate the package.

## Publishing (maintainers)

```bash
python -m pip install --upgrade build twine
python -m build            # writes dist/*.whl and dist/*.tar.gz
twine check dist/*
twine upload dist/*        # or: twine upload --repository testpypi dist/*
```

`jsaimanoj/pyds_ai` is the real repo path, used consistently across
`pyproject.toml`, this README's badges and screenshot URLs, `mkdocs.yml`,
`CONTRIBUTING.md`, and `.github/ISSUE_TEMPLATE/config.yml`. Two things
depend on it working correctly, not just looking right: the README's two
screenshots are absolute `raw.githubusercontent.com` URLs (a relative
path renders fine on GitHub but shows broken on the PyPI project page,
which doesn't resolve repo-relative links) — they'll go live once the
`docs/images/*.png` files are actually pushed to `main`; and the CI/
docs-deploy workflows trigger on push to `main`, this repo's default
branch.

## License

Apache-2.0 — see [LICENSE](LICENSE).
