Metadata-Version: 2.4
Name: drakun-lakehouse-core
Version: 0.6.1
Summary: The lakehouse as the signed-in user: Lakekeeper catalog browsing, dataset files and previews (shared by the JupyterLab explorer, the agent MCP server and the IDE extension).
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.28
Requires-Dist: pyarrow>=14
Provides-Extra: preview
Requires-Dist: pandas>=2; extra == "preview"
Requires-Dist: openpyxl>=3; extra == "preview"
Provides-Extra: iceberg
Requires-Dist: pyiceberg[pyarrow]>=0.10; extra == "iceberg"
Provides-Extra: mcp
Requires-Dist: fastmcp>=2; extra == "mcp"
Provides-Extra: local
Requires-Dist: msal>=1.28; extra == "local"
Requires-Dist: msal-extensions>=1.2; extra == "local"
Provides-Extra: spark
Requires-Dist: pyspark-client==4.1.2; extra == "spark"
Provides-Extra: notebooks
Requires-Dist: pycrdt>=0.12; extra == "notebooks"
Requires-Dist: jupyter-ydoc>=3; extra == "notebooks"
Requires-Dist: websockets>=13; extra == "notebooks"
Provides-Extra: all
Requires-Dist: drakun-lakehouse-core[iceberg,local,mcp,notebooks,preview,spark]; extra == "all"
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: drakun-lakehouse-core[preview]; extra == "test"

# lakehouse_core

The lakehouse as the signed-in user: Lakekeeper catalog browsing, dataset
files and previews. One implementation behind three front ends - the
JupyterLab Lakehouse explorer, the agents' MCP server and (V1) the VS Code
extension. Lakekeeper decides every permission. Plan:
[`docs/data-platform/vscode-extension-plan.md`](../../../../../docs/data-platform/vscode-extension-plan.md).

| Module | What |
|---|---|
| `config` | catalog URI, warehouse, files root (env or `configure()`) |
| `auth` | the user's token: `set_provider(fn)`, `LAKEHOUSE_ACCESS_TOKEN`, JupyterHub `auth_state`, or the laptop sign-in (MSAL) |
| `catalog` | `Browser` (REST: children, table details); `iceberg_catalog`, `describe_table`, `preview_table` (PyIceberg) |
| `lakefiles` | dataset files (`import lakefiles` in notebooks is a shim for this module) |
| `preview` | first rows / text of csv, tsv, parquet, xlsx, json, jsonl (+ .gz) |
| `mcp_server` | `python -m lakehouse_core.mcp_server` - read-only tools for agents (honours `agent_mode`) |
| `guide` | the agent guide: `guide.md` -> variants `pod` (= the mounted ai-agent/AGENTS.md) and `local` |
| `notebooks`, `notebook_mcp` | `python -m lakehouse_core.notebook_mcp` - read, edit and run notebooks in the user's JupyterHub server through its collaboration room and the notebook's own kernel, and run a local `.py` file's `# %%` cells on a JupyterHub kernel (`run_py_cell`) (`notebooks` extra) |
| `agents` | `python -m lakehouse_core agent-setup` - guide + MCP config for Claude Code, Cursor, Copilot, Codex |
| `hub` | the user's JupyterHub server from a laptop: token exchange, start/stop, connection info for remote kernels |
| `spark` | Spark on the platform from a local Python (V4): `from lakehouse_core.spark import spark`, same helpers as the Spark kernel; via the Spark Connect gateway |
| `metadata` | OpenMetadata context (V3a, dormant until a profile sets `openmetadata_url`) |
| `rpc` | JSON-lines backend of the VS Code extension (`ide/vscode-lakehouse/`) |
| `profiles.json` | one entry per platform: catalog URL, Entra tenant / client / scope, `agent_mode` |

Deployed by mounting the `.py` files into the notebook image's
site-packages (`jupyterhub.tf` in `gcp-platform` and `lakehouse`), so a
change needs only a server restart.

On a laptop:

```bash
pip install 'drakun-lakehouse-core[all]'               # PyPI (import name: lakehouse_core), or the .vsix wheel
python -m lakehouse_core login --profile gcp-dev       # browser sign-in once (--device for SSH)
python -m lakehouse_core agent-setup                   # give your coding agent the guide + tools (lakehouse + jupyterhub MCP)
```

```python
from lakehouse_core import catalog, lakefiles as lf
df = catalog.iceberg_catalog().load_table("demo.sales").scan(limit=10_000).to_pandas()
lf.ls("files.media.landing/meta_ads")

from lakehouse_core.spark import spark      # Spark in the cluster, as you (python -m lakehouse_core spark-check)
spark.table("demo.sales").groupBy("region").count().show()
```

Changing the guide: edit `guide.md`, then
`python -m lakehouse_core guide --variant pod > ../../../gcp-platform/files/ai-agent/AGENTS.md`
(a test fails while they differ).

Tests (stub Lakekeeper, no cluster):

```bash
pip install -e '.[test]' && pytest        # in this directory
```
