Metadata-Version: 2.4
Name: vdstudio-dbtfabric-notebook
Version: 0.2.0
Summary: Fabric notebook dbt job runner (clone + run + persist) on top of the official dbt-fabricspark adapter.
Project-URL: Homepage, https://github.com/pradipsodha-acceleratedata/vdstudio-dbtfabric-notebook
Project-URL: Issues, https://github.com/pradipsodha-acceleratedata/vdstudio-dbtfabric-notebook/issues
Author-email: Pradip Sodha <pradip.sodha@acceleratedata.ai>
License: Apache-2.0
Keywords: dbt,fabric,fabricspark,notebook,vdstudio
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

# vdstudio-dbtfabric-notebook

A thin dbt job runner for Microsoft Fabric notebooks, built on top of the
official [`dbt-fabricspark`](https://pypi.org/project/dbt-fabricspark/) adapter.

A Fabric deploy notebook installs this package and makes a single call —
`run_dbt_job(config)` — which clones the dbt project over SSH using a
deploy key fetched from the attached lakehouse, runs the dbt command(s),
and persists logs and artifacts to the lakehouse.

## Install

```bash
pip install vdstudio-dbtfabric-notebook
```

The Fabric notebook runtime must also have `dbt-core` and `dbt-fabricspark`
available (the runner shells out to the `dbt` CLI). No other runtime
dependencies — the package has zero third-party Python deps.

## Use

Inside a Fabric notebook:

```python
from vdstudio_dbtfabric_notebook import (
    run_dbt_job,
    DbtJobConfig,
    RepoConfig,
    ConnectionConfig,
)

config = DbtJobConfig(
    command=["dbt deps", "dbt build --target prod"],
    repo=RepoConfig(
        url="git@github.com:org/repo.git",
        branch="main",
        deploy_key_onelake_path="Files/_studio/<domain-slug>/deploy-keys/<repo>.id_ed25519",
    ),
    connection=ConnectionConfig(
        lakehouse_name=lakehouse_name,
        lakehouse_id=lakehouse_id,
        workspace_id=workspace_id,
        workspace_name=workspace_name,
        schema_name=schema_name,
    ),
)

result = run_dbt_job(config)
```

## What it does

`run_dbt_job(config)`:

1. **Environment setup** — sets `LAKEHOUSE`, `LAKEHOUSE_ID`, `SCHEMA`,
   `WORKSPACE_ID`, `WORKSPACE_NAME`, `DBT_JOB_NAME` so the cloned project's
   `profiles.yml` resolves via `env_var()`.
2. **Clone over SSH** — reads the deploy key from
   `/lakehouse/default/<deploy_key_onelake_path>` (the notebook's attached
   default lakehouse is mounted there by Fabric), copies it to a `0600`
   temp file, and runs `git clone --depth 1 --branch <branch>` with
   `GIT_SSH_COMMAND` configured to use that key. The temp key is deleted
   immediately after, even on failure.
3. **Run** — `dbt deps`, then the dbt command(s) in order. Logs are
   emitted as JSON; `dbt.log`, `run_results.json`, and `manifest.json`
   are persisted to
   `/lakehouse/default/Files/logs/dbt/{YYYY}/{MM}/{DD}/{invocation_id}/`.

Commands can be a single string or a list run sequentially; the returned
`DbtResult` is from the last command.

## OneLake deploy-key convention

```
<lakehouse>/Files/_studio/<domain-slug>/deploy-keys/<repo-name>.id_ed25519
```

- One read-only deploy key **per repo**, generated and uploaded by studio
  at provisioning time.
- The notebook never sees the GitHub App PEM. It only reads back the
  per-repo private key from the lakehouse it's attached to.
- `deploy_key_onelake_path` is interpreted relative to `/lakehouse/default/`
  unless an absolute path is supplied.

## Ephemeral validation

The runner consumes whatever `ConnectionConfig`, `branch`, and `command`
it is given. A wrapper pipeline can override the lakehouse/workspace
parameters, the branch, and swap `--target prod` for an ephemeral target
to validate against a throwaway lakehouse — no code change required.

## Security

- **No PEM and no Key Vault calls anywhere.** The notebook's only secret
  exposure is the per-repo deploy key, scoped read-only.
- The staged temp key has `0600` permissions and is deleted in a
  `finally` block before the function returns.
- SSH host verification uses `StrictHostKeyChecking=accept-new` with a
  per-clone `UserKnownHostsFile` (`/tmp/known_hosts_deploy`). Pin the
  github.com fingerprint there if you want stricter guarantees.

## Development

```bash
pip install -e ".[dev]"
pytest
```

## License

Apache-2.0
