Metadata-Version: 2.4
Name: pyfabric-dev
Version: 0.5.2
Summary: Local development framework for Microsoft Fabric notebooks: source-of-truth Python, generated .Notebook artifacts, mocked Fabric APIs, parallel local testing.
Author-email: CashHero <engineering@cashhero.ai>
License: Apache-2.0
Project-URL: Homepage, https://github.com/CashHero/pyfabric-dev
Project-URL: Repository, https://github.com/CashHero/pyfabric-dev
Project-URL: Issues, https://github.com/CashHero/pyfabric-dev/issues
Project-URL: Changelog, https://github.com/CashHero/pyfabric-dev/blob/main/CHANGELOG.md
Keywords: microsoft-fabric,pyspark,delta-lake,notebook,medallion,lakehouse
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pyspark>=3.5.0
Requires-Dist: delta-spark>=3.2.0
Requires-Dist: azure-identity>=1.15.0
Requires-Dist: azure-keyvault-secrets>=4.7.0
Requires-Dist: python-dotenv>=1.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Dynamic: license-file

# pyfabric-dev

A development framework for [Microsoft Fabric](https://www.microsoft.com/fabric)
notebooks. Write plain Python modules in `src/`, generate Fabric `.Notebook`
artifacts from them, and run + test everything locally with mocked Fabric APIs.

> **Status:** `v0.5.2`. APIs are stabilizing but pre-1.0; minor versions may break.
>
> **Disclaimer:** not affiliated with or endorsed by Microsoft.

## Why

Fabric notebooks have two pain points:

1. **They can't `import` Python modules.** Code must be inlined into notebook
   cells, which fights every Python convention.
2. **They're nearly impossible to test locally.** `mssparkutils` and
   `notebookutils` only exist in the Fabric runtime.

`pyfabric-dev` solves both:

- A **source-of-truth pattern**: you write modules under `src/`; the generator
  compiles them into `.Notebook/` artifacts (committed to git, ready for Fabric
  Git integration).
- A **local execution layer**: mocks `mssparkutils` / `notebookutils`, sets up
  PySpark + Delta with a per-checkout Derby metastore, and runs your notebooks
  and pipelines off-cloud.
- A **parallel test runner** that mirrors a Fabric pipeline's batching, so the
  same suite that passes locally is the one that runs in CI.

## Install

Requires **Python 3.13 or 3.14** and a JDK 17+ on `$PATH` (for PySpark).

```bash
pip install pyfabric-dev
```

## CLIs

After installing, four console scripts are on your `PATH`:

| Command | Purpose |
|---|---|
| `pyfabric-generate` | Compile `src/` modules into Fabric `.Notebook/` artifacts |
| `pyfabric-run-notebook` | Execute one notebook locally (resolves `%run` deps) |
| `pyfabric-run-pipeline` | Execute a `pipeline-content.json` DAG locally |
| `pyfabric-test` | Parallel pytest runner with isolated per-worker metastores |

All four accept `--project-root <dir>` (default: CWD).

## Five-minute walkthrough

```bash
git clone https://github.com/CashHero/pyfabric-dev.git
cd pyfabric-dev/examples/minimal-medallion

pyfabric-generate    --project-root .
pyfabric-test        --project-root . --config config/test_batches.json
pyfabric-run-pipeline --project-root . --dry-run sales_etl
```

`examples/minimal-medallion/` is a working bronze/silver/gold pipeline over
toy sales data. See its `README.md` for details.

## Documentation

- [Getting started](docs/getting-started.md)
- [Source-of-truth pattern](docs/source-of-truth-pattern.md) — how `src/` → `.Notebook/` works
- [Local execution](docs/local-execution.md) — how the runner mocks Fabric APIs
- [Testing](docs/testing.md) — fixtures, parallelism, batching
- [Hooks reference](docs/hooks-reference.md) — customizing the runners for your project

## Used in production by

- CashHero — the internal pipeline this framework was extracted from.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). PRs welcome.

## License

[Apache 2.0](LICENSE). See [NOTICE](NOTICE) for attribution.
