Metadata-Version: 2.4
Name: datamuru
Version: 0.4.0a0
Summary: Open source enterprise data infrastructure framework
Author: DataMuru
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/AjayAJ2000/DataMuru
Project-URL: Documentation, https://ajayaj2000.github.io/DataMuru/
Project-URL: Repository, https://github.com/AjayAJ2000/DataMuru
Project-URL: Issues, https://github.com/AjayAJ2000/DataMuru/issues
Project-URL: Changelog, https://github.com/AjayAJ2000/DataMuru/blob/main/CHANGELOG.md
Keywords: databricks,data-platform,data-mesh,data-governance,infrastructure
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Systems Administration
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click<9,>=8.1
Requires-Dist: pydantic<3,>=2.0
Requires-Dist: PyYAML<7,>=6.0
Requires-Dist: requests<3,>=2.31
Requires-Dist: rich<14,>=13.0
Provides-Extra: databricks
Requires-Dist: databricks-sdk<1,>=0.18; extra == "databricks"
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python<4,>=3.10; extra == "snowflake"
Provides-Extra: aws
Requires-Dist: boto3<2,>=1.34; extra == "aws"
Provides-Extra: azure
Requires-Dist: azure-identity<2,>=1.15; extra == "azure"
Requires-Dist: azure-keyvault-secrets<5,>=4.7; extra == "azure"
Requires-Dist: azure-storage-blob<13,>=12.19; extra == "azure"
Provides-Extra: gcp
Requires-Dist: google-cloud-secret-manager<3,>=2.16; extra == "gcp"
Requires-Dist: google-cloud-storage<3,>=2.13; extra == "gcp"
Provides-Extra: dev
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: mypy<2,>=1.8; extra == "dev"
Requires-Dist: pre-commit<4,>=3.6; extra == "dev"
Requires-Dist: pytest<9,>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio<1,>=0.23; extra == "dev"
Requires-Dist: pytest-cov<6,>=4.1; extra == "dev"
Requires-Dist: pytest-mock<4,>=3.12; extra == "dev"
Requires-Dist: respx<1,>=0.21; extra == "dev"
Requires-Dist: ruff<1,>=0.2; extra == "dev"
Requires-Dist: twine<7,>=5; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
Requires-Dist: mkdocs-git-revision-date-localized-plugin<2,>=1.2; extra == "docs"
Requires-Dist: mkdocs-material<10,>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]<1,>=0.25; extra == "docs"
Provides-Extra: test
Requires-Dist: pytest<9,>=8.0; extra == "test"
Requires-Dist: pytest-cov<6,>=4.1; extra == "test"
Requires-Dist: pytest-mock<4,>=3.12; extra == "test"
Dynamic: license-file

<p align="center">
  <img src="docs/assets/datamuru-mark-canva.png" alt="DataMuru logo" width="240" height="240">
</p>

<h1 align="center">DataMuru</h1>

<p align="center"><strong>Provider-agnostic data infrastructure, governed by design.</strong></p>

[![CI](https://github.com/AjayAJ2000/DataMuru/actions/workflows/ci.yml/badge.svg)](https://github.com/AjayAJ2000/DataMuru/actions/workflows/ci.yml)
[![Documentation](https://github.com/AjayAJ2000/DataMuru/actions/workflows/docs.yml/badge.svg)](https://ajayaj2000.github.io/DataMuru/)
[![PyPI](https://img.shields.io/pypi/v/datamuru.svg)](https://pypi.org/project/datamuru/)
![License](https://img.shields.io/badge/License-Apache%202.0-0D7377)
[![Python](https://img.shields.io/pypi/pyversions/datamuru.svg)](https://pypi.org/project/datamuru/)

DataMuru is an Apache-2.0, Python-first data infrastructure framework.
It provisions and governs provider-backed data estates from declarative configuration.
This public repository is the canonical home of the DataMuru Open Source Edition,
the shared configuration contract, the `datamuru` PyPI package, and the public documentation.

This repository contains the `v0.3 alpha` implementation with the following scope:

- Foundation layer: config loading, validation, local state, planning, apply, and destroy.
- Azure-first Databricks provider abstraction with multi-cloud-ready interfaces.
- Basic governance compilation: taxonomy, RBAC, and masking integration points.
- Core CLI surface: `init`, `validate`, `plan`, `apply`, and `destroy`.
- MkDocs-based product documentation written from an international SaaS product perspective.
- Live Databricks catalog, schema, ACL, import-discovery, and supported identity workflows.

## Current stage

DataMuru is currently in the `v0.3 alpha` stage.

The package and CLI execute supported live Databricks operations when `execution_mode: live-apply` is configured. Alpha support currently covers catalogs, schemas, Unity Catalog ACLs, import discovery/config generation, and capability-aware account SCIM identity operations.

## Delivery status

Product execution is tracked in a private DataMuru Product Roadmap GitHub
Project. The repository snapshot is maintained in
[PROJECT_STATUS.md](PROJECT_STATUS.md), including readiness, risks, milestones,
completed work, blocked work, and next recommended actions.

## Open-core model

- **DataMuru OSS:** this public Apache-2.0 repository and PyPI package.
- **DataMuru Enterprise:** a private extension repository for paid capabilities such as multi-workspace orchestration, advanced compliance automation, hosted services, SSO/SAML, SIEM integrations, and SLA-backed operations.

Enterprise extends the public core; it does not maintain a competing fork of the core configuration model or CLI.

## Design constraints

- Multi-cloud is an architectural requirement, but not a parity requirement for the alpha slice.
- `open-source` vs `enterprise` is the only runtime packaging boundary.
- Zero external data-tool runtime dependencies means no Terraform, dbt, Airflow, Great Expectations, Fivetran, or Airbyte required for the framework to operate.

## Quick start

```bash
pip install datamuru
datamuru validate --config datamuru.yml
datamuru doctor --config datamuru.yml
datamuru plan --config datamuru.yml
```

## Installation

From PyPI:

```bash
pip install datamuru
```

## Documentation

This repository now includes a full MkDocs documentation site.

- MkDocs config: [mkdocs.yml](mkdocs.yml)
- Docs source: [docs](docs/)
- Published documentation: [ajayaj2000.github.io/DataMuru](https://ajayaj2000.github.io/DataMuru/)
- Support and documentation feedback:
  [Support and feedback](https://ajayaj2000.github.io/DataMuru/operations/support/)

To work on the documentation locally:

```bash
python -m pip install -e ".[docs]"
python -m mkdocs serve
```

## Product usage guidance

For operator guidance and rollout practices, start with:

- [Product Usage Guidelines](docs/operations/usage-guidelines.md)
- [Team Adoption Guidelines](docs/operations/team-adoption-guidelines.md)

## Trying it with Databricks Free Edition

If you want to try the framework with your own Databricks personal workspace, start here:

- [Databricks Free Edition Setup](docs/getting-started/databricks-free-edition.md)

For package-oriented team usage, also read:

- [Packaging and Distribution](docs/product/packaging-and-distribution.md)
- [Product Usage Guidelines](docs/operations/usage-guidelines.md)

## Repository standards

- `datamuru/`: shared installable Python package
- `docs/`: versioned product documentation published through GitHub Pages
- `schemas/`: public configuration contracts
- `tests/`: unit, provider-contract, and end-to-end tests
- `.github/workflows/`: CI, documentation deployment, link validation, and
  trusted PyPI publishing

See [CONTRIBUTING.md](CONTRIBUTING.md) for the required local quality gate.
