Metadata-Version: 2.5
Name: pydabs-genie-agent-retarget
Version: 0.1.1
Summary: Rewrite Unity Catalog references in Genie Agent (Genie space) definitions; a PyDABs mutator for Declarative Automation Bundles
Project-URL: Homepage, https://github.com/alexott/pydabs-genie-agent-retarget
Project-URL: Issues, https://github.com/alexott/pydabs-genie-agent-retarget/issues
Author-email: Alex Ott <alexott@gmail.com>
License:                                Databricks License
                                Copyright (2026) Databricks, Inc.
        
            Definitions.
        
            Agreement: The agreement between Databricks, Inc., and you governing
            the use of the Databricks Services, as that term is defined in
            the Master Cloud Services Agreement (MCSA) located at
            www.databricks.com/legal/mcsa.
        
            Licensed Materials: The source code, object code, data, and/or other
            works to which this license applies.
        
            Scope of Use. You may not use the Licensed Materials except in
            connection with your use of the Databricks Services pursuant to
            the Agreement. Your use of the Licensed Materials must comply at all
            times with any restrictions applicable to the Databricks Services,
            generally, and must be used in accordance with any applicable
            documentation. You may view, use, copy, modify, publish, and/or
            distribute the Licensed Materials solely for the purposes of using
            the Licensed Materials within or connecting to the Databricks Services.
            If you do not agree to these terms, you may not view, use, copy,
            modify, publish, and/or distribute the Licensed Materials.
        
            Redistribution. You may redistribute and sublicense the Licensed
            Materials so long as all use is in compliance with these terms.
            In addition:
        
                -   You must give any other recipients a copy of this License;
                -   You must cause any modified files to carry prominent notices
                    stating that you changed the files;
                -   You must retain, in any derivative works that you distribute,
                    all copyright, patent, trademark, and attribution notices,
                    excluding those notices that do not pertain to any part of
                    the derivative works; and
                -   If a "NOTICE" text file is provided as part of its
                    distribution, then any derivative works that you distribute
                    must include a readable copy of the attribution notices
                    contained within such NOTICE file, excluding those notices
                    that do not pertain to any part of the derivative works.
        
            You may add your own copyright statement to your modifications and may
            provide additional license terms and conditions for use, reproduction,
            or distribution of your modifications, or for any such derivative works
            as a whole, provided your use, reproduction, and distribution of
            the Licensed Materials otherwise complies with the conditions stated
            in this License.
        
            Termination. This license terminates automatically upon your breach of
            these terms or upon the termination of your Agreement. Additionally,
            Databricks may terminate this license at any time on notice. Upon
            termination, you must permanently delete the Licensed Materials and
            all copies thereof.
        
            DISCLAIMER; LIMITATION OF LIABILITY.
        
            THE LICENSED MATERIALS ARE PROVIDED “AS-IS” AND WITH ALL FAULTS.
            DATABRICKS, ON BEHALF OF ITSELF AND ITS LICENSORS, SPECIFICALLY
            DISCLAIMS ALL WARRANTIES RELATING TO THE LICENSED MATERIALS, EXPRESS
            AND IMPLIED, INCLUDING, WITHOUT LIMITATION, IMPLIED WARRANTIES,
            CONDITIONS AND OTHER TERMS OF MERCHANTABILITY, SATISFACTORY QUALITY OR
            FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. DATABRICKS AND
            ITS LICENSORS TOTAL AGGREGATE LIABILITY RELATING TO OR ARISING OUT OF
            YOUR USE OF OR DATABRICKS’ PROVISIONING OF THE LICENSED MATERIALS SHALL
            BE LIMITED TO ONE THOUSAND ($1,000) DOLLARS.  IN NO EVENT SHALL
            THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
            OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
            ARISING FROM, OUT OF OR IN CONNECTION WITH THE LICENSED MATERIALS OR
            THE USE OR OTHER DEALINGS IN THE LICENSED MATERIALS.
License-File: LICENSE
Keywords: dabs,genie-agents
Requires-Python: >=3.10
Provides-Extra: bundles
Requires-Dist: databricks-bundles>=1.19.0; extra == 'bundles'
Description-Content-Type: text/markdown

# pydabs-genie-agent-retarget

Rewrite Unity Catalog references in Genie Agent (formerly Genie space)
definitions when you deploy them with Declarative Automation Bundles (DABs).

A serialized Genie space hardcodes `catalog.schema.table` names in table
identifiers, join specs, SQL snippets, example and benchmark SQL, and text
instructions. `genie_agent_retarget` is a Python mutator for DABs that rewrites
all of them per target with prefix rules:

| Rule level | Example                                              |
|------------|------------------------------------------------------|
| table      | `main.intel.oss_iocs: ${var.catalog}.intel.iocs`     |
| schema     | `main.gold: ${var.catalog}.${var.gold_schema}`       |
| catalog    | `main: ${var.catalog}`                               |

The most specific rule wins, and each name is rewritten once (rules never
chain). Matching is case-insensitive and handles backtick-quoted names, in the
space and in the rules. A rule whose two sides have a different number of parts
fails `databricks bundle validate`.

After a rewrite, the mutator sorts `data_sources.tables` by identifier, because
the Genie API rejects an unsorted list. If a table rule renames a table, the
mutator also renames the short table qualifiers in SQL fields: join specs, SQL
snippets (filters, measures, expressions), example SQL, and benchmark SQL
(`oss_iocs.ioc_type = 'URL'` becomes `iocs.ioc_type = 'URL'`). It skips a short
name that another table of the space also has, because it is ambiguous.

## Usage

1. Add the package to the bundle's virtual environment (with `uv`):

   ```toml
   [dependency-groups]
   dev = [
       "databricks-bundles==1.19.0",
       "pydabs-genie-agent-retarget @ git+https://github.com/alexott/pydabs-genie-agent-retarget",
   ]
   ```

   Or copy `src/genie_agent_retarget.py` next to `databricks.yml`. It has no
   dependencies except `databricks-bundles`.

2. Register the mutator and define the `genie_table_mapping` variable in
   `databricks.yml`:

   ```yaml
   python:
     venv_path: .venv
     mutators:
       - 'genie_agent_retarget:retarget_genie_spaces'

   variables:
     genie_table_mapping:
       type: complex
       default:
         main.gold: ${var.catalog}.${var.gold_schema}
   ```

3. Keep each space as `src/<name>.geniespace.json` (the output of
   `databricks bundle generate genie-space`) and reference it with `file_path`.

Use `genie_table_mapping: {}` in a target that needs no rewriting.

## Command line

The same code works on a single file. This is useful when you export a space
from an environment that is not the authoring one, and must convert it back
to the authoring names:

```sh
python -m genie_agent_retarget src/x.geniespace.json --in-place \
  -m dev_catalog.gold_dev=main.gold
```

## Example

`example/` deploys 3 Genie spaces (CVE analysis, IoC analysis, and Lakewatch
security operations over 22 OCSF gold tables). Each data source location is a
variable (`ocsf_schema`, `cve_schema`, `iocs_table`):

- `dev` moves the spaces to other schemas and renames one table.
- `azure` deploys them to another workspace, where the data is in other
  catalogs and schemas.
- `prod` keeps the names, so the files are deployed without changes.

```sh
cd example
uv sync
databricks bundle validate -t dev
databricks bundle deploy -t dev
```

The example uses names from our environments: the CLI profiles (`lw-field`,
`az-alexott`), the catalogs and schemas of each target, and the warehouse name.
The JSON files keep the names they were authored against
(`fe_lakewatch_catalog...`), because the rules map from these names. To try the
example in your workspace, change `workspace.profile`, `warehouse_name`, and the
`ocsf_schema`, `cve_schema`, and `iocs_table` variables of a target to your
tables. The tables must have the same columns, otherwise the instructions and
example SQL of the spaces do not match the data.

The Genie API checks that all tables exist when it creates or updates a space,
so the tables must exist before you deploy.

If the `DATABRICKS_CONFIG_PROFILE` environment variable is set, the CLI uses
that profile instead of the target's `workspace.profile`. Unset it when you
deploy targets that use different workspaces.

## Tests

```sh
uv sync && uv run pytest
```

## License

This project is licensed under the [Databricks License](LICENSE).
