Metadata-Version: 2.4
Name: google-cloud-datacatalog-lineage-producer-client
Version: 0.2.7
Summary: Producer library for GCP data lineage API.
Author: Google
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: cachetools>=5.0.0
Requires-Dist: google-cloud-datacatalog-lineage<1.0.0,>=0.3.11
Dynamic: license-file

# Data Lineage Producer Python Library

Python producer library for [Data Lineage](https://cloud.google.com/dataplex/docs/about-data-lineage).

The library provides synchronous and asynchronous clients for interacting with
the Data Lineage API.

## Installation

Requires Python 3.9 or later.

To install using pip:

```shell
pip install google-cloud-datacatalog-lineage-producer-client
```

## Before you begin

1. [Enable the Data Lineage API](https://console.cloud.google.com/apis/library/datalineage.googleapis.com)
   in your Google Cloud project.
2. Make sure your principal has a role that allows reporting lineage, such as
   `roles/datalineage.producer` or `roles/datalineage.editor`.
3. Set up Application Default Credentials:

   ```shell
   gcloud auth application-default login
   ```

## Composing an OpenLineage message

```python
open_lineage_message = {
    "eventType": "COMPLETE",
    "eventTime": "2023-04-04T13:21:16.098Z",
    "producer": "producer-1",
    "schemaURL": "https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunEvent",
    "job": {
        "namespace": "namespace-1",
        "name": "job-1",
    },
    "run": {
        "runId": "1234-5678",
    },
    "inputs": [
        {
            "namespace": "bigquery",
            "name": "input_project.input_dataset.input_table",
        }
    ],
    "outputs": [
        {
            "namespace": "bigquery",
            "name": "output_project.output_dataset.output_table",
        }
    ],
}
```

or, to convert it from an existing JSON string:

```python
import json

open_lineage_message = json.loads(json_string)
```

## Performing calls

To perform a call, you need to:

1. Create a client
2. Create a request object
3. Call a corresponding method

`parent` is the project and location that the lineage is reported to, in the
format `projects/PROJECT_ID/locations/LOCATION`.

```python
from google.cloud.datacatalog.lineage.producer_client.v1 import SyncLineageClient
from google.cloud.datacatalog_lineage_v1 import ProcessOpenLineageRunEventRequest

client = SyncLineageClient()
parent = "projects/PROJECT_ID/locations/LOCATION"
request = ProcessOpenLineageRunEventRequest(
    parent=parent,
    open_lineage=open_lineage_message,
)
response = client.process_open_lineage_run_event(request)
print(response)
```

For asynchronous calls:

```python
import asyncio

from google.cloud.datacatalog.lineage.producer_client.v1 import AsyncLineageClient
from google.cloud.datacatalog_lineage_v1 import ProcessOpenLineageRunEventRequest

async def main():
    client = AsyncLineageClient()
    parent = "projects/PROJECT_ID/locations/LOCATION"
    request = ProcessOpenLineageRunEventRequest(
        parent=parent,
        open_lineage=open_lineage_message,
    )
    response = await client.process_open_lineage_run_event(request)
    print(response)

asyncio.run(main())
```

The response contains the names of the created process, run and lineage events.
