Metadata-Version: 2.4
Name: datahub-airbyte-source
Version: 0.2.0
Summary: Custom Airbyte source plugin for DataHub ingestion
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: acryl-datahub>=1.5.0
Requires-Dist: requests>=2.31.0

# datahub-airbyte-source

Custom Airbyte source plugin for DataHub ingestion.

## What this provides

- Registers `source.type: airbyte` via DataHub plugin entry point.
- Extracts Airbyte connections as `DataFlow` + `DataJob`.
- Extracts recent sync jobs as `DataProcessInstance` events.
- Emits table-level lineage and optional column-level lineage when stream field mappings are present.

## Install in DataHub ingestion

Use the package name in DataHub UI **Advanced -> Extra Pip Libraries**:

`datahub-airbyte-source==0.2.0`

Or install from a git URL/wheel URL.

## Example recipe

```yaml
source:
  type: airbyte
  config:
    api:
      # Auto detects Cloud when server_url host is api.airbyte.com.
      deployment_type: auto
      server_url: "https://api.airbyte.com"
      api_key: ${AIRBYTE_API_KEY}
    connection_patterns:
      allow:
        - ".*"
    history_sync_lookback_period: 7
    include_column_lineage: true

sink:
  type: datahub-rest
  config:
    server: "http://gms:8080"
```

## Notes

- Supports both OSS `/api/v1/*` and Airbyte Cloud/public API endpoints.
- `deployment_type` can be `auto`, `oss`, or `cloud`.

