Skip to content

# Source Types

## Execution Model

Every query ultimately executes through the federation engine, which provides federation across all sources. Sources fall into three categories based on their connectivity. [tool-verified: `provisa/core/models.py` lines 84–132] (REQ-550)

Category | Has Direct Driver | Has Federated Connector | Examples
--- | --- | --- | ---
Direct-capable | Yes | Yes | PostgreSQL, MySQL, MariaDB, SingleStore, SQL Server, Oracle, DuckDB
Federation only | No | Yes | Redshift, Druid, Exasol, Hive, Iceberg, Delta Lake, Hive (S3-backed)
Direct-read (replica) | Yes | Yes | Snowflake, Databricks, ClickHouse — driver reads data and lands a replica; queries run against the replica in the active engine
Materialize → Federation | No | No | REST/OpenAPI, remote GraphQL, gRPC, Neo4j Cypher, SPARQL, WebSocket, RSS, CSV, SQLite, Parquet, Ingest (push receiver), GovData, SharePoint, Splunk

Direct-capable sources execute single-source queries via their native driver (sub-100ms), bypassing the federation engine (REQ-027, REQ-229). They retain full connector support and participate in federation when joined with other sources (REQ-028).

Federation only sources are always queried through the federation layer. No direct driver exists (REQ-229).

Direct-read (replica) sources have a DirectDriver that reads from the warehouse natively (Arrow-native where available), lands a replica into the active engine's materialization store, and then queries run against that replica. See Warehouses as Named Sources.

Materialize sources have no federated connector. Provisa fetches their data (on startup or at query time) and caches it as Parquet in S3 or in PostgreSQL, making it reachable by the federation engine for cross-source queries (REQ-309).

## All Sources

Reference for every source type Provisa supports. "Direct driver" means single-source queries execute against the source natively (sub-100ms) (REQ-027). "Connector Name" is the federated connector used when the source participates in multi-source JOINs (REQ-028). [tool-verified: `provisa/core/source_registry.py` `SOURCE_TO_DIALECT`; `provisa/federation/trino_connectors.py` `trino_connector_name`]

### RDBMS

Source Type | Direct Driver | Connector Name | SQLGlot Dialect | Mutations
--- | --- | --- | --- | ---
`postgresql` | asyncpg | postgresql | postgres | Yes
`mysql` | aiomysql | mysql | mysql | Yes
`mariadb` | aiomysql | mariadb | mysql | Yes
`singlestore` | — | singlestore | singlestore | Federated
`sqlserver` | aioodbc | sqlserver | tsql | Yes
`oracle` | oracledb | oracle | oracle | Yes
`duckdb` | duckdb | memory | duckdb | Yes
`cockroachdb` | asyncpg (pg wire) | postgresql | postgres | Yes
`yugabytedb` | asyncpg (pg wire) | postgresql | postgres | Yes
`greenplum` | asyncpg (pg wire) | postgresql | postgres | Yes
`tidb` | aiomysql (mysql wire) | mysql | mysql | Yes

Wire-compatible databases reuse a base wire's JDBC driver, native async driver, and SQLGlot dialect — CockroachDB, YugabyteDB, and Greenplum ride the PostgreSQL wire; TiDB rides the MySQL wire. They need only registry entries, no new connector code. [tool-verified: `provisa/core/source_registry.py` `_PG_WIRE_TYPES`, `_MYSQL_WIRE_TYPES`] (REQ-950)

`firebird` (Firebird 3/4/5) and `airport` (Arrow Flight server) are registered source types reached in place via DuckDB community extensions when DuckDB is the active engine — no direct driver, no federated connector. [tool-verified: `provisa/core/models.py` lines 44, 93] (REQ-899)

### Cloud Data Warehouses

[tool-verified: `executor/drivers/snowflake.py`, `executor/drivers/databricks.py`, `executor/drivers/registry.py`]

Source Type | Direct Driver | Connector Name | SQLGlot Dialect | Mutations | Notes
--- | --- | --- | --- | --- | ---
`snowflake` | SnowflakeDriver | snowflake | snowflake | Federated | Reads via snowflake-connector-python; lands replica; `account`/`warehouse`/`role` in `federation_hints` (REQ-988)
`bigquery` | — | bigquery | bigquery | Federated | No DirectDriver; reaches via federation engine or BigQuery engine ATTACH
`databricks` | DatabricksDriver | delta_lake | databricks | Federated | Reads via databricks-sql-connector (Cloud Fetch, Arrow); lands replica; `http_path` required in `federation_hints` (REQ-987)
`redshift` | — | redshift | redshift | Federated | —
`fabric` | MssqlWarehouseDriver | — | tsql | Federated | Microsoft Fabric Warehouse; T-SQL over TDS, Azure AD auth; lands replica (REQ-995)
`synapse` | MssqlWarehouseDriver | — | tsql | Federated | Azure Synapse SQL; T-SQL over TDS, Azure AD auth; lands replica (REQ-995)
`trino` | SQLAlchemyDriver | — | — | Federated | Remote Trino/Presto coordinator read via the SQLAlchemy trino dialect; lands replica on any engine (REQ-994)

### Analytics / OLAP

[tool-verified: `executor/drivers/clickhouse.py`]

Source Type | Direct Driver | Connector Name | SQLGlot Dialect | Mutations | Notes
--- | --- | --- | --- | --- | ---
`clickhouse` | ClickHouseDriver | clickhouse | clickhouse | Federated | Reads via clickhouse-connect (HTTP); `secure: "true"` in `federation_hints` for TLS (REQ-986)
`druid` | — | druid | druid | No | —
`exasol` | — | exasol | exasol | No | —
`elasticsearch` | — | elasticsearch | — | No | Connector properties come from the type's mapping DSL [tool-verified: `trino_connectors.py:309`]
`pinot` | — | pinot | — | No | Trino `pinot` connector; `pinot.controller-urls` = host:port of the Pinot controller [tool-verified: `trino_connectors.py:199`]

### Data Lake / Open Table Formats

These source types are federation-only — no direct driver, no SQLGlot dialect. [tool-verified: `LAKE_ONLY_SOURCES` in `provisa/core/source_registry.py`] (REQ-229)

Source Type | Connector Name | Time Travel | Notes
--- | --- | --- | ---
`iceberg` | iceberg | Yes (`as_of` argument, REQ-372) | —
`delta_lake` | delta_lake | Yes (`as_of` argument, REQ-372) | —
`hive` | hive | No | —
`hive_s3` | hive | No | S3-backed Hive

### NoSQL

`mongodb`, `cassandra`, and `redis` have Trino connectors (`redis` builds its properties from the type's mapping DSL). [tool-verified: `provisa/federation/trino_connectors.py`; `provisa/core/models.py`] (REQ-017, REQ-1097)

Source Type | Connector Name | Mutations
--- | --- | ---
`mongodb` | mongodb | No
`cassandra` | cassandra | No
`redis` | redis | No

### Streaming

Source Type | Mechanism | Mutations
--- | --- | ---
`kafka` | Federated Kafka connector; schema via Confluent Schema Registry (Avro, Protobuf, JSON Schema), manual definition, or sample inference (REQ-147, REQ-150) | Sink only (REQ-176)
`websocket` | External WebSocket feed — connect, subscribe, receive events; results materialized (REQ-338) | No
`rss` | RSS 2.0 / Atom feed — poll, watermark by pubDate/updated; results materialized (REQ-342, REQ-343) | No

### Push Receiver

Source Type | Mechanism | Mutations
--- | --- | ---
`ingest` | External services POST JSON events; results materialized (REQ-331, REQ-335) | No

### Graph & Semantic

Source Type | Mechanism | Mutations
--- | --- | ---
`neo4j` | Cypher via HTTP API, results cached in PostgreSQL (REQ-295) | No
`sparql` | SPARQL 1.1 POST, results cached in PostgreSQL (REQ-297) | No

### File-Based

Two mechanisms cover files. Both use the `path` field instead of `host`/`port`. [tool-verified: `provisa/core/models.py`] (REQ-553)

Single-file sources — `sqlite`, `csv`, `parquet` point `path` at one file.

Source Type | Transports | Mutations
--- | --- | ---
`sqlite` | local | Yes
`csv` | local | No
`parquet` | local, `s3://` | No

Private buckets need credentials (AWS region and keys from the environment). For CSV over `s3://` or `http(s)://`, or to register many files at once, use the `files` source. [tool-verified: `provisa/file_source/source.py`]

`files` source — points `path` at a glob, crawls it recursively, and registers the directory as a federated catalog of tables. It reads many formats over many transports; the sets below come from the file connector (kenstott/calcite fork). [tool-verified: `provisa/core/catalog.py` `files` branch and `provisa/core/models.py` `SOURCE_TO_CONNECTOR`; format and transport lists from the calcite `file` adapter — `FileSchema.java`, `storage/StorageProviderFactory.java`]

Formats | Transports
--- | ---
CSV, TSV, JSON, YAML, Excel (XLS/XLSX), Parquet, Arrow, and documents converted to tables — HTML, Markdown, DOCX, PPTX | Local filesystem, HTTP(S), `s3://`, `hdfs://`, `ftp://`/`ftps://`, `sftp://`, `iceberg://`, SharePoint (REST and Microsoft Graph)

    - id: sales_files
      type: files
      path: s3://bucket/sales/**/*.csv   # glob; local and http(s):// also supported

### Observability & Other

`prometheus` has a Trino connector (properties built from the type's mapping DSL). `google_sheets` is a registered source type with no Trino connector and materializes through the API cache pipeline. [tool-verified: `provisa/federation/trino_connectors.py:314`; `provisa/core/models.py` lines 87–88]

Source Type | Connector Name | Mutations
--- | --- | ---
`google_sheets` | — (materialized) | No
`prometheus` | prometheus | No

### Enterprise SaaS Connectors

SharePoint and Splunk register through Apache Calcite connectors (kenstott/calcite fork). Neither has a direct driver — Provisa materializes their rows by launching the connector's bundled Calcite pgwire server (`pgwire-sharepoint`, `pgwire-splunk`), connecting to it as a generic PostgreSQL endpoint, and landing the rows into the materialize store for federation (REQ-954). Both connectors always enable case-insensitive name matching, matching each product's own case-insensitive semantics (REQ-725, REQ-730). [tool-verified: `provisa/core/models.py` lines 99–100; `provisa/federation/trino_connectors.py` lines 223–286]

#### `sharepoint`

SharePoint lists are enumerated as schemas and exposed as queryable tables (REQ-726, REQ-731). Two auth methods: `CLIENT_CREDENTIALS` (default) and certificate-based via a PFX certificate (REQ-727). Secret values in `mapping` are resolved through the secrets engine before reaching the connector (REQ-729). [tool-verified: `provisa/federation/trino_connectors.py` lines 230–252]

Source field | Connector property | Notes
--- | --- | ---
`base_url` or `host` | `site-url` | SharePoint site URL
`username` | `client-id` | Azure app client ID
`password` | `client-secret` | Azure app client secret
`database` | `tenant-id` | Azure tenant UUID
`mapping.auth_type` | `auth-type` | `CLIENT_CREDENTIALS` (default) or `CERTIFICATE`
`mapping.certificate_path` | `certificate-path` | PFX path when `auth_type: CERTIFICATE`
`mapping.certificate_password` | `certificate-password` | PFX password

When the connector does not expose `information_schema.columns`, register the table with explicit column definitions (obtained from the Microsoft Graph API) via the `registerTable` mutation (REQ-732).

    - id: hr-sharepoint
      type: sharepoint
      base_url: https://kenstott.sharepoint.com
      username: ${env:SP_CLIENT_ID}
      password: ${env:SP_CLIENT_SECRET}
      database: ${env:SP_TENANT_ID}
      mapping:
        auth_type: CLIENT_CREDENTIALS

#### `splunk`

Splunk search results are queryable as tables (e.g. `internal_server`) (REQ-721). The connector URL comes from `base_url`, or is constructed as `https://{host}:{port}` with a default port of `8089` (REQ-722). Auth: when `mapping.use_token` is `true` (the default), `password` is passed as the API token; when `false`, `username` and `password` are passed as separate credentials (REQ-723). [tool-verified: `provisa/federation/trino_connectors.py` lines 262–286]

Source field | Connector property | Notes
--- | --- | ---
`base_url` / `host` + `port` | `url` | `base_url`, else `https://host:port` (port default 8089)
`password` | `token` or `password` | token when `use_token: true`
`username` | `user` | only when `use_token: false`
`database` | `app` | restrict to a Splunk app
`mapping.datamodel_filter` | `datamodel-filter` | filter to a data model
`mapping.disable_ssl_validation` | `disable-ssl-validation` | for self-signed certs (REQ-724)

    - id: ops-splunk
      type: splunk
      host: splunk
      port: 8089
      password: ${env:SPLUNK_TOKEN}
      mapping:
        use_token: true
        disable_ssl_validation: true

### API Sources

Register any HTTP endpoint as a queryable table. [tool-verified: `provisa/core/models.py` `SourceType` enum] (REQ-314, REQ-307, REQ-322)

API Type | Discovery | Column Inference
--- | --- | ---
`openapi` | OpenAPI spec parsing (REQ-314, REQ-316) | Primitives → native, objects → JSONB
`graphql_remote` | Schema introspection (REQ-307, REQ-308) | Primitives → native, objects → JSONB
`grpc_remote` | Server reflection (REQ-322, REQ-325) | Primitives → native, objects → JSONB

API responses are fetched, cached in PostgreSQL (configurable TTL), and exposed as GraphQL types (REQ-309, REQ-318, REQ-327). Cached tables participate in federated queries like any other source (REQ-313).

JSONB rules: Complex columns (objects, arrays) stored as JSONB are not filterable (REQ-119). Sub-field access uses `->>` extraction in SQL (REQ-151). Relationships are declared between tables using scalar FK columns — JSONB blob columns are not join targets. Use JSONB promotion to convert nested fields into native scalar columns when filtering or joining on them is needed (REQ-119).

### GovData

U.S. government open data. Access is partitioned by subject grouping. [tool-verified: `provisa/core/models.py` lines 543–609]

Each `govdata` source selects one subject. That subject determines which GovData schemas are exposed. The `ref` and `geo` schemas are always included as linker schemas — they are not listed per subject but are always present. [tool-verified: `provisa/core/models.py` line 562–563 comment]

Subject | Schemas Exposed
--- | ---
`COMMERCE` | `sec`, `patents`
`ECONOMY` | `econ`
`EDUCATION` | `census`, `edu`
`HEALTH` | `health`
`CYBER` | `cyber_threat`, `cyber_vuln`
`PUBLIC_SAFETY` | `crime`
`ENVIRONMENT` | `lands`
`WEATHER` | `weather`
`GOVERNMENT` | `fedregister`, `fec`
`ALL` | Every schema above

    sources:

      - id: federal-commerce
        type: govdata
        subject: COMMERCE
        domain_id: federal-analytics
        description: U.S. commerce and securities data
        governance: pre-approved

Field | Required | Default | Description
--- | --- | --- | ---
`id` | Yes | — | Unique identifier
`subject` | Yes | — | One of the subject values above
`domain_id` | Yes | — | Domain this source belongs to
`description` | No | `""` | Human-readable description
`governance` | No | `pre-approved` | Governance level for all tables in this source

## Warehouses as Named Sources

Snowflake, Databricks, and ClickHouse can be registered as named sources independently of which federation engine is active. [tool-verified: `executor/drivers/snowflake.py` (REQ-988), `executor/drivers/databricks.py` (REQ-987), `executor/drivers/clickhouse.py` (REQ-986)]

When registered, Provisa reads the warehouse via the source's DirectDriver and lands a replica into the active engine's materialization store. The query then runs against that replica. This differs from the traditional direct-capable path (asyncpg, aiomysql) where the engine is bypassed entirely — here the engine still executes the query, but against a local replica rather than over the wire to the warehouse on every request.

Reads are Arrow-native where the warehouse supports it: Databricks uses Cloud Fetch, Snowflake uses `fetch_arrow_table`, and ClickHouse uses the native columnar HTTP interface.

Extended connection parameters that the standard `host`/`port`/`username`/`password` fields cannot carry go in `federation_hints`:

    sources:
      - id: my-databricks
        type: databricks
        host: my-workspace.azuredatabricks.net
        password: ${env:DATABRICKS_TOKEN}
        federation_hints:
          http_path: /sql/1.0/warehouses/xxxx   # required — the SQL Warehouse connection detail

      - id: my-snowflake
        type: snowflake
        host: org.snowflakecomputing.com
        username: svc_provisa
        password: ${env:SNOWFLAKE_PASSWORD}
        federation_hints:
          account: myorg-myaccount    # required — Snowflake account identifier
          warehouse: COMPUTE_WH       # optional — virtual warehouse to use
          role: PROVISA_ROLE          # optional — Snowflake role

      - id: my-clickhouse
        type: clickhouse
        host: ch.example.com
        port: 8123
        database: analytics
        username: default
        password: ${env:CLICKHOUSE_PASSWORD}
        federation_hints:
          secure: "true"              # optional — enables TLS on the HTTP interface

Registration as a named source is independent of selecting the same warehouse as the federation engine. A Snowflake source on a DuckDB engine lands a replica into DuckDB, not into Snowflake.

Cloud object/lake data (parquet, csv, iceberg, delta_lake files on S3 / GCS / R2) is a separate source type that attaches in place when the active engine has an ATTACH connector for that type. No replica is landed — the engine scans the object storage directly. Credentials for those sources also go in `federation_hints`:

    sources:
      - id: r2-events
        type: parquet
        path: s3://my-bucket/events/2026/*.parquet
        federation_hints:
          access_key_id: ${env:R2_ACCESS_KEY}
          secret_access_key: ${env:R2_SECRET}
          account_id: ${env:R2_ACCOUNT_ID}     # Cloudflare R2 account (S3-compatible)

## Source Configuration Fields

All sources share a common set of fields. [tool-verified: `provisa/core/models.py` `Source` class, lines 138–204]

Field | Required | Default | Description
--- | --- | --- | ---
`id` | Yes | — | Unique identifier; alphanumeric with hyphens/underscores
`type` | Yes | — | Source type (see tables above)
`host` | No | `""` | Hostname or IP
`port` | No | `0` | Port number
`database` | No | `""` | Database name
`username` | No | `""` | Username
`password` | No | `""` | Password; use `${env:VAR}` for secret resolution
`path` | No | `null` | File path or cloud URI for file-based and object/lake sources
`base_url` | No | `null` | Base URL for OpenAPI sources
`pool_min` | No | `1` | Minimum connection pool size (REQ-052)
`pool_max` | No | `5` | Maximum connection pool size (REQ-052)
`use_pgbouncer` | No | `false` | Route connections through PgBouncer (REQ-053)
`pgbouncer_port` | No | `6432` | PgBouncer port (REQ-053)
`cache_enabled` | No | `true` | Enable API response caching
`cache_ttl` | No | `null` | Cache TTL in seconds; inherits global default when null
`cache_catalog` | No | `null` | Federated catalog for API cache; defaults to source's own catalog
`cache_schema` | No | `api_cache` | Schema within the cache catalog
`naming_convention` | No | `null` | Override global naming convention for this source (REQ-194)
`federation_hints` | No | `{}` | Session properties passed to the federation engine, and extended connection params for warehouse sources (REQ-278, REQ-281)
`mapping` | No | `{}` | Type-specific connector settings for NoSQL and SaaS sources (e.g. SharePoint `auth_type`, Splunk `use_token`) (REQ-251)
`allowed_domains` | No | `[]` | Restrict source to specific domains; empty = unrestricted
`description` | No | `""` | Human-readable description

## Kafka Sources

Kafka topics are configured separately under `kafka_sources`, keyed by the source `id` of a registered `kafka` source. [tool-verified: `config/provisa.yaml` lines 138–151] (REQ-147)

    kafka_sources:

      - id: kafka-support
        topics:

          - id: tickets
            topic: support.tickets
            domain_id: sales-analytics
            description: "Inbound support tickets"
            default_window: 1h
            columns:

              - name: id
              - name: subject
              - name: status
              - name: created_at

Field | Description
--- | ---
`id` | Must match the `id` of a source with `type: kafka`
`topics[].id` | Logical name for this topic within Provisa
`topics[].topic` | Kafka topic name
`topics[].domain_id` | Domain this topic belongs to
`topics[].description` | Human-readable description
`topics[].default_window` | Default time window for windowed queries (e.g. `1h`) (REQ-148)
`topics[].columns` | Column definitions for the topic schema (REQ-150)

## Governance

Every registered table has a `governance` field. [tool-verified: `provisa/core/models.py` `GovernanceLevel` enum, lines 73–76]

Value | Behaviour
--- | ---
`pre-approved` | Queries run against this table with user rights alone; no registry approval required (REQ-003)

## Column Visibility

The `visible_to` field on each column is a list of role IDs that can see that column. [tool-verified: `provisa/core/models.py` `Column` class line 248; `config/provisa.yaml` lines 39–51]

    columns:

      - name: email
        visible_to: [admin]        # only admin role sees this column

      - name: region
        visible_to: [admin, analyst]  # both roles see this column

Columns omitted from a role's `visible_to` list do not appear in that role's GraphQL schema and cannot be queried or referenced in filters (REQ-039).

## Relationships

Relationships connect two registered tables and appear as nested fields in GraphQL. [tool-verified: `provisa/core/models.py` `Relationship` class lines 323–343; `config/provisa.yaml` lines 103–110] (REQ-019)

    relationships:

      - id: orders-to-customers
        source_table_id: orders
        target_table_id: customers
        source_column: customer_id
        target_column: id
        cardinality: many-to-one

Field | Required | Description
--- | --- | ---
`id` | Yes | Unique identifier for this relationship
`source_table_id` | Yes | Table that holds the foreign key
`target_table_id` | Yes | Table being referenced; empty for computed relationships
`source_column` | Yes | Column on the source table
`target_column` | Yes | Column on the target table; empty for computed relationships
`cardinality` | Yes | `many-to-one` or `one-to-many` (REQ-019)
`materialize` | No | Auto-create a materialized view for cross-source joins (REQ-158)
`refresh_interval` | No | MV refresh interval in seconds (default: 300)
`target_function_name` | No | DB function name for computed relationships
`function_arg` | No | Which function argument receives the source column value
`alias` | No | Human-readable relationship type (e.g. `WORKS_FOR`)
`graphql_alias` | No | Names the SDL field this relationship exposes on the parent type. When absent, the name is derived from the target table's `field_name` and relationship cardinality. [tool-verified: `provisa/compiler/schema_gen.py:1050`]
`disable_cypher` | No | When `true`, exclude this relationship from Cypher graph edges
`source_json_key` | No | Extract this key from source column as a JSON object before JOIN

Cardinality values [tool-verified: `provisa/core/models.py` `Cardinality` enum, lines 79–81]:

- `many-to-one` — each source row maps to one target row (FK to PK)
- `one-to-many` — each source row maps to multiple target rows (inverse of above)

## Row-Level Security Rules

RLS rules inject `WHERE` clauses at query time, scoped to a role and optionally to a table or domain. [tool-verified: `provisa/core/models.py` `RLSRule` class lines 391–395; `config/provisa.yaml` lines 128–131] (REQ-041)

    rls_rules:

      - table_id: orders          # applies to orders table only
        role_id: analyst
        filter: "region = current_setting('provisa.user_region')"

      - domain_id: sales-analytics  # applies to every table in domain (REQ-402)
        role_id: analyst
        filter: "tenant_id = current_setting('provisa.tenant_id')"

When both a domain-level and a table-level rule exist for the same role, the table-level rule takes precedence (REQ-403).

Field | Required | Description
--- | --- | ---
`table_id` | Conditional | Table to apply the rule to; mutually exclusive with `domain_id`
`domain_id` | Conditional | Domain to apply the rule to; applies to all tables in the domain (REQ-402)
`role_id` | Yes | Role this rule applies to
`filter` | Yes | SQL predicate injected into `WHERE`; may reference session variables (REQ-041)

## Functions and Webhooks

### DB Functions

Track a database function and expose it as a GraphQL query or mutation. [tool-verified: `provisa/core/models.py` `Function` class lines 423–438; `config/provisa.yaml` lines 152–164] (REQ-205)

Database sources can also auto-discover their stored procedures and functions from the vendor catalog (`pg_proc`, `information_schema.routines`, or vendor equivalents), removing the need to hand-register each one. Discovery reads `prokind` and `provolatile`: immutable/stable functions register as parameterized relations (proc arguments become query parameters, the same shape as OpenAPI GET tables), and volatile procedures register as mutations/tracked functions. Discovered routines flow through Stage-2 governance identically to hand-registered ones. [tool-verified: `provisa/api/admin/introspect.py:541`, `provisa/api/admin/introspect.py:593`] (REQ-887)

    functions:

      - name: get_customers_by_region
        source_id: sales-pg
        schema: public
        function_name: get_customers_by_region
        returns: customers
        domain_id: sales-analytics
        description: "Returns customers filtered by region"
        visible_to: [admin, analyst]
        kind: query
        arguments:

          - name: p_region
            type: String

Field | Required | Default | Description
--- | --- | --- | ---
`name` | Yes | — | GraphQL field name
`source_id` | Yes | — | Source containing the function
`schema` | No | `public` | Database schema
`function_name` | Yes | — | Actual database function name
`returns` | Yes | — | Registered table ID the function returns (REQ-207)
`arguments` | No | `[]` | List of `{name, type}` argument definitions (REQ-211)
`visible_to` | No | `[]` | Roles that can call this function
`writable_by` | No | `[]` | Roles that can call this as a mutation
`domain_id` | No | `""` | Domain this function belongs to
`description` | No | `null` | GraphQL field description
`kind` | No | `mutation` | `"query"` or `"mutation"` (REQ-205)

### Webhooks

Expose an external HTTP endpoint as a GraphQL query or mutation. [tool-verified: `provisa/core/models.py` `Webhook` class lines 441–455; `config/provisa.yaml` lines 166–178] (REQ-209)

    webhooks:

      - name: notify_support
        url: http://localhost:9999/notify
        method: POST
        timeout_ms: 3000
        domain_id: sales-analytics
        description: "Send a support notification"
        visible_to: [admin]
        kind: mutation
        arguments:

          - name: message
            type: String

Field | Required | Default | Description
--- | --- | --- | ---
`name` | Yes | — | GraphQL field name
`url` | Yes | — | Webhook endpoint URL
`method` | No | `POST` | HTTP method
`timeout_ms` | No | `5000` | Request timeout in milliseconds
`returns` | No | `null` | Registered table ID, or null for inline type
`inline_return_type` | No | `[]` | List of `{name, type}` fields for custom return shapes (REQ-210)
`arguments` | No | `[]` | List of `{name, type}` argument definitions
`visible_to` | No | `[]` | Roles that can call this webhook
`domain_id` | No | `""` | Domain this webhook belongs to
`description` | No | `null` | GraphQL field description
`kind` | No | `mutation` | `"query"` or `"mutation"`

## Authentication

Auth is configured under the `auth` key. [tool-verified: `provisa/core/models.py` `AuthConfig` class lines 467–477] (REQ-120)

Provider | Description
--- | ---
`none` | No authentication; all requests treated as the `default_role`
`firebase` | Firebase Authentication; requires `project_id` and `service_account_key` (REQ-121)
`keycloak` | Keycloak OIDC (REQ-122)
`oauth` | Generic OAuth 2.0 (REQ-123)
`simple` | Username/password without an external provider (REQ-124)

    auth:
      provider: firebase
      assignments_source: provisa   # "claims" or "provisa"
      default_role: analyst
      default_assignments:

        - role_id: analyst
          domain_id: "*"
      firebase:
        project_id: ${env:FIREBASE_PROJECT_ID}
        service_account_key: ${env:FIREBASE_SERVICE_ACCOUNT_KEY}

`assignments_source: claims` reads role assignments from JWT claims. `assignments_source: provisa` reads them from Provisa's own assignment store. [tool-verified: `provisa/core/models.py` line 476] (REQ-551)

## Execution Routing

Direct execution — Single-source RDBMS queries route to the native driver for sub-100ms latency (REQ-027). Sources require both a `SOURCE_TO_DIALECT` entry and a `SOURCE_TO_CONNECTOR` entry to support this path (REQ-229).

Federated execution — Multi-source queries and sources without a direct driver route through the federation engine (REQ-028). Provisa includes an embedded federation engine; point to your own compatible cluster for large-scale deployments (REQ-226).

Statistics — On registration, Provisa runs `ANALYZE` against each published table to prime the cost-based optimizer (row counts, null fraction, distinct values, min/max). Failures are logged and do not block registration (REQ-275).

## Graph & Semantic Sources

### Neo4j

Register a Neo4j graph database as a queryable source. Stewards author Cypher queries that project scalar values; Provisa caches results and exposes them as GraphQL types (REQ-295).

Cypher queries must use property accessors in the `RETURN` clause (`RETURN n.id AS id, n.name AS name`) — returning node objects is rejected at registration time (REQ-296).

    # Register via admin API (no YAML config required)
    POST /admin/sources/neo4j
    {
      "source_id": "graph",
      "host": "neo4j",
      "port": 7474,
      "database": "neo4j"
    }

    # Register a table (preview + validate before persisting)
    POST /admin/sources/neo4j/graph/tables
    {
      "table_name": "person_skills",
      "cypher": "MATCH (p:Person)-[:HAS_SKILL]->(s:Skill) RETURN p.name AS name, s.skill AS skill, p.experience AS years",
      "ttl": 300
    }

The preview endpoint (`POST /admin/sources/neo4j/{id}/preview`) returns sample rows and blocks registration if the Cypher returns node objects (REQ-296).

### SPARQL

Register any SPARQL 1.1 compliant triplestore (Apache Jena Fuseki, Virtuoso, Stardog, etc.) as a queryable source (REQ-297).

Queries must be `SELECT` queries. Variable names in the `SELECT` clause become column names automatically (REQ-297).

    # Register via admin API
    POST /admin/sources/sparql
    {
      "source_id": "knowledge-graph",
      "endpoint_url": "http://fuseki:3030/ds/sparql",
      "default_graph_uri": "http://example.org/graph"
    }

    # Register a table (executes LIMIT 5 probe to validate and infer columns)
    POST /admin/sources/sparql/knowledge-graph/tables
    {
      "table_name": "product_categories",
      "sparql_query": "SELECT ?product ?label ?category WHERE { ?product a :Product ; rdfs:label ?label ; :hasCategory ?category . }",
      "ttl": 600
    }

Both connectors use the API source cache pipeline — results are stored in PostgreSQL with configurable TTL, making them available for cross-source federated JOINs (REQ-295, REQ-297, REQ-299).

## Connection Examples

### PostgreSQL

    - id: sales-pg
      type: postgresql
      host: postgres
      port: 5432
      database: provisa
      username: provisa
      password: ${env:PG_PASSWORD}

### Snowflake

    - id: analytics-sf
      type: snowflake
      host: org.snowflakecomputing.com
      port: 443
      database: ANALYTICS
      username: svc_provisa
      password: ${env:SNOWFLAKE_PASSWORD}
      federation_hints:
        account: myorg-myaccount
        warehouse: COMPUTE_WH

### Databricks

    - id: lakehouse-db
      type: databricks
      host: my-workspace.azuredatabricks.net
      password: ${env:DATABRICKS_TOKEN}
      federation_hints:
        http_path: /sql/1.0/warehouses/xxxx

### MongoDB

    - id: reviews-mongo
      type: mongodb
      host: mongodb
      port: 27017
      database: provisa
      username: ""
      password: ""

### Cross-Source Query

    {
      orders(where: {region: {eq: "us"}}) {
        id
        amount
        customers {       # PostgreSQL
          name
          email
        }
        productReviews {  # MongoDB (federated)
          rating
          comment
        }
      }
    }

Single-source portions route directly (REQ-027). Cross-source JOINs federate with automatic type coercion (REQ-028, REQ-552).