Metadata-Version: 2.5
Name: autoair
Version: 0.1.0
Summary: Type-safe, extensible no-code DAG builder for Apache Airflow 3
Project-URL: Homepage, https://github.com/ctolon/autoair
Project-URL: Documentation, https://ctolon.github.io/autoair/
Project-URL: Repository, https://github.com/ctolon/autoair
Project-URL: Issues, https://github.com/ctolon/autoair/issues
Project-URL: Changelog, https://github.com/ctolon/autoair/blob/main/CHANGELOG.md
Project-URL: Release notes, https://github.com/ctolon/autoair/releases
Project-URL: Container image, https://github.com/ctolon/autoair/pkgs/container/autoair
Author-email: ctolon <cevatbatuhan.tolon@gmail.com>
Maintainer-email: ctolon <cevatbatuhan.tolon@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: airflow,airflow-plugin,apache-airflow,dag,dag-builder,data-engineering,etl,low-code,mcp,no-code,orchestration,visual-editor,workflow
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Web Environment
Classifier: Framework :: Apache Airflow
Classifier: Framework :: FastAPI
Classifier: Framework :: Pydantic :: 2
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: JavaScript
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: System :: Distributed Computing
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: cryptography>=42
Requires-Dist: fastapi>=0.115
Requires-Dist: fsspec>=2024.6
Requires-Dist: httpx2>=2.12
Requires-Dist: jedi>=0.19
Requires-Dist: jinja2>=3.1
Requires-Dist: pillow>=10
Requires-Dist: prometheus-client>=0.20
Requires-Dist: pydantic-settings>=2.7
Requires-Dist: pydantic>=2.11
Requires-Dist: pyyaml>=6.0
Requires-Dist: typer>=0.16
Provides-Extra: ai
Requires-Dist: pydantic-ai-slim<3,>=2.43; extra == 'ai'
Provides-Extra: ai-all
Requires-Dist: pydantic-ai-slim[anthropic,bedrock,cohere,google,groq,mistral,openai]<3,>=2.43; extra == 'ai-all'
Provides-Extra: ai-anthropic
Requires-Dist: pydantic-ai-slim[anthropic]<3,>=2.43; extra == 'ai-anthropic'
Provides-Extra: ai-bedrock
Requires-Dist: pydantic-ai-slim[bedrock]<3,>=2.43; extra == 'ai-bedrock'
Provides-Extra: ai-cohere
Requires-Dist: pydantic-ai-slim[cohere]<3,>=2.43; extra == 'ai-cohere'
Provides-Extra: ai-google
Requires-Dist: pydantic-ai-slim[google]<3,>=2.43; extra == 'ai-google'
Provides-Extra: ai-groq
Requires-Dist: pydantic-ai-slim[groq]<3,>=2.43; extra == 'ai-groq'
Provides-Extra: ai-guardrails
Requires-Dist: llm-guard>=0.3; extra == 'ai-guardrails'
Provides-Extra: ai-mistral
Requires-Dist: pydantic-ai-slim[mistral]<3,>=2.43; extra == 'ai-mistral'
Provides-Extra: ai-ollama
Requires-Dist: pydantic-ai-slim[openai]<3,>=2.43; extra == 'ai-ollama'
Provides-Extra: ai-openai
Requires-Dist: pydantic-ai-slim[openai]<3,>=2.43; extra == 'ai-openai'
Provides-Extra: ai-rerank
Requires-Dist: sentence-transformers>=3.0; extra == 'ai-rerank'
Provides-Extra: airflow
Requires-Dist: apache-airflow-providers-standard>=1.0; extra == 'airflow'
Requires-Dist: apache-airflow<4,>=3.0; extra == 'airflow'
Provides-Extra: all
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.29; extra == 'all'
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.50b0; extra == 'all'
Requires-Dist: opentelemetry-sdk>=1.29; extra == 'all'
Requires-Dist: pycrdt-websocket>=0.16; extra == 'all'
Requires-Dist: pycrdt>=0.12; extra == 'all'
Requires-Dist: pydantic-ai-slim<3,>=2.43; extra == 'all'
Requires-Dist: pydantic-ai-slim[anthropic]<3,>=2.43; extra == 'all'
Requires-Dist: pydantic-ai-slim[openai]<3,>=2.43; extra == 'all'
Requires-Dist: pyyaml>=6.0; extra == 'all'
Requires-Dist: ruff>=0.9; extra == 'all'
Requires-Dist: shellcheck-py>=0.10; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.30; extra == 'all'
Provides-Extra: collab
Requires-Dist: pycrdt-websocket>=0.16; extra == 'collab'
Requires-Dist: pycrdt>=0.12; extra == 'collab'
Provides-Extra: collab-redis
Requires-Dist: redis>=5.0; extra == 'collab-redis'
Provides-Extra: format
Requires-Dist: ruff>=0.9; extra == 'format'
Provides-Extra: git
Provides-Extra: kb-elasticsearch
Provides-Extra: kb-pgvector
Requires-Dist: psycopg[binary]>=3.2; extra == 'kb-pgvector'
Requires-Dist: sqlalchemy>=1.4.49; extra == 'kb-pgvector'
Provides-Extra: lsp
Requires-Dist: python-lsp-ruff>=2.2; extra == 'lsp'
Requires-Dist: python-lsp-server>=1.12; extra == 'lsp'
Provides-Extra: lsp-pyright
Requires-Dist: basedpyright>=1.20; extra == 'lsp-pyright'
Provides-Extra: mcp
Requires-Dist: fastmcp>=4.0; extra == 'mcp'
Requires-Dist: mcp>=2.3; extra == 'mcp'
Provides-Extra: otel
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.29; extra == 'otel'
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.50b0; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.29; extra == 'otel'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.2; extra == 'postgres'
Requires-Dist: sqlalchemy>=1.4.49; extra == 'postgres'
Provides-Extra: shell
Requires-Dist: shellcheck-py>=0.10; extra == 'shell'
Provides-Extra: sqlite
Requires-Dist: sqlalchemy>=1.4.49; extra == 'sqlite'
Provides-Extra: standalone
Requires-Dist: uvicorn[standard]>=0.30; extra == 'standalone'
Provides-Extra: yaml
Requires-Dist: pyyaml>=6.0; extra == 'yaml'
Description-Content-Type: text/markdown

# AutoAir: a no-code DAG builder for Apache Airflow 3

AutoAir lets you design Airflow DAGs on a visual canvas and produces **readable, type-checked
Python** (or a JSON spec with a runtime loader, an Airflow 3.4 native YAML Dag, or
dag-factory YAML). It validates with Airflow's own `DagBag`, deploys to a **local folder, object
storage or Git (direct push or pull requests)**, and lets you trigger, debug and version DAGs
from the Airflow UI. AI agents can use the same operations through AutoAir's **MCP server**.

* **Airflow 3.0 or later, below 4.** CI covers 3.0.6, 3.1.8, 3.2.2 and 3.3.2, plus the 3.4.0b1
  pre-release, on Python 3.10–3.14. Generated imports follow the target Airflow version (for
  example, timetables come from `airflow.timetables.*` on 3.0/3.1 and from `airflow.sdk` on 3.2+).
  The `mcp` extra cannot be installed next to Airflow 3.1 (FastMCP 4 needs Starlette 1.x); see
  the [compatibility matrix](https://ctolon.github.io/autoair/docs/installation/compatibility).
* Runs **inside Airflow** as a plugin (Browse → *DAG Builder* and an *AutoAir* tab on every DAG
  page on 3.1+; a full page at `/autoair/ui/` on 3.0) **or standalone** (`autoair serve`).

## Install

```bash
pip install "autoair[airflow]"      # into the Airflow environment: the plugin loads automatically
pip install "autoair[standalone]"   # standalone server: autoair serve --airflow-url http://airflow:8080
uvx --from 'autoair[standalone,airflow]' autoair local --open   # try it on a laptop, no Airflow server
```

AI is optional and runs on [pydantic-ai](https://ai.pydantic.dev) (like Airflow's common.ai
provider): add `ai` plus a provider extra, e.g. `pip install "autoair[ai,ai-anthropic]"`
(`ai-openai`, `ai-google`, `ai-ollama`, `ai-bedrock`, `ai-groq`, `ai-mistral`, `ai-cohere`,
`ai-all`). Other optional extras: `git`, `mcp`, `collab`, `collab-redis`, `sqlite`, `postgres`,
`otel`, `format`, `shell`, `lsp`, `lsp-pyright`, `ai-rerank`, `ai-guardrails`, and `all` for the
common set. The container image
`ghcr.io/ctolon/autoair` is the official Airflow image with AutoAir installed. See
[deployment](https://github.com/ctolon/autoair/blob/main/docs/deployment.md).

## Features

**Canvas.** React Flow editor with integration logos on nodes (S3, Jira, dbt, BigQuery and more,
plus custom icon upload), notes, task groups, insert-on-edge, a command palette (⌘/Ctrl+K),
context menus, copy/paste with id and XCom remapping, alignment guides, undo/redo, ELK
auto-layout, run-state and version-diff overlays, and collapsible panels (⌘/Ctrl+B, I, J).
English and Turkish UI, light and dark themes.

**Authoring.** Every installed operator, sensor and transfer, from official and third-party
providers, plugins, DAG folders and packages, through a static AST catalog that does not import
the modules. Also:
- TaskFlow tasks (`@task`, `@task.branch`, `short_circuit`, `virtualenv`, `sensor`, `bash`,
  `docker`, `kubernetes`, `@task.stub`), dynamic mapping (`expand`, `expand_kwargs`,
  `map_index_template`), task loops (3.4), setup/teardown and trigger rules.
- Callbacks and notifiers, retry policies (3.3), deadlines (3.1), human-in-the-loop operators
  with a form designer, OpenLineage toggles and disabled tasks.
- DAG params with a live trigger-form preview, and environment overlays (dev/staging/prod).
- Parameters that take a callable accept an import path or a function written in place. Script
  parameters (Bash, Spark, SQL and others) accept an embedded script file that is written to
  every worker at parse time.
- Hard-coded passwords and tokens are flagged with a hint to use Connections or Variables.

**Scheduling.** DAG time zone (DST aware), cron, presets, intervals, multi-cron, event dates,
asset expressions, event-driven assets (AssetWatcher with SQS, Kafka, Pub/Sub, Redis, Service Bus
or Kinesis), assets-or-time, assets-and-time (3.4) and partitioned schedules (3.2), with a "next
runs" preview computed by Airflow's timetable engine.

**Editors.** Jinja-aware CodeMirror fields with completion for context, params, variables,
connections, macros and `ti.xcom_pull`; drag-to-map upstream XComs; connection pickers filtered
by type, with inline connection creation and testing; Python and shell editors with jedi
completion, hover and signature help for the installed Airflow and providers, live ruff lint and
ShellCheck diagnostics. Operator docstrings render like an IDE hover. Every field shows its Python
type, whether it is required, and its description; clicking a problem jumps to the field.

**Lifecycle.**
- Live validation and code preview (Python, Airflow YAML, dag-factory, spec JSON), backed by a
  warm DagBag import-check pool.
- Test a single task or the whole DAG without the metadata database.
- Deploy with ETag optimistic locking and a three-way semantic merge dialog, with protection for
  hand-edited (ejected) files.
- Version history with a visual graph diff.
- Git pull-request mode (GitHub, GitLab, Bitbucket, Azure DevOps, Gitea, or any remote through
  the git CLI) with environment promotion.
- A deploy-target dialog that discovers where Airflow reads DAGs (DAG folder, GitDagBundles, a
  git-sync checkout).
- Trigger, backfill (dry run), clear, and pause or unpause from the builder; a run overlay with
  logs and XComs.
- Templates, reusable fragments, and import from hand-written Python DAGs, Airflow and YAML.
- An audit trail.

**Collaboration and access.** Live co-editing (Yjs) with presence, remote cursors and per-user
undo; WebSocket access through short-lived tickets that require DAG edit permission. IAM on top
of Airflow's auth manager with four-eyes approvals ([IAM](https://github.com/ctolon/autoair/blob/main/docs/iam.md)). Signed (Ed25519) template
marketplaces (`autoair marketplace keygen|build|verify`).

**AI and MCP.** Generate a DAG from a prompt, edit with AI (reviewed as a diff), explain and fix
problems, write docs and troubleshoot runs, using Anthropic, OpenAI or OpenAI-compatible servers,
Gemini or Ollama, with a central or per-user model ([AI architecture](https://github.com/ctolon/autoair/blob/main/docs/ai-architecture.md)).
The MCP server exposes authoring tools to Claude Code, Claude Desktop and other agents over stdio
or streamable HTTP ([MCP](https://github.com/ctolon/autoair/blob/main/docs/mcp.md)).

**Operations.** `/api/health`, `/api/ready` and Prometheus `/api/metrics` (under `/autoair` in the
plugin), optional OpenTelemetry tracing, organisation policy rules ([policy](https://github.com/ctolon/autoair/blob/main/docs/policy.md)), state
in files, SQLite or PostgreSQL, a container image with a precomputed catalog, and Helm values for
the official Airflow chart.

## Architecture

```
React 19 + @xyflow/react ── typed client (hey-api, generated from OpenAPI) ──► FastAPI (AutoAir API, MCP)
  zustand + zundo · RJSF · CodeMirror · i18next                                 │
                                                                               ├─ spec/        Pydantic v2 DagSpec + migrations (single source of truth)
                                                                               ├─ catalog/     static AST provider index (+ subprocess fallback, disk cache)
                                                                               ├─ codegen/     spec → AST → ruff · YAML (AIP-85) · dag-factory · version-aware imports
                                                                               ├─ loader/      the same AST compiled at parse time (+ environment overlays)
                                                                               ├─ validation/  graph · catalog · Jinja · policy · version gates · warm DagBag pool
                                                                               ├─ storage/     local · object (fsspec) · git (CLI, GitHub, GitLab, Bitbucket, Azure DevOps, Gitea)
                                                                               ├─ runner/      task and DAG test runs without the metadata DB, in a sandbox process
                                                                               ├─ importer/    Python DAG → spec, bulk import, code sync
                                                                               ├─ iam/         roles and permissions on top of Airflow's auth manager
                                                                               ├─ ai/ · mcp/   AI assistants and the MCP server
                                                                               ├─ collab · lsp Yjs rooms (ticket auth) · language services
                                                                               └─ gateway/     Airflow REST API v2 (caller's token forwarded → Airflow RBAC)
```

## Development

```bash
uv sync && (cd frontend && npm ci && npm run build)
./scripts/dev-airflow.sh   # Airflow 3 + plugin at http://localhost:8080 → Browse → DAG Builder
# Git mode: AUTOAIR_DEV_GIT_REPO=file:///path/to/bare.git ./scripts/dev-airflow.sh  (adds a GitDagBundle)
uv run autoair serve --dags-folder /path/to/dags --airflow-url http://localhost:8080   # standalone
uv run autoair mcp         # MCP server (stdio)
```

```bash
make check     # ruff, mypy --strict, hard-coded audit, config reference, tsc, eslint, prettier
make test      # pytest (unit + real-Airflow integration) and vitest
make matrix    # nox: Airflow 3.0 / 3.1 / 3.2 / 3.3, the 3.4 pre-release, Python 3.10
make e2e       # Playwright inside the Airflow UI
make crawl     # UI crawler: opens every menu, dialog, tab and node inspector; any error fails
make codegen   # regenerate the TypeScript client after API changes
```

See [CONTRIBUTING.md](https://github.com/ctolon/autoair/blob/main/CONTRIBUTING.md) for the full workflow. `autoair --help` lists the CLI
commands: `serve`, `local`, `generate`, `validate`, `mcp`, `mcp-serve`, `ai-serve`,
`import-python`, `import-yaml`, `export-yaml`, `export-dag-factory`, `export-schema`,
`export-openapi`, `new-node`, `catalog`, `marketplace`, `iam`, `config`, `kb` and `db`.

## Documentation

User documentation (installation, user guide, administration, AI, extending):
https://ctolon.github.io/autoair/ (source in [`website/`](https://github.com/ctolon/autoair/blob/main/website/README.md); `make docs-serve`).

In-repo reference: [configuration](https://github.com/ctolon/autoair/blob/main/docs/configuration.md) ·
[configuration reference](https://github.com/ctolon/autoair/blob/main/docs/configuration-reference.md) · [deployment](https://github.com/ctolon/autoair/blob/main/docs/deployment.md) ·
[IAM](https://github.com/ctolon/autoair/blob/main/docs/iam.md) · [SSO with Airflow](https://github.com/ctolon/autoair/blob/main/docs/sso.md) · [policy](https://github.com/ctolon/autoair/blob/main/docs/policy.md) ·
[AI architecture](https://github.com/ctolon/autoair/blob/main/docs/ai-architecture.md) · [AI assistants](https://github.com/ctolon/autoair/blob/main/docs/ai-assistants.md) ·
[AI troubleshooting](https://github.com/ctolon/autoair/blob/main/docs/ai-troubleshooting.md) · [state and scaling](https://github.com/ctolon/autoair/blob/main/docs/architecture/state-and-scaling.md) ·
[MCP](https://github.com/ctolon/autoair/blob/main/docs/mcp.md) · [extending](https://github.com/ctolon/autoair/blob/main/docs/extending.md) · [roadmap](https://github.com/ctolon/autoair/blob/main/docs/ROADMAP.md) ·
[changelog](https://github.com/ctolon/autoair/blob/main/CHANGELOG.md)

## License

Apache License 2.0. See [LICENSE](https://github.com/ctolon/autoair/blob/main/LICENSE) and [NOTICE](https://github.com/ctolon/autoair/blob/main/NOTICE). Security issues: see
[SECURITY.md](https://github.com/ctolon/autoair/blob/main/SECURITY.md).
