Metadata-Version: 2.4
Name: overture-serverless
Version: 0.1.0
Summary: Base classes for portable, framework-agnostic serverless Python jobs.
Author: Overture Maps Foundation
License-Expression: MIT
Project-URL: Homepage, https://github.com/OvertureMaps/overture-core
Project-URL: Source, https://github.com/OvertureMaps/overture-core/tree/main/packages/overture_serverless
Project-URL: Issues, https://github.com/OvertureMaps/overture-core/issues
Keywords: overture,overturemaps,serverless,jobs
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: fargate
Requires-Dist: apache-airflow<3,>=2.9; extra == "fargate"
Requires-Dist: apache-airflow-providers-amazon<9,>=8.13.0; extra == "fargate"
Provides-Extra: dev
Requires-Dist: pytest>=9.1.1; extra == "dev"
Requires-Dist: pytest-cov>=7.0.0; extra == "dev"
Requires-Dist: apache-airflow<3,>=2.9; extra == "dev"
Requires-Dist: apache-airflow-providers-amazon<9,>=8.13.0; extra == "dev"
Dynamic: license-file

# overture-serverless

[![PyPI](https://img.shields.io/pypi/v/overture-serverless.svg)](https://pypi.org/project/overture-serverless/)
[![Python versions](https://img.shields.io/pypi/pyversions/overture-serverless.svg)](https://pypi.org/project/overture-serverless/)

Base class for portable, framework-agnostic "job" classes, plus the Airflow-facing backends that launch them. A published package that gets installed at run time wherever a job actually executes, and provides the contract your job subclasses.

It contains:
- `ServerlessPythonJob` — abstract base class your jobs subclass (`execute_job()`), with parameter parsing (`get_param()`) and logging (`log()`). Plain Python, no cloud dependencies — runs identically on any backend or on your laptop.
- `backends.fargate.serverless_python_task_group` — an Airflow `TaskGroup` factory that runs a job on AWS ECS Fargate. Requires the `fargate` extra (`pip install overture-serverless[fargate]`), which pulls in `apache-airflow` and `apache-airflow-providers-amazon`; the base install stays dependency-free since `ServerlessPythonJob` runs inside the job container, not inside Airflow.

## Writing a job

```python
from overture_serverless.job import ServerlessPythonJob


class CollectionJob(ServerlessPythonJob):
    def execute_job(self) -> None:
        path = self.get_param("input_path")
        self.log(f"Processing {path}")
        # ... your logic
```

## Testing locally

```bash
cd packages/overture_serverless
uv run python -c "
from overture_serverless.job import ServerlessPythonJob

class ExampleJob(ServerlessPythonJob):
    def execute_job(self) -> None:
        self.log(self.get_param('input_path'))

ExampleJob().run('{\"input_path\": \"s3://...\"}')
"
```

## Launching a job on Fargate

`serverless_python_task_group` builds an Airflow `TaskGroup` that provisions an ECS Fargate task, runs your job's runner container, and tears the task definition down afterward. It has no opinion on your AWS account's VPC layout, IAM roles, or container registry — you resolve those and pass them in:

```python
from overture_serverless.backends.fargate import serverless_python_task_group

collect = serverless_python_task_group(
    group_id="collection_job",
    module_name="overture_addresses.collect",
    class_name="CollectionJob",
    python_packages="overture-addresses",
    task_role_arn=my_resolved_role_arn,  # e.g. via STS in your own DAG code
    network_config=my_ecs_network_config,  # ECS `networkConfiguration` dict
    image_uri=my_resolved_runner_image_uri,  # e.g. from your own ECR-URI builder
    ecs_task_builder_factory=MyEcsTaskBuilder,  # your register/run/teardown builder
)
```

See the `serverless_python_task_group` docstring for the full parameter list (sizing presets, CodeArtifact coordinates, retries, etc.).

## Publishing

See [`PACKAGE_VERSIONING.md`](../PACKAGE_VERSIONING.md) for how a version bump here turns into a PyPI release.
