Metadata-Version: 2.4
Name: ga-parallel
Version: 0.1.1
Summary: Pluggable parallel task execution helpers (threading, joblib, dask, ray).
Author: Andrea Gemma
License-Expression: MIT
Project-URL: Documentation, https://github.com/andreagemma/parallel#readme
Project-URL: Issues, https://github.com/andreagemma/parallel/issues
Project-URL: Source, https://github.com/andreagemma/parallel
Keywords: parallel,concurrency,dask,ray,joblib,multiprocessing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Distributed Computing
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ga-serializer>=0.1.0
Provides-Extra: dask
Requires-Dist: dask>=2026.1; extra == "dask"
Requires-Dist: distributed>=2026.1; extra == "dask"
Provides-Extra: ray
Requires-Dist: ray>=2.9; extra == "ray"
Provides-Extra: joblib
Requires-Dist: joblib>=1.4; extra == "joblib"
Provides-Extra: all
Requires-Dist: dask[distributed]>=2024.1; extra == "all"
Requires-Dist: distributed>=2024.1; extra == "all"
Requires-Dist: ray>=2.9; extra == "all"
Requires-Dist: joblib>=1.4; extra == "all"
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == "test"
Requires-Dist: pytest-cov>=5.0; extra == "test"
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: twine>=5.1; extra == "dev"
Dynamic: license-file

# ga-parallel

[![PyPI](https://img.shields.io/pypi/v/ga-parallel.svg)](https://pypi.org/project/ga-parallel/)
[![Python](https://img.shields.io/pypi/pyversions/ga-parallel.svg)](https://pypi.org/project/ga-parallel/)

`parallel` provides a single `Parallel` helper class that runs a function over
a collection of tasks using a selectable execution engine: plain threading,
[joblib](https://joblib.readthedocs.io), [dask](https://www.dask.org/), or
[ray](https://www.ray.io/). The engine is chosen at runtime, so the same code
can scale from a laptop to a cluster without changes.

The PyPI distribution is named `ga-parallel`; the import package is named
`parallel`.

## Installation

```bash
python -m pip install ga-parallel
```

`dask`, `ray`, and `joblib` are optional third-party engines and are not
installed by default. Install only the engines you need as extras:

```bash
python -m pip install "ga-parallel[dask]"
python -m pip install "ga-parallel[ray]"
python -m pip install "ga-parallel[joblib]"
```

Install every optional engine at once with:

```bash
python -m pip install "ga-parallel[all]"
```

Development and test tools are available as extras:

```bash
python -m pip install -e ".[test]"
python -m pip install -e ".[dev]"
```

## Quick Start

```python
from parallel import Engine, Parallel


def double(tasks: list[dict]) -> list[int]:
    return [task["value"] * 2 for task in tasks]


tasks = [{"value": i} for i in range(10)]

runtime = Parallel(num_cpus=2, engine=Engine.MULTITHREADING)
results = runtime.map_list(double, tasks)

runtime.shutdown()
```

`engine` may also be passed as a string, for example `"threading"` or `"ray"`.
The library normalizes it via `Engine.parse(...)`.

For scoped execution, use the runtime as a context manager:

```python
with Parallel(num_cpus=2, engine=Engine.MULTITHREADING) as runtime:
    results = runtime.map_list(double, tasks)
```

## Engines

Each `Parallel` instance owns its engine state and supports the following
engines, selected via the `engine` argument of the constructor, `configure(...)`
or `map(...)`:

- `Engine.NONE` - sequential execution (default with a single CPU).
- `Engine.MULTITHREADING` - `concurrent.futures.ThreadPoolExecutor`.
- `Engine.JOBLIB` - `joblib.Parallel` with the `loky` backend
  (requires the `joblib` extra).
- `Engine.DASK` - a local Dask `distributed` cluster with
  multi-process workers (requires the `dask` extra).
- `Engine.DASK_MULTITHREADING` - a local Dask `distributed` client
  with a single multi-threaded worker (requires the `dask` extra).
- `Engine.RAY` - a local or remote Ray cluster (requires the `ray` extra).

If an engine's dependency is missing, `Parallel` falls back to
`Engine.MULTITHREADING` and logs a warning.

## API Overview

- `Parallel(num_cpus=None, engine=None, log=None, **kwargs)` creates and
  configures an independent runtime instance.
- `Parallel.configure(num_cpus=None, engine=None, log=None, **kwargs)`
  configures that instance and returns the number of usable CPUs.
- `Parallel.map(fn, tasks, engine=None, n_workers=None, chunk_size=None, **kwargs)`
  splits `tasks` into chunks and yields the result of `fn` for each chunk.
- `Parallel.map_list(fn, tasks, engine=None, n_workers=None, chunk_size=None,
  **kwargs)` eagerly executes `map` and returns a flat list of results.
- `Parallel.apply(fn, params, engine=None, **kwargs)` runs `fn` once with a
  single task payload.
- `Parallel.session(...)` creates a configured instance suitable for a
  `with` block.
- `Parallel.shutdown(force=False)` releases the resources owned by that
  instance.
