Metadata-Version: 2.4
Name: gigq
Version: 0.5.1
Summary: A lightweight job queue system with SQLite backend and zero dependencies
Author-email: GigQ Team <info@gigq.dev>
License: MIT License
        
        Copyright (c) 2025 GigQ Authors
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/kpouianou/gigq
Project-URL: Documentation, https://kpouianou.github.io/GigQ/
Project-URL: Issues, https://github.com/kpouianou/gigq/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: examples
Requires-Dist: requests; extra == "examples"
Requires-Dist: pandas; extra == "examples"
Requires-Dist: schedule; extra == "examples"
Requires-Dist: scikit-learn; extra == "examples"
Provides-Extra: docs
Requires-Dist: mkdocs-material; extra == "docs"
Requires-Dist: pymdown-extensions; extra == "docs"
Requires-Dist: mkdocstrings[python]; extra == "docs"
Requires-Dist: mkdocs-git-revision-date-localized-plugin; extra == "docs"
Requires-Dist: mkdocs-minify-plugin; extra == "docs"
Requires-Dist: mike; extra == "docs"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: coverage; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Provides-Extra: all
Requires-Dist: gigq[dev,docs,examples]; extra == "all"
Dynamic: license-file

<h1 align="center">
  <span style="color: #4f81e6;">Gig</span><span style="color: #60cdff;">Q</span>
</h1>
<p align="center">Lightweight SQLite Job Queue</p>

<p align="center">
  <a href="https://pypi.org/project/gigq/"><img alt="PyPI" src="https://img.shields.io/pypi/v/gigq.svg?style=flat-square"></a>
  <a href="https://pypi.org/project/gigq/"><img alt="Python Versions" src="https://img.shields.io/pypi/pyversions/gigq.svg?style=flat-square"></a>
  <a href="https://github.com/kpouianou/gigq/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/github/license/kpouianou/gigq?style=flat-square"></a>
  <a href="https://github.com/kpouianou/gigq/actions/workflows/ci.yml"><img alt="Build Status" src="https://img.shields.io/github/actions/workflow/status/kpouianou/gigq/ci.yml?branch=main&style=flat-square"></a>
</p>

GigQ is a Python job queue backed by SQLite. It fits teams and projects that have outgrown a raw `for` loop with `try`/`except`, but don't want to run Redis or any other broker. Define functions, enqueue them, and run one or more workers on any machine that can reach the database file.

```python
from gigq import task, JobQueue, Worker

@task()
def greet(name="world"):
    return f"Hello, {name}!"

queue = JobQueue("jobs.db")
greet.submit(queue, name="Alice")
Worker("jobs.db").start()
```

You get **retries with backoff**, **crash recovery** (stuck work is reclaimed), and **queryable status and results** from the same DB the workers use.

## When to use GigQ

**Good fit**

- Local or single-server automation, ETL, scraping batches, ML prep, or any scriptable work you want off the request path
- A few concurrent worker threads or processes against one SQLite file
- You are fine storing queue state in a file next to your app

**Not the right tool**

- Multi-datacenter orchestration, strict global ordering, or millions of jobs per second
- When you already run Redis/Kafka/SQS and need their ecosystem

## Workflows and `parent_results`

Wire `@task` functions into a `Workflow`, declare dependencies, and submit once. Dependent tasks can declare a `parent_results` argument; GigQ injects a dict of **parent job id → deserialized result** so fan-out and fan-in steps can pass data without going through job parameters.

```python
from gigq import task, JobQueue, Workflow, Worker

@task()
def source():
    return {"items": [1, 2, 3]}

@task()
def branch_a(parent_results):
    n = next(iter(parent_results.values()))["items"]
    return {"branch": "a", "len": len(n)}

@task()
def branch_b(parent_results):
    n = next(iter(parent_results.values()))["items"]
    return {"branch": "b", "len": len(n)}

@task()
def merge(parent_results):
    return {"combined": list(parent_results.values())}

queue = JobQueue("jobs.db")
wf = Workflow("fan")
s = wf.add_task(source)
a = wf.add_task(branch_a, depends_on=[s])
b = wf.add_task(branch_b, depends_on=[s])
wf.add_task(merge, depends_on=[a, b])
wf.submit_all(queue)
Worker("jobs.db").start()
```

## Installation

```bash
pip install gigq
```

## CLI

The `gigq` command uses `--db` (default `gigq.db`) and a subcommand:

| Command                          | Purpose                                                  |
| -------------------------------- | -------------------------------------------------------- |
| `gigq worker`                    | Run a worker; add `--concurrency N` for threaded workers |
| `gigq list`                      | List jobs; optional `--status pending` (etc.)            |
| `gigq status <id> --show-result` | Inspect a job and its result                             |
| `gigq stats`                     | Aggregate counts by status                               |
| `gigq submit`                    | Enqueue by import path `module.function`                 |

## How it works

Jobs, dependencies, and results live in **SQLite** tables (`jobs`, `job_executions`). Workers claim work in **transactions** so only one worker runs a given job at a time; multiple threads or processes coordinate through the database file (WAL mode is enabled for less contention). Retries and timeouts are enforced in the worker loop.

## Examples

| Example                                                                  | Description                                                                |
| ------------------------------------------------------------------------ | -------------------------------------------------------------------------- |
| [`examples/parallel_tasks.py`](examples/parallel_tasks.py)               | Fan-out workers plus a fan-in task using `parent_results`                  |
| [`examples/data_pipeline.py`](examples/data_pipeline.py)                 | Linear pipeline: generate → transform → format via `parent_results`        |
| [`examples/hyperparameter_tuning.py`](examples/hyperparameter_tuning.py) | scikit-learn search with sequential vs parallel workers and crash recovery |

## Integrations

- **[`mcp_server/`](mcp_server/)** — **Model Context Protocol** server (`gigq-mcp`) so agents can submit jobs, inspect queues, and read results. See the [MCP integration docs](https://kpouianou.github.io/GigQ/integrations/mcp/).

## Documentation

Full guides and API reference: **[https://kpouianou.github.io/GigQ/](https://kpouianou.github.io/GigQ/)**

## License

This project is licensed under the MIT License — see [LICENSE](LICENSE).
