Metadata-Version: 2.5
Name: cabsallm
Version: 0.1.1
Summary: Configurable LLM annotation with adaptive batching and BOHB-style tuning.
Project-URL: Homepage, https://aclanthology.org/2026.acl-short.51/
Project-URL: Documentation, https://aclanthology.org/2026.acl-short.51/
Author: CodeEditsCues contributors
License: MIT
License-File: LICENSE
Keywords: adaptive-batching,annotation,bohb,llm,optimization
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.10
Requires-Dist: openai>=1.0
Requires-Dist: openpyxl>=3.1
Requires-Dist: pandas>=2.0
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: all-providers
Requires-Dist: anthropic>=0.40; extra == 'all-providers'
Requires-Dist: google-genai>=1.0; extra == 'all-providers'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: google-genai>=1.0; extra == 'gemini'
Provides-Extra: tokens
Requires-Dist: tiktoken>=0.7; extra == 'tokens'
Description-Content-Type: text/markdown

# CaBSALLM

**Efficient, context-aware LLM annotation for conversational and tabular text data.** CaBSALLM lets researchers configure a dataset, text and group columns, annotation prompt, provider/model, batch controller, and optimization strategy without editing pipeline code.

It supports resumable, structured annotation for CSV, TSV, Excel, JSON, and JSONL inputs, with provider presets for OpenAI, Claude, Gemini, Qwen, Kimi, and OpenAI-compatible endpoints.

## Install

```powershell
pip install cabsallm
```

Install only the provider extras you need:

```powershell
pip install "cabsallm[anthropic]"
pip install "cabsallm[gemini]"
pip install "cabsallm[all-providers]"
```

The command-line tools are `cabsallm` and `cuebatch`. The Python import is `codeedit_cues`.

## Quick start

1. Inspect a dataset and create a configuration template:

   ```powershell
   cabsallm inspect messages.csv
   cabsallm init cabsallm.yaml
   ```

2. Put the provider credential in a local `.env` file beside the configuration. Never commit this file.

   ```dotenv
   OPENAI_API_KEY=<set-this-locally>
   ```

3. Edit `cabsallm.yaml` to identify the data, columns, prompt, model, and output path. Run a no-cost validation first, then a small pilot:

   ```powershell
   cabsallm run cabsallm.yaml --dry-run
   cabsallm run cabsallm.yaml --max-rows 50
   ```

4. Run the full job after checking the pilot output:

   ```powershell
   cabsallm run cabsallm.yaml
   ```

CaBSALLM writes annotations, a resumable progress state, run events, and a run summary to the configured output directory.

## Configuration

This minimal configuration annotates a text column while preserving each row's identifier and conversation/group context:

```yaml
input_path: data/messages.csv
output_dir: runs/my-study

columns:
  id: message_id
  text: message_text
  group: conversation_id

output:
  label_column: annotation
  rationale_column: annotation_rationale

prompt:
  system: prompts/system.txt
  user_template: prompts/user.txt

llm:
  provider: openai
  model: gpt-5.4
  api_key_env: OPENAI_API_KEY
  reasoning_effort: medium
  temperature: 0.0

batch:
  strategy: hybrid
  initial_size: 8
  min_size: 1
  max_size: 32

optimization:
  enabled: false
```

Prompts should request structured JSON and explicitly require the original row identifiers. CaBSALLM rejects malformed responses and records recoverable failures rather than silently assigning labels to the wrong rows.

## Providers and credentials

Credentials may be supplied through an environment variable, a local `.env` file, or application code. Keep keys out of YAML, prompts, notebooks, outputs, and version control.

| Provider preset | Example model | Default key variable | Notes |
| --- | --- | --- | --- |
| `openai` | `gpt-5.4` | `OPENAI_API_KEY` | Supports `reasoning_effort`. |
| `anthropic` | `claude-sonnet-4-5` | `ANTHROPIC_API_KEY` | Install `cabsallm[anthropic]`. |
| `gemini` | `gemini-2.5-pro` | `GEMINI_API_KEY` | Install `cabsallm[gemini]`; supports `thinking_budget`. |
| `qwen` | `qwen-plus` | `DASHSCOPE_API_KEY` | Use its OpenAI-compatible endpoint when required. |
| `kimi` | `moonshot-v1-8k` | `MOONSHOT_API_KEY` | Use its OpenAI-compatible endpoint when required. |
| `openai_compatible` | provider-specific | your choice | Set `base_url` and `api_key_env`. |

Example for a Qwen-compatible endpoint:

```yaml
llm:
  provider: qwen
  model: qwen-plus
  api_key_env: DASHSCOPE_API_KEY
  base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
```

Use `reasoning_effort: low`, `medium`, or `high` when supported by the selected model. For Gemini, use `thinking_budget` when that model supports a controllable thinking budget. Provider-specific controls are forwarded only when compatible with the selected preset.

## Batch control and comfortable progress reporting

Choose the controller that fits the reliability and cost profile of the task:

| Strategy | Use it when |
| --- | --- |
| `fixed` | You need a constant, known batch size. |
| `aimd` | You want conservative additive growth and quick backoff after failures. |
| `ewma` | You want batch size to follow a smoothed latency/error signal. |
| `hybrid` | You want adaptive control with safety limits; this is the usual default. |

Make long runs easier to supervise:

```yaml
reporting:
  verbosity: detailed       # quiet, normal, or detailed
  show_eta: true
  show_batch_metrics: true
  show_token_estimates: true
  show_controller_updates: true
  show_error_details: true
  write_events: true
  write_partials: true
  update_every_batches: 1
```

Set `verbosity: quiet` for unattended runs, `normal` for concise checkpoints, or `detailed` to include batch sizes, controller decisions, latency, errors, estimates, and ETA.

## Tune the batch controller

BOHB is the default tuning method. Tuning always requires an explicit list of hyperparameters, so the search space is visible and reproducible. Other available methods are `successive_halving`, `random`, `grid`, and `greedy`.

```yaml
optimization:
  enabled: true
  method: bohb
  objective: cost_adjusted_throughput
  budget: 24
  hyperparameters:
    - name: batch.initial_size
      type: int
      min: 2
      max: 16
    - name: batch.max_size
      type: int
      min: 16
      max: 64
    - name: batch.strategy
      type: categorical
      values: [aimd, ewma, hybrid]
```

Run tuning with:

```powershell
cabsallm tune cabsallm.yaml
```

The selected configuration, trial history, metrics, and recommendation are saved beneath the tuning output directory. Review the recommendation before using it for a full production run.

## Python API and command examples

Run an annotation project from Python:

```python
from codeedit_cues import AnnotationRunner, load_config

config = load_config("cabsallm.yaml")
runner = AnnotationRunner(config)
summary = runner.run(max_rows=50)
print(summary)
```

Run tuning from Python:

```python
from codeedit_cues import load_config
from codeedit_cues.tuning import Tuner

config = load_config("cabsallm.yaml")
result = Tuner(config).tune()
print(result.best_config)
```

Runnable scripts are included in the source distribution under `examples/scripts/`:

```powershell
python examples/scripts/create_project.py
python examples/scripts/inspect_data.py data/messages.csv
python examples/scripts/dry_run.py cabsallm.yaml
python examples/scripts/run_annotation.py cabsallm.yaml --max-rows 50
python examples/scripts/tune.py cabsallm.yaml
python examples/scripts/cite_paper.py --style bibtex
```

Use `cabsallm --help` or `cabsallm <command> --help` for every command and option.

## Citation

CaBSALLM is built on:

> Abolhasani Mohammadsadegh, Reza Mousavi, and Paul Jen-Hwa Hu. 2026. *CaBSALLM: Efficient Context-Aware Batch Annotation of Conversational Streams with Large Language Models.* In *Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)*, pages 615–636. Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.acl-short.51

Print a ready-to-use citation from the command line:

```powershell
cabsallm cite --style acl
cabsallm cite --style bibtex
cabsallm cite --style markdown
cabsallm cite --style doi
```

Or from Python:

```python
from codeedit_cues import citation

print(citation("bibtex"))
```

Read the paper at the [ACL Anthology](https://aclanthology.org/2026.acl-short.51/).

## Responsible use

Pilot prompts and models before a full run. Inspect samples for systematic errors, use an appropriate human-review process for consequential labels, and follow the data-use, privacy, and provider requirements that apply to your study.
