Metadata-Version: 2.5
Name: axm-smelt
Version: 0.3.0
Summary: Deterministic token compaction for LLM inputs
Project-URL: Homepage, https://github.com/axm-protocols/axm-smelt
Project-URL: Documentation, https://axm-protocols.github.io/axm-smelt/
Project-URL: Repository, https://github.com/axm-protocols/axm-smelt.git
Project-URL: Issues, https://github.com/axm-protocols/axm-smelt/issues
Author-email: AXM Protocols <gabriel@axm-protocols.io>
License: Apache-2.0
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: axm
Requires-Dist: pydantic>=2.12.5
Requires-Dist: pyyaml>=6.0
Requires-Dist: tiktoken>=0.9
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/axm-protocols/axm-forge/main/assets/logo.png" alt="AXM Logo" width="180" />
</p>

<p align="center">
  <strong>axm-smelt — Deterministic token compaction for LLM inputs</strong>
</p>


<p align="center">
  <a href="https://github.com/axm-protocols/axm-forge/actions/workflows/ci.yml"><img src="https://github.com/axm-protocols/axm-forge/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
  <a href="https://forge.axm-protocols.io/audit/"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/axm-protocols/axm-forge/gh-pages/badges/axm-smelt/axm-audit.json" alt="axm-audit"></a>
  <a href="https://forge.axm-protocols.io/init/"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/axm-protocols/axm-forge/gh-pages/badges/axm-smelt/axm-init.json" alt="axm-init"></a>
  <a href="https://github.com/axm-protocols/axm-forge/actions/workflows/axm-quality.yml"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/axm-protocols/axm-forge/gh-pages/badges/axm-smelt/coverage.json" alt="Coverage"></a>
  <a href="https://pypi.org/project/axm-smelt/"><img src="https://img.shields.io/pypi/v/axm-smelt" alt="PyPI"></a>
  <img src="https://img.shields.io/badge/python-3.12%2B-blue" alt="Python 3.12+">
  <a href="https://forge.axm-protocols.io/smelt/"><img src="https://img.shields.io/badge/docs-live-brightgreen" alt="Docs"></a>
</p>

---

`axm-smelt` reduces token consumption for LLM inputs by applying deterministic compaction strategies — whitespace collapsing, structural transforms, and optional lossy simplifications. It works through the **Python API** and three registered **AXMTools**, which provide MCP, AXM CLI, and DAG-node access from one declaration.

📖 **[Full documentation](https://forge.axm-protocols.io/smelt/)**

## Features

- **Format detection** — auto-detect JSON, YAML, XML, TOML, CSV, Markdown, and plain text
- **Token counting** — always via tiktoken; Claude and unknown models route to the `o200k_base` proxy (approximate, no network)
- **10 strategies** — `minify`, `drop_nulls`, `flatten`, `tabular`, `round_numbers`, `strip_quotes`, `dedup_values_with_refs`, `collapse_whitespace`, `compact_tables`, `strip_html_comments`
- **Composable pipeline** — chain strategies explicitly or use presets (`safe`, `moderate`, `aggressive`)
- **AXMTools** — `smelt`, `smelt_check`, and `smelt_count`, available through MCP, `axm`, and DAG nodes
- **MCP tool** — `SmeltTool` for use by AI agents via `axm-mcp`
- **Modern Python** — 3.12+ with strict typing

## Installation

```bash
uv add axm-smelt
```

## Quick Start

### AXM CLI

The CLI surface is derived from the registered AXMTools. Each command accepts redirected text or a UTF-8 file.

Compact data with `smelt`:

```bash
printf '{"name": "Alice", "notes": null}\n' | axm smelt
# smelt | json | 11->9 tok (-18.18%) | minify | tiktoken
# {"name":"Alice","notes":null}
axm smelt --input-path ./payload.json
```

Analyze possible savings with `smelt_check`:

```bash
printf '{"name": "Alice", "notes": null}\n' | axm smelt_check
# smelt_check | json | 11 tok
#   drop_nulls: -54.55%
#   minify: -18.18%
#   flatten: -18.18%
#   round_numbers: -18.18%
#   strip_quotes: -18.18%
axm smelt_check --input-path ./payload.json
```

Count tokens with `smelt_count`:

```bash
printf 'alpha beta gamma delta epsilon\n' | axm smelt_count
# smelt_count | 6 tokens | 31 chars | o200k_base | tiktoken
axm smelt_count --input-path ./payload.txt
```

Explicit data takes precedence over `--input-path`, which takes precedence over non-interactive stdin. If the designated path does not exist or its contents are not valid UTF-8, the command exits with a non-zero status and a diagnostic that names that path.

There is no standalone `axm-smelt` executable and `python -m axm_smelt` is intentionally unsupported.

### Python API

```python
from axm_smelt import smelt, check, count

# Compact using the safe preset (default)
report = smelt('{\n  "name": "Alice",\n  "age": 30\n}')
print(f"{report.savings_pct:.1f}% saved")
# Tokens: 14 -> 9

# Compact with explicit strategies
report = smelt(data, strategies=["minify", "drop_nulls"])

# Compact with a preset
report = smelt(data, preset="aggressive")

# Analyze without transforming
report = check('{"data": [1, 2, 3]}')
for strat, pct in report.strategy_estimates.items():
    print(f"  {strat}: {pct:.1f}%")

# Count tokens
tokens = count("hello world")
```

### MCP (AI Agent)

`axm-smelt` is available through [`axm-mcp`](https://github.com/axm-protocols/axm-forge/tree/main/packages/axm-mcp). AI agents can call `smelt(data, preset="moderate")`, `smelt_check(...)`, or `smelt_count(...)` directly.

See the [MCP how-to guide](https://forge.axm-protocols.io/smelt/howto/mcp/) for details.

## AXMTool Commands

| Command | Description |
|---|---|
| `axm smelt` | Compact text or structured data |
| `axm smelt_check` | Analyze token waste without transforming the input |
| `axm smelt_count` | Count input tokens from explicit data, a UTF-8 file, or redirected stdin |

These commands come from the `axm.tools` registry; the same definitions power MCP and DAG nodes. Use `--help` for their generated CLI signatures. The removed standalone façade has no compatibility alias.

## Strategies

| Name | Category | Description |
|---|---|---|
| `minify` | whitespace | Compact JSON/YAML/XML whitespace (parse + re-serialize; preserves data) |
| `drop_nulls` | structural | Recursively remove `None`, `""`, `[]`, `{}` values |
| `flatten` | structural | Collapse single-child wrapper dicts (`{"a":{"b":1}}` → `{"a.b":1}`) |
| `tabular` | structural | Convert `list[dict]` JSON to pipe-separated tables |
| `round_numbers` | cosmetic | Round floats to N decimal places (default: 2) |
| `strip_quotes` | cosmetic | Remove quotes on simple alphanumeric JSON keys |
| `dedup_values_with_refs` | structural | Replace repeated long strings (≥20 chars, ≥2 occurrences) with aliases. **Output is wrapped in a `{_refs, _data}` envelope — not format-preserving.** |
| `collapse_whitespace` | whitespace | Collapse consecutive blank lines and strip trailing whitespace on prose/Markdown (skips structured formats and fenced code blocks) |
| `compact_tables` | whitespace | Remove padding whitespace from Markdown table cells (skips fenced code blocks) |
| `strip_html_comments` | cosmetic | Remove `<!-- … -->` HTML comments from Markdown/plain text (skips fenced code blocks) |

## Presets

| Preset | Strategies | Use when |
|---|---|---|
| `safe` | `minify`, `collapse_whitespace` | Data-preserving — keeps the parsed value identical for structured formats (JSON/YAML/…) and never touches fenced code. **Not byte/whitespace-lossless on prose/Markdown**: `collapse_whitespace` collapses blank-line runs and strips trailing whitespace. |
| `moderate` | `minify`, `drop_nulls`, `flatten`, `dedup_values_with_refs`, `tabular`, `strip_quotes`, `collapse_whitespace`, `compact_tables`, `strip_html_comments` | Structural transforms are acceptable |
| `aggressive` | `minify`, `drop_nulls`, `flatten`, `tabular`, `round_numbers`, `dedup_values_with_refs`, `strip_quotes`, `collapse_whitespace`, `compact_tables`, `strip_html_comments` | Maximum savings, may alter float precision |

## Development

This package is part of the [**axm-forge**](https://github.com/axm-protocols/axm-forge) workspace.

```bash
git clone https://github.com/axm-protocols/axm-forge.git
cd axm-forge
uv sync --all-groups
uv run --package axm-smelt --directory packages/axm-smelt pytest -x -q
```

## License

Apache-2.0 — © 2026 axm-protocols
