Metadata-Version: 2.4
Name: codetool-shell
Version: 0.1.1
Summary: Compress shell output text with Python and optional Rust backends
Project-URL: Homepage, https://github.com/pbi-agent/codetool-shell
Project-URL: Repository, https://github.com/pbi-agent/codetool-shell
Project-URL: Issues, https://github.com/pbi-agent/codetool-shell/issues
Project-URL: Changelog, https://github.com/pbi-agent/codetool-shell/releases
Author-email: drod <naceur.bs@gmail.com>
Maintainer-email: drod <naceur.bs@gmail.com>
Keywords: agent,compression,developer-tools,llm,rust,shell,terminal
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Shells
Classifier: Topic :: Text Processing
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# codetool-shell

Python interface library for compressing plain shell-output text before it is
returned to an LLM/tool consumer.

```python
from codetool_shell import compress_text

compressed = compress_text("one  \n\n\n two\t\n")
print(compressed)
```

The core pipeline is intentionally conservative:

- normalize CRLF/CR line endings to LF
- trim trailing whitespace on each line
- run dedicated output filters only when they make the text smaller
- collapse repeated blank lines to `max_blank_lines` (default: `1`)

Initial dedicated filters cover:

- git output: short and long status, tabbed name-status, diff stats, and patch
  hunks/binary-patch payloads with verbose unchanged or encoded context
  summarized
- test-runner output: conservative ANSI-aware pytest (including quiet
  progress/summary output), unittest-like, vitest-like, Playwright, and
  cargo-test summaries that preserve failure identifiers, assertion/error text,
  source locations, artifacts/traces, and final totals
- CI/job logs: conservative GitHub Actions / `gh run`-style summaries that
  collapse repeated timestamped step noise while preserving step names,
  annotations, warnings/errors, file locations, artifact/URL/path references,
  and job/workflow conclusions
- standalone Python tracebacks: conservative traceback summaries that preserve
  exception type/message, file stack frames, chained-exception separators,
  source/caret context, and small command noise while collapsing repeated or
  internal stack frames
- structured tables: conservative psql, SQLite/column, MySQL boxed, and long
  Markdown pipe table summaries that preserve headers, footers, first/last
  rows, and warning/error/URL/path/ID rows while counting omitted middle rows
- file/tree/listing output: conservative path-only, `tree`, and safe `ls -R`
  reducers that render repeated path prefixes as compact trees while preserving
  visible summary totals and leaving metadata-rich listings unchanged
- ripgrep output: direct `path:line[:column]:text`, heading-mode, context-mode
  `path-line-context` with `--` separators, and ANSI-colored matches grouped
  into a compact path tree when repeated paths or prefixes would otherwise be
  duplicated, while diagnostic/table/prose-looking output is left alone
- generic application logs: conservative long timestamp/level-prefixed log
  summaries that collapse repeated INFO/DEBUG/TRACE patterns while preserving
  first/last context, warnings/errors/fatal/panic/exception lines,
  file paths/URLs, failure summaries, and unusual levels
- package-manager logs: conservative uv/pip, npm, bun, and cargo package
  install/update summaries that collapse progress/download/resolve/install
  noise while preserving warnings/errors, package summaries, versions,
  lock/frozen conflicts, audit/vulnerability summaries, hints, paths, and URLs
- build/compiler diagnostics: conservative cargo/rustc, tsc, ESLint, and
  mypy-like reducers that collapse build progress while preserving
  warnings/errors, codes, file spans, source/caret snippets, notes/help, rule
  IDs, and final summaries

## Backends

- `backend="python"` uses the pure-Python implementation.
- `backend="rust"` requires the optional Rust CLI backend.
- `backend="auto"` prefers Rust when a compatible binary is discoverable and
  falls back to Python otherwise.

Rust discovery order:

1. `CODETOOL_SHELL_RUST_BINARY`
2. packaged wheel binaries under `codetool_shell/bin/<platform>/`
3. development binaries under `rust/target/{release,debug}/`
4. `PATH`

Build and stage the Rust CLI before creating a target wheel with:

```bash
python scripts/package_rust_binary.py --target-runtime linux-x86_64
```

Then build the Python distributions with:

```bash
uv build
```

During local development you can also rely on the development build directly:

```bash
cd rust
cargo build
```

## Packaging and release

`codetool-shell` publishes target-specific wheels that include one matching
Rust helper binary under `codetool_shell/bin/<runtime>/`. Supported runtime keys
are:

- `linux-x86_64`
- `linux-aarch64`
- `macos-x86_64`
- `macos-arm64`
- `windows-x86_64`
- `windows-arm64`

To build a Linux x86_64 wheel locally (replace the runtime key for other
targets):

```bash
uv run python scripts/package_rust_binary.py --target-runtime linux-x86_64
CODETOOL_SHELL_TARGET_RUNTIME=linux-x86_64 uv build --wheel
```

The release workflow builds an sdist, manylinux wheels, native macOS/Windows
wheels, checks the distributions, and publishes to PyPI via trusted publishing
for tagged releases (`v*`) or manual dispatch with `publish=true`.

## Development

```bash
uv sync --dev
uv run pytest
uv run python benchmarks/benchmark_compress_text.py
uv run python scripts/benchmark_filters.py --iterations 1000
```