Metadata-Version: 2.5
Name: aitextutils-py
Version: 0.1.0
Summary: Text cleaning, chunking, and prompt helpers for AI pipelines
Author: alexcarter
License-Expression: MIT
License-File: LICENSE
Keywords: ai,chunking,llm,prompt,text,tokens
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: aitextkit-py>=0.1.0
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == 'dev'
Description-Content-Type: text/markdown

# aitextutils

Text cleaning, chunking, and prompt helpers for AI pipelines.

This package is the text-only counterpart to `aitextkit`. It has no OS installer and no `system.py`.

## Install

```bash
pip install aitextutils
```

Local development:

```bash
cd aitextutils
pip install -e ".[dev]"
```

## Usage

```python
from aitextutils import (
    build_chat_messages,
    build_prompt,
    clean_text,
    chunk_text,
    estimate_tokens,
    extract_code_block,
    extract_json_block,
    redact_secrets,
    truncate_to_tokens,
)

clean_text("Hello   <b>world</b>")
# "Hello world"

estimate_tokens("abcd")
# 1

build_prompt("Summarize this.", system="You are concise.", context="A long article.")
build_chat_messages("Hello", system="Be brief.")
extract_code_block("```python\nprint(1)\n```", "python")
extract_json_block('Answer: {"ok": true}')
redact_secrets("contact me@site.com")
truncate_to_tokens(long_text, 200)
```

## API

- `clean_text(value, strip_html=True, strip_urls=False)`
- `chunk_text(value, max_chars=1000, overlap_chars=100)`
- `estimate_tokens(value)`
- `truncate_to_tokens(value, max_tokens)`
- `build_prompt(instruction, system="", context="", max_context_tokens=None)`
- `build_chat_messages(user, system="", context="")`
- `extract_code_block(value, language=None)`
- `extract_json_block(value)`
- `redact_secrets(value)`

## License

MIT
