Metadata-Version: 2.5
Name: aitextkit-py
Version: 0.1.1
Summary: Text cleaning, chunking, and token estimation for AI pipelines
Author: alexcarter
License-Expression: MIT
License-File: LICENSE
Keywords: ai,chunking,llm,text,tokens
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == 'dev'
Description-Content-Type: text/markdown

# aitextkit

Text cleaning, chunking, and token estimation for AI pipelines.

## Install

```bash
pip install aitextkit
```

For local development:

```bash
pip install -e .
```

## Usage

```python
from aitextkit import (
    build_chat_messages,
    build_prompt,
    clean_text,
    extract_code_block,
    extract_json_block,
    redact_secrets,
    truncate_to_tokens,
)

clean_text("Hello   <b>world</b>")
# "Hello world"

prompt = build_prompt(
    "Summarize this.",
    system="You are concise.",
    context="A long article.",
    max_context_tokens=500,
)

messages = build_chat_messages("Hello", system="Be brief.")
extract_code_block("```python\nprint(1)\n```", "python")
extract_json_block('Answer: {"ok": true}')
redact_secrets("contact me@site.com")
truncate_to_tokens(long_text, 200)
```

## API

- `clean_text(value, strip_html=True, strip_urls=False)`
- `chunk_text(value, max_chars=1000, overlap_chars=100)`
- `estimate_tokens(value)`
- `truncate_to_tokens(value, max_tokens)`
- `build_prompt(instruction, system="", context="", max_context_tokens=None)`
- `build_chat_messages(user, system="", context="")`
- `extract_code_block(value, language=None)`
- `extract_json_block(value)`
- `redact_secrets(value)`
- `get_os_version()` — returns `windows`, `mac`, or `linux` in `name`

## License

MIT
