Metadata-Version: 2.4
Name: tokenizebot
Version: 0.1.3
Summary: A simple rule-based tokenizer for English text
License: MIT
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown

# TokenizeBot ![PyPI version](https://img.shields.io/pypi/v/tokenizebot) ![License](https://img.shields.io/pypi/l/tokenizebot) ![Python Versions](https://img.shields.io/pypi/pyversions/tokenizebot)

A lightweight, rule-based tokenizer for handling English clitics and punctuation.

## Installation
```bash
pip install tokenizebot
```

## Usage
```python
from tokenizebot import TokenizeBot
bot = TokenizeBot()
tokens = bot.tokenize("They've been busy.", lowercase=True)
print(tokens)
```
