Metadata-Version: 2.2
Name: codeweave
Version: 0.1.0
Summary: A code-optimized tokenizer for better understanding and processing of Python source code
Home-page: https://github.com/vaifai/codeweave
Author: Vaibhav Pandey
Author-email: vaifaipandey1996@gmail.com
Project-URL: Bug Tracker, https://github.com/vaifai/codeweave/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: General
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: regex
Requires-Dist: tokenize-rt
Requires-Dist: tqdm
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# CodeWeave

A code-optimized tokenizer designed specifically for source code, offering better compression and understanding of code structures compared to general-purpose tokenizers.

## Installation

```bash
pip install codeweave
```

**Quick Start**
```bash
from codeweave import CodeTokenizer

tokenizer = CodeTokenizer()
code = """
def hello_world():
    print("Hello, World!")
"""
tokens = tokenizer.tokenize(code)
```
