⚡ Chunkr Wasm
Browser Demo
High-throughput document and text chunking running 100% client-side via WebAssembly.
Input Document
# Chunkr: Blazingly Fast Document Chunking Chunkr is an ultra-fast document and text chunking library engineered in Rust with native bindings for Python and WebAssembly. ## Key WebAssembly Features - **100% Client-Side**: No sensitive documents or proprietary text ever leave the user's browser. - **Edge Compatible**: Direct synchronous loading in Cloudflare Workers and lightweight serverless runtimes. - **In-Memory PDF Parsing**: Extract and chunk multi-page PDFs directly from raw bytes. - **OpenAI Tokenizer**: Exact BPE token sizing using cl100k_base and o200k_base encodings. ## Post-Processing Pipeline Pipeline stages allow chaining min/max length filtering, exact deduplication, bin-packing, and metadata enrichment.
Chunking Strategy
Recursive (Natural Boundaries)
Markdown (Header Hierarchy)
Token (OpenAI BPE cl100k)
Sentence Boundaries
Paragraph Boundaries
Code Syntax
Semantic (Lexical Distance)
Chunk Size (Chars/Tokens)
Overlap
Loading Wasm...