# Vendored BPE rank tables for offline tiktoken (--mode openai).
#
# Files:
#   cl100k_base.tiktoken
#   o200k_base.tiktoken
#
# Upstream mirror (refresh via scripts/pin_tiktoken.sh):
#   https://raw.githubusercontent.com/niieani/gpt-tokenizer/main/data/
#
# Do NOT fetch from openaipublic.blob.core.windows.net in this project.
