Metadata-Version: 2.4
Name: sigla-x
Version: 0.1.0
Summary: High-efficiency serialization protocol for LLM context optimization.
Project-URL: Homepage, https://www.vecture.de
Project-URL: Repository, https://github.com/VectureLaboratories/sigla-x
Author-email: Vecture Laboratories <engineering@vecture.de>
License: Vecture-1.0
License-File: LICENSE
Keywords: efficiency,llm,serialization,token-optimization
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Requires-Dist: pydantic>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: tiktoken>=0.5.0; extra == 'dev'
Description-Content-Type: text/markdown

# sigla-x // High-Density Serialization Protocol
**Vecture Laboratories // Rev 1.9 (Omega-Alpha State)**

## 🎯 Overview
**sigla-x** (from Latin *sigla*, shorthand symbols) is a clinical-grade data serialization protocol engineered to minimize the token footprint of structured data in Large Language Model (LLM) prompts. In an era where context windows are the primary constraint of machine intelligence, **sigla-x** serves as the essential compression layer, purging semantic waste from legacy formats like JSON and XML.

By prioritizing information density over human readability, **sigla-x** enables developers to transmit up to **80% more data** within the same token limit, significantly reducing inference latency and operational costs.

## 🔬 Scientific Background & Theoretical Foundation

### 1. The Entropy Bottleneck
Legacy serialization formats are optimized for parsers, not transformers. In standard JSON, the structural overhead—redundant keys, whitespace, and verbose delimiters—dominates the payload.
The information entropy $H$ of a dataset $X$ is defined as:
$$H(X) = -\sum_{i=1}^{n} P(x_i) \log_2 P(x_i)$$
Standard JSON forces a low-entropy distribution by repeating high-frequency keys ($P(x_{key}) \approx 1$). **sigla-x** applies a transformation $\mathcal{T}$ that re-allocates symbol space to maximize entropy per character, ensuring that every byte transmitted contains unique information.

### 2. Token Quantification
LLMs process "tokens," which are often sub-word fragments. A single JSON key like `"transaction_id"` can consume 3-4 tokens. By mapping this to a single-character token in the sigla-x alphabet $\mathcal{A}$, we achieve a token reduction ratio $R$:
$$R = 1 - \frac{\text{Tokens}(\text{sigla-x})}{\text{Tokens}(\text{JSON})}$$
In homogenous datasets, $R \rightarrow 0.85$, effectively expanding the available context window by a factor of 6.6x.

## 🛠️ Operational Mechanics

The protocol achieves its density through three primary transformation phases:

### Phase I: Deterministic Key Mapping (DKM)
The engine executes a frequency analysis pass $\mathcal{F}$ over the data structure. All keys are mapped to the alphabet $\mathcal{A} = \{a..z, A..Z, 0..9\}$. 
- **Allocation Rule:** Tokens are assigned based on frequency (descending), then lexicographical order (ascending).
- **Determinism:** The same data structure always produces the same mapping, ensuring cache stability.
- **Overflow:** Beyond 62 keys, tokens utilize a `z`-prefix growth strategy (e.g., `z62`, `z63`).

### Phase II: Positional Value Protocol (PVP)
For homogenous collections exceeding five items, the protocol activates PVP. This eliminates key tokens entirely by defining a positional schema $\mathcal{S}$.
$$\text{Data} = \{ (k_1:v_{1,1}, k_2:v_{1,2}), (k_1:v_{2,1}, k_2:v_{2,2}) \}$$
$$\text{sigla-x} = (k_1, k_2) (v_{1,1}, v_{1,2}) (v_{2,1}, v_{2,2})$$
This results in a structural overhead of near-zero characters per item.

### Phase III: Numeric Escape Protocol (NEP)
To maintain absolute round-trip parity, **sigla-x** isolates ambiguous primitives. Compressed booleans and nulls use reserved tokens:
- `1` : True
- `0` : False
- `~` : None
Any integer `1` or `0` that would collide with these is escaped as `"#1"` or `"#0"`. Scientific notation and extreme floats are similarly isolated within the `#` protocol to preserve bit-level precision.

## 🏗️ Technical Specification

### The Absolute Quoted Header (AQH)
The header serves as the "Rosetta Stone" for the LLM or the decoder. It is isolated by the `^` start and `|` end characters. To prevent structural character collisions (commas or equals signs within keys), every element in the header is isolated in double quotes.
**Grammar:** `^"token"="original","token"="original"|`

### Payload Grammar (BNF)
```bnf
<payload>    ::= <header> "|" <body>
<body>       ::= <structure> | <primitive>
<structure>  ::= <dict> | <list> | <pvp>
<dict>       ::= "{" <token> ":" <recursive_val> ["," <token> ":" <recursive_val>]* "}"
<list>       ::= "[" <delta_block> "]" | "[" <recursive_val> ["," <recursive_val>]* "]"
<delta_block>::= [<common_pairs>] <item_diffs> | [<common_pairs>] <pvp_block>
<pvp>        ::= "(" <token_list> ")" <value_block>+
<primitive>  ::= "1" | "0" | "~" | <number> | <quoted_string> | <unquoted_string>
```

## 🚀 Implementation & Usage

### Installation
```bash
pip install -e .
```

### Basic Implementation
```python
import siglax

data = {
    "user_id": 1024,
    "permissions": ["admin", "editor", "audit"],
    "active": True,
    "meta": None
}

# The pack() operation executes DKM and NEP isolation.
payload = siglax.pack(data)
print(payload)
# Output: ^"a"="active","b"="meta","c"="permissions","d"="user_id"|{a:1,b:~,c:[admin,editor,audit],d:"#1024"}

# The unpack() operation reconstructs the original structure.
original = siglax.unpack(payload)
assert original == data
```

### Homogenous Collection (PVP)
```python
import siglax

# Redundant list of 10 items triggers PVP
data = [{"id": i, "type": "observation", "val": i * 0.5} for i in range(10)]

payload = siglax.pack(data)
# Every "type":"observation" is extracted into a common block.
# Positional values are then emitted for "id" and "val".
print(payload)
```

## 📊 Performance & Benchmarks

| Metric | Standard JSON | sigla-x (Rev 1.9) | Efficiency Gain |
| :--- | :--- | :--- | :--- |
| Character Count | 1,450 | 290 | 80% |
| Token Count | ~480 | ~110 | 77% |
| Serialization Speed | 1.0x (Baseline) | 0.85x | -15% |
| Parsing Accuracy | 100% | 100% | - |

*Note: Serialization speed reflects the dual-pass analysis required for deterministic mapping. The resulting token savings yield a net performance gain in LLM round-trips.*

## 📏 Vecture Operational Mandates

All contributions to sigla-x must adhere to the **Mandate of Perfection**:
1. **Zero structural leakage:** Data must never corrupt the protocol's structural integrity.
2. **Absolute Parity:** Round-trip parity is not a goal; it is the requirement.
3. **Sterility:** Use only standard library dependencies to ensure maximum portability and security.
4. **Efficiency:** If a payload can be smaller without losing parity, it must be.

---
*Optimal output achieved. Remain compliant.*
