How laya-compactor works

Score every retrieved chunk against the query in one shared forward pass, then keep the most relevant chunks that fit the token budget. Nothing kept is rewritten. Everything cut gets a reason.

query + retrieved docs

? Who invented the telephone?

Bell patent for the telephone, 1876

2.48

First transcontinental call, 1915

1.64

Bell's research on hearing and speech

1.56

Edison's carbon microphone

1.36

Elisha Gray's caveat, same day

budget exhausted: needs 17 tokens, 6 left

1.21

The Berlin Wall fell in 1989

score 0.20 below min_score 1.00

0.20

Jenner's smallpox vaccination, 1796

score 0.05 below min_score 1.00

0.05

Stephenson's Rocket at Rainhill

score 0.04 below min_score 1.00

0.04

score, all at once

laya

predict_batch

one shared forward pass
local checkpoint, no API

0 irr · 1 bg · 2 rel · 3 ess

compact context

Bell patent, 1876

transcontinental call

Bell's research

Edison microphone

84/136 tokens — 38% saved

Measured on SQuAD (200 questions): answers identical to full context on 30% of the tokens. Multi-hop HotpotQA: 87% of full exact match on 68% of the tokens — losses published, not hidden.