Aggregate long ranges with a handful of requests, not thousands.
A per-bucket client pays one network round trip per output row. The Hybrid Recursive Ladder decomposes a range into complete calendar units, fetches each level once at the next finer aggregation, and folds locally—so the request count depends on the coarse units, not the minute buckets.
The 139-second query
A production ThingsBoard query over a VPN took 139 seconds for 1,140 statistical buckets. The arithmetic was trivial; the network was not. Each bucket needed up to four HTTP calls.
The obvious fixes—one coarse interval, pre-aggregation, pagination—all sacrifice something: output resolution, write access to the source, or the number of round trips.
The useful question: can the source return a coarser or finer aggregation over a contiguous run, so one request fills many output units?
2026-01-01 → 2026-10-21
month Jan … Sep 9
day Oct 1 … Oct 20 20
hour Oct 21 00–17 18
minute Oct 21 18:00–18:48 49
tail 18:49–18:49:59 1
───
97 segments
~5–7 requests
423,049 minute buckets avoided
How it works
Decompose
Split the range into complete calendar units. Emit only complete units; push partial left and right edges to the next finer level.
Fetch once per level
Fetch each maximal contiguous run sharing a fetch level at the next finer aggregation: months as days, days as hours, hours as minutes.
Fold locally
Sum the returned buckets into the output units. Optionally split against local coverage and fetch only the uncovered sub-ranges.
Measured on a production deployment
| Scenario | Segments | Requests | Time |
|---|---|---|---|
| 2 days, minute level | 69 | ~4 | 0.64 s |
| 1 year, day level | 42 | ~4 | 1.18 s cold / 0.20 s warm |
| 10 months, mixed levels | 97 | ~5–7 | sub-second |
| Local read, 42–90 segments | 42–90 | 0 | 9–19 ms |
Production numbers over a VPN; latency depends on network, source load and data density. The reproducible benchmark prints segment and request counts without a server.
What it is—and is not
It is a query-planning composition
- calendar hierarchy as the query plan, not just the display bucket;
- next-finer fetch plus local fold;
- fetch-level + contiguity grouping;
- coverage-aware gap filling;
- request count bounded by the source's per-request bucket limit.
It is not a new database or a new theory
- RRDtool / Whisper: multi-resolution archives;
- BTrDB: hierarchical time-partitioned summaries;
- TimescaleDB: hierarchical continuous aggregates;
- Druid: calendar period granularity;
- Prometheus / Grafana: point caps and single-step downsampling;
- segment trees and timing wheels: aligned decomposition and level cascade.
The method is published as a public disclosure and released under the MIT License. The prior-art review documents the closest systems and the differences.
Cite this work
@software{hybrid_recursive_ladder_2026,
author = {MagicSquarekey},
title = {Hybrid Recursive Ladder: Query Planning for Bucket-Limited Time-Series APIs},
year = {2026},
version = {0.0.6},
license = {MIT},
doi = {10.5281/zenodo.23683292},
url = {https://github.com/MagicSquarekey/hybrid_recursive_ladder_algorithm}
}
Archived at Zenodo: 10.5281/zenodo.23683292. GitHub's "Cite this repository" button reads CITATION.cff.
Try it
pip install https://github.com/MagicSquarekey/hybrid_recursive_ladder_algorithm/releases/download/v0.0.6/hybrid_recursive_ladder-0.0.6-py3-none-any.whl hybrid-ladder demo hybrid-ladder decompose --start 2026-01-01 --end 2026-10-21T18:49:59 python examples/benchmark.py
No database, no network, no configuration required for the demo. Documentation: README · algorithm spec · 中文文章.