Metadata-Version: 2.4
Name: scrollscout
Version: 0.1.0
Summary: Scout Vesuvius Challenge scrolls: stream and preview Herculaneum CT volumes from a laptop without downloading terabytes or tripping rate limits.
Author: Alexandre Dufour-Richard
License: MIT
Project-URL: Homepage, https://github.com/vonduffen/scrollscout
Project-URL: Repository, https://github.com/vonduffen/scrollscout
Project-URL: Issues, https://github.com/vonduffen/scrollscout/issues
Project-URL: Vesuvius Challenge, https://scrollprize.org
Keywords: vesuvius-challenge,herculaneum,ome-zarr,ct,papyrology
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: zarr<3,>=2.14
Requires-Dist: numpy>=1.23
Requires-Dist: fsspec>=2023.1
Requires-Dist: aiohttp>=3.8
Requires-Dist: requests>=2.28
Requires-Dist: Pillow>=9.0
Provides-Extra: catalog
Requires-Dist: vesuvius>=0.1.9; extra == "catalog"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# scrollscout

**Scout [Vesuvius Challenge](https://scrollprize.org) scrolls from a laptop — stream and preview the Herculaneum CT volumes without downloading terabytes or tripping the data server's rate limiter.**

The scroll volumes are enormous. Scroll 1 alone is **14376 × 7888 × 8096 voxels ≈ 918 GB** at full resolution. If you're on a laptop with limited disk, you can't (and shouldn't) download them to take a look. The data *is* streamable as multiscale OME-Zarr — but two things bite newcomers immediately:

1. **You don't know which resolution level to read.** Ask for a full-res cross-section and you'll try to pull hundreds of GB.
2. **Naive reads get rate-limited.** A large `array[z, y0:y1, x0:x1]` makes zarr fire one HTTP request per chunk, all at once. The server replies `429 Too Many Requests` and your read dies:

   ```
   aiohttp.client_exceptions.ClientResponseError: 429, message='Too Many Requests',
   url='https://dl.ash2txt.org/.../54keV_7.91um_Scroll1A.zarr/0/56/19/26'
   ```

`scrollscout` handles both for you: it **auto-selects the coarsest pyramid level** that satisfies your requested pixel width, and reads it in **small, chunk-aligned tiles with exponential backoff on 429**, so a read *slows down* under pressure instead of failing.

![Scroll 1 mid cross-section](https://raw.githubusercontent.com/vonduffen/scrollscout/main/examples/scroll1_mid.png)

*Scroll 1 (PHerc. Paris 4), middle cross-section — streamed with `scrollscout thumb 1 -w 1000`. Total transfer: a few MB, not 918 GB.*

## Install

```bash
pip install scrollscout
# optional: use the live scroll catalogue from the official `vesuvius` package
pip install "scrollscout[catalog]"
```

You must accept the Vesuvius Challenge [data license](https://scrollprize.org/data) before accessing scroll data. scrollscout does not redistribute any scroll data; it streams the public server on your behalf.

## Command line

```bash
# list a scroll's multiscale pyramid — see the sizes before you read anything
scrollscout info 1

# stream a middle cross-section to a PNG (auto-picks the pyramid level)
scrollscout thumb 1 --width 1000 -o scroll1.png

# a shallower slice of scroll 2, larger
scrollscout thumb 2 --z-frac 0.25 --width 1200 -o scroll2_quarter.png
```

`scrollscout info 1` prints:

```
Scroll 1
  pyramid levels (6):
    level             shape (z,y,x)          chunks         scale  approx full size
        0       (14376, 7888, 8096)  (128, 128, 128)  (1.0, 1.0, 1.0)    918.07 GB
        1        (7188, 3944, 4048)  (128, 128, 128)  (2.0, 2.0, 2.0)    114.76 GB
        2        (3594, 1972, 2024)  (128, 128, 128)  (4.0, 4.0, 4.0)     14.34 GB
        3         (1797, 986, 1012)  (128, 128, 128)  (8.0, 8.0, 8.0)      1.79 GB
        4           (899, 493, 506)  (128, 128, 128)  (16.0,16.0,16.0)     0.22 GB
        5           (450, 247, 253)  (128, 128, 128)  (32.0,32.0,32.0)     0.03 GB
```

## Python API

```python
import scrollscout as ss

vol = ss.open_scroll(1)            # streams only ~KB of metadata
print(vol.level_for_width(1000))   # -> Level(path='3', ...)

# get the cross-section as a numpy array (throttle-safe under the hood)
arr = ss.thumbnail(1, z_frac=0.5, width=1000, out="scroll1.png", verbose=True)
print(arr.shape, arr.dtype)        # (986, 1012) uint8
```

## How the throttle-safe read works

- `level_for_width(target_px)` picks the **coarsest** pyramid level whose in-plane
  width still meets your target. A 1000 px view comes from level 3 (~64 chunks),
  never level 0 (~4000 chunks). Under-fetching is the first line of defense.
- `read_slice()` walks the chosen slice in `tile`-sized blocks with a short pause
  between tiles, and retries each tile with exponential backoff (`base_delay * 2**n`)
  when the server returns 429. Non-rate-limit errors propagate immediately.

## Scope & limitations

- Reads scroll **volumes** (the `volumes_zarr*` OME-Zarr). Segment/surface
  meshes and ink predictions are out of scope for v0.1.
- Preview/QA tool, not an analysis pipeline — it gets you *looking* at the data
  fast so you can decide where to work.
- Cross-sections are on the `z` axis (the scan slice axis) for now.

## License

MIT (see `LICENSE`). Scroll data itself is governed by the Vesuvius Challenge /
EduceLab-Scrolls terms and is **not** included or redistributed here.

If you use scroll data in published work, cite EduceLab-Scrolls: Parsons, S.,
Parker, C. S., Chapman, C., Hayashida, M., & Seales, W. B. (2023).
*EduceLab-Scrolls: Verifiable Recovery of Text from Herculaneum Papyri using
X-ray CT.* arXiv:2304.02084.
