Metadata-Version: 2.5
Name: cattensors
Version: 0.2.0
Summary: Fast safetensors loading for distributed filesystems
Requires-Python: >=3.11
Requires-Dist: safetensors
Requires-Dist: torch
Provides-Extra: dev
Requires-Dist: flashpack; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest>=9.0.2; extra == 'dev'
Requires-Dist: safetensors>=0.7.0; extra == 'dev'
Requires-Dist: torch>=2.10.0; extra == 'dev'
Requires-Dist: ty>=0.0.19; extra == 'dev'
Description-Content-Type: text/markdown

# cattensors

Faster `safetensors` loading for JuiceFS.

> Note: This library was developed and benchmarked against JuiceFS, but in theory should work with any network backed distributed filesystem.

`cattensors` monkey-patches `safetensors` to pre-warm the OS page cache before tensor reads. On network backed filesystems, the first read of a large model file is slow. `cattensors` reads the file into page cache using parallel threads so subsequent `mmap` based access hits RAM instead of the network.

## Compatibility

- POSIX platforms (Linux/macOS)
- Not supported on Windows (`fcntl.flock` is used for cross-process coordination)

## Installation

```bash
pip install git+https://github.com/badayvedat/cattensors
```

## Usage

Installing or importing `cattensors` does not patch anything. Call `patch()` explicitly in each process that loads models:

```python
import cattensors

cattensors.patch()

from safetensors.torch import load_file
```

Call it before importing libraries such as diffusers or transformers that may cache references to safetensors functions. Importing `torch` first is fine; existing `from safetensors import safe_open` or `from safetensors.torch import load_file` references cannot be updated by patching later. Calling `patch()` before any torch/safetensors-related imports is the simplest safe convention.

Flashpack patching is separate and opt-in:

```python
from cattensors.flashpack_patch import patch as patch_flashpack

patch_flashpack()
```

Call this before importing code that caches flashpack reader functions.

No startup `.pth` file is installed, and `CATTENSORS_AUTO_PATCH` is no longer supported. When upgrading an older installation, ensure its `cattensors.pth` has been removed from `site-packages` (a normal package upgrade should remove it).


## Configuration

Environment variables:

| Variable | Default | Description |
|---|---|---|
| `CATTENSORS_WARM_THREADS` | `8` | Parallel reader threads per file |
| `CATTENSORS_PARALLEL_SHARDS` | `false` | Pre-warm sibling shards in background when loading sharded models |
| `HF_ENABLE_PARALLEL_LOADING` | `false` | If `true`, `cattensors` skips sibling-shard parallel warming to avoid overlapping I/O |
| `CATTENSORS_DEBUG` | `false` | Enable debug logs to stderr |

Flags are read when used, so changes affect subsequent load calls.
