scitex_storage

scitex-storage: research-data storage triage.

MVP scope (see README for the full roadmap): read-only directory-tree scan — discovery, size x staleness scoring, and an optional (size+hash) duplicate-file report. Nothing in this release moves, deletes, or otherwise mutates anything on disk.

Public names are exposed via PEP 562 __getattr__ (lazy, on first access) so import scitex_storage itself stays cheap — every CLI invocation and every shell tab-completion pays this cost (see the python-api skill’s “PEP 562 module __getattr__” section).

class scitex_storage.FileEntry(path, size, atime)

Bases: object

One scanned file: its path (relative to the scan root), size, and last-access time.

path: Path

size: int

atime: float

days_since_access(now=None)

Return type:

float

score(now=None)

score = size_bytes * days_since_last_access (design-doc heuristic).

Return type:

float

__init__(path, size, atime)

class scitex_storage.ScanResult(root, files_scanned=0, dirs_scanned=0, total_size=0, skipped_size=0, skipped_dirs=<factory>, entries=<factory>, duplicate_groups=<factory>, scan_time=<factory>)

Bases: object

root: Path

files_scanned: int = 0

dirs_scanned: int = 0

total_size: int = 0

skipped_size: int = 0

skipped_dirs: set[str]

entries: list[FileEntry]

duplicate_groups: list[list[Path]]

scan_time: float

top_candidates(top=20)

Return type:

list[FileEntry]

__init__(root, files_scanned=0, dirs_scanned=0, total_size=0, skipped_size=0, skipped_dirs=<factory>, entries=<factory>, duplicate_groups=<factory>, scan_time=<factory>)

scitex_storage.find_duplicates(entries)

Group files that share (size, sha256) — a cheap, exact duplicate detector in the spirit of fdupes/rdfind.

Files are first bucketed by size (free — already known from the walk); only files sharing a size bucket with >= 2 members are hashed, so a tree with no same-size collisions costs zero hashing.

Return type:

list[list[Path]]

scitex_storage.scan(root, top=20, dedupe=True, exclude_dirs=frozenset({'.eggs', '.git', '.mypy_cache', '.pytest_cache', '.ruff_cache', '.tox', '.venv', '__pycache__', 'build', 'dist', 'env', 'node_modules', 'venv'}))

Scan root and return a ScanResult.

Read-only: only stats and (for dedupe) reads file bytes. Never writes, moves, or deletes anything.

Return type:

ScanResult

scitex_storage.walk_tree(root, exclude_dirs=frozenset({'.eggs', '.git', '.mypy_cache', '.pytest_cache', '.ruff_cache', '.tox', '.venv', '__pycache__', 'build', 'dist', 'env', 'node_modules', 'venv'}))

Walk root, returning (entries, dirs_scanned, skipped_size).

Directories whose name matches exclude_dirs are pruned entirely (never descended into); their on-disk size is best-effort summed into skipped_size for the report, but their contents are not scanned.

Return type:

tuple[list[FileEntry], int, int]