Metadata-Version: 2.4
Name: hugpy-storage
Version: 0.2.18
Summary: Hugpy storage: model download queue and daemon, Hugging Face transport and token, resumable transfers, model sync, physical inventory and on-disk model layout
Author-email: putkoff <support@hugpy.ai>
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://hugpy.ai
Project-URL: UI, https://www.npmjs.com/package/@hugpy/ui
Project-URL: Documentation, https://github.com/hugpy/hugpy/blob/main/py/storage/hugpy_storage/README.md
Project-URL: Repository, https://github.com/hugpy/hugpy
Project-URL: Source, https://github.com/hugpy/hugpy/tree/main/py/storage/hugpy_storage
Project-URL: Issues, https://github.com/hugpy/hugpy/issues
Project-URL: Changelog, https://github.com/hugpy/hugpy/releases
Project-URL: Architecture, https://github.com/hugpy/hugpy/blob/main/PARTITION.md
Keywords: hugpy,llm,self-hosted,huggingface,downloads,model-storage,gguf
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: abstract_essentials
Requires-Dist: huggingface_hub>=0.20
Requires-Dist: requests>=2.28
Requires-Dist: pydantic>=2
Requires-Dist: flask>=2
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: pytest-timeout; extra == "test"
Provides-Extra: engine
Requires-Dist: hugpy-engine; extra == "engine"
Dynamic: license-file

# hugpy-storage

> **STATUS: OPTIONAL, not part of the live fleet path (2026-10-15).**
> Nothing live runs this package's downloader, sync or inventory role.
>
> - **What replaced it.** Central's Hub downloader, central drives, copies and
>   transfers live in `hugpy-core` (`hugpy_core/basic/downloads.py`, `drives.py`
>   and `scan.py`). Parts of that code were lifted from this package. Each
>   worker's `hugpy-wrapper` discovers, hashes, serves and pulls its own model
>   files (`fitevict/serve/transfer.py`, `worker_pull.py` and `drives.py`).
> - **What still imports it.**
>   - `hugpy-media`'s runners (`embed`, `keywords`, `summarizers`, `imagegen`,
>     `vision_analysis`, `tts`) call `hugpy_storage.download_models.ensure_model`
>     for **in-process local inference**. After the operator-approved change of
>     2026-10-15 (hugpy-media 0.3.4 and later), that import is lazy and
>     `hugpy-storage` is the optional extra `hugpy-media[local]`.
>   - `hugpy-engine` loses its last lazy uses in the same change (0.3.9 and
>     later; see [hugpy_engine](../hugpy_engine/README.md)).
>   - Core's legacy Flask app (`hugpy_core/web/`, `registry/`, `relay/`) imports
>     it. That app is not live, and core's venv does not install this package.
>   - `hugpy-worker` depends on it, and is retired.
> - **Where it is installed.** Until the change lands, 0.2.10 is installed in
>   `/opt/hugpy-media/venv` and `/opt/hugpy-video/venv` as a dependency of
>   `hugpy-media` 0.3.3. No unit runs `hugpy-storage` or `hugpy-downloader`.
>
> See [docs/packages.md](../../docs/packages.md).

Distribution `hugpy-storage` **0.2.10**, import `hugpy_storage`. Dependencies:
`abstract_essentials`, `huggingface_hub>=0.20`, `requests>=2.28`,
`pydantic>=2`, `flask>=2`. Extras: `engine` (`hugpy-engine`), `test`.

## What it contains

This package holds bytes at rest and in transit, as designed for the pre-basic
"orbit" runtime:

| module | role |
|---|---|
| `downloader/` (`daemon`, `queue`, `engine`, `presence`) | the model download queue and daemon |
| `transfer/downloads.py` | `ensure_model(key)`, `download_one`, `ensure_models`. On a worker (`WORKER_CENTRAL_URL` set) `ensure_model` resolves the model locally or from central and never reaches Hugging Face. |
| `transfer/provision.py`, `transfer/sync.py` | resumable central and Hugging Face transfer and model sync |
| `hub/` (`api`, `token`, `metadata`, `options/`) | external provider transport, credentials, metadata, search and install options |
| `models/` (`physical`, `status_cache`, `presence`) | physical inventory and its caches |
| `models/` (`paths`, `marker`, `gguf`, `formats`, `config`, `preview`, `archive`, `admission`) | on-disk model layout, markers, inspection and lifecycle metadata |
| `catalog/` (`source`, `providers`, `events`, `bus`) | registry Protocol, injected hooks and catalog events |
| `web/` (`api`, `transfer`) | legacy Flask blueprints; `hugpy.api: storage` points to `web.api:storage_bp` |
| `imports/` (`constants`, `functions`, `classes`, `init_imports`) | Core-style shared vocabulary, pure helpers and common types |
| `config/roots.py`, `platform/`, `schemas/`, `jobs/store.py` | environment-dependent roots, platform helpers, wire schemas and persistent jobs |
| `commands/cli.py`, `console/`, `deploy/` | command-line entry point, console helpers and service resources |

Console scripts:

- `hugpy-storage` (`daemon`, `sync`, `status`)
- `hugpy-downloader`, an alias for the download daemon

No unit runs either on the live hosts.

`docs/REGISTRY-DB-INDEX.md` records the 2026-09-10 decision to keep
`hugpy.json` markers as the per-model source of truth and to rebuild any
aggregate index from them. The wrapper's marker store follows the same rule.

## Package layout

The source follows Core's nested-package and shared-import convention:

```text
hugpy_storage/
  imports/       constants.py, functions.py, classes.py, init_imports.py
  catalog/       source.py, providers.py, events.py, bus.py
  commands/      cli.py
  config/        roots.py
  console/       UI-facing adapters
  downloader/    daemon.py, engine.py, queue.py, presence.py
  hub/           api.py, token.py, metadata.py, options/
  jobs/          store.py
  models/        paths.py, marker.py, physical.py, admission.py, ...
  platform/      filesystem/process/environment helpers
  schemas/       models.py, install.py, download_schemas.py
  transfer/      downloads.py, provision.py, sync.py
  web/           api.py, transfer.py
  deploy/        packaged service resources
```

Internal imports and package entry points use these canonical paths. Existing
flat paths and `_lifted` modules remain compatibility aliases to the **same
module objects**, including private symbols and mutable state. For example,
`hugpy_storage.providers` and `hugpy_storage.catalog.providers` share one set
of injected hooks; importing the old name does not create another registry,
metadata store, cache or bus. New integrations should use the nested paths.
The executable compatibility paths `python -m hugpy_storage.cli` and
`python -m hugpy_storage.model_sync` also delegate to the nested implementations.

`imports.constants` holds pure shared vocabulary; `imports.functions` holds
environment parsing and timestamp helpers; `imports.classes` re-exports the
lightweight catalog/provider contracts. `imports.init_imports` is the optional
one-stop import surface. These modules do not initialize storage roots or
clients. Environment-dependent roots retain their existing initialization in
`config.roots`; importing it may create directories and seed cache environment
variables, just as the old `_lifted.constants` did.
