Metadata-Version: 2.4
Name: dlmhub
Version: 0.1.0
Summary: A provider-agnostic, fully parametrable model hub for developers (not tied to Hugging Face Hub).
Author-email: Abdelkrime Aries <kariminfo0@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/kariminf/dlmhub
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Intended Audience :: Developers
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.25
Requires-Dist: tqdm>=4.60
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-mock>=3.10; extra == "dev"
Dynamic: license-file

# dlmhub

A small, provider-agnostic model hub for Python libraries. `dlmhub` gives
your users a `MyClass.load_from_hub("org/model")` API without hard-wiring
your project to Hugging Face Hub (or any single provider).

## Why

Many libraries want "download my pretrained files, cache them locally,
load them" behavior, but end up tightly coupled to `huggingface_hub`.
`dlmhub` extracts that pattern into a standalone, dependency-light
package where:

- **Providers are pluggable.** Hugging Face and GitHub releases ship as
  built-ins; register your own (S3, a private server, git-lfs, ...) with
  a few lines.
- **The cache directory is fully parametrable.** Set it globally, per
  class, or per call — whichever fits your app.
- **The mixin is the only requirement.** Add `HubMixin` to any class,
  declare the files it needs, implement how to build an instance from
  them, and you're done.

## Install

```bash
pip install -e .            # from a local checkout
# or, once published:
pip install dlmhub
```

## Quickstart

```python
from dlmhub import HubMixin

class MyTokenizer(HubMixin):
    files = ["vocab.json", "merges.txt"]

    @classmethod
    def _load_from_files(cls, local_files, **kwargs):
        # local_files: {"vocab.json": Path(...), "merges.txt": Path(...)}
        ...
        return cls(...)

tok = MyTokenizer.load_from_hub("org/model")                  # Hugging Face by default
tok = MyTokenizer.load_from_hub("org/repo", provider="github") # GitHub releases
tok = MyTokenizer.load_from_hub("org/model", variant="fp16")   # variant-prefixed files
tok = MyTokenizer.load_from_hub("org/model", token="hf_...")   # private/gated repos
```

## Configuring the cache directory

Resolution order, highest priority first:

1. Per call: `MyTokenizer.load_from_hub(..., cache_dir="/data/cache")`
2. Per class: `MyTokenizer.CACHE_DIR = Path("/data/cache")`
3. Globally, in code: `dlmhub.config.set_cache_dir("/data/cache")`
4. Globally, via environment: `DLMHUB_CACHE_DIR=/data/cache`
5. Built-in default: `~/.cache/dlmhub`

```python
import dlmhub
dlmhub.config.set_cache_dir("/data/model_cache")
```

## Adding a custom provider

```python
from dlmhub.providers import Provider, default_registry

# Simple case: just a URL template with {model} and {file} placeholders.
default_registry.register(Provider(
    name="my-s3",
    url_pattern="https://my-bucket.s3.amazonaws.com/{model}/{file}",
))

# Full control: supply your own download logic (auth, SDKs, checksums...).
def _download_from_internal_server(model, file, dest_path, headers, **kwargs):
    ...  # write bytes to dest_path yourself

default_registry.register(Provider(
    name="internal",
    download_fn=_download_from_internal_server,
))
```

Need full isolation instead of touching the shared registry? Create your
own `ProviderRegistry()` and assign it to `MyTokenizer.PROVIDER_REGISTRY`.

## Compact hub-path strings

For CLIs/config files, a `"provider:model[:variant]"` string can be
parsed with:

```python
from dlmhub import process_hub_path

process_hub_path("huggingface:org/model:fp16")
# {"provider": "huggingface", "model": "org/model", "variant": "fp16"}
```

## Project layout

```
src/dlmhub/
    __init__.py     public API surface
    config.py       global, overridable configuration (cache dir, ...)
    providers.py    Provider / ProviderRegistry, built-in HF + GitHub providers
    hub.py          HubMixin: load_from_hub() and download orchestration
    hubpath.py      "provider:model:variant" string parsing
    fileutils.py    small filesystem helpers (list files/subfolders, read lines)
    exceptions.py   DlmHubError and friends
tests/
```

## License

Copyright (C) 2026 Abdelkrime Aries

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
