Metadata-Version: 2.4
Name: xiaoshiai-hub
Version: 0.1.2
Summary: Python SDK for XiaoShi AI Hub - Upload, download, and manage AI models and datasets with encryption support
Home-page: https://github.com/poxiaoyun/moha-sdk
Author: XiaoShi AI
Author-email: XiaoShi AI <support@xiaoshiai.cn>
Maintainer: XiaoShi AI Team
Maintainer-email: XiaoShi AI Team <support@xiaoshiai.cn>
License: Apache-2.0
Project-URL: Homepage, https://github.com/poxiaoyun/moha-sdk
Project-URL: Repository, https://github.com/poxiaoyun/moha-sdk
Project-URL: Issues, https://github.com/poxiaoyun/moha-sdk/issues
Keywords: xiaoshi,ai-hub,machine-learning,deep-learning,model-hub,dataset,encryption,sdk,api-client
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.20.0
Requires-Dist: tqdm>=4.62.0
Provides-Extra: upload
Requires-Dist: gitpython>=3.1.0; extra == "upload"
Requires-Dist: cryptography>=3.4.0; extra == "upload"
Provides-Extra: encryption
Requires-Dist: cryptography>=3.4.0; extra == "encryption"
Requires-Dist: gmssl>=3.2.1; extra == "encryption"
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0; extra == "dev"
Requires-Dist: pytest-mock>=3.6.0; extra == "dev"
Requires-Dist: black>=22.0; extra == "dev"
Requires-Dist: flake8>=4.0; extra == "dev"
Requires-Dist: mypy>=0.950; extra == "dev"
Requires-Dist: isort>=5.10.0; extra == "dev"
Requires-Dist: build>=0.8.0; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Requires-Dist: gitpython>=3.1.0; extra == "dev"
Requires-Dist: cryptography>=3.4.0; extra == "dev"
Requires-Dist: gmssl>=3.2.1; extra == "dev"
Provides-Extra: all
Requires-Dist: gitpython>=3.1.0; extra == "all"
Requires-Dist: cryptography>=3.4.0; extra == "all"
Requires-Dist: gmssl>=3.2.1; extra == "all"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: maintainer
Dynamic: requires-python

# XiaoShi AI Hub Python SDK

[![PyPI version](https://badge.fury.io/py/xiaoshiai-hub.svg)](https://badge.fury.io/py/xiaoshiai-hub)
[![Python Support](https://img.shields.io/pypi/pyversions/xiaoshiai-hub.svg)](https://pypi.org/project/xiaoshiai-hub/)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

XiaoShi AI Hub Python SDK 是一个功能强大的 Python 库，用于与 XiaoShi AI Hub 平台进行交互。它提供了简单易用的 API，支持模型和数据集的上传、下载、加密等功能。

[English](README_EN.md) | 简体中文

## ✨ 特性

- 🚀 **简单易用** - 类似 Hugging Face Hub 的 API 设计，上手即用
- 📥 **下载功能** - 支持下载单个文件或整个仓库
- 📤 **上传功能** - 支持上传文件和文件夹到仓库
- 🔐 **加密支持** - 内置多种加密算法（AES、SM4、RSA、SM2）
- 🎯 **模式匹配** - 支持使用 allow/ignore 模式过滤文件
- 📊 **进度显示** - 下载和上传时显示进度条
- 🔑 **多种认证** - 支持用户名/密码和 Token 认证
- 🌐 **环境变量配置** - 灵活的 Hub URL 配置
- 💾 **缓存支持** - 高效的文件缓存机制
- 🔍 **类型提示** - 完整的类型注解，IDE 友好

## 📦 安装

### 基础安装（仅下载功能）

```bash
pip install xiaoshiai-hub
```

### 完整安装（包含上传功能）

```bash
pip install xiaoshiai-hub[upload]
```

### 开发安装

```bash
pip install xiaoshiai-hub[dev]
```

### 完整安装（所有功能）

```bash
pip install xiaoshiai-hub[all]
```

## 🚀 快速开始

### 下载单个文件

```python
from xiaoshiai_hub import moha_hub_download

# 下载单个文件
file_path = moha_hub_download(
    repo_id="demo/demo",
    filename="config.yaml",
    repo_type="models",  # 或 "datasets"
    username="your-username",
    password="your-password",
)
print(f"文件已下载到: {file_path}")
```

### 下载整个仓库

```python
from xiaoshiai_hub import snapshot_download

# 下载整个仓库
repo_path = snapshot_download(
    repo_id="demo/demo",
    repo_type="models",
    username="your-username",
    password="your-password",
)
print(f"仓库已下载到: {repo_path}")
```

### 使用过滤器下载

```python
from xiaoshiai_hub import snapshot_download

# 只下载 YAML 和 Markdown 文件
repo_path = snapshot_download(
    repo_id="demo/demo",
    allow_patterns=["*.yaml", "*.yml", "*.md"],
    ignore_patterns=[".git*", "*.log"],
    username="your-username",
    password="your-password",
)
```

### 上传文件

```python
from xiaoshiai_hub import upload_file

# 上传单个文件
commit_hash = upload_file(
    path_or_fileobj="./config.yaml",
    path_in_repo="config.yaml",
    repo_id="demo/my-model",
    repo_type="models",
    commit_message="Upload config file",
    username="your-username",
    password="your-password",
)
print(f"提交哈希: {commit_hash}")
```

### 上传文件夹

```python
from xiaoshiai_hub import upload_folder

# 上传整个文件夹
commit_hash = upload_folder(
    folder_path="./my_model",
    repo_id="demo/my-model",
    repo_type="models",
    commit_message="Upload model files",
    ignore_patterns=["*.log", ".git*"],  # 忽略这些文件
    username="your-username",
    password="your-password",
)
print(f"提交哈希: {commit_hash}")
```

### 加密上传和下载

```python
from xiaoshiai_hub import upload_file, moha_hub_download
from xiaoshiai_hub.encryption import EncryptionAlgorithm

# 上传加密文件
commit_hash = upload_file(
    path_or_fileobj="./secret.txt",
    path_in_repo="secret.txt",
    repo_id="demo/encrypted-repo",
    encryption_key="your-32-character-secret-key",
    encryption_algorithm=EncryptionAlgorithm.AES_256_CBC,
    username="your-username",
    password="your-password",
)

# 下载并解密文件
file_path = moha_hub_download(
    repo_id="demo/encrypted-repo",
    filename="secret.txt",
    decryption_key="your-32-character-secret-key",
    decryption_algorithm=EncryptionAlgorithm.AES_256_CBC,
    username="your-username",
    password="your-password",
)
```

### 使用 HubClient API

```python
from xiaoshiai_hub import HubClient

# 创建客户端
client = HubClient(
    username="your-username",
    password="your-password",
)

# 获取仓库信息
repo_info = client.get_repository_info("demo", "models", "my-model")
print(f"仓库名称: {repo_info.name}")
print(f"组织: {repo_info.organization}")

# 列出分支
branches = client.list_branches("demo", "models", "my-model")
for branch in branches:
    print(f"分支: {branch.name} (commit: {branch.commit_sha})")

# 浏览仓库内容
content = client.get_repository_content("demo", "models", "my-model", "main")
for entry in content.entries:
    print(f"{entry.type}: {entry.name}")
```

## 🔐 加密功能

SDK 支持多种加密算法：

### 对称加密

- **AES-256-CBC** - 标准 AES 加密
- **AES-256-GCM** - AES 加密（带认证）
- **SM4-CBC** - 国密 SM4 加密
- **SM4-GCM** - 国密 SM4 加密（带认证）

### 非对称加密

- **RSA-OAEP** - RSA 加密（推荐）
- **RSA-PKCS1V15** - RSA 加密（兼容性）
- **SM2** - 国密 SM2 加密

### 生成加密密钥

```python
import secrets
import string

# 生成对称加密密钥（32 字符）
symmetric_key = ''.join(secrets.choice(string.ascii_letters + string.digits) for _ in range(32))
print(f"对称密钥: {symmetric_key}")

# 生成 RSA 密钥对
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import serialization

private_key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048,
)
public_key = private_key.public_key()

# 序列化公钥
public_pem = public_key.public_bytes(
    encoding=serialization.Encoding.PEM,
    format=serialization.PublicFormat.SubjectPublicKeyInfo
).decode()

# 序列化私钥
private_pem = private_key.private_bytes(
    encoding=serialization.Encoding.PEM,
    format=serialization.PrivateFormat.PKCS8,
    encryption_algorithm=serialization.NoEncryption()
).decode()
```

### 部分文件加密

```python
from xiaoshiai_hub import upload_folder
from xiaoshiai_hub.encryption import EncryptionAlgorithm

# 上传文件夹，只加密部分文件
commit_hash = upload_folder(
    folder_path="./my_model",
    repo_id="demo/my-model",
    encryption_key="your-secret-key",
    encryption_algorithm=EncryptionAlgorithm.AES_256_CBC,
    encryption_exclude=["README.md", "*.yaml"],  # 这些文件不加密
    username="your-username",
    password="your-password",
)
```

## ⚙️ 配置

### 环境变量

```bash
# Hub 服务端点
export MOHA_ENDPOINT="https://your-hub-url.com/moha"

# 认证信息
export MOHA_USERNAME="your-username"
export MOHA_PASSWORD="your-password"

# 加密密钥（可选）
export ENCRYPTION_KEY="your-32-character-encryption-key"
export DECRYPTION_KEY="your-32-character-decryption-key"
```

### 使用 .env 文件

创建 `.env` 文件：

```bash
MOHA_ENDPOINT=https://your-hub-url.com/moha
MOHA_USERNAME=your-username
MOHA_PASSWORD=your-password
ENCRYPTION_KEY=your-encryption-key
```

然后在代码中加载：

```python
from dotenv import load_dotenv
load_dotenv()

from xiaoshiai_hub import moha_hub_download

# 自动使用环境变量中的认证信息
file_path = moha_hub_download(
    repo_id="demo/demo",
    filename="config.yaml",
)
```

## 📚 示例代码

项目包含丰富的示例代码，位于 `examples/` 目录：

- **01_basic_usage.py** - 基础使用示例
- **02_download_file.py** - 下载文件示例
- **03_download_repository.py** - 下载仓库示例
- **04_upload_file.py** - 上传文件示例
- **05_upload_folder.py** - 上传文件夹示例
- **06_encryption.py** - 加密功能示例
- **07_complete_workflow.py** - 完整工作流示例

运行示例：

```bash
# 设置环境变量
export MOHA_USERNAME="your-username"
export MOHA_PASSWORD="your-password"

# 运行示例
python examples/01_basic_usage.py
python examples/07_complete_workflow.py
```

详细说明请查看 [examples/README.md](examples/README.md)

## 🧪 测试

运行测试：

```bash
# 安装开发依赖
pip install xiaoshiai-hub[dev]

# 运行所有测试
pytest

# 运行特定测试文件
pytest tests/test_client.py

# 运行测试并显示覆盖率
pytest --cov=xiaoshiai_hub --cov-report=html

# 运行测试（详细输出）
pytest -v
```

## 📖 API 文档

### 核心函数

#### `moha_hub_download()`

下载单个文件。

**参数：**
- `repo_id` (str): 仓库 ID，格式为 "organization/repo_name"
- `filename` (str): 要下载的文件名
- `repo_type` (str): 仓库类型，"models" 或 "datasets"，默认 "models"
- `revision` (str, optional): 分支、标签或提交哈希，默认 "main"
- `local_dir` (str, optional): 本地保存目录
- `username` (str, optional): 用户名
- `password` (str, optional): 密码
- `token` (str, optional): 认证 Token
- `decryption_key` (str, optional): 解密密钥
- `decryption_algorithm` (str, optional): 解密算法

**返回：** 下载文件的本地路径

#### `snapshot_download()`

下载整个仓库。

**参数：**
- `repo_id` (str): 仓库 ID
- `repo_type` (str): 仓库类型，默认 "models"
- `revision` (str, optional): 分支、标签或提交哈希，默认 "main"
- `local_dir` (str, optional): 本地保存目录
- `allow_patterns` (List[str], optional): 允许下载的文件模式
- `ignore_patterns` (List[str], optional): 忽略的文件模式
- `username` (str, optional): 用户名
- `password` (str, optional): 密码
- `token` (str, optional): 认证 Token
- `decryption_key` (str, optional): 解密密钥
- `decryption_algorithm` (str, optional): 解密算法

**返回：** 下载仓库的本地路径

#### `upload_file()`

上传单个文件。

**参数：**
- `path_or_fileobj` (str | Path | bytes): 文件路径或文件对象
- `path_in_repo` (str): 仓库中的文件路径
- `repo_id` (str): 仓库 ID
- `repo_type` (str): 仓库类型，默认 "models"
- `revision` (str): 分支名称，默认 "main"
- `commit_message` (str, optional): 提交信息
- `commit_description` (str, optional): 提交描述
- `username` (str, optional): 用户名
- `password` (str, optional): 密码
- `token` (str, optional): 认证 Token
- `encryption_key` (str, optional): 加密密钥
- `encryption_algorithm` (str, optional): 加密算法

**返回：** 提交哈希

#### `upload_folder()`

上传整个文件夹。

**参数：**
- `folder_path` (str | Path): 文件夹路径
- `repo_id` (str): 仓库 ID
- `repo_type` (str): 仓库类型，默认 "models"
- `revision` (str): 分支名称，默认 "main"
- `commit_message` (str, optional): 提交信息
- `commit_description` (str, optional): 提交描述
- `ignore_patterns` (List[str], optional): 忽略的文件模式
- `username` (str, optional): 用户名
- `password` (str, optional): 密码
- `token` (str, optional): 认证 Token
- `encryption_key` (str, optional): 加密密钥
- `encryption_algorithm` (str, optional): 加密算法
- `encryption_exclude` (List[str], optional): 不加密的文件模式

**返回：** 提交哈希

### HubClient 类

#### 初始化

```python
client = HubClient(
    base_url="https://hub.example.com/moha",
    username="your-username",
    password="your-password",
)
```

#### 方法

- `get_repository_info(org, repo_type, repo_name)` - 获取仓库信息
- `list_branches(org, repo_type, repo_name)` - 列出分支
- `list_tags(org, repo_type, repo_name)` - 列出标签
- `get_repository_content(org, repo_type, repo_name, branch, path)` - 获取仓库内容
- `download_file(org, repo_type, repo_name, branch, file_path, local_path)` - 下载文件
- `get_moha_encryption(org, repo_type, repo_name, branch)` - 获取加密元数据

## 🔧 开发

### 设置开发环境

```bash
# 克隆仓库
git clone https://github.com/poxiaoyun/XiaoShi-Moha.git
cd XiaoShi-Moha/python-sdk

# 创建虚拟环境
python -m venv venv
source venv/bin/activate  # Linux/macOS
# 或
venv\Scripts\activate  # Windows

# 安装开发依赖
pip install -e .[dev]
```

### 代码格式化

```bash
# 使用 black 格式化代码
black xiaoshiai_hub tests examples

# 使用 flake8 检查代码
flake8 xiaoshiai_hub tests

# 使用 mypy 检查类型
mypy xiaoshiai_hub
```

### 构建和发布

```bash
# 构建包
python -m build

# 检查包
twine check dist/*

# 上传到 TestPyPI
twine upload --repository testpypi dist/*

# 上传到 PyPI
twine upload dist/*
```

