Metadata-Version: 2.4
Name: zibanu-django-repository
Version: 4.0.0
Summary: Zibanu repository library for django projects
Author-email: Mario Cerón <mario.ceron@cqinversiones.co>
License-Expression: GPL-3.0-or-later
Keywords: django,zibanu,library,repository,filemanager,files
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Development Status :: 5 - Production/Stable
Classifier: Framework :: Django
Classifier: Framework :: Django :: 3.2
Classifier: Framework :: Django :: 4
Classifier: Framework :: Django :: 5
Classifier: Environment :: Web Environment
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: av
Requires-Dist: Django>=3.2
Requires-Dist: Pillow
Requires-Dist: zibanu-django>=3.0.0
Requires-Dist: pymupdf
Requires-Dist: xhtml2pdf>=0.2.11
Dynamic: license-file

# Zibanu Django Repository Package

**zibanu.django.repository** — A comprehensive file repository manager for Django with versioning, metadata extraction, and thumbnail generation capabilities.

## Overview

The Zibanu Django Repository package provides a complete solution for managing file storage in Django projects. It handles file uploads, versioning, structured storage with multi-level directories, automatic metadata extraction, thumbnail generation, and file categorization. Files are stored in a structured, date-partitioned directory hierarchy and can be retrieved through a unified API.

### Key Features

- **File Versioning**: Track and manage multiple versions of uploaded files
- **Structured Storage**: Multi-level directory organization with date-based partitioning (year/month)
- **Metadata Extraction**: Automatic extraction from PDFs, images, videos, and Office documents
- **Thumbnail Generation**: Create thumbnails for image files
- **File Categorization**: Organize files using a hierarchical category system
- **File Validation**: Enforce file type restrictions and security policies
- **Content Hashing**: Multiple hash algorithm support (MD5, SHA1, SHA256, SHA384, SHA512)
- **MIME Type Detection**: Automatic MIME type resolution
- **Base64 Serialization**: JSON-compatible file transport via Base64 encoding
- **REST API**: Built-in endpoints for file retrieval and download

## Installation

### 1. Add to Django Project

Ensure the package is available in your Python environment, then add it to your Django project's `INSTALLED_APPS`:

```python
# settings.py
INSTALLED_APPS = [
    # ...
    "zibanu.django.repository",
]
```

### 2. Configure Django Settings

Set the required media directories and repository-specific settings (see **Configuration** section).

### 3. Run Migrations

```bash
python manage.py migrate zibanu.django.repository
```

## Quick Start

### Basic File Upload and Storage

```python
from django.core.files.uploadedfile import UploadedFile
from zibanu.django.repository.lib.controllers import FileController
from zibanu.django.repository.models import Category, File

# Get or create a category
category = Category.objects.first()  # or create one

# Create a controller with an uploaded file
controller = FileController(
    file_handler=uploaded_file,
    category=category,
    request=request
)

# Create the file record
file_instance = controller.create_file()

print(f"File created: {file_instance.uuid}")
print(f"File URL: {file_instance.file_url}")
```

### Retrieving and Downloading Files

```python
from zibanu.django.repository.lib.controllers import FileController

# Load an existing file
controller = FileController(uuid="your-file-uuid", request=request)

# Access file properties
print(f"File size: {controller.file_size} bytes")
print(f"File hash: {controller.file_hash}")
print(f"MIME type: {controller.mime_type}")

# Get file content as Base64 (for JSON transport)
base64_content = controller.file_content_base64

# Get file content as bytes
content = controller.file_content
```

### Working with File Versions

When versioning is enabled, each file update creates a new version:

```python
# Load a specific file version
controller = FileController(
    uuid="your-file-uuid",
    file_version=1,  # Load version 1
    request=request
)

file_content = controller.file_content
```

### Extracting Metadata

Metadata is automatically extracted for supported file types when the category has `extract_metadata=True`:

```python
controller = FileController(uuid="your-file-uuid", request=request)

# Access extracted metadata
metadata = controller.metadata
print(metadata)  # e.g., {'pages': 10, 'author': 'John Doe', ...}
```

### Generating Thumbnails

Thumbnails are created for image files when the category has `gen_thumb=True`:

```python
controller = FileController(uuid="your-file-uuid", request=request)

# Get thumbnail as bytes
thumbnail_content = controller.thumbnail_content

# Or get the file handler
thumbnail_handler = controller.thumbnail_handler
```

## Features

### File Versioning

File versioning allows tracking changes over time. When enabled via `ZB_REPOSITORY_VERSIONING=True`, each new file upload or update creates a version entry:

- Access historical versions using `FileController(file_version=N)`
- Version numbers start at 1
- All versions are stored in the `FileVersion` model with their own UUIDs

### Multi-Level Directory Structure

Files are organized in a hierarchical structure:

```
MEDIA_ROOT/
└── ZbRepository/              # ZB_REPOSITORY_ROOT_DIR
    └── ZbFiles/               # ZB_REPOSITORY_FILES_DIR
        ├── 2024/
        │   ├── 1/             # Month 1 (January)
        │   │   ├── file-uuid-1.pdf
        │   │   └── file-uuid-2.jpg
        │   └── 2/             # Month 2 (February)
        │       └── file-uuid-3.docx
        └── 2025/
            └── 3/
                └── file-uuid-4.xlsx
```

### File Categories

Categories organize files hierarchically:

```python
from zibanu.django.repository.models import Category

# Create root category
root = Category.objects.create(name="Documents", parent=None)

# Create subcategory
invoices = Category.objects.create(
    name="Invoices",
    parent=root,
    gen_thumb=False,
    extract_metadata=True,
    file_types="pdf,jpg,png"
)

# Upload file to category
controller = FileController(file_handler=file, request=request)
file_instance = controller.create_file()
file_instance.file_extended.category = invoices
file_instance.file_extended.save()
```

### File Categorization Rules

Categories support several configuration options:

- **`gen_thumb`**: Generate thumbnails for image files
- **`extract_metadata`**: Automatically extract file metadata
- **`convert_to_mp4`**: Convert compatible video formats to MP4
- **`file_types`**: Comma-separated list of allowed file extensions (e.g., `"pdf,jpg,png"`)
- **`is_public`**: Make category accessible to unauthenticated users
- **`published`**: Visibility flag for the category

#### Upload Rejection Rules

When a file is uploaded, the controller (`FileController.__validate_file`) rejects
it — raising `ValidationError` — under any of these conditions:

- The extension is in the global `ZB_REPOSITORY_INVALID_EXTENSIONS` denylist.
- The target category `is_root` — root categories never store files directly.
- The category's `files_allowed` is `False`. This depends on the two flags below:
  when `ZB_REPOSITORY_MULTILEVEL_FILES_ALLOWED` is `False`, only the deepest allowed
  level may hold files; when `ZB_REPOSITORY_MIX_FILES_CATS_ALLOWED` is `False`, a
  category that already has child categories cannot hold files.
- `category.validate_file_type(extension)` fails. When `file_types` is set, the file
  extension must match one of the comma-separated entries **exactly**
  (case-insensitive) — not as a substring.

The same category checks are re-applied by `change_category()` when moving a file to
another category.

### Metadata Extraction

Supported file types for automatic metadata extraction:

| File Type | Metadata Extracted |
|-----------|-------------------|
| PDF | Pages, author, title, creation date, producer |
| Images (JPG, PNG, etc.) | EXIF data, dimensions, color space, camera info |
| Videos (MP4, MOV, etc.) | Duration, resolution, codec, bit rate, frame rate |
| Office (DOCX, XLSX, PPTX) | Author, creation date, title, subject, number of sheets/pages |

### Thumbnail Generation

Thumbnails are automatically generated for supported image formats:

- **Supported formats**: JPG, PNG, GIF, BMP, TIFF
- **Configurable width**: Set via `ZB_REPOSITORY_THUMBNAIL_WIDTH` (default: 150px)
- **Configurable format**: Set via `ZB_REPOSITORY_THUMBNAIL_EXTENSION` (default: "png")

## API Reference

### FileController

**Path**: `zibanu.django.repository.lib.controllers.FileController`

The main controller for managing repository files.

#### Initialization

```python
# Load existing file by UUID
controller = FileController(
    uuid="file-uuid-string",
    request=request  # Optional: for resolving owner
)

# Load file with custom parameters
controller = FileController(
    uuid="file-uuid-string",
    file_version=2,  # Optional: load specific version
    thumbnail_width=200,  # Optional: thumbnail dimensions
    request=request
)

# Create new file (adding mode)
from io import BytesIO
file_like = BytesIO(b"file content")
file_like.name = "document.pdf"

controller = FileController(
    file_handler=file_like,
    category=category,
    request=request
)
```

#### Properties

| Property | Type | Description |
|----------|------|-------------|
| `is_valid` | `bool` | Whether file passed validation |
| `file_size` | `int` | File size in bytes |
| `extension` | `str` | File extension (lowercase) |
| `file_name` | `str` | Storage filename (UUID-based) |
| `file_hash` | `str` | Computed file checksum |
| `mime_type` | `str` | Detected MIME type |
| `metadata` | `dict` | Extracted metadata |
| `file_content` | `bytes` | Full file content (memory-intensive for large files) |
| `file_content_base64` | `str` | Base64-encoded content for JSON transport |
| `thumbnail_content` | `bytes` | Thumbnail bytes (if available) |
| `thumbnail_handler` | `File` | File handler for thumbnail |
| `full_filename` | `str` | Full storage path (relative) |
| `full_path_filename` | `str` | Absolute filesystem path |
| `random_file_name` | `str` | Generate random filename with current extension |
| `random_thumbnail_name` | `str` | Generate random thumbnail filename |

#### Methods

##### `create_file() -> File`

Creates a new file record from the currently loaded file handler.

**Returns**: File model instance

**Example**:
```python
controller = FileController(file_handler=uploaded_file, request=request)
file_instance = controller.create_file()
```

##### `save_from_file(file: Any, attrs: dict[str, Any]) -> None`

Saves or updates a file from a handler and attribute dictionary.

**Parameters**:
- `file`: File-like object (Django uploaded file or similar)
- `attrs`: Dictionary containing file metadata and extended properties

> **Owner required:** `save_from_file` resolves the owner from the request user
> (`controller.request.user`) or, when no request is available, from the `owner=`
> kwarg passed to `FileController`. If neither yields a user it raises
> `ValidationError("Owner not found.")`.

**Example**:
```python
attrs = {
    "description": "Invoice for April 2024",
    "file_extended": {
        "category": invoice_category,
        "title": "Invoice_2024_04.pdf",
        "published": True
    }
}

controller.save_from_file(file=uploaded_file, attrs=attrs)
```

##### `delete_file_from_storage(artifact_paths: list[str] = None) -> None`

Deletes the current file binary and, when `artifact_paths` is provided, the listed
historical version files. Those version paths are normally collected automatically
by the `pre_delete` signal receiver, so calling this method directly is rarely
needed. Thumbnails are generated on the fly and never persisted, so there is nothing
to clean up for them.

**Parameters**:
- `artifact_paths` (optional): List of absolute filesystem paths to version files to remove. Deletion is best-effort; paths that do not exist are skipped.

**Example**:
```python
controller = FileController(uuid="file-uuid", request=request)
controller.delete_file_from_storage()
```

##### `change_category(new_category_uuid: str) -> None`

Changes the file's category.

**Parameters**:
- `new_category_uuid`: UUID of the new category

**Example**:
```python
controller = FileController(uuid="file-uuid", request=request)
controller.change_category(new_category_uuid="new-category-uuid")
```

### File Model

**Path**: `zibanu.django.repository.models.File`

Core model for storing file metadata.

#### Fields

| Field | Type | Description |
|-------|------|-------------|
| `code` | CharField | Unique validation code |
| `uuid` | UUIDField | Unique file identifier |
| `file_uuid` | UUIDField | Physical file identifier (changes with versioning) |
| `owner` | ForeignKey | User who created the file |
| `generated_at` | DateTimeField | File creation timestamp |
| `description` | TextField | File description |
| `file_type` | CharField | File extension |
| `checksum` | CharField | File hash value |
| `hash_method` | CharField | Algorithm used for hashing |
| `mime_type` | CharField | Detected MIME type |
| `version` | IntegerField | Current version number |

#### Properties

```python
file = File.objects.get(uuid=uuid)

# Access computed paths
print(file.file_name)        # "uuid-hex.ext"
print(file.thumbnail_name)   # "uuid-hex.png"
print(file.file_url)         # "zbrepo/download/uuid-hex/" (ROOT_URL + DOWNLOAD_URL)
print(file.full_file_name)   # "ZbRepository/ZbFiles/2024/3/uuid.ext"
print(file.icon_class)       # Font Awesome class (e.g., "fa-regular fa-file-pdf")
```

#### Related Objects

```python
# Access extended properties
extended = file.file_extended
print(extended.title)       # User-friendly title
print(extended.category)    # Associated category
print(extended.metadata)    # Extracted metadata
print(extended.published)   # Publication status

# Access version history
for version in file.file_versions.all():
    print(f"Version {version.version}: {version.generated_at}")
```

### FileVersion Model

**Path**: `zibanu.django.repository.models.FileVersion`

Stores historical versions of files.

#### Fields

| Field | Type | Description |
|-------|------|-------------|
| `file` | ForeignKey | Reference to the main File |
| `uuid` | UUIDField | Physical file UUID for this version |
| `version` | IntegerField | Version number |
| `owner` | ForeignKey | User who created this version |
| `generated_at` | DateTimeField | Version creation timestamp |

#### Properties

```python
# Access version information
version = file.file_versions.get(version=1)
print(version.file_name)      # "version-uuid.ext"
print(version.version)        # 1
print(version.generated_at)   # DateTime
```

### Category Model

**Path**: `zibanu.django.repository.models.Category`

Hierarchical file organization.

#### Fields

| Field | Type | Description |
|-------|------|-------------|
| `name` | CharField | Category name |
| `level` | IntegerField | Hierarchy level (read-only) |
| `gen_thumb` | BooleanField | Generate thumbnails for this category |
| `extract_metadata` | BooleanField | Extract metadata for this category |
| `convert_to_mp4` | BooleanField | Convert videos to MP4 format |
| `file_types` | CharField | Comma-separated allowed extensions |
| `is_public` | BooleanField | Public category (no auth required) |
| `published` | BooleanField | Category visibility |
| `parent` | ForeignKey | Parent category (null for root) |
| `uuid` | UUIDField | Unique identifier |
| `sort_by` | IntegerField | Field to sort by (name, date, size) |
| `sort_type` | IntegerField | Sort order (ascending/descending) |

#### Properties

```python
category = Category.objects.first()

print(category.is_root)          # True if no parent
print(category.has_children)     # True if has subcategories
print(category.has_files)        # True if contains files
print(category.root_category)    # Get root ancestor
print(category.files_allowed)    # Check upload permissions
print(category.level_changed)    # Check if level was modified
```

### REST API Endpoints

Both endpoints are served by `FileViewSet` (`api/services/file.py`), routed in
`urls.py` under `app_name = "zb_repository"`, and are `GET`-only with `AllowAny`
permission (per-file access is enforced from the stored `FileExtended`/`Category`
flags). There are no POST/PUT/DELETE endpoints — the public surface is
read/download only.

#### Get File (JSON/Base64)

**Endpoint**: `GET /zbrepo/get/{file_uuid}/`  ·  action `get_file`  ·  URL name `zb_repository:file-get`

**Query Parameters**:
- `feature` (optional, default `0`): Selects the payload — `0` (FILE) returns the file content, `1` (THUMBNAIL) returns content plus its thumbnail, `3` (METADATA) returns the metadata dictionary.
- `version` (optional): Historical file version number. Only honored together with `feature=0`.

**Response** (`feature=0`):
```json
{
    "file": "base64_encoded_content",
    "hash": "hash_value"
}
```

With `feature=1` the payload also carries a `thumbnail` (Base64) key; with
`feature=3` the response body is the raw metadata dictionary.

#### Download File (Streaming)

**Endpoint**: `GET /zbrepo/download/{file_uuid}/`  ·  action `download_file`  ·  URL name `zb_repository:file-download`

The `download/` path segment is configurable via `ZB_REPOSITORY_DOWNLOAD_URL`.

**Query Parameters**:
- `feature` (optional, default `0`): `0` (FILE) streams the file, `1` (THUMBNAIL) streams the thumbnail.
- `type` (optional, default `0`): `0` (FILE) streams the raw bytes, `1` (BASE64) returns the same body Base64-encoded as `text/plain` (named with a `.b64` suffix).
- `version` (optional): Historical file version number (only with `feature=0`).
- `video_format` (optional): Target container (e.g. `mp4`) for stored videos; the file is converted once and the result cached on disk.
- `inline` (optional): Serve inline (`true`) or as an attachment (`false`, the default).

**Response**: File stream (HTTP streaming)

## Configuration

All repository settings are prefixed with `ZB_REPOSITORY_` and defined in Django settings.

### Directory Settings

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `ZB_REPOSITORY_ROOT_DIR` | str | `"ZbRepository"` | Base directory for all repository files |
| `ZB_REPOSITORY_FILES_DIR` | str | `"ZbFiles"` | Subdirectory for physical files |
| `ZB_REPOSITORY_CACHE_DIR` | str | `"ZbCache"` | Subdirectory for cached artifacts |
| `ZB_REPOSITORY_ML_DIR` | str | `"ZbML"` | Subdirectory for ML artifacts |
| `ZB_REPOSITORY_ROOT_URL` | str | `"zbrepo/"` | Base URL path for repository endpoints |

#### Example Configuration

```python
# settings.py
ZB_REPOSITORY_ROOT_DIR = "MyRepository"
ZB_REPOSITORY_FILES_DIR = "StoredFiles"
ZB_REPOSITORY_CACHE_DIR = "CachedContent"
ZB_REPOSITORY_ML_DIR = "MachineLearning"
ZB_REPOSITORY_ROOT_URL = "files/"
```

> **Cache maintenance:** The `ZB_REPOSITORY_CACHE_DIR` directory stores
> converted-video artifacts keyed by source hash and target format. It has
> **no automatic invalidation or size limit** — entries accumulate as new
> files/formats are requested. Purge it with an external scheduled job
> (e.g. remove files older than N days) to bound disk usage.

### Hashing and Security Settings

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `ZB_REPOSITORY_HASH_METHOD` | str | `"sha384"` | Primary hash algorithm |
| `ZB_REPOSITORY_ALLOWED_HASH_METHODS` | list | `["md5", "sha1", "sha224", "sha256", "sha384", "sha512"]` | Allowed hash algorithms |
| `ZB_REPOSITORY_INVALID_EXTENSIONS` | list | Conservative denylist | Blocked file extensions |

#### Example Configuration

```python
# settings.py
ZB_REPOSITORY_HASH_METHOD = "sha256"
ZB_REPOSITORY_ALLOWED_HASH_METHODS = ["sha256", "sha512", "sha384"]
ZB_REPOSITORY_INVALID_EXTENSIONS = [
    "php", "js", "py", "exe", "bat", "sh", "pl", "ps1",
    "cmd", "ts", "htm", "html", "css", "scr", "vbs", "vbe",
    "wsf", "msi", "msc", "jar", "com", "hta", "cpl", "pif",
    "reg", "inf"
]
```

### Feature Flags

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `ZB_REPOSITORY_VERSIONING` | bool | `True` | Enable file versioning |
| `ZB_REPOSITORY_MULTILEVEL_FILES_ALLOWED` | bool | `True` | Allow nested subdirectories |
| `ZB_REPOSITORY_MIX_FILES_CATS_ALLOWED` | bool | `False` | Allow mixing categories in same directory |
| `ZB_REPOSITORY_MAX_LEVEL_ALLOWED` | int | `3` | Maximum directory nesting level |

#### Example Configuration

```python
# settings.py
ZB_REPOSITORY_VERSIONING = True
ZB_REPOSITORY_MULTILEVEL_FILES_ALLOWED = True
ZB_REPOSITORY_MIX_FILES_CATS_ALLOWED = False
ZB_REPOSITORY_MAX_LEVEL_ALLOWED = 3
```

### Thumbnail Settings

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `ZB_REPOSITORY_THUMBNAIL_EXTENSION` | str | `"png"` | Thumbnail file extension |
| `ZB_REPOSITORY_THUMBNAIL_WIDTH` | int | `150` | Thumbnail width in pixels |

#### Example Configuration

```python
# settings.py
ZB_REPOSITORY_THUMBNAIL_EXTENSION = "webp"
ZB_REPOSITORY_THUMBNAIL_WIDTH = 200
```

### Download and API Settings

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `ZB_REPOSITORY_DOWNLOAD_URL` | str | `"download/"` | Base URL path for downloads |

#### Example Configuration

```python
# settings.py
ZB_REPOSITORY_DOWNLOAD_URL = "files/download/"
```

### Icon Mapping Settings

| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `ZB_REPOSITORY_ICON_CLASSES` | dict | FontAwesome mapping | Icon classes for file types |

#### Example Configuration

```python
# settings.py
ZB_REPOSITORY_ICON_CLASSES = {
    "default": "fa-regular fa-file",
    "pdf": "fa-regular fa-file-pdf",
    "word": "fa-regular fa-file-word",
    "excel": "fa-regular fa-file-excel",
    "powerpoint": "fa-regular fa-file-powerpoint",
    "image": "fa-regular fa-file-image",
}
```

## Usage Examples

### Example 1: File Upload in a Django View

```python
from django.http import JsonResponse
from django.views.decorators.http import require_http_methods
from django.contrib.auth.decorators import login_required
from zibanu.django.repository.lib.controllers import FileController
from zibanu.django.repository.models import Category

@login_required
@require_http_methods(["POST"])
def upload_file(request):
    if "file" not in request.FILES:
        return JsonResponse({"error": "No file provided"}, status=400)
    
    uploaded_file = request.FILES["file"]
    category_uuid = request.POST.get("category_uuid")
    
    try:
        category = Category.objects.get(uuid=category_uuid)
        
        controller = FileController(
            file_handler=uploaded_file,
            category=category,
            request=request
        )
        
        file_instance = controller.create_file()
        
        return JsonResponse({
            "success": True,
            "file_uuid": str(file_instance.uuid),
            "file_url": file_instance.file_url,
            "file_size": controller.file_size
        })
    
    except Exception as e:
        return JsonResponse({"error": str(e)}, status=400)
```

### Example 2: File Versioning and History

```python
from zibanu.django.repository.lib.controllers import FileController

# Load current version
current = FileController(uuid="file-uuid", request=request)
print(f"Current version: {current._instance.version}")

# Load specific older version
version_1 = FileController(
    uuid="file-uuid",
    file_version=1,
    request=request
)
print(f"Version 1 hash: {version_1.file_hash}")

# List all versions
file_instance = current._instance
for version in file_instance.file_versions.all():
    print(f"Version {version.version}: {version.generated_at}")
```

### Example 3: Batch Metadata Extraction

```python
from zibanu.django.repository.models import File

# Extract metadata from all files in a category
files = File.objects.filter(file_extended__category__name="Invoices")

for file in files:
    controller = FileController(uuid=str(file.uuid))
    metadata = controller.metadata
    
    print(f"{file.file_extended.title}: {metadata}")
```

### Example 4: Thumbnail Generation

```python
from django.http import FileResponse
from zibanu.django.repository.lib.controllers import FileController

def download_thumbnail(request, file_uuid):
    try:
        controller = FileController(uuid=file_uuid, request=request)
        
        if not controller._instance.file_extended.category.gen_thumb:
            return JsonResponse({"error": "Thumbnails not allowed"}, status=403)
        
        thumbnail_content = controller.thumbnail_content
        return FileResponse(thumbnail_content, content_type="image/png")
    
    except FileNotFoundError:
        return JsonResponse({"error": "Thumbnail not found"}, status=404)
```

### Example 5: Category Management and Organization

```python
from zibanu.django.repository.models import Category

# Create hierarchical categories
root = Category.objects.create(
    name="Documents",
    published=True
)

invoices = Category.objects.create(
    name="Invoices",
    parent=root,
    file_types="pdf,jpg,png",
    extract_metadata=True,
    published=True
)

contracts = Category.objects.create(
    name="Contracts",
    parent=root,
    file_types="pdf,docx",
    extract_metadata=True,
    published=True
)

# Query categories
print(f"Root categories: {Category.objects.filter(parent__isnull=True).count()}")
print(f"Invoices path: {invoices}")  # Documents // Invoices
print(f"Has children: {root.has_children}")
print(f"Total files: {root.has_files}")
```

### Example 6: Advanced File Operations

```python
from zibanu.django.repository.lib.controllers import FileController
from zibanu.django.repository.models import File

# Update existing file
controller = FileController(uuid="file-uuid", request=request)

attrs = {
    "description": "Updated description",
    "file_extended": {
        "title": "New Title",
        "published": True
    }
}

# Save new version
controller.save_from_file(file=new_file_handler, attrs=attrs)

# Retrieve all file information
print(f"Size: {controller.file_size} bytes")
print(f"Hash: {controller.file_hash}")
print(f"MIME: {controller.mime_type}")
print(f"Metadata: {controller.metadata}")
```

## Testing

### Run Unit Tests

```bash
# Run all repository tests
python manage.py test zibanu.django.repository

# Run specific test class
python manage.py test zibanu.django.repository.tests.TestFileController

# Run with verbose output
python manage.py test zibanu.django.repository -v 2

# Run tests with logging
python manage.py test zibanu.django.repository --debug-mode
```

### Run with Coverage

```bash
# Generate coverage report
coverage run --source='zibanu.django.repository' manage.py test zibanu.django.repository
coverage report

# Generate HTML coverage report
coverage html
# Open htmlcov/index.html in browser
```

### Testing Best Practices

- Use Django's test client for view testing
- Mock external services (file storage, image processing)
- Test both success and error scenarios
- Verify database state after operations
- Clean up test files from media directory

## Performance Considerations

### File Content Loading

The `file_content` property loads the entire file into memory. For large files:

- **Avoid** using `file_content` for files > 100MB
- **Use** `file_handler` to stream content instead
- **Prefer** HTTP streaming endpoints for downloads

#### Example of Streaming Large Files

```python
def download_large_file(request, file_uuid):
    controller = FileController(uuid=file_uuid, request=request)
    handler = controller.file_handler
    
    # Stream file instead of loading into memory
    response = FileResponse(handler)
    response['Content-Disposition'] = f'attachment; filename="{controller.file_name}"'
    return response
```

### Metadata Extraction

Metadata extraction can be CPU-intensive for:

- Large PDFs (100+ pages)
- High-resolution images (4K+)
- Long videos (1+ hour)

**Recommendation**: Schedule metadata extraction as background tasks for large files.

### Thumbnail Generation

Thumbnail generation for large images can use significant CPU and memory.

**Recommendation**: Pre-generate thumbnails during upload or use lazy generation with caching.

### Database Query Optimization

```python
# Optimized with prefetch_related and select_related
files = File.objects.prefetch_related(
    'file_versions',
    'file_extended__category',
    'owner'
).select_related('owner').all()

# More efficient than N+1 queries
for file in files:
    print(file.file_extended.title)
    print(file.owner.username)
```

### Storage Optimization

- Use solid-state drives (SSDs) for better I/O performance
- Implement file cleanup policies for old versions
- Monitor disk space usage regularly
- Consider archiving old files to external storage

## Requirements

### Python and Django Versions

- Python 3.11+
- Django 4.0+

### Core Dependencies

| Dependency | Purpose | Min Version |
|-----------|---------|------------|
| Django | Web framework | 4.0+ |
| zibanu.django | Base utilities | Latest |
| Pillow | Image processing | 8.0+ |

### Optional Dependencies

| Feature | Library | Purpose |
|---------|---------|---------|
| PDF Metadata | `pymupdf` or `pypdf` | Extract PDF metadata |
| Image Processing | `Pillow` | Metadata extraction, thumbnails |
| Video Processing | `PyAV` or `ffmpeg-python` | Video metadata, MP4 conversion |
| Advanced Imaging | `opencv-python` | Advanced image analysis |

### Media Storage

- **Minimum free space**: 100MB (recommended based on expected file volume)
- **Permissions**: Write access required on `MEDIA_ROOT`
- **Performance**: SSD recommended for concurrent operations
- **Backup**: Regular backups recommended for production environments

### System Configuration

```python
# settings.py - Django Media Configuration
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'

# Ensure proper permissions
# Linux/Mac: chmod 755 media/
```

## Troubleshooting

### "File not found" Error

**Symptom**: `ValidationError: File not found.`

**Cause**: File was deleted from filesystem but database record remains.

**Solution**:
```python
# Identify orphaned records
from zibanu.django.repository.models import File
import os
from django.conf import settings

media_root = settings.MEDIA_ROOT
for file in File.objects.all():
    full_path = os.path.join(media_root, file.full_file_name)
    if not os.path.exists(full_path):
        print(f"Orphaned: {file.uuid}")
        # Consider deleting: file.delete()
```

### Thumbnail Generation Fails

**Symptom**: `FileNotFoundError` or empty thumbnail.

**Causes**:
- Pillow not installed
- Category has `gen_thumb=False`
- Unsupported image format

**Solution**:
```bash
# Install Pillow
pip install Pillow

# Verify category settings
python manage.py shell
>>> from zibanu.django.repository.models import Category
>>> cat = Category.objects.get(id=1)
>>> print(cat.gen_thumb)  # Should be True
```

### "Invalid hash method" Error

**Symptom**: `ValidationError: Invalid hash method.`

**Cause**: Hash method not in allowed list.

**Solution**:
```python
# settings.py
from django.conf import settings

# Verify current method
print(settings.ZB_REPOSITORY_HASH_METHOD)  # Should be 'sha384' or 'sha256'

# Update if needed
ZB_REPOSITORY_HASH_METHOD = "sha256"
ZB_REPOSITORY_ALLOWED_HASH_METHODS = ["sha256", "sha384", "sha512"]
```

### Metadata Extraction Returns Empty Dict

**Symptom**: `metadata = {}` despite `extract_metadata=True`.

**Note**: Images now degrade gracefully. An unreadable EXIF field — for example a GPS
coordinate stored as an invalid `0/0` rational (a camera that wrote a GPS block
without a fix) — is simply omitted, while the format, dimensions and the rest of the
EXIF are still returned. A single bad tag no longer wipes the whole dictionary, so a
genuinely empty `{}` for an image points to a missing `Pillow`, an unreadable/corrupted
file, or an image type outside `SUPPORTED_MIME_TYPES`.

**Causes of a truly empty dict**:
- The MIME type is unsupported by design — e.g. legacy `application/msword` (`.doc`) files, which are not listed in `SUPPORTED_MIME_TYPES`.
- The required extractor library is not installed (`pymupdf` for PDF, `Pillow` for images, `PyAV` for video).
- A PDF, Office/OpenDocument or video file is corrupted or unreadable.

**Solution**:
```bash
# Install optional dependencies
pip install pymupdf Pillow PyAV

# Verify supported types
python manage.py shell
>>> from zibanu.django.repository.lib.classes import FileMetadata
>>> from zibanu.django.lib import FileUtils
>>> file = FileUtils(file_name="test.pdf", file_dir="/path")
>>> meta = FileMetadata(file)
>>> print(meta.metadata)
```

### URL Reverse Errors or 404 on Endpoints

**Symptom**: `NoReverseMatch` or file download returns 404.

**Cause**: Repository URL patterns not included in main `urls.py`.

**Solution**:
```python
# main urls.py
from django.urls import path, include

urlpatterns = [
    # ...
    path("zbrepo/", include("zibanu.django.repository.urls")),
]

# Test the URL pattern (app_name is "zb_repository")
python manage.py shell
>>> from django.urls import reverse
>>> reverse('zb_repository:file-get', args=['file-uuid'])
>>> reverse('zb_repository:file-download', args=['file-uuid'])
```

### Performance Issues with Large Files

**Symptom**: Slow uploads or downloads, high memory usage.

**Solution**:
- Use file streaming instead of loading entire content
- Implement chunked uploads for large files
- Configure separate storage backend for files
- Use CDN for file distribution

```python
# Use Django storages for external storage
STORAGES = {
    'default': {
        'BACKEND': 'storages.backends.s3boto3.S3Boto3Storage',
    },
}
```

### Missing Metadata or MIME Types

**Symptom**: `mime_type` is None or metadata extraction skipped.

**Causes**:
- `extract_metadata=False` in category
- File not properly uploaded
- Corrupted file header

**Solution**:
```python
# Enable metadata extraction
category.extract_metadata = True
category.save()

# Re-process file
controller = FileController(uuid="file-uuid", request=request)
print(controller.mime_type)
print(controller.metadata)
```

## License

Copyright © 2019-2025 CQ Inversiones SAS. All rights reserved.

Licensed under the Apache License 2.0. See LICENSE file for details.

## Support

For issues, feature requests, or contributions:

### Reporting Issues

1. **Include Information**:
   - Error traceback and full message
   - Django and Python versions
   - Relevant settings configuration
   - File UUID and file type
   - Steps to reproduce the issue

2. **Debug Information**:
```python
python manage.py shell
>>> from zibanu.django.repository.models import File
>>> file = File.objects.get(uuid="uuid")
>>> print(file.__dict__)
>>> print(file.file_extended.__dict__)
```

3. **Check Logs**:
   - Enable Django logging for `zibanu.django.repository`
   - Review media directory permissions
   - Verify disk space availability

### Contact

- **Issues**: File an issue with reproduction steps
- **Documentation**: Refer to this README
- **Community**: Consult zibanu.django documentation

# Changelog

All notable changes to `zibanu.django.repository` are documented here, one section per
principal version (`x.y.z`). Pre-releases of the same core (`-dev`, `-alpha`, `-beta`)
are folded into their section; several intermediate plain-core bumps in this package's
history (e.g. `2.0.6`–`2.0.9`, `2.1.0`–`2.1.1`, `2.2.0-alpha.0`, `3.0.4`, `3.0.6`–`3.0.7`)
were never tagged or shipped as an independent installer build and are folded the same
way, into whichever tagged release they led up to.

## [4.0.0] — 2026-10-01

### Added
- Category listing (`get_categories_files_allowed`) now respects each category's
  `published` flag when computing the allowed set.
- The file admin now persists the selected category across "Save and add another",
  and gives precedence to a category passed via the query string over a previously
  persisted one.
- Corrupted/damaged packets in a source video no longer abort the whole conversion —
  they are discarded with a warning and the rest of the frames are converted.
- Bundled OpenAPI/Swagger specs (with a DEBUG-only Swagger UI), and a refreshed
  `README.md` / `README.es.md` and public docstrings.

### Changed
- **Breaking:** the file/category manager class was renamed `File` → `FileManager`,
  and the category-tree helper `set_children_publish()` was renamed to
  `unpublish_children()` — its *publish* direction had been a silent no-op, so the
  corrected, renamed method is the first version that actually publishes children.
- **Breaking:** a category's allowed file-type restriction can no longer be bypassed
  with a substring match against the extension list; every call site now routes
  through `Category.allowed_file_types`.
- **Breaking:** `access_type` and `category.is_public` now also constrain
  **authenticated** callers, not only anonymous ones — closing a gap where a logged-in
  user could read files from a private category.
- A `File` row is now written to the database only after the physical upload
  succeeds; a failed disk write no longer leaves an orphaned database record.
- Video frame timestamps (constant-frame-rate re-timing) and audio timestamps are now
  computed explicitly instead of trusting the source container or letting the encoder
  invent them, fixing out-of-order frames and audio/video drift on re-encoded videos.
- Non-finite float values (`NaN`/`Infinity`) are sanitized out of extracted file
  metadata before it is stored or serialized.
- Image metadata extraction no longer discards the whole metadata payload when a
  single GPS coordinate is invalid — the rest of the metadata is preserved.
- A further internal hardening pass (~70 items) removed dead code and unused
  imports/parameters, corrected docstrings/messages that no longer matched behavior,
  and closed minor consistency gaps across the API services, `apps.py`, PDF/metadata
  generation, thumbnailing, video conversion, filters and enums — none of it
  user-observable beyond the items already listed here and under Fixed.

### Fixed
- Saving a new version of an existing file wrote the binary under the file's
  *original* year/month directory instead of the current one, because a
  positional-argument shift silently bound today's date to the wrong parameter.
- `get_path()` no longer resolves to a path outside its configured base directory.
- `publish_categories()` is now atomic per row and reports a per-row result instead of
  an all-or-nothing update.
- Specific `ValidationError`s raised during file/category validation are no longer
  swallowed and rewritten into a generic error by an overly broad `except Exception`.
- PDF generation now closes the source file handle before cleaning up its temporary
  files.
- A category's cached `level` is refreshed after `save()`, so a subsequent
  `level_changed` check no longer reads stale state.
- `change_category()` now validates the target category and reports an error instead
  of failing silently when it doesn't exist.
- The background file-deletion thread now handles its own exceptions, runs as a
  daemon thread, and closes its own database connection instead of leaking one.
- Thumbnail generation from an RGBA source image no longer fails silently when saved
  as JPEG.
- A ZIP handle opened while extracting Office-document metadata is now closed.
- Admin category/file forms replaced an unguarded `Category.objects.get(...)` with a
  field-scoped `ValidationError`, so a missing category shows a normal form error
  instead of a server error.
- File deletion no longer orphans version files on disk: a `pre_delete` signal
  handler now captures each version's physical path before the database cascade
  removes the `FileVersion` rows, so the background cleanup can still find them.
- Admin category filters, `get_root_category()` and `set_children_level()` no longer
  run one query per row/node (`select_related`, a cycle guard, and one query pair per
  tree depth respectively); `Category.save()` skips cascade-recalculation queries on
  insert; the file admin changelist uses `list_select_related`.

> `urls.py`'s `file_id` path converter was briefly tightened to `<uuid:file_id>` and
> then reverted back to `<str:file_id>` for compatibility before this version shipped
> — no net change for callers of the current `main`-tracked API.

## [3.0.5] — 2026-07-21

### Fixed
- Video conversion (the `video_format` query parameter) is now only applied when the
  stored file's MIME type is an actual video document; images, PDFs and other
  documents are always streamed as their original file even if a client requests a
  video format.
- Fixed conversion of `.m4v` sources — they are now force-converted to `.mp4` instead
  of failing.

## [3.0.3] — 2026-07-20

- No user-facing changes. Published minutes after `3.0.2` to finalize the version
  number for distribution; the code is unchanged from `3.0.2`.

## [3.0.2] — 2026-07-20

### Added
- `FileController` can now change a file's category after upload, not only at
  creation time.

### Changed
- **Breaking:** video metadata extraction and thumbnail generation now use PyAV
  instead of OpenCV + ffmpeg; the `opencv-python` and `ffmpeg-python` dependencies
  were dropped from the package. This also fixes swapped red/blue color channels and
  an always-zero duration that OpenCV reported for video thumbnails/metadata.
  Deployments that installed those packages specifically for this library can remove
  them.
- Video downloads are remuxed (no re-encode) when the source codec already matches
  the requested container, falling back to a full transcode only when needed; FFmpeg
  threading is enabled and the x264 preset changed to `veryfast` (roughly 4x faster
  transcodes).
- Converted videos are now cached on disk, keyed by file hash and target format,
  under the new `ZB_REPOSITORY_CACHE_DIR` setting (default `"ZbCache"`) — only the
  first request for a given file/format pays the conversion cost.
- A filesystem path can now be passed directly as the conversion source, avoiding an
  in-memory/temp-file copy (a chunked-copy fallback still applies for non-local
  storage handlers).

### Fixed
- Muxing AVI/MP3 audio into MP4 no longer raises `EINVAL`: decoded audio frame
  timestamps are cleared and re-timed by the AAC encoder instead of reused as-is.
- Fixed silent loss of the last frames of a converted video: end-of-file flush
  packets are now decoded instead of skipped.
- `get_converted_video()` returns `None` on failure instead of an unusable
  `File(None)`.
- `FileController.full_path_filename` is now a real property (it was missing its
  `@property` decorator).

## [3.0.1] — 2026-05-15

### Added
- PDF files can now generate a thumbnail (previously only image and video sources
  were supported).
- Permission validation on the (now on-demand) thumbnail-generation flow.

### Changed
- **Breaking:** thumbnails are now generated on demand instead of eagerly at upload
  time; the automatic thumbnail-creation step at save time was removed.
- `create_file` was reworked to fix the file-saving error described below and related
  file-creation issues.

### Removed
- **Breaking:** the now-unused `ZB_REPOSITORY_THUMBNAILS_DIR` setting was removed,
  along with the internal helper that persisted thumbnails at save time.

### Fixed
- Fixed an error saving a new file ("error save from file").
- Cleaned up assorted unused variables left over from earlier development.

## [3.0.0] — 2026-03-07

### Added
- Video support: thumbnail generation from a representative video frame, and
  extraction of video metadata (duration, codecs, frame size, GPS/creation info).
- A `hash` field on the file model, exposed through the extended GET/download
  services for integrity and de-duplication checks.
- File version date tracking (`generated_at`) as its own persisted field, added via
  migration independently of `created_at`.
- A configurable list of forbidden file extensions, to block risky uploads.
- Root-category creation now reports a clear validation error instead of an obscure
  one.
- Support for streaming file content directly (`file_content`) in addition to file
  paths.
- Documentation and initial scaffolding for an optional MQTT client entry point in
  `apps.py` (groundwork for the sibling `zibanu.django.mqtt` package's later
  integration; not itself a working client in this package).

### Changed
- **Breaking:** the `full_file_name` property was replaced by `full_filename` plus a
  new `full_path_filename` property, fixing an extension-handling bug in
  `file_utils.py` / `file_controller.py` in the same pass.
- Video-to-format conversion was narrowed to `.avi` sources specifically, after
  broader automatic conversion proved unreliable.

### Fixed
- Assorted errors surfaced during this development cycle around file deletion,
  thumbnail storage/generation, and file-extension handling.

> This range spans roughly seven months (Aug 2025 – Mar 2026) across ten `alpha` and
> eight `beta` pre-releases before the production tag. It is the package's biggest
> feature build-out (over 30 tracked user stories, `ZDR-3`…`ZDR-36`); the summary
> above groups them by user-visible capability rather than listing each one. The
> history also briefly touched an abandoned `2.2.0-alpha.0` / `2.1.6-beta.x` /
> `2.1.7-dev.0` line that was discarded in favor of `3.0.0` and never shipped.

## [2.1.5] — 2025-08-23

### Added
- User-configurable icon customization for categories/files, together with a switch
  from `BigAutoField` to `AutoField` for the package's auto-incrementing primary keys.

### Fixed
- Category level recalculation, including a persistent bug in nested-category level
  recalculation.
- Unpublishing a category no longer leaves the public documents it contains publicly
  reachable.

## [2.1.2] — 2025-06-21

### Added
- Signal-based auditing of user file/category access.

### Changed
- The repository's "published" filter now also applies to a category's parent
  categories, not only to leaf categories (paired with a sort-order parameter added
  to the sibling `zb_query_filter` template tag, shipped from
  `zibanu.django.template`).

### Fixed
- Root-category creation no longer fails.
- A `UniqueConstraint` on `FileVersion` that raised on Django versions older than 4.1
  was corrected for compatibility.
- A version-numbering bug when adding new files.

## [2.0.5] — 2025-04-21

### Added
- Thumbnail generation from the original image, via a new `FileThumbnail` class and
  supporting fields on the `File` model.

### Fixed
- A small `file_controller.py` bug carried over from the `2.0.4` release.

## [2.0.4] — 2025-04-05

### Fixed
- A small `file_controller.py` bug ahead of the PyPI `2.0.4` release.

## [2.0.2] — 2025-04-01

### Added
- Base `Geometry` model and its supporting enum/migration (foundation later reused by
  the split-out `zibanu.django.gis` package).

### Fixed
- `FileAdmin`: an error in `FileUtils`'s file-type validation, and a missing
  file-existence check in `file_admin_view.py`'s `clean()` that triggered file-type
  validation even when it wasn't required.

## [2.0.0] — 2025-03-14

### Added
- Initial release of `zibanu.django.repository` as its own installable package, split
  out of the `zibanu-django` monorepo.
