Metadata-Version: 2.4
Name: nerfstudio-pixelnerf
Version: 0.0.21
Summary: Unofficial Implementation of `pixelNeRF: Neural Radiance Fields from One or Few Images` [Yu et al.] for NeRFStudio
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: dotmap>=1.3.30
Requires-Dist: gdown>=6.0.0
Requires-Dist: lpips>=0.1.4
Requires-Dist: nerfstudio
Requires-Dist: pyhocon>=0.3.63

# nerfstudio-pixel-nerf

Unofficial pixelNeRF integration for Nerfstudio. A framework for training and rendering few-shot Neural Radiance Fields using image-conditioned feature extraction.

## Install

```bash
uv pip install -e .
```

## Verify method registration

```bash
ns-train --help | grep -i pixel-nerf
```

## Dataset Format
This integration uses a custom pipeline to inject source views dynamically into the RayBundle metadata. It is compatible with the standard `NerfstudioDataParserConfig`. Point the dataparser to a scene root containing a valid `transforms.json`.

## Configure PixelNeRF

### Basic Training (Full Network Training - Default)
By default, the entire network (both ResNet encoder and NeRF MLPs) is trainable (`freeze_mode = "none"`):

```bash
ns-train pixel-nerf --data /path/to/scene_root
```

### Transfer Learning & Freezing Strategies

When using pretrained weights (or transferring priors across distinct image domains such as histology datasets), you can choose how to freeze the encoder via `--pipeline.model.freeze_mode`:

1. **Full Training (`none` - Default):**
   ```bash
   ns-train pixel-nerf \
     --data /path/to/scene_root \
     --pipeline.model.transfer_learning True \
     --pipeline.model.pretrained_ckpt_path /path/to/pretrained.pth \
     --pipeline.model.freeze_mode none
   ```

2. **Freeze Late Layers (`late`):**
   Keeps low-level stem and early layers (`conv1`, `bn1`, `layer1`, `layer2`) trainable to adapt to distinct low-level image distributions (e.g. histopathology, microscopy), while freezing high-level semantic layers (`layer3`, `layer4`):
   ```bash
   ns-train pixel-nerf \
     --data /path/to/scene_root \
     --pipeline.model.transfer_learning True \
     --pipeline.model.pretrained_ckpt_path /path/to/pretrained.pth \
     --pipeline.model.freeze_mode late
   ```

3. **Freeze Early Layers (`early`):**
   Freezes low-level feature extractors (`conv1`, `bn1`, `layer1`, `layer2`) and trains only high-level layers (`layer3`, `layer4`):
   ```bash
   ns-train pixel-nerf \
     --data /path/to/scene_root \
     --pipeline.model.transfer_learning True \
     --pipeline.model.pretrained_ckpt_path /path/to/pretrained.pth \
     --pipeline.model.freeze_mode early
   ```

4. **Freeze All Encoder Weights (`all`):**
   Freezes the entire feature extractor ResNet:
   ```bash
   ns-train pixel-nerf \
     --data /path/to/scene_root \
     --pipeline.model.transfer_learning True \
     --pipeline.model.pretrained_ckpt_path /path/to/pretrained.pth \
     --pipeline.model.freeze_mode all
   ```

### Differential Learning Rates
You can configure different learning rates for the image encoder and NeRF MLPs directly via CLI optimizer flags:

```bash
ns-train pixel-nerf \
  --data /path/to/scene_root \
  --pipeline.model.transfer_learning True \
  --optimizers.encoder.optimizer.lr 1e-5 \
  --optimizers.nerf.optimizer.lr 1e-4
```
