Advanced Usage

This guide covers advanced topics and customization options for Bio Transformations. It’s intended for users who are already familiar with the basic usage of the package and want to explore its full capabilities.

Customizing BioConverter

The BioConverter class accepts several parameters that allow you to fine-tune the bio-inspired modifications. Here’s an example of creating a BioConverter with custom settings:

from bio_transformations import BioConverter

converter = BioConverter(
    fuzzy_learning_rate_factor_nu=0.2,
    dampening_factor=0.7,
    crystal_thresh=5e-05,
    rejuvenation_parameter_dre=10.0,
    weight_splitting_Gamma=3,
    apply_dales_principle=True,
    base_lr=0.05,
    stability_factor=2.5,
    lr_variability=0.15
)

Refer to the API documentation for detailed explanations of each parameter.

Implementing Custom Activation Functions

You can implement custom activation functions for weight splitting. Here’s an example:

import torch

def custom_activation(x):
    return torch.tanh(x) * x

converter = BioConverter(
    weight_splitting_activation_function=custom_activation,
    weight_splitting_Gamma=2
)

Selective Application of Bio-Inspired Features

You can selectively apply bio-inspired features to specific layers of your model:

class CustomModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 20)
        self.fc2 = nn.Linear(20, 5)
        self.fc3 = nn.Linear(5, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        return self.fc3(x)

model = CustomModel()

# Mark fc3 to skip weight splitting
BioConverter.mark_skip_weight_splitting(model.fc3)

# Convert the model
bio_model = converter(model)

Extending BioModule with Custom Methods

You can extend the BioModule class with your own bio-inspired methods:

from bio_transformations.bio_module import BioModule

class CustomBioModule(BioModule):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def custom_bio_method(self):
        # Your custom bio-inspired logic here
        pass

# Update BioModule.exposed_functions to include your new method
BioModule.exposed_functions += ("custom_bio_method",)

# Use CustomBioModule in your BioConverter
class CustomBioConverter(BioConverter):
    def _bio_modulize(self, module):
        if isinstance(module, (nn.Linear, nn.Conv2d)):
            module.add_module('bio_mod', CustomBioModule(lambda: module, **self.bio_module_params))

# Use your custom converter
custom_converter = CustomBioConverter()
bio_model = custom_converter(model)

# Now you can use your custom method
bio_model.custom_bio_method()

Monitoring Bio-Inspired Modifications

To monitor the effects of bio-inspired modifications during training:

class MonitoredBioModule(BioModule):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.weight_changes = []

    def rejuvenate_weights(self):
        before = self.get_parent().weight.data.clone()
        super().rejuvenate_weights()
        after = self.get_parent().weight.data
        self.weight_changes.append((before - after).abs().mean().item())

# Use MonitoredBioModule in your BioConverter
# ... (similar to the CustomBioModule example above)

# After training, you can analyze the weight changes
import matplotlib.pyplot as plt

plt.plot(bio_model.fc1.bio_mod.weight_changes)
plt.title('Weight Changes Due to Rejuvenation')
plt.xlabel('Rejuvenation Events')
plt.ylabel('Average Absolute Weight Change')
plt.show()

These advanced usage examples should help you customize and extend Bio Transformations to suit your specific needs. Remember to refer to the API documentation for detailed information on each class and method.