spacr.utils
===========

.. py:module:: spacr.utils








Module Contents
---------------

.. py:data:: spacr_path

.. py:function:: merge_split_objects(mask_src, intensity_img_src=None, intensity_channel=None, perimeter_fraction=0.5, intensity_merge=False, intensity_split=False, area_multiplier=2.0, min_distance=10, min_object_area=100, intensity_threshold_method='mean', intensity_percentile=75, min_area=0, max_area=0, remove_border_objects=False, min_intensity_percentile=0, max_intensity_percentile=100, n_jobs=1, progress_callback=None, op_name='')

   Split, merge, and filter labeled objects across a directory of masks.

   Runs the split -> merge -> filter pipeline on each mask file in
   ``mask_src`` in parallel, overwriting each mask in place.

   :param mask_src: directory containing mask .tif/.tiff/.npy files.
   :param intensity_img_src: directory of matched intensity images, or ``None``.
   :param intensity_channel: channel index to pull from multi-channel intensity images.
   :param perimeter_fraction: minimum shared-boundary fraction for perimeter-based merging.
   :param intensity_merge: enable boundary-intensity-based merging.
   :param intensity_split: enable watershed splitting of oversized objects.
   :param area_multiplier: split objects with area > multiplier * median.
   :param min_distance: minimum pixel distance between watershed seeds.
   :param min_object_area: absolute minimum area below which objects are never split.
   :param intensity_threshold_method: ``'mean'`` or ``'percentile'`` boundary comparison.
   :param intensity_percentile: percentile used when method is ``'percentile'``.
   :param min_area: remove objects smaller than this (px); 0 disables.
   :param max_area: remove objects larger than this (px); 0 disables.
   :param remove_border_objects: drop objects touching the image border.
   :param min_intensity_percentile: drop objects below this intensity percentile; 0 disables.
   :param max_intensity_percentile: drop objects above this intensity percentile; 100 disables.
   :param n_jobs: parallel worker count.
   :param progress_callback: optional callback(fov_index, total, duration, op_name).
   :param op_name: label passed to the progress callback.
   :returns: None.


.. py:function:: debug(enabled=True, logger_name=None)

   Decorator that temporarily sets the given logger to DEBUG for the wrapped call.

   :param enabled: no-op when ``False``.
   :param logger_name: logger name to tweak; defaults to the function's module logger.
   :returns: decorator function.


.. py:function:: filepaths_to_database(img_paths, settings, source_folder, crop_mode)

   Insert cropped PNG filepaths and parsed well/object IDs into the measurements DB.

   :param img_paths: iterable of PNG paths for cropped objects.
   :param settings: settings dict; ``timelapse`` toggles time_id parsing.
   :param source_folder: experiment root; DB is written to ``measurements/measurements.db``.
   :param crop_mode: one of ``'cell'``, ``'nucleus'``, ``'pathogen'``, ``'cytoplasm'``.
   :returns: None.


.. py:function:: activation_maps_to_database(img_paths, source_folder, settings)

   Insert activation-map PNG paths and parsed well IDs into the dataset DB.

   :param img_paths: iterable of PNG paths for activation-map images.
   :param source_folder: experiment root; DB written to ``measurements/<dataset>.db``.
   :param settings: settings dict; must contain ``dataset`` and ``cam_type``.
   :returns: None.


.. py:function:: activation_correlations_to_database(df, img_paths, source_folder, settings)

   Merge per-image correlation stats with parsed well IDs and insert into the dataset DB.

   :param df: DataFrame of correlation stats indexed by ``file_name``.
   :param img_paths: iterable of PNG paths matching rows of ``df``.
   :param source_folder: experiment root; DB written to ``measurements/<dataset>.db``.
   :param settings: settings dict; must contain ``dataset`` and ``cam_type``.
   :returns: None.


.. py:function:: calculate_activation_correlations(inputs, activation_maps, file_names, manders_thresholds=None)

   Compute per-image Pearson and Manders correlations between input and activation channels.

   :param inputs: input image batch, tensor of shape ``(B, C, H, W)``.
   :param activation_maps: activation-map batch, tensor of shape ``(B, C, H, W)`` or ``(B, H, W)``.
   :param file_names: file names corresponding to each image in the batch.
   :param manders_thresholds: intensity percentiles used for Manders coefficients. Default ``[15, 50, 75]``.
   :returns: DataFrame with one row per image and one column per channel-pair statistic.


.. py:function:: load_settings(csv_file_path, show=False, setting_key='setting_key', setting_value='setting_value')

   Load a two-column key/value settings CSV into a Python dict.

   Values are parsed to booleans, ``None``, ints, floats, lists, tuples, or
   nested dicts where possible, otherwise kept as strings.

   :param csv_file_path: path to the CSV file.
   :param show: display the raw DataFrame for debugging.
   :param setting_key: name of the key column.
   :param setting_value: name of the value column.
   :returns: dict of parsed settings.
   :raises ValueError: if the required columns are missing.


.. py:function:: save_settings(settings, name='settings', show=False)

   Persist a settings dict to ``<src>/settings/<name>.csv``.

   Forces ``test_mode`` and ``plot`` off in the saved copy so the reloaded
   settings are safe for a full run.

   :param settings: settings dict; must contain ``src``.
   :param name: base filename; ``_list`` suffix appended when ``src`` is a list.
   :param show: display the DataFrame before writing.
   :returns: None.


.. py:function:: print_progress(files_processed, files_to_process, n_jobs, time_ls=None, batch_size=None, operation_type='')

   Print a one-line progress report with an ETA derived from mean step time.

   :param files_processed: number of items done (int or list).
   :param files_to_process: total items to do (int or list).
   :param n_jobs: parallelism used to compute ETA.
   :param time_ls: list of per-step durations (seconds) for ETA; ``None`` skips ETA.
   :param batch_size: batch size when ``time_ls`` is per batch rather than per image.
   :param operation_type: label printed alongside the progress line.
   :returns: None.


.. py:function:: reset_mp()

   Set the multiprocessing start method appropriate for the current OS.

   Uses ``spawn`` on Windows and ``fork`` on Linux/macOS.

   :returns: None.


.. py:function:: is_multiprocessing_process(process)

   Return ``True`` if ``process`` cmdline contains ``multiprocessing``.


.. py:function:: close_file_descriptors()

   Close file descriptors from 3 up to the soft NOFILE limit.


.. py:function:: close_multiprocessing_processes()

   Terminate all detected multiprocessing child processes and close file descriptors.


.. py:function:: check_mask_folder(src, mask_fldr)

   Return ``True`` if masks in ``src/masks/mask_fldr`` still need generating.

   :param src: experiment root containing ``masks/`` and ``stack/`` subfolders.
   :param mask_fldr: subfolder name under ``masks/``.
   :returns: ``True`` when the mask folder is missing or has fewer ``.npy`` files
       than the stack folder.


.. py:function:: smooth_hull_lines(cluster_data)

   Return the x, y coordinates of a smoothed convex-hull outline of a 2-D point set.

   :param cluster_data: 2-D array of point coordinates.
   :returns: tuple ``(x, y)`` of spline-interpolated hull coordinates (100 samples).


.. py:function:: mask_object_count(mask)

   Return the number of nonzero labeled objects in ``mask``.


.. py:function:: is_list_of_lists(var)

   Return ``True`` if ``var`` is a list whose every element is also a list.


.. py:function:: normalize_to_dtype(array, p1=2, p2=98, percentile_list=None, new_dtype=None)

   Percentile-normalize each channel of an image stack into the target dtype range.

   :param array: input stack of shape ``(H, W, C)``.
   :param p1: lower percentile. Default ``2``.
   :param p2: upper percentile. Default ``98``.
   :param percentile_list: per-channel ``(low, high)`` pairs; overrides ``p1``/``p2``.
   :param new_dtype: target dtype (``np.uint8``/``np.uint16`` or their string forms).
   :returns: normalized stack with the same shape as ``array``.


.. py:function:: annotate_conditions(df, cells=None, cell_loc=None, pathogens=None, pathogen_loc=None, treatments=None, treatment_loc=None)

   Annotate ``df`` with host cell, pathogen, treatment, and combined ``condition`` columns.

   :param df: DataFrame to annotate; must contain ``rowID``/``columnID``.
   :param cells: host cell types (str or list).
   :param cell_loc: per-cell-type list-of-lists of row/column identifiers.
   :param pathogens: pathogens (str or list).
   :param pathogen_loc: per-pathogen list-of-lists of row/column identifiers.
   :param treatments: treatments (str or list).
   :param treatment_loc: per-treatment list-of-lists of row/column identifiers.
   :returns: annotated DataFrame with ``host_cells``, ``pathogen``, ``treatment``, ``condition`` columns.


.. py:class:: Cache(max_size)

   LRU cache with a fixed maximum size.

   :param max_size: maximum number of entries retained; oldest is evicted on overflow.


   .. py:attribute:: cache


   .. py:attribute:: max_size


   .. py:method:: get(key)

      Return the cached value for ``key`` and mark it most-recently-used, or ``None``.



   .. py:method:: put(key, value)

      Insert ``value`` under ``key``, evicting the least-recently-used entry if full.



.. py:class:: ScaledDotProductAttention(d_k)

   Bases: :py:obj:`torch.nn.Module`


   Standard scaled dot-product attention layer.

   :param d_k: dimensionality of key/query vectors used in the scaling factor.


   .. py:attribute:: d_k


   .. py:method:: forward(Q, K, V)

      Return ``softmax(QK^T / sqrt(d_k)) V``.

      :param Q: query tensor.
      :param K: key tensor.
      :param V: value tensor.
      :returns: attention-weighted value tensor.



.. py:class:: SelfAttention(in_channels, d_k)

   Bases: :py:obj:`torch.nn.Module`


   Linear-projected self-attention layer.

   :param in_channels: input feature dimension.
   :param d_k: projected key/query/value dimension.


   .. py:attribute:: W_q


   .. py:attribute:: W_k


   .. py:attribute:: W_v


   .. py:attribute:: attention


   .. py:method:: forward(x)

      Return self-attention over ``x`` of shape ``(B, in_channels)``.



.. py:class:: EarlyFusion(in_channels)

   Bases: :py:obj:`torch.nn.Module`


   1x1 convolution that fuses input channels down to 64 feature maps.

   :param in_channels: number of input channels.


   .. py:attribute:: conv1


   .. py:method:: forward(x)

      Return the 64-channel fused feature map.



.. py:class:: SpatialAttention(kernel_size=7)

   Bases: :py:obj:`torch.nn.Module`


   Spatial attention gate that reweights features by pooled channel statistics.

   :param kernel_size: convolution kernel width used to fuse average+max pooled maps.


   .. py:attribute:: conv1


   .. py:attribute:: sigmoid


   .. py:method:: forward(x)

      Return the spatial attention map for ``x`` in ``[0, 1]``.



.. py:class:: MultiScaleBlockWithAttention(in_channels, out_channels)

   Bases: :py:obj:`torch.nn.Module`


   Dilated conv block followed by a 1x1 attention convolution.

   :param in_channels: input channel count.
   :param out_channels: output channel count.


   .. py:attribute:: dilated_conv1


   .. py:attribute:: spatial_attention


   .. py:method:: custom_forward(x)

      Apply dilated conv + ReLU followed by the 1x1 spatial attention.



   .. py:method:: forward(x)

      Forward pass; delegates to :meth:`custom_forward`.



.. py:class:: CustomCellClassifier(num_classes, pathogen_channel, use_attention, use_checkpoint, dropout_rate)

   Bases: :py:obj:`torch.nn.Module`


   Small classifier stacking :class:`EarlyFusion` and a multi-scale attention block.

   :param num_classes: output class count.
   :param pathogen_channel: reserved for downstream use; kept for API compatibility.
   :param use_attention: reserved for downstream use; kept for API compatibility.
   :param use_checkpoint: run the forward pass through ``torch.utils.checkpoint``.
   :param dropout_rate: reserved for downstream use; kept for API compatibility.


   .. py:attribute:: early_fusion


   .. py:attribute:: multi_scale_block_1


   .. py:attribute:: fc1


   .. py:attribute:: use_checkpoint


   .. py:method:: custom_forward(x)

      Return the class logits for a batch ``x`` of shape ``(B, 3, H, W)``.



   .. py:method:: forward(x)

      Forward pass, optionally through activation checkpointing.



.. py:class:: TorchModel(model_name: str = 'resnet50', pretrained: bool = True, dropout_rate: Optional[float] = None, use_checkpoint: bool = False, num_classes: int = 2, multilabel: bool = False)

   Bases: :py:obj:`torch.nn.Module`


   Thin wrapper around TorchVision classification backbones that:
     1) Loads a requested backbone with (optional) pretrained weights
     2) Strips its classification head to expose features
     3) Adds a simple Linear 'spacr' classifier with `num_classes` outputs
     4) Optionally applies dropout before the final classifier
     5) Supports gradient checkpointing
   Works with most TorchVision **classification** models. Non-classification
   (detection/segmentation) models are rejected with a clear error.


   .. py:attribute:: model_name
      :value: ''



   .. py:attribute:: use_checkpoint
      :value: False



   .. py:attribute:: num_classes
      :value: 2



   .. py:attribute:: multilabel
      :value: False



   .. py:attribute:: use_dropout


   .. py:attribute:: base_model


   .. py:attribute:: num_ftrs


   .. py:attribute:: spacr_classifier


   .. py:method:: forward(x: torch.Tensor) -> torch.Tensor

      Return classification logits of shape ``(N, num_classes)`` for input batch ``x``.



.. py:class:: TorchModel_v2(model_name: str = 'resnet50', pretrained: bool = True, dropout_rate: float = None, use_checkpoint: bool = False, num_classes: int = 2, multilabel: bool = False)

   Bases: :py:obj:`torch.nn.Module`


   TorchVision backbone with a SPACR linear head (streamlined variant of :class:`TorchModel`).

   :param model_name: TorchVision classification model to load.
   :param pretrained: use ImageNet-pretrained weights when available.
   :param dropout_rate: dropout probability applied to backbone and SPACR head; ``None`` disables.
   :param use_checkpoint: enable gradient checkpointing through the backbone.
   :param num_classes: output class count.
   :param multilabel: informational flag consumed by external loss/metrics code.


   .. py:attribute:: model_name
      :value: 'resnet50'



   .. py:attribute:: use_checkpoint
      :value: False



   .. py:attribute:: num_classes
      :value: 2



   .. py:attribute:: multilabel
      :value: False



   .. py:attribute:: base_model


   .. py:attribute:: num_ftrs


   .. py:method:: forward(x: torch.Tensor) -> torch.Tensor

      Return classification logits of shape ``(N, num_classes)`` for input batch ``x``.



.. py:class:: FocalLossWithLogits(alpha=1.0, gamma=2.0, reduction='mean')

   Bases: :py:obj:`torch.nn.Module`


   Focal loss for binary, multiclass, and multilabel targets.

   Auto-selects the BCE or cross-entropy branch based on the shapes of
   ``logits`` and ``target``:
     - binary: logits ``(N,)`` or ``(N,1)``; target float ``(N,)`` in ``{0,1}``.
     - multiclass: logits ``(N,C)``; target long ``(N,)`` in ``[0..C-1]``.
     - multilabel: logits ``(N,C)``; target float ``(N,C)`` in ``{0,1}``.

   :param alpha: class-balancing factor (float or 1-D tensor of shape ``(C,)``).
   :param gamma: focusing parameter.
   :param reduction: one of ``'mean'``, ``'sum'``, ``'none'``.


   .. py:attribute:: gamma


   .. py:attribute:: reduction
      :value: 'mean'



   .. py:attribute:: alpha
      :value: 1.0



   .. py:method:: forward(logits, target)

      Return the focal loss value for the chosen ``reduction`` mode.



.. py:class:: ResNet(resnet_type='resnet50', dropout_rate=None, use_checkpoint=False, init_weights='imagenet')

   Bases: :py:obj:`torch.nn.Module`


   ResNet backbone with a two-layer SPACR binary-classification head.

   :param resnet_type: one of ``'resnet18'``/``'resnet34'``/``'resnet50'``/``'resnet101'``/``'resnet152'``.
   :param dropout_rate: dropout probability before the final linear layer; ``None`` disables.
   :param use_checkpoint: enable gradient checkpointing through the ResNet backbone.
   :param init_weights: ``'imagenet'`` for pretrained weights or ``'none'`` for random init.


   .. py:method:: initialize_base(base_model_dict, dropout_rate, use_checkpoint, init_weights)

      Build the backbone (with or without pretrained weights) and the two-layer head.

      :param base_model_dict: dict with keys ``func`` (model constructor) and ``weights``.
      :param dropout_rate: dropout probability applied between the two linear layers.
      :param use_checkpoint: enable gradient checkpointing through the backbone.
      :param init_weights: ``'imagenet'`` or ``'none'``.
      :raises ValueError: if ``init_weights`` is neither ``'imagenet'`` nor ``'none'``.



   .. py:method:: forward(x)

      Return the flattened single-logit prediction for input batch ``x``.



.. py:function:: split_my_dataset(dataset, split_ratio=0.1)

   Randomly split ``dataset`` into ``(train, val)`` subsets.

   :param dataset: source dataset.
   :param split_ratio: fraction of samples reserved for validation.
   :returns: ``(train_subset, val_subset)``.


.. py:function:: classification_metrics(all_labels, prediction_pos_probs, loss, epoch)

   Return a one-row DataFrame of accuracy, PR-AUC, and optimal-threshold stats.

   :param all_labels: ground-truth binary labels.
   :param prediction_pos_probs: predicted positive-class probabilities.
   :param loss: loss tensor for the epoch (``.item()`` is called).
   :param epoch: epoch number used as the row index.
   :returns: DataFrame indexed by epoch with accuracy, per-class accuracy, loss,
       PR-AUC, and optimal threshold columns.
   :raises ValueError: if ``all_labels`` and ``prediction_pos_probs`` have different lengths.


.. py:function:: compute_irm_penalty(losses, dummy_w, device)

   Return the IRM penalty as the sum of squared gradient dot-products across environments.

   :param losses: per-environment loss tensors.
   :param dummy_w: scalar dummy weight used for gradient computation.
   :param device: torch device on which to compute the penalty.
   :returns: scalar IRM penalty value.


.. py:function:: choose_model(model_type: str, device: torch.device, init_weights: bool = True, dropout_rate: float = 0.0, use_checkpoint: bool = False, channels: int = 3, height: int = 224, width: int = 224, chan_dict: Optional[dict[str, Any]] = None, num_classes: int = 2, verbose: bool = False) -> Optional[torch.nn.Module]

   Instantiate a classification model by name for binary or multiclass problems.

   :param model_type: TorchVision model name (e.g. ``'resnet50'``, ``'vit_b_16'``) or ``'custom'``.
   :param device: target device (the caller moves the returned model).
   :param init_weights: load pretrained weights when available.
   :param dropout_rate: dropout probability before the classifier head (``None``/``0`` disables).
   :param use_checkpoint: enable gradient checkpointing for the backbone.
   :param channels: input channel count (pretrained backbones assume 3).
   :param height: nominal input height used for a forward sanity check.
   :param width: nominal input width used for a forward sanity check.
   :param chan_dict: optional dict forwarded to a custom model builder.
   :param num_classes: output class count; ``1`` yields a single-logit BCE head.
   :param verbose: print the model structure when ``True``.

   :returns:
       nn.Module or None if invalid.


.. py:function:: calculate_loss(output, target, prefer_focal=False, gamma=2.0, alpha=1.0, reduction='mean')

   Auto-select and return a loss for binary, multiclass, or multilabel problems.

   Dispatches based on the shapes/dtypes of ``output`` and ``target``:
     - binary: logits ``(N,1)``, float targets in ``{0,1}`` -> BCE / focal-BCE.
     - multiclass: logits ``(N,C)``, long targets ``(N,)`` -> CE / focal-CE.
     - multilabel: logits ``(N,C)``, float targets ``(N,C)`` -> BCE / focal-BCE.

   :param output: model logits.
   :param target: ground-truth labels.
   :param prefer_focal: use the focal-loss variant instead of plain CE/BCE.
   :param gamma: focal-loss focusing parameter.
   :param alpha: focal-loss class-balancing factor.
   :param reduction: one of ``'mean'``, ``'sum'``, ``'none'``.
   :returns: scalar loss tensor (or per-sample tensor when ``reduction='none'``).


.. py:function:: pick_best_model(src)

   Return the path to the ``.pth`` file in ``src`` with the highest ``epoch/acc`` tag.

   :param src: directory of checkpoint files named ``..._epoch_<N>_acc_<A>...``.
   :returns: absolute path to the top-ranked checkpoint.


.. py:function:: get_paths_from_db(df, png_df, image_type='cell_png')

   Return rows of ``png_df`` whose path contains ``image_type`` and whose ``prcfo`` is in ``df``.

   :param df: DataFrame indexed by ``prcfo`` identifiers.
   :param png_df: DataFrame of PNG metadata with ``png_path`` and ``prcfo`` columns.
   :param image_type: substring that must appear in ``png_path``.
   :returns: filtered subset of ``png_df``.


.. py:function:: save_file_lists(dst, data_set, ls)

   Write ``ls`` as a single-column CSV named ``<data_set>.csv`` under ``dst``.

   :param dst: destination directory.
   :param data_set: column name and file stem.
   :param ls: iterable of values to persist.
   :returns: None.


.. py:function:: augment_single_image(args)

   Save six augmentations of one image (original, 90/180/270 rotations, H/V flips).

   :param args: ``(img_path, dst)`` tuple.
   :returns: None.


.. py:function:: augment_images(file_paths, dst)

   Run :func:`augment_single_image` in parallel over ``file_paths``.

   :param file_paths: iterable of source image paths.
   :param dst: destination folder (created if missing).
   :returns: None.


.. py:function:: suggest_training_changes(dst, train_csv=None, val_csv=None, last_k=25, min_epochs=10, gap_threshold_acc=0.05, plateau_eps=0.001, noisy_var_ratio=0.03)

   Inspect saved training/validation progress CSVs and propose concrete training changes.

   :param dst: folder where progress CSVs were saved.
   :param train_csv: explicit train-CSV path; auto-detected in ``dst`` if ``None``.
   :param val_csv: explicit val-CSV path; auto-detected in ``dst`` if ``None``.
   :param last_k: number of recent epochs used for trend and plateau checks.
   :param min_epochs: minimum epochs before most suggestions are issued.
   :param gap_threshold_acc: accuracy generalization-gap threshold (train - val).
   :param plateau_eps: absolute slope threshold used to declare a plateau.
   :param noisy_var_ratio: instability flag threshold on ``stdev/mean`` of recent val loss.
   :returns: dict with ``summary`` (key scalars), ``flags`` (short codes),
       and ``suggestions`` (ordered suggestion strings).


.. py:function:: estimate_class_counts(loader, num_classes: int, src=None, classes=None) -> torch.Tensor

   Return per-class sample counts as a ``LongTensor`` of length ``num_classes``.

   When ``src`` and ``classes`` are provided the counts are taken from the file
   listings under ``src/<class>``, avoiding a slow DataLoader iteration on NAS.

   :param loader: fallback DataLoader iterated only when folder info is missing.
   :param num_classes: number of output classes.
   :param src: parent folder containing per-class subfolders.
   :param classes: ordered class-folder names matching ``src``.
   :returns: ``LongTensor`` of per-class counts.


.. py:function:: build_loss(loss_type: str = 'ce', num_classes: int = 2, class_counts: Optional[torch.Tensor] = None, label_smoothing: float = 0.0, focal_gamma: float = 2.0, focal_alpha: Optional[float] = None, logit_adjust_tau: float = 0.0, asl_gamma_pos: float = 0.0, asl_gamma_neg: float = 4.0, asl_clip: float = 0.05)

   Return a closure ``loss_fn(logits, target)`` implementing the requested loss.

   Supported ``loss_type`` values: ``'ce'``, ``'ce_smooth'``, ``'ce_weighted'``,
   ``'focal_ce'``, ``'bce'``, ``'focal_bce'``, ``'logit_adjust_ce'``, ``'asl'``, ``'auto'``.
   ``num_classes==1`` selects binary (BCE variants); ``>=2`` selects multiclass (CE variants).

   :param loss_type: loss identifier (see above).
   :param num_classes: output class count.
   :param class_counts: per-class sample counts used to derive weights or logit adjustment.
   :param label_smoothing: label-smoothing epsilon for ``ce_smooth``.
   :param focal_gamma: focal-loss focusing parameter.
   :param focal_alpha: focal-loss class-balancing factor (float or per-class tensor).
   :param logit_adjust_tau: strength of the Menon-et-al. logit adjustment; 0 disables.
   :param asl_gamma_pos: asymmetric-loss gamma for positives.
   :param asl_gamma_neg: asymmetric-loss gamma for negatives.
   :param asl_clip: asymmetric-loss negative-probability clip.
   :returns: ``loss_fn(logits, target)`` callable returning a scalar tensor.
   :raises ValueError: if ``loss_type`` is unknown or incompatible with ``num_classes``.


.. py:function:: augment_classes(dst, nc, pc, generate=True, move=True)

   Augment negative and positive class images and split them into train/test folders.

   :param dst: destination root; augmented images land under ``aug_nc``/``aug_pc`` and
       move into ``aug/{train,test}/{nc,pc}``.
   :param nc: negative-class source image paths.
   :param pc: positive-class source image paths.
   :param generate: run augmentation before moving files.
   :param move: split augmented images into train/test folders.
   :returns: None.


.. py:function:: annotate_predictions(csv_loc)

   Read prediction CSV and add plate/well/field/object columns plus a ``cond`` label.

   :param csv_loc: path to a predictions CSV with a ``path`` column of PNG paths.
   :returns: DataFrame enriched with parsed metadata and a ``cond`` column
       (``'screen'``/``'pc'``/``'nc'`` from the plate/well convention).


.. py:function:: initiate_counter(counter_, lock_)

   Initialize shared multiprocessing ``counter`` and ``lock`` globals.

   :param counter_: shared ``multiprocessing.Value`` counter.
   :param lock_: shared ``multiprocessing.Lock`` guarding the counter.
   :returns: None.


.. py:function:: add_images_to_tar(paths_chunk, tar_path, total_images)

   Add ``paths_chunk`` images to ``tar_path``, updating the shared counter for progress.

   :param paths_chunk: list of image paths to add.
   :param tar_path: destination tar archive path.
   :param total_images: overall image count used to render progress.
   :returns: None.


.. py:function:: generate_fraction_map(df, gene_column, min_frequency=0.0)

   Return a wells-by-genes fraction matrix, dropping columns below ``min_frequency``.

   :param df: long-format DataFrame with ``prc``, ``count``, ``well_read_sum`` columns.
   :param gene_column: column identifying the gene/guide.
   :param min_frequency: drop columns whose maximum fraction is below this cutoff.
   :returns: DataFrame indexed by ``prc`` with per-gene fractions.


.. py:function:: fishers_odds(df, threshold=0.5, phenotyp_col='mean_pred')

   Fisher's exact test per mutant column against a binarized phenotype label.

   :param df: DataFrame with per-mutant presence columns plus ``phenotyp_col``.
   :param threshold: cutoff below which ``phenotyp_col`` is called "high phenotype".
   :param phenotyp_col: name of the phenotype column.
   :returns: DataFrame with columns ``Mutant``, ``OddsRatio``, ``PValue``, ``AdjustedPValue``.


.. py:function:: model_metrics(model)

   Print RMSE/MAE/Durbin-Watson and show residual/QQ/scale-location diagnostic plots.

   :param model: fitted statsmodels regression result.
   :returns: None.


.. py:function:: check_multicollinearity(x)

   Checks multicollinearity of the predictors by computing the VIF.


.. py:function:: lasso_reg(merged_df, alpha_value=0.01, reg_type='lasso')

   Fit Lasso or Ridge on one-hot-encoded gene/grna/plate/row/column predictors.

   :param merged_df: DataFrame with ``gene``, ``grna``, ``plateID``, ``rowID``, ``columnID``, ``pred``.
   :param alpha_value: regularization strength.
   :param reg_type: ``'lasso'`` or ``'ridge'``.
   :returns: DataFrame with ``Feature`` and ``Coefficient`` columns.


.. py:function:: MLR(merged_df, refine_model)

   Fit a multiple-linear regression on gene:grna interactions plus plate/row/column terms.

   :param merged_df: DataFrame with ``gene``, ``grna``, ``plate``, ``row``, ``column``, ``pred`` columns.
   :param refine_model: refit after removing outliers by residuals and Cook's distance.
   :returns: tuple ``(max_effects, max_effects_pvalues, model, df)``.


.. py:function:: get_files_from_dir(dir_path, file_extension='*')

   Return glob matches for ``dir_path/file_extension``.


.. py:function:: create_circular_mask(h, w, center=None, radius=None)

   Return a boolean circular mask of shape ``(h, w)`` centered on ``center``.

   :param h: image height.
   :param w: image width.
   :param center: ``(x, y)`` center; defaults to the image middle.
   :param radius: circle radius; defaults to the largest circle fitting inside.
   :returns: boolean ndarray where ``True`` marks pixels within ``radius``.


.. py:function:: apply_mask(image, output_value=0)

   Zero out (or set to ``output_value``) pixels outside a circular mask fit to ``image``.


.. py:function:: invert_image(image)

   Return the intensity-inverted image, using the dtype max as the pivot.


.. py:function:: resize_images_and_labels(images, labels, target_height, target_width, show_example=True)

   Resize aligned image/label lists to ``target_height`` x ``target_width``.

   :param images: iterable of source images (2-D or 3-D).
   :param labels: matching iterable of label masks, or ``None``.
   :param target_height: output height in pixels.
   :param target_width: output width in pixels.
   :param show_example: display an example of the resized pair when ``True``.
   :returns: ``(resized_images, resized_labels)`` lists.


.. py:function:: resize_labels_back(labels, orig_dims)

   Resize a list of label masks back to their original ``(width, height)``.

   :param labels: iterable of label masks.
   :param orig_dims: matching iterable of ``(width, height)`` tuples.
   :returns: list of resized label masks.
   :raises ValueError: if lengths differ or ``orig_dims`` entries are malformed.


.. py:function:: calculate_iou(mask1, mask2)

   Return the intersection-over-union of two binary masks after zero-padding to a common shape.


.. py:function:: match_masks(true_masks, pred_masks, iou_threshold)

   Greedy match each predicted mask to a still-unmatched true mask above ``iou_threshold``.

   :param true_masks: iterable of ground-truth masks.
   :param pred_masks: iterable of predicted masks.
   :param iou_threshold: minimum IoU to count as a match.
   :returns: list of ``(true_mask, pred_mask)`` matched pairs.


.. py:function:: compute_average_precision(matches, num_true_masks, num_pred_masks)

   Return ``(precision, recall)`` given match count, true count, and predicted count.


.. py:function:: pad_to_same_shape(mask1, mask2)

   Zero-pad ``mask1`` and ``mask2`` to their element-wise maximum shape.


.. py:function:: compute_ap_over_iou_thresholds(true_masks, pred_masks, iou_thresholds)

   Return the area under the precision-recall curve swept over ``iou_thresholds``.


.. py:function:: compute_segmentation_ap(true_masks, pred_masks, iou_thresholds=np.linspace(0.5, 0.95, 10))

   Return the COCO-style segmentation AP by matching connected components across IoU thresholds.


.. py:function:: jaccard_index(mask1, mask2)

   Return the Jaccard/IoU index of two binary masks.


.. py:function:: dice_coefficient(mask1, mask2)

   Return the Dice similarity of two masks, treating any nonzero value as foreground.


.. py:function:: extract_boundaries(mask, dilation_radius=1)

   Return the boundary of a binary mask via morphological dilation minus erosion.

   :param mask: label or binary mask.
   :param dilation_radius: half-width of the structuring element.
   :returns: boolean boundary mask.


.. py:function:: boundary_f1_score(mask_true, mask_pred, dilation_radius=1)

   Return the boundary F1 score between two masks with tolerance ``dilation_radius``.


.. py:function:: merge_touching_objects(mask, threshold=0.25)

   Merge touching labeled objects whose shared boundary exceeds ``threshold`` of the smaller perimeter.

   :param mask: labeled mask.
   :param threshold: fraction of the smaller perimeter required to merge.
   :returns: merged label mask.


.. py:function:: remove_intensity_objects(image, mask, intensity_threshold, mode)

   Drop labeled objects whose mean intensity is on the wrong side of ``intensity_threshold``.

   :param image: intensity image.
   :param mask: labeled mask aligned to ``image``.
   :param intensity_threshold: cutoff value.
   :param mode: ``'low'`` removes below-threshold objects, ``'high'`` removes above.
   :returns: filtered label mask.


.. py:class:: SelectChannels(channels)

   Callable transform that zeroes out image channels not present in ``channels``.

   :param channels: iterable of 1-based channel indices to keep (1=red, 2=green, 3=blue).


   .. py:attribute:: channels


.. py:class:: SaliencyMapGenerator(model)

   Generate saliency maps and predictions for a binary classifier.

   :param model: trained PyTorch model with a single-logit binary output.


   .. py:attribute:: model


   .. py:method:: compute_saliency_maps(X, y)

      Return absolute-gradient saliency maps for inputs ``X`` given labels ``y``.



   .. py:method:: compute_saliency_and_predictions(X)

      Return ``(saliency, predictions)`` computed against the model's own predicted classes.



   .. py:method:: plot_activation_grid(X, saliency, predictions, overlay=True, normalize=False)

      Render a grid overlaying saliency maps on inputs with predicted-class labels.



   .. py:method:: percentile_normalize(img, lower_percentile=2, upper_percentile=98)

      Per-channel percentile-normalize ``img`` into ``[0, 1]``.



.. py:class:: GradCAMGenerator(model, target_layer, cam_type='gradcam')

   Grad-CAM (and variants) map generator for binary classifiers.

   :param model: trained model to inspect.
   :param target_layer: dotted attribute path to the convolutional layer to probe.
   :param cam_type: variant identifier, e.g. ``'gradcam'``.


   .. py:attribute:: model


   .. py:attribute:: target_layer


   .. py:attribute:: cam_type
      :value: 'gradcam'



   .. py:attribute:: gradients
      :value: None



   .. py:attribute:: activations
      :value: None



   .. py:attribute:: target_layer_module


   .. py:method:: hook_layers()

      Register forward/backward hooks that capture activations and gradients.



   .. py:method:: get_layer(model, target_layer)

      Resolve a dotted attribute path into the referenced submodule.



   .. py:method:: compute_gradcam_maps(X, y)

      Return the min-max normalized Grad-CAM map for a single-sample batch ``X`` and label ``y``.



   .. py:method:: compute_gradcam_and_predictions(X)

      Return ``(gradcam_maps, predictions)`` for every sample in the batch ``X``.



   .. py:method:: plot_activation_grid(X, gradcam, predictions, overlay=True, normalize=False)

      Render a grid overlaying Grad-CAM maps on inputs with predicted-class labels.



   .. py:method:: percentile_normalize(img, lower_percentile=2, upper_percentile=98)

      Per-channel percentile-normalize ``img`` into ``[0, 1]``.



.. py:function:: preprocess_image(image_path, normalize=True, image_size=224, channels=None)

   Load and preprocess ``image_path`` into a batched tensor ready for classification.

   :param image_path: path to the source image.
   :param normalize: apply ImageNet mean/std normalization.
   :param image_size: square resize dimension.
   :param channels: reserved for downstream use; kept for API compatibility.
   :returns: ``(pil_image, input_tensor)`` where the tensor has shape ``(1, 3, H, W)``.


.. py:function:: class_visualization(target_y, model_path, dtype, img_size=224, channels=None, l2_reg=0.001, learning_rate=25, num_iterations=100, blur_every=10, max_jitter=16, show_every=25, class_names=None)

   Synthesize an input image that maximizes the classifier score for ``target_y``.

   :param target_y: target class index.
   :param model_path: path to the trained model checkpoint.
   :param dtype: tensor dtype; overridden internally based on CUDA availability.
   :param img_size: square image size (pixels).
   :param channels: input channels (defaults to ``[0, 1, 2]``).
   :param l2_reg: L2 regularization weight on the pixel norm.
   :param learning_rate: gradient-ascent step size.
   :param num_iterations: optimization iteration count.
   :param blur_every: interval (iterations) between periodic Gaussian blurs.
   :param max_jitter: maximum pixel jitter applied per iteration.
   :param show_every: interval (iterations) between preview plots.
   :param class_names: display names for the classes (defaults to ``['nc', 'pc']``).
   :returns: deprocessed image as a numpy array.


.. py:function:: get_submodules(model, prefix='')

   Return all dotted submodule names of ``model`` in traversal order.

   :param model: PyTorch module to walk.
   :param prefix: optional prefix prepended to returned names.
   :returns: list of dotted submodule names.


.. py:class:: GradCAM(model, target_layers=None, use_cuda=True)

   Named-hook Grad-CAM implementation for arbitrary target layers.

   :param model: trained model to inspect.
   :param target_layers: list of dotted layer names to hook.
   :param use_cuda: run the model on CUDA when available.


   .. py:attribute:: model


   .. py:attribute:: target_layers
      :value: None



   .. py:attribute:: cuda
      :value: True



   .. py:method:: forward(input)

      Return the model output for ``input``.



.. py:function:: show_cam_on_image(img, mask)

   Return ``img`` overlaid with a jet colormap of ``mask`` as an 8-bit RGB image.


.. py:function:: recommend_target_layers(model)

   Return ``([last_conv_layer], all_conv_layers)`` from ``model``.

   :param model: PyTorch module to scan for ``Conv2d`` layers.
   :returns: tuple ``(recommended, all)`` of layer-name lists.
   :raises ValueError: if the model contains no convolutional layers.


.. py:class:: IntegratedGradients(model)

   Compute integrated-gradients attributions for a classifier.

   :param model: trained PyTorch model.


   .. py:attribute:: model


   .. py:method:: generate_integrated_gradients(input_tensor, target_label_idx, baseline=None, num_steps=50)

      Return integrated gradients from ``baseline`` to ``input_tensor`` for ``target_label_idx``.

      :param input_tensor: input sample tensor.
      :param target_label_idx: target class index whose logit is attributed.
      :param baseline: reference tensor (defaults to zeros of the same shape).
      :param num_steps: number of Riemann-sum interpolation steps.
      :returns: attribution ndarray with the shape of ``input_tensor``.



.. py:function:: get_db_paths(src)

   Return the standard ``measurements/measurements.db`` paths for one or more source roots.


.. py:function:: get_sequencing_paths(src)

   Return the standard ``sequencing/sequencing_data.csv`` paths for one or more source roots.


.. py:function:: load_image_paths(c, visualize)

   Load the ``png_list`` table into a DataFrame indexed by ``prcfo`` and optionally filter by object.

   :param c: open sqlite3 cursor.
   :param visualize: object-type prefix (``'cell'``/``'nucleus'``/...) or falsy to keep all rows.
   :returns: DataFrame of PNG metadata indexed by ``prcfo``.


.. py:function:: merge_dataframes(df, image_paths_df, verbose)

   Merge ``df`` into ``image_paths_df`` on the shared ``prcfo`` index.

   :param df: feature DataFrame with a ``prcfo`` column.
   :param image_paths_df: DataFrame indexed by ``prcfo``.
   :param verbose: display the merged DataFrame.
   :returns: merged DataFrame.


.. py:function:: filter_columns(df, filter_by)

   Return ``df`` restricted to columns matching ``filter_by`` (or morphology columns).

   :param df: source DataFrame.
   :param filter_by: substring required in column names, or ``'morphology'`` to drop channel columns.
   :returns: column-filtered DataFrame.


.. py:function:: reduction_and_clustering(numeric_data, n_neighbors, min_dist, metric, eps, min_samples, clustering, reduction_method='umap', verbose=False, embedding=None, n_jobs=-1, mode='fit', model=False)

   Reduce ``numeric_data`` to 2-D and cluster the embedding.

   :param numeric_data: numeric data matrix.
   :param n_neighbors: UMAP ``n_neighbors`` or t-SNE perplexity (fraction or int).
   :param min_dist: UMAP ``min_dist``.
   :param metric: distance metric used by UMAP/DBSCAN.
   :param eps: DBSCAN ``eps``.
   :param min_samples: DBSCAN ``min_samples`` or KMeans cluster count.
   :param clustering: ``'dbscan'`` or ``'kmeans'``.
   :param reduction_method: ``'umap'`` or ``'tsne'``.
   :param verbose: print progress.
   :param embedding: precomputed embedding (skips reducer fit).
   :param n_jobs: parallel worker count.
   :param mode: ``'fit'`` to train a new reducer, otherwise transform with ``model``.
   :param model: existing reducer to reuse when ``mode != 'fit'``.
   :returns: ``(embedding, labels, reducer)``.
   :raises ValueError: on unsupported ``reduction_method`` or missing model.


.. py:function:: remove_noise(embedding, labels)

   Drop rows of ``embedding`` (and ``labels``) whose label is DBSCAN noise (``-1``).


.. py:function:: plot_embedding(embedding, image_paths, labels, image_nr, img_zoom, colors, plot_by_cluster, plot_outlines, plot_points, plot_images, smooth_lines, black_background, figuresize, dot_size, remove_image_canvas, verbose)

   Plot a 2-D embedding with cluster outlines, points, and optional image overlays.

   :returns: matplotlib ``Figure``.


.. py:function:: generate_colors(num_clusters, black_background)

   Return an RGBA color palette for ``num_clusters`` clusters with fixed accent colors first.


.. py:function:: assign_colors(unique_labels, random_colors)

   Return a ``(colors, label_to_index)`` mapping keyed by ``unique_labels``.


.. py:function:: setup_plot(figuresize, black_background)

   Return a ``(fig, ax)`` with light or dark theme applied globally.


.. py:function:: plot_clusters(ax, embedding, labels, colors, cluster_centers, plot_outlines, plot_points, smooth_lines, figuresize=10, dot_size=50, verbose=False)

   Draw cluster outlines, points, and centroid labels onto ``ax`` for a 2-D embedding.

   :param ax: Matplotlib axes to draw into.
   :param embedding: ``(N, 2)`` array of 2-D points (e.g. UMAP output).
   :param labels: length-``N`` cluster labels; ``-1`` denotes noise.
   :param colors: iterable of per-cluster colors, one per unique label.
   :param cluster_centers: iterable of ``(x, y)`` centroids, one per unique label.
   :param plot_outlines: draw a hull/smoothed outline around each cluster.
   :param plot_points: render the scatter points (otherwise plotted invisibly).
   :param smooth_lines: use a smoothed hull polyline instead of the convex hull edges.
   :param figuresize: base size in inches used to scale axis label and tick fonts. Default ``10``.
   :param dot_size: scatter marker size in points. Default ``50``.
   :param verbose: unused placeholder kept for API compatibility. Default ``False``.
   :returns: None.


.. py:function:: plot_umap_images(ax, image_paths, embedding, labels, image_nr, img_zoom, colors, plot_by_cluster, remove_image_canvas, verbose)

   Overlay sample images from ``image_paths`` on the UMAP embedding in ``ax``.


.. py:function:: plot_images_by_cluster(ax, image_paths, embedding, labels, image_nr, img_zoom, colors, cluster_indices, remove_image_canvas, verbose)

   Overlay up to ``image_nr`` images per cluster on the embedding in ``ax``.


.. py:function:: plot_image(ax, x, y, img, img_zoom, remove_image_canvas=True)

   Place a zoomed thumbnail of ``img`` at ``(x, y)`` on ``ax``.


.. py:function:: remove_canvas(img)

   Return ``img`` as an RGBA array whose alpha channel masks out zero pixels.


.. py:function:: plot_clusters_grid(embedding, labels, image_nr, image_paths, colors, figuresize, black_background, verbose)

   Plot a grid of example images per cluster label discovered in ``labels``.


.. py:function:: plot_grid(cluster_images, colors, figuresize, black_background, verbose)

   Render one column per cluster of representative images with colored borders and labels.


.. py:function:: generate_path_list_from_db(db_path, file_metadata)

   Return all ``png_path`` values from ``db_path`` optionally filtered by ``file_metadata`` substrings.

   :param db_path: path to the measurements SQLite DB.
   :param file_metadata: substring or list of substrings to LIKE-match against ``png_path``.
   :returns: list of PNG paths.


.. py:function:: correct_paths(df, base_path, folder='data')

   Rewrite PNG paths (in a DataFrame or list) so they live under ``base_path/folder``.

   :param df: DataFrame with a ``png_path`` column, or a list of paths.
   :param base_path: destination root to prepend.
   :param folder: intermediate folder name that anchors the rewrite.
   :returns: DataFrame + list, or list, mirroring the input type.


.. py:function:: delete_folder(folder_path)

   Recursively delete ``folder_path`` if it exists (files and subdirectories included).


.. py:function:: measure_test_mode(settings)

   Copy a random subset of source files into a ``test/merged`` folder when ``test_mode`` is on.

   :param settings: settings dict; must contain ``src``, ``test_mode``, ``test_nr``.
   :returns: settings dict with ``src`` optionally redirected to the test folder.


.. py:function:: preprocess_data(df, filter_by, remove_highly_correlated, log_data, exclude, column_list=False)

   Prepare a feature matrix by filtering, decorrelating, log-transforming, and scaling ``df``.

   :param df: input DataFrame.
   :param filter_by: channel of interest passed to :func:`filter_dataframe_features`; ``None`` disables.
   :param remove_highly_correlated: correlation cutoff (float) or ``True`` to use ``0.95``; ``False`` disables.
   :param log_data: apply ``log(x + 1e-6)`` to numeric columns.
   :param exclude: features to exclude from filtering.
   :param column_list: optional explicit column subset applied before selecting numeric columns.
   :returns: standard-scaled ``ndarray`` of numeric features.
   :raises ValueError: if no numeric columns remain after filtering.


.. py:function:: remove_low_variance_columns(df, threshold=0.01, verbose=False)

   Drop numeric columns whose variance is below ``threshold``.

   :param df: input DataFrame.
   :param threshold: variance cutoff.
   :param verbose: print the dropped column names.
   :returns: filtered DataFrame.


.. py:function:: remove_highly_correlated_columns(df, threshold=0.95, verbose=False)

   Drop numeric columns whose absolute correlation with a prior column exceeds ``threshold``.

   :param df: input DataFrame.
   :param threshold: correlation cutoff.
   :param verbose: print the dropped column names.
   :returns: decorrelated DataFrame.


.. py:function:: filter_dataframe_features(df, channel_of_interest, exclude=None, remove_low_variance_features=True, remove_highly_correlated_features=True, verbose=False)

   Restrict a features DataFrame to a channel of interest and clean up correlated/low-variance columns.

   :param df: input DataFrame.
   :param channel_of_interest: int, str, list, or ``'morphology'`` to select feature groups.
   :param exclude: features to drop from the final list.
   :param remove_low_variance_features: apply :func:`remove_low_variance_columns`.
   :param remove_highly_correlated_features: apply :func:`remove_highly_correlated_columns`.
   :param verbose: print filter details.
   :returns: ``(filtered_df, features)``.


.. py:function:: check_overlap(current_position, other_positions, threshold)

   Return ``True`` if ``current_position`` is within ``threshold`` of any point in ``other_positions``.


.. py:function:: find_non_overlapping_position(x, y, image_positions, threshold, max_attempts=100)

   Return a nearby ``(x, y)`` jittered position that does not collide with ``image_positions``.

   :param x: original x.
   :param y: original y.
   :param image_positions: previously placed points.
   :param threshold: minimum allowed spacing.
   :param max_attempts: retry budget before giving up.
   :returns: ``(x, y)`` tuple; original position if no non-overlapping spot is found.


.. py:function:: search_reduction_and_clustering(numeric_data, n_neighbors, min_dist, metric, eps, min_samples, clustering, reduction_method, verbose, reduction_param=None, embedding=None, n_jobs=-1)

   Variant of :func:`reduction_and_clustering` accepting extra reducer kwargs via ``reduction_param``.

   :param numeric_data: numeric data matrix.
   :param n_neighbors: UMAP ``n_neighbors`` or t-SNE perplexity (int or fraction).
   :param min_dist: UMAP ``min_dist``.
   :param metric: distance metric.
   :param eps: DBSCAN ``eps``.
   :param min_samples: DBSCAN ``min_samples`` or KMeans cluster count.
   :param clustering: ``'dbscan'`` or ``'kmeans'``.
   :param reduction_method: ``'umap'`` or ``'tsne'``.
   :param verbose: print progress.
   :param reduction_param: extra kwargs forwarded to the reducer.
   :param embedding: precomputed embedding to skip fitting.
   :param n_jobs: parallel worker count.
   :returns: ``(embedding, labels)``.
   :raises ValueError: on unsupported ``reduction_method`` or ``clustering``.


.. py:function:: load_image(image_path)

   Load and preprocess an image.


.. py:function:: extract_features(image_paths, resnet=resnet50)

   Extract features from images using a pre-trained ResNet model.


.. py:function:: check_normality(series)

   Helper function to check if a feature is normally distributed.


.. py:function:: random_forest_feature_importance(all_df, cluster_col='cluster')

   Random Forest feature importance.


.. py:function:: perform_statistical_tests(all_df, cluster_col='cluster')

   Perform ANOVA or Kruskal-Wallis tests depending on normality of features.


.. py:function:: combine_results(rf_df, anova_df, kruskal_df)

   Combine the results into a single DataFrame.


.. py:function:: cluster_feature_analysis(all_df, cluster_col='cluster')

   Perform Random Forest feature importance, ANOVA for normally distributed features,
   and Kruskal-Wallis for non-normally distributed features. Combine results into a single DataFrame.


.. py:function:: process_mask_file_adjust_cell(file_name, parasite_folder, cell_folder, nuclei_folder, organelle_folder=None, overlap_threshold=5, perimeter_threshold=30)

   Load one triple of parasite/cell/nuclei masks, merge cells in place, and return the elapsed time.

   :param file_name: mask file name (must exist in all folders).
   :param parasite_folder: folder of parasite masks.
   :param cell_folder: folder of cell masks (overwritten in place).
   :param nuclei_folder: folder of nuclei masks.
   :param organelle_folder: optional folder of organelle masks.
   :param overlap_threshold: fractional overlap threshold used by the merger.
   :param perimeter_threshold: shared-perimeter threshold used by the merger.
   :returns: elapsed seconds.
   :raises ValueError: if the matching cell or nuclei mask file is missing.


.. py:function:: adjust_cell_masks(parasite_folder, cell_folder, nuclei_folder, organelle_folder=None, overlap_threshold=5, perimeter_threshold=30, n_jobs=None)

   Run :func:`process_mask_file_adjust_cell` in parallel across matching mask files.

   :param parasite_folder: folder of parasite masks.
   :param cell_folder: folder of cell masks (overwritten in place).
   :param nuclei_folder: folder of nuclei masks.
   :param organelle_folder: optional folder of organelle masks.
   :param overlap_threshold: fractional overlap threshold used by the merger.
   :param perimeter_threshold: shared-perimeter threshold used by the merger.
   :param n_jobs: worker count; defaults to ``cpu_count() - 2``.
   :returns: None.
   :raises ValueError: if the three folders contain different numbers of files.


.. py:function:: process_masks(mask_folder, image_folder, channel, batch_size=50, n_clusters=2, plot=False)

   Cluster object morphology/intensity across a mask folder and keep the largest cluster in place.

   :param mask_folder: folder of ``.npy`` masks.
   :param image_folder: matching folder of ``.npy`` intensity images.
   :param channel: channel index used for intensity measurements.
   :param batch_size: number of files to load per batch.
   :param n_clusters: number of KMeans clusters.
   :param plot: show a PCA scatter of the clustered objects.
   :returns: None.


.. py:function:: merge_regression_res_with_metadata(results_file, metadata_file, name='_metadata')

   Merge regression outputs with gene metadata on the parsed ``gene`` column.

   :param results_file: path to a regression results CSV with a ``feature`` column.
   :param metadata_file: path to a gene metadata CSV with a ``Gene ID`` column.
   :param name: suffix appended to the output filename.
   :returns: merged DataFrame (also written to ``<results_file><name>.csv``).


.. py:function:: process_vision_results(df, threshold=0.5)

   Split image paths into well identifiers and binarize the ``pred`` column.

   :param df: DataFrame with ``path`` and ``pred`` columns.
   :param threshold: cutoff used to derive ``cv_predictions``.
   :returns: enriched DataFrame with ``plateID``, ``rowID``, ``columnID``, ``fieldID``, ``prc``, ``cv_predictions``.


.. py:function:: get_ml_results_paths(src, model_type='xgboost', channel_of_interest=1)

   Return the standard set of ML output paths for the given model and channel selection.

   :param src: experiment root.
   :param model_type: model identifier (used in the results folder name).
   :param channel_of_interest: int, list, ``'morphology'``, or ``None`` (aliased to ``all_features``).
   :returns: 10-tuple of paths ``(data, permutation, feature_importance, model_metrics,
       permutation_fig, feature_importance_fig, shap_fig, plate_heatmap, settings, ml_features)``.
   :raises ValueError: if ``channel_of_interest`` has an unsupported type.


.. py:function:: augment_image(image)

   Return a list of PIL images covering 4 rotations x 2 horizontal reflections of ``image``.


.. py:function:: augment_dataset(dataset, is_grayscale=False)

   Expand ``dataset`` by 8x through rotation and horizontal reflection of every image tensor.

   :param dataset: iterable of ``(tensor, label, filename)``.
   :param is_grayscale: informational flag (retained for API compatibility).
   :returns: list of augmented ``(tensor, label, filename)`` tuples.
   :raises TypeError: if an image is not a ``torch.Tensor``.


.. py:function:: convert_and_relabel_masks(folder_path)

   Converts all int64 npy masks in a folder to uint16 with relabeling to ensure all labels are retained.

   Parameters:
   - folder_path (str): The path to the folder containing int64 npy mask files.

   Returns:
   - None


.. py:function:: correct_masks(src)

   Convert cell masks under ``src/masks/cell_mask_stack`` to uint16 and re-stack arrays.

   Relabels masks so they fit in ``uint16`` and then re-concatenates the four
   array folders under ``src`` in the layout expected downstream.

   :param src: Root folder of a spacr run containing a ``masks/`` subfolder.
   :returns: None.


.. py:function:: count_reads_in_fastq(fastq_file)

   Return the number of reads in a gzipped FASTQ file.

   Counts total lines and divides by four (the FASTQ record length).

   :param fastq_file: Path to a ``.fastq.gz`` file.
   :returns: Integer read count.


.. py:function:: get_cuda_version()

   Return the installed CUDA toolkit version as a digit-only string, or ``None``.

   Parses the ``nvcc --version`` output; the dots are stripped so ``11.8`` becomes ``"118"``.

   :returns: Version string without dots, or ``None`` if ``nvcc`` is missing or fails.


.. py:function:: all_elements_match(list1, list2)

   Return ``True`` if every element of ``list1`` is contained in ``list2``.

   :param list1: iterable of items to test.
   :param list2: iterable acting as the reference set.
   :returns: ``True`` when ``list1`` is a subset of ``list2``, else ``False``.


.. py:function:: prepare_batch_for_segmentation(batch)

   Cast a batch to ``float32`` and per-image max-normalize any image whose max exceeds 1.

   :param batch: ``(N, ...)`` numpy array of images.
   :returns: The same array cast to ``float32`` with each image scaled to ``[0, 1]``.


.. py:function:: check_index(df, elements=5, split_char='_')

   Validate that every index label in ``df`` splits into ``elements`` parts on ``split_char``.

   :param df: DataFrame whose index labels are compound identifiers.
   :param elements: Expected number of parts after splitting. Default ``5``.
   :param split_char: Delimiter used to split each index label. Default ``'_'``.
   :returns: None.
   :raises ValueError: if any index label does not split into ``elements`` parts.


.. py:function:: map_condition(col_value, neg='c1', pos='c2', mix='c3')

   Map a column-ID value to one of ``'neg'``, ``'pos'``, ``'mix'``, or ``'screen'``.

   :param col_value: Column identifier from the plate metadata.
   :param neg: Column ID that corresponds to negative controls. Default ``'c1'``.
   :param pos: Column ID that corresponds to positive controls. Default ``'c2'``.
   :param mix: Column ID that corresponds to mixed controls. Default ``'c3'``.
   :returns: Condition label; any unlisted column returns ``'screen'``.


.. py:function:: download_models(repo_id='einarolafsson/models', retries=5, delay=5)

   Downloads all model files from Hugging Face and stores them in the `resources/models` directory
   within the installed `spacr` package.

   :param repo_id: The repository ID on Hugging Face (default is 'einarolafsson/models').
   :type repo_id: str
   :param retries: Number of retry attempts in case of failure.
   :type retries: int
   :param delay: Delay in seconds between retries.
   :type delay: int

   :returns: The local path to the downloaded models.
   :rtype: str


.. py:function:: generate_cytoplasm_mask(nucleus_mask, cell_mask)

   Generates a cytoplasm mask from nucleus and cell masks.

   Parameters:
   - nucleus_mask (np.array): Binary or segmented mask of the nucleus (non-zero values represent nucleus).
   - cell_mask (np.array): Binary or segmented mask of the whole cell (non-zero values represent cell).

   Returns:
   - cytoplasm_mask (np.array): Mask for the cytoplasm (1 for cytoplasm, 0 for nucleus and pathogens).


.. py:function:: add_column_to_database(settings)

   Adds a new column to the database table by matching on a common column from the DataFrame.
   If the column already exists in the database, it adds the column with a suffix.
   NaN values will remain as NULL in the database.

   :param settings: A dictionary containing the following keys:
                    csv_path (str): Path to the CSV file with the data to be added.
                    db_path (str): Path to the SQLite database (or connection string for other databases).
                    table_name (str): The name of the table in the database.
                    update_column (str): The name of the new column in the DataFrame to add to the database.
                    match_column (str): The common column used to match rows.
   :type settings: dict

   :returns: None


.. py:function:: fill_holes_in_mask(mask)

   Fill holes in each object in the mask while keeping objects separated.

   :param mask: A labeled mask where each object has a unique integer value.
   :type mask: np.ndarray

   :returns: A mask with holes filled and original labels preserved.
   :rtype: np.ndarray


.. py:function:: correct_metadata_column_names(df)

   Rename legacy metadata columns to the canonical spacr names.

   Handles the common aliases (``plate_name`` -> ``plateID``, ``col`` -> ``columnID``,
   ``row_name`` -> ``rowID``, ``grna_name`` -> ``grna``) and splits ``plate_row``
   into ``plateID`` and ``rowID``.

   :param df: DataFrame whose columns may use legacy names.
   :returns: The same DataFrame with columns renamed in place.


.. py:function:: control_filelist(folder, mode='columnID', values=None)

   Return filenames in ``folder`` whose row or column ID matches one of ``values``.

   The filename is split on ``_`` and the second token is inspected: characters
   after the first (``mode='columnID'``) or the leading character
   (``mode='rowID'``) are matched against ``values``.

   :param folder: Directory to scan.
   :param mode: ``'columnID'`` matches trailing digits, ``'rowID'`` matches leading letter.
       Default ``'columnID'``.
   :param values: Iterable of allowed ID strings. Defaults to ``['01', '02']``.
   :returns: List of matching filenames.


.. py:function:: rename_columns_in_db(db_path)

   Rename legacy plate-metadata columns across every table in a SQLite database.

   Renames each of ``row``/``column``/``col``/``plate``/``field``/``channel`` to
   the canonical spacr column name (``rowID``/``columnID``/``plateID``/…). Skips
   a table when the target name already exists to avoid clashes.

   :param db_path: Path to the SQLite database file to update in place.
   :returns: None.


.. py:function:: group_feature_class(df, feature_groups=None, name='compartment')

   Add a column tagging each feature with its compartment (or other group) label.

   Matches feature names against the tokens in ``feature_groups`` and stores the
   result in a new column ``name``. When ``name == 'channel'``, unmatched
   features are relabeled ``'morphology'``.

   :param df: DataFrame with a ``feature`` column.
   :param feature_groups: Iterable of substrings/regex tokens to look for in each
       feature name. Defaults to ``['cell', 'cytoplasm', 'nucleus', 'pathogen']``.
   :param name: Name of the column added to ``df``. Default ``'compartment'``.
   :returns: ``df`` with the new group column populated.


.. py:function:: delete_intermedeate_files(settings)

   Remove intermediate per-channel and stack folders under ``settings['src']``.

   Safeguarded to only run when a ``merged/`` folder is present and the ``orig/``
   backup folder exists, so raw inputs are preserved.

   :param settings: Dict with an ``'src'`` key naming the run's root folder.
   :returns: None.


.. py:function:: filter_and_save_csv(input_csv, output_csv, column_name, upper_threshold, lower_threshold)

   Reads a CSV into a DataFrame, filters rows based on a column for values > upper_threshold and < lower_threshold,
   and saves the filtered DataFrame to a new CSV file.

   :param input_csv: Path to the input CSV file.
   :type input_csv: str
   :param output_csv: Path to save the filtered CSV file.
   :type output_csv: str
   :param column_name: Column name to apply the filters on.
   :type column_name: str
   :param upper_threshold: Upper threshold for filtering (values greater than this are retained).
   :type upper_threshold: float
   :param lower_threshold: Lower threshold for filtering (values less than this are retained).
   :type lower_threshold: float

   :returns: None


.. py:function:: extract_tar_bz2_files(folder_path)

   Extracts all .tar.bz2 files in the given folder into subfolders with the same name as the tar file.

   :param folder_path: Path to the folder containing .tar.bz2 files.
   :type folder_path: str


.. py:function:: calculate_shortest_distance(df, object1, object2)

   Calculate the shortest edge-to-edge distance between two objects (e.g., pathogen and nucleus).

   Parameters:
   - df: Pandas DataFrame containing measurements
   - object1: String, name of the first object (e.g., "pathogen")
   - object2: String, name of the second object (e.g., "nucleus")

   Returns:
   - df: Pandas DataFrame with a new column for shortest edge-to-edge distance.


.. py:function:: format_path_for_system(path)

   Takes a file path and reformats it to be compatible with the current operating system.

   :param path: The file path to be formatted.
   :type path: str

   :returns: The formatted path for the current operating system.
   :rtype: str


.. py:function:: normalize_src_path(src)

   Ensures that the 'src' value is properly formatted as either a list of strings or a single string.

   :param src: The input source path(s).
   :type src: str or list

   :returns:

             A correctly formatted list if the input was a list (or string representation of a list),
                          otherwise a single string.
   :rtype: list or str


.. py:function:: generate_image_path_map(root_folder, valid_extensions=('tif', 'tiff', 'png', 'jpg', 'jpeg', 'bmp', 'czi', 'nd2', 'lif'))

   Recursively scans a folder and its subfolders for images, then creates a mapping of:
   {original_image_path: new_image_path}, where the new path includes all subfolder names.

   :param root_folder: The root directory to scan for images.
   :type root_folder: str
   :param valid_extensions: Tuple of valid image file extensions.
   :type valid_extensions: tuple

   :returns: A dictionary mapping original image paths to their new paths.
   :rtype: dict


.. py:function:: copy_images_to_consolidated(image_path_map, root_folder)

   Copies images from their original locations to a 'consolidated' folder,
   renaming them according to the generated dictionary.

   :param image_path_map: Dictionary mapping {original_path: new_path}.
   :type image_path_map: dict
   :param root_folder: The root directory where the 'consolidated' folder will be created.
   :type root_folder: str


.. py:function:: correct_metadata(df)

   Normalize a metadata DataFrame to the canonical spacr column names and plate ID form.

   Strips a duplicated ``pp`` prefix from plate IDs, promotes legacy
   ``*_name`` columns to their ID equivalents, and renames
   ``row``/``col``/``column``/``field`` (and their ``*_name`` variants) to
   ``rowID``/``columnID``/``fieldID``.

   :param df: Metadata DataFrame that may still use legacy naming.
   :returns: The DataFrame with canonical columns.


.. py:function:: remove_outliers_by_group(df, group_col, value_col, method='iqr', threshold=1.5)

   Removes outliers from `value_col` within each group defined by `group_col`.

   :param df: The input DataFrame.
   :type df: pd.DataFrame
   :param group_col: Column name to group by.
   :type group_col: str
   :param value_col: Column containing values to check for outliers.
   :type value_col: str
   :param method: 'iqr' or 'zscore'.
   :type method: str
   :param threshold: Threshold multiplier for IQR (default 1.5) or z-score.
   :type threshold: float

   :returns: A DataFrame with outliers removed.
   :rtype: pd.DataFrame


