metric_tracking package

Submodules

metric_tracking.combined_per_epoch_metric module

Module containing a metrics class for tracking and aggregating several metrics related to multiple 3D MRT scans whose slices may be scattered across different batches.

class metric_tracking.combined_per_epoch_metric.CombinedPerEpochMetric(metrics, id_to_class_names, image_ids, slices_per_image, include_background_in_reduced_metrics=False, multi_label=False, confidence_levels=None, reduction_across_classes='mean', reduction_across_images='mean', ignore_nan_in_reduction=False, stage=None)

Bases: torchmetrics.metric.Metric

A metrics class that tracks the metrics of multiple 3D images whose slices may be scattered across different batches. Different metrics can be tracked per image, e.g. dice score and Hausdorff distance, and in case of multi-label segmentation tasks, the metrics can also be tracked for different confidence levels. If reduction is not “none”, the per-scan metrics are aggregated into global per-epoch metric values.

Parameters

• metrics (Iterable[str]) – A list of metric names to be tracked. Available options: "dice_score" | "sensitivity"``| ``"specificity", and "hausdorff95".

• id_to_class_names (Dict[int, str]) – A mapping of class indices to descriptive class names.

• image_ids (Iterable[str]) – List of the ids of all images for which the metrics are to be tracked.

• slices_per_image (Union[int, List[int]]) – Number of slices per 3d image. If a single integer value is provided, it is assumed that all images of the dataset have the same number of slices.

• include_background_in_reduced_metrics (bool, optional) – if False, class channel index 0 (background class) is excluded from the calculation of aggregated metrics. This parameter is used only if multi_label is set to False. Defaults to True.

• multi_label (bool, optional) – Determines whether the data is multi-label or not (default = False).

• confidence_levels – A list of confidence levels for which the metrics are to be tracked separately. This parameter is used only if multi_label is set to True. Defaults to [0.5].

Note

If multi_label is False, the prediction tensor is expected to be either the output of the final softmax layer of a segmentation model or a label-encoded, sharp prediction. In the first case, the prediction tensor must be of floating point type and have the shape \((N, C, X, Y)\) or \((N, C, X, Y, Z)\) where N = batch size and C = number of classes. In the second case, the prediction tensor must be of integer type and have the shape \((N, X, Y)\) or \((N, X, Y, Z)\). The target tensor is expected to be label-encoded in both cases. Thus, it needs to have the shape \((N, X, Y)\) or \((N, X, Y, Z)\) and be of integer type.

If multi_label is True, the prediction tensor is expected to be either the output of the final sigmoid layer of a segmentation model or a sharp prediction. In the first case, the prediction tensors needs to be of floating point type and in the second type of integer type. the In both cases the prediction tensor needs to have the shape \((N, C, X, Y)\) or \((N, C, X, Y, Z)\). The target tensor is expected to contain sharp predictions and to have the shape \((N, C, X, Y)\) or \((N, C, X, Y, Z)\).

Shape:

• Prediction: \((N, C, X, Y, ...)\) or \((N, X, Y, ...)\), where N = batch size and `C = number of

  classes` (see Notes above).

• Target: \((N, C, X, Y, ...)\), or \((N, X, Y, ...)\) (see Notes above).

• Image_ids: \((N)\), where N = batch size.

compute()

Computes the metrics for each scan and aggregates them if reduction is not “none”.

Returns

Mapping of metric names to metric values. If reduction is “none”, the keys have the form <metric name>_<confidence_level>_<image ID>. Otherwise the keys have the form <reduction>_<metric name>_<confidence_level> If “metrics_to_aggregate” is provided and reduction is not “none”, the dictionary additionally contains the keys <reduction>_aggregated_<confidence_level>.

Return type

Dict[string, Tensor]

get_metric_names()

Returns

A list containing the keys of the dictionary returned by the compute() method of this module.

Return type

List[string]

reset()

Resets internal state such that metric ready for new data.

update(prediction, target, image_ids)

Takes the prediction and target of a given batch and updates the metrics accordingly.

Parameters

• prediction (Tensor) – A batch of predictions.

• target (Tensor) – A batch of targets.

• image_ids (Iterable[string]) – Image IDs of each slice in the prediction and target batches.

metric_tracking.combined_per_image_metric module

Module containing a metrics class for tracking several metrics related to one 3d image

class metric_tracking.combined_per_image_metric.CombinedPerImageMetric(metrics, id_to_class_names, slices_per_image, multi_label=False, confidence_levels=None)

Bases: torchmetrics.metric.Metric

A metrics class that tracks several metrics related to one 3d image whose slices may be scattered across different batches. In case of multi-label segmentation tasks, the metrics can be tracked for different confidence levels.

Parameters

• metrics (Iterable[str]) – A list of metric names to be tracked. Available options: "dice_score" | "sensitivity"``| ``"specificity", and "hausdorff95".

• id_to_class_names (Dict[int, str]) – A mapping of class indices to descriptive class names.

• confidence_levels (Iterable[float], optional) – A list of confidence levels for which the metrics are to be tracked separately. This parameter is used only if multi_label is set to True. Defaults to [0.5].

• multi_label (bool, optional) – Determines whether the task is a multi-label segmentation task or not (default = False).

• slices_per_image (int) – Number of slices per 3d image.

Note

If multi_label is False, the prediction tensor is expected to be either the output of the final softmax layer of a segmentation model or a label-encoded, sharp prediction. In the first case, the prediction tensor must be of floating point type and have the shape \((C, X, Y)\) or \((C, X, Y, Z)\) where C = number of classes. In the second case, the prediction tensor must be of integer type and have the shape \((X, Y)\) or \((X, Y, Z)\). The target tensor is expected to be label-encoded in both cases. Thus, it needs to have the shape \((X, Y)\) or \((X, Y, Z)\) and be of integer type.

If multi_label is True, the prediction tensor is expected to be either the output of the final sigmoid layer of a segmentation model or a sharp prediction. In the first case, the prediction tensors needs to be of floating point type and in the second type of integer type. the In both cases the prediction tensor needs to have the shape \((C, X, Y)\) or \((C, X, Y, Z)\). The target tensor is expected to contain sharp predictions and to have the shape \((C, X, Y)\) or \((C, X, Y, Z)\).

Shape:

• Prediction: \((C, X, Y, ...)\), where C = number of classes (see Notes above).

• Target: \((X, Y, ...)\), or \((C, X, Y, ...)\) (see Notes above).

compute()

Computes the metrics for the image.

Returns

Mapping of metric names to metric values. The keys have the form <metric name>_<class_name>_<confidence_level> for multi-label segmentation tasks and <metric name>_<class_name> for single-label segmentation tasks.

Return type

Dict[string, Tensor]

reset()

Resets internal state such that metric ready for new data.

update(prediction, target, slice_ids=None)

Takes a prediction and a target slice of the 3d image and updates the metrics accordingly.

Parameters

• prediction (Tensor) – A prediction slice or a whole 3d image.

• target (Tensor) – A target slice or a whole 3d image.

• slice_ids (Union[int, List[int]]) – Indices of the image slices represented by prediction and target. Must be provided if target is a single slice or a subset of slices taken from a 3d image.

Module contents