sportslabkit.metrics#

Overview#

Classes#
`BaseCostMatrixMetric`	A base class for computing the cost matrix between trackers and
`CosineCMM`	Compute the Cosine Cost Matrix Metric between trackers and
`EuclideanCMM`	Compute the Euclidean Cost Matrix Metric between trackers and
`EuclideanCMM2D`	A base class for computing the cost matrix between trackers and
`IoUCMM`	Compute the IoU Cost Matrix Metric between trackers and detections.

Function#
`hota_score`(bboxes_track, bboxes_gt)	Calculates the HOTA metrics for one sequence.
`identity_score`(bboxes_track, bboxes_gt)	Calculates ID metrics for one sequence.
`mota_score`(bboxes_track, bboxes_gt)	Calculates CLEAR metrics for one sequence.
`ap_score`(bboxes_det_per_class, bboxes_gt_per_class, iou_threshold)	Calculate average precision.
`ap_score_range`(bboxes_det_per_class, bboxes_gt_per_class, start_threshold, end_threshold, step)	Calculate average precision in the specified range.
`convert_to_x1y1x2y2`(bbox)	Convert bbox to x1y1x2y2 format.
`iou_score`(bbox_det, bbox_gt)	-
`map_score`(bboxes_det, bboxes_gt, iou_threshold)	Calculate mean average precision.
`map_score_range`(bboxes_det, bboxes_gt, start_threshold, end_threshold, step)	Calculate mean average precision.

Classes#

class sportslabkit.metrics.BaseCostMatrixMetric[source]#

Bases: abc.ABC

A base class for computing the cost matrix between trackers and detections.

Overview

Methods#
`compute_metric`(trackers, detections)	abc Calculate the metric between trackers and detections.

Members

abstract compute_metric(trackers: collections.abc.Sequence[sportslabkit.types.tracklet.Tracklet], detections: collections.abc.Sequence[sportslabkit.types.detection.Detection]) → numpy.ndarray#

Calculate the metric between trackers and detections.

Parameters:

trackers – A list of trackers.
detections – A list of detections.

Returns:

An array of containing the metric between trackers and detections.

class sportslabkit.metrics.CosineCMM[source]#

Bases: BaseCostMatrixMetric

Compute the Cosine Cost Matrix Metric between trackers and detections.

Overview

Methods#
`compute_metric`(trackers, detections)	Calculate the metric between trackers and detections.

Members

compute_metric(trackers: collections.abc.Sequence[sportslabkit.types.tracklet.Tracklet], detections: collections.abc.Sequence[sportslabkit.types.detection.Detection]) → numpy.ndarray#

Calculate the metric between trackers and detections.

Parameters:

trackers – A list of trackers.
detections – A list of detections.

Returns:

An array of containing the metric between trackers and detections.

class sportslabkit.metrics.EuclideanCMM(use_pred_box=False, im_shape: tuple[float, float] = (1080, 1920))[source]#

Bases: BaseCostMatrixMetric

Compute the Euclidean Cost Matrix Metric between trackers and detections.

Overview

Methods#
`compute_metric`(trackers, detections)	Calculate the metric between trackers and detections.

Members

compute_metric(trackers: collections.abc.Sequence[sportslabkit.types.tracklet.Tracklet], detections: collections.abc.Sequence[sportslabkit.types.detection.Detection]) → numpy.ndarray#

Calculate the metric between trackers and detections.

Parameters:

trackers – A list of trackers.
detections – A list of detections.

Returns:

An array of containing the metric between trackers and detections.

class sportslabkit.metrics.EuclideanCMM2D(use_pred_pt=False, im_shape: tuple[float, float] = (1080, 1920))[source]#

Bases: BaseCostMatrixMetric

A base class for computing the cost matrix between trackers and detections.

Overview

Methods#
`compute_metric`(trackers, detections)	Calculate the metric between trackers and detections.

Members

compute_metric(trackers: collections.abc.Sequence[sportslabkit.types.tracklet.Tracklet], detections: collections.abc.Sequence[sportslabkit.types.detection.Detection]) → numpy.ndarray#

Calculate the metric between trackers and detections.

Parameters:

trackers – A list of trackers.
detections – A list of detections.

Returns:

An array of containing the metric between trackers and detections.

class sportslabkit.metrics.IoUCMM(use_pred_box=False)[source]#

Bases: BaseCostMatrixMetric

Compute the IoU Cost Matrix Metric between trackers and detections.

Overview

Methods#
`compute_metric`(trackers, detections)	Calculate the metric between trackers and detections.

Members

compute_metric(trackers: collections.abc.Sequence[sportslabkit.types.tracklet.Tracklet], detections: collections.abc.Sequence[sportslabkit.types.detection.Detection]) → numpy.ndarray#

Calculate the metric between trackers and detections.

Parameters:

trackers – A list of trackers.
detections – A list of detections.

Returns:

An array of containing the metric between trackers and detections.

Functions#

sportslabkit.metrics.hota_score(bboxes_track: sportslabkit.BBoxDataFrame, bboxes_gt: sportslabkit.BBoxDataFrame) → dict[str, Any][source]#

Calculates the HOTA metrics for one sequence.

Parameters:

bboxes_track (BBoxDataFrame) – Bbox Dataframe for tracking in 1 sequence
bboxes_gt (BBoxDataFrame) – Bbox Dataframe for ground truth in 1 sequence

Returns:

HOTA metrics

Return type:

dict[str, Any]

Note: The description of each evaluation indicator will be as follows: “HOTA” : The overall HOTA score. “DetA” : The detection accuracy. “AssA” : The association accuracy. “DetRe” : The detection recall. “DetPr” : The detection precision. “AssRe” : The association recall. “AssPr” : The association precision. “LocA” : The localization accuracy. “RHOTA” : The robust HOTA score. “HOTA(0)” : The overall HOTA score with a threshold of 0.5. “LocA(0)” : The localization accuracy with a threshold of 0.5. “HOTALocA(0)” : The overall HOTA score with a threshold of 0.5 and the localization accuracy with a threshold of 0.5.

This is also based on the following original paper and the github repository. paper : https://link.springer.com/article/10.1007/s11263-020-01375-2 code : https://github.com/JonathonLuiten/TrackEval

sportslabkit.metrics.identity_score(bboxes_track: sportslabkit.BBoxDataFrame, bboxes_gt: sportslabkit.BBoxDataFrame) → dict[str, Any][source]#

Calculates ID metrics for one sequence.

Parameters:

bboxes_track (BBoxDataFrame) – Bbox Dataframe for tracking in 1 sequence
bboxes_gt (BBoxDataFrame) – Bbox Dataframe for ground truth in 1 sequence

Returns:

ID metrics

Return type:

dict[str, Any]

Note: The description of each evaluation indicator will be as follows: “IDTP” : The number of true positive identities. “IDFN” : The number of false negative identities. “IDFP” : The number of false positive identities. “IDF1” : The F1 score of the identity detection. “IDR” : The recall of the identity detection. “IDP” : The precision of the identity detection.

This is also based on the following original paper and the github repository. paper : https://arxiv.org/abs/1609.01775 code : https://github.com/JonathonLuiten/TrackEval

sportslabkit.metrics.mota_score(bboxes_track: sportslabkit.BBoxDataFrame, bboxes_gt: sportslabkit.BBoxDataFrame) → dict[str, Any][source]#

Calculates CLEAR metrics for one sequence.

Parameters:

bboxes_track (BBoxDataFrame) – Bbox Dataframe for tracking in 1 sequence
bboxes_gt (BBoxDataFrame) – Bbox Dataframe for ground truth in 1 sequence

Returns:

CLEAR metrics

Return type:

dict[str, Any]

Note: The description of each evaluation indicator will be as follows: “MOTA” : Multi-Object Tracking Accuracy. “MOTAL” : MOTA with a logarithmic penalty for ID switches. “MOTP” : The average dissimilarity between all true positives and their corresponding ground truth targets.

res[“MOTP_sum”] / np.maximum(1.0, res[“CLR_TP”])

“MODA” : Multi-Object Detection Accuracy. This measure combines false positives and missed targets. “CLR_Re”: MOTA’s Recall. [“CLR_TP”] / np.maximum(1.0, res[“CLR_TP”] + res[“CLR_FN”]). “CLR_Pr”: MOTA’s Precision. [“CLR_TP”] / np.maximum(1.0, res[“CLR_TP”] + res[“CLR_FP”]). “MTR” : MT divided by the number of unique IDs in gt. “PTR” : PT divided by the number of unique IDs in gt. “MLR” : ML divided by the number of unique IDs in gt. “sMOTA” : Sum of similarity scores for matched bboxes. “CLR_TP” : Number of TPs. “CLR_FN” : Number of FNs. “CLR_FP” : Number of FPs. “IDSW” : Number of IDSW. “MT” : Mostly tracked trajectory. A target is mostly tracked if it is successfully tracked for at least 80% of its life span. “PT” : Partially tracked trajectory. All trajectories except MT and ML are PT. “ML” : Mostly lost trajectory. If a track is only recovered for less than 20% of its total length, it is said to be mostly lost (ML). “Frag” : Number of fragments. A fragment is a sub-trajectory of a track that is interrupted by a large gap in detection.

This is also based on the following original paper and the github repository. paper : https://arxiv.org/pdf/1603.00831.pdf code : https://github.com/JonathonLuiten/TrackEval

sportslabkit.metrics.ap_score(bboxes_det_per_class: list[list[float, float, float, float, float, str, str]], bboxes_gt_per_class: list[list[float, float, float, float, float, str, str]], iou_threshold: float) → dict[str, Any][source]#

Calculate average precision.

Parameters:

bboxes_det_per_class (list) – bbox of detected object per class.
bboxes_gt_per_class (list) – bbox of ground truth object per class.
IOUThreshold (float) – iou threshold. it is usually set to 50%, 75% or 95%.

Returns:

dict containing information about average precision

Return type:

ap(dict)

Note

bboxes_det_per_class: [bbox_det_1, bbox_det_2, …] bboxes_gt_per_class: [bbox_gt_1, bbox_gt_2, …]

#The elements of each bbox variable are as follows, each element basically corresponding to a property of the BoundingBox class of Object-Detection-Metrics. https://github.com/rafaelpadilla/Object-Detection-Metrics/blob/master/lib/BoundingBox.py

bbox_det_n(tuple): (xmin, ymin, width, height, confidence, class_id, image_name) bbox_gt_n(tuple): (xmin, ymin, width, height, 1.0, class_id, image_name)

xmin(float): xmin ymin(float): ymin width(float): width height(float): height confidence(float): class confidence class_id(str): class id image_name(str): image name

#index variable, this is written as a global variable in the def main() function. X_INDEX = 0 Y_INDEX = 1 W_INDEX = 2 H_INDEX = 3 CONFIDENCE_INDEX = 4 CLASS_ID_INDEX = 5 IMAGE_NAME_INDEX = 6

sportslabkit.metrics.ap_score_range(bboxes_det_per_class: list[float, float, float, float, float, str, str], bboxes_gt_per_class: list[float, float, float, float, float, str, str], start_threshold: float = 0.5, end_threshold: float = 0.95, step: float = 0.05) → float[source]#

Calculate average precision in the specified range.

Parameters:

bboxes_det_per_class (list) – bbox of detected object per class.
bboxes_gt_per_class (list) – bbox of ground truth object per class.
start_threshold (float) – start threshold of IOU. default is 0.5.
end_threshold (float) – end threshold of IOU. default is 0.95.
step (float) – step of updating threshold. default is 0.05.

Returns:

list of average precision in the specified range. ap_range(float): average of ap in the specified range.

Return type:

ap_results(list)

sportslabkit.metrics.convert_to_x1y1x2y2(bbox: list[int]) → list[int][source]#: Convert bbox to x1y1x2y2 format.

sportslabkit.metrics.iou_score(bbox_det: list[int], bbox_gt: list[int]) → float[source]#

Calculate mean average precision.

Parameters:

det_df (pd.DataFrame) – dataframe of detected object.
gt_df (pd.DataFrame) – dataframe of ground truth object.
IOUThreshold (float) – iou threshold

Returns:

mean average precision

Return type:

map(float)

Calculate mean average precision.

Parameters:

det_df (pd.DataFrame) – dataframe of detected object.
gt_df (pd.DataFrame) – dataframe of ground truth object.
start_threshold (float) – start threshold of IOU. default is 0.5.
end_threshold (float) – end threshold of IOU. default is 0.95.
step (float) – step of updating threshold. default is 0.05.

Returns:

average of map in the specified range. (0.5 to 0.95 in increments of 0.05)

Return type:

map_range(float)