lfxai.utils.metrics module

class AverageMeter(name)

Bases: object

Computes and stores the average and current value

reset()

update(val, n=1)

compute_metrics(data: ndarray, metrics: callable) → list

cos_saliency(saliency: ndarray) → ndarray

Computes the average cosine between different saliency maps :param saliency: saliency maps stacked together (indexed by the first tensor dimension)

Returns: cosine between saliency maps

count_activated_neurons(saliency: ndarray) → ndarray

Count the average number of neurons sensitive to a feature :param saliency: saliency maps stacked together (indexed by the first tensor dimension)

Returns: Average number of neurons sensitive to a feature

entropy_saliency(saliency: ndarray) → ndarray

Computes the entropy of different saliency maps :param saliency: saliency maps stacked together (indexed by the first tensor dimension)

Returns: Entropy between saliency maps

off_diagonal_sum(mat: ndarray) → ndarray

Computes the sum of of-diagonal matrix elements :param mat: matrix

Returns: sum of the off diagonal elements of mat

pearson_saliency(saliency: ndarray) → ndarray

Computes the average Pearson correlation between different saliency maps :param saliency: saliency maps stacked together (indexed by the first tensor dimension)

Returns: Pearson correlation between saliency maps

similarity_rates(example_importance: Tensor, labels_subtrain: Tensor, labels_test: Tensor, n_top_list: list = [1, 2, 5, 10, 20, 30, 40, 50]) → tuple

Computes the similarity rate metric (see paper) :param example_importance: attribution for each example :param labels_subtrain: labels of the train examples :param labels_test: labels of the test examples :param n_top_list: number of training examples to consider per test example

Returns: Similary rates of most and least important examples for each element in n_to_list

spearman_saliency(saliency: ndarray) → ndarray

Computes the average Spearman correlation between different saliency maps :param saliency: saliency maps stacked together (indexed by the first tensor dimension)

Returns: Spearman correlation between saliency maps