foxai.explainer.computer_vision.algorithm package

Submodules

foxai.explainer.computer_vision.algorithm.conductance module

File with Conductance algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_conductance.py.

class foxai.explainer.computer_vision.algorithm.conductance.LayerConductanceCVExplainer[source]

Bases: Explainer

Layer Conductance algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, baselines: Union[None, int, float, Tensor] = None, additional_forward_args: Optional[Any] = None, n_steps: int = 50, method: str = 'gausslegendre', internal_batch_size: Union[None, int] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with Layer Conductance algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which layer conductance is computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
baselines –
Baselines define the starting point from which integral is computed and can be provided as:
- a single tensor, if inputs is a single tensor, with
  exactly the same dimensions as inputs or the first dimension is one and the remaining dimensions match with inputs.
- a single scalar, if inputs is a single tensor, which will
  be broadcasted for each input value in input tensor.
In the cases when baselines is not provided, we internally use zero scalar corresponding to each input tensor. Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. For a tensor, the first dimension of the tensor must correspond to the number of examples. It will be repeated for each of n_steps along the integrated path. For all other types, the given argument is used for all forward evaluations. Note that attributions are not computed with respect to these arguments. Default: None
n_steps – The number of steps used by the approximation method. Default: 50.
method – Method for approximating the integral, one of riemann_right, riemann_left, riemann_middle, riemann_trapezoid or gausslegendre. Default: gausslegendre if no method is provided.
internal_batch_size – Divides total #steps * #examples data points into chunks of size at most internal_batch_size, which are computed (forward / backward passes) sequentially. internal_batch_size must be at least equal to 2 * #examples. For DataParallel models, each batch is split among the available devices, so evaluations on each available device contain internal_batch_size / num_devices examples. If internal_batch_size is None, then all evaluations are processed in one batch. Default: None
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer inputs, otherwise it will be computed with respect to layer outputs. Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False

Returns:

Conductance of each neuron in given layer input or output. Attributions will always be the same size as the input or output of the given layer, depending on whether we attribute to the inputs or outputs of the layer which is decided by the input flag attribute_to_layer_input. Attributions are returned in a tuple if the layer inputs / outputs contain multiple tensors, otherwise a single tensor is returned.

Raises:

ValueError – if model does not contain conv layers.
RuntimeError – if attribution has shape (0)

create_explainer(model: Module, layer: Module) → LayerConductance[source]

Create explainer object.

model: The forward function of the model or any: modification of it.
layer: Layer for which attributions are computed.: Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer.

Returns:: Explainer object.

foxai.explainer.computer_vision.algorithm.deconv module

File with Deconvolution algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/guided_backprop_deconvnet.py.

class foxai.explainer.computer_vision.algorithm.deconv.BaseDeconvolutionCVExplainer[source]

Bases: Explainer

Base Deconvolution algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, **kwargs) → Tensor[source]

Generate model’s attributes with Deconvolution algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
inputs – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order, following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None

Returns:

The deconvolution attributions with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module) → Deconvolution[source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.deconv.DeconvolutionCVExplainer[source]

Bases: BaseDeconvolutionCVExplainer

Base Deconvolution algorithm explainer.

create_explainer(model: Module) → Deconvolution[source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

foxai.explainer.computer_vision.algorithm.deeplift module

File with DeepLIFT algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/deep_lift.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_deep_lift.py.

class foxai.explainer.computer_vision.algorithm.deeplift.BaseDeepLIFTCVExplainer[source]

Bases: Explainer

Base DeepLIFT algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, baselines: Union[None, int, float, Tensor] = None, additional_forward_args: Optional[Any] = None, custom_attribution_func: Union[None, Callable[[...], Tuple[Tensor, ...]]] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with DeepLIFT algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
baselines –
Baselines define reference samples that are compared with the inputs. In order to assign attribution scores DeepLift computes the differences between the inputs/outputs and corresponding references. Baselines can be provided as:
- a single tensor, if inputs is a single tensor, with
  exactly the same dimensions as inputs or the first dimension is one and the remaining dimensions match with inputs.
- a single scalar, if inputs is a single tensor, which will
  be broadcasted for each input value in input tensor.
In the cases when baselines is not provided, we internally use zero scalar corresponding to each input tensor. Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order, following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
custom_attribution_func –
A custom function for computing final attribution scores. This function can take at least one and at most three arguments with the following signature:
- custom_attribution_func(multipliers)
- custom_attribution_func(multipliers, inputs)
- custom_attribution_func(multipliers, inputs, baselines)
In case this function is not provided, we use the default logic defined as: multipliers * (inputs - baselines) It is assumed that all input arguments, multipliers, inputs and baselines are provided in tuples of same length. custom_attribution_func returns a tuple of attribution tensors that have the same length as the inputs. Default: None
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer input, otherwise it will be computed with respect to layer output. Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False

Returns:

Attribution score computed based on DeepLift rescale rule with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[DeepLift, LayerDeepLift][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of DeepLift, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by (inputs - baselines). This flag applies only if custom_attribution_func is set to None.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.deeplift.DeepLIFTCVExplainer[source]

Bases: BaseDeepLIFTCVExplainer

DeepLIFTC algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, eps: float = 1e-10, **kwargs) → Union[DeepLift, LayerDeepLift][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of DeepLift, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by (inputs - baselines). This flag applies only if custom_attribution_func is set to None.
eps – A value at which to consider output/input change significant when computing the gradients for non-linear layers. This is useful to adjust, depending on your model’s bit depth, to avoid numerical issues during the gradient computation. Default: 1e-10

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.deeplift.LayerDeepLIFTCVExplainer[source]

Bases: BaseDeepLIFTCVExplainer

Layer DeepLIFT algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, layer: Optional[Module] = None, **kwargs) → Union[DeepLift, LayerDeepLift][source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of DeepLift, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by (inputs - baselines). This flag applies only if custom_attribution_func is set to None.

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

foxai.explainer.computer_vision.algorithm.deeplift_shap module

File with DeepLIFT SHAP algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/deep_lift.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_deep_lift.py.

class foxai.explainer.computer_vision.algorithm.deeplift_shap.BaseDeepLIFTSHAPCVExplainer[source]

Bases: Explainer

Base DeepLIFT SHAP algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, baselines: Union[None, int, float, Tensor] = None, additional_forward_args: Optional[Any] = None, custom_attribution_func: Union[None, Callable[[...], Tuple[Tensor, ...]]] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with DeepLIFT SHAP algorithm explainer.

Under the hood this method is calling DeepLiftShap.attribute function and therefore uses the same arguments. Source of parameters documentation: https://github.com/pytorch/captum/blob/master/captum/attr/_core/deep_lift.py.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
baselines –
Baselines define reference samples that are compared with the inputs. In order to assign attribution scores DeepLift computes the differences between the inputs/outputs and corresponding references. Baselines can be provided as:
- a single tensor, if inputs is a single tensor, with
  exactly the same dimensions as inputs or the first dimension is one and the remaining dimensions match with inputs.
- a single scalar, if inputs is a single tensor, which will
  be broadcasted for each input value in input tensor.
In the cases when baselines is not provided, we internally use zero scalar corresponding to each input tensor. Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order, following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
custom_attribution_func –
A custom function for computing final attribution scores. This function can take at least one and at most three arguments with the following signature:
- custom_attribution_func(multipliers)
- custom_attribution_func(multipliers, inputs)
- custom_attribution_func(multipliers, inputs, baselines)
In case this function is not provided, we use the default logic defined as: multipliers * (inputs - baselines) It is assumed that all input arguments, multipliers, inputs and baselines are provided in tuples of same length. custom_attribution_func returns a tuple of attribution tensors that have the same length as the inputs. Default: None
attribute_to_layer_input – Argument present only for LayerDeepLiftShap. Indicates whether to compute the attributions with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer inputs, otherwise it will be computed with respect to layer outputs. Note that currently it assumes that both the inputs and outputs of internal layers are single tensors. Support for multiple tensors will be added later. Default: False

Returns:

Attribution score computed based on DeepLift rescale rule with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[DeepLiftShap, LayerDeepLiftShap][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of DeepLift, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by (inputs - baselines). This flag applies only if custom_attribution_func is set to None.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.deeplift_shap.DeepLIFTSHAPCVExplainer[source]

Bases: BaseDeepLIFTSHAPCVExplainer

DeepLIFTC SHAP algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[DeepLiftShap, LayerDeepLiftShap][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of LayerDeepLiftShap, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by layer activations for inputs - layer activations for baselines This flag applies only if custom_attribution_func is set to None.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.deeplift_shap.LayerDeepLIFTSHAPCVExplainer[source]

Bases: BaseDeepLIFTSHAPCVExplainer

Layer DeepLIFT SHAP algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, layer: Optional[Module] = None, **kwargs) → Union[DeepLiftShap, LayerDeepLiftShap][source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of LayerDeepLiftShap, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by layer activations for inputs - layer activations for baselines This flag applies only if custom_attribution_func is set to None.

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

foxai.explainer.computer_vision.algorithm.gradcam module

File with GradCAM algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/guided_grad_cam.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/grad_cam.py.

class foxai.explainer.computer_vision.algorithm.gradcam.BaseGradCAMCVExplainer[source]

Bases: Explainer

Base GradCAM algorithm explainer.

abstract calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate features image with GradCAM algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output in LayerGradCam. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer inputs, otherwise it will be computed with respect to layer outputs. Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False

Returns:

Element-wise product of (upsampled) GradCAM and/or Guided Backprop attributions. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned. Attributions will be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If the GradCAM attributions cannot be upsampled to the shape of a given input tensor, None is returned in the corresponding index position.

abstract create_explainer(model: Module, layer: Module, **kwargs) → Union[GuidedGradCam, LayerBaseGradCAM][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.gradcam.GuidedGradCAMCVExplainer[source]

Bases: BaseGradCAMCVExplainer

GuidedGradCAM algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, attribute_to_layer_input: bool = False, interpolate_mode: str = 'nearest', **kwargs) → Tensor[source]

Generate model’s attributes with GradCAM algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
interpolate_mode – Method for interpolation, which must be a valid input interpolation mode for torch.nn.functional. These methods are “nearest”, “area”, “linear” (3D-only), “bilinear” (4D-only), “bicubic” (4D-only), “trilinear” (5D-only) based on the number of dimensions of the chosen layer output (which must also match the number of dimensions for the input tensor). Note that the original GradCAM paper uses “bilinear” interpolation, but we default to “nearest” for applicability to any of 3D, 4D or 5D tensors. Default: “nearest”
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output in LayerGradCam. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer inputs, otherwise it will be computed with respect to layer outputs. Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False

Returns:

Element-wise product of (upsampled) GradCAM and/or Guided Backprop attributions. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned. Attributions will be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If the GradCAM attributions cannot be upsampled to the shape of a given input tensor, None is returned in the corresponding index position.

Raises:

ValueError – if model does not contain conv layers.
RuntimeError – if attributions has shape (0)

create_explainer(model: Module, layer: Module, **kwargs) → Union[GuidedGradCam, LayerBaseGradCAM][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.gradcam.LayerBaseGradCAM(target_layer: Module)[source]

Bases: ABC

Layer GradCAM for object detection task.

property activations: Tensor

abstract forward(input_img: Tensor) → Union[Tensor, ObjectDetectionOutput][source]

Forward pass of GradCAM aglorithm.

Parameters:: input_img – Input image with shape of (B, C, H, W).
Returns:: ObjectDetectionOutput object for object detection and tensor with saliency map for classification.

get_saliency_map(height: int, width: int, gradients: Tensor, activations: Tensor) → Tensor[source]

Generate saliency map.

Parameters:

height – Original image height.
width – Original image width.
gradients – Layer gradients.
activations – Layer activations.

Returns:

Saliency map.

property gradients: Tensor

class foxai.explainer.computer_vision.algorithm.gradcam.LayerGradCAMCVExplainer[source]

Bases: BaseGradCAMCVExplainer

Layer GradCAM algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, attribute_to_layer_input: bool = False, relu_attributions: bool = False, **kwargs) → Tensor[source]

Generate features image with GradCAM algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
inputs_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided. If forward_func takes multiple tensors as input, a tuple of the input tensors should be provided. It is assumed that for all given input tensors, dimension 0 corresponds to the number of examples, and if multiple input tensors are provided, the examples must be aligned appropriately.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output in LayerGradCam. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer inputs, otherwise it will be computed with respect to layer outputs. Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False
relu_attributions – Indicates whether to apply a ReLU operation on the final attribution, returning only non-negative attributions. Setting this flag to True matches the original GradCAM algorithm, otherwise, by default, both positive and negative attributions are returned. Default: False

Returns:

Element-wise product of (upsampled) GradCAM and/or Guided Backprop attributions. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned. Attributions will be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If the GradCAM attributions cannot be upsampled to the shape of a given input tensor, None is returned in the corresponding index position.

Raises:

ValueError – if model does not contain conv layers.
RuntimeError – if attributions has shape (0)

create_explainer(model: Module, layer: Module, **kwargs) → Union[GuidedGradCam, LayerBaseGradCAM][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.gradcam.LayerGradCAMObjectDetection(model: Module, target_layer: Module)[source]

Bases: LayerBaseGradCAM

Layer GradCAM for object detection task.

forward(input_img: Tensor) → Tensor[source]

Forward pass of GradCAM aglorithm.

Parameters:: input_img – Input image with shape of (B, C, H, W).
Returns:: Tensor with saliency map.

class foxai.explainer.computer_vision.algorithm.gradcam.LayerGradCAMObjectDetectionExplainer(model: BaseObjectDetector, target_layer: Module)[source]

Bases: LayerBaseGradCAM

Layer GradCAM for object detection task.

Code based on https://github.com/pooya-mohammadi/yolov5-gradcam.

forward(input_img: Tensor) → ObjectDetectionOutput[source]

Forward pass of GradCAM aglorithm.

Parameters:: input_img – Input image with shape of (B, C, H, W).
Returns:: ObjectDetectionOutput object.

foxai.explainer.computer_vision.algorithm.gradient_shap module

File with Gradient SHAP algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/gradient_shap.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_gradient_shap.py.

class foxai.explainer.computer_vision.algorithm.gradient_shap.BaseGradientSHAPCVExplainer[source]

Bases: Explainer

Base Gradient SHAP algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, baselines: Union[None, int, float, Tensor] = None, n_samples: int = 5, stdevs: Union[float, Tuple[float, ...]] = 0.0, additional_forward_args: Optional[Any] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with Gradient SHAP algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which SHAP attribution values are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
baselines –
Baselines define the starting point from which expectation is computed and can be provided as:
- a single tensor, if inputs is a single tensor, with
  the first dimension equal to the number of examples in the baselines’ distribution. The remaining dimensions must match with input tensor’s dimension starting from the second dimension.
It is recommended that the number of samples in the baselines’ tensors is larger than one.
n_samples – The number of randomly generated examples per sample in the input batch. Random examples are generated by adding gaussian random noise to each sample. Default: 5 if n_samples is not provided.
stdevs – The standard deviation of gaussian noise with zero mean that is added to each input in the batch. If stdevs is a single float value then that same value is used for all inputs. If it is a tuple, then it must have the same length as the inputs tuple. In this case, each stdev value in the stdevs tuple corresponds to the input with the same index in the inputs tuple. Default: 0.0
additional_forward_args –
If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It can contain a tuple of ND tensors or any arbitrary python type of any shape. In case of the ND tensor the first dimension of the tensor must correspond to the batch size. It will be repeated for each n_steps for each randomly generated input sample.

Note that the gradients are not computed with respect to these arguments. Default: None
attribute_to_layer_input (bool, optional) –
Indicates whether to compute the attribution with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer input, otherwise it will be computed with respect to layer output.

Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False

Returns:

Attribution score computed based on GradientSHAP with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[GradientShap, LayerGradientShap][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then this type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of gradient shap, if multiply_by_inputs is set to True, the sensitivity scores of scaled inputs are being multiplied by (inputs - baselines).

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.gradient_shap.GradientSHAPCVExplainer[source]

Bases: BaseGradientSHAPCVExplainer

Gradient SHAP algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[GradientShap, LayerGradientShap][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then this type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of gradient shap, if multiply_by_inputs is set to True, the sensitivity scores of scaled inputs are being multiplied by (inputs - baselines).

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.gradient_shap.LayerGradientSHAPCVExplainer[source]

Bases: BaseGradientSHAPCVExplainer

Layer Gradient SHAP algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, layer: Optional[Module] = None, **kwargs) → Union[GradientShap, LayerGradientShap][source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in then this type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of layer gradient shap, if multiply_by_inputs is set to True, the sensitivity scores for scaled inputs are being multiplied by layer activations for inputs - layer activations for baselines.

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

foxai.explainer.computer_vision.algorithm.guided_backprop module

File with Guided Backpropagation algorithm explainer class.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/guided_backprop_deconvnet.py.

class foxai.explainer.computer_vision.algorithm.guided_backprop.BaseGuidedBackpropCVExplainer[source]

Bases: Explainer

Base Guided Backpropagation algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, **kwargs) → Tensor[source]

Generate model’s attributes with Guided Backpropagation algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided. If forward_func takes multiple tensors as input, a tuple of the input tensors should be provided. It is assumed that for all given input tensors, dimension 0 corresponds to the number of examples (aka batch size), and if multiple input tensors are provided, the examples must be aligned appropriately.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order, following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None

Returns:

The guided backprop gradients with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, **kwargs) → GuidedBackprop[source]

Create explainer object.

Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.guided_backprop.GuidedBackpropCVExplainer[source]

Bases: BaseGuidedBackpropCVExplainer

Guided Backpropagation algorithm explainer.

create_explainer(model: Module, **kwargs) → GuidedBackprop[source]

Create explainer object.

Returns:: Explainer object.

foxai.explainer.computer_vision.algorithm.input_x_gradient module

File with Input X Gradient algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/input_x_gradient.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_gradient_x_activation.py.

class foxai.explainer.computer_vision.algorithm.input_x_gradient.BaseInputXGradientSHAPCVExplainer[source]

Bases: Explainer

Base Input X Gradient algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with Input X Gradient algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer input, otherwise it will be computed with respect to layer output. Default: False

Returns:

The input x gradient with respect to each input feature or gradient and activation for each neuron in given layer output. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, **kwargs) → Union[InputXGradient, LayerGradientXActivation][source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.input_x_gradient.InputXGradientCVExplainer[source]

Bases: BaseInputXGradientSHAPCVExplainer

Input X Gradient algorithm explainer.

create_explainer(model: Module, **kwargs) → Union[InputXGradient, LayerGradientXActivation][source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.input_x_gradient.LayerInputXGradientCVExplainer[source]

Bases: BaseInputXGradientSHAPCVExplainer

Layer Input X Gradient algorithm explainer.

create_explainer(model: Module, layer: Optional[Module] = None, multiply_by_inputs: bool = True, **kwargs) → Union[InputXGradient, LayerGradientXActivation][source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in, then this type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of layer gradient x activation, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by layer activations for inputs.

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

foxai.explainer.computer_vision.algorithm.integrated_gradients module

File with Integrated Gradients algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/integrated_gradients.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_integrated_gradients.py.

class foxai.explainer.computer_vision.algorithm.integrated_gradients.BaseIntegratedGradientsCVExplainer[source]

Bases: Explainer

Base Integrated Gradients algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, baselines: Union[None, int, float, Tensor] = None, additional_forward_args: Optional[Any] = None, n_steps: int = 50, method: str = 'gausslegendre', internal_batch_size: Union[None, int] = None, attribute_to_layer_input: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with Integrated Gradients algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which layer integrated gradients are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
baselines –
Baselines define the starting point from which integral is computed and can be provided as:
- a single tensor, if inputs is a single tensor, with
  exactly the same dimensions as inputs or the first dimension is one and the remaining dimensions match with inputs.
- a single scalar, if inputs is a single tensor, which will
  be broadcasted for each input value in input tensor.
In the cases when baselines is not provided, we internally use zero scalar corresponding to each input tensor. Default: None
additional_forward_args –
If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs.

For a tensor, the first dimension of the tensor must correspond to the number of examples. It will be repeated for each of n_steps along the integrated path. For all other types, the given argument is used for all forward evaluations.

Note that attributions are not computed with respect to these arguments. Default: None
n_steps – The number of steps used by the approximation method. Default: 50.
method – Method for approximating the integral, one of riemann_right, riemann_left, riemann_middle, riemann_trapezoid or gausslegendre. Default: gausslegendre if no method is provided.
internal_batch_size –
Divides total #steps * #examples data points into chunks of size at most internal_batch_size, which are computed (forward / backward passes) sequentially. internal_batch_size must be at least equal to #examples.

For DataParallel models, each batch is split among the available devices, so evaluations on each available device contain internal_batch_size / num_devices examples. If internal_batch_size is None, then all evaluations are processed in one batch. Default: None
attribute_to_layer_input –
Indicates whether to compute the attribution with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer input, otherwise it will be computed with respect to layer output.

Note that currently it is assumed that either the input or the output of internal layer, depending on whether we attribute to the input or output, is a single tensor. Support for multiple tensors will be added later. Default: False

Returns:

Integrated gradients with respect to layer’s inputs or outputs. Attributions will always be the same size and dimensionality as the input or output of the given layer, depending on whether we attribute to the inputs or outputs of the layer which is decided by the input flag attribute_to_layer_input.

For a single layer, attributions are returned in a tuple if the layer inputs / outputs contain multiple tensors, otherwise a single tensor is returned.

For multiple layers, attributions will always be returned as a list. Each element in this list will be equivalent to that of a single layer output, i.e. in the case that one layer, in the given layers, inputs / outputs multiple tensors: the corresponding output element will be a tuple of tensors. The ordering of the outputs will be the same order as the layers given in the constructor.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[IntegratedGradients, LayerIntegratedGradients][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in, then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of integrated gradients, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by (inputs - baselines).

In case of layer integrated gradients, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by layer activations for inputs - layer activations for baselines.

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.integrated_gradients.IntegratedGradientsCVExplainer[source]

Bases: BaseIntegratedGradientsCVExplainer

Integrated Gradients algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, **kwargs) → Union[IntegratedGradients, LayerIntegratedGradients][source]

Create explainer object.

Parameters:

model – The forward function of the model or any modification of it.
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in, then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of integrated gradients, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by (inputs - baselines).

Returns:

Explainer object.

class foxai.explainer.computer_vision.algorithm.integrated_gradients.LayerIntegratedGradientsCVExplainer[source]

Bases: BaseIntegratedGradientsCVExplainer

Layer Integrated Gradients algorithm explainer.

create_explainer(model: Module, multiply_by_inputs: bool = True, layer: Optional[Module] = None, **kwargs) → Union[IntegratedGradients, LayerIntegratedGradients][source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None
multiply_by_inputs –
Indicates whether to factor model inputs’ multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs’ multiplier isn’t factored in, then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104

In case of layer integrated gradients, if multiply_by_inputs is set to True, final sensitivity scores are being multiplied by layer activations for inputs - layer activations for baselines.

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

foxai.explainer.computer_vision.algorithm.lrp module

File with LRP algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/lrp.py and https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_lrp.py.

class foxai.explainer.computer_vision.algorithm.lrp.BaseLRPCVExplainer[source]

Bases: Explainer

Base LRP algorithm explainer.

add_rules(model: Module) → Module[source]

Add rules for the LRP explainer, according to https://arxiv.org/pdf/1910.09840.pdf.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Modified DNN object.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, additional_forward_args: Optional[Any] = None, attribute_to_layer_input: bool = False, verbose: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with LRP algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which relevance is propagated. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order, following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None
attribute_to_layer_input – Indicates whether to compute the attribution with respect to the layer input or output. If attribute_to_layer_input is set to True then the attributions will be computed with respect to layer input, otherwise it will be computed with respect to layer output.
verbose – Indicates whether information on application of rules is printed during propagation. Default: False

Returns:

Features matrix.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, **kwargs) → Union[LRP, LayerLRP][source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.lrp.LRPCVExplainer[source]

Bases: BaseLRPCVExplainer

LRP algorithm explainer.

create_explainer(model: Module, **kwargs) → Union[LRP, LayerLRP][source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.lrp.LayerLRPCVExplainer[source]

Bases: BaseLRPCVExplainer

Layer LRP algorithm explainer.

create_explainer(model: Module, layer: Optional[Module] = None, **kwargs) → Union[LRP, LayerLRP][source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

foxai.explainer.computer_vision.algorithm.noise_tunnel module

File with Noise Tunnel algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/noise_tunnel.py.

class foxai.explainer.computer_vision.algorithm.noise_tunnel.BaseNoiseTunnelCVExplainer[source]

Bases: Explainer

Base Noise Tunnel algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, nt_type: str = 'smoothgrad', nt_samples: int = 5, nt_samples_batch_size: Union[None, int] = None, stdevs: Union[float, Tuple[float, ...]] = 1.0, draw_baseline_from_distrib: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with Noise Tunnel algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which integrated gradients are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
nt_type – Smoothing type of the attributions. smoothgrad, smoothgrad_sq or vargrad Default: smoothgrad if type is not provided.
nt_samples – The number of randomly generated examples per sample in the input batch. Random examples are generated by adding gaussian random noise to each sample. Default: 5 if nt_samples is not provided.
nt_samples_batch_size – The number of the nt_samples that will be processed together. With the help of this parameter we can avoid out of memory situation and reduce the number of randomly generated examples per sample in each batch. Default: None if nt_samples_batch_size is not provided. In this case all nt_samples will be processed together.
stdevs – The standard deviation of gaussian noise with zero mean that is added to each input in the batch. If stdevs is a single float value then that same value is used for all inputs. If it is a tuple, then it must have the same length as the inputs tuple. In this case, each stdev value in the stdevs tuple corresponds to the input with the same index in the inputs tuple. Default: 1.0 if stdevs is not provided.
draw_baseline_from_distrib – Indicates whether to randomly draw baseline samples from the baselines distribution provided as an input tensor. Default: False

Returns:

Attribution with respect to each input feature. attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned.

Raises:

RuntimeError – if attribution has shape (0).

abstract create_explainer(model: Module, **kwargs) → NoiseTunnel[source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

class foxai.explainer.computer_vision.algorithm.noise_tunnel.LayerNoiseTunnelCVExplainer[source]

Bases: BaseNoiseTunnelCVExplainer

Layer Noise Tunnel algorithm explainer.

create_explainer(model: Module, layer: Optional[Module] = None, **kwargs) → NoiseTunnel[source]

Create explainer object.

Uses parameter layer from kwargs. If not provided function will call get_last_conv_model_layer function to obtain last torch.nn.Conv2d layer from provided model.

Parameters:

model – The forward function of the model or any modification of it.
layer – Layer for which attributions are computed. Output size of attribute matches this layer’s input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Default: None

Returns:

Explainer object.

Raises:

ValueError – if model does not contain conv layers.

class foxai.explainer.computer_vision.algorithm.noise_tunnel.NoiseTunnelCVExplainer[source]

Bases: BaseNoiseTunnelCVExplainer

Noise Tunnel algorithm explainer.

create_explainer(model: Module, **kwargs) → NoiseTunnel[source]

Create explainer object.

Parameters:: model – The forward function of the model or any modification of it.
Returns:: Explainer object.

foxai.explainer.computer_vision.algorithm.occlusion module

File with Occulusion algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/occlusion.py.

class foxai.explainer.computer_vision.algorithm.occlusion.OcclusionCVExplainer[source]

Bases: Explainer

Occlusion algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, sliding_window_shapes: Union[Tuple[int, ...], Tuple[Tuple[int, ...], ...]] = (1, 1, 1), strides: Union[None, int, Tuple[int, ...], Tuple[Union[int, Tuple[int, ...]], ...]] = None, baselines: Union[None, int, float, Tensor] = None, additional_forward_args: Optional[Any] = None, perturbations_per_eval: int = 1, show_progress: bool = False, **kwargs) → Tensor[source]

Generate model’s attributes with Occlusion algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which occlusion attributions are computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which difference is computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
sliding_window_shapes – Shape of patch (hyperrectangle) to occlude each input. For a single input tensor, this must be a tuple of length equal to the number of dimensions of the input tensor - 1, defining the dimensions of the patch. If the input tensor is 1-d, this should be an empty tuple. Default: (1, 1, 1)
strides – This defines the step by which the occlusion hyperrectangle should be shifted by in each direction for each iteration. For a single tensor input, this can be either a single integer, which is used as the step size in each direction, or a tuple of integers matching the number of dimensions in the occlusion shape, defining the step size in the corresponding dimension. To ensure that all inputs are covered by at least one sliding window, the stride for any dimension must be <= the corresponding sliding window dimension if the sliding window dimension is less than the input dimension. If None is provided, a stride of 1 is used for each dimension of each input tensor. Default: None
baselines –
Baselines define reference value which replaces each feature when occluded. Baselines can be provided as:
- a single tensor, if inputs is a single tensor, with
  exactly the same dimensions as inputs or broadcastable to match the dimensions of inputs
- a single scalar, if inputs is a single tensor, which will
  be broadcasted for each input value in input tensor.
In the cases when baselines is not provided, we internally use zero scalar corresponding to each input tensor. Default: None
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. For a tensor, the first dimension of the tensor must correspond to the number of examples. For all other types, the given argument is used for all forward evaluations. Note that attributions are not computed with respect to these arguments. Default: None
perturbations_per_eval – Allows multiple occlusions to be included in one batch (one call to forward_fn). By default, perturbations_per_eval is 1, so each occlusion is processed individually. Each forward pass will contain a maximum of perturbations_per_eval * #examples samples. For DataParallel models, each batch is split among the available devices, so evaluations on each available device contain at most (perturbations_per_eval * #examples) / num_devices samples. Default: 1
show_progress – Displays the progress of computation. It will try to use tqdm if available for advanced features (e.g. time estimation). Otherwise, it will fallback to a simple output of progress. Default: False

Returns:

The attributions with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

foxai.explainer.computer_vision.algorithm.saliency module

File with Saliency algorithm explainer classes.

Based on https://github.com/pytorch/captum/blob/master/captum/attr/_core/saliency.py.

class foxai.explainer.computer_vision.algorithm.saliency.SaliencyCVExplainer[source]

Bases: Explainer

Saliency algorithm explainer.

calculate_features(model: Module, input_data: Tensor, pred_label_idx: Union[None, int, Tuple[int, ...], Tensor, List[Tuple[int, ...]], List[int]] = None, abs_value: bool = True, additional_forward_args: Optional[Any] = None, **kwargs) → Tensor[source]

Generate model’s attributes with Saliency algorithm explainer.

Parameters:

model – The forward function of the model or any modification of it.
input_data – Input for which saliency is computed. If forward_func takes a single tensor as input, a single input tensor should be provided.
pred_label_idx –
Output indices for which gradients are computed (for classification cases, this is usually the target class). If the network returns a scalar value per example, no target index is necessary. For general 2D outputs, targets can be either:
- a single integer or a tensor containing a single
  integer, which is applied to all input examples
- a list of integers or a 1D tensor, with length matching
  the number of examples in inputs (dim 0). Each integer is applied as the target for the corresponding example.
For outputs with > 2 dimensions, targets can be either:
- A single tuple, which contains #output_dims - 1
  elements. This target index is applied to all examples.
- A list of tuples with length equal to the number of
  examples in inputs (dim 0), and each tuple containing #output_dims - 1 elements. Each tuple is applied as the target for the corresponding example.
Default: None
abs_value – Returns absolute value of gradients if set to True, otherwise returns the (signed) gradients if False. Default: True
additional_forward_args – If the forward function requires additional arguments other than the inputs for which attributions should not be computed, this argument can be provided. It must be either a single additional argument of a Tensor or arbitrary (non-tuple) type or a tuple containing multiple additional arguments including tensors or any arbitrary python types. These arguments are provided to forward_func in order following the arguments in inputs. Note that attributions are not computed with respect to these arguments. Default: None

Returns:

The gradients with respect to each input feature. Attributions will always be the same size as the provided inputs, with each value providing the attribution of the corresponding input index. If a single tensor is provided as inputs, a single tensor is returned. If a tuple is provided for inputs, a tuple of corresponding sized tensors is returned.

Raises:

RuntimeError – if attribution has shape (0).

foxai.explainer.computer_vision.algorithm package

Submodules

foxai.explainer.computer_vision.algorithm.conductance module

foxai.explainer.computer_vision.algorithm.deconv module

foxai.explainer.computer_vision.algorithm.deeplift module

foxai.explainer.computer_vision.algorithm.deeplift_shap module

foxai.explainer.computer_vision.algorithm.gradcam module

foxai.explainer.computer_vision.algorithm.gradient_shap module

foxai.explainer.computer_vision.algorithm.guided_backprop module

foxai.explainer.computer_vision.algorithm.input_x_gradient module

foxai.explainer.computer_vision.algorithm.integrated_gradients module

foxai.explainer.computer_vision.algorithm.lrp module

foxai.explainer.computer_vision.algorithm.noise_tunnel module

foxai.explainer.computer_vision.algorithm.occlusion module

foxai.explainer.computer_vision.algorithm.saliency module

Module contents