Layer-wise Relevance Propagation (LRP)  is a novel technique has been used by authors in multiple use-cases (apart from this publication) to demonstrate the robustness and advantage of a *decomposition* method over other heatmap generation methods. Such heatmap generation methods are very crucial for increasing interpretability of Deep Learning models as such.  Apart from LRP relevance, authors also discuss quantitative ways to measure the accuracy of the heatmap generated.

### LRP & Alternatives

What is LRP ?

LRP is a principled approach to decompose a classification decision into pixel-wise relevances indicating the contributions of a pixel to the overall classification score. The approach is derived
from a layer-wise conservation principle , which forces the propagated quantity (e.g. evidence for a predicted class) to be preserved between neurons of two adjacent layers. 

Denoting by R(l) [i] the relevance associated to the ith neuron of layer and by R (l+1) [j] the relevance associated to the jth neuron in the next layer, the conservation principle requires that

![](https://i.imgur.com/GQxrnCT.png)

where R(l) [i] is given as 
![](https://i.imgur.com/FD7AAfF.png)


where z[i,j]  is the activation of jth neuron because of input from ith neuron 

As per authors this is not necssarily the only relevance funtion which is conserved. The intuition behind using such a function is  that lower-layer neurons that mostly contribute to the activation of the higher-layer neuron receive a larger share of the relevance Rj of the neuron j.

A downside of this propagation rule (at least if *epsilon* = 0) is that the denominator may tend to zero if lower-level contributions to neuron j cancel each other out. The numerical instability can be overcome by setting *epsilon* > 0. However in that case, the conservation idea is relaxated in order to gain better numerical properties.  To conserve relevance, it can be formulated as sum of positive and negative activations
![](https://i.imgur.com/lo7f8AI.png)
 such that *alpha* - *beta* = 1

#### Alternatives to LRP for heatmap

**Senstiivity measurement**

In such methods of generating heamaps, gradient of the output with respect to input is used for generating heatmap. This quantity measures how much small changes in the pixel value locally affect the network output. 
##### Disadvantages
Given most models use ReLU as activation function, the gradient flows only through activation with positive output - thereby making  makes the backward mapping discontinuous, and consequently strongly local. Also same applies for maxpool activations - wherein gradients only flow through neurons with maximum intensity in local neighbourhood.

Also, given most of these methods use absolute impact on prediction cause by changes in pixel intensities, the granularity of whether the pixel intensity was in favour or against evidence is lost.

**Deconvolutional Networks**

##### Disadvantages

Here the backward discontinuity problem of sensitivity based methods are absent, hence global features can be captured. However, since the method only takes in activation from final layer (which learns the presence or absence of features mostly) , using this for generating heatmaps is likely to yield avergae maps, lacking image specific localisation effects

LRP is able to counter the effects nicely because of the way it uses relevance

#### Performance of heatmaps

Few concerns that the authors raise are 
- A heatmap is not a segmentation mask  on the contrary missing evidence or the context may be very important for classification
- Salient features represent average explanations of what distinguishes one image category from another. For individual images these explanations may be meaningless or even wrong. For instance, salient features for the class ‘bicycle’ may be the wheels and the handlebar. However, in some images a bicycle
may be partly occluded so that these parts of a bike are not visible. In these images salient features fail to explain the classifier’s decision (which still may be correct).

Authors propose a novel method (MoRF - *Most Relevant First* ) of objectively quantifying quality of a heatmap. A good detailed idea of the measure can best be obtained from the paper. To give an idea, the most reliable method should ideally rank the most relevant regions in the same order even if small perturbations in pixel intensities are observed (in non-relevant areas.

The quantity of interest in this case is the area over the MoRF perturbation curve (AOPC).

#### Observation

Most of the sensitivity based methods answer to the question - *what change would make the image more or less belong to the category car* which isn't really the classifier's question.  LRP plans to answer the real classifier question *what speaks for the presence of a car in the image*

An image below would be a good example of how LRPs can denoise heatmaps generated on the basis of sensitivity.

![](https://i.imgur.com/Sq0b5yg.png)