Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, Emily L.
and
Zaremba, Wojciech
and
Bruna, Joan
and
LeCun, Yann
and
Fergus, Rob
Neural Information Processing Systems Conference - 2014 via Local Bibsonomy
Keywords:
dblp
The paper addresses the problem of speeding up the evaluation of pre-trained image classification ConvNets. To this end, a number of techniques are proposed, which are based on the tensor representation of the conv. layer weight matrix. Namely, the following techniques are considered (Sect. 3.2-3.5):
1. SVD decomposition of the tensor
2. outer product decomposition of the tensor
3. monochromatic approximation of the first conv. layer - projecting RGB colors to a 1-D space, followed by clustering
4. biclustering tensor approximation - clustering input and output features to split the tensor into a number of sub-tensors, each of which is then separately approximated
5. fine-tuning of approximate models to (partially) recover the lost accuracy