Processing math: 100%

A Brief Walk Through Neural Network's Feature Visualisation

A Brief Walk Through Neural Network's Feature Visualisation
Jithin James
January 19th, 2019
jithinjk.github.io/blog
Visualizing and Understanding CNNs [Zeiler13] · Understanding NNs Through Deep Visualization [Yosinki15] · Wrapping Up · Bibliography

In this paper review, we will dive into following two papers on feature visualizaion for neural networks:

Some of the factors that lead to a renewed interest in CNN based models are: 1) availability of large training sets, 2) powerful GPUs, and, 3) better regularization strategies

In this review, we'll see several visualisations and try to understand CNN feature activations.


   

Visualizing and Understanding CNNs [Zeiler13]

In this first paper, [Zeiler13] proposes a visualization technique that uses a multi-layered Deconvolutional Network (deconvnet), as proposed by [Zeiler11], to project the feature activations back to the input pixel space.

In this method, we can see what input pattern caused a given activation in the feature maps using a deconvnet. A deconvnet can be thought of as a convnet model that uses the same components (filtering, pooling) but in reverse(mapping features to pixels).

 Figure 1: Feature visualization layer 1 and 2. Source: [Zeiler13]

In the above figure, in Layer 1, we can find groups of pixels that have lines in different directions, gradients that go from yellow to blue, etc. This is achieved using simple convolutional filters, which are simply matrices(square, in most cases). They are multiplied and slide across the entirety of an image to find these patterns.

Following Layer 1 , the next layer takes the results of previous layer filter activations and does another round of computation. Layer 2 learns about curves, and circles, etc. In each layer, this representation/dimensionality increases to represent more and more features.

 Figure 2: Feature visualization layer 3. Source: [Zeiler13]

In the above figure, we can interesting patterns objects/persons. Again, Layer 3 builds up on bottom layer features to learn other representations.

 Figure 3: Feature visualization layer 4 and 5. Source: [Zeiler13]

Going forward, Layer 4 combines them together and can recognize dog faces, tyres, round objects, etc. By layer 5, we got features that identifies faces, cycles, sign labels, etc.

   

Understanding NNs Through Deep Visualization [Yosinki15]

Jason Yosinski et. al [Yosinki15] developed tools to visualize and interpret neural networks. The first tool visualizes the activations produced on each layer of a trained convnet as it processes an image or video. The second tool enables visualizing features at each layer of a DNN via regularized optimization in image space.

Tools are open-source and avaliable at http://yosinski.com/deepvis.

 Figure 4: deepvis tool. Source: http://yosinski.com/deepvis

   

Intuitions gained from results

 Figure 5: deepvis tool. Source: http://yosinski.com/deepvis

   

Regularized Optimization

The authors proposed several regularization methods to bias images found via optimization toward more visually interpretable examples. These are called “natural image priors” or “regularization”. The optimization problem is to find an image x where

x=argmaxx(ai(x)Rθ(x))

where x is an image xϵRC×H×W, where C = 3 color channels and the height (H) and width (W). The neural network causes an an activation ai(x) for some unit i, where i is an index that runs over all units on all layers. We use a parameterized regularization function Rθ(x) that penalizes iamges in following ways:

 Figure 6: All layers. Source: http://yosinski.com/deepvis

From the above figure, you could easily see important features, such as edges, corners, wheels, eyes, faces, etc. Feature complexity and pattern variation increases in as we move higher up the layers as simpler features are combined from lower layers.

   

Wrapping Up

I hope that this paper review was helpful for you. If you think something's missing, feel free to refer to the original paper to clarify your doubts.

More blog posts could be found at https://jithinjk.github.io/blog

   

Bibliography

[ Anh14] Nguyen, Anh, Yosinski, Jason, and Clune, Jeff. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. ArXiv e-prints, December 2014.
[ Dumitru09] Erhan, Dumitru, Bengio, Yoshua, Courville, Aaron, and Vincent, Pascal. Visualizing higher-layer features of a deep network. Technical report, University of Montreal, 2009.
[ Erhan09] Erhan, D., Bengio, Y., Courville, A., and Vincent, P. Visualizing higher-layer features of a deep network. In Technical report, University of Montreal, 2009.
[ Feifei06] Fei-fei, L., Fergus, R., and Perona, P. One-shot learning of object categories. IEEE Trans. PAMI, 2006.
[ Hannun14] Hannun, A., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., Coates, A., and Ng, A. Y. Deep Speech: Scaling up end-to-end speech recognition. ArXiv e-prints, December 2014.
[ Hinton06] Hinton, G. E., Osindero, S., and The, Y. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527{1554, 2006.
[ Hinton12] Hinton, Geoffrey E, Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
[ Krizhevsky12] Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoff. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, pp. 1106–1114, 2012.
[ Nguyen14] Nguyen, Anh, Yosinski, Jason, and Clune, Jeff. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. ArXiv e-prints, December 2014.
[ Yosinki14] Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. How transferable are features in deep neural networks? In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 27, pp. 3320–3328. Curran Associates, Inc., December 2014.
[ Yosinki15] Understanding Neural Networks Through Deep Visualization. arXiv:1506.06579
[ Zeiler11] Zeiler, M., Taylor, G., and Fergus, R. Adaptive deconvolutional networks for mid and high level feature learning. In ICCV, 2011.
[ Zeiler13] Zeiler, Matthew D and Fergus, Rob. Visualizing and understanding convolutional neural networks. arXiv preprint arXiv:1311.2901, 2013.

formatted by Markdeep 1.03