Deconvolution Layer == Convolution Layer?
I stumbled across a note which expands an already brilliant paper: Realtime Image/Video Super Resolution using a Sub-Pixel CNN (which I really should write about soon). The note basically tries to answer the question: Is the decovolution layer the same as a convolution layer in Low Resolution (LR) space?
TL;DR: Yes.
Their reasoning was what really caught my eye. The crux of it is:
We notice that the different sets of weights in the kernel (1,1,4,4) are activated independently from each other.So we can easily break them into kernels as shown in the figure on the right.This operation is invertible (4,1,2,2) because each set of the weights are independent from each other during the convolution. In our paper,instead of convolving the kernel with the unpooleded sub-pixel image, we notice that the different sets of weights in the kernel (1,1,4,4) are activated independently from each other. So we can easily break them into kernels as shown in the figure on the right.This operation is invertible (4,1,2,2) because each set of the weights are independent from each other during the convolution.In our paper,instead of convolving the kernel with the unpooled subÂpixel image, we convolve (1,1,4,4) the kernel with the LR input directly as illustrated by the following figure.
Another interesting part of the note was the realization that the representation power of a network in LR is mucn more than that in HR (where you first upscale using, say, Bicubic and then pass this as input to the network).
Go on, read the note (and the paper it expands) and the fascinating breakdown of the logic.
Is the deconvolution layer the same as a convolutional layer?: Wenzhe Shi, Jose Caballero, Lucas Theis, Ferenc Huszar, Andrew Aitken,Alykhan Tejani, Johannes Totz, Christian Ledig, Zehan Wang
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network : Wenzhe Shi, Jose Caballero, Lucas Theis, Ferenc Huszar, Andrew Aitken,Alykhan Tejani, Johannes Totz, Christian Ledig, Zehan Wang