Pooling and Convolution with "SAME" mode


#1

Hello Community,

How do I achieve “SAME” mode in Pooling operator?
MXNet support only “VALID” and “FULL”.

I am implementing MXNet backend for Keras. Keras supports “VALID” and “SAME” modes.
The workaround I am thinking:

  • Calculating the padding to get the same output shape after pooling. (How ?)
  • Call sym.pad() and followed by sym.pool

Any suggestions would be greatly helpful.

Also, if we choose the above-mentioned padding route, will it have any impact on pooling operation? How do I calculate the padding size to achieve pooling with SAME mode.

Thanks,
Sandeep


#2

For Convolution operator, I achieved “SAME” mode with this code block - https://github.com/deep-learning-tools/keras/blob/keras2_mxnet_backend/keras/backend/mxnet_backend.py#L3876-L3891


#3

Hi,

I achieve “SAME” padding (for odd kernels), with the following (assuming you know your kernel size and dilation rate) for stride = 1:


# Ensures padding = 'SAME' for ODD kernel selection 
p0 = dilation_rate[0] * (kernel_size[0] - 1)/2
p1 = dilation_rate[1] * (kernel_size[1] - 1)/2
p = (p0,p1)

nfilters = 32 # some number

gluon.nn.Conv2D(channels = nfilters,  kernel_size =  kernel_size, 
                                           padding=p, dilation= dilation_rate)

you can find the arithmetic that is followed for the output kernel, here:

out_height = floor((height+2*padding[0]-dilation[0]*(kernel_size[0]-1)-1)/stride[0])+1
out_width = floor((width+2*padding[1]-dilation[1]*(kernel_size[1]-1)-1)/stride[1])+1

by demanding out_height == height and assuming stride = 1, you can solve the equation for padding (given kernel size and dilation). Now, the floor function perhaps complicates things, but for odd kernels works for me. Additional information for convolution arithmetic can be found here but note that mxnet has (perhaps?) slightly different definitions.

Hope this helps.


#4

The padding requirements between pooling and convolution are basically the same, in the sense that they both run a kernel over the data+padding. What makes this a bit challenging with MXNet is that the padding is applied symmetrically, which means that getting the same length on output as input is not possible when a kernel dimension is even. The steps to take to get ‘SAME’ padding in 2-D pooling:

  1. set pad to (kernel[0]//2, kernel[1]//2)
  2. if any kernel dimension is even, slice off the first row/column of the output (this will replicate the implementation in TF)

Note that because of the slice operation, there is an extra memcopy associated with even dimensioned kernels.

# Example Code:
# Assuming symetric kernel of (k x k) and data shape of (batch, c, h, w)
pool = mx.sym.Pooling(data, kernel=(k,k), pad=(k//2, k//2))
if k%2 == 0:
    pool = pool[:,:,1:,1:]

#5

Thank you @feevos … This was very helpful.


#6

Thank you @safrooze - Your solution worked for me.