Which layer in a CNN able to detect spinned and translated objects?

Question

Does the conv layer, max pooling layer, or anything else do the job? In my opinion, the Conv layer or max pooling layer is able to do the job only when the rotations or translations are not too big.

Answer 1

Convolutional layers are not equivariant to rotation, and pooling layers only help with invariance to small rotations. "Invariance" of the whole classifier to rotations is not part of the inductive bias, but it's actually learned through heavy data augmentation.

However, for each group action there exists a corresponding group convolution operator which is equivariant to it. This concept is used, for example, in 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data by Weiler, Geiger, Welling, Boomsma and Cohen, 2018, to design layers which are equivariant to 3D rotations:

https://arxiv.org/pdf/1807.02547.pdf

Answer 2

For translational invariance, you can follow the discussion here. In general, pooling layer is the important player in local translational invariance by removing the spatial dimension in, for example, max-pooling. For instance, if an object slightly moves towards some direction, max-pooling still captures the max element and the same output will appear after the pooling. The convolutional layer is actually equivariant in translation.

Neither layers are rotation-invariant. Though, the network can exhibit this behavior if the properties of the data, and the overall architecture permit. A NIPS paper addresses this issue and use Spatial Transformers to improve CNNs invariance to rotation, scale and translation.