Upstream our changes to PyTorch r(2+1)d architecture

[daniel-j-h]

The torchvision r(2+1)d architecture needs two modifications to get it in sync with the official Caffe2 implementation (see facebookresearch/VMZ#89) and our provided code:

• Number of midplanes in the downsampling blocks
• Batchnorm

We should upstream both modifications to torchvision.

[bjuncek]

see pytorch/vision#1265

need some discussion, but is easily done

[daniel-j-h]

@bjuncek great, this would be sweet to have upstream! ❤️ Please consider adapting the Batchnorm blocks, too. Then the architectures are 100% in sync and weights can easily be transfered.

[daniel-j-h]

Closing this ticket since there is nothing actionable on our end. We will follow up with you folks on vmz to upstream some of our changes when we ported the weights.