nd convolution and pooling revision

[lfz]

This is a revision of #2824 , since it is already deprecated.

I fixed some bugs mentioned in the discussion, now it supports cudnn v3

an example of using:

layer {
  name: "conv1"
  type: "NdConvolution"
  bottom: "3d_data" # actually 5 dimension here, you may need reshape
  top: "conv1"
  convolution_param {
    num_output: 10
    stride:1
    # specifies the index of the "channels" axis --
    # may be omitted as 1 is the default
    kernel_shape { dim: 4 dim: 4 dim: 3}
    weight_filler {type: "xavier"}
  }
}



# layer {
#   name: "pool1"
#   type: "NdPooling"
#   bottom: "conv1" 
#   top: "pool1" 

#   pooling_param {
#     pool: MAX
#     kernel_shape {dim: 3,dim: 3,dim: 2}
#     stride_shape  {dim: 3,dim: 3, dim: 2}
#   }
# }

[lfz]

I am sorry that this only works with a gpu,

and because of that the checks didn't pass without cuda.

And I don't know how to bypass this

[seanbell]

Is this a bug? The legacy shape accessors should not allow for 5D blobs. This change should not be made.

[lfz]

it's because I want to use the xavier initializer, I'll change it

[ddetone]

caffe/src/caffe/layers/cudnn_ndpooling_layer.cpp line 13 has a typo, const is written twice

[lfz]

Thanks @ddetone , it has been fixed

[YOUNGING]

Is it possible to implement a ND deconvolutional layer using cudnn?

[YOUNGING]

Is there anyone can make sure that this PR works?I tried this Ndpooling and the master branch 3D convolutionLayer,but I have to reduce my learningrate to 1e-18 to avoid loss rising to Nan.Is this normal?

[lfz]

@YOUNGING really? Please supply me with the sample proto and result

[YOUNGING]

@lfz When compiled with "USE_CUDNN" option , caffe will use CUDNN engine,right?
But most of cudnn layers don't have a ND implementation.
It will give an error like this:

neural layer like ReLU can use CAFFE engine by setting engine to CAFFE,but SoftmaxWithLoss layer don't have an engine option,so what should I do to make things right?

BTW,reshaping the input of loss to 2D may solve the problem,but I want to find a more efficient way.Any ideas?

Thanks.

[chuckcho]

This function takes an additional input arg in CUDNN v4 (or possibly earlier version too)
Something like following will take care of this:

#if CUDNN_VERSION >= 4000
  CUDNN_CHECK(cudnnSetConvolutionNdDescriptor(*conv,
              pad.size(), pad.data(), stride.data(), upscale.data(),
              CUDNN_CROSS_CORRELATION, cudnn_type));
#else
   CUDNN_CHECK(cudnnSetConvolutionNdDescriptor(*conv,
               pad.size(), pad.data(), stride.data(), upscale.data(),
               CUDNN_CROSS_CORRELATION));
#endif

[futurely]

Any updates? FYI, #3983 supports cuDNN v5.