concept - Deep Learning

Convolutional Neural Network

CONV layer is a set of filters that spatially slide across input volume and output dot product during each step.

  • why: much less parameters compared to a fully connected layer, parameter sharing, sparsity of connection
  • convolve: the action of sliding over width and height
  • local connectivity: each filter only look at input volume within the filter window, NOT fully connected
  • filter: matrix of size (f,f,c), f is mostly odd, c is the same as input volume depth (n,n,c)
  • depth: hyperparameter, number of filters, depth column, or fiber
  • stride: Step size when convolve the filter, mostly 1 or 2
  • pooling: reduce size of representation, speeds up computation, more robust features.
    • hyperparameter: (f,f,s) s is step size
    • output: n+2p-f+1 / s,
      • Max Pooling: max(spatial filter)
      • Average Pooling: less common, used to collapse
  • structure: input -> layer 1 (conv1, pool1) -> layer 2(conv2, pool2) -> layer 3(fully connected), layer 4(FC) -> softmax