Scale layer in Caffe

Question

I am looking through the Caffe prototxt for deep residual networks and have noticed the appearance of a \"Scale\" layer.

layer {
    bottom: \"r         


        

Answer 1

There's also some documentation on it in the caffe.proto file, you can search for 'ScaleParameter'.

Thanks a heap for your post :) Scale layer was exactly what I was looking for. In case anyone wants an example for a layer that scales by a scalar (0.5) and then "adds" -2 (and those values shouldn't change):

layer {
  name: "scaleAndAdd"
  type: "Scale"
  bottom: "bot"
  top: "scaled"
  param {
    lr_mult: 0
    decay_mult: 0
  }
  param {
    lr_mult: 0
    decay_mult: 0
  }
  scale_param {
    filler {
      value: 0.5    }
    bias_term: true
    bias_filler {
      value: -2
    }
  }
}

(Probably, the decay_mult's are unnecessary here though. But dunno. See comments.) Other than that:

• lr_mult: 0 - switches off learning for "that param" - I think the first "param {" always(?) refers to the weights, the second to bias (lr_mult is not ScaleLayer specific)
• filler: a "FillerParameter" [see caffe.proto] telling how to fill the ommited second blob. Default is one constant "value: ...".
• bias_filler: parameter telling how to fill an optional bias blob
• bias_term: whether there is a bias blob

All taken from caffe.proto. And: I only tested the layer above with both filler values = 1.2.