Is it possible to run caffe models on the data-set which is not stored in data-source like LMDB?

雨燕双飞 提交于 2019-11-28 14:22:35

You can write your own python data layer. See discussions here and implementation for of input data layer for video stream here.

Basically you will need add to you network description layer like:

layer {
  type: 'Python'
  name: 'data'
  top: 'data'
  top: 'label'
  python_param {
    # the module name -- usually the filename -- that needs to be in $PYTHONPATH
    module: 'filename'
    # the layer name -- the class name in the module
    layer: 'CustomInputDataLayer'
  }
}

and implement the layer interface in Python:

class CustomInputDataLayer(caffe.Layer):
    def setup(self):
         ...

    def reshape(self, bottom, top)
        top[0].reshape(BATCH_SIZE, your_data.shape)
        top[1].reshape(BATCH_SIZE, your_label.shape)

    def forward(self, bottom, top):
        # assign output
        top[0].data[...] = your_data
        top[1].data[...] = your_label

    def backward(self, top, propagate_down, bottom):
        pass

Other than defining custom python layers, you can use the following options:

  • use ImageData layer: it has a source parameter (source: name of a text file, with each line giving an image filename and label)

  • use MemoryData layer: using which you can load input images directly from memory to your network using ‘setinputarrays‘ method in python. Be cautious about using this layer as it only accepts labels which are single values and you cannot use images as labels (e.g. In semantic segmentation)

  • use a deploy version of your network like this:

    input: "data"
    input_shape {
    dim: n # batch size
    dim: c # number of channels
    dim: r # image size1
    dim: w # image size2
    }
    
    input: "label"
    input_shape {
    dim: n # batch size
    dim: c # number of channels
    dim: r # label image size1
    dim: w # label image size2
    }
     ... #your other layers to follow
    
  • use an HDF5 input layer (more or less ine lmdb, but lmdb is more computationally efficient)

You can find the details of these layers here: http://caffe.berkeleyvision.org/tutorial/layers.html

There are examples available online as well.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!