How to create this CNN architecture in Caffe?

问题

I have two images/patches (the larger one includes the smaller one and some context information around it) of different sizes (I dont wont to create a new image by just concatenating them).

I want to create a CNN as shown in the figure.

How can I create it using Caffe prototxt file?

Can I directly use Caffe to train it?

Both CNNs can share the same weights.

来源：https://stackoverflow.com/questions/38226378/how-to-create-this-cnn-architecture-in-caffe