So I have installed anaconda with python 2.7 and installed all of the requirements for Caffe library. I ensured that opencv is installed by
import cv2
Adding
LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda/lib
in to Makefile.config worked.