I downloaded the latest pre-built opencv2.framework from the OpenCV SourceForge page. then in Xcode6-Beta3, I added opencv2.framework as a required linked framework under the \
It's easiest if you start with a simple project all ready to compile and run out of the box.
This xcode project works with Swift, XCode 6.1, OpenCV 2, and is MIT licensed: http://whitneyland.com/2014/10/opencv-swift.html
Here's one I haven't tried yet but there are so few still it's worth looking at: https://github.com/foundry/OpenCVSwiftStitch