I\'m working with WCF at the moment and attempting to implement a custom security model based around an API key and signature (similar to how Facebook/Flickr/OAuth etc. work
Take a look at Juval Lowy's excellent "Programming WCF Services," 2nd Edition:
Here's the link to Amazon's page on it.
Chapter 10 is completely devoted to security.
Microsoft has released a WCF Security Guide - a free(!) eBook. You can find it here.
That's an awful lot of information to wade through. Good luck!