I had similar problems while trying to call WCF web services within SQL CLR.
It loads so much assemblies from .NET and I couldn't make it work.
Eventually, I use generic WebClient to make HTTP request to the WCF server which works really well and it requires min assemblies loaded into SQL Server.