A web search will turn up lots of amateur radio BPSK and RTTY/FSK solutions. Much of this code was written for older slower CPUs, so should run just fine on an iPhone. You can use the Audio Queue API or the RemoteIO Audio Unit for iOS audio IO to the codec.
