I\'ve been trying to use Zsh within my emacs session, without emacs remapping all the Zsh keys. I found ansi-term works pretty well for this but, I\'m still having some problems
Hmmm. I don't think I've ever seen any fancy editing work out well within ansi-term, although I haven't tried it in maybe 20 years. I'll just ask, in passing, if you've tried shell-mode (M-x shell) as it's a lot more natural with EMACS anyway.
That said, reading through the term.el file, it kind of looks like ansi-term is doing a lot of manging of its own. you might want to look at the term raw mode code, starting around line 1230, at least in EMACS 22.3.