In the GNOME Terminal, Bash does smart auto-completion. For example
apt-get in
becomes
apt-get install
I don't know the answer to this. But the reason that it doesn't work as you expect is probably because the completion in emacs shells is handled by emacs internally (by the comint-dynamic-complete function), and doesn't have those smart completion functions built-in.
I'm afraid it is not an easy thing to fix.
Edit: njsf's suggestion of using term-mode is probably as good as it gets. Start it with
M-x termIt is included in the standard emacs distribution (and in emacs21-common or emacs22-common on Ubuntu and Debian at least).
I use Prelude and when I hit Meta+Tab it completes for me.
Also, Ctrl+i seems to do the same thing.
I use helm mode. It's has this functionality (after press "TAB"):
I know this question is three years old, but it's something that I've also been interested in solving. A Web search directed me to a piece of elisp that makes Emacs use bash for completion in shell mode. It works for me, in any case.
Check it out at https://github.com/szermatt/emacs-bash-completion .
Like Matli said, it is not an easy task, since bash is started with --noediting and TAB is bound to comint-dynamic-complete.
One could possibly rebind TAB to self-insert-command in shell-comand-hook with local-set-key and make shell-mode not start with --noediting by M-x customize-variable RET explicit-bash-args, but I suspect that it will not sit well with all other editing.
You might want to try term-mode, but it has another set of problems, because some of the other regular keybindings are overtaken by term-mode.
EDIT: By other regular keybidings being overtaken by term-mode, I mean all but C-c which becomes the escape to be able to switch buffers. So instead of C-x k to kill the buffer you'd have to C-c C-x k. Or to switch to another buffer 'C-c C-x o' or 'C-c C-x 2'
In the emacs shell, it's actually emacs doing the auto-completion, not bash. If the shell and emacs are out of sync (e.g. by using pushd, popd or some bash user function that changes the shell's current directory), then auto-completion stops working.
To fix this, just type 'dirs' into the shell and things get back in sync.
I also have the following in my .emacs:
(global-set-key "\M-\r" 'shell-resync-dirs)
Then just hitting Esc-return resyncs the auto-completion.