In a for loop like this one:
for f in `ls *.avi`; do echo $f; ffmpeg -i $f $f.mp3; done
$f will be the complete filename, including the extensi
This should do it:
for i in *.m4a; do ffmpeg -i $i ${i%%.*}.mp3 done