If I have html like this:
This is some text First span text
This untested, but I think you may be able to try something like this:
$('#listItem').not('span').text();
http://api.jquery.com/not/