This is not possible with pure CSS without adding new tags. In order to do this with pure CSS/HTML, you will need to add a tag to either the image's [a href] link.. or add a tag to the [a hrer] links that you want to display with an underline.
You could write a small piece of code in javascript that would alter the border property of an element on hover quite easily. You would just need to check if the element is an IMG.
Try this trick. It works.
a { text-decoration:none; border-bottom:2px solid; }
a img { border:none; vertical-align:top; }
Another way - PHP:
$text = preg_replace('#<a(.*?)<img(.*?)/>(.*?)</a>#is', "<a class='imgshow' \\1 <img\\2 />\\3</a>", $text);
Sorry, what you'd want is some kind of :parent pseudo-class that selects the child but applies to the parent, which unfortunately does not exist (not even in CSS3).
You'd have to do a bit of Javascript, selecting all elements matching #sidebar a:hover, then applying the red bottom border on the condition that they don't have a child IMG element.