As suggested in the comments, one approach is to style everything, and then undo the styling for subsequent elements. The only real downside is that if you're over the container div but in between spans, then you get everything styled red. One fix would be to put the spans all on one line, so that there's no whitespace between.
#parent {
line-height: 0;
display: inline-block;
}
#parent span {
display:inline-block;
width: 32px;
height: 32px;
}
#parent:hover span {
background-color:red;
}
#parent:hover span:hover~span {
background-color:#eee;
}