ggplot scale_fill_manual within groups

Question

I\'m trying to change the color of the individual bars of my ggplot using scale_fill_manual, but am having trouble because the bars are within