When a standard WPF button is clicked, it gets highlighted in blue (probably using the blue color from whatever Windows theme is set), and it stays highlighted until you interac
This is the default look for Aero buttons when they have focus. You can either set Focusable="False"
or use a custom Style, that doesn't render it differently when the button has focus. Something like:
xmlns:theme="clr-namespace:Microsoft.Windows.Themes;assembly=PresentationFramework.Aero"
You'd need to add a reference to PresentationFramework.Aero.dll