When a standard WPF button is clicked, it gets highlighted in blue (probably using the blue color from whatever Windows theme is set), and it stays highlighted until you interac
Try to set Focusable to false. The button will be clickable but will not remain focused.