Im setting a background color when my Item is selected and undo when selection canceled like this:
holder.mView.setOnTouchListener(new View.OnTouc