I have the following code in my activity. In my xml, the video view is inside the linear layout. However, when the view is clicked, the onTouchListener never fires.
ACTION_UP is never being sent to your listener because you return false and therefor don't "consume" the event. Return true and you'll get the start event (ACTION_DOWN) as well as all the subsequent ones (ACTION_MOVE and then ACTION_UP).