Easy way is to just finish the current activity when you are moving to the next activity.
Just do a finish() after your startActivity(), then on back press from the any activity you can simply call finish() to come out of the application as all the other activities are not there in the stack.