I\'m playing WAVs on my Android phone by loading the file and feeding the bytes into AudioTrack.write() via the FileInputStream > BufferedInputStream > DataInputStream method.
I ran into a similar problem, although I was using a RandomAccessFile, instead of a BufferedInputStream, to read the PCM data. The issue was that the file I/O was too slow. I suspect you will have this problem even with a buffered stream, because the I/O is still taking place on the same thread as audio processing.
The solution is to have two threads: A thread that reads buffers from a file and queues them into memory, and another thread that reads from this queue and writes to the audio hardware. I used a ConcurrentLinkedQueue to accomplish this.
I used the same technique for recording, using AudioRecord, but in the reverse direction. The key is to place the file I/O on a separate thread.