Android MediaExtractor and mp3 stream

Question

I am trying to play an mp3 stream using MediaExtractor/MediaCodec. MediaPlayer is out of the question due to latency and long buffer size.

The only sample code i have found is this: http://dpsm.wordpress.com/category/android/

The code samples are only parcial (?) and use a File instead of a stream.

I have been trying to adapt this example to play an Audio Stream but i can't get my head around how this is supposed to work. The Android documentation as usual is no help.

I understand that first we get information about the stream, presumably setup the AudioTrack with this information ( code sample does include AudioTrack initialization ?) and then open an input buffer and output buffer.

I have recreated code for this, with what i can guess would be the missing parts, but no audio comes out of this.

Can someone point me in the right direction to understand how this is supposed to work?

public final String LOG_TAG = "mediadecoderexample";
private static int TIMEOUT_US = -1;
MediaCodec codec;
MediaExtractor extractor;

MediaFormat format;
ByteBuffer[] codecInputBuffers;
ByteBuffer[] codecOutputBuffers;
Boolean sawInputEOS = false;
Boolean sawOutputEOS = false;
AudioTrack mAudioTrack;
BufferInfo info;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    String url = "http://82.201.100.9:8000/RADIO538_WEB_MP3";
    extractor = new MediaExtractor();

    try {
        extractor.setDataSource(url);
    } catch (IOException e) {
    }

    format = extractor.getTrackFormat(0);
    String mime = format.getString(MediaFormat.KEY_MIME);
    int sampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);

    Log.i(LOG_TAG, "===========================");
    Log.i(LOG_TAG, "url "+url);
    Log.i(LOG_TAG, "mime type : "+mime);
    Log.i(LOG_TAG, "sample rate : "+sampleRate);
    Log.i(LOG_TAG, "===========================");

    codec = MediaCodec.createDecoderByType(mime);
    codec.configure(format, null , null , 0);
    codec.start();

    codecInputBuffers = codec.getInputBuffers();
    codecOutputBuffers = codec.getOutputBuffers();

    extractor.selectTrack(0); 

    mAudioTrack = new AudioTrack(
            AudioManager.STREAM_MUSIC, 
            sampleRate, 
            AudioFormat.CHANNEL_OUT_STEREO, 
            AudioFormat.ENCODING_PCM_16BIT, 
            AudioTrack.getMinBufferSize (
                    sampleRate, 
                    AudioFormat.CHANNEL_OUT_STEREO, 
                    AudioFormat.ENCODING_PCM_16BIT
                    ), 
            AudioTrack.MODE_STREAM
            );

    info = new BufferInfo();


    input();
    output();


}

private void output()
{
    final int res = codec.dequeueOutputBuffer(info, TIMEOUT_US);
    if (res >= 0) {
        int outputBufIndex = res;
        ByteBuffer buf = codecOutputBuffers[outputBufIndex];

        final byte[] chunk = new byte[info.size];
        buf.get(chunk); // Read the buffer all at once
        buf.clear(); // ** MUST DO!!! OTHERWISE THE NEXT TIME YOU GET THIS SAME BUFFER BAD THINGS WILL HAPPEN

        if (chunk.length > 0) {
            mAudioTrack.write(chunk, 0, chunk.length);
        }
        codec.releaseOutputBuffer(outputBufIndex, false /* render */);

        if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
            sawOutputEOS = true;
        }
    } else if (res == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
        codecOutputBuffers = codec.getOutputBuffers();
    } else if (res == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        final MediaFormat oformat = codec.getOutputFormat();
        Log.d(LOG_TAG, "Output format has changed to " + oformat);
        mAudioTrack.setPlaybackRate(oformat.getInteger(MediaFormat.KEY_SAMPLE_RATE));
    }

}

private void input()
{
    Log.i(LOG_TAG, "inputLoop()");
    int inputBufIndex = codec.dequeueInputBuffer(TIMEOUT_US);
    Log.i(LOG_TAG, "inputBufIndex : "+inputBufIndex);

    if (inputBufIndex >= 0) {   
        ByteBuffer dstBuf = codecInputBuffers[inputBufIndex];

        int sampleSize = extractor.readSampleData(dstBuf, 0);
        Log.i(LOG_TAG, "sampleSize : "+sampleSize);
        long presentationTimeUs = 0;
        if (sampleSize < 0) {
            Log.i(LOG_TAG, "Saw input end of stream!");
            sawInputEOS = true;
            sampleSize = 0;
        } else {
            presentationTimeUs = extractor.getSampleTime();
            Log.i(LOG_TAG, "presentationTimeUs "+presentationTimeUs);
        }

        codec.queueInputBuffer(inputBufIndex,
                               0, //offset
                               sampleSize,
                               presentationTimeUs,
                               sawInputEOS ? MediaCodec.BUFFER_FLAG_END_OF_STREAM : 0);
        if (!sawInputEOS) {
            Log.i(LOG_TAG, "extractor.advance()");
            extractor.advance();

        }
     }

}
}

Edit: adding logcat output for extra ideas.

03-10 16:47:54.115: I/mediadecoderexample(24643): ===========================
03-10 16:47:54.115: I/mediadecoderexample(24643): url ....
03-10 16:47:54.115: I/mediadecoderexample(24643): mime type : audio/mpeg
03-10 16:47:54.115: I/mediadecoderexample(24643): sample rate : 32000
03-10 16:47:54.115: I/mediadecoderexample(24643): ===========================
03-10 16:47:54.120: I/OMXClient(24643): Using client-side OMX mux.
03-10 16:47:54.150: I/Reverb(24643):  getpid() 24643, IPCThreadState::self()->getCallingPid() 24643
03-10 16:47:54.150: I/mediadecoderexample(24643): inputLoop()
03-10 16:47:54.155: I/mediadecoderexample(24643): inputBufIndex : 0
03-10 16:47:54.155: I/mediadecoderexample(24643): sampleSize : 432
03-10 16:47:54.155: I/mediadecoderexample(24643): presentationTimeUs 0
03-10 16:47:54.155: I/mediadecoderexample(24643): extractor.advance()
03-10 16:47:59.085: D/HTTPBase(24643): [2] Network BandWidth = 187 Kbps
03-10 16:47:59.085: D/NuCachedSource2(24643): Remaining (64K), HighWaterThreshold (20480K)
03-10 16:48:04.635: D/HTTPBase(24643): [3] Network BandWidth = 141 Kbps
03-10 16:48:04.635: D/NuCachedSource2(24643): Remaining (128K), HighWaterThreshold (20480K)
03-10 16:48:09.930: D/HTTPBase(24643): [4] Network BandWidth = 127 Kbps
03-10 16:48:09.930: D/NuCachedSource2(24643): Remaining (192K), HighWaterThreshold (20480K)
03-10 16:48:15.255: D/HTTPBase(24643): [5] Network BandWidth = 120 Kbps
03-10 16:48:15.255: D/NuCachedSource2(24643): Remaining (256K), HighWaterThreshold (20480K)
03-10 16:48:20.775: D/HTTPBase(24643): [6] Network BandWidth = 115 Kbps
03-10 16:48:20.775: D/NuCachedSource2(24643): Remaining (320K), HighWaterThreshold (20480K)
03-10 16:48:26.510: D/HTTPBase(24643): [7] Network BandWidth = 111 Kbps
03-10 16:48:26.510: D/NuCachedSource2(24643): Remaining (384K), HighWaterThreshold (20480K)
03-10 16:48:31.740: D/HTTPBase(24643): [8] Network BandWidth = 109 Kbps
03-10 16:48:31.740: D/NuCachedSource2(24643): Remaining (448K), HighWaterThreshold (20480K)
03-10 16:48:37.260: D/HTTPBase(24643): [9] Network BandWidth = 107 Kbps
03-10 16:48:37.260: D/NuCachedSource2(24643): Remaining (512K), HighWaterThreshold (20480K)
03-10 16:48:42.620: D/HTTPBase(24643): [10] Network BandWidth = 106 Kbps
03-10 16:48:42.620: D/NuCachedSource2(24643): Remaining (576K), HighWaterThreshold (20480K)
03-10 16:48:48.295: D/HTTPBase(24643): [11] Network BandWidth = 105 Kbps
03-10 16:48:48.295: D/NuCachedSource2(24643): Remaining (640K), HighWaterThreshold (20480K)
03-10 16:48:53.735: D/HTTPBase(24643): [12] Network BandWidth = 104 Kbps
03-10 16:48:53.735: D/NuCachedSource2(24643): Remaining (704K), HighWaterThreshold (20480K)
03-10 16:48:59.115: D/HTTPBase(24643): [13] Network BandWidth = 103 Kbps
03-10 16:48:59.115: D/NuCachedSource2(24643): Remaining (768K), HighWaterThreshold (20480K)
03-10 16:49:04.480: D/HTTPBase(24643): [14] Network BandWidth = 103 Kbps
03-10 16:49:04.480: D/NuCachedSource2(24643): Remaining (832K), HighWaterThreshold (20480K)
03-10 16:49:09.955: D/HTTPBase(24643): [15] Network BandWidth = 102 Kbps

Answer 1

The code in onCreate() suggests you have a misconception about how MediaCodec works. Your code is currently:

onCreate() {
    ...setup...
    input();
    output();
}

MediaCodec operates on access units. For video, each call to input/output would get you a single frame of video. I haven't worked with audio, but my understanding is that it behaves similarly. You don't get the entire file loaded into an input buffer, and it doesn't play the stream for you; you take one small piece of the file, hand it to the decoder, and it hands back decoded data (e.g. a YUV video buffer or PCM audio data). You then do whatever is necessary to play that data.

So your example would, at best, decode a fraction of a second of audio. You need to be doing submit-input-get-output in a loop with proper handling of end-of-stream. You can see this done for video in the various bigflake examples. It looks like your code has the necessary pieces.

You're using a timeout of -1 (infinite), so you're going to supply one buffer of input and wait forever for a buffer of output. In video this wouldn't work -- the decoders I've tested seem to want about four buffers of input before they'll produce any output -- but again I haven't worked with audio, so I'm not sure if this is expected to work. Since your code is hanging I'm guessing it's not. It might be useful to change the timeout to (say) 10000 and see if the hang goes away.

I'm assuming this is an experiment and you're not really going to do all this in onCreate(). :-)

Answer 2

For anyone still looking for an answer to the problem of playing streamin audio reliably, you might want to have a look at this project (based on the MediaCodec API)

https://code.google.com/p/android-openmxplayer/

Answer 3

There are two issues with the above code. First, as the accepted answer states, only one read is done from the input stream. However, secondly, a call to .play() is needed on the AudioTrack.

This modification fixes OPs code:

mAudioTrack.play();

do {
    input();
    output();
} while (!sawInputEOS);