Audio performance with Javafx for Android (MediaPlayer and NativeAudioService)

Question

I have created, with JavaFX, a game on desktop that works fine (20000 Java lines. As it is a game, the Real Time constraint is important (response time of player\'s actions)

Answer 1

I've changed a little bit the implementation you mentioned, given that you have a bunch of short audio files to play, and that you want a very short time to play them on demand. Basically I'll create the AssetFileDescriptor for all the files once, and also I'll use the same single MediaPlayer instance all the time.

The design follows the pattern of the Charm Down library, so you need to keep the package names below.

EDIT

After the OP's feedback, I've changed the implementation to have one MediaPlayer for each audio file, so you can play any of them at any time.

1. Source Packages/Java:

package: com.gluonhq.charm.down.plugins

AudioService interface

public interface AudioService {
    void addAudioName(String audioName);
    void play(String audioName, double volume);
    void stop(String audioName);
    void pause(String audioName);
    void resume(String audioName);
    void release();
}

AudioServiceFactory class

public class AudioServiceFactory extends DefaultServiceFactory<AudioService> {

    public AudioServiceFactory() {
        super(AudioService.class);
    }

}

2. Android/Java Packages

package: com.gluonhq.charm.down.plugins.android

AndroidAudioService class

public class AndroidAudioService implements AudioService {

    private final Map<String, MediaPlayer> playList;
    private final Map<String, Integer> positionList;

    public AndroidAudioService() {
        playList = new HashMap<>();
        positionList = new HashMap<>();
    }

    @Override
    public void addAudioName(String audioName) {
        MediaPlayer mediaPlayer = new MediaPlayer();
        mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
        mediaPlayer.setOnCompletionListener(m -> pause(audioName)); // don't call stop, allows reuse
        try {
            mediaPlayer.setDataSource(FXActivity.getInstance().getAssets().openFd(audioName));
            mediaPlayer.setOnPreparedListener(mp -> {
                System.out.println("Adding  audio resource " + audioName);
                playList.put(audioName, mp);
                positionList.put(audioName, 0);
            });
            mediaPlayer.prepareAsync();
        } catch (IOException ex) {
            System.out.println("Error retrieving audio resource " + audioName + " " + ex);
        }

    }

    @Override
    public void play(String audioName, double volume) {
        MediaPlayer mp = playList.get(audioName);
        if (mp != null) {
            if (positionList.get(audioName) > 0) {
                positionList.put(audioName, 0);
                mp.pause();
                mp.seekTo(0);
            }
            mp.start();
        }

    }

    @Override
    public void stop(String audioName) {
        MediaPlayer mp = playList.get(audioName);
        if (mp != null) {
            mp.stop();
        }
    }

    @Override
    public void pause(String audioName) {
        MediaPlayer mp = playList.get(audioName);
        if (mp != null) {
            mp.pause();
            positionList.put(audioName, mp.getCurrentPosition());
        }
    }

    @Override
    public void resume(String audioName) {
        MediaPlayer mp = playList.get(audioName);
        if (mp != null) {
            mp.start();
            mp.seekTo(positionList.get(audioName));
        }
    }

    @Override
    public void release() {
        for (MediaPlayer mp : playList.values()) {
            if (mp != null) {
                mp.stop();
                mp.release();
            }
        }

    }

}

3. Sample

I've added five short audio files (from here), and added five buttons to my main view:

@Override
public void start(Stage primaryStage) throws Exception {

    Button play1 = new Button("p1");
    Button play2 = new Button("p2");
    Button play3 = new Button("p3");
    Button play4 = new Button("p4");
    Button play5 = new Button("p5");
    HBox hBox = new HBox(10, play1, play2, play3, play4, play5);
    hBox.setAlignment(Pos.CENTER);

    Services.get(AudioService.class).ifPresent(audio -> {

        audio.addAudioName("beep28.mp3");
        audio.addAudioName("beep36.mp3");
        audio.addAudioName("beep37.mp3");
        audio.addAudioName("beep39.mp3");
        audio.addAudioName("beep50.mp3");

        play1.setOnAction(e -> audio.play("beep28.mp3", 5));
        play2.setOnAction(e -> audio.play("beep36.mp3", 5));
        play3.setOnAction(e -> audio.play("beep37.mp3", 5));
        play4.setOnAction(e -> audio.play("beep39.mp3", 5));
        play5.setOnAction(e -> audio.play("beep50.mp3", 5));
    });

    Scene scene = new Scene(new StackPane(hBox), Screen.getPrimary().getVisualBounds().getWidth(), 
                        Screen.getPrimary().getVisualBounds().getHeight());
    primaryStage.setScene(scene);
    primaryStage.show();
}

@Override
public void stop() throws Exception {
    Services.get(AudioService.class).ifPresent(AudioService::release);
}

The prepare step takes place when the app is launched and the service is instanced, so when playing later on any of the audio files, there won't be any delay.

I haven't checked if there could be any memory issues when adding several media players with big audio files, as that wasn't the initial scenario. Maybe a cache strategy will help in this case (see CacheService in Gluon Charm Down).