Crash when using the GvrAudioProcessor with mono audio channel video

[Oleur]

Issue description

Videos with one audio channel cannot be played when the player uses the GvrAudioProcessor. An exception (UnhandledFormatException) is thrown from the GvrAudioProcessor in the configure method.

Reproduction steps

Override the buildAudioProcessors method in SimpleExoPlayer.

player = new SimpleExoPlayer(...) {
  @Override
  public AudioProcessor[] buildAudioProcessors() {
    gvrAudioProcessor = new GvrAudioProcessor();
    return new AudioProcessor[] {gvrAudioProcessor};
  }
};

Setup the player with the mono audio channel video and start playback.

Link to test content

Every video with only one audio channel

Version of ExoPlayer being used

ExoPlayer 2.3.1

Device(s) and version(s) of Android being used

Nexus 5X - Android 7.1.2

A full bug report captured from the device

04-20 11:06:46.458 19432-19642/cinemur.apptest E/ExoPlayerImplInternal: Renderer error.
                                                                        com.google.android.exoplayer2.ExoPlaybackException
                                                                            at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:250)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506)
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466)
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300)
                                                                            at android.os.Handler.dispatchMessage(Handler.java:98)
                                                                            at android.os.Looper.loop(Looper.java:154)
                                                                            at android.os.HandlerThread.run(HandlerThread.java:61)
                                                                            at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40)
                                                                         Caused by: com.google.android.exoplayer2.audio.AudioTrack$ConfigurationException: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                            at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:492)
                                                                            at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                            at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                            at android.os.Looper.loop(Looper.java:154) 
                                                                            at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                            at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 
                                                                         Caused by: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                            at com.google.android.exoplayer2.ext.gvr.GvrAudioProcessor.configure(SourceFile:106)
                                                                            at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:490)
                                                                            at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                            at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                            at android.os.Looper.loop(Looper.java:154) 
                                                                            at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                            at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 
04-20 11:06:46.460 19432-19432/cinemur.apptest E/EventLogger: playerFailed [0.53]
                                                              com.google.android.exoplayer2.ExoPlaybackException
                                                                  at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:250)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506)
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466)
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300)
                                                                  at android.os.Handler.dispatchMessage(Handler.java:98)
                                                                  at android.os.Looper.loop(Looper.java:154)
                                                                  at android.os.HandlerThread.run(HandlerThread.java:61)
                                                                  at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40)
                                                               Caused by: com.google.android.exoplayer2.audio.AudioTrack$ConfigurationException: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                  at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:492)
                                                                  at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                  at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                  at android.os.Looper.loop(Looper.java:154) 
                                                                  at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                  at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 
                                                               Caused by: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                  at com.google.android.exoplayer2.ext.gvr.GvrAudioProcessor.configure(SourceFile:106)
                                                                  at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:490)
                                                                  at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                  at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                  at android.os.Looper.loop(Looper.java:154) 
                                                                  at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                  at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 

Partial fix

I came up with a solution but I don't know if this is the right way to do it. (https://gist.github.com/Oleur/b2770cf7c225bf5c0d3861925d6fb576). It would be nice to pass through the spatialization if the audio format is not supported by the GvrAudioSurround and process the audio normaly.

[andrewlewis]

I'm following up to find out whether GVR could handle mono audio. I'm not sure that just passing through the the input is the best option, as the audio wouldn't respond to head rotation.

If you only need to handle mono input, as a temporary workaround you could try duplicating the channel to make it stereo. One way to do that is to make ChannelMappingAudioProcessor publicly visible and add one before the GvrAudioProcessor in your buildAudioProcessors method. Then call setChannelMap(new int[] {0, 0}); to make it output a stereo stream with the first channel of input duplicated. I'm not sure about the correctness of this option from a spatial audio standpoint, though.

[andrewlewis]

There's work underway to add support for handling mono input in the GVR audio SDK. I don't have an ETA yet but I'll update this issue when the functionality is available via GvrAudioProcessor.

[Oleur]

Thanks for the update!