Using the Media Panel in Google Chrome to Debug Media Players

It is critical to have the right tools to build and debug media players, and today, we shall talk about the Media Panel in Google Chrome’s Developer Tools. There is a ton of information that the Media Panel exposes that is very useful for media engineering.

In this tutorial, I’ll be using a mix of Amazon Prime Video, YouTube, and Netflix to demonstrate different aspects of the Media Panel in Google Chrome.

Let’s start!

Where is the Media Panel in Google Chrome?

First off, you need to locate and open the Media Panel within Google Chrome’s Developer Tools. The Media Panel in Google Chrome is hidden as it’s not frequently used, and to access it, you need to do the following –

1. Right-Click on the screen, and choose “Inspect.” Or press Ctrl + Shift + I on a Windows machine.
2. Click on the More Options button (three dots) > More tools > Media to open the Media panel.

Once you click on “Media”, you get access to the Media Panel in Google Chrome and are now ready to start debugging your players.

Let’s explore each all the tools and information inside the Media Panel tool in the following sections.

Properties Tab in the Media Panel

I opened a video on YouTube (https://www.youtube.com/watch?v=Z1ut5GVm-HU) and then opened the Media Panel on Google Chrome.

The first tab that you see when you open the Media Panel is the Media Panel’s Properties tab. It shows the Playback Resolution, Playback URL, Playback Title, and information about the Video and Audio decoders, as you can see below.

If there are multiple video players in the webpage, then you can choose them one at a time using the “Players” panel on the left.

In this example, you can see that the Properties tab is telling us that the

1. 1920x1080 (1080p) variant of the video is being played back.
2. it gives you the video URL
3. it gives you the title of the video (Aussies grab late wickets after Green, Smith fire | Vodafone Test Series 2020-21)
4. MojoVideoDecoder is the video decoder, and the video is being decoded at the hardware level (hw-acceleration).
5. FFmpegAudioDecoder is being used to decode the audio track.

With that quick intro to the Properties tab, let’s move on to the Events Tab.

Events Tab in the Media Panel

As the name suggests, this logs all the messages that the player throws and it contains a whole lot of handy information.

This time, I turned on Netflix and played Season 1, Episode 1 of Bodyguard. The Events tab is useful in tracking all the information sent by the player embedded inside Chrome. A list of the player states can be accessed in this header file from the Chromium project.

For example, the first event kWebMediaPlayerCreated means the follow from the Chromium project’s source code –

  // The media stack implementation of the blink media player has been created
  // but may not be fully initialized.
  kWebMediaPlayerCreated,

By referring to the definitions of the events, its easy to follow along and figure out if anything is going wrong in the media playback.

If you follow the series of events from the native player, you can see that it starts playing back the 720 variant, and this is presumably because Netflix throttles video quality to 720p on HTML5 and Chrome (refer: https://help.netflix.com/en/node/23742).

Then, I pressed the Pause button and waiting a long time. This put the player into the Suspend state (obvious from the events in the Log), and when I pressed Play again, it went into the Resume state and then back to the Playing state after ensuring that the buffers are full.

Okay, now let’s see what sort of messages are thrown when the signal strength changes frequently or drops abruptly. What happens to the playback, and how can we inspect the impact?

Inspecting Playback using the Events Tab

Switching back to YouTube and the same video, I decided to throttle the signal strength using the Network tab in Chrome and chose “Slow 3G” to force the player to change the playback resolution from what it started with, initially.

This resulted in a very descriptive and interesting series of events that I have copy-pasted below for you. As you can see from the color-coded messages, the player starts at 360p and moves up to 720p quickly upon detecting a good network connection. But then I turned on the Slow 3G mode, which throttled the network connection, forcing the player to drop the playback resolution quickly.

You can clearly see that the player is in an Underflow state.

This is because the sudden drop in the network speed does not allow the video segments to arrive in time. This will usually result in the video player retrying to download the same segment, but at a lower resolution. Finally, in my case, the player settles at a poor 144p, which it can sustain!

0.000	kWebMediaPlayerCreated	{origin_url: “https://www.youtube.com/”}
0.000	kLoad	{url: “blob:https://www.youtube.com/4233359f-d08a-4ca1-ac3d-41e762dc9a81”}
0.000	kPipelineStateChange	{pipeline_state: “kStarting”}
0.678	kPlay	{}
1.038	kPipelineStateChange	{pipeline_state: “kPlaying”}
1.039	kBufferingStateChanged	{audio_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
1.063	kBufferingStateChanged	{video_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
0.681	kDurationChanged	{duration: 509.401}
1.201	kVideoSizeChanged	{dimensions: “640×360”}
1.202	kBufferingStateChanged	{pipeline_buffering_state: {for_suspended_start: false, state: “BUFFERING_HAVE_ENOUGH”}}
6.067	kVideoSizeChanged	{dimensions: “1280×720”}
12.162	kPause	{}
12.267	kSeek	{seek_target: 27.312516}
12.267	kPipelineStateChange	{pipeline_state: “kSeeking”}
12.268	kBufferingStateChanged	{audio_buffering_state: {state: “BUFFERING_HAVE_NOTHING”}}
12.268	kBufferingStateChanged	{video_buffering_state: {state: “BUFFERING_HAVE_NOTHING”}}
12.268	kPipelineStateChange	{pipeline_state: “kPlaying”}
12.270	kBufferingStateChanged	{audio_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
12.283	kPlay	{}
12.380	kBufferingStateChanged	{video_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
12.388	kBufferingStateChanged	{pipeline_buffering_state: {for_suspended_start: false, state: “BUFFERING_HAVE_ENOUGH”}}
15.080	kBufferingStateChanged	{audio_buffering_state: {reason: “DEMUXER_UNDERFLOW”, state: “BUFFERING_HAVE_NOTHING”}}
15.080	kBufferingStateChanged	{,…}
17.874	kBufferingStateChanged	{audio_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
17.874	kBufferingStateChanged	{pipeline_buffering_state: {for_suspended_start: false, state: “BUFFERING_HAVE_ENOUGH”}}
27.588	kBufferingStateChanged	{video_buffering_state: {reason: “DEMUXER_UNDERFLOW”, state: “BUFFERING_HAVE_NOTHING”}}
30.589	kBufferingStateChanged	{,…}pipeline_buffering_state: {for_suspended_start: false, reason: “DEMUXER_UNDERFLOW”, state: “BUFFERING_HAVE_NOTHING”}for_suspended_start: falsereason: “DEMUXER_UNDERFLOW”state: “BUFFERING_HAVE_NOTHING”
48.405	kVideoSizeChanged	{dimensions: “854×480”}
48.407	kBufferingStateChanged	{video_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
48.407	kBufferingStateChanged	{pipeline_buffering_state: {for_suspended_start: false, state: “BUFFERING_HAVE_ENOUGH”}}
55.380	kVideoSizeChanged	{dimensions: “426×240”}
55.744	kBufferingStateChanged	{audio_buffering_state: {reason: “DEMUXER_UNDERFLOW”, state: “BUFFERING_HAVE_NOTHING”}}
55.744	kBufferingStateChanged	{,…}
56.317	kBufferingStateChanged	{audio_buffering_state: {state: “BUFFERING_HAVE_ENOUGH”}}
56.319	kBufferingStateChanged	{pipeline_buffering_state: {for_suspended_start: false, state: “BUFFERING_HAVE_ENOUGH”}}
65.880	kVideoSizeChanged	{dimensions: “256×144”}
77.328	kPause	{}

YouTube’s “Stats for Nerds” confirms this as well … you can see that the Connection Speed is around 223 kbps (close to 3G), which is not sufficient for good quality 720p streaming. This is why the native video player in Chrome had to down-switch to reach 144p.

Access to such tools is very important when debugging a media service.

Messages Tab in the Media Panel

The Messages tab has very interesting information that is critical in player debugging. Here is an example from Amazon Prime – all I did was click on the Play button and then Paused the video.

There is a ton of information that Google Chrome has exposed. For example.

Selected DecryptingVideoDecoder for video decoding, config: codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [1280,720], visible rect: [0,0,1280,720], natural size: [1280,720], has extra data: false, encryption scheme: CENC, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

This says that

1. The video codec is H.264/AVC High Profile, and it’s not able to parse the Level information
2. the video uses CENC encryption (if you don’t know what that is, check out our Fundamentals of DRM series).
3. The video uses BT709 color-space.

The same goes for the audio portion of the media. Here we see, that the audio decoder being used is FFmpegAudioDecoder; the audio is in aac format, 2-channel Stereo at 48kHz, and is unencrypted.

Selected FFmpegAudioDecoder for audio decoding, config: codec: aac, profile: unknown, bytes_per_channel: 2, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Signed 16-bit, bytes_per_frame: 4, seek_preroll: 0us, codec_delay: 0, has extra data: false, encryption scheme: Unencrypted, discard decoder delay: false

The messages then walk you through the DRM process where the player says that a decryption key was unavailable, and after receiving the key from the License servers, it resumes the decryption process.

DecryptingVideoDecoder: no key for key ID 385E014610FF4ECC8E055EE1EE757739; will resume decoding after new usable key is available

DecryptingDemuxerStream: key was added, resuming decrypt

Here is another example from Netflix where I switched the playback speed (a new feature in Netflix) and it showed up in the Messages tab of the Media Panel.

Timeline Tab in the Media Panel

The Timeline tool in Google Chrome’s Media Panel is another fantastic tool for debugging players. It gives you a visual representation of the playback status and the buffering status during video playback on a timeline. For a list of all the status messages, player states used inside Chromium’s media engine, please refer to this header file.

Here you can see the timeline of a session that I played in Amazon Prime Video. I basically let it play for ~20 seconds and Paused and Resumed several times. The dark orange visually represents the “Pause” periods, and the regions in lighter orange represent the player in its “Playing” state.

Below the playback timeline is the buffering timeline. To induce buffering, all I did was to seek 10 seconds ahead several times, and as you can see, it resulted in a small buffering state each time. This is understandable as the player has to request future segments, and this *may* cause a momentary pause in video playback.

To get more information, all you need to do is hover over the bars and the pop-up tells you the player’s state and the duration in which it was in that state.

However, I think the biggest drawback to the Timeline tab is the lack of pinch-and-zoom. I wish I had the ability to select a region of the graph and zoom into it. This is very useful when debugging long playback sessions in which all the different player states are basically squished into tiny lines!

Inspecting the HLS.js player

As a final example, I headed over to the HLS.js demo page (a popular open-source player for playing back HLS streams) and inspected its playback performance. Here is what the Messages tab reported. Lots of useful information, right?

All said and done, this is an excellent tool for streaming media debugging. With the information it provides, it should be easy to track down issues with the streaming manifests, or DRM configuration or Key delivery, or issues with the streams resulting in decoding problems.

Have you used the Media Panel in Google Chrome? Or, do you have an alternate tool that you prefer to the Media Panel? Do let me know! Thank you and see you next time on OTTVerse.com