Common-Media-Client-Data (CMCD) Explanation with Sample Player

One of the biggest problems that video streaming companies face is taking disjoint sets of information from different points in the video delivery pipeline and making sense of it.

If a client was buffering while playing Spiderman Returns from New York at 9 pm EST on a Friday, whose fault was it? Did the manifest have a problem in it? Or did the client’s ABR algorithm request for the wrong rendition for the measured bandwidth? Or did the CDN not have the media segments in its cache and had to perform several costly round-trips to the Origin (thundering herds?)?

Whose line fault is it anyway?

Need for the Common-Media-Client-Data (CMCD)?

From a practical and troubleshooting standpoint, correlating the client’s information/metrics/data with the CDN logs is very important. 

Let’s see why with an example –

• Assume you have 1 million subscribers logging on to your service each day and watching a mix of movies, short-form content, trailers, and songs.
• Your audience is geographically-spread and is serviced by several different PoPs.
• They use a variety of devices to watch the content from several different connection types (Broadband, 3G/4G/5G); some receive MPEG-DASH content and some receive HLS, and so on.

When you run an operation as diverse as this, it is very important to gather as much information as possible from different stages of your delivery pipeline, and then *somehow* bring these disjoint pieces of information together and make sense of it.

Let’s suppose a customer complains that he experienced a lot of buffering at 11 pm EST, sitting in New York.

• What can you get from CDN logs?
• Are you able to filter the logs down to a particular subscriber?
• How do you narrow it down to a particular session?
• A particular movie?

You can gather very detailed statistics from the media clients using QoE/QoS analytics services such as Conviva, NPAW, MediaMelon, Mux. These products can tell you quite precisely, who watched what, from where, and how. By this I mean, when you use a commercial analytics solution, you can typically get per-subscriber and per-session level information as seen from the media client.

Most commercial analytics solutions send data back from the players to their servers every N seconds with a lot of segment-level information (measured bandwidth, the data rates of the segments requested, buffering, playback events, and much more). In essence, you can get a snapshot of an entire streaming session – from start to finish.

Similarly, you can get logs from your CDN provider that’ll paint a different picture – as seen from the CDN.

Now, the difficulty is in bringing the client-side data and the CDN-data together and making sense out of it. The question is, do you have a way of,

• sifting through the CDN logs and looking at all the logs of a particular subscriber watching a certain movie?
• “joining/merging” the CDN and client-side logs for every video streaming session, individually?

This is a challenging problem because

• the volume of data is enormous on both ends of the delivery pipeline.
• a protocol needs to exist so that the information definition, exchange, and logging can be standardized.

However, if such a standard exists, then you have a lot more information on your hands – information that captures the journey of a media object from the CDN to the player, and then to the screen. You can

• take several thousand lines of server logs and tag them to the same session and subscriber
• identify which CDN (or PoP) is leading to buffering issues
• segment your data in terms of Live and VOD and then monitor the performance of your CDNs
• and much, much more!

Right?

Common-Media-Client-Data (CMCD)

In order to solve the problem of joining/associating client-side data with the CDN logs, the CMCD specification was developed by the CTA-WAVE (more about the CTA-WAVE project can be read here).

The purpose of CMCD is very clearly mentioned as follows –

This document outlines a simple means by which every media player can communicate data with each media object request and have it received and processed consistently by every CDN.

CTA-5004 Document

I like the wording in this sentence a lot.

It clearly says that “every” media player can communicate with “every” CDN and have the data received and processed “consistently.” This is important as you don’t want to have multiple different specifications for different player + CDN combinations. Having a uniform and useful spec such as the CMCD spec makes engineering lives easier on both ends of the pipeline!

For a deep understanding of CMCD, I recommend that you read the specification. It’s only a dozen (or so) pages long and written in simple, easy-to-follow language.

Metrics Defined in CMCD

Here are a list of all the metrics taken from the CMCD specification along with their definitions.

As you can see, the list of metrics covers very critical pieces of video transmission technology such as the encoded bitrate, buffer length, content ID, measured throughput, session ID, playback rate, etc. Gathering all this information will allow for very close communication between the client and CDN and aid in further testing, tracing errors, tuning, and troubleshooting.

Description	Key Name	Header Name	Type & Unit	Value definition
Encoded bitrate	br	CMCD-Object	Integer kbps	The encoded bitrate of the audio or video object being requested. This may not be known precisely by the player, however, it MAY be estimated based upon playlist/manifest declarations. If the playlist declares both peak and average bitrate values, the peak value should be transmitted.
Buffer length	bl	CMCD-Request	Integer milliseconds	The buffer length associated with the media object being requested. This value MUST be rounded to the nearest 100 ms. This key SHOULD only be sent with an object type of ‘a’,‘v’ or ‘av’.
Buffer starvation	bs	CMCD-Status	Boolean	Key is included without a value if the buffer was starved at some point between the prior request and this object request, resulting in the player being in a rebuffering state and the video or audio playback being stalled.  This key MUST NOT  be sent if the buffer was not starved since the prior request.   If the object type ‘ot’ key is sent along with this key, then the ‘bs’ key refers to the buffer associated with the particular object type. If no object type is communicated, then the buffer state applies to the current session.
Content ID	cid	CMCD-Session	String	A unique string identifying the current content. Maximum length is 64 characters. This value is consistent across multiple different sessions and devices and is defined and updated at the discretion of the service provider.
Object duration	d	CMCD-Object	Integer milliseconds	The playback duration in milliseconds of the object being requested. If a partial segment is being requested, then this value MUST indicate the playback duration of that part and not that of its parent segment. This value can be an approximation of the estimated duration if the explicit value is not known.
Deadline	dl	CMCD-Request	Integer milliseconds	Deadline from the request time until the first sample of this Segment/Object needs to be available in order to not create a buffer underrun or any other playback problems. This value MUST be rounded to the nearest 100ms. For a playback rate of 1, this may be equivalent to the player’s remaining buffer length.
Measured throughput	mtp	CMCD-Request	Integer kilobits per second (kbps)	The throughput between client and server, as measured by the client and MUST be rounded to the nearest 100 kbps. This value, however derived, SHOULD be the value that the client is using to make its next Adaptive Bitrate switching decision. If the client is connected to multiple servers concurrently, it must take care to report only the throughput measured against the receiving server. If the client has multiple concurrent connections to the server, then the intent is that this value communicates the aggregate throughput the client sees across all those connections.
Next object request	nor	CMCD-Request	String	Relative path of the next object to be requested. This can be used to trigger pre-fetching by the CDN.This MUST be a path relative to the current request. This string MUST be URLEncoded [5]. The client SHOULD NOT depend upon any pre-fetch action being taken – it is merely a request for such a pre-fetch to take place.
Next range request	nrr	CMCD-Request	String of the form “<range-start>-<range-end>”	If the next request will be a partial object request, then this string denotes the byte range to be requested. If the ‘nor’ field is not set, then the object is assumed to match the object currently being requested. The client SHOULD NOT depend upon any pre-fetch action being taken – it is merely a request for such a pre-fetch to take place. Formatting is similar to the HTTP Range header, except that the unit MUST be ‘byte’, the ‘Range:’ prefix is NOT required and specifying multiple ranges is NOT allowed. Valid combinations are:”<range-start>-“”<range-start>-<range-end>””-<suffix-length>”
Object type	ot	CMCD-Object	Token – one of [m, a,v,av,i,c,tt,k,o]	The media type of the current object being requested: * m = text file, such as a manifest or playlist * a = audio only * v = video only * av = muxed audio and video * i = init segment *c = caption or subtitle *tt = ISOBMFF timed text track *k = cryptographic key, license or certificate. *o = other If the object type being requested is unknown, then this key MUST NOT be used.
Playback rate	pr	CMCD-Session	Decimal	1 if real-time, 2 if double speed, 0 if not playing. SHOULD only be sent if not equal to 1.
Requested maximum throughput	rtp	CMCD-Status	Integer kilobits per second (kbps)	The requested maximum throughput that the client considers sufficient for delivery of the asset. Values MUST be rounded to the nearest 100kbps. For example, a client would indicate that the current segment, encoded at 2Mbps, is to be delivered at no more than 10Mbps, by using rtp=10000.   Note: This can benefit clients by preventing buffer saturation through over-delivery and can also deliver a community benefit through fair-share delivery. The concept is that each client receives the throughput necessary for great performance, but no more. The CDN may not support the rtp feature.
Streaming format	sf	CMCD-Session	Token – one of [d,h,s,o]	The streaming format which defines the current request * d = MPEG DASH * h = HTTP Live Streaming (HLS) * s = Smooth Streaming * o = other If the streaming format being requested is unknown, then this key MUST NOT be used.
Session ID	sid	CMCD-Session	String	A GUID identifying the current playback session. A playback session typically ties together segments belonging to a single media asset. The maximum length is 64 characters. It is RECOMMENDED to conform to the UUID specification [7].
Stream type	st	CMCD-Session	Token – one of [v,l]	* v = all segments are available e.g. VOD * l = segments become available over time e.g. LIVE
Startup	su	CMCD-Request	Boolean	Key is included without a value if the object is needed urgently due to startup, seeking or recovery after a buffer-empty event. The media SHOULD not be rendering when this request is made. This key MUST not be sent if it is FALSE.
Top bitrate	tb	CMCD-Object	Integer Kbps	The highest bitrate rendition in the manifest or playlist that the client is allowed to play, given current codec, licensing and sizing constraints.
CMCD version	v	CMCD-Session	Integer	The version of this specification used for interpreting the defined key names and values. If this key is omitted, the client and server MUST interpret the values as being defined by version 1. Client SHOULD omit this field if the version is 1.

This table was taken from the CMCD Spec. Please refer to it for the latest definitions and updates.

How do you send CMCD data from the Client?

There are clear guidelines laid out in the specification – so, it’s best to refer to it for the latest recommendations on transmitting information to the CDN in addition to guidelines for CDNs on processing incoming data.

In brief, the client data can be sent to the CDN by one of three means (pretty standard!):

• As a custom HTTP request header
• As an HTTP query argument 
• As a JSON object independent of the HTTP object request.

The thought process behind this is that, in any case, media clients need to communicate with the CDNs to receive video segments or manifests/playlists. Then, why not send some additional data to the CDNs while they are at it?

This enables the tagging of every request from a player to the CDN and can form the basis of future analysis and trouble-shooting.

Some of the requirements while sending data are (not a complete list) –

1. All the information must be sent in key-value pairs, and a comma should separate each pair.
2. An equals-to sign should separate the individual key and value.
3. Key-value pairs should be sequenced in alphabetical order of the key name to reduce the fingerprinting surface exposed by the player.
4. The recommendation is that the players supply a sid value for all the requests made for a particular session (manifest/playlist request, init files, segment requests, DRM, etc.). All the keys are optional and should only be sent if they make sense.

There are well-defined guidelines for how CDNs should process in-coming data. For e.g., if the keys are not recognized, a CDN should reject it and so on.

Let’s See CMCD in Action

dash.js 3.0.3 was released with a CMCD example and here is the link to a demo dash.js player with CMCD implemented in it. However, it appears that this player has some out-of-date code because the keys are not in alphabetical order.

Instead, you can head over to dash.js’s GitHub repo, download the latest tag (https://github.com/Dash-Industry-Forum/dash.js/archive/v3.2.0.zip), unzip it, and use the CMCD example (dash.js-3.2.0/samples/advanced/cmcd.html) to test the functionality.

Opening the cmcd.html file, you can see the following snippet of code in the player initialization step –

 player.updateSettings({
    streaming: {
        cmcd: {
            enabled: true, /* enable reporting of cmcd parameters */
            sid: 'b248658d-1d1a-4039-91d0-8c08ba597da5', /* session id send with each request */
            cid: '21cf726cfe3d937b5f974f72bb5bd06a', /* content id send with each request */
        }
    }
 });

Dummy cid and sid are created and sent with every payload. This is helpful in parsing and filtering through million of lines of CDN logs to narrow down on a certain session and content ID combination.

Another JS function in the same HTML file shows that the player events are sent as keys after being alphabetically sorted.

    function handleCmcdDataGeneratedEvent(event) {
        log("type: " +  event.mediaType);
        log("file: " + event.url.split("/").pop())
        var keys = Object.keys(event.cmcdData);
        keys = keys.sort();
        for (var key of keys) {
            log(key.padEnd(4) + ": " + event.cmcdData[key]);
        }
        log("");
    }

Here is how the demo player with CMCD enables looks. The demo player is set up to show the video on the left and a log of all the key-value pairs on the right – a very convinient way of testing the CMCD functionality.

And, finally, here is a snippet of the CMCD logs after I played back the video for a few seconds.

type: stream
file: bbb_30fps.mpd
cid : 21cf726cfe3d937b5f974f72bb5bd06a
ot  : m
sid : b248658d-1d1a-4039-91d0-8c08ba597da5
v   : 1

type: video
file: bbb_30fps_480x270_600k_0.m4v
cid : 21cf726cfe3d937b5f974f72bb5bd06a
ot  : i
sf  : d
sid : b248658d-1d1a-4039-91d0-8c08ba597da5
st  : v
su  : true
v   : 1

type: audio
file: bbb_a64k_0.m4a
cid : 21cf726cfe3d937b5f974f72bb5bd06a
ot  : i
sf  : d
sid : b248658d-1d1a-4039-91d0-8c08ba597da5
st  : v
su  : true
v   : 1

type: video
file: bbb_30fps_480x270_600k_1.m4v
bl  : 0
br  : 759
cid : 21cf726cfe3d937b5f974f72bb5bd06a
d   : 4000
dl  : 0
ot  : v
sf  : d
sid : b248658d-1d1a-4039-91d0-8c08ba597da5
st  : v
su  : true
tb  : 14932
v   : 1

type: audio
file: bbb_a64k_1.m4a
bl  : 0
br  : 67
cid : 21cf726cfe3d937b5f974f72bb5bd06a
d   : 4011
dl  : 0
ot  : a
sf  : d
sid : b248658d-1d1a-4039-91d0-8c08ba597da5
st  : v
su  : true
tb  : 67
v   : 1

As you can see, the cid and sid values are sent with every payload (or object request) to the CDN. This allows the CDN to precisely tag every incoming request to a specific session ID and a content ID.

Then, you can see the information sent for the manifest and video + audio segments.

• MPEG-DASH is being used (sf = d) for a VOD session st = v.
• The manifest is tagged with an object-type ot = m,
• The video and audio are tagged as v and a, respectively.
• Most of the keys are missing from the object sent with the manifest request (such as rtp, br, etc.) because they (throughput and bitrate) are irrelevant for the DASH manifest file.
• the keys are arranged in an alphabetical order,
• etc.

This sample CMCD implementation should prove to be quite useful for implementing it on media players.

Update: Feb. 13, 2021

Based on a comment by Will Law (Akamai’s Chief Architect), you can access the latest CMCD implementation within the dash.js player by going here.

“@Krishna – well written, accurate and informative article. One update – the dash.js guys have been hard at work and if you reference the /latest/ branch for the samples then you’ll pick up all their newest additions around CMCD, especially ‘nor’ and ‘nrr’ which are two of the hardest keys to implement. Reference implementation here: http://reference.dashif.org/dash.js/latest/samples/advanced/cmcd.html“

Conclusion

The CMCD is a great step towards being able to analyze video streaming performance using both client and CDN logs. I hope to see all the CDN vendors and analytics providers embrace this and help collectively drive a better video streaming experience.

Appendix

Below are a few useful resources on CMCD in general.

1. CTA WAVE: https://cta.tech/Resources/Standards/WAVE-Project
2. CTA Wave’s GitHub repo for reporting issues on CMCD: https://github.com/cta-wave/common-media-client-data/issues
3. Very nice blog post on CMCD: https://websites.fraunhofer.de/video-dev/dash-js-common-media-client-data-cmcd/
4. Will Law’s presentation at Mile High Video 2020: https://mile-high.video/files/mhv2020/session-6/Session-6-2_Law.pdf