CDNs or Content Delivery Networks are geographically distributed servers that are vital in live video streaming and VOD streaming infrastructures. A CDN sits between the video players (clients) and the origin servers to deliver video content across geographical regions and scale efficiently to ensure a smooth viewing experience for the clients.
In this article, we shall learn about how a CDN works, what happens when a CDN is not present, and also take a look at Cache-Hits and Cache-Misses and what they mean. Let’s get started!
Video Streaming Architecture
The following steps are common to most live and VOD streaming architectures.
- The source video is ingested and sent to a transcoder.
- The transcoder resizes the video and compresses it into different bitrate-resolution combinations (called Profiles).
- The compressed videos are sent to a packager and prepared for delivery via ABR technologies like HLS and MPEG-DASH. (Learn everything about packaging here).
- The packaged videos (chunks and manifests/playlists) are then stored on a streaming server or Origin Server. Finally, the URLs for the videos are published via the CMS and made known to the apps.
- When the user presses play on a video, the Origin Server responds to the requests from the players and delivers the requested chunks.
We just described the architecture of a live or VOD streaming service that * might * work well at the beginning and will soon fall apart when you start adding more users, subscribers, content, or suddenly a video goes viral.
Let’s talk about the viral video use-case for a minute.
Let’s assume that someone recorded a song and uploaded it to your UGC (User Generated Content) platform, and suddenly, it went viral.
The whole world wants to watch the song and the number of requests from the players to your origin server are off the charts.
What do you think is going to happen next?
Well, for starters, your origin server is going to get hammered with requests … 1000s every second for video chunks from the same 3-min long video. Like a stampede!
Note: Read this article to learn more about the Thundering Herd Problem in CDNs.
What is the origin server going to do in this situation??
Your origin server (or webserver) will struggle to serve all the requests. Even if the server is powerful, it won’t keep up due to the sheer volume of requests coming in.
Some players might be asking for the first segment of the video, while others might be asking for the last segment at different bitrates and resolutions. The origin server will soon not serve many of the requests either due to processing or network I/O limitations.
In the end, your end users will suffer several problems such as,
- Video buffering because the server isn’t able to serve them fast enough.
- Startup-delay because the server is overloaded and can’t serve the videos.
- Inferior video quality because the player starts switching down to lower bitrates because it cannot get high-quality (high-bitrate) video from the server promptly. That’s how ABR works!
All of the above lead to a terrible experience, and it is not the way to run a video-streaming service. And what we’ve described is not a rare situation – it is something that can happen almost daily for a popular video streaming service.
So, what is the solution?
Content Delivery Networks or CDNs
Let’s try to solve the problem by looking at the symptoms and what we’ve seen so far –
- A single server is forced to serve 100s or 1000s of clients (or video players) and cannot keep up with the demand.
- A single server in a particular location cannot serve video segments to clients in far-away locations or geographies.
- It’s a single-point-of-failure leading to a poor user experience.
- But, most importantly, a lot of the requests are for the same segments of video.
If we look at the above reasons, a pattern seems to be emerging and might lead us to an answer.
If all the clients ask for the same or a subset of the video segments, why not cache the video segments like what’s done on a computer’s cache? Why go to the disk each time?
So, let’s add a cache layer in front of the Origin Server that can cache frequently-asked video segments and serve them without going to the Origin each time.
Then, to serve different geographies, we can set up several of these caches worldwide to cater to users near them and provide fast responses.
This in-fact is the start of the design of a very simple Content Delivery Network or a CDN.
Okay, now, let us go a little deeper and understand how CDNs work.
- The packaged videos (perhaps, HLS or DASH) are stored on the origin server, and that path is made known to the CDN. So the CDN knows that all the movies from a streaming provider live on a bunch of servers and their IP addresses.
- Then the video players are programmed to ask the CDN for the video and not go directly to the Origin Server. So, the players are given the CDN’s URLs.
- When the very first play request reaches the CDN, the CDN most likely does not have the content in its cache. So, it relays the request to the Origin Server, and when it receives the response from the Origin, it caches the content and sends it to the player.
- The next time another video player (or the same one!) asks for the same video segment, the CDN first checks its cache to see if it already has it cached. If yes, the CDN serves that video from its cache, and if not, then the CDN asks the Origin Server to send the segment to it.
There are a couple of terminologies you need to know regarding CDNs –
- Cache Miss: A cache-miss occurs when a client requests the CDN for some particular content, and the CDN has not cached that content. When a cache miss occurs, the CDN sends a request back to the origin server for that missing content. When the Origin responds, the CDN caches the content and serves it to the client.
- Cache hit: A cache hit occurs when a client requests the CDN for some particular content, and the CDN has cached that content. The CDN, in this case, will serve the content back to the client’s device.
- Time to Live: A CDN does not cache segments or any other media indefinitely. It uses a variable called “Time to Live” or TTL to discard or flush its cache of the infrequently requested content. This cache-flushing is required to make space for newer content and to manage disk space intelligently.
Advantages of using a CDN
There are very clear advantages of using a CDN for video streaming – Live or VOD. Let’s take a look at a few of them.
- CDNs lead to less load on the Origin Server, and the CDN’s faster response ensures that a player can request high bitrate video chunks when its bandwidth is good and be certain that it will receive the segments in time to prevent buffer underflows.
- Reduced Infrastructure requirement in terms of Origin Servers as most of the load is absorbed by the CDN. In addition, this leads to lower bandwidth consumption for the Origin Servers.
- Easier to serve a large geographical region or several different geographies due to the CDNs distributed Points of Presence (PoPs).
- Security: CDNs also play a role in preventing DDoS attacks on origin servers since customers can set up rules to drop requests from certain clients or IP addresses. CDNs can thus act as the first line of defense.
I hope this explanation helped you understand what a CDN is, how it works, and the advantages of using a CDN. Several commercial CDNs like Akamai, Fastly, Cloudfare, KeyCDN, LimeLight, Medianova, and others do an excellent job delivering content to the end-users and improving the video watching experience – for different use-cases, architectures, and budgets.
In future articles of our video streaming series, we’ll look at concepts in CDN technologies such as Multi-CDNs, Edge Caches, Edge Computing, and more.
Until then, take care and happy streaming!