What is Live Streaming and How Does Live Streaming Work?

Live streaming is the live transmission of audio and video content live over the Internet. As live video becomes more popular, live streaming has become a critical part of many businesses and organizations’ marketing strategies. Live streams can be used for everything from broadcasting an event to providing customer service to hosting webinars.

The article will explore what live streaming is, how it works, and how you can use it in your business or organization. We look at the architecture of a typical live streaming setup and walk you through the process of transcoding, packaging, DRM, ad insertion, CDN-based delivery, playback, and other services that make up part of the live streaming pipeline.

At every stage, we’ll give you a quick idea of what happens and provide you with links to articles that take the topic further.

Note that the live streaming pipeline contains components, tools, and systems that we won’t cover in this article to keep the scope contained. For example, we aren’t going to do deep-dives into OBS Studio, cameras, mics, green screens, graphics, overlays, rendering, post-processing, subtitles, etc. But, stay tuned for deep dives into all of these 🙂

Anyways, with this quick introduction, let’s get started on our live streaming journey.

Table of Contents

Video Capture and Transfer to a Live Streaming Software/Platform

The first aspect of live streaming is to capture the video using a video camera. The size, configuration, and capabilities of the camera are dependent on the application. If you are live streaming yourself to your friends, perhaps, your laptop’s inbuilt camera might suffice. But, if you are live streaming a concert, or a sporting event, then you are better off using professional equipment (lights, cameras, rigs, etc.).

For large events (political rallies, concerts, sporting events), the camera feed is transmitted using an OB van to broadcast stations via satellites. At these local/national broadcasting stations, the videos are received and processed to add graphics, slates, ads, ticker-text, etc. and then sent off to the rest of the livestreaming pipeline for transcoding, etc.

For smaller-scale productions, the camera feed can be processed on-site on a computer using dedicated software like OBS Studio that can be used to add graphics, slates, etc. and then either compressed on-site or in the cloud using dedicated live streaming platforms.

In the next section, let’s take a look at video transcoding – one of the most important steps in live streaming.

Video Transcoding

Video Encoding is the art and science of reducing a video’s size or bitrate without adversely impacting its quality ( as perceived by a human) and is also referred to as compression. Video transcoding (another word you’ll come across often) typically refers to re-encoding compressed files, and it involves an additional step to decode the incoming video before compressing it.

Typically video transcoders have a lot of capabilities, such as the ability to

Encode and Decode bitstreams of different video codecs such as H.264/AVC, HEVC, AV1, VP9, etc.
Convert to and from different container formats (mp4, ts, fmp4, WebM, mkv, etc.).
Resize videos by upscaling and downscaling to produce outputs of different resolutions. This is critical to ABR video streaming or multi-rate broadcast. As an example, a 1080p input might be used to create 1080p, 720p, 640p, and 480p outputs.

Video transcoding is performed by a set of algorithms and tools referred to as a video codec. Groups of companies and people develop these after a lot of discussion, debate, and experimentation. Once a video codec is standardized, anyone can purchase a license to produce an encoder/decoder as per the specification. Popular video codecs involve H.264/AVC, HEVC, AV1, VP9, LCEVC, VVC, EVC, and more!

Transcoding in Live Streaming — **H.264/AVC vs. LCEVC with H.264/AVC as the base codec**

Here are some articles to help you understand more about video compression and codecs –

ABR Live Streaming using HLS and MPEG-DASH

After transcoding, the encoded bitstreams are sent to a Packager to create ABR-Compliant streams.

What is ABR Streaming? ABR or Adaptive BitRate Streaming is a technique of video delivery that adapts to the player’s bandwidth conditions and player’s buffer fullness. If the bandwidth conditions at the player are good, then the player requests higher quality versions of the video, and if the player’s bandwidth conditions or buffer fullness drops, then the player requests lower quality versions of the video to prevent buffering.

For an introduction to ABR, please read OTTVerse’s Introduction to ABR Streaming article.

image 768x258 1 — MPEG-DASH and HLS (HTTP Live Streaming) are the dominant HTTP-based ABR video streaming protocols

To support HTTP-based ABR video streaming, the output of the transcoder needs to be transformed, and converting videos into HLS or MPEG-DASH compliant streams is called Packaging. Packaging involves splitting the video into chunks and recording the chunking numbering, order of delivery, side information, metadata, etc., into a file called the manifest (DASH) or a playlist (HLS).

The manifest is first delivered to the player. Using the manifest as a guide, a video player can request the appropriate video chunks based on their buffer fullness and available bandwidth.

For more information on packaging, HLS, and MPEG-DASH, please read the following articles on OTTVerse.com.

The packaged streams are then protected against piracy and theft using DRM and Content Protection Technologies. Let’s look at that next.

Digital Rights Management (DRM) for Live Streaming

Digital Rights Management (DRM) refers to a system or technology used to protect your videos from piracy, illegal re-streaming, downloading, and other forms of unauthorized video content consumption. DRM gives you the flexibility to choose who gets to consume the streamed content via business rules and a secure, protected communication protocol and is a very important part of a live streaming service.

How does DRM Work?

DRM works by encrypting content using symmetric-key algorithms where encryption and decryption are performed using the same key.
First, the content is encrypted using a Key (typically, AES-128) and then delivered to the client. The key is served securely in a dedicated server.
When the client wants to play the encrypted video, it sends a request to the DRM server for the decryption key.
The server checks the client’s authenticity, and if the client is authentic, the server sends the decryption key to it, along with licensing rules.
After receiving the decryption key, the client uses secure software called the CDM (Content Decryption Module) to decrypt the video, decode it, and securely send it to the screen.

There are multiple licensing rules that streaming providers can configure to control who gets to watch the video, when, and where. In most commercial DRM implementations, you can decide to,

block people from certain countries,
allow access to the content for a certain period (rentals, TVOD),
prevent a user from casting the movie onto a screen,
block free users from accessing premium content,
block playback on specific devices, and more.

DRM plays a vital role in reducing piracy and ensuring that content creators can monetize their efforts. Commercially, there are many trusted DRM technologies such as Microsoft’s PlayReady, Google’s Widevine, and Apple’s FairPlay. And there are DRM vendors who provide additional infrastructure around these DRM solutions by adding more business rules, analytics, and tools for publishers.

For a detailed look at DRM, please dive into OTTVerse.com’s Hitchhiker’s Guide to DRM. Here are some of the articles that make up the DRM series.

Ad Insertion in Live Streaming

AVOD or Advertising-based Video On Demand is a VOD business model that generates revenue by inserting advertisements into the video streaming.

Typically, AVOD services are free-to-watch because the publisher (your streaming provider) makes money every time you watch an ad or click on it. However, streaming providers can bundle advertising into Freemium services, where you can watch ad-free videos if you sign up for a subscription (AVOD-SVOD hybrid).

Ad insertion via CSAI and SSAI for live streaming

The dominant ad serving standards are VAST (Video Ad Serving Template) and VPAID (Video Player-Ad Interface Definition), as defined by the IAB. Using these standards, the entire ad ecosystem can function cohesively right from creating, tagging, serving, and tracking ads.

In regards to ad delivery, there are two ways of inserting ads into videos – CSAI (Client-Side Ad Insertion) and SSAI (Server-Side Ad Insertion).

CSAI works by making calls from the video player to an ad server that responds with the ad creative. Thus, CSAI is driven by the player.
SSAI works by inserting (or stitching) the ad media directly into the video stream, avoiding the need for making calls to a server to receive ads. SSAI’s advantage is its ability to avoid ad-blockers because the ads are initiated from the server and not the client.

But, how does a player know when to play ads? This information is transmitted to the player via a manifest/playlist that has been decorated with ad-break information. The manifest will clearly specify when an ad is to be played, where to fetch the ad, and where to send ad-tracking data.

Where can ads be inserted? Ads can be inserted either before the video (pre-roll), during the video (mid-roll), or after the video ends (post-roll), and the choice is up to the streaming service provider which combination of ad-positions to use. They can also choose to bunch videos together into ad-pods (i.e., back-to-back ads at a given ad break).

For more information on ad insertion, check out these articles from OTTVerse.com

CDN For Live Streaming

CDNs or Content Delivery Networks are geographically distributed servers used to deliver live streaming video quickly and efficiently. A CDN sits between the video players (clients) and the origin servers to deliver video content across vast geographical regions. They scale efficiently with traffic, shield the origin servers from getting overloaded, reduce latency, and ensure a smooth viewing experience.

Here is a brief description of how a CDN for live streaming works –

After transcoding, packaging, DRM, ad-insertion, etc., the final streams are placed on an Origin Server (a web server, basically).
A CDN is configured with the Origin’s URLs (i.e., the end-points where the video streams are available).
The video players are programmed to ask the CDN for the video segments and not go directly to the Origin Server.
When the first play request reaches the CDN, it most likely does not have the content in its cache. So, it relays the request to the Origin Server, and when it receives the response from the Origin, it caches the content and sends it to the player.
When a player asks for the same video segment, the CDN first checks its cache to see if it already has it cached. If yes, the CDN serves that video from its cache, and if not, then the CDN asks the Origin Server to send the segment to it.

A few of the benefits of using CDNs for live streaming are –

Scalable video delivery.
Reduced latency and buffering.
Reducing the load on the origin servers.
Improving the streaming video quality.
Protecting the origins from DDoS attacks.

For more information on CDNs, check out these articles on OTTVerse.com

Video Playback for Multiple Screens

Until now, we have transcoded, packaged, protected the streams using DRM, inserted Ads or Ad Markers, stored the videos on origin servers, and set up a CDN for delivery. Now what’s left is to develop video players for people to play videos and enjoy!

Video Players come in all shapes and sizes, and they need to be custom-built for the operating system and environment. This is because people watch videos on laptops, gaming consoles, smartphones (iOS, Android), Smart TVs, tablets, connected TV interfaces (Roku, Chromecast, Amazon FireTV). Due to this diverse ecosystem, video player development and maintenance require a diverse skill set, including programming in different programming languages, understanding DRM, Ad Insertion, API usage, etc.

Let’s look at some video players for different platforms –

Android: Exoplayer
iOS, tvOS: AVPlayer
Roku (written in Brightscript), Amazon Fire TV, Chromecast for Connected TVs
Web/HTML5
Samsung TV’s AVPlay for the Tizen operating system and LG WebOS Players

There are several open-source players like Exoplayer, VideoJS, HLS.js, dash.js, Shaka player, etc. Apart from this, many companies provide video players for a fee and offer support. These commercial players are tested for compatibility with HLS, DASH, DRM, Ad Insertion, Subtitles, and (often) come with extensive APIs for gathering playback information.

For a list of free online players, check out these articles on

Video Analytics – QoE and QoS

The world runs on data – so why shouldn’t your live streaming platform? Live streaming providers should use an analytics service or build their own to collect usage, QoE, QoS statistics for their live streams.

video analytics for a live streaming service

Not only will this data tell you how your service is performing, but it’s a proven fact that almost every part of a video streaming pipeline can be optimized with the right quality and quantity of data.

Some of the most important stats that can be collected at the player are –

Latency or Start-Up Delay: This measures how long it takes for the video to show up after the user presses Play.
Buffering Duration: how long has the video been buffering?
Buffering Percentage: For a certain amount of video watched, how much has been spent in a buffering state?
Number of Plays (or Starts)
Number of Completed Plays
Exit Before Video Start: After the user presses Play, does the user leave the video before the first frame shows up? If yes, why?
For AVOD providers, it’s important to track the Ad Fill Rates, Ad QoE, Completion Rates, Quartiles of Completion, etc., to optimize ad delivery.

Capturing information about the subscribers, videos, etc., and tying them together can give your marketing, sales, and content production/curation teams a lot of useful data to optimize their future campaigns.

To learn how analytics can impact your service, please read this article on Ad Fill Rates and their Calculation on OTTVerse.com.

Subscription & Payment Services

Subscription & Payment Services are necessary to manage your subscribers, accept payments, follow-up on renewals, etc., to keep your PVOD, SVOD, TVOD (PPV) business running. Most people only think of encoders and CDNs when they imagine building a live streaming service, but, in fact, auxiliary services like Billing, Payment, Subscription services are crucial to setting up and running your business!

TVOD (pay-per-view) providers also need services and tools to manage payments, allow only paying users to access content, restrict access for a limited time period, and impose certain restrictions. This can be achieved with a payment processing service, subscription management service, and DRM with rental policies.

There is no point in re-inventing the wheel and spending 100s of hours building these services for both subscription management and payment services you are better off using existing providers.

Conclusion

As you’ve seen, Live Streaming requires many moving components that need to work in tight unison for a great user experience. You can either set up all the components yourself using open-source or COTS applications; or, you can opt-in for OVPs (Online Video Platforms) that typically contain all these components and present a short time-to-market.

There is always a trade-off between cost, time-to-market, maintenance costs, quality, etc., between DIY and buying off-the-shelf products. You’re better off deciding between the two after closely examining all the factors that affect your business.

I hope this article was informative and you understand what live streaming is and how it works. Until next time, stay safe, and happy streaming!

Subscribe to OTTVerse, and you’ll get articles like this directly to your inbox every two weeks.

Krishna Rao Vijayanagar

Website

Krishna Rao Vijayanagar, Ph.D., is the Editor-in-Chief of OTTVerse, a news portal covering tech and business news in the OTT industry.

With extensive experience in video encoding, streaming, analytics, monetization, end-to-end streaming, and more, Krishna has held multiple leadership roles in R&D, Engineering, and Product at companies such as Harmonic Inc., MediaMelon, and Airtel Digital. Krishna has published numerous articles and research papers and speaks at industry events to share his insights and perspectives on the fundamentals and the future of OTT streaming.