RTMP (Real-Time Messaging Protocol) – How Does It Work

What is RTMP or the Real-Time Messaging Protocol?

RTMP or Real-Time Messaging Protocol is a proprietary, two-way communication protocol for low-latency, real-time audio, video, and data streaming over the Internet developed by Macromedia, which Adobe then acquired. RTMP works by establishing and maintaining a communication pathway between an RTMP Client and an RTMP Server for fast, and reliable data transmission.

RTMP was originally intended for streaming media with Adobe’s Flash player, but as we all know, Flash has been deprecated as of Dec. 2020. Does that mean RTMP is dead and dusted? Nope!

RTMP still finds several use-cases in the modern video streaming pipeline, especially at the ingest and egress from transcoders. And this is owing to the low-latency, real-time streaming nature of RTMP.

Most of the industry-standard encoders (encoding.com, Bitmovin, Harmonic, AWS Elemental, etc.) can produce RTMP output feeds. Similarly, streaming services like Twitch, YouTube, Facebook Live, and other live streaming platforms such as Dacast, Ant Media, Wowza can ingest RTMP feeds.

This article takes an in-depth look at the RTMP protocol including,

• RTMP’s history
• How does RTMP streaming work?
• How to establish an RTMP Connection?
• Alternatives to RTMP Video Streaming
• Pros and cons of RTMP Streaming.

Without further ado, let’s dive right into the history of the RTMP protocol.

History of RTMP (Real Time Messaging Protocol)

RTMP was introduced by Adobe for use in its super-popular Adobe Flash Player which was used by millions of websites to show video to their users. At its prime, Adobe Flash Player was probably on more than 90 – 95% of the websites that carried video content.

Adobe defines RTMP as follows –

The Real-Time Messaging Protocol (RTMP) was designed for high-performance transmission of audio, video, and data between Adobe Flash Platform technologies, including Adobe Flash Player and Adobe AIR. RTMP is now available as an open specification to create products and technology that enable the delivery of video, audio, and data in the open AMF, SWF, FLV, and F4V formats compatible with Adobe Flash Player.

Adobe

However, with the deprecation of Flash, RTMP is no longer used for video delivery to video players and is witnessing heavy competition from the likes of MPEG-DASH and HLS which are HTTP-based video transmission protocols. But, RTMP but still plays an important role for video transmission to and from encoders as we shall in future sections.

How does RTMP Streaming Work?

As we saw in the introduction, RTMP (Real Time Messaging Protocol) is a TCP-based communication protocol for two-way communication of data, audio, and video.

RTMP works by establishing and maintaining a communication pathway between an RTMP Client and an RTMP Server for fast, and reliable data transmission.

Similar to how HTTP-based transmission protocols like HLS and DASH behave, RTMP too, breaks a multimedia stream into fragments that are usually, 64 bytes for audio, and 128 bytes for video. The size of the fragments can be negotiated between the client and the server.

Conventional wisdom dictates that the size should be neither too small nor too big. Large fragment sizes can cause delays in write-operations and very small fragments can increase the load on the CPU.

By breaking the streams into fragments, RTMP can interleave fragments from different streams and transmit them on a single connection. This is generally referred to as “multiplexing,” similar to statistical-multiplexing in video broadcasting. However, in practice, packets containing several fragments are interleaved, and this makes RTMP streaming more efficient and allows RTMP to create multiple virtual, addressable streaming channels. At the decoder-side, these interleaved packets can be demultiplexed to retrieve the original audio and video data.

RTMP Connection Setup – Handshake, Connection, Stream

Now, let’s understand how an RTMP connection is created so that we have a better appreciation of how the RTMP protocol works. RTMP connection establishment can be divided into three important steps – Handshake, Connection, and Streaming.

Let’s take a look at each of these three steps.

Step 1: Handshake

The handshake process is relatively straightforward in RTMP streaming, and it is performed after establishing a TCP connection. In this handshake process, each side (client & server) sends three packets referred to as C0, C1, C2 and S0, S1, and S2 for the client and server packets.

Here’s an explanation of the RTMP handshake process –

1. The Client sends C0 to the server containing the RTMP version it’s requesting.
2. Then the Client sends C1 containing 1536 random bytes of data, and this is sent without waiting for the server to indicate that it has received C0.
3. At this point, the server MUST wait till it has received C0 to respond with S0 and, optionally, with S1. The server at this stage knows what RTMP version the client is requesting. The server responds with S0 and S1, which are essentially copies of C0 and C1.
4. The client and server then exchange C2 and S2, after which the handshake is considered to be complete, and a connection is established.

Step 2: Connection

The Connection step follows the Handshake step between the RTMP client and the RTMP server. In the connection process, the client and server exchange messages encoded using AMF encoding.

AMF stands for Action Message Format and is used to send messages between an Adobe Flash client and a Flash media server. Or, programmers can use AMF for serializing object graphs in ActionScript and XML. AMF is used in RTMP streaming to communicate between the client and server and to indicate the type and content of the message. For more on AMF, read this Wiki entry.

Here is an example is taken from here, and it shows the message sent by the client to the RTMP server. The connection URL, audio codec, video codec, and the AMF version used. In this example, it is version 3.0.

(Invoke) "connect"
(Transaction ID) 1.0
(Object1) { app: "sample", flashVer: "MAC 10,2,153,2", swfUrl: null,
              tcUrl: "rtmpt://127.0.0.1/sample ", fpad: false,
              capabilities: 9947.75 , audioCodecs: 3191, videoCodecs: 252,
              videoFunction: 1 , pageUrl: null, objectEncoding: 3.0 }

To which the RTMP server will respond with a message like this –

(Invoke) "_result"
(transaction ID) 1.0
(Object1) { fmsVer: "FMS/3,5,5,2004", capabilities: 31.0, mode: 1.0 }
(Object2) { level: "status", code: "NetConnection.Connect.Success",
                   description: "Connection succeeded",
                   data: (array) { version: "3,5,5,2004" },
                   clientId: 1728724019, objectEncoding: 3.0 }

During this step, the client and server also exchange “Set Peer Bandwidth” and “Window Acknowledgement Size” protocol messages. When successfully executed, these messages indicate the establishment of a connection, and now the server can stream the video data. For details on the values of audio/video codecs and other parameters, please refer to the RTMP specification.

Step 3: Stream

After the RTMP handshake and connection steps, a connection has been established between the RTMP client and the RTMP server, and now data can be served. To do this, the RTMP specification defines several commands such as,

• createStream
• play
• play2
• deleteStream
• closeStream
• receiveAudio
• receiveVideo
• publish
• seek
• pause

With the help of these commands, it is possible to transit video using the RTMP protocol.

Now that you have a basic understanding of how an RTMP connection works, let’s look at the definitions of some of the popular RTMP variants.

RTMPS, RTMPT, RTMFP, RTMPE, RTMP Proper – Variants of RTMP

There are several variants of RTMP for specific purposes and in this section, we take a brief look at them, starting off with RTMPS.

RTMPS: RTMPS is simply RTMP over a TLS/SSL connection. Setting up and using RTMPS is considered a little more involved and complicated than using RTMPE, but it guarantees a level of security. If you plan to stream to Facebook Live using RTMP, you need to use RTMPS (source).

RTMPE: RTMPE uses industry-standard cryptography consisting of the Diffie–Hellman key exchange and HMACSHA256. It generates a pair of RC4 keys where,

• the first one is used to encrypt the media data sent from the server to the client
• the second key is used to encrypt the data sent to the server.

RTMP Proper: This is just another name used for basic RTMP streaming over TCP using port number 1935.

RTMPT:  RTMPT is the use of RTMP with encapsulation within HTTP requests. This allows RTMP messages to pass through firewalls, and the message that is encapsulated can be either RTMP Proper, RTMPS, or RTMPE packets within.

RTMFP: RTMPF uses RTMP with the UDP protocol instead of TCP, and the RTMP Chunk Stream is not used in RTMFP. The RTMFP was designed for low latency, real-time, live audio, and video communication directly between peers (P2P) without the need for going through an RTMP server. For more details, go here.

Audio and Video Codec Support in RTMP

Before moving ahead, let’s take a look at the support for audio and video codecs in RTMP streaming. The header file (rtmp.h) reveals support for the following codecs –

• Audio: AAC, MP3 (go here to learn about audio codecs)
• Video: H.264/AVC, VP6 in FLV containers.

Where is RTMP Supported?

Several commercial and open-source encoders and streaming engines support RTMP – either at the input (ingest) or can produce an RTMP feed (egress). Notably, you can use

1. OBS Studio, a free broadcasting and live-streaming software to produce an RTMP feed.
2. FFmpeg (tutorial on RTMP streaming using FFmpeg)
3. Dacast.com
4. Bitmovin.com
5. Ant Media Server
6. Wowza

and more to either provide/produce RTMP feeds at the ingest or egress.

Alternatives to RTMP Streaming for Egress (Output)

With RTMP facing a slightly uncertain future owing to Adobe ending support for Flash, there are other alternatives you could consider for encoder egress (i.e., coming out of the encoder).

HLS or HTTP Live Streaming is a popular alternative to RTMP Egress. HLS is an established standard in the streaming community and is supported widely in the video ecosystem starting from encoders, packagers, encryption (DRM), CDNs, and playback on devices.

Another alternative is MPEG-DASH which is also an HTTP-based video streaming protocol. DASH has widespread support as well, similar to HLS and can be seen as an alternative to RTMP egress.

A problem with HTTP-based protocols is the delay that they add to the system. Typically, in HLS and DASH, a certain number of chunks of video have to be pushed out of the encoder before a DASH manifest or an HLS playlist is created. Without the playlist or the manifest, the players (or the rest of the ecosystem) cannot understand the streams being produced. Waiting for these playlists/manifests adds to the latency and can often add up to 45 seconds – 1 min of delay to the system. Click here to learn more about how these ABR-systems work.

However, low-latency DASH and HLS protocols and systems being developed could help reduce the latency of HTTP-based streaming and mitigate some of the headaches that come with HTTP-based streaming protocols.

Conclusion

I hope this introductory article on RTMP or the Real-Time Messaging Protocol was useful and informative to you. In future articles, we shall be looking into RTSP, the differences between RTMP and RTSP, and how to stream using RTMP with popular tools such as OBS Studio.

Until next time, take care and keep streaming!

Reference

1. RTMP specification released by Macromedia
2. RTMP v 2.0