After the release of Qiniu Live Cloud in June 2016, it has helped customers to solve various problems, such as live broadcast stuck, mosaic, flower screen, black screen, noise, and the lack of synchronization of sound and picture. Some of these are network reasons, some are the use posture of developers, and some are parameter configuration errors. Of course, There are also some problems with the SDK itself.
To sum up, if developers can have a deeper understanding of some basic knowledge in the live broadcast field and master some basic troubleshooting methods, many problems can be solved quickly by themselves, or even better prevented.
Therefore, following Detailed Explanation of Live Broadcasting Technology After the series of articles, we launched this new series, "Troubleshooting Difficulties and Miscellaneous Problems of Live Broadcasting". We will gradually share our experience in helping customers solve live broadcasting problems, and at the same time, we will also intersperse some basic knowledge and optimization experience in audio and video development, hoping to help developers in the direct broadcasting field.
The contents covered in this series include but are not limited to the following topics:
-
Playback failed
-
Live broadcast of Carton
-
Slow start
-
High delay
-
The sound and picture are not synchronized
-
Mosaic serious
-
Play black screen, flower screen and green screen
-
Play noise, noise, echo
-
Inaccurate on-demand dragging
-
Live broadcast heating problem
-
Other questions (to be continued)
This is the fourth article in the "Live Troubleshooting" series. Let's take a look Delay of live broadcast 。
### Delay measurement
Generally, the simplest way to measure the delay is to face the same clock at the streaming end and the playback end, and then subtract the time displayed at the streaming end from the time displayed at the playback end to get a rough live broadcast delay.
### Analysis of high delay
First, let's look at the modules that may cause delay:
- Image processing delay, such as picture clipping, beautification and special effect processing
- Video encoding/decoding delay
- Delay of network transmission
- Buffer in business code
Generally, the delay caused by image processing, data copying, encoding and decoding is at the ms level. The places that really cause large delay are the network transmission delay on the Internet and the buffer in the business code.
Network transmission delay
When data is transmitted on the network, it is inevitable that there will be physical delay when it is forwarded from one node to another through multi-level servers. The following table gives the theoretical network transmission time of data in the optical fiber (the delay in the actual scenario is often much larger than this, because it involves bandwidth, network jitter and other interferences):
It can be seen from the table that the closer the physical distance between the playback end and the streaming end or edge server node, the smaller the delay will be.
Buffer in business code
The buffer in the business code is mainly the buffer at the streaming end and the buffer at the playback end. For a 30 fps video stream, the delay will increase by 1s for every 30 frames left in the buffer. So how do they generate buffered data?
>>>>How does the data on the streaming end "accumulate"?
Collection ->Coding ->Data Transmission ->[Server] When the network jitters, the "data transmission" will slow down, resulting in a certain amount of blocking, so that these data will be "accumulated" in the transmission buffer of the streaming end.
>>>>How does the data on the player end "accumulate"?
[Server] ->Data reception ->decoding ->rendering When the network jitters, the server's data cannot be transmitted to the player in a "timely manner". Due to the reliability of the TCP protocol, all data will be accumulated by the server. When the network recovers well, it will be transmitted to the player in a fast speed, and these data will be passively "accumulated" in the receive buffer.
>>>>How to eliminate the cumulative delay of the service buffer?
The sending buffer of the streaming end can quickly send out when the network recovers well, thus eliminating the cumulative delay.
The receiving buffer at the playback end can quickly consume the data in the buffer by dropping frames or accelerating playback, thus eliminating the cumulative delay.
Protocol delay
Generally, there are three standard live broadcast protocols: RTMP, HLV, and HLS. Generally, the delay of RTMP/HLV protocol is 1~3s, while that of HLS protocol is greater. For live broadcast applications that pay attention to delay, most of them choose RTMP/HLV protocol. These protocols are tcp based protocols, The multiple features of tcp protocol lead to its delay being significantly higher than that of udp based private protocols, mainly in the following aspects:
- Three handshakes to establish a connection
- ACK mechanism
- Packet loss retransmission
Therefore, if you want to essentially solve the problem of live broadcast delay, you still need to change to a private protocol based on udp to transmit data.
## Summary
This is about the problem of high playback delay. In the next chapter, we will discuss The sound and picture are not synchronized This topic is discussed.
Author: Lu Jun @ Qi Niuyun 。 If you are interested in questions that are not in the above list, you can also write** lujun.hust@gmail.com **Exchange, welcome to follow Sina Weibo @ Lu Jun perhaps WeChat official account @ Jhuster Get the latest articles and information.
