Guide: In the field of real-time audio and video, how to realize the fully automated evaluation of video is a common problem in the industry. No reference (NR) technology does not need any information about the reference video, which is undoubtedly the dawn of fully automated video evaluation. This article come from Netease Smart Enterprise Senior Test Engineer , Primary This paper introduces a video non reference evaluation method recently used by Yunxin Test Group and its automation application scheme. The source and author of articles reproduced in various channels shall be indicated 。
1、 Background
The purpose of video quality assessment is to accurately measure the perception of human eyes on video content. After the source video is transmitted to the human eye through acquisition, encoding, transmission, decoding and other modules, it will inevitably introduce some compression distortion. In serious cases, there will even be green screen, vase, mosaic and other problems. In the industry There are two common methods to evaluate video quality: subjective video quality assessment and objective video quality assessment.
The common subjective experiment process is generally as follows:
1. Select representative source videos
2. Add the video processing scheme to be measured
3. Design subjective experiments according to ITU standards
4. Recruit volunteers to watch the video and give a score
5. Collect subjective scores and eliminate invalid data
6. Data modeling and experimental conclusions.
Objective assessment of video quality Quantify video quality through some evaluation criteria, main There are three types ， Full Reference (FR), Partial Reference (RR), No Reference (NR).
The full reference evaluation compares the difference between each pixel of the two videos before and after processing.
Some characteristics of the two videos are extracted from the partial reference evaluation, and they are scored accordingly.
The full reference assessment and partial reference assessment are all participatory assessments. Common assessment algorithms include PSNR, SSIM VIF VMAF PEVQ, etc. The assessment results can be obtained only by inputting and outputting video, which is suitable for offline testing.
No reference evaluation is conducted without reference to any original video, which is more suitable for online video evaluation, video enhancement, video merging and other scenarios.
Obviously, subjective evaluation is a long cycle, time-consuming and laborious process. It is not feasible to verify all video quality through subjective scoring. Objective assessment passed some Mathematical model similar to subjective quality assessment results Quantifying the perception of the human eye on video content can improve the efficiency of some evaluations.
In the video effect evaluation test of Netease Yunxin, the tests of encoder optimization (H264, X264 effect optimization), encoder upgrade (H264 upgrade to H265, H264 upgrade to VP8), Qos optimization, etc. are mainly subjective evaluation, supplemented by objective evaluation. Among the objective evaluation methods, the full reference evaluation methods such as PSNR, SSIM, VMA, etc. are mainly used in the optimization and upgrade test of the encoder. Of course, in addition to the full reference evaluation method, the Netease Yunxin test team is also trying to apply the non reference evaluation to audio and video related tests. The following will introduce a video reference free evaluation method recently used by Netease Yunxin.
2、 Video Quality Indicators
Video Quality Indicators is an open source audio and video non reference evaluation scheme, which provides non reference analysis of video jamming, noise, stripes, picture loss, exposure, etc. stay int El Open source QosTestFramework uses Video Quality Indicators as a method of non reference evaluation. Video Quality Indicators（http: // vq.kt.agh.edu.pl //m etrics.html）， Support windows, mac, linux. Evaluate the quality of video files by analyzing yuv files. The official website provides a variety of call schemes, including Bash, Python, Matlab, etc. It supports the analysis of sticking, picture missing, blocking effect, horizontal lines, noise spots, picture flicker, etc. The fields that mainly support analysis are as follows:
See http: //vq.kt.agh.edu.pl//metrics.html The following video shows the analysis effect of Video Quality Indicators (intercepted from the official website):
http://m.v.qq.com/play/play.html?vid=p3134yyavun&url_from=share&second_share=0&share_from=copy
3、 QosService framework design
Video Quality Indicators provide the analysis results of 15 video related indicators. Video Quality Indicators analyze images, that is, the comparison results between frames. Therefore, for a 5-minute video, at an average frame rate of 25 frames, Video Quality Indicators will provide 7500 data points. There are many original data types and a large amount of data. In order to facilitate analysis, reduce the threshold for tool use, and improve tool utilization, the non reference analysis function provided by Video Quality Indicators has been serviced. In the service process, we used tornado as the service framework. Based on the QosTestFramework, we persisted the analysis results to the database, and finally used the front end to display the analysis results. The overall implementation architecture is as follows:
The user inputs UID and CID to trigger the analysis of video files and display the analysis results. DL VIDEO: Interactively download the recording file with the recording server to the local Ffmpeg: transcode the corresponding file to the yuv format (Video Quality Indicators only support the yuv format) VQ: Video Quality Indicators analyze the corresponding yuv file and persist the data to the database front-end for final presentation:
No reference analysis examples:
The source video is: http://m.v.qq.com/play/play.html?vid=l3135pqxd80&url_from=share&second_share=0&share_from=copy
It can be seen that the picture near the moving calf in the video is blurred and there are some noise points. The results of QosService analysis are as follows:
Blur indicates the definition of the picture. The blur value is greater than 5, indicating that the picture is fuzzy. The larger the blur value, the more fuzzy the picture is. The analysis result of the software as shown in the figure shows abnormal definition, which is consistent with the actual situation
Noise indicates the noise in the picture. The higher the value, the more noise, and the worse the picture quality. Noise greater than 30 means that the noise of the picture is too large. As shown in the figure, the analysis result of the software indicates abnormal noise, which is consistent with the actual situation
Blockloss: indicates that there is a picture missing in the picture. This video picture has good integrity, and the calculated value is consistent with the actual situation
Contrast: refers to the contrast of the picture. The contrast of the image in this video is acceptable to the naked eye, and the measured value is consistent with the actual situation
Freezing: indicates the picture's stuck condition. There is no stuck condition in this video. The calculated value is consistent with the actual situation
4： Summary and outlook
In the field of real-time audio and video, how to realize the fully automated evaluation of video is a common problem in the industry. No reference (NR) technology does not need any information about the reference video, which is undoubtedly the dawn of fully automated video evaluation. To quote Mr. Lu Xun's famous saying: Actually there is no road on the ground, and more people walk, it becomes a road. I would like to pay tribute to the industry compatriots who have made great efforts to fully automated video assessment.
The source and author of articles reproduced in various channels shall be indicated 。
understand Netease Cloud AV call >>>
understand Netease Yunxin Communication and video cloud services from NetEase core architecture>>
For more technical goods, please pay attention to the vx official account "Netease Smart Enterprise Technology+". A series of courses can be viewed in advance. Boutique gifts are free, and you can also talk to CTO directly.
