Video quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amounts of distortion or artifacts in the video signal, so video quality evaluation is an important problem.

From analog to digital video

Since the time when the world's first video sequence was recorded, many video processing systems have been designed. In the ages of analog video systems, it was possible to evaluate quality of a video processing system by calculating the system's frequency response using some traditional test signal (for example, a collection of color bars and circles).

Nowadays digital video systems are replacing analog ones, and evaluation methods have changed. Performance of a digital video processing system can vary significantly and depends on dynamic characteristics of input video signal (e.g. amount of motion or spatial details). That's why digital video quality should be evaluated on diverse video sequences, often from user's database.

Objective video quality

Objective video evaluation techniques are mathematical models that approximate results of subjective quality assessment, but are based on criteria and metrics that can be measured objectively and automatically evaluated by a computer program. Objective methods are classified based on the availability of the original video signal, which is considered to be of high quality (generally not compressed). Therefore, they can be classified as Full Reference Methods (FR), Reduced Reference Methods (RR) and No-Reference Methods (NR). FR metrics compute the quality difference by comparing every pixel in each image of the distorted video to its corresponding pixel in the original video. RR metrics extract some features of both videos and compare them to give a quality score. They are used when all the original video is not available, e.g. in a transmission with a limited bandwidth. NR metrics try to assess the quality of a distorted video without any reference to the original video. These metrics are usually used when the video coding method is known.

The most traditional ways of evaluating quality of digital video processing system (e.g. video codec like DivX, XviD) are calculation of the signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR) between the original video signal and signal passed through this system. PSNR is the most widely used objective video quality metric. However, PSNR values do not perfectly correlate with a perceived visual quality due to non-linear behavior of human visual system. Recently a number of more complicated and precise metrics were developed, for example UQI, VQM, PEVQ, SSIM and CZD. Based on a benchmark by the Video Quality Experts Group (VQEG) in the course of the Multimedia Test Phase 2007-2008 some metrics were standardized as ITU-T Rec. J.246 (RR) and J.247 (FR) in 2008.

The performances of an objective video quality metric are evaluated by computing the correlation between the objective scores and the subjective tests results. The latters are called mean opinion score (MOS). The most frequently used correlation coefficients are : linear correlation coefficient, Spearman's rank correlation coefficient, kurtosis, kappa coefficient and outliers ratio.

When estimating quality of a video codec, all the mentioned objective methods may require repeating post-encoding tests in order to determine the encoding parameters that satisfy a required level of visual quality, making them time consuming, complex and impractical for implementation in real commercial applications. For this reason, a lot of research has been focused on developing novel objective evaluation methods which enable prediction of the perceived quality level of the encoded video before the actual encoding is performed

Subjective video quality

Main article: Subjective video quality

The main goal of many objective video quality metrics is to automatically estimate average user (viewer) opinion on a quality of video processed by the system. Sometimes however, measurement of subjective video quality can also be challenging because it may require a trained expert to judge it. Many “subjective video quality measurements” are described in ITU-T recommendation BT.500. Their main idea is the same as in Mean Opinion Score for audio: video sequences are shown to the group of viewers and then their opinion is recorded and averaged to evaluate the quality of each video sequence. However details of testing may vary greatly.

References

• ITU-T Rec. J.247 (08/08) Objective perceptual multimedia video quality measurement in the presence of a full reference
• ITU-T Rec. J.246 (08/08) Perceptual audiovisual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference
• Digital Video Quality, Stefan Winkler, Wiley, March 2005, ISBN 0-470-02404-6

See also

• Quality
• Subjective video quality
• Video codecs
• Mean Opinion Score
• PSNR
• Perceptual Evaluation of Video Quality (PEVQ, ITU-T J.247)
• MDI

Further reading

• Video Quality Experts Group
• Objective Video Quality Assessment
• ITU-T page with recommendations on subjective video quality assessments
• The Universal Image Quality Index - an alternative image quality measurement
• Video Quality Lab (VQLab) - Professional Video Quality Assesment Software
• Video Clarity - Quantitative, Repeatable Video Quality Testing Systems