6. Conclusions and Future Directions

There has been increasing interest in the development of objective quality measurement techniques that can automatically predict the perceptual quality of images and video streams. Such methods are useful tools for video database systems and are also desired for a broad variety of applications, such as video acquisition, compression, communication, displaying, analysis, watermarking, restoration and enhancement. In this chapter, we reviewed the basic concepts and methods in objective image and video quality assessment research and discussed various quality prediction approaches. We also laid out a new paradigm for quality assessment based on structural distortion measures, which has some promising advantages over traditional perceptual error sensitivity based approaches.

After decades of research work, image and video quality assessment is still an active and evolving research topic. An important goal of this chapter is to discuss the challenges and difficulties encountered in developing objective quality assessment approaches, and provide a vision for future directions.

For error sensitivity based approaches, the four major problems discussed in Section 2.5 also laid out the possible directions that may be explored to provide improvements of the current methods. One of the most important aspects that requires the greatest effort is to investigate various masking/facilitation effects, especially the masking and facilitation phenomena in the suprathreshold range and in cases where the background is composed of broadband natural image (instead of simple patterns). For example, the contrast matching experimental study on center-surround interactions [88] may provide a better way to quantitatively measure the image distortions at the suprathreshold [89] level. The Modelfest phase one dataset [90] collected simple patterns such as Gabor patches as well as some more complicated broadband patterns including one natural image. Comparing and analysing the visual error prediction capability of different error sensitivity based methods with these patterns may help researchers to better understand whether HVS features measured with simple patterns can be extended to predict the perceptual quality of complex natural images.

The structural distortion based framework is at a very preliminary stage. The newly proposed image quality indexing approach is attractive not only because of its promising results, but also its simplicity. However, it is perhaps too simple and the combination of the three factors is ad hoc. More theoretical analysis and subjective experimental work is needed to provide direct evidence on how it is connected with visual perception and natural image statistics. Many other issues may also be considered, such as multiscale analysis, adaptive windowing and space-variant pooling using a statistical fixation model. Furthermore, under the new paradigm of structural distortion measurement, other approaches may emerge that could be very different from the proposed quality indexing algorithm. The understanding of "structural information" would play a key role in these innovations.

Another interesting point is the possibility of combining the advantages of the two paradigms. This is a difficult task without a deeper understanding of both. One possible connection may be as follows: use the structural distortion based method to measure the amount of structural information loss, and the error sensitivity based approach to help determine whether such information loss can be perceived by the HVS.

The fields of NR and RR quality assessment are very young, and there are many possibilities for the development of innovative metrics. The philosophy of doing NR or RR quality assessment will continue to be that of blind distortion measurement with respect to features that best separate the undistorted signals from the distortion. The success of statistical models for natural scenes that are more suited to certain distortion types and applications will drive the success of NR and RR metrics in the future. A combination of natural scene models with HVS models may also prove beneficial for NR and RR quality assessment.