ERCIM News No.44 - January 2001 [contents]
ERCIM News No.44 - January 2001 [contents]
by Ann McNamara
Increased application of computer graphics in areas which demand high levels of realism has made it necessary to examine the manner in which images are evaluated and validated. Research carried out at Trinity College Dublin explores the need for including the human observer in any process that attempts to quantify the level of realism achieved by the image synthesis method, and involves running sets of experiments that compare a simple physical test scene to a number of graphical images which are intended to represent it.
The aim of realistic image synthesis is the creation of accurate, high quality imagery that faithfully represents a physical environment, the ultimate goal being to create images that are perceptually indistinguishable from an actual scene. Reliable image quality assessments are necessary to help determine how close image synthesis systems come to achieving that goal. Typically the quality of an image synthesis method is evaluated using numerical techniques that attempt to quantify fidelity using image-to-image comparisons (often comparisons are made with a photograph of the scene that the image is intended to depict). However, this may not reveal specific errors in the portrayal of lighting and shadows; furthermore it could be argued that one needs a measure of fidelity which involves assessment of performance on a specific visual task, performance on that task in a real scene can be compared with performance on that task in a graphical scene to give a measure of fidelity. By conducting such a set of experiments, based on psychophysics, it can be estimated how close a rendered image is to its original counterpart.
The Experimental Set-Up
The test environment was a simple five-sided cube. Several small objects were placed within the box for examination. The test scene was then modelled and rendered on computer. In total ten graphical images were considered for comparison to the real scene, these included rendered images of varying degrees of quality and a digital photograph. A small, front-silvered, high quality mirror was incorporated into the set up to allow the viewing conditions to facilitate alternation between the two settings, viewing of the original scene or viewing of the modelled scene on the computer monitor. When the optical mirror was in position, subjects viewed the original scene. In the absence of the optical mirror the computer representation of the original scene was viewed, see Figure 2.
The Experiment
Each participant was presented with a series of images, in a random order, one of which was the real environment. The images presented were the real scene, the photograph and the 9 rendered images. The task - that of matching materials in the scene against a display of originals - was chosen because the task has a number of attractive features. First, it has been shown that the perception of lightness (the perceptual correlate of reflectance) is strongly dependent on the human visual system’s rendition of both illumination and 3-D geometry. These are key features of perception of any scene and are in themselves complex attributes. However, the simple matching procedure used here depends critically on the correct representation of the above parameters. Therefore, the task should be sensitive to any mismatch between the original and the rendered scene.
Results and Conclusions
The results show there are indeed differences between the judgements of lightness in the real scene and the synthetic images. The main conclusions from the analysis of the experimental results are that there is indeed hard evidence that the certain images are perceptually degraded compared to the real scene. However, there is no evidence that the others images in this study are perceptually inferior to the real scene. From this we can conclude that the certain images are of the same perceptual quality as the real scene.
The main focus and motivation behind this project was the development of a process that led to the evaluation of computer graphics images with respect to the real world scene they depict. This has resulted in a new experimental framework that facilitates human comparison of real and rendered scenes.
The ultimate goal of this work, which is continuing in the Image Synthesis Group at Trinity College Dublin, is to develop a computational algorithm to judge the quality of computer images that mimics the quality judgement of a human being. In effect this would mean designing a form of „visible differences predictor‰ which matches human judgement of an image to its real world scene. This would eliminate the need for further, time consuming psychophysical studies, by incorporating the actual response of the HVS into a difference metric. This new metric would then give the same results as image comparison by human observers, facilitating subjective comparison of computer graphics imagery.
Links:
http://www.cs.tcd.ie/
Please contact:
Ann McNamara - Trinity College Dublin
Tel: +353 1 608 3681
E-mail: Ann.McNamara@tcd.ie
http://www.cs.tcd.ie/people/Ann.McNamara