We are interested in using perceptually driven criteria to accelerate interactive rendering. A great deal of excellent research has studied how to exploit the limitations of the human visual system to accelerate realistic rendering, such as global illumination algorithms. Metrics such as those published by Bolin and Meyer [Bolin 98 and Ramasubramanian et al [Ramasubramanian 00] bring sophisticated perceptual models to guide those algorithms. Their models account for coarse effects such as Weber's Law (threshold-vs-intensity) and contrast sensitivity over varying spatial frequency, as well as more subtle effects such as contrast masking. While many issues certainly remain to be studied in this realm, that research is at least well on its way.

On the other hand, very little work has investigated applying such metrics to the field of interactive rendering. Acceleration schemes in interactive rendering, such as the geometric simplification of polygonal models, is largely ad hoc. For example, the polygonal simplification community has long recognized that the human visual system deduces much about an object's shape from its silhouette, and so simplification algorithms have long included heuristics to preserve regions of high curvature, which are likely to form silhouettes [e.g., Rossignac 93]. However, the perception of shape also depends on the appearance and movement of specular highlights, so researchers have proposed heuristics to preserve regions where specular highlights appear and disappear [Xia 96].

We are attempting to approach the integration of simplification and perceptual issues from a new and more systematic direction. One of our first principles is that taking advantage of perceptual effects generally requires the fine-grained control provided by a view-dependent framework. These frameworks depart from the traditional approach to simplification, which creates several discrete levels of detail (LODs) in a preprocess. Instead, view-dependent simplification systems create a hierarchy of local simplification operations in which each operation eliminates only a few polygons [Luebke 97, Hoppe 97, Xia 96, DeFloriani 98]. In our work we consider the perceptual effect of each local simplification operation on the rendered image. We can achieve imperceptible simplification by only performing those operations for which the resulting change in the image is judged imperceptible. Our system is built as a set of callbacks on top of VDSlib, a flexible public-domain toolkit for performing view-dependent polygonal simplification.

In our preliminary technical report [Hallen 00], we describe a set of view-dependent simplification criteria that evaluate vertex merge operations according to the worst-case contrast and spatial frequency the merge could introduce in the final image. Using well-known measures from the psychophysical literature (e.g., [Campbell 68], [Kelly 75]), we derive a metric for the perceptibility of the merge operation, and only perform the merge when the resulting image will be indistinguishable from the original. User studies were used to verify the imperceptibility of the resulting view-dependent simplification. If "best-effort" rather than imperceptible simplification is the goal, our equations can order the merge operations according to the distance at which the user may be able to perceive the result. A greedy algorithm can then simplify the model to a set polygon budget while minimizing perceptual artifacts.

A particularly interesting aspect of this work is the optional incorporation of an eye-tracker to provide gaze-directed rendering. Gaze-directed rendering is not a new idea, but the fine granularity of view-dependent simplification seems to greatly improve on previous results by Oshima and Reddy [Oshima 96] [Reddy 97]. We have extended the perceptual model to account for eccentricity, or falloff in visual acuity in the peripheral vision [Figure 1], and performed user studies to verify the imperceptibility of the resulting simplifications. These preliminary experiments are encouraging indicators of the potential of gaze-directed rendering.

We have also applied our perceptually-driven framework to a completely different rendering paradigm, namely the point-based rendering system QSplat. Initial results are encouraging: while QSplat's standard high-quality mode only simplifies the model until each averaged sample is no larger than a pixel, we are often able to use much larger sample sizes (several pixels across) imperceptibly because of locally low contrast [Figure 2].

This research is in nascent form; while we believe that it breaks important intellectual ground, the overall rendering speedup is not nearly what we believe possible. Our immediate avenues of future work include studying more accurate techniques for evaluating the spatial frequency and contrast induced by a vertex merge, as well as implementing a more sophisticated perceptual model. Our current model is simple and somewhat bewhiskered, deriving from classic but decades-old psychophysical research. We would like increase our efficiency and accuracy and incorporate additional perceptual phenomenon such as velocity and contrast masking.

[Bolin 98] M. Bolin and G. Meyer. "A Perceptually Based Adaptive Sampling Algorithm", Computer Graphics, Vol. 32 (SIGGRAPH 98).

[Campbell 68] Campbell, F.W. and Robson, J.G. An Application of Fourier Analysis to the Visibility of Contrast Gratings, Journal of Physiology, 187 (1968)

[DeFloriani 98] L. De Floriani, P. Magillo, E. Puppo, Efficient Implementation of Multi-Triangulations, Proceedings IEEE Visualization '98, Research Triangle Park, NC, October 18-23, 1998.

[Hallen 00] Hallen, B., Luebke, D., Newfield, D., and Watson, B. "Perceptually Driven Simplification Using Gaze-Directed Rendering", University of Virginia Technical Report CS-2000-04, 2000.

[Hoppe 97] Hoppe, Hughes. "View-Dependent Refinement of Progressive Meshes", Computer Graphics, Vol. 31 (SIGGRAPH 97).

[Kelly 75] Kelly, D.H. Spatial Frequency Selectivity in the Retina, Vision Research, 15 (1975).

[Luebke 97] Luebke, David, and C. Erikson. "View-Dependent Simplification of Arbitrary Polygonal Environments", Computer Graphics, Vol. 31 (SIGGRAPH 97).

[Oshima 96] Oshima, Toshikazu, H. Yamammoto, and H. Tamura. "Gaze-Directed Adaptive Rendering for Interacting with Virtual Space", Proceedings of VRAIS 96 (1996).

[Ramasubramanian 99], Mahesh Ramasubramanian, S. Pattanaik, and D. Greenberg. "A perceptually based physical error metric for realistic image synthesis", Computer Graphics, Vol. 33 (SIGGRAPH 99).

[Reddy 97] Reddy, Martin. "Perceptually-Modulated Level of Detail for Virtual Environments", Ph.D. thesis, University of Edinburgh, 1997.

[Rossignac 93] Rossignac, Jarek, and P. Borrel. "Multi-Resolution 3D Approximations for Rendering Complex Scenes", pp. 455-465 in Geometric Modeling in Computer Graphics, Springer-Verlag (1993), Genova, Italy. Also published as IBM Research Report RC17697 (77951) 2/19/92.

[Xia 96] Xia, Julie, and A. Varshney. "Dynamic View-Dependent Simplification for Polygonal Models", Proceedings of IEEE Visualization 96, 1996.

© Copyright is held by the author, David Luebke, 2001