First Irish Workshop on Eye-Tracking: Abstracts

Abstracts

Level of Detail Control for Real-time Computer Graphics and Virtual Reality: Applications of Eye-tracking.

Carol O'Sullivan, Marc Janott, Mark Watson, John Dingliana.
Image Synthesis Group, Computer Science Department, Trinity College Dublin.

Producing complex images and simulations on a computer can be very expensive in terms of computational power, and storage requirements. When such simulations need to be provided at interactive frame-rates, such as those required in Virtual Reality (VR) applications and games, it is not always possible to achieve the highest level of accuracy required. In such situations, Level Of Detail (LOD) approaches allow the application to trade accuracy for speed, thus degrading the realism of the images and/or the simulation to some degree. We will present some applications in which we have used eye-tracking to optimise the quality in such situations.

Introduction

In recent years there has been an increase in interest in the use of eye-tracking technology in real-time computer graphical applications such as Virtual Reality, games and simulations. Deering (1992) states that: "(eye-tracking) is a promising long term solution, since gaze direction can be exploited for other purposes such as identifying the region of screen space - corresponding to the foveal portion of the retina - that deserves to be rendered with high spatial detail."

When considering LOD control in real-time simulations, we are not only referring to the image which is rendered at each frame, but also to the simulation of the motions and interactions of the entities in the virtual environment. There are many opportunities to use information about the viewer's current region of interest to reduce the negative impact of reduced accuracy. For example, we can pre-process models of objects to ensure that perceptually important features are retained longer; When actually rendering scenes and objects, the viewer's current fixation point can be used to determine which parts of the scene to render at high detail; Similarly, the animation of objects and events which are currently being watched needs to be much more convincing than those happening peripherally.

Multiresolution Meshes

In time-critical computer graphics applications, such as Virtual Reality (VR), three-dimensional objects are often represented as meshes of triangles. Real-time frame rates limit the number of triangles the graphics engine is able to display per frame. Therefore, meshes with a high number of triangles often have to be replaced by meshes with a lower number in order to achieve acceptable display rates; i.e. the level of detail (LOD) needs to be reduced.

Building the Mesh
When building these meshes, it is possible to determine which triangles should be retained at highest detail for longest. Several techniques have been developed to derive meshes of lower LOD from one with a high resolution. These approaches are based on the mathematical or logical properties of the original mesh. The results differ in visual quality depending on the strategy used. Visual quality can be described in terms of the probability of the degradation being recognised by the viewer. Interactive Perception of Multiresolution Meshes (IPoMM), is an interest-dependent strategy for decreasing the LOD of a given object. The most interesting regions (i.e. triangle clusters) of the object are retained at high resolution for as long as possible, while less interesting regions are coarsened earlier. Thus, the reduction in accuracy should be less recognisable by the viewer (See Figure 1).

358 patches, evenly distributed over the whole object 358 patches, high resolution at the head leaves less detail for the hind legs

Figure 1: "Perceptual Moulding" of a multi-resolution mesh.

A viewer's interest in a triangle can not be determined a priori. Hence the object in question is shown to a test viewer, while simultaneously their eye-movements are tracked. We associate a counter with each triangle to measure the frequency with which they fixate it. The number of fixations is then interpreted as a measure for the test viewer's interest in a triangle, and the order with which regions are coarsened is updated accordingly. In other words, the lower LOD meshes get moulded interactively by the test viewer's interest. Meshes created in this way showed significant improvements over meshes created by more traditional methods, even when the LOD was significantly reduced. Striking results could be achieved within seconds, depending on the number of triangles and the topology of the object.

Figure 2: A Terrain Mesh.

View-dependent Continuous Level of Detail
When rendering large meshes on the computer, both storage and rendering time can be problematic. Even if the memory and power were available, the time spent rendering triangles that don't make a perceivable difference would be better spent improving the image quality of triangles that do. When an eye-tracker is used, detail can be added wherever the user is looking. This will give the impression that the mesh is much more detailed than it actually is. In the specific problem of rendering detailed terrain, there are several LOD techniques available (see Figure 2). An existing method called ROAM, presented by Duchaineau et al. 1997, has been adapted to incorporate foveation. The focus of the viewer's attention is assumed to be the intersection between the line from the viewer's position in the direction of gaze, and the mesh. The distance of this intersection point from the viewer is used to determine how big a section of the terrain needs to be drawn in higher detail. Similar approaches may be used in any application which renders objects modelled by meshes.

Simulation Levels of Detail

In order to create believable animation of objects and characters, the physics of the real world need to be taken into account. In particular, when objects come into contact with each other, a suitable response must occur. However, often it is not possible to perform the computations which will enable an entity to react in a realistic way to un-predictable events. Firstly, simply determining whether they have collided or not is expensive. For a highly accurate response to be performed, the contacts between the objects need to be modelled in detail, and finally the physical response must be determined. Again, an eye-tracker can indicate which collisions are the most important and this information is then used to schedule the processing of such events. A hierarchy of spheres is used to approximate the topology of the object, as in Hubbard 1996 (See Figure 3). When the time available for collision processing has expired, response occurs at the resulting level of detail.

Figure 3: An object approximated at different levels of detail for collision handling.

References
Deering, M. (1992). High Resolution Virtual Reality. Computer Graphics, 26(2) 195-202
Duchaineau, M. et al. (1997). ROAMing Terrain: real-time optimally adapting meshes. Proceedings Visualization'97, IEEE, 91-88
Hubbard, P.M. (1996) Approximating Polyhedra with Spheres for Time-Critical Collision Detection. ACM Transactions on Graphics, 15(3) 179-210

Pages maintained by Carol O'Sullivan.