ERCIM News No.44 - January 2001 [contents]

3D Scanning for Computer Graphics

by Holly Rushmeier

Three dimensional scanning has recently become a very active area in computer graphics. The requirements for computer graphics are different from those of traditional scanning applications. At IBM Watson Research we are developing scanning systems for producing virtual objects that can be rendered with high visual quality. 

The capture of 3D representations of objects has been an area of research for decades in computer vision, robotics and metrology. Traditional uses for digital objects include industrial inspection, autonomous robot navigation and object recognition. Each of these areas has different requirements and different approaches for data acquisition and processing. For inspection applications, metric accuracy is critical, and scanners with great precision have been developed. Autonomous navigation applications, such as the use of robots in hazardous environments, need to be very robust and justify the use of expensive scanning equipment. Object recognition may use both the shape and surface properties, but only to the extent they are needed to compute a unique signature for the object.

Unlike traditional applications, the end product of scanning for computer graphics is a model that can be used to render a realistic image of the object under novel conditions, ie in a location or under lighting conditions that exist only in computer simulation. The emphasis is on visual, rather than metric accuracy. A systematic error in the shape may be less important than errors in the color or apparent shininess of the object. This translates into a greater concern for estimating the spatial variation of the spectral bidirectional reflectance of a surface, and less concern for precision range measurements.

Applications for computer graphics rendering of scanned objects include virtual museums, e-commerce and games. In virtual museums, institutions can allow visitors to interact with virtual copies of objects that are physically too delicate to touch, or possibly even to keep on display. In e-commerce, a vendor may offer a potential consumer the capability to view an object in a variety of customized configurations. In games, scanned objects can be used to populate visually rich synthetic environments. These applications, while affecting a wide population, can only justify a modest expenditure on individual scanners, dictating the use of commodity components in their construction. Also, these applications require interactive display of the digital objects. This requirement dictates that the output of the scanning processing pipeline must be a form that can be rendered in real time.

At IBM Watson Research, we have focused on the development of scanning systems that acquire surface properties as well as shape, use commodity digital cameras, and produce texture-mapped triangle meshes that can be efficiently rendered by graphics hardware. Our first system was built around a commercial shape scanner that used projected light stripes and multi-baseline stereo. We enhanced the scan with a novel photometric system that acquired surface normals and albedo at a spatial resolution 4 times the base geometry. We developed a processing pipeline for the acquired data that included new algorithms for meshing point clouds, for computing consistent normals from photometric data, and for the alignment and integration of surface texture maps.

Our first major project using our scanning system was creating a digital model of Michelangeloís Florentine Pieta`. The sculpture was scanned on site in the Museum of the Opera del Duomo in Florence. The resulting model was used by art historian Jack Wasserman to study issues such as how the work appeared after damage (now repaired) inflicted by the artist, and how it would have appeared in its originally intended site in a niche above the artists tomb.

Many European research groups are pursuing the development of inexpensive scanners for computer graphics. Cultural heritage is a leading application for many of these projects. For example, the Visual Computer Group at CNR Pisa, Italy, has developed a scanner based on a digital camera and video projector. They have recently used this scanner to capture a model of a bronze Minerva in Florence. The VISICS group at the Katholieke Universitat Leuven, Belgium has developed numerous capture techniques, including a method for generating texture-mapped models from uncalibrated video streams. This method was used in reconstructing the site of ancient Sagalassos in Turkey.

At IBM Research we are continuing to develop new scanning systems. We are exploring the use of alternative shape capture systems based on digital cameras. We are exploring the trade-offs between shape and texture resolution. We are also improving the processing pipeline to reduce user intervention to make scanning accessible to a wide range of consumers.

Links:
IBM Visual and Geometric Computing group:
http://www.research.ibm.com/vgc