Augmented Virtual Table Tennis with Real Racquets

by Charles Woodward and Petri Honkamaa

A group of researchers at VTT have developed a computer system for natural interaction in an augmented virtual environment, enabling people to play table tennis over Internet/LAN with real racquets. Apart from the web cameras, no special hardware is required. The players can see each other in the camera image, which is streamed in real time compressed over the network. The position of the racquets is computed by marker detection from the image. The multicast implementation also enables a network audience to view the game.

Among the various fields of computer graphics, real time marker detection from a camera-produced video stream is most closely related to Augmented Reality (AR). In AR applications, virtual objects are superimposed on the user’s view of the real world, which is displayed, for example, through a computer monitor or HMD devices.

A popular application of AR technology is collaborative gaming. For example, AR table tennis for two people has been previously implemented using magnetic trackers and sharing the same computer processing unit. In another case, a virtual reality (VR) table tennis game has been implemented over a computer network, but it does not involve video image augmenting and is based on the traditional mouse interface.

Figure 1: Computer system and virtual objects for augmented virtual table tennis.

Our solution combines the virtues of these different approaches for an augmented virtual table tennis game in a computer network system. Natural user interaction is accomplished by real-time marker detection from the camera image. Thus two people at remote locations can play virtual table tennis against each other using real racquets, with no special hardware required other than the web cameras.
Figure 1 shows a schematic drawing of the computer system for the game. The hardware for each player consists of a PC workstation and a web camera. The PCs are connected by Internet or LAN network. The cameras are aimed towards the players, who hold in their hands real table tennis racquets. Each racquet has a marker, which enables the racquet’s position (distance and angle) to be computed from the video image.

A computer program is synchronised between the players, defining a shared virtual ball, table and net. When the racquet’s location meets the virtual ball within given distance, a hit is registered, the new ball trajectory is computed, and the ball parameters (time of impact, new trajectory) are sent via multicast address to the other player. Additionally, hits and bounces are accented with sound effects.

Marker Detection
The marker is a coloured rectangle of given size (10x10 cm), glued to both sides of the racquet. Green was chosen as the marker colour, as it occurs less frequently than other basic colours in room interiors or on clothing. Depending on lighting conditions, the image brightness and saturation can be adjusted to preserve bright green colours.

Marker corner points are located from the video image by fitting two diagonals to the detected marker image, and refined by fitting four lines to the marker edges. The marker position in the virtual object’s coordinate system is then calculated using laws of weak perspective projection. A simple camera calibration routine is performed when installing the system.

Player’s View
Figure 2 shows a screenshot of the player’s view of the game. Each player sees on the computer monitor a virtual racquet corresponding to the position of his or her real racquet. The video image of each player is compressed in real time, streamed continuously to the other player, who sees their opponent at the other side of the table. VTT’s proprietary MVQ (Motion Vector Quantization) software is used for efficient video compression and streaming.

Figure 2: Player’s view of the game.	Figure 3: Audience view.

Solo Game
A simple solo game mode is provided for playing against a ’wall’. In the more interesting version, a player can actually play against him/herself, as the video image of the player is shown at the other end of the table (see Figure 2 again). But which side should then hit the ball, the ’virtual me’ (the racquet), or the ’video me’ (the image at the other end of the table)? Our solution is to have both sides hit the ball at the same time. Consequently, the game contains two balls, which the user has to hit twice as frequently as normally.

Network Audience
A network audience option can facilitate, for example, the viewing of game tournaments. Thus the video stream and ball parameters from both players are multicast to people in the network who have joined to watch the game. The audience members see on their screens the same virtual/video game content as the players, with the ball trajectory as an added component. The audience may also rotate the view to different sides and angles (see Figure 3).

Simplifications
We have not attempted to implement a full correspondence with the real game, and we do not account for air resistance, friction etc. Adding more physical realism would be possible, but may not be worth the effort. For one thing, the inevitable imprecision in marker pose detection sets some limitations in reaching full realism. In our experience, the game can actually be more fun with some simplifications. Thus, to keep the ball ’live’ longer than normal, the ball is directed slightly towards the table, and hits are registered at a larger diameter than that of the actual racquet. Slow motion balls can also be useful for practice, and they provide a nice exercise tool for elderly and handicapped people.

Future Work
The game is available for free download from our Web pages http://www.vtt.fi/ multimedia. Ongoing and future work includes detecting a serve by gesture, a foursome game using different marker types, an audio component for chat between players, and implementing the game for video HMD glasses with stereo virtual objects. The table tennis implementation described here can also be generalised to a wide variety of other augmented virtual game concepts.

Link:
http://www.vtt.fi/multimedia/

Please contact:
Charles Woodward, VTT Information Technology, Finland
Tel: +358 9 4565629
E-mail: charles.woodward@vtt.fi