Mobile Multimodal Virtual Reality for Personal Teleservices
by Petri Pulli
Augmented reality merging real and virtual environments to provide totally new services and interfaces to mobile observers is the basis of the mobile multimodal virtual reality system for personal teleservices, developed in the PIHVI (Picocellular based personal virtual products and services) project at VTT Electronics and University of Oulu Infotech Research Center. The research focuses on the characteristics and components of wearable personal virtual reality systems with augmented reality display systems, tracking systems, wireless communications and wearable computing. The backbone of the mobile virtual reality is a wireless picocellular personal surrounding network (PSN). Currently, experiments are carried out with a multimodal personal augmented reality user interface for Nokia 9000 Communicator product, which is a small-sized combination of a notepad computer and a cellular phone.Two major approaches to extend the use of computing resources beyond a single workstation can be recognized: ubiquitous computing and augmented reality. Ubiquitous computing is a term coined by Mark Weiser to mean a situation, where small computational devices are embedded into our everyday environment in a way that they can be operated seamlessly and transparently. Augmented reality is a research approach that attempts to integrate some form of computer media with the real world.
The core idea of our approach is that by using a very short distance radio communication network it is possible to mix these two approaches. Thus we suggest ubiquitous computing in the form where different devices are computationally active. But instead of a multitude of different displays and interaction devices we suggest that the interaction with all these devices would take place in an 'augmented reality', for example by using a head-mounted see-through display and a mobile phone/remote controller.
Essential requirements for augmented reality display systems are a stereoscopic see-through head-mounted display and haptic displays providing touch and force feedback for artificial surfaces. Tracking systems are needed for head movement to synchronize and overlay virtual world with real world; eye viewing direction for context sensitive viewing; hand and finger position tracking for gesture recognition; I/O device tracking for data and command entry and object tracking for synchronizing real and virtual worlds.
An essential requirement for personal augmented reality is that they should work wireless, otherwise the human user is tied with cables and the freedom of movement is lost. Wireless communication is needed between components of the system and also between personal augmented reality system and networks services, such as world models and other users or avatars. Wearable computing means that the computing equipment is attached to the user's body or clothes.
The backbone of mobile virtual reality is a wireless picocellular personal surrounding network (PSN). The PSN network connects user's personal mobile terminals like a head-mounted-display or a pen-shaped input device and provides mobile access to other mobile and fixed networks.
The benefits of using very small cells in mobile virtual reality are obvious. The smaller the cell size, the higher the throughput, because there are fewer users in each cell and higher transmission frequencies can be used. The diameter of a PSN cell in our system is going to be some three meters, which enables the construction of small very low-powered hand held terminals still capable of transmitting high bandwidth multimedia data required by virtual reality applications.
Adjacent PSNs can change information directly without using a fixed base station. This does not only enable wireless communication between user's personal terminals but also makes it possible for two users to transmit data to each other directly. In this way the number of expensive base stations can be greatly reduced. To decrease the amount of computation power needed by a PSN most of the heavy processes should be distributed across the fixed network and executed in different network nodes.
The main components of the system are a fixed base station providing access to fixed networks and their services, Personal mobile terminals including virtual reality devices and a mobile base station providing connections between mobile terminals and a fixed network.
Although we have experimented mainly with small electronics and tele-communication products, we believe that our approach has a wide utilisation reserve. The PIHVI project is co-funded by Technology Development Centre of Finland (TEKES), CCC Software Professionals, Elektrobit, Nokia Mobile Phones, Polar Electro, and Telecom Finland. We are interested in co-operation with other ERCIM members, eventually in joint EU projects.
Please contact:
Petri Pulli - VTT Electronics
Tel: +358 8 551 2111
E-mail: Petri.Pulli@vtt.fiTino Pyssysalo - University of Oulu
Tel: +358 8 5531011
E-mail: tino@tks.oulu.fi