Generating Multimedia Presentations: It's All in the Game

by Frank Nack and Lynda Hardman

In a time when the human attention span seems to grow ever shorter, computer games are able to keep people focused for hours. Compelling storylines and interactive environments draw players into the game. At CWI these elements are applied to the generation of multimedia presentations.

Online information systems, such as the Web site of the Rijksmuseum in Amsterdam (http://rijksmuseum.nl/), provide users with a huge amount of information in the form of texts, photos, video and animation. The Rijksmuseum site displays, for example, 1200 of its top exhibits, with links connecting and associating objects from many different departments of the museum's vast collection. For instance, a painting of a nineteenth-century landscape could be associated with its seventeenth-century forerunner. Access is via four categories: artists' names, themes, encyclopaedic terms and a systematic catalogue.

The challenge for the designer of such a media-based information environment is to foresee the circumstances and presuppositions of the user at the time of accessing the information. The designer's vision might be an informative and graphical experience, but it is impossible to cover all presentational aspects of dynamic systems when neither the individual user's requirements nor the requested material can be predicted in advance.

A solution to this problem is the automatic generation of multimedia presentations. The aim is to present the right information for a particular user in a compelling way. In particular, the aspect of an appealing presentation relates research in automatic presentation generation to that of game design.

Computer and video games can be extremely engaging, and their magnetic effect is all too familiar. The reason for this is that game development focuses on what players experience as they work their way through the game. The goal for the game designer is to make this experience as compelling as possible. The two main aspects of user experience are to keep players engaged with the goals they follow and the appropriate environment they can follow them in. The strategies used in game design to achieve these two aspects are the carving of an engaging storyline and the provision of a highly interactive environment.

The storyline is a carefully constructed net of events that tries to create a self-contained, internally consistent world. The navigation of the user through the story world is important, because it is direct interaction with the material that provides immersion, just as immersion provides the grounds for motivation and concentration to achieve the goal. A game is, therefore, a skilful exploitation of narrative structures that give players the perception of free will, even though at any time their options are actually quite limited.

It is in particular the strategy of narration and interactivity we wish to adopt and integrate in our approach towards automatically generated user-adapted multimedia presentation. A potential solution for the Rijksmuseum is, for example, to identify typical categories of users and provide ways of communicating in a form tailored to each individual group.

Figure 1: Presentation in the storyteller style.

Figure 2: Presentation in the encyclopaedia style.

The storyteller presentation style, for example, establishes a non-interactive presentation, where the system presents the relevant material in the form of a narrated story. The system should be able to present in various styles, such as prosaic, documentary, or voyeuristic. The storyteller style serves the needs of a user who does not wish to explore the repository of the Rijksmuseum independently, but wishes rather to be guided based on a few submitted preferences such as artist, style, image or epoch. Figure 1 presents a stage within a presentation about Vermeer, where the highlighted image (top left) is the current focus of an audio narration.

On the contrary, if the user wishes to take a more active role in investigating the repository, the encyclopaedia style might be more applicable. Here the system presents the relevant information in a simple, concise manner to facilitate fast and highly interactive information retrieval. This style expects users to know exactly what they want. The main goal for the system in this style is, therefore, to offer access options without restricting user interaction. Figure 2 displays a snapshot of a presentation about Vermeer in the encyclopaedia style.

In this way we can generate non-permanent presentations of non-static spaces, where the abstract, stylised and non-real presentation is a feature. In our approach towards automated multimedia presentation, the 'experience' is provided by the narrative and appropriate presentation aesthetics. This requires, however, that the various levels of event and context need to be made as explicit as possible. In our research we address four aspects:

• modelling domain-dependent and domain-independent discourse to steer the presentation generation process
• investigating the dependencies of the user and domain models in the generation process
• investigating the characteristics of media types for presenting information to the user
• investigating to what extent graphic design knowledge can be included in the generation process.

In addition, we are also investigating the contribution of Web and Semantic Tools to this research.

Part of the research described here was funded by the Dutch national Token2000/CHIME and NWO/NASH projects. The authors wish to thank in particular Katharina Schwarz, who provided the interface study from which the images in this article are taken.

Links:
http:// www.cwi.nl/ins2

Katharina Schwarz: An investigation on the relationship between the user model and
graphic representations for the automated generation of multimedia presentations. Diploma Media System Design, University of Applied Sciences Darmstadt, Germany, 2003:

Please contact:
Frank Nack and Linda Hardmann, CWI
E-mail: Frank.Nackcwi.nl, Lynda.Hardmancwi.nl