Joint DELOS-NSF Workshop on Personalisation and Recommender Systems in Digital Libraries

by Alan Smeaton and Jamie Callan

The Joint DELOS-NSF Workshop on Personalisation and Recommender Systems in Digital Libraries held at Dublin City University June 18-20, 2001, brought together 57 researchers and practitioners from 14 countries to discuss the development of personalisation and recommender systems and techniques, particularly as they apply to digital libraries.

The concept of personalisation is about making systems different for individual people or groups of people and one type of personalisation that is growing in use is recommender systems. These take input directly or indirectly from users and based on user needs, preferences and usage patterns, they make personalised recommendations of products or services. These vary from recommending books to buy or TV programs to watch, to suggesting web pages to visit. The ultimate goal of such recommender systems is to reduce the amount of explicit user input and to operate, effectively, based on usage patterns alone, thus giving users what they want without them having to ask.

Within the digital library environment, personalisation and recommender systems may have different characteristics because individual user behavior and traffic patterns may differ significantly from those of Web and E-commerce environments. For example, few digital libraries will see millions of transactions within a short period of time, digital library resources may be structured and more stable as compared with commercial sites, and digital library characteristics and usage patterns may provide opportunities for types of long-term learning that would be difficult or impractical in other environments.

Fifteen papers and three invited talks were presented at the workshop, while moderated discussion sessions at the end of each day provided an opportunity for participants to reflect on the day’s talks and to address recurring themes and issues. Although the workshop included talks on software architectures, several deployed systems, and a user study, most of the presentations focused on algorithms for making recommendations. In spite of this emphasis the papers were generally consistent with past research in this area, which has been adaptation of existing technologies, mostly from Information Retrieval or Machine Learning, often combined with creative methods of acquiring training data. This suggests that the area is not yet mature enough to have spawned its own specific and tailored techniques, or the theoretical models that would support them, which is a sign that there is much more work yet to be done.

Workshop participants naturally felt that personalisation is an important research area, and that it will receive greater attention in the coming years but there was less agreement on how it will be applied in the context of digital libraries. Most of the ‘success’ stories are systems that encourage additional consumption, for example of movies and books but it is harder to find systems that provide other types of improvements that might be more applicable in a digital library environment.

As is often the case, research has been influenced strongly by the data available and movie recommender systems are a popular research vehicle because movie databases are freely available on the Web. Some researchers have access to more interesting datasets, but these are often proprietary, which makes research results difficult to evaluate and reproduce, and which raises a barrier to entry by new researchers. Workshop participants agreed that there is a strong need for a more diverse set of generally-available data resources for personalisation research, and encourage the funding community to consider this in their funding decisions.

Evaluation is a related problem. Most of the systems discussed in the workshop were evaluated in some manner, but few of the evaluations could be called rigorous. It might be clear how to evaluate a movie recommender system, but it is less clear how to evaluate more complex systems. Researchers in this area need to begin considering a broad approach to evaluation that embraces not just quantitative evaluation, but also methods and tools from disciplines such as sociology. While recognising the importance of evaluation, we worry that an overemphasis on evaluation will stifle new ideas in this fledgling field. Finding the right balance between creative development of new ideas and scientific evaluation remains an issue.

Finally, prior research on personalisation and recommender systems has focused on relatively short periods of time. Systems are beginning to be deployed that are intended for daily use over long periods of time (two were discussed in this workshop), but little is known about how such systems and their users might evolve over time. It is important to begin studying long-term personalisation issues, for example following a particular group of users over a several year period. Workshop participants felt that funding bodies need to give greater attention to longer-term projects.

The complete workshop proceedings are available at http://www.ercim.eu/ publication/ws-proceedings/DelNoe02/.

Please contact:
Alan Smeaton - Dublin City University, Ireland
E-mail: asmeatoncompapp.dcu.ie 
Jamie Callan - Carnegie Mellon University, USA
E-mail: callancs.cmu.edu