Dietmar Jannach

Beyond accuracy: evaluating recommender systems by coverage and serendipity

When we evaluate the quality of recommender systems (RS), most approaches only focus on the predictive accuracy of these systems. Recent works suggest that beyond accuracy there is a variety of other metrics that should be considered when evaluating a RS. In this paper we focus on two crucial metrics in RS evaluation: coverage and serendipity. Based on a literature review, we first discuss both measurement methods as well as the trade-off between good coverage and serendipity. We then analyze the role of coverage and serendipity as indicators of recommendation quality, present novel ways of how they can be measured and discuss how to interpret the obtained measurements. Overall, we argue that our new ways of measuring these concepts reflect the quality impression perceived by the user in a better way than previous metrics thus leading to enhanced user satisfaction.

Consistency-based diagnosis of configuration knowledge bases

Configuration problems are a thriving application area for declarative knowledge representation that currently experiences a constant increase in size and complexity of knowledge bases. Automated support of the debugging of such knowledge bases is a necessary prerequisite for effective development of configurators. We show that this task can be achieved by consistency-based diagnosis techniques. Based on the formal definition of consistency-based configuration we develop a framework suitable for diagnosing configuration knowledge bases. During the test phase of configurators, valid and invalid examples are used to test the correctness of the system. In case such examples lead to unintended results, debugging of the knowledge base is initiated. Starting from a clear definition of diagnosis in the configuration domain we develop an algorithm based on conflicts. Our framework is general enough for its adaptation to diagnosing customer requirements to identify unachievable conditions during configuration sessions.

UML AS DOMAIN SPECIFIC LANGUAGE FOR THE CONSTRUCTION OF KNOWLEDGE-BASED CONFIGURATION SYSTEMS

In many domains, software development has to meet the challenges of developing highly adaptable software very rapidly. In order to accomplish this task, domain specific, formal description languages and knowledge-based systems are employed. From the viewpoint of the industrial software development process, it is important to integrate the construction and maintenance of these systems into standard software engineering processes. In addition, the descriptions should be comprehensible for the domain experts in order to facilitate the review process. For the realization of product configuration systems, we show how these requirements can be met by using a standard design language (UML-Unified Modeling Language) as notation in order to simplify the construction of a logic-based description of the domain knowledge. We show how classical description concepts for expressing configuration knowledge can be introduced into UML and be translated into logical sentences automatically. These sentences are exploited by a general inference engine solving the configuration task.

Conceptual modeling for configuration of mass-customizable products

Abstract The development and maintenance of product configuration systems is faced with increasing challenges caused by the growing complexity of the underlying knowledge bases. Effective knowledge acquisition is needed since the product and the corresponding configuration system have to be developed in parallel. In this paper, we show how to employ a standard design language (Unified Modeling Language, UML) for modeling configuration knowledge bases. The two constituent parts of the configuration model are the component model and a set of corresponding functional architectures defining which requirements can be imposed on the product. The conceptual configuration model is automatically translated into an executable logic representation. Using this representation we show how to employ model-based diagnosis techniques for debugging faulty configuration knowledge bases, detecting infeasible requirements, and for reconfiguring old configurations.

An Integrated Environment for the Development of Knowledge-Based Recommender Applications

The complexity of the product assortments offered by on-line selling platforms makes selection a challenging task. Customers differ in respect to expertise and product knowledge, but intelligent recommender systems offer personalized dialogues that support the product-selection process. This paper describes CWAdvisor, a domain-independent, knowledge-based recommender environment that provides users with consistently appropriate solutions, identifies additional selling opportunities, and explains solutions. The discussion uses examples from several application domains to show how model-based diagnosis, personalization, and intuitive knowledge-acquisition techniques support customer-oriented sales dialogues. Experience obtained in industrial projects is reported, and successfully deployed recommender applications are evaluated.

A framework for the development of personalized, distributed web-based configuration systems

For the last two decades, configuration systems relying on AI techniques have successfully been applied in industrial environments. These systems support the configuration of complex products and services in shorter time with fewer errors and, therefore, reduce the costs of a mass-customization business model. The European Union-funded project entitled CUSTOMER-ADAPTIVE WEB INTERFACE FOR THE CONFIGURATION OF PRODUCTS AND SERVICES WITH MULTIPLE SUPPLIERS (CAWICOMS) aims at the next generation of web-based configuration applications that cope with two challenges of todays open, networked economy: (1) the support for heterogeneous user groups in an open-market environment and (2) the integration of configurable subproducts provided by specialized suppliers.This article describes the CAWICOMS WORKBENCH for the development of configuration services, offering personalized user interaction as well as distributed configuration of products and services in a supply chain. The developed tools and techniques rely on a harmonized knowledge representation and knowledge-acquisition mechanism, open XML-based protocols, and advanced personalization and distributed reasoning techniques. We exploited the workbench based on the real-world business scenario of distributed configuration of services in the domain of information processing-based virtual private networks.

Configuration knowledge representations for Semantic Web applications

Todays economy exhibits a growing trend toward highly specialized solution providers cooperatively offering configurable products and services to their customers. This paradigm shift requires the extension of current standalone configuration technology with capabilities of knowledge sharing and distributed problem solving. In this context a standardized configuration knowledge representation language with formal semantics is needed in order to support knowledge interchange between different configuration environments. Languages such as Ontology Inference Layer (OIL) and DARPA Agent Markup Language (DAML+OIL) are based on such formal semantics (description logic) and are very popular for knowledge representation in the Semantic Web. In this paper we analyze the applicability of those languages with respect to configuration knowledge representation and discuss additional demands on expressivity. For joint configuration problem solving it is necessary to agree on a common problem definition. Therefore, we give a description logic based definition of a configuration problem and show its equivalence with existing consistency-based definitions, thus joining the two major streams in knowledge-based configuration (description logics and predicate logic/constraint based configuration).

Automated Generation of Music Playlists: Survey and Experiments

Most of the time when we listen to music on the radio or on our portable devices, the order in which the tracks are played is governed by so-called playlists. These playlists are basically sequences of tracks that traditionally are designed manually and whose organization is based on some underlying logic or theme. With the digitalization of music and the availability of various types of additional track-related information on the Web, new opportunities have emerged on how to automate the playlist creation process. Correspondingly, a number of proposals for automated playlist generation have been made in the literature during the past decade. These approaches vary both with respect to which kind of data they rely on and which types of algorithms they use. In this article, we review the literature on automated playlist generation and categorize the existing approaches. Furthermore, we discuss the evaluation designs that are used today in research to assess the quality of the generated playlists. Finally, we report the results of a comparative evaluation of typical playlist generation schemes based on historical data. Our results show that track and artist popularity can play a dominant role and that additional measures are required to better characterize and compare the quality of automatically generated playlists.

A knowledge-based framework for multimedia adaptation

AbstractPersonalized delivery of multimedia content over the Internet opens new business perspectives for future multimedia applications and thus plays an important role in the ongoing MPEG-7 and MPEG-21 multimedia standardization efforts. Based on these standards, next-generation multimedia services will be able to automatically prepare the digital content before delivery according to the clients device capabilities, the network conditions, or even the users content preferences. However, these services will have to deal with a variety of different end user devices, media formats, as well as with additional metadata when adapting the original media resources. In parallel, an increasing number of commercial or open-source media transformation tools will be available, capable of exploiting such descriptive metadata or dealing with new media formats; thus it is not realistic that a single tool will support all possible transformations.In this paper, we present a novel, fully knowledge-based approach for building such multimedia adaptation services, addressing the above mentioned issues of openness, extensibility, and concordance with existing and upcoming standards. In our approach, the original media is transformed in multiple adaptation steps performed by an extensible set of external tools, where the construction of adequate adaptation sequences is solved in an Artificial Intelligence planning process. The interoperability issue is addressed by exploiting standardized Semantic Web Services technology. This technology allows us to express tool capabilities and execution semantics in a declarative and well-defined form. In this context, existing multimedia standards serve as a shared domain ontology.The presented approach was implemented and successfully evaluated in an official ISO/IEC MPEG (Moving Picture Experts Group) Core Experiment and is currently under further evaluation by the standardization body.

What Recommenders Recommend – An Analysis of Accuracy, Popularity, and Sales Diversity Effects

In academic studies, the evaluation of recommender system (RS) algorithms is often limited to offline experimental designs based on historical data sets and metrics from the fields of Machine Learning or Information Retrieval. In real-world settings, however, other business-oriented metrics such as click-through-rates, customer retention or effects on the sales spectrum might be the true evaluation criteria for RS effectiveness. In this paper, we compare different RS algorithms with respect to their tendency of focusing on certain parts of the product spectrum. Our first analysis on different data sets shows that some algorithms – while able to generate highly accurate predictions – concentrate their top 10 recommendations on a very small fraction of the product catalog or have a strong bias to recommending only relatively popular items than others. We see our work as a further step toward multiple-metric offline evaluation and to help service providers make better-informed decisions when looking for a recommendation strategy that is in line with the overall goals of the recommendation service.