Neil Rubens

Active learning strategies for rating elicitation in collaborative filtering: A system-wide perspective

The accuracy of collaborative-filtering recommender systems largely depends on three factors: the quality of the rating prediction algorithm, and the quantity and quality of available ratings. While research in the field of recommender systems often concentrates on improving prediction algorithms, even the best algorithms will fail if they are fed poor-quality data during training, that is, garbage in, garbage out. Active learning aims to remedy this problem by focusing on obtaining better-quality data that more aptly reflects a users preferences. However, traditional evaluation of active learning strategies has two major flaws, which have significant negative ramifications on accurately evaluating the systems performance (prediction error, precision, and quantity of elicited ratings). (1) Performance has been evaluated for each user independently (ignoring system-wide improvements). (2) Active learning strategies have been evaluated in isolation from unsolicited user ratings (natural acquisition). In this article we show that an elicited rating has effects across the system, so a typical user-centric evaluation which ignores any changes of rating prediction of other users also ignores these cumulative effects, which may be more influential on the performance of the system as a whole (system centric). We propose a new evaluation methodology and use it to evaluate some novel and state-of-the-art rating elicitation strategies. We found that the system-wide effectiveness of a rating elicitation strategy depends on the stage of the rating elicitation process, and on the evaluation measures (MAE, NDCG, and Precision). In particular, we show that using some common user-centric strategies may actually degrade the overall performance of a system. Finally, we show that the performance of many common active learning strategies changes significantly when evaluated concurrently with the natural acquisition of ratings in recommender systems.

Understanding Business Ecosystem Dynamics: A Data-Driven Approach

Business ecosystems consist of a heterogeneous and continuously evolving set of entities that are interconnected through a complex, global network of relationships. However, there is no well-established methodology to study the dynamics of this network. Traditional approaches have primarily utilized a single source of data of relatively established firms; however, these approaches ignore the vast number of relevant activities that often occur at the individual and entrepreneurial levels. We argue that a data-driven visualization approach, using both institutionally and socially curated datasets, can provide important complementary, triangulated explanatory insights into the dynamics of interorganizational networks in general and business ecosystems in particular. We develop novel visualization layouts to help decision makers systemically identify and compare ecosystems. Using traditionally disconnected data sources on deals and alliance relationships (DARs), executive and funding relationships (EFRs), and public opinion and discourse (POD), we empirically illustrate our data-driven method of data triangulation and visualization techniques through three cases in the mobile industry Google’s acquisition of Motorola Mobility, the coopetitive relation between Apple and Samsung, and the strategic partnership between Nokia and Microsoft. The article concludes with implications and future research opportunities.

E-Learning 3.0: Anyone, Anywhere, Anytime, and AI

The concept of e-Learning 2.0 has become well established and widely accepted. Just like how e-Learning 2.0 replaced its predecessor, we are again on the verge of a transformation. Both previous generations of e-Learning 1.0 and 2.0 closely parody the prevalent technologies available in their kin Web versions 1.0 and 2.0, respectively. In order to acquire a better perspective to assess what technologies will be available in the Web 3.0 and therefore e-Learning 3.0, we take a historical glance at the previous generations of e-Learning and theWeb. We then survey some existing predictions for e-Learning 3.0 and finally provide our own. Previous surveys tend to identify educational needs for e-Learning, and then discuss what technologies are required to satisfy these needs. Educational needs are an important factor, but the required technologies may not reach fruition. Gauging past trends we take the reverse approach by first identifying technologies that are likely to be brought forth by the Web 3.0, and only then looking at how these technologies could be utilized in the learning domain. In particular, we pin-point Artificial Intelligence more specifically Machine Learning and Data Mining as a major driving force behind the Web 3.0. We therefore examine the influence that AI might exert on the development of e-Learning 3.0.

A batch ensemble approach to active learning with model selection

Optimally designing the location of training input points (active learning) and choosing the best model (model selection)-which have been extensively studied-are two important components of supervised learning. However, these two issues seem to have been investigated separately as two independent problems. If training input points and models are simultaneously optimized, the generalization performance would be further improved. In this paper, we propose a new approach called ensemble active learning for solving the problems of active learning and model selection at the same time. We demonstrate by numerical experiments that the proposed method compares favorably with alternative approaches such as iteratively performing active learning and model selection in a sequential manner.

Insights for orchestrating innovation ecosystems: the case of EIT ICT Labs and data-driven network visualisations

This paper explores opportunities for supporting the orchestration of innovation ecosystems, hence contributing to a fundamental capability in the networked world. We present analysis, evaluation and interpretation toward the objective of decision support and insights for transforming innovation ecosystems with a case study of EIT ICT Labs, a major initiative intended to turn Europe into a global leader in ICT innovation. Towards this, we use a data-driven, relationship-based and network centric approach to operationalise the ‘innovation ecosystems transformation framework’. Our results indicate that with coordinated and continuously improved use of visual and quantitative social network analysis, special characteristics, significant actors and connections in the innovation ecosystem can be revealed to develop new insights. We conclude that the IETF transformation framework can be used to develop shared vision and to support the orchestration of innovation ecosystem transformations.

Active Learning in Collaborative Filtering Recommender Systems

In Collaborative Filtering Recommender Systems user’s preferences are expressed in terms of rated items and each rating allows to improve system prediction accuracy. However, not all of the ratings bring the same amount of information about the user’s tastes. Active Learning aims at identifying rating data that better reflects users’ preferences. Active learning Strategies are used to selectively choose the items to present to the user in order to acquire her ratings and ultimately improve the recommendation accuracy. In this survey article, we review recent active learning techniques for collaborative filtering along two dimensions: (a) whether the system requested ratings are personalised or not, and, (b) whether active learning is guided by one criterion (heuristic) or multiple criteria.

Ostinato: The Exploration-Automation Cycle of User-Centric, Process-Automated Data-Driven Visual Network Analytics

Network analysis is a valuable method for investigating and mapping the social structure driving phenomena and sharing the findings with others. The interactive visual analytics approach transforms data into views that allow the visual exploration of the structures and processes of networks represented by data, therefore increasing the transparency of editorial processes on social media as well as networked structures in innovation ecosystems and other phenomena. Although existing tools have opened many new exploratory opportunities, new tools in development promise investigators even greater freedom to interact with the data, refine and analyze the data, and explore alternative explanations for networked processes. This chapter presents the Ostinato Model—an iterative, user-centric, process-automated model for data-driven visual network analytics. The Ostinato Model simultaneously supports the automation of the process and enables interactive and transparent exploration. The model has two phases, Data Collection and Refinement and Network Creation and Analysis. The Data Collection and Refinement phase is further divided into Entity Index Creation, Web/API Crawling, Scraping, and Data Aggregation. The Network Construction and Analysis phase is composed of Filtering in Entities, Node and Edge Creation, Metrics Calculation, Node and Edge Filtering, Entity Index Refinement, Layout Processing and Visual Properties Configuration. A cycle of exploration and automation characterizes the model and is embedded in each phase.

Adapting to natural rating acquisition with combined active learning strategies

The accuracy of collaborative-filtering recommender systems largely depends on the quantity and quality of the ratings added to the system over time. Active learning (AL) aims to improve the quality of ratings by selectively finding and soliciting the most informative ratings. However previous AL techniques have been evaluated assuming a rather artificial scenario: where AL is the only source of rating acquisition. However, users do frequently rate items on their own, without being prompted by the AL algorithms (natural acquisition). In this paper we show that different AL strategies work better under different conditions, and adding naturally acquired ratings changes these conditions and may result in a decreased effectiveness for some of them. While we are unable to control the naturally occurring changes in conditions, we should adaptively select the AL strategies which are well suited for the conditions at hand. We show that choosing AL strategies adaptively outperforms any of the individual AL strategies.

Automatic Group Formation for Informal Collaborative Learning

Ability to find appropriate collaborators and learning materials is crucial for informal collaborative learning. However, traditional group formation models are not applicable/effective in informal learning settings since little is known about learners and learning materials and teachers assistance is not available. We propose the data-driven group formation model that automatically extracts information about learners and learning materials from multiple data sources (databases of academic publications, wikis, social networking cites, blogs, forums, etc) and automatically forms collaborative learning groups. The open source implementation of the model (a part of WebClass-RAPSODY learning management system) consists of loosely coupled modules (implementing the proposed methods for data mashup, mining and inference) integrated through the web services interface; allowing for easy adaptation, extension and customization of the model.

Utilizing Social Media for Goal Setting Based on Observational Learning

Goal-Setting enhances learning by providing a sense of direction and purpose. Often only a few goals are suggested, as a result many learners fail to find the goals that they can relate to. To address this problem, we propose to extract a large number and variety of goals from social media. Learners can then observe goal-based messages from others and adopt the ones they find useful. Conceptually, this approach could be considered a combination of Goal-Setting and Observational Learning. To provide a practical implementation, we automate this process by (1) retrieving a large number of messages from Twitter, (2) classifying which ones contain goals, (3) determining what those goals are.