The TagRec Framework as a Toolkit for the Development of Tag-Based Recommender Systems
TThe TagRec Framework as a Toolkit for the Development ofTag-Based Recommender Systems
Dominik Kowald
Know-Center & Graz University ofTechnologydkowald@know-center . at Simone Kopeinik
Graz University ofTechnologysimone . kopeinik@tugraz . at Elisabeth Lex
Graz University ofTechnologyelisabeth . lex@tugraz . at ABSTRACT
Recommender systems have become important tools to supportusers in identifying relevant content in an overloaded informationspace. To ease the development of recommender systems, a numberof recommender frameworks have been proposed that serve a widerange of application domains. Our
TagRec framework is one of thefew examples of an open-source framework tailored towards devel-oping and evaluating tag-based recommender systems . In this paper,we present the current, updated state of
TagRec , and we summarizeand reflect on four use cases that have been implemented with
TagRec : (i) tag recommendations, (ii) resource recommendations,(iii) recommendation evaluation, and (iv) hashtag recommendations.To date,
TagRec served the development and/or evaluation processof tag-based recommender systems in two large scale Europeanresearch projects, which have been described in 17 research papers.Thus, we believe that this work is of interest for both researchersand practitioners of tag-based recommender systems.
KEYWORDS
Recommender Systems; Recommender Framework; Recommenda-tion Evaluation; Tag Recommendation; Hashtag Recommendation
Recommender systems aim to predict the probability that a spe-cific user will like a specific resource. Therefore, recommendersystems utilize the past user behavior (e.g., resources previouslyconsumed by this user) in order to generate a personalized list ofpotentially relevant resources [32]. Popular application domains ofrecommender systems include online marketplaces (e.g., Amazonand Zalando), movie and music streaming services (e.g., Netflix andSpotify), job portals (e.g., LinkedIn and Xing), and social taggingsystems (e.g., BibSonomy and CiteULike).Social tagging systems bear particularly great potential for rec-ommender systems as, by nature, they produce a vast amount ofuser-generated resource-annotations (i.e., tags). Thus, possible usecases of these tag-based recommender systems include the sugges-tion of resources to extend a user’s set of bookmarks [4, 38] and the
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected].
Conference’17, Washington, DC, USA © 2016 ACM. 978-x-xxxx-xxxx-x/YY/MM...$15.00DOI: 10 . . nnnnnnn suggestion of tags to assist in the annotation of these bookmarks.The latter one is known as the field of tag recommendations [13].Over the past years, various recommendation frameworks andlibraries have been developed in order to support the developmentand evaluation of recommender systems (see Section 4). Whilethese frameworks cover a wide range of application domains, to thebest of our knowledge, an open-source recommendation frameworkto design and evaluate tag-based recommender systems was stilllacking. Therefore, in 2014, we have started developing TagRec ,a standardized tag recommender benchmarking framework [19].In the initial development phase of the framework, we mainlyfocused on evaluating tag recommendation algorithms. In 2015, theframework was extended by including resource recommendationalgorithms that are based on social tagging data [36].The aim of this paper, however, is to present the current, updatedstate of
TagRec . This includes the extension of the framework for(i) the analysis of tag reuse practices [21], (ii) the evaluation oftag recommendations in real-world folksonomy and TechnologyEnhanced Learning settings [16, 20], and (iii) hashtag recommen-dations in Twitter [22].Apart from that, we provide an updated framework description(see Section 2) as well as a summary of use cases in the field ofrecommender research that have been completed using
TagRec (seeSection 3). Research areas encompass tag recommendations, re-source recommendations, recommendation evaluation and hashtagrecommendations. To date,
TagRec has served the recommenderdevelopment and/or evaluation processes in two large-scale Euro-pean research projects, which have been published in 17 researchpapers. We conclude the paper with a discussion on future workand potential improvements of the framework (see Section 5).We believe that our work contributes to the rich portfolio oftechnical frameworks in the area of recommender systems. Further-more, this paper presents an overview of use cases which can berealized with
TagRec , and should be of interest for both researchersand developers of tag-based recommender systems.
TagRec is a Java-based recommendation framework for tag-basedinformation retrieval settings. It is open-source software and freelyavailable via our Github repository . The Github page also containsa detailed technical description on the usage of the framework.Figure 1 illustrates TagRec ’s system architecture. The frame-work consists of (i) a data processing component, which processesdata sources, (ii) a data model and analytics component, which en-ables access to the processed data, (iii) recommendation algorithms, https://github . com/learning-layers/TagRec a r X i v : . [ c s . I R ] J a n agRec DataProcessing Data Model& AnalyticsRecommendationAlgorithmsEvaluationEngineRecommendationResults
ClientMetricsData
Figure 1: System architecture of
TagRec . Here, the data pro-cessing component processes data sources in order to cre-ate a data model and data analytics. Then, this data modelis used by recommendation algorithms to create recommen-dation results that are either forwarded to an evaluation en-gine or to a client application. which calculate recommendations, (iv) an evaluation engine, whichevaluates the algorithms, and (v) recommendation results, whichcan be passed to a client application. The mentioned componentsare described in more detail in the remainder of this section. Apartfrom that, we describe practical aspects of the framework thatshould be helpful when implementing and/or evaluating a recom-mendation algorithm. Finally, Table 1 provides an overview of thesupported datasets, recommendation algorithms and evaluationmetrics.
Data Processing.
The data processing component is responsi-ble for parsing and processing external data sources. Currentlysupported datasets are listed in Table 1. These datasets serve awide range of application domains such as social bookmarking sys-tems, learning environments, microblogging tools and music/moviesharing portals. The set of datasets can easily be extended by im-plementing custom data pre-processing strategies.Furthermore, this component supports various data enrichmentand transformation methods such as p -core pruning [8], topic mod-eling [25], training/test set splitting [20] and data conversion intorelated formats (e.g., for MyMediaLite [9]). Data Model and Analytics.
The data model is created based ondescribed data processing steps and provides an object-orientedrepresentation of the data in order to ease the implementationprocess of a novel recommendation algorithm. Thus, it enables easyaccess to the entities in the datasets via powerful query functionality(e.g., get the set of tags a user has used in the past). Furthermore,the data model of
TagRec is connected to Apache Solr and thus,enables fast access to content-based data of entities.Another role of this component is the provision of basic dataanalytics functionality to get a better understanding of the datasetcharacteristics. For example, dataset statistics, such as the totalnumber of distinct tags or the average number of bookmarks peruser, can be retrieved. Recommendation Algorithms.
TagRec contains a wide range ofrecommendation algorithms (see Table 1). As later described in http://lucene . apache . org/solr/ Dataset Description
Flickr Image sharing [20]CiteULike Scientific references [20]BibSonomy Publication sharing [20]Delicious Social bookmarking [20]LastFM Music sharing [20]MovieLens Movie rating [20]Twitter Microblogging [22]TravelWell Learning resource exchange [16]Aposdle Work-integrated learning [16]MACE Informal learning [16]KDD15 KDD 2015 cup [16]
Algorithm Description
MostPopular Frequency-based [21]CF Collaborative Filtering [30]FolkRank / APR Graph-based [13]FM / PITF Factorization Machines [31]LDA Topic modeling [25]MostRecent / GIRP Time-based [21]3Layers Human categorization theory [17, 23, 35]BLL / BLL AC Human memory theory [18, 22, 24, 37]CIRTT Tag- and time-based [27]SUSTAIN Human category learning [15, 34]SimRank Content-based [39]BLL I , S , C Temporal hashtag patterns [22]
Metric Description
Recall Accuracy [20]Precision Accuracy [20]F1-score Accuracy [20]MRR Ranking [20]MAP Accuracy & ranking [20]nDCG Accuracy & ranking [20]AILD Diversity [20]AIP Novelty [20]Runtime Computational costs [20]Memory Computational costs [20]
Table 1: Datasets, recommendation algorithms and evalua-tion metrics supported by
TagRec . A complete list of thefeatures is provided on the framework’s Github page.
Section 3, algorithms for tag recommendations, resource recom-mendations and hashtag recommendations are provided. Thesealgorithms can be used as baseline approaches for a newly im-plemented algorithm. The complete list of all variants of thesealgorithms is provided on the
TagRec ’s Github page.A key contributions of
TagRec is that next to well-establishedapproaches, such as Collaborative Filtering, MostPopular and Fac-torization machines, it also encompasses approaches based uponcognitive models of information retrieval, human memory the-ory and category learning. In [20], it has been shown that thesecognitive-inspired approaches achieve high prediction accuracyestimates in comparison to classic recommendation algorithms. Be-sides, the algorithms have demonstrated their suitability for sparsedatasets such and narrow folksonomies. igure 2: Screenshot of a user interface for the online evalu-ation of tag recommendations using the
TagRec framework.Evaluation Engine.
The evaluation engine quantifies the qualityof implemented recommendation strategies by applying a rich setof evaluation metrics as listed in Table 1. One drawback of mostrecommendation evaluation frameworks is their focus on accu-racy and ranking estimates, which restricts the evaluation to theperformance of recommender systems [2].To fill this gap,
TagRec supports a variety of evaluation metrics toalso offer indicators for diversity, novelty, runtime performance andmemory consumption of algorithms. For evaluating an algorithm,
TagRec has to be provided with three parameters, where the firstone specifies the algorithm, the second one specifies the datasetdirectory and the third one specifies the file name of the datasetsample. For example, java jar tagrec.jar cf bib bib sample runsCollaborative Filtering on a sample of the BibSonomy dataset. Thecalculated metrics are then either written to a “metrics” file orprinted to the console.
Recommendation Results.
As indicated in Figure 1, the algo-rithms’ recommendation results can be either forwarded to theevaluation engine to retrieve evaluation metrics or to a client appli-cation for further processing (e.g., visualization). The KnowBraintool [7] is an example of such a client application. It is an opensource social bookmarking tool, which has been extended to caterthe requirements of tag recommender evaluations in online settings.A screenshot of KnowBrain’s graphical user interface is shown inFigure 2. It enables the bookmarking of Web links and their annota-tions by (i) selecting from a pre-defined set of categories, and by (ii)assigning a variable number of tags. The user’s tagging process issupported by a list of recommended tags that are selected based on
Tag recommendations Research papers
Model of human categorization [17, 23, 35]Activation processes in human memory [18, 21, 24, 37]Informal learning settings [5–7]
Resource recommendations Research papers
Attention-interpretation dynamics [15, 34]Tag and time information [27, 28]
Recommendation evaluation Research papers
Real-world folksonomies [20]Technology enhanced learning settings [16]
Hashtag recommendations Research papers
Temporal effects on hashtag reuse [22]
Table 2: Use cases realized with
TagRec . To date,
TagRec sup-ported the recommender development and/or evaluationprocesses described in 17 research papers. algorithms of the
TagRec framework. The elicitation of categories al-lows for semantic context-based recommendation algorithms suchas 3Layers [35]. Furthermore, the comparison of actually used tagswith recommended tags gives insights into the online performance(i.e., user acceptance) of recommendation strategies.
In this section, we describe use cases that have been implementedusing the
TagRec framework. To date,
TagRec supported the recom-mender development and/or evaluation processes in two large-scaleEuropean research projects. Results have been published in 17 re-search papers (see Table 2).
Tag recommendation systems assist users in finding descriptivetags to annotate resources. In other words, given a specific userand a specific resource, a tag recommendation algorithm predicts aset of tags a user is likely to apply in annotating the resource [13].Within this context,
TagRec was used for the creation of (i)cognitive-inspired algorithms, and (ii) the evaluation of approachessuitable for formal and informal learning settings.
Tag Recommendations Using a Model of Human Categoriza-tion.
In [17, 23, 35], the authors introduced a tag recommendationalgorithm based on the human categorization models ALCOVE[26] and MINERVA2 [12]. This algorithm is called and sim-ulates categorization processes in human memory. Therefore, thecategories assigned to a given resource, which a user is going toannotate, are matched against already annotated resources of thisuser. Based on this matchmaking process, a set of tags associatedwith semantically related resources is recommended.Since
TagRec enables to link a list of categories to a resource, itsupported the development of by providing functions foranalyzing and deriving category information of resources (e.g., viaLDA topic modeling [25]).
Utilizing Activation Processes in Human Memory.
Activa-tion processes in human memory describe the general and context-dependent usefulness of information. It was shown that theserocesses (especially usage frequency, recency and semantic con-text) greatly influence the reuse probability of tags [21]. Basedon this, a set of time-aware tag recommendation approaches (see[18, 24, 37]) was developed that utilize the activation equation ofthe cognitive architecture ACT-R [1].Therefore,
TagRec was used to analyze the timestamps of tagassignments and to calculate tag co-occurrences for reflecting thesemantic context of social tagging. Furthermore,
TagRec enabled thehybrid combination of the components of the model (e.g., combiningtime-aware and context-aware recommendations).
Tag Recommendations in Informal Learning Settings.
In thecourse of the European-funded project Learning Layers , whichaims at supporting informal learning at the workplace, tag recom-mendations were used to support the individual user in findingdescriptive tags and the collective in consolidating a shared tagvocabulary. These tag recommendations were used in two tools: (i)the Dropbox-like environment KnowBrain [7], and (ii) the Sense-making interface Bits & Pieces [6].To achieve this, TagRec was integrated as a tag recommendationlibrary into the Social Semantic Server [5], which was used as thetechnical back-end for KnowBrain and Bits & Pieces. This showsthat
TagRec cannot only be used as a standalone tool but also as aprogramming library (or toolkit) to include recommendation func-tionality in existing software. A similar approach will be followedin another European-funded project called AFEL , which engagesin design and development of analytics for everyday learning. Resource recommender systems suggest potentially relevant webitems (e.g., movies, books, learning resources, URLs, etc.) to users.Most of these recommender systems are based on CollaborativeFiltering (CF) techniques, which aim to calculate similarities be-tween users to suggest the most suitable web resources to them[33].
TagRec was applied to support and improve the developmentof CF approaches in tag-based online environments.
Mimicking Attention-Interpretation Dynamics.
Seitlinger etal. [34] introduced the first version of a CF-based recommenda-tion approach that takes into consideration non-linear user-artifactdynamics, modeled by means of SUSTAIN. SUSTAIN (
Supervisedand Unsupervised STratified Adaptive Incremental Network ) is a flex-ible network model of human category learning that is thoroughlydiscussed in [29]. It assumes that learning is a dynamic processthat takes place through the encounter of new examples (e.g., Webresources). Throughout the learning trajectory, categories emergeand learners’ attention foci shift. In [15], an advanced, adaptedversion of the initial approach was presented and analyzed in de-tail. The resulting approach
SU STAI N + CF , firstly applies CF tocalculate the most suitable resources for a user, and secondly re-ranks this list depending on a user’s category learning model (i.e.,SUSTAIN’s user model).The algorithm has been implemented and developed within the TagRec framework. Features of the framework allowed for continu-ous evaluation and analysis of single factors of the model and with http://learning-layers . eu/ http://afel-project . eu/ it, the associated change of recommendation performance. Withthis data, it was possible to gain deeper insight into the algorithmicapproach and its parameters and thus, to further adapt the modelto the requirements of our application area. Resource Recommendations using Tag and Time Informa-tion.
In [27], the
Collaborative Item Ranking Using Tag and TimeInformation (CIRTT) approach was presented. CIRTT uses Collabo-rative Filtering to identify a set of candidate resources and re-ranksthese candidate resources by incorporating tag and time informa-tion. This is achieved via the Base-Level-Learning (BLL) equation,which is one component of ACT-R’s activation equation [1].Since
TagRec contains a full implementation of the activationequation, it could be easily adapted for the task of resource recom-mendations as well. Apart from that,
TagRec was used to compare
CIRTTT to other related resource recommendation methods (e.g.,[40]). Another study of the recency effects in Collaborative Filteringrecommender systems was provided in [28].
One of the most challenging tasks in the area of recommendersystems, is the reproducible evaluation of recommendation results[11].
TagRec aims to support this process by providing standard-ized data processing methods, baseline algorithm implementations,evaluation protocols and metrics.
Evaluating Tag Recommendations in Real-World Folksono-mies.
Because of the sparse nature of social tagging systems, mosttag recommendation evaluation studies were conducted using p -core pruned datasets. This means that all users, resources and tags,which do not appear at least p times in the dataset, are removed.This clearly does not reflect a real-world folksonomy setting asshown by [8].To overcome this problem, TagRec was used in [20] to comparea rich set of tag recommendation algorithms using a wide rangeof evaluation metrics on six unfiltered social tagging datasets (i.e.,Flickr, CiteULike, BibSonomy, Delicious, LastFM and MovieLens).The results showed that the efficacy of a recommendation algo-rithm greatly depends on the given dataset characteristics, and thatcognitive-inspired approaches provide the most robust results, evenin sparse data folksonomy settings.
Comparing Recommendation Algorithms in Technology En-hanced Learning Settings.
Kopeinik et al. [16] is another exam-ple of using
TagRec for the evaluation of a variety of algorithmson different offline datasets. The paper focused on technology-enhanced formal and informal learning environments, where dueto fast changing domains and characteristic group learning set-tings, data is typically sparse. The evaluation was divided in twosettings, the performance of (i) resource recommendation strate-gies, and (ii) tag recommendation strategies. In both cases, theauthors compared the recommendation accuracy of a number ofcomputationally-inexpensive recommendation algorithms on six of-fline datasets retrieved from various educational settings (i.e., socialbookmarking systems, social learning environments and massiveopen online courses). Investigated approaches are either state-of-the-art recommendation approaches, or strategies that have beenexplicitly suggested in the context of TEL systems.o address the goals of this study, the
TagRec framework al-ready provided a wide range of required functionality such as theimplemented data processing component, evaluation metrics andstate-of-the-art algorithms. In the context of this research paper, itwas further extended by a couple of algorithms that are consideredparticularly relevant to learning settings and by additional statistics,which were needed to interpret evaluation results properly.
Over the past years, hashtags have become very popular in systemssuch as Twitter, Instagram and Facebook. Similar to social tags,hashtags are freely-chosen keywords to categorize resources suchas Twitter posts (i.e., tweets). One of the biggest advantages ofhashtags is that they can be easily used by integrating them inthe tweet text. Unsurprisingly, this has led to the development ofhashtag recommendation algorithms that aim to support users inapplying the most descriptive hashtags to their tweets [39].
Temporal Effects on Hashtag Reuse.
In [22], a time-dependentand cognitive-inspired hashtag recommendation approach was pro-posed. In this paper, temporal effects on hashtag reuse in Twitterhave been analyzed with the help of
TagRec in order to design ahashtag recommendation approach, which utilizes the BLL equa-tion of the cognitive architecture ACT-R [1]. Therefore,
TagRec was extended with functions to access Apache Solr (see Section 2),which enables the content-based analysis of tweets using TF-IDF(see [22]).
In recent years, a multitude of recommendation engines have beencreated and made available either for commercial use or as open-source systems. In [14], a well-structured overview of such ap-proaches is presented. The author differentiates between Software-as-a-Service (i.e., SaaS), non-SaaS, open-source, academic and bench-marking recommender systems. We consider open-source andbenchmarking frameworks that implement recommendation strate-gies fitting academic purposes as most relevant to our work.A considerable contribution to this area is
LibRec , a Java-basedlibrary that, so far, comprises around 70 resource recommendationalgorithms and evaluation modules [10]. Another Java-based, open-source framework is RankSys , which focuses on the evaluationof ranking problems and supports the investigation of novelty aswell as diversity for academic research [3], which is reflected in itsdesign (e.g., data input interfaces work with a triple of user, itemand features).Other examples of open-source recommender software are My-MediaLite , an item recommender library that focuses on ratingand ranking predictions in collaborative filtering approaches [9], CARSKit , a recommendation library specifically designed for context-aware recommendations, and Tag Recommender , a software compo-nent that implements Tensor Factorization models for personalizedtag recommendations in C++ [31]. http://wiki . librec . net/doku . php http://ranksys . org/ . mymedialite . net/ https://github . com/irecsys/CARSKit . libfm . org/tagrec . html However, to the best of our knowledge, an open-source recom-mender framework that implements a wide range of tag and re-source recommendation algorithms (including a number of cognitive-inspired approaches) for the design and evaluation of personalizedtag-based recommendation strategies was still missing.
In this paper, we presented the
TagRec framework as a toolkit forthe development and evaluation of tag-based recommender sys-tems.
TagRec is open-source software written in Java and can befreely downloaded from Github. The framework consists of fivecomponents: (i) a data processing component, which processesdata sources, (ii) a data model and analytics component, which en-ables access to the processed data, (iii) recommendation algorithms,which calculate recommendations, (iv) an evaluation engine, whichevaluates the algorithms, and (v) recommendation results, whichcan be passed to client applications.Apart from that, we summarized various use cases realized with
TagRec from the fields of tag recommendations, resource recommen-dations, recommendation evaluation and hashtag recommendations.To date,
TagRec supported the development and/or evaluation pro-cess described in 17 research papers. Specifically, our frameworkwas used for the realization of recommendation algorithms basedon models of cognitive science. In these papers, it was shown thatthe cognitive-inspired approaches provided the most robust results,even in sparse data folksonomy settings.We believe that
TagRec extends the already rich portfolio ofrecommender frameworks with a toolkit that is specifically tailoredto fit tag-based settings. Furthermore, the presentation of
TagRec ’suse cases should be of interest for both researchers and developersof tag-based recommender systems.
Limitations & future work.
Currently, one limitation of
TagRec is that the data access is not standardized. Thus, social taggingdata is accessed from folksonomy files, whereas resource-relatedmetadata (e.g., tweet content) is accessed from Apache Solr.Thus, our first plan for future work is to implement a mecha-nism that integrates all data into Apache Solr. Apart from that, wewant to further work on the stability and code quality of the frame-work. For example, we want to enhance the build and dependencymanagement of the software using Apache Maven . This work was supported by the Know-Center Graz, the European-funded projects AFEL (GA: 687916) and Learning Layers (GA: 318209),and the Austrian Science Fund (FWF) project OMFix (Grant NoP27709-G22). The Know-Center Graz is funded within the AustrianCOMET Program - Competence Centers for Excellent Technolo-gies - under the auspices of the Austrian Ministry of Transport,Innovation and Technology, the Austrian Ministry of Economicsand Labor and by the State of Styria. COMET is managed by theAustrian Research Promotion Agency (FFG). https://maven . apache . org/ EFERENCES [1] John R Anderson, Daniel Bothell, Michael D Byrne, Scott Douglass, ChristianLebiere, and Yulin Qin. 2004. An integrated theory of the mind.
Psychologicalreview
Proceedings of the22nd ACM international conference on Information & Knowledge Management .ACM, 109–118.[3] Pablo Castells, Neil J Hurley, and Saul Vargas. 2015. Novelty and diversity inrecommender systems. In
Recommender Systems Handbook . Springer, 881–918.[4] Pasquale De Meo, Giovanni Quattrone, and Domenico Ursino. 2010. A queryexpansion and user profile enrichment approach to improve the performanceof recommender systems operating on a folksonomy.
User Modeling and User-Adapted Interaction
20, 1 (2010), 41–86.[5] Sebastian Dennerlein, Dominik Kowald, Elisabeth Lex, Dieter Theiler, EmanuelLacic, and Tobias Ley. 2015. The social semantic server: a flexible frameworkto support informal learning at the workplace. In
Proceedings of the 15th Inter-national Conference on Knowledge Technologies and Data-driven Business . ACM,26.[6] Sebastian Dennerlein, Matthias Rella, Vladimir Tomberg, Dieter Theiler, TamsinTreasure-Jones, Micky Kerr, Tobias Ley, Mohammad Al-Smadi, and ChristophTrattner. 2014. Making sense of bits and pieces: A sensemaking tool for informalworkplace learning. In
European Conference on Technology Enhanced Learning .Springer, 391–397.[7] Sebastian Dennerlein, Dieter Theiler, Peter Marton, Patricia Santos Rodriguez,John Cook, Stefanie Lindstaedt, and Elisabeth Lex. 2015. Knowbrain: An onlinesocial knowledge repository for informal workplace learning. In
Design forTeaching and Learning in a Networked World . Springer, 509–512.[8] Stephan Doerfel and Robert J¨aschke. 2013. An analysis of tag-recommenderevaluation procedures. In
Proceedings of the 7th ACM conference on Recommendersystems . ACM, 343–346.[9] Zeno Gantner, Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2011. MyMediaLite: A free recommender system library. In
Proceedingsof the fifth ACM conference on Recommender systems . ACM, 305–308.[10] Guibing Guo, Jie Zhang, Zhu Sun, and Neil Yorke-Smith. 2015. LibRec: A JavaLibrary for Recommender Systems.. In
UMAP Workshops .[11] Jonathan L Herlocker, Joseph A Konstan, Loren G Terveen, and John T Riedl.2004. Evaluating collaborative filtering recommender systems.
ACM Transactionson Information Systems (TOIS)
22, 1 (2004), 5–53.[12] Douglas L Hintzman. 1984. MINERVA 2: A simulation model of human memory.
Behavior Research Methods, Instruments, & Computers
16, 2 (1984), 96–101.[13] Robert J¨aschke, Leandro Marinho, Andreas Hotho, Lars Schmidt-Thieme, andGerd Stumme. 2007. Tag recommendations in folksonomies. In
European Confer-ence on Principles of Data Mining and Knowledge Discovery . Springer, 506–514.[14] Graham Jenson. 2017. List of Recommender Systems. (2017). https://github . com/grahamjenson/list of recommender systems[15] Simone Kopeinik, Dominik Kowald, Ilire Hasani-Mavriqi, and Elisabeth Lex.2017. Improving Collaborative Filtering Using a Cognitive Model of HumanCategory Learning. The Journal of Web Science
2, 1 (2017).[16] Simone Kopeinik, Dominik Kowald, and Elisabeth Lex. 2016. Which AlgorithmsSuit Which Learning Environments? A Comparative Study of Recommender Sys-tems in TEL. In
European Conference on Technology Enhanced Learning . Springer,124–138.[17] Dominik Kowald. 2015. Modeling Cognitive Processes in Social Tagging to Im-prove Tag Recommendations. In
Proceedings of the 24th International Conferenceon World Wide Web . ACM, 505–509.[18] Dominik Kowald, Simone Kopeinik, Paul Seitlinger, Tobias Ley, Dietrich Albert,and Christoph Trattner. 2015. Refining frequency-based tag reuse predictionsby means of time and semantic context. In
Mining, Modeling, and Recommend-ing’Things’ in Social Media . Springer, 55–74.[19] Dominik Kowald, Emanuel Lacic, and Christoph Trattner. 2014. Tagrec: Towardsa standardized tag recommender benchmarking framework. In
Proceedings ofthe 25th ACM conference on Hypertext and social media . ACM, 305–307.[20] Dominik Kowald and Elisabeth Lex. 2015. Evaluating tag recommender algo-rithms in real-world folksonomies: A comparative study. In
Proceedings of the9th ACM Conference on Recommender Systems . ACM, 265–268. [21] Dominik Kowald and Elisabeth Lex. 2016. The influence of frequency, recencyand semantic context on the reuse of tags in social tagging systems. In
Proceedingsof the 27th ACM Conference on Hypertext and Social Media . ACM, 237–242.[22] Dominik Kowald, Subhash Pujari, and Elisabeth Lex. 2017. Temporal Effectson Hashtag Reuse in Twitter: A Cognitive-Inspired Hashtag RecommendationApproach. In
Proceedings of the 26th International Conference on World Wide Web(WWW ’17) . ACM.[23] Dominik Kowald, Paul Seitlinger, Simone Kopeinik, Tobias Ley, and ChristophTrattner. 2015. Forgetting the words but remembering the meaning: Modelingforgetting in a verbal and semantic tag recommender. In
Mining, Modeling, andRecommending’Things’ in Social Media . Springer, 75–95.[24] Dominik Kowald, Paul Seitlinger, Christoph Trattner, and Tobias Ley. 2014. Longtime no see: The probability of reusing tags as a function of frequency andrecency. In
Proceedings of the 23rd International Conference on World Wide Web .ACM, 463–468.[25] Ralf Krestel, Peter Fankhauser, and Wolfgang Nejdl. 2009. Latent dirichletallocation for tag recommendation. In
Proceedings of the third ACM conferenceon Recommender systems . ACM, 61–68.[26] John K Kruschke. 1992. ALCOVE: An exemplar-based connectionist model ofcategory learning.
Psychological review
99, 1 (1992), 22.[27] Emanuel Lacic, Dominik Kowald, Paul Seitlinger, Christoph Trattner, and DenisParra. 2014. Recommending Items in Social Tagging Systems Using Tag andTime Information. In
Proceedings of the 1st International Workshop on SocialPersonalisation co-located with the 25th ACM Conference on Hypertext and SocialMedia (HT ’14) . CEUR-WS, CEUR-WS.[28] Santiago Larrain, Christoph Trattner, Denis Parra, Eduardo Graells-Garrido,and Kjetil Nørv˚ag. 2015. Good times bad times: A study on recency effects incollaborative filtering for social tagging. In
Proceedings of the 9th ACM Conferenceon Recommender Systems . ACM, 269–272.[29] Bradley C Love, Douglas L Medin, and Todd M Gureckis. 2004. SUSTAIN: anetwork model of category learning.
Psychological review
Data Analysis, Machine Learning and Applications . Springer,533–540.[31] Steffen Rendle and Lars Schmidt-Thieme. 2010. Pairwise interaction tensorfactorization for personalized tag recommendation. In
Proceedings of the thirdACM international conference on Web search and data mining . ACM, 81–90.[32] Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011.
Introduction to recom-mender systems handbook . Springer.[33] Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-basedcollaborative filtering recommendation algorithms. In
Proceedings of the 10thinternational conference on World Wide Web . ACM, 285–295.[34] Paul Seitlinger, Dominik Kowald, Simone Kopeinik, Ilire Hasani-Mavriqi, Elisa-beth Lex, and Tobias Ley. 2015. Attention please! a hybrid resource recommendermimicking attention-interpretation dynamics. In
Proceedings of the 24th Interna-tional Conference on World Wide Web . ACM, 339–345.[35] Paul Seitlinger, Dominik Kowald, Christoph Trattner, and Tobias Ley. 2013.Recommending tags with a model of human categorization. In
Proceedings of the22nd ACM international conference on Conference on information & knowledgemanagement . ACM, 2381–2386.[36] Christoph Trattner, Dominik Kowald, and Emanuel Lacic. 2015. TagRec: towardsa toolkit for reproducible evaluation and development of tag-based recommenderalgorithms.
ACM SIGWEB Newsletter
Winter (2015), 3.[37] Christoph Trattner, Dominik Kowald, Paul Seitlinger, Tobias Ley, SimoneKopeinik, and others. 2016. Modeling Activation Processes in Human Memoryto Predict the Use of Tags in Social Bookmarking Systems.
The Journal of WebScience
2, 1 (2016).[38] Valentina Zanardi and Licia Capra. 2008. Social ranking: uncovering relevantcontent using tag-based recommender systems. In
Proceedings of the 2008 ACMconference on Recommender systems . ACM, 51–58.[39] Eva Zangerle, Wolfgang Gassler, and Gunther Specht. 2011. Recommending
Proceedings of the Workshop on Semantic Adaptive Social Web(SASWeb 2011). CEUR Workshop Proceedings , Vol. 730. 67–78.[40] Nan Zheng and Qiudan Li. 2011. A recommender system based on tag and timeinformation for social tagging systems.