An Overview of Direct Diagnosis and Repair Techniques in the WeeVis Recommendation Environment
Alexander Felfernig, Stefan Reiterer, Martin Stettinger, Michael Jeran
AAn Overview of Direct Diagnosis and Repair Techniquesin the W EE V IS Recommendation Environment ∗ Alexander Felfernig and Stefan Reiterer and Martin Stettinger and Michael Jeran Graz University of Technology, Graz, Austriae-mail: {alexander.felfernig, stefan.reiterer, martin.stettinger, mjeran}@ist.tugraz.at
Abstract
Constraint-based recommenders support users inthe identification of items (products) fitting theirwishes and needs. Example domains are finan-cial services and electronic equipment. In this pa-per we show how divide-and-conquer based (di-rect) diagnosis algorithms (no conflict detection isneeded) can be exploited in constraint-based rec-ommendation scenarios. In this context, we pro-vide an overview of the MediaWiki-based recom-mendation environment W EE V IS . Constraint-based recommenders [1; 2] support the identifi-cation of relevant items from large and often complex as-sortments. Example item domains are electronic equipment[3] and financial services [4]. In contrast to collaborativefiltering [5] and content-based filtering [6], constraint-basedrecommendation relies on an explicit representation of rec-ommendation knowledge. Two major types of knowledgesources are exploited for the definition of a constraint-basedrecommendation task [2]. First, knowledge about the givenset of customer requirements. Second, recommendationknowledge that is represented as a set of items and a setof constraints that help to establish a relationship betweenrequirements and the item assortment.Diagnosis techniques can be useful in the following sit-uations: (1) in situations where it is not possible to find asolution for a given set of user (customer) requirements, i.e.,the requirements are inconsistent with the recommendationknowledge base and the user is in the need for repair propos-als to find a way out from the "no solution could be found"dilemma; (2) if a recommendation knowledge base is incon-sistent with a set of test cases that has been defined for thepurpose of regression testing, the knowledge engineer needssupport in figuring out the responsible faulty constraints.For situation (1) we sketch how model-based diagnosis[7] can be applied for the identification of faulty constraintsin a given set of customer requirements. In this context ef-ficient divide-and-conquer based algorithms can be appliedto the diagnosis and repair of inconsistent requirements. Ina similar fashion, such algorithms can be applied for the di-agnosis of inconsistent recommender knowledge bases (theknowledge base itself can be inconsistent, or alternatively, ∗ This work has been conducted in the project P
EOPLE V IEWS funded by the Austrian Research Promotion Agency (843492). inconsistencies can be induced by test cases used for regres-sion testing).The diagnosis approaches presented in this paper havebeen integrated into W EE V IS which is a MediaWiki-basedrecommendation environment for complex products andservices. In the line of the Wikipedia idea to support com-munities of users in the cooperative development of Webcontent, W EE V IS is an environment that supports all thefunctionalities available for the creation of Wiki pages. Ad-ditionally, it allows the inclusion of constraint-based recom-mender applications that help to work up existing knowl-edge and present this in a compressed and intuitive fashion.The contributions of this paper are the following. First,we sketch how efficient divide-and-conquer based algo-rithms can be applied for solving diagnosis and repair tasksin constraint-based recommendation scenarios. Second, wesketch how diagnosis and repair approaches can be inte-grated into Wiki technologies and with this be made ac-cessible to a large user group. Third, we discuss challengesfor future research that have to be tackled to advance thestate-of-the-art in constraint-based recommendation.The remainder of this paper is organized as follows. InSection 2 we discuss properties of constraint-based recom-mendation tasks. Thereafter, we introduce an example rec-ommendation knowledge base. In Section 3 we show howdivide-and-conquer based algorithms can be applied for thediagnosis and repair of inconsistent requirements. There-after we show how such algorithms can be applied to theidentification of faulty constraints in knowledge bases (seeSection 4). Related and future work are discussed in Section5. We conclude the paper with Section 6. In the remainder of this paper we will use personal com-puter recommendation as working example. Roughlyspeaking, a recommendation task consists of selecting thoseitems that match the user requirements. In the context ofpersonal computers, the recommender user has to specifyhis/her requirements regarding, for example, the intendedusage, the maximum accepted price, and the cpu type. SinceW EE V IS is a MediaWiki-based environment, the definitionof a recommender knowledge base is supported in a textualfashion (see Figure 1).On the basis of a set of requirements, the recommendersystem determines alternative solutions (the consideration a r X i v : . [ c s . I R ] F e b igure 1: Example W EE V IS PC Recommender definition (MediaWiki "Edit" mode).set) and presents these to the user. If no solution could befound for the given requirements, repair alternatives are de-termined which support users in getting out of the "no solu-tion could be found" dilemma (see Figure 3).Constraint-based recommendation requires the explicitdefinition of questions (representing alternatives for user re-quirements), properties of the items, and constraints. Anexample of a recommendation knowledge base is shownin Figure 1. The W EE V IS tag &QUESTIONS enumeratesvariables that describe user requirements where usage spec-ifies the intended use of the computer, eefficiency representsthe required energy efficiency, maxprice denotes the up-per price limit specified by the user, country represents thecountry of the user, mb represents the type of motherboard,and cpu the requested central processing unit. If a variableis associated with a keep tag, this variable is not taken intoaccount in the diagnosis process. For example, country? isassociated with a keep tag; for this reason, it will not be partof any diagnosis presented to the recommender user. Otherexamples of such attributes are a person’s age and gender.In addition to variables representing potential user re-quirements, a recommendation knowledge base includes the definition of variables that represent item properties (repre-sented by the W EE V IS tag &PRODUCTS). In our example,cpu p represents the CPU included in the item, mb p speci-fies the included motherboard, os p represents the installedoperating system, and price p is the overall price. Further-more, the set of items (products) must be specified that canbe recommended to users. A simplified item assortment isincluded in Figure 1 as part of the item properties. Our ex-ample assortment of items consists of the entries hw1 , hw2 ,and energystar .Incompatibility constraints describe combinations of re-quirements that lead to an inconsistency. The descriptionrelated to the W EE V IS tag &CONSTRAINTS includes anincompatibility relationship between the variable usage andthe variable cpu . For example, computers with a CPUDmust not be sold to users interested in scientific calculations.Filter constraints describe the relationship between userrequirements and items. A simple example of such a filterconstraint is maxprice ≥ price p , i.e., the price of an rec-ommended item must be equal or below the maximum ac-cepted price specified by the customer (see the W EE V IS tag&CONSTRAINTS in Figure 1).inally, W EE V IS supports the definition of test cases (seealso Section 4) which can be used to specify the intendedbehavior of a recommender knowledge base (W EE V IS tag&TEST). After changes to the knowledge base, regressiontests can be triggered on the basis of the defined test suite.The |show| tag specifies whether the recommender systemuser interface should show the status of the test case (satis-fied or not) – see, for example, Figure 4.On a formal level, a recommendation knowledge base canbe represented as a constraint satisfaction problem [8] withtwo sets of variables V = U ∪ P and the corresponding con-straints C = COMP ∪ PROD ∪ FILT. In this context, u i ∈ U are variables describing possible user requirements (e.g., us-age or maxprice ) and p i ∈ P are variables describing item(product) properties (e.g., mb p or price p ).The recommendation knowledge base specified in Figure1 can be transformed into a constraint satisfaction prob-lem where &QUESTIONS represents U , &PRODUCTSrepresents P and P ROD , and &CONSTRAINTS repre-sents
COM P and
F ILT . Given such a recommendationknowledge base we are able to determine concrete recom-mendations on the basis of a specified set of user (customer)requirements. Requirements collected are represented interms of constraints, i.e., R = { r , r , ..., r k } represents aset of user requirements.After having identified the set of alternative solutions(recommended items or consideration set), this result is pre-sented to the user. In constraint-based recommendation sce-narios, the ranking of items is often performed on the basisof Multi-Attribute Utility Theory (MAUT) where items areevaluated on the basis of a given set of interest dimensions.For further details on the ranking of items in constraint-based recommendation scenarios we refer to [9]. In situations where the given set of requirements r i ∈ R (unary constraints defined on variables of U such as max-price ≤ ) become inconsistent with the recommendationknowledge base (C), we are interested in repair proposalsthat indicate for a subset of these requirements change oper-ations with a high probability of being accepted by the user.On a more formal level we now introduce a definition of acustomer requirements diagnosis task and a correspondingdiagnosis (see Definition 1). Definition 1 (Requirements Diagnosis Task) . Given a setof requirements R and a set of constraints C (the recom-mendation knowledge base), the diagnosis task it to identifya minimal set ∆ of constraints (the diagnosis) that have tobe removed from R such that R - ∆ ∪ C is consistent.An example of a set of requirements for which no solu-tion can be identified is R = { r : usage = Scientific , r : eef-ficiency = high , r : maxprice = , r : country = Austria , r : mb = MBSilver , r : cpu = CPUD }. The recommenda-tion knowledge base induces two minimal conflict sets (CS)[10] in R which are CS : { r , r } and CS : { r , r }. Forthese conflict sets we have two alternative diagnoses whichare ∆ :{ r , r } and ∆ :{ r }. The pragmatics, for example,of ∆ is that at least r and r have to be adapted in order tobe able to find a solution. How to determine such diagnoseson the basis of a HSDAG (hitting set directed acyclic graph)is shown, for example, in [11]. PROD is assumed to be represented as a single constraint indisjunctive normal form where each conjunct is an item.
Approaches based on the construction of hitting setstypically rely on conflict detection [10; 11]. In interac-tive settings, where only preferred diagnoses (leading di-agnoses) should be presented, hitting set based approachestend to become too inefficient since conflict sets have tobe determined before a diagnosis can be presented [12;13]. This was the major motivation for the development ofthe F
AST D IAG algorithm [12; 13; 14], which is a divide-and-conquer based algorithm that enables the determinationof minimal diagnoses without the need of conflict determi-nation and HSDAG construction. This way of determiningminimal diagnoses can also be denoted as direct diagnosis since no conflict set determination is needed in this context.F
AST D IAG can be seen as an inverse Q
UICK XP LAIN [10]type algorithm which relies on the following basic principle(see Figure 2). Given, for example, a set R = { r , r , ..., r }and a diagnosis (see Definition 1) is contained in { r , r , r }(first part of the split), then there is no need of further eval-uating { r , r , r }, i.e., the latter set is consistent. The sim-ilarity to Q UICK XP LAIN is the following. If a minimalconflict is contained in { r , r , r } there is no need to fur-ther search for conflicts in { r , r , r } since the algorithmdetermines one minimal conflict set at a time. Both algo-rithms (F AST D IAG and Q
UICK XP LAIN ) rely on a total lex-icographical ordering [10; 12] which allows the determina-tion of preferred minimal diagnoses (minimal conflict sets).A minimal (preferred) diagnosis ∆ can be used as a basisfor the determination of corresponding repair actions, i.e.,concrete measures to change user requirements in R in afashion such that the resulting R’ is consistent with C. Definition 2 (Repair Task) . Given a set of requirements R= { r , r , ..., r k } inconsistent with the constraints in C anda corresponding diagnosis ∆ ⊆ R ( ∆ = { r l , ..., r o }), thecorresponding repair task is to determine an adaption A ={ r l ’, ..., r o ’} such that R - ∆ ∪ A is consistent with C.Figure 2: Divide-and-conquer principle of F
AST D IAG ( CS and CS are assumed to be conflict sets). The set of require-ments R = { r , ..., r } is split in the middle. If a diagnosis isalready contained in the first part of the split ( R - { r , r , r }is consistent), there is no need to further investigate the rightpart for further diagnosis elements. This way, half of thepotential diagnosis elements can be eliminated in one step(consistency check).In W EE V IS , repair actions are determined conform toDefinition 2. For each diagnosis ∆ determined by F AST -D IAG (currently, the first n=3 leading diagnoses are deter-mined – for details see [12]), the corresponding solutionsearch for R - ∆ ∪ C returns a set of alternative repairactions (represented as adaptation A). In the following, allproducts that satisfy R - ∆ ∪ A are shown to the user (seethe right hand side of Figure 3).In the current W EE V IS implementation, the total lex-icographical ordering is derived from the order in whicha user has entered his/her requirements. For example, ifigure 3: PC recommender UI (MediaWiki "Read" mode). If the user selects the item energystar on the right-hand side, adiagnosis with corresponding repair actions is depicted on the left-hand side. r : usage = Scientific has been entered before r : mb = MBSilver and r : cpu = CPUD then the underlying as-sumption is that r and r are of lower importance forthe user and thus have a higher probability of being partof a diagnosis. In our working example ∆ = { r , r }.The corresponding set of repair actions (solutions for R- ∆ ∪ C) is A = { r ’:mb=MBDiamond, r ’:cpu=CPUS}, i.e.,{ r , r , r , r , r , r } - { r , r } ∪ { r ’, r ’} is consistent.The item that satisfies R - ∆ ∪ A is {hw1} (see the firstentry in Figure 3). In a similar fashion, repair actions aredetermined for ∆ - the recommended item is {energystar}.The identified items (p) are finally ranked according to theirsupport value (see Formula 1). support ( p ) = repair actions in R (cid:48) requirements in R (1) Recommendation knowledge is often subject to change op-erations. Due to frequent changes it is important to supportquality assurance of recommendation knowledge. W EE V IS supports the definition and execution of test cases whichdefine the intended behavior of the recommender knowl-edge base. If some test cases become inconsistent with anew version of the knowledge base, the causes of the un-intended behavior must be identified. On a formal level a W EE V IS supports the definition of positive test cases (testcases that should be consistent with the knowledge base). recommendation knowledge base (RKB) diagnosis task canbe defined as follows (see Definition 3). Definition 3 (RKB Diagnosis Task) . Given a set C(the recommendation knowledge base) and a set T ={ t , t , ..., t q } of test cases t i , the corresponding diagnosistask is it to identify a minimal set ∆ of constraints (the di-agnosis) that have to be removed from C such that ∀ t i ∈ T : C − ∆ ∪ t i is consistent.An example test case which induces an inconsistencywith the constraints in C is t : usage = Scientific and cpu = CPUD and mb = MBSilver (see Figure 1). t inducestwo conflicts in the recommendation knowledge base whichare CS : ¬ ( usage = Scientific ∧ cpu = CPUD ) and CS : ¬ ( usage = Scientific ∧ mb = MBSilver ). In order to makeC consistent with t , both incompatibility constraints have tobe deleted from C, i.e., are part of the diagnosis ∆ .Similar to the diagnosis of inconsistent requirements, thehitting set based determination of diagnoses for inconsis-tent knowledge bases is shown in [11]. This approach relieson the construction of a HSDAG determined on the basisof minimal conflict sets provided by conflict detection algo-rithm such as Q UICK XP LAIN . Diagnoses are determined ina breadth-first fashion, i.e., minimal cardinality diagnosesof faulty constraints in C are returned first.In contrast to [11], W EE V IS includes a F AST D IAG basedapproach to knowledge base debugging that can also beapplied in interactive settings. In this case, diagnoses aresearched in C. In the case of requirements diagnosis, thetotal ordering of the requirements is related to user prefer-ences (in W EE V IS derived from the instantiation order ofigure 4: PC recommender knowledge base: result of the diagnosis process presented in W EE V IS .variables). Total orderings of constraints in the context ofknowledge base diagnosis are determined using criteria dif-ferent from the diagnosis of inconsistent requirements, forexample, age of constraints, frequency of quality assurance,and structural constraint complexity (see [15]). An exam-ple screenshot of the W EE V IS diagnosis presentation is de-picted in Figure 4. Diagnosing Inconsistent Requirements . Junker [10] intro-duced the Q
UICK XP LAIN algorithm which is a divide-and-conquer based approach to the determination of min-imal conflict sets (one conflict set at a time). Combin-ing Q
UICK XP LAIN with the hitting set directed acyclicgraph (HSDAG) algorithm [7] allows for the calculationof the complete set of minimal conflicts. O’Sullivan etal. [16] show how to determine representative explana-tions (diagnoses) which fulfill the requirement that minimalsubsets ∆ S of the complete set of diagnoses ∆ C shouldbe determined that fulfill the criteria that if a constraint c i is contained in a diagnosis of ∆ C it must also be partof at least one diagnosis in ∆ S . Felfernig et al. [17;18] show how to integrate similarity metrics, utility-, andprobability-based approaches to the determination of lead-ing diagnoses on the basis HSDAG-based search.Felfernig and Schubert [13] introduce F LEX D IAG whichis a top-down version of F
AST D IAG allowing a kind ofanytime diagnosis due to the fact that diagnosis granularity(size of constraints regarded as one component in the diag-nosis process) can be parametrized. Felfernig et al. [12;13] introduce the F
AST D IAG algorithm that allows for amore efficient determination of diagnoses due to the fact thethere is no need for determining conflict sets (= direct di-agnosis ). F
AST D IAG is a Q
UICK XP LAIN style algorithmthat follows a divide-and-conquer approach for the determi-nation of minimal diagnoses. Note that in contrast to tradi-tional HSDAG based approaches, F
AST D IAG does not focuson the determination of minimal cardinality but preferredminimal diagnoses. A major issue for future work will bethe development of diagnosis algorithms that are capable ofperforming intra-constraint debugging an thus help to betterfocus on the sources of inconsistencies. F
AST D IAG is notrestricted to the application in knowledge-based recommen- dation scenarios but generally applicable in consistency-based settings [19]. For example, the same principles can beapplied in knowledge-based configuration [20; 21; 22]. Fur-ther approaches to the determination of diagnoses for incon-sistent knowledge bases can be found, for example, in [23;24; 25; 26; 27].
Knowledge Base Maintenance . The application of model-based diagnosis for the debugging of inconsistent constraintsets was first presented in [28]. Felfernig et al. [11] showhow to exploit test cases for the induction of conflict sets inknowledge bases which are then resolved on the basis of ahitting set based approach. In the line of the work of [12;13] the performance of knowledge debugging can be im-proved on the basis of F
AST D IAG . A detailed evaluationof the performance gains of F
AST D IAG in the context ofknowledge base debugging is within the focus of our futurework. A detailed comparison between the performance ofF
AST D IAG and conflict-driven diagnosis of inconsistent re-quirements can be found, for example, in [12].Identifying redundant constraints is an additional issuein the context of knowledge base development and main-tenance. Redundant constraints can deteriorate runtime per-formance and also be the cause of additional overheads indevelopment and maintenance operations [29]. Redundancydetection can be based on Q
UICK XP LAIN especially in thecase of an increasing number of redundant constraints. For adetailed discussion of alternative algorithms for redundancydetection in knowledge bases we refer to [29]. A major fo-cus of our future research will be the development of anintra-constraint redundancy detection, i.e., it will be possi-ble to identify redundant subexpressions.
In this paper we provide an overview of the W EE V IS en-vironment with a special focus on the integrated diagno-sis support. Diagnosis techniques integrated in W EE V IS are the result of research in model-based diagnosis witha special focus on divide-and-conquer based (direct) algo-rithms that make diagnosis search more efficient in the casethat leading diagnoses are required. W EE V IS is a publiclyavailable MediaWiki-based environment for developing andmaintaining constraint-based recommender applications. eferences [1] A. Felfernig, E. Teppan, and B. Gula. Knowledge-based recommender technologies for marketing andsales. International Journal of Pattern Recognitionand Artificial Intelligence , 21(2):333–354, 2006. Spe-cial issue of Personalization Techniques for Recom-mender Systems and Intelligent User Interfaces.[2] A. Felfernig and R. Burke. Constraint-based recom-mender systems: Technologies and research issues. In
ACM ICEC 08 , pages 17–26, 2008.[3] A. Felfernig, G. Friedrich, D. Jannach, and M. Zanker.An Integrated Environment for the Development ofKnowledge-based Recommender Applications.
Intl.Journal of Electronic Commerce (IJEC) , 11(2):11–34,2006.[4] K. Szabo A. Felfernig, K. Isak and P. Zachar. TheVITA Financial Services Sales Support Environment.In
AAAI/IAAI 2007 , pages 1692–1699, Vancouver,Canada, 2007.[5] J. Konstan, B. Miller, D. Maltz, J. Herlocker, L. Gor-don, and J. Riedl. Grouplens: applying collaborativefiltering to usenet news full text.
Communications ofthe ACM , 40(3):77–87, 1997.[6] M. Pazzani and D. Billsus. Learning and revising userprofiles: the identification of interesting websites.
Ma-chine Learning , 27(3):313–331, 1997.[7] R. Reiter. A theory of diagnosis from first principles.
AI Journal , 32(1):57–95, 1987.[8] A. Mackworth. Consistency in Networks of Relations.
AI Journal , 8(1):99–118, 1977.[9] A. Felfernig, S. Schippel, G. Leitner, F. Reinfrank,K. Isak, M. Mandl, P. Blazek, and G. Ninaus. Au-tomated Repair of Scoring Rules in Constraint-basedRecommender Systems.
AICom , 26(2):15–27, 2013.[10] Ulrich Junker. QUICKXPLAIN: preferred explana-tions and relaxations for over-constrained problems. In
AAAI’04 , pages 167–172, 2004.[11] A. Felfernig, G. Friedrich, D. Jannach, and M. Stumpt-ner. Consistency-based diagnosis of configurationknowledge bases.
AI Journal , 152(2):213–234, 2004.[12] A. Felfernig, M. Schubert, and C. Zehentner. An ef-ficient diagnosis algorithm for inconsistent constraintsets.
AIEDAM , 26(1):53–62, 2012.[13] A. Felfernig and M. Schubert. Fastdiag: A diagnosisalgorithm for inconsistent constraint sets. In
DX 2010 ,pages 31–38, 2010.[14] J. Galindo A. Felfernig, D. Benavides and F. Rein-frank. Towards Anomaly Explanation in FeatureModels. In
Workshop on Configuration , pages 117–124, Vienna, Austria, 2013.[15] A. Felfernig, S. Reiterer, M. Stettinger, F. Reinfrank,M. Jeran, and G. Ninaus. Recommender Systems forConfiguration Knowledge Engineering. In
Workshopon Configuration , pages 51–54, 2013.[16] B. O’Sullivan, A. Papadopoulos, B. Faltings, andP. Pu. Representative explanations for over-constrai-ned problems. In
AAAI’07 , pages 323–328, 2007. [17] A. Felfernig, M. Mairitsch, M. Mandl, M. Schubert,and E. Teppan. Utility-based repair of inconsistent re-quirements. In
Proceedings of IEA/AIE’09 , volume5579 of
Springer Lecture Notes in Computer Science ,pages 162–171, Tainan, Taiwan, 2009.[18] A. Felfernig, M. Schubert, and S. Reiterer. Person-alized Diagnosis for Over-Constrained Problems. In
IJCAI 2013 , pages 1990–1996, 2013.[19] L. Hotz, A. Felfernig, M. Stumptner, A. Ryabokon,C. Bagley, and K. Wolter. Configuration KnowledgeRepresentation & Reasoning. In A. Felfernig, L. Hotz,C. Bagley, and J. Tiihonen, editors,
Knowledge-basedConfiguration – From Research to Business Cases ,chapter 6, pages 59–96. Morgan Kaufmann Publish-ers, 2013.[20] M. Stumptner. An Overview of Knowledge-basedConfiguration.
AI Communications , 10(2):111–126,1997.[21] Daniel Sabin and Reiner Weigel. Product Configura-tion Frameworks - A Survey.
IEEE Intelligent Systems ,13(4):42–49, 1998.[22] A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen.
Knowledge-based Configuration: From Research toBusiness Cases . Elsevier/Morgan Kaufmann, 2014.[23] W. Liu K. McAreavey and P. Miller. Computationalapproaches to finding and measuring inconsistency inarbitrary knowledge bases.
International Journal ofApproximate Reasoning , pages 1–35, 2014.[24] J. Marques-Silva and A. Previti. On Computing Pre-ferred MUSes and MCSes. In
SAT 2014 , pages 58–74,2014.[25] M. Janota A. Previti J. Marques-Silva, F. Heras andA. Belov. On computing minimal correction subsets.In
IJCAI’2013 , pages 615–622, 2013.[26] A. Previti Y. Malitsky, B. O’Sullivan and J. Marques-Silva. A Portfolio Approach to Enumerating Mini-mal Correction Subsets for Satisfiability Problems. In
CPAIOR’2014 , pages 368–376, 2014.[27] R. Walter, C. Zengler, and W. Küchlin. Applications ofMaxSAT in Automotive Configuration. In
Workshopon Configuration , pages 21–28, Vienna, Austria, 2013.[28] R. Bakker, F. Dikker, F. Tempelman, and P. Wog-mim. Diagnosing and solving over-determined con-straint satisfaction problems. In
IJCAI 1993 , pages276–281, 1993.[29] A. Felfernig, C. Zehentner, and P. Blazek. CoreDiag:Eliminating redundancy in constraint sets. In