Implementing WordNet Measures of Lexical Semantic Similarity in a Fuzzy Logic Programming System
UUnder consideration for publication in Theory and Practice of Logic Programming (TPLP) Implementing WordNet Measures of Lexical SemanticSimilarity in a Fuzzy Logic Programming System ∗ PASCUAL JULI ´AN-IRANZO
Dept. of Information Technologies and Systems, University of Castilla-La Mancha,13071 Ciudad Real, Spain ( e-mail: [email protected] ) FERNANDO S ´AENZ-P ´EREZ
Faculty of Computer Science, Complutense University of Madrid,28040 Madrid, Spain ( e-mail: [email protected] ) submitted 25 November 2019; revised 25 November 2020; accepted – – – Abstract
This paper introduces techniques to integrate WordNet into a Fuzzy Logic Programming system. SinceWordNet relates words but does not give graded information on the relation between them, we have im-plemented standard similarity measures and new directives allowing the proximity equations linking twowords to be generated with an approximation degree. Proximity equations are the key syntactic structureswhich, in addition to a weak unification algorithm, make a flexible query-answering process possible inthis kind of programming language. This addition widens the scope of Fuzzy Logic Programming, allowingcertain forms of lexical reasoning, and reinforcing Natural Language Processing applications.
Under consideration in Theory and Practice of Logic Programming (TPLP)KEYWORDS : Fuzzy Logic Programming, WordNet, Proximity Equations, System Implementation
Fuzzy Logic Programming (Lee 1972) integrates concepts coming from fuzzy logic (Zadeh 1965)into logic programming (van Emden and Kowalski 1976) in order to deal with the essentialvagueness of some problems by using declarative techniques. In recent years there has beenrenewed interest in this field, involving multiple lines of work. When the fuzzy unification algo-rithm is weakened using a similarity relation (i.e., a reflexive, symmetric, transitive, fuzzy binaryrelation) the approach is usually called
Similarity-based Logic Programming (Fontana and For-mato 1999; Fontana and Formato 2002; Loia et al. 2001; Sessa 2002).We have extended Similarity-based Logic Programming by introducing new theoretical con-cepts and developing two fuzzy logic programming systems:
Bousi ∼ Prolog ( BPL for short) ∗ Work partially funded by the State Research Agency (AEI) of the Spanish Ministry of Science and Innovation undergrant PID2019-104735RB-C42 (SAFER), by the Spanish Ministry of Economy and Competitiveness, under the grantsTIN2016-76843-C4-2-R (MERINET), TIN2017-86217-R (CAVI-ART-2), and by the Comunidad de Madrid, underthe grant S2018/TCS-4339 (BLOQUES-CM), co-funded by EIE Funds of the European Union. a r X i v : . [ c s . P L ] J a n Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez %% PROXIMITY EQUATIONSancestor~ascendant=1.0. ancestor~progenitor=0.9.%% FACTSfather(abraham,isaac). father(isaac,esau). father(isaac,jacob).mother(sara,isaac). mother(rebeca,jacob). mother(rebeca,esau).%% RULESdirect_ancestor(X,Y) :- father(X,Y); mother(X,Y).ancestor(X,Z) :- direct_ancestor(X,Z).ancestor(X,Z) :- direct_ancestor(X,Y), ancestor(Y,Z).
Fig. 1. A BPL program fragment.(Rubio-Manzano and Juli´an-Iranzo 2014; Juli´an-Iranzo and Rubio-Manzano 2017) and
Fuzzy-DES (Juli´an-Iranzo and S´aenz-P´erez 2017; Juli´an-Iranzo and S´aenz-P´erez 2018b). Their syntaxis based on the clausal form, and they embody a
Weak SLD (WSLD) resolution operational se-mantics, which uses a fuzzy unification algorithm based on the concept of proximity relation (i.e., a fuzzy binary relation supporting unification that, although reflexive and symmetric, is notnecessarily transitive) (Juli´an-Iranzo and Rubio-Manzano 2015; Juli´an-Iranzo and S´aenz-P´erez2018a). A proximity relation is defined by proximity equations , denoted by a ∼ b = α , whoseintuitive reading is that two constants (either n -ary function symbols or n -ary predicate symbols), a and b , are approximate or similar with a certain degree α .For instance, assume a deductive database that stores information about people and their familyrelationships encoded using the Bousi ∼ Prolog language (see Fig. 1). In a
Prolog system (withoutproximity equations), asking about the progenitors of isaac with the query progenitor(X,isaac) produces no answer. However,
Bousi ∼ Prolog answers
X=abraham with 0.9 and
X=sara with0.9 thanks to its proximity-based unification algorithm. Since we have specified that progenitor is close to ancestor with degree 0.9, these two terms can “weakly” unify with approximationdegree 0.9, leading to a refutation.Here, the proximity equations are axiomatically given by the programmer. It would be inter-esting if the system could provide assistance through its connection to a lexical resource suchas WordNet (Fellbaum 1998; Fellbaum 2006; Miller 1995). This study, therefore, deals with thegeneration of the set of proximity equations both automatically and with a minimal interven-tion by the programmer. However, the motivation for integrating WordNet into our logic systemsgoes beyond this simple help function. We provide the Prolog implementation, wn connect , toconnect to WordNet with a number of similarity measures and convenient predicates to be usedeither in isolation or integrated into
Bousi ∼ Prolog , allowing reasoning with linguistic terms. Un-like Distributional Semantic Models such as Word Embeddings or other statistical approaches,WordNet-based techniques do not require training and facilitate explainability (Santus et al.2018).The usefulness of this proposal lies in its inclusion of applications such as text mining in in-formation retrieval, text classification, and even sentiment analysis (Allahyari et al. 2017; Baeza-Yates and Ribeiro-Neto 2011; Serrano-Guerrero et al. 2015) (see also 6.2). mplementing WordNet Lexical Similarity Measures WordNet is a lexical English language database. Words of the same syntactic category are groupedinto sets of synonyms called synsets . Roughly speaking, the words of a synset have the samemeaning in a specific context and they represent a concept (or word sense). Each synset has a synset ID . Because a word has different senses (meanings), it can belong to different synsets.WordNet is structured as a semantic net where words are interlinked by lexical relations, andsynsets by semantic relations. Synonymy and antonymy are the major lexical relations. Semanticrelations serve to build knowledge structures (i.e., networks of synsets –concepts–). Nouns, aswell as verbs, are interconnected by the hyponymy relation (IS-A relation), which links specificconcepts to more general ones. Hypernymy is the opposite relation, that is, a hypernym is a wordwhose meaning includes a group of other words. Both relations are transitive. Note also that bothnouns and verbs are organized as separate hierarchical structures.WordNet can be accessed either via a web interface or locally. There exists a WordNet 3.0database version released by Eric Kafe which can be found at: https://github.com/ekaf/wordnet-prolog . The information stored in WordNet is provided as a collection of
Prolog files. Each file contains the definition of what is called an operator , corresponding to a WordNetrelation. Files are named wn < operator >.pl , where < operator > is the name of a specific op-eration (relation). Therefore, each WordNet relation is represented by a Prolog predicate which isstored in a separate file and defined by a set of
Prolog facts. The specifications of these predicatesare detailed in (Fellbaum and et al. wn s.pl contains all the information about words stored in WordNet. It defines the s operator, which has an entry for each word. The structure of the s operator is s(Synset id,W num,Word,Ss type,Sense number,Tag count) , where W num , if given, indicates whichword in the synset is being referred to. The words in a synset are numbered serially, starting with1. The third argument is the word itself (which is represented by a
Prolog atom). The
Ss type parameter is a one character code indicating the synset type: n (noun); v (verb); a (adjective); s (satellite adjective) and r (adverb). The Sense number parameter specifies the sense of theword, within the part of speech encoded in the
Synset id . The higher the sense number, theless common the word. Finally, the
Tag count indicates the number of times the word sensewas found in the sense-tagged text corpus of Semantic Concordances (Miller et al. 1993), whichwas generated from the Brown Corpus (Francis and Kucera 1979), using WordNet as a lexicon.The Brown Corpus was inspected word by word, including sense-tags for each one. A highertag count number means that the word is more common than others with a lower tag count. InSection 5.4 we illustrate the meaning of some of these parameters through an example.The file wn hyp.pl stores hypernymy relations in the binary predicate hyp(synset ID1,synset ID2) specifying that the second synset is a hypernym of the first synset. This semanticrelation only holds for nouns and verbs. Because hyponymy is the inverse relation of hypernymy,the operator hyp also specifies that the first synset is a hyponym of the second synset.
WordNet relates words but does not give their degree of relationship. Measuring lexical seman-tic similarity has many applications for Natural Language Processing (NLP), and its integra-tion into a fuzzy logic programming system such as
Bousi ∼ Prolog is appropriate because of
Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez its proximity-based operational semantics. The syntax of our language uses symbols (words)that are endowed with a fuzzy semantics via proximity equations. We are therefore interested intechniques for measuring the similarity degree between words to facilitate the construction ofproximity equations with linguistic criteria. Semantic similarity quantifies how alike two wordsare (more precisely: how similar the concepts they denote are).Similarity measures are limited to noun pairs and verb pairs because WordNet organizes nounsand verbs into hyponymy/hypernymy-based hierarchies of concepts (synsets).Although a large number of measures of semantic relatedness and similarity have been pro-posed (Budanitsky and Hirst 2006), they are only implemented by a limited number of tools.WordNet::Similarity (Pedersen et al. 2004) is perhaps the most prominent. This tool has threesimilarity measures based on counting edges between concepts (PATH, WUP (Wu and Palmer1994) and LCH (Leacock and Chodorow 1998)), and another three based on information content(RES (Resnik 1995), JCN (Jiang and Conrath 1997) and LIN (Lin 1998)).Table 1 summarizes some features of these measures, with the measure name in the first col-umn, its type (either counting Edges Based –EB– or Information Content –IC–) in the second,its description in the third, and its range in the last. In order to understand the description of sim-ilarity measures accurately, we introduce the following standard definitions and notations usedwhen working in the framework of WordNet: • We differentiate between “words” and “concepts”. We use the term “ word ” as shorthandfor “word form,” and the term “ concept ” (i.e., “synset”) to refer to a specific sense or wordmeaning. Words will be denoted by the letter w , and concepts by the letter c , possibly withsubscripts. A concept can also be seen as a word w of type t with sense s and denoted by w : t : s (which we often call word term or pattern ). • Similarity measures use so-called HyperTrees (Hypernym Trees). These are IS-A hier-archies which are a consequence of the hyponymy relation between concepts. Despitetheir name, HyperTrees are not really trees because a concept can be linked to a hyper-nym concept through different paths. Moreover, in practice, the branches of a HyperTreeare manipulated independently as hypernym chains. Given a HyperTree, the length of theshortest path from synset c to synset c is denoted by len ( c , c ) . The depth of a node c isthe length of the shortest path from the global root to c , i.e., depth ( c ) = len ( root , c ) . The“global root” is a virtual root that we introduce into the IS-A hierarchy of either nouns orverbs for technical reasons. • The least common subsumer (LCS) of two concepts c and c is the most specific conceptthey share as an ancestor. It is denoted by lcs ( c , c ) . An example illustrating the notion ofLCS is shown in Section 5.4.RES, JCN and LIN measures are based on the notion of Information Content (IC) (Resnik1995). For a concept c , IC ( c ) = − ln ( p ( c )) , where p is the probability of finding an instance ofthe concept c in a corpus. In our case, this probability is measured in terms of a relative frequencyof use (or frequency count ) of the concept c stored in WordNet, which is a measure of the number Note that “lexical semantic relatedness” is a broad concept that subsumes “lexical semantic similarity”. There aremany different forms in which two words can be related without being similar: for instance, “car” and “petrol” areclosely related, but they are not similar. From a pragmatic point of view, and to distinguish one type of measure fromanother, it is usual to reserve the name “relatedness” for those that measure features other than similarity. mplementing WordNet Lexical Similarity Measures
Measure Type Description Range
PATH EB. sim
PATH ( c , c ) = / len ( c , c ) [ , ] WUP EB. sim
WUP ( c , c ) = × depth ( lcs ( c , c )) Depth ( c )+ Depth ( c ) [ , ] LCH EB. sim
LCH ( c , c ) = − log ( len ( c , c ) × max { depth ( c ) | c ∈ WordNet } ) [ , ∞ ] RES IC sim
RES ( c , c ) = IC ( lcs ( c , c )) [ , ∞ ] JCN IC sim
JCN ( c , c ) = IC ( c )+ IC ( c ) − × IC ( lcs ( c , c )) [ , ∞ ] LIN IC sim
LIN ( c , c ) = × IC ( lcs ( c , c )) IC ( c )+ IC ( c ) [ , ] of times that it occurs in the corpus of Semantic Concordances. Specifically, p ( c ) = Frequency ( c ) / Frequency ( Root ) where Frequency ( c ) is computed by adding the Tag count of the concepts subsumed by theconcept c and Root is the concept (virtual or not) on the top of the concept hierarchy.
This section recalls and extends some formal background of
BPL and its relation to lexicalsimilarity. Given a universe U , proximity equations extensionally define a binary fuzzy relation R : U × U −→ [ , ] . A λ -cut is a user-defined threshold such that R λ = {(cid:104) x , y (cid:105) | R ( x , y ) ≥ λ } . Afuzzy relation can have some properties attached, for any e , e , e , e ∈ U : Reflexive ( R ( e , e ) = R ( e , e ) = R ( e , e ) )), and (cid:52) -Transitive ( R ( e , e ) ≥ R ( e , e ) (cid:52) R ( e , e ) ), wherethe operator (cid:52) is an arbitrary t-norm. A fuzzy relation with the reflexive and symmetric propertiesis a proximity relation. If in addition it has the (cid:52) -transitive property, it is a similarity relation. A weak unification of terms builds upon the notion of weak unifier of level λ for two expres-sions E and E with respect to R (or λ -unifier): a substitution θ such that R ( E θ , E θ ) ≥ λ ,which is the unification degree of E and E with respect to θ and R . There are several weakunification algorithms (Juli´an-Iranzo and S´aenz-P´erez 2018a) based on this notion and on the proximity-based unification relation ⇒ , which defines a transition system (based on (Martelliand Montanari 1982)). This relation, applied to a set of unification problems { E i ≈ E (cid:48) i | ≤ i ≤ n } can yield either a successful or a failed sequence of transition steps. In the first case, both asuccessful substitution and a unification degree are obtained (detailed in, e.g., (Juli´an-Iranzoand S´aenz-P´erez 2018a)). The notion of weak most general unifier (wmgu) θ between two ex-pressions, denoted by wmgu λ R ( E , E ) , is defined as a λ -unifier of E and E such that there isno other λ -unifier which is more general than θ . Unlike in the classical case, the wmgu is notunique. However, our weak unification algorithm computes a representative wmgu with approx-imation degree greater than or equal to any other wmgu.Given a fuzzy logic program Π with rules (cid:104) ( A ← Q ) ; δ (cid:105) , where A is an atomic formula, Q iseither empty or a conjunction of n ≥ B i , and δ is the degree of the rule,an operational semantics can be defined as a transition system with a transition relation ⇒ WSLD ,which, in particular, includes the (transition) rule: Lexical semantic similarity and a similarity relation are two different concepts. The first simply provides a degree ofsimilarity between words, but it is not a similarity relation in the sense defined above, with the reflexive, symmetricand (cid:52) -transitive properties. On the practical side, we use the WordNet similarity measures to obtain the approximationdegree that we use when automatically constructing the proximity equations.
Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez (cid:104) ( ← A (cid:48) ∧ Q (cid:48) ) , θ , α (cid:105) ⇒ WSLD (cid:104) ( ← Q ∧ Q (cid:48) ) σ , θ σ , δ (cid:52) β (cid:52) α (cid:105) if (cid:104) ( A ← Q ) ; δ (cid:105) ∈ Π , σ = wmgu λ R ( A , A (cid:48) ) (cid:54) = f ail , R ( A σ , A (cid:48) σ ) = β ≥ λ , and ( δ (cid:52) β (cid:52) α ) ≥ λ .A fuzzy logic program Π is translated into a logic program by: linearizing heads, making theweak unification explicit, and explicitly computing the approximation degree. Essentially, givena graded rule (cid:104) p ( t n ) ← Q ; δ (cid:105) , for each R ( p , q ) = α ∈ Π with α ≥ λ , generate the clause: q ( x n ) ← ( δ (cid:52) α ) ∧ x ≈ t ∧ . . . ∧ x n ≈ t n ∧ Q where ≈ is the weak unification operator, t i are terms, x i are variables, and δ (cid:52) α abbreviates thegoal δ (cid:52) α ≥ λ .We assume 3-arity predicates for lexical similarity measures with the pattern ( w : t : s , w : t : s , d ) , where w i : t i : s i are word terms and d ∈ ( , ] is a normalized semantic similarity degree.Any element e in the semantics of a lexical similarity measure m can be used to generate a prox-imity equation R ( w , w ) = d defining R . Depending on the fuzzy relation we decide to workwith, Bousi ∼ Prolog generates several types of closure starting from the proximity equationsdefining R . Specifically, since a similarity relation requires all of the three properties (in partic-ular, transitivity), its intension is its reflexive, symmetric, (cid:52) -transitive closure. This allows forboth manual and automatic generation of proximity equations relating similar words, includingwords that are not directly related by m (cf. Section 5).Section 5 will show how to integrate WordNet and the aforementioned lexical semantic simi-larity measures into the state-of-the-art fuzzy logic programming system Bousi ∼ Prolog . ∼ Prolog
Bousi ∼ Prolog comprises three subsystems with a total of nine modules. The wn-connect sub-system provides the basis for the connection between WordNet and the Bousi ∼ Prolog system. wn-connect is a software application in itself with ten Prolog modules, which implements pred-icates for managing synsets, hypernyms and hyponyms, giving support to the wn sim measures and wn ic measures modules which, in addition, implement the standard similarity measuresdefined in Section 3. We now offer a summary of the base modules: • The wn synsets module implements predicates to retrieve information about words andsynsets stored in WordNet. It uses the wn module implemented by Jan Wielemaker whichexploits SWI-Prolog demand-loading and Quick Load Files (QLF) for ‘just-in-time’ fastloading. • The wn hypernyms module implements predicates to retrieve information about hyper-nyms of a concept (synset). It uses the modules wn synsets and wn utilities . Notably,the predicate wn hypernyms/2 returns a list
List SynSet HyperNym of hypernyms (assynset identifiers) for a word term
Hyponym . • The wn hyponyms module implements predicates to retrieve information about hyponymsof a concept (synset). Remarkably, the predicate wn gen all hyponyms of/2 generatesall the hyponyms of a concept (
Synset ID ), and is especially useful for computing theinformation content of a concept. https://dectau.uclm.es/bousi-prolog (Juli´an-Iranzo and S´aenz-P´erez 2019) gives a more detailed description of this subsystem from the user’s point ofview. Also, https://dectau.uclm.es/bousi-prolog/2018/08/27/applications/ supplies the source fileswith the code and detailed comments of its implementation. https://github.com/JanWielemaker/wordnet . mplementing WordNet Lexical Similarity Measures A first step for a more ambitious goal is to automatically extract semantic similarity informationfrom WordNet IS-A hierarchies, and other attributes as the frequency of use as explained be-fore in Section 3. Here, we describe in broad strokes the implementation of similarity measuresbased on edge-counting (module wn sim measures ) and some insights about those based oninformation content (module wn ic measures ).To a greater or lesser extent, all edge-counting similarity measures are based on the compu-tation of the LCS of two words (more accurately, concepts). The predicate wn sim measures:lcs/6 returns the LCS of two words
Word1 and
Word2 , and also measures depth in their re-spective HyperTrees. Roughly speaking, it computes the HyperTrees of
Word1 and
Word2 andcompares them from their roots, returning the synset ID previous to the first mismatch (whichis the LCS of both concepts). Additionally, the predicate lcs/6 returns the depths for LCS,
Word1 and
Word2 for reasons of efficiency: we want to go through the hypernym lists only once,so these quantities are calculated when computing the LCS.The computation of the hypernyms of a concept is carried out by the predicate wn hypernyms:hypernym chain/2 , which computes a list (
SynSet HyperNyms ) of synset IDs designatingthe hypernyms of a concept (
SynSet Hyponym ). It thus computes a HyperTree that will be usedin the former comparison to compute the LCS.Once the depths of the LCS and the words to be compared are known, it is easy to computethe relationship degree between them by following the guidelines given in Section 3. For ex-ample, the WUP measure is implemented by the predicate wn wup/3 , which takes two concepts(expressed as word terms of the form
Word:SS type:Sense num ) and returns the degree of sim-ilarity between them. It relies on the private predicate wup/3 that calls lcs/6 to generate andinspect a pair of HyperTrees associated to
Word1 and
Word2 , and obtains the similarity degreebetween both words (according to that pair of HyperTrees). Because a concept can have morethan one HyperTree, several pairs of HyperTrees are possibly considered, and a list of similaritydegrees is obtained for each of these pairs of HyperTrees. Finally, the maximum degree in thelist is selected as a result.Regarding similarity measures based on the information content, the key idea lies in the im-plementation of the notion of frequency of use. The operator wn s/6 stores information on howcommon a word is. The tag number indicates the number of times the word was found in a textcorpus: the higher the number, the more common the word is. This parameter can therefore beemployed to obtain the use of a word and, summing the tag number of all words in a synset, thespecific use associated to a whole synset (i.e., to a concept) can be obtained. Then, the frequencyof use of a concept is obtained by adding the “synset tag num” of all concepts subsumed by thatconcept.As explained in Section 3, the information content of a concept is a function of the ratiobetween the frequency of use of that concept and the frequency of use of the root concept of thehierarchy. Finally, the information-content-based measures are computed as shown in Table 1 forspecific predicates. Note that we have taken the option of smoothing the frequencies of use witha value of 0, which we substitute for a very small number close to 0. So, some relationship valuesdo not exactly match those that would be obtained when using tools like wordnet::similarity (Pedersen et al. 2004).Finally, 6.1 includes a performance comparison between the similarity measures implementedfor
Bousi ∼ Prolog and other systems.
Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez
Bousi ∼ Prolog can load both ontologies (consisting of proximity equations) and fuzzy logic pro-grams (with fuzzy logic rules and possibly proximity equations). Thus, it would be of interestto use the similarity measures implemented in the last section to automatically construct suchontologies.In order to define the semantic similarity between selected concepts, we provide a
Bousi ∼ Prolog directive for automatically generating the proximity equations which define an ontology: • :-wn gen prox equations(+Msr, +LL of Pats) where Msr is the similarity measure which can be any of: path , wup , lch , res , jcn and lin . The second argument LL of Pats is a list for which each element is another listcontaining the patterns that must be related by proximity equations. The pattern can beeither a word or the structure
Word:Type:Sense , where
Word is the word,
Type is itstype (either n for noun or v for verb), and Sense is the sense number in its synset. Notethat, because similarity measures only relate nouns with nouns, and verbs with verbs, thewords of a set must be of the same part of speech. If the pattern is simply a word, then asense number of 1 is assumed, and its type is made to match all other words in the samelist.An example of this directive is: :-wn_gen_prox_equations(wup,[[man,human,person],[grain:n:8,wheat:n:2]]).
In this case, as only words are provided in the first list, the sense number is 1, and their typesare equal by pairs (nouns for these words). The second list explicitly specifies the pattern of eachword to be related. Then, excluding, for reasons of simplicity, reflexive and symmetric entries,the following proximity equations are generated for a lambda cut of 0: sim(man, human, 1, 0.56).sim(man, person, 1, 0.8888888888888888).sim(person, man, 1, 0.8888888888888888).sim(human, person, 1, 0.6086956521739131).sim(grain, wheat, 0, 0.2608695652173913).
Note that there are two blocks, numbered with 1 for the first four equations, and with 0 forthe last one. Clearly, words in the first list are not made to be related to those in the second list,and therefore they must occur in different blocks. In addition, proximity equations are generatedonly for the words stored in WordNet.Another form of this directive automatically builds an ontology in terms of the tokens in theBPL program by including auto in its second argument. Only the symbols that occur in a pro-gram are related, because it would not be practical to relate the symbols of the program with allthose that occur in WordNet.
Bousi ∼ Prolog processes a file (either a program or an ontology) with the load command ld file of the BPL
Shell module (named bplShell ), where its argument is the name of the file toload (with default extension bpl ). Upon execution of this command, a source file ( file .bpl ) isparsed, compiled to Prolog ( file .tpl ), and consulted.When parsing a directive :-wn gen prox equations , it is first checked for validity, and then mplementing WordNet Lexical Similarity Measures
Prolog file with the proximity equations corresponding to the pairs formedwith the symbols derived from its arguments. As explained, there are two cases for this directive,and they are handled in a different way: • Explicit indication of words to be related.Here, the proximity equations can be directly generated from each list of words, kept inthe memory (as asserted
Prolog facts) and outputted to the translated program in the .tpl file at a later stage. The procedure is as would be expected: for each pair of differentwords W1 and W2 in a list, generate the proximity equation sim(W1,W2,D) , where D isthe approximation degree for the normalized measure given as the first parameter of thedirective. Normalization is required because measures are generally not on the interval ( , ] which is the range for proximity equations. • Automatic generation of proximity equations.This case is different from the former because, when processing the directive, the rulesin the program have not yet been parsed, so their tokens are not available. It is thereforeprocessed after parsing the remaining program, by performing a syntactic analysis in orderto extract the sets of constant, functor and predicate identifiers and adding the resultingproximity equations for each separate set of tokens (with the same shape as in the othercase) to the memory.The directives that generate proximity equations are based on the private predicate gen proxequation . It generates a proximity equation sim(Word1, Word2, NormalizedDegree ) interms of a given measure (
Measure ) and a pair of words, which can be completely specified witheither a pattern or only with its syntactic form as plain words. In this last case, their first sensenumber is selected and the same word type is enforced.
The wn connect subsystem must be made visible before using the built-ins (public predicates)defined in its modules. In
Bousi ∼ Prolog , WordNet and a wide repertoire of built-in predicateswhich are implemented by the wn connect modules can be accessed either by the directive :-wn connect in a program or interactively with ensure loaded(wn(wn connect)) at thecommand prompt.Nearly all the predicates implemented in the wn-connect subsystem are crisp, returning thetop approximation degree. For instance, the predicate wn word info/1 merges the informationprovided by the predicate wn s/6 (which stores information about a synset) and wn g/2 (whichcontains an explanation/definition of the concept represented by the synset and example sen-tences). Figure 2 shows the first answer to the query wn word info(cat) . This is telling us thatthe first sense (Sense number = 1) of the word form “cat” in the part of speech of nouns (Synsettype = n —i.e., a noun—) refers to the concept: “feline mammal usually having thick soft furand no ability to roar etc.”. There are six more answers for noun-related senses and two more forverb-related senses.However, the binary similarity predicates ( wn path/2 , wn wup/2 , wn lch/2 , etc.) are fuzzypredicates that return the similarity degree of two concepts. We also maintain ternary predicatesavailable to programmers, since they provide direct access to the approximation degree D , whichcan be very useful for its explicit handling. Thanks to the repertoire of built-in predicates imple-mented in the wn-connect subsystem, the user of the BPL system can extract information from0
Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez ?- wn_word_info(cat).INFORMATION ABOUT THE WORD ’cat’ :Synset_id = 102121620Word Order num. = 1Synset type (n:NOUN, v:VERB, a:ADJ., s:ADJ. SAT., r:ADV.) = nSense number = 1Tag_count = 18-----------Gloss:feline mammal usually having thick soft fur and no ability to roar: domesticcats; wildcats-----------true
Fig. 2. A query for obtaining all relevant information about a word (e.g. the word “cat”)Fig. 3. Hypernyms of the word god (all senses)WordNet, deepening into the structure of the relationships between its linguistic terms. This be-comes especially evident for the predicate wn display graph hypernyms/1 . Figure 3 showsits outcome for the hypernym hierarchy of all senses of the word god . Moreover, with these built-in predicates, a certain form of linguistic reasoning is possible. Forexample, the predicate wn lcs/2 , which computes the LCS of a set of concepts, can help to ob-tain the most specific generalization of a set of concepts and to contribute to knowledge discovery. In Figure 3, each node draws the representative word of the respective synset (i.e., those with
W num equal to one).This figure also illustrates how a concept can be linked to a hypernym concept through different paths in (a subset of)the WordNet IS-A hierarchy. mplementing WordNet Lexical Similarity Measures ?- wn_lcs([lion, leopard, cougar, cat], LCS_SS_ID),wn_synset_components(LCS_SS_ID, Words_LCS_SS_ID).LCS_SS_ID = 102120997,Words_LCS_SS_ID = [feline:n:1, felid:n:1]. Fig. 4. A query for obtaining the LCS or most specific generalization of a set of concepts.Although in a database there only exists direct information about, e.g., lion, leopard, cougar andcat, it is possible to discover that this information is also pertinent for feline by using the pred-icate wn lcs/2 . In Figure 4, the concept referred to by the synset ID 102120997 (grouping [feline:n:1, felid:n:1] ) is the most specific concept of (the synsets of) lion , leopard , cougar and cat , that they share as a common ancestor in the IS-A hierarchy of WordNet. Fur-thermore, the predicate wn gen hyponyms upto level/3 , which generates all the hyponymsof a concept ( Synset ID ) up to a certain depth level (
Level ), can also be used to generate anontology of closely related terms to the given concept that can be used to implement flexiblequeries and text mining tasks. In particular, 6.2 illustrates an application of this work to textclassification, also including some performance measures.
In the following two subsections we perform experiments to find the performance of the imple-mented similarity measures and the cost of integrating WordNet into
Bousi ∼ Prolog . Instructions,programs and data to reproduce the experiments in these appendices have been made availableat https://dectau.uclm.es/bousi-prolog/wp-content/uploads/sites/3/2020/07/Published.zip . In this section we evaluate the computational cost of the measures implemented in Subsec-tion 5.1, comparing the results with other systems.Specifically, we are using an implementation of WordNet::Similarity for Java (WS4J) devel-oped by Hideki Shima. We use WS4J because it provides some time information that allowsthe cost of these measures to be appreciated. We are also using our own implementation of theWordNet-based similarity measures, but executed both by
SWI-Prolog and
Bousi ∼ Prolog . Thisallows us not only to compare the performance of our measures integrated into
Bousi ∼ Prolog with WS4J, but also the overhead introduced by our implementation of
Bousi ∼ Prolog w.r.t.
SWI-Prolog .In the first experiment, we selected twelve words with the highest number of senses. Then wepair them obtaining six pairs of words. Afterwards, for each word in that pair we generate theCartesian product of all their senses (
Word1:n:Sense1 , Word2:n:Sense2 ). Finally, we com-pute the similarity degree of these pairs, thus mimicking how WS4J operates, measuring theoverall time cost of the computation.Table 2 shows the costs involved in the computation of the similarity degree of these six pairs WS4J is available at https://github.com/Sciss/ws4j and also has a web interface WS4J Demo at http://ws4jdemo.appspot.com . For WS4J, the parameter MFS (Most Frequent Sense) is set to false. Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez of words for the three systems and the three edge-based measures. For each measure, ‘Time’shows the elapsed time in milliseconds, ‘Lat.’ the latency in milliseconds/pair, and ‘Thr.’ thethroughput in pairs/second. While BPL is at a small disadvantage with respect to SWIPL, WS4Jis roughly four times as fast.Table 2. Comparing
Bousi ∼ Prolog , SWI-Prolog and WS4J on six pairs of words with the high-est number of senses
MeasureSystem PATH WUP LCHTime Lat. Thr. Time Lat. Thr. Time Lat. Thr.BPL 966 0.02 44,572 959 0.02 45,400 959 0.02 44,943SWIPL 895 0.02 48,004 920 0.02 46,540 950 0.02 45,117WS4J 211 0.01 204,410 430 0.01 101,024 212 0.01 205,464
In a second experiment, we randomly generate pairs of noun and verb patterns (
Word1:Type:Sense1 , Word2:Type:Sense2 ) so
Type is either n or v . Then, we generate the calls to a sim-ilarity measure, and finally, we measure the performance of Bousi ∼ Prolog w.r.t.
SWI-Prolog . Table 3 shows the results of this experiment. The numbers are the average after tree executions.Table 3. Comparing
Bousi ∼ Prolog and
SWI-Prolog on randomly generated pairs of patterns
MeasureSystem Edge-based Information Content-basedPATH WUP LCH RES JCN LIN(milliseconds/10,000 pairs) (milliseconds/250 pairs)BPL 1,438 1,403 1,597 34,680 36,221 34,982SWIPL 242 242 275 33,521 35,088 35,073
The analysis of the data in Table 3 reveals that for the BPL system the average Latency of edge-based measures is 0.14 ms/pair and the average Latency of IC-based measures is 141.17 ms/pair,while for
SWI-Prolog they are 0.03 ms/pair and 138.24 ms/pair respectively. These results leadto an average ratio between both systems of 5.84 for edge-based measures and only 1.02 forIC-based measures, showing an acceptable overhead of
Bousi ∼ Prolog relative to
SWI-Prolog for these tasks. In the first case, the overhead is more noticeable when traversing 10K wordpairs than only 250 because tail recursion optimization is lost in the
Bousi ∼ Prolog to Prologtranslation. Thus, optimizing this translation will be the subject of future work.
Bousi ∼ Prolog is well suited to making the query-answering process more flexible, due to itsweak unification algorithm. In (Rubio-Manzano and Juli´an-Iranzo 2014) we discussed severalpractical applications where it can be useful, such as flexible deductive databases, knowledge-based systems, information retrieval, and approximate reasoning.
Bousi ∼ Prolog has been usedin a number of real applications such as: text classification (or cataloging) (Romero et al. 2013), https://code.google.com/archive/p/ws4j/wikis/DraftNextVersion.wiki describes a similar experi-ment for WS4J, but we were unable to replicate it because the results of this kind of experiments depends strongly onthe list of word pairs. mplementing WordNet Lexical Similarity Measures Bousi ∼ Prolog toweakly unify. The proposed method consists of the following steps:1.
Knowledge Base Building : The categories are (semantically) defined by extracting a setof proximity equations from standard thesauri and ontologies (WordNet in our case). Theset of proximity equations form the significative subset of the thesaurus or ontology that wewill use in the classification process and, by abuse of language, we name it the “ontology”file.2.
Flexible Search and Computing Occurrence Degrees : For each document content, thewords close to a category are searched in order to classify them, and their degrees ofoccurrence are obtained. The occurrence degree of a word is an aggregation of the numberof occurrences of the word (in a document) and its approximation degree with regard tothe category analyzed.3.
Computing Document Compatibility Degrees : The compatibility degrees of the doc-uments with regard to a category are computed using a specific compatibility measure.A compatibility measure is an operation which uses the occurrence degrees of the wordsclose to a category to calculate a document compatibility degree, that is, an index of howcompatible the document is with regard to the category analysed.4.
Classification Process : Finally, each document is classified as pertaining to the categoryor categories that return a higher compatibility degree. We assign to a document all the cat-egories that have a compatibility index between the maximum compatibility,
Max , obtainedfor that document and a minimum
Min=0.9*Max .It is noteworthy that our approach to text classification does not need a pre-classified set oftraining documents. The proposed method only requires the category names as user input. Hence,our method is not based on category occurrence frequency, but depends greatly on the definitionof that category and how the text fits that definition.Thanks to the integration with WordNet, we can generate the ontology files without humanintervention, starting from the set of predefined categories. Ontology files are computed eitherby: i) generating several level of hyponyms of a category and obtaining the similarity degreebetween them and the category by using a similarity measure (PATH, WUP, etc.); or ii) takingthe gloss of a category (which can be seen as the definition of the category), extracting a list ofwords using natural language processing techniques, and then obtaining the degree of relationbetween them and the category by using a similarity measure.Once the ontology file is generated and the categories from which we start are defined (interms of their semantic relationship with other words), we can then apply the remaining steps ofour classification algorithm.An application implementing the method described above can be found at the URL https://dectau.uclm.es/bousi-prolog/applications/ , and a preliminary paper on this subjectis (Al-Sayadi et al. 2020). The results shown in that paper are encouraging in terms of
Precision ,4 Pascual Juli´an-Iranzo and Fernando S´aenz-P´erezRecall and
F-measure . For instance, for the dataset “News Wires-2 (Reuters-10)”, which is aset of short texts (news limited up to 160 characters long) selected from Reuters-21578, weobtain an average Precision , Recall and
F-measure of 73.07%, 55.59% and 62.99% respectively.Our immediate goal is to improve
Recall and to contribute to provide explainable results.In order to show the feasibility of integrating WordNet into
Bousi ∼ Prolog , we undertookan experimental assessment of the cost of generating several ontologies and classifying somedatasets. The results are shown in Tables 4 and 5, with CPU runtime in seconds, the numberof inferences performed during the computation, and information on memory consumption inmegabytes.Table 4. Performance of automatic generation of ontologies based on WordNet hyponyms
Ontology file Equs. Runtime (s) Inferences Global Stack (Mb) Local Stack (Mb)Using similarity measures based on counting edgesodp hyp 263 0.038 287,140 0.823461 0.512711enviweb hyp 352 0.486 399,114 0.767675 0.723183reutersshorts hyp 278 0.039 286,338 0.348183 0.554588reuters10 hyp 314 0.044 328,070 0.541951 0.618492AVERAGE 302 0.152 325,165 0.620317 0.602243Using similarity measures based on information contentodp hyp ic 263 44.020 190,808,568 0.245071 0.512710enviweb hyp ic 352 28.529 124,793,835 0.344101 0.723183reutersshorts hyp ic 278 1.641 7,382,626 0.807327 0.554588reuters10 hyp ic 314 25.871 111,821,332 0.323959 0.618492AVERAGE 302 25.015 108,701,590 0.430115 0,602243
Table 5. Performance of classifying datasets from WordNet-generated ontologies
Dataset Runtime (s) Inferences Global Stack (Mb) Local Stack (Mb)Web Snippets (ODP): 115documents 1.453 14,510,417 2.49 2.31News Snippets (EnviWeb):116 documents 1.668 16,508,815 1.72 1.71News Wires-1 (Reuters-Short): 267 documents 2.766 27,308,333 3.32 4.39News Wires-2 (Reuters-10): 8 .
599 documents 422.729 3,586,006,321 247.62 341.49
In Table 4, each row groups the average data obtained when generating ontologies for a givendata set using three different similarity measures. The column “Equs.” shows the number ofproximity equations generated per ontology. The first part of the table presents data related tosimilarity measures based on counting edges (PATH, WUP and LCH) while the second partgives those based on information content (RES, JCN, LIN).Note that, for the ontologies which use similarity measures based on counting edges, the cost Precision : percentage of total positive classifications w.r.t. the total of classifications performed by the classifiermethod. In this case, ’positive classification’ means a classification where the classifier and the expert judgment co-incide.
Recall : percentage of total positive classifications w.r.t. the total of classifications performed by the expertclassifier.
F-measure : the harmonic mean between precision and recall. mplementing WordNet Lexical Similarity Measures We have presented techniques to embody the information stored in the lexical database Word-Net (Fellbaum 1998; Fellbaum 2006; Miller 1995) into the fuzzy logic programming language
Bousi ∼ Prolog (Rubio-Manzano and Juli´an-Iranzo 2014; Juli´an-Iranzo and Rubio-Manzano 2017;Juli´an-Iranzo and S´aenz-P´erez 2018a). However, the techniques developed can be used to con-nect WordNet to any logic programming language that uses an operational semantics based onsome variant of WSLD resolution.The main contributions of this study are the following:1. We have implemented, in Prolog, all the usual similarity measures (based on countingedges and on information content) to be found in standard tools such as wordnet::simila-rity (Pedersen et al. 2004).2. A whole
BPL subsystem ( wn-connect ) has been developed, providing those measuresand several built-in predicates to obtain useful information about words and synsets inWordNet. This subsystem can be used independently in a
Prolog interpreter.3. We have implemented directives to generate proximity equations from a set of words,linking them with an approximation degree. Hence, the significance of this work is tomake a fuzzy treatment of concepts via proximity relations possible, and also to endow
Bousi ∼ Prolog with linguistic characteristics.4. Because
Bousi ∼ Prolog allows WordNet databases to be accessed easily, it is possible touse interesting relations (antonymy, meronymy, etc.), or to use causal relations, to reason.5. We have provided the system implementing these techniques as a desktop application (forWindows, Mac and Linux OS’s – dectau.uclm.es/bousi-prolog ), and also an onlineweb interface ( dectau.uclm.es/bplweb ).6. Finally, we have undertaken an experimental assessment: Firstly, measuring the perfor-mance of the implemented WordNet-based similarity measures and the cost of generat-ing hyponymy-based ontologies; and, secondly, executing a text classification applica-tion implemented using
Bousi ∼ Prolog and its connection to WordNet, concluding that
Bousi ∼ Prolog has a reasonable performance w.r.t. other systems.As future work, experiments suggest enhancing the performance of
Bousi ∼ Prolog by intro-ducing memorizing techniques, and optimizing their compilation by leveraging tail recursionoptimization, and also, implementing relatedness measures based on other techniques such asword embeddings.6
Pascual Juli´an-Iranzo and Fernando S´aenz-P´erez
Acknowledgements
We would like to express our gratitude to the anonymous reviewers and the area editor for their valuablecomments that have greatly improved the final version of our paper.
References A L -S AYADI , S. H., J
ULI ´ AN -I RANZO , P., R
OMERO , F. P.,
AND
S ´
AENZ -P ´
EREZ , F. 2020. A Fuzzy Declar-ative Approach to Classify Unlabeled Short Texts based on Automatically Constructed WordNet Ontolo-gies. In
Proc. of the 12th European Symposium on Computational Intelligence and Mathematics, ESCIM2020 . 1–6.A
LLAHYARI , M., P
OURIYEH , S., A
SSEFI , M., S
AFAEI , S., T
RIPPE , E., G
UTIERREZ , J.,
AND K OCHUT ,K. 2017. A Brief Survey of Text Mining: Classification, Clustering and Extraction Techniques.
CoRR abs/1707.02919 .B AEZA -Y ATES , R.
AND R IBEIRO -N ETO , B. 2011.
Modern Information Retrieval - The Concepts andTechnology behind Search, Second Edition . Pearson Education Ltd., Harlow, England.B
UDANITSKY , A.
AND H IRST , G. 2006. Evaluating WordNet-based Measures of Lexical Semantic Relat-edness.
Computational Linguistics 32,
1, 13–47.C¸
AKIR , E.
AND U LUKAN , H. Z. 2019. A Fuzzy Logic Programming Environment for Recycling FacilitySelection. In
Proc. of the 11th International Joint Conference on Computational Intelligence, IJCCI2019 . ScitePress, 367–374.C¸
AKIR , E.
AND U LUKAN , H. Z. 2020. A Fuzzy Linguistic Programming for Sustainable Ecotourism Ac-tivities. In
Proc. of the 10th Annual Computing and Communication Workshop and Conference, CCWC2020 . IEEE, 121–126.F
ELLBAUM , C. 1998.
WordNet: An Electronic Lexical Database . MIT Press.F
ELLBAUM , C. 2006. WordNet(s). In
Encyclopedia of Language & Linguistics, Second Edition , K. B. E.in Chief), Ed. Vol. 13. Elsevier, Oxford., 665–670.F
ELLBAUM , C.
AND et al. https://wordnet.princeton.edu/documentation/prologdb5wn .F ONTANA , F. A.
AND F ORMATO , F. 1999. Likelog: A Logic Programming Language for Flexible DataRetrieval. In
Proc. of the 1999 ACM Symposium on Applied Computing (SAC’99) . 260–267.F
ONTANA , F. A.
AND F ORMATO , F. 2002. A Similarity-based Resolution Rule.
International Journal ofIntelligent Systems 17,
9, 853–872.F
RANCIS , W. N.
AND K UCERA , H. 1964, 1971, 1979. A Standard Corpus of Present-Day Edited AmericanEnglish, for use with Digital Computers (Brown). .J IANG , J. J.
AND C ONRATH , D. W. 1997. Semantic Similarity Based on Corpus Statistics and Lexical Tax-onomy. In
Proc. of the 10th Research on Computational Linguistics International Conference, ROCLING1997 . The Association for Computational Linguistics and Chinese Language Processing (ACLCLP), 19–33.J
ULI ´ AN -I RANZO , P.
AND R UBIO -M ANZANO , C. 2015. Proximity-based Unification Theory.
Fuzzy Setsand Systems 262 , 21–43.J
ULI ´ AN -I RANZO , P.
AND R UBIO -M ANZANO , C. 2017. A Sound and Complete Semantics for a Similarity-based Logic Programming Language.
Fuzzy Sets and Systems 317 , 1–26.J
ULI ´ AN -I RANZO , P.
AND
S ´
AENZ -P ´
EREZ , F. 2017. FuzzyDES or how DES met Bousi Prolog. In
Proc. ofthe 2017 IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2017 . 1–6.J
ULI ´ AN -I RANZO , P.
AND
S ´
AENZ -P ´
EREZ , F. 2018a. An Efficient Proximity-based Unification Algorithm.In
Proc. of the 2018 IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2018 . 1–8.J
ULI ´ AN -I RANZO , P.
AND
S ´
AENZ -P ´
EREZ , F. 2018b. A Fuzzy Datalog Deductive Database System.
IEEETransactions on Fuzzy Systems 26 , 2634–2648.J
ULI ´ AN -I RANZO , P.
AND
S ´
AENZ -P ´
EREZ , F. 2019. WordNet and Prolog: why not? In
Proc. of the 11thConference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019 . 1–8. mplementing WordNet Lexical Similarity Measures L EACOCK , C.
AND C HODOROW , M. 1998. Combining Local Context and Wordnet Similarity for WordSense Identification. In
WordNet: An Electronic Lexical Database , C. Fellbaum, Ed. MIT Press., 265–283.L EE , R. 1972. Fuzzy Logic and the Resolution Principle. Journal of the ACM 19,
1, 119–129.L IN , D. 1998. An Information-Theoretic Definition of Similarity. In Proc. of the 15th International Con-ference on Machine Learning . Morgan Kaufmann, 296–304.L
OIA , V., S
ENATORE , S.,
AND S ESSA , M. I. 2001. Similarity-based SLD Resolution and Its Implemen-tation in an Extended Prolog System. In
Proc. of the 2001 IEEE International Conference on FuzzySystems, FUZZ-IEEE 2001 . 650–653.M
ARTELLI , A.
AND M ONTANARI , U. 1982. An Efficient Unification Algorithm.
ACM Transactions onProgramming Languages and Systems 4 , 258–282.M
ILLER , G. A. 1995. WordNet: A Lexical Database for English.
Comms. of the ACM 38,
11, 39–41.M
ILLER , G. A., L
EACOCK , C., T
ENGI , R.,
AND B UNKER , R. 1993. A Semantic Concordance. In
Proc.of the Workshop on Human Language Technology, HLT 1993 . 303–308.P
EDERSEN , T., P
ATWARDHAN , S.,
AND M ICHELIZZI , J. 2004. WordNet::Similarity - Measuring theRelatedness of Concepts. In
Proc. of the Nineteenth National Conference on Artificial Intelligence,Sixteenth Conference on Innovative Applications of Artificial Intelligence . AAAI Press / The MIT Press,1024–1025.R
ESNIK , P. 1995. Using Information Content to Evaluate Semantic Similarity in a Taxonomy. In
Proc.of the Fourteenth International Joint Conference on Artificial Intelligence, IJCAI 95, 2 Volumes . MorganKaufmann, 448–453.R
OMERO , F. P., J
ULI ´ AN -I RANZO , P., F
ERREIRA -S ATLER , M.,
AND G ALLARDO -C ASERO , J. 2013. Clas-sifying Unlabeled Short Texts using a Fuzzy Declarative Approach.
Language Resources and Evalua-tion 47,
1, 151–178.R
UBIO -M ANZANO , C.
AND J ULI ´ AN -I RANZO , P. 2014. Fuzzy Linguistic Prolog and its Applications.
Journal of Intelligent and Fuzzy Systems 26 , 1503–1516.R
UBIO -M ANZANO , C.
AND J ULI ´ AN -I RANZO , P. 2015. Incorporation of Abstraction Capability in a Logic-based Framework by using Proximity Relations.
Journal of Intelligent and Fuzzy Systems 29,
4, 1671–1683.R
UBIO -M ANZANO , C.
AND T RIVI ˜ NO , G. 2016. Improving Player Experience in Computer Games by us-ing Players’ Behavior Analysis and Linguistic Descriptions. International Journal of Human-ComputerStudies 95 , 27 – 38.S
ANTUS , E., W
ANG , H., C
HERSONI , E.,
AND Z HANG , Y. 2018. A Rank-Based Similarity Metric for WordEmbeddings. In
Proc. of the 56th Annual Meeting of the Association for Computational Linguistics ACL2018 (2) . Association for Computational Linguistics, 552–557.S
ERRANO -G UERRERO , J., O
LIVAS , J. A., R
OMERO , F. P.,
AND H ERRERA -V IEDMA , E. 2015. SentimentAnalysis: A Review and Comparative Analysis of Web Services.
Inf. Sci. 311 , 18–38.S
ESSA , M. I. 2002. Approximate Reasoning by Similarity-based SLD Resolution.
Theoretical ComputerScience 275,
VAN E MDEN , M.
AND K OWALSKI , R. 1976. The Semantics of Predicate Logic as a Programming Lan-guage.
Journal of the ACM 23,
4, 733–742.W U , Z. AND P ALMER , M. S. 1994. Verb Semantics and Lexical Selection. In
Proc. of the 32nd AnnualMeeting of the Association for Computational Linguistics . Morgan Kaufmann Publishers / ACL, 133–138.Z
ADEH , L. 1965. Fuzzy Sets.