Context based Analysis of Lexical Semantics for Hindi Language
CContext based Analysis of Lexical Semantics for Hindi Language
Mohd Zeeshan Ansari and Lubna Khan
Department of Computer Engineering, Jamia Millia Islamia, New Delhi, India. [email protected], [email protected]
Abstract.
A word having multiple senses in a text introduces the lexical semantic task to find out which particular sense is appropriate for the given context. One such task is Word sense disambiguation which refers to the identification of the most appropriate meaning of the polysemous word in a given context using computational algorithms. The language processing research in Hindi, the official language of India, and other Indian languages is restricted by unavailability of the standard corpus. For Hindi word sense disambiguation also, the large corpus is not avail-able. In this work, we prepared the text containing new senses of certain words leading to the enrichment of the sense-tagged Hindi corpus of sixty polysemous words. Furthermore, we analyzed two novel lexical associa-tions for Hindi word sense disambiguation based on the contextual fea-tures of the polysemous word. The evaluation of these methods is carried out over learning algorithms and favorable results are achieved.
Keywords:
Natural Language Processing, Lexical Semantics, Word Sense Dis-ambiguation. Introduction
All natural languages have certain words with multiple meanings called homonyms. The automatic selection of the appropriate meaning of such words is a challenging task in natural language processing. Humans can easily comprehend the appropriate mean-ing of such word using the context in which it is used. However, the relationship be-tween the meaning and the context is not well understood by machines because the computational representation of context is considerably complex. When a particular word has different meanings, also called senses, pertaining to the contexts in which it is used, it is called polysemous. This characteristic produces a great deal of complexity in the processing of natural languages. In lexical semantics of natural languages, the context is closely related to the specific task, domain and underlying language. Since certain words or phrases that occur in a given text may be interpreted in multiple ways, the significance of context is to determine its appropriate sense. Therefore, for the au- tomatic identification of the particular sense of a word, the context analysis is neces-sary. Consequently, the Word Sense Disambiguation (WSD) task is to determine the most appropriate sense of a word in a given context. Most of the WSD techniques consider context as the text surrounding the polysemous word, usually in a fixed size window keeping the word in the middle. Several approaches to solve WSD task for English and European languages are present in literature. Most of them are classified under three major approaches: Knowledge-based, supervised and unsupervised ap-proaches [1,2,7,8,13,15,24]. Enough WSD research based on semi-supervised and hy-brid approaches is carried out, with English being the primary language. The studies on complex language processing tasks such as machine translation, in-formation extraction, question answering, sentiment analysis, etc. involving Indian lan-guages, especially Hindi, the official language of India, is constrained by unavailability of large standard corpora. As WSD is involved in many such tasks, it is therefore, quiet challenging for Indian languages as they are rich in morphology. Also, the development of several resources like WordNet, machine-readable dictionaries, language corpora, etc. are still under progress [32,35]. In this article, we present the work on Hindi written in Devanagari script, which is the official language of India. We explore the Hindi WSD task by the interpretation and analysis of the context in a variety of ways. The Sense Tagged Hindi Corpus, used in this work was developed under the Technology Devel-opment for the Indian Languages (TDIL) project, Government of India [30]. This cor-pus is available for research on Hindi Word Sense Disambiguation which consist of polysemous words and their instances, see Table 1. We enriched this corpus by the inclusion of more senses in the case of two existing words. This enriched sense tagged Hindi corpus is used to investigate lexical and semantic attributes significantly to Hindi WSD task. Our contribution to Hindi WSD is three folds (1) propose additional sense of two words (i) ‘ बाल ’ meaning ‘ भुट्टे का बाल ’ (corn silk), and (ii) ‘ कदम ’ meaning ‘ कदम का पेड़ ’ (bur flower) and their instances to the existing corpus (2) we explore two novel attribute associations for Hindi WSD and test them on a range of window size, (3) pre-sent comparative analysis of their performance with respect to the methods found in literature. For a comparative evaluation of our methods, we also constructed the attrib-utes defined by Singh et. al. [29]. In this work, we investigate various attribute associations for Hindi WSD task. The feature vector is constructed using the associations of local context, collocation, bag of words after stop word removal and vibhakti.
These vectors of features when tested with the different window size of text produce significant results. The overall work is orga-nized in the sections as follows. Section 2 shows related work on general WSD task, highlighting the challenges faced by researchers in WSD. Section 3 introduces Hindi WSD tasks. Section 4 gives the basic definitions of attributes of WSD task. Section 5 gives the detailed description of our work, i.e., the suggested attribute associations, the existing methods with illustrative examples. Section 6 gives detailed description of the dataset preparation, experimental setup and the analysis and discussion of results. Sec-tion 7 presents the conclusion and future scope work.
Table 1.
Corpus statistics. Original Corpus Enriched Corpus Number of words 381875 383008 Number of instances 7506 7570 Number of polysemous words 60 60 Related Word
In lexical semantics, the word sense disambiguation principally involves two key tasks, firstly the determination of potential senses (or meanings) of each word, and secondly, the classification of each word to its probable sense with low error and computational complexity. The challenges identified for the WSD task are (1) discreteness of the senses, (2) differences between dictionaries, (3) amount of samples and semantic knowledge available. The discreteness of the senses can be divided into coarse-grained and fine-grained levels. A human can easily comprehend coarse-grained which deals with homographs, but it is quite difficult for a human to understand fine-grained level. The number of samples and semantic knowledge available can be enhanced by building them manually which involves large costs. Numerous approaches and algorithms are built for disambiguating words based on a given context. In initial phases, the disam-biguation methods were typically evaluated on a small dataset consisting of words with scarce and strong sense differences, e.g. Yarowsky [4] reported 96% precision for twelve words with only two strong sense distinctions. Subsequently, large number of approaches were examined and huge effort was dedicated to solve this task. ACL SIGLEX Workshop concluded with an open challange “Tagging Text with Lexical Se-mantics: Why, What and How? WSD is perhaps the great open problem at the lexical level of NLP” [7,8,10]. Similarly, the outcomes of the Senseval, Senseval-2 and Sense-val-3 in which systems submitted in these conferences did not achieve eighty percent accuracy on lexical sample as well as all words tasks [14,16,23]. Bayesian classification models applied to several investigations in WSD are consid-erably successful [22,31]. Due to its simplicity, the Naive Bayes algorithm is the pre-ferred choice to obtain the preliminary results on supervised WSD [12,18,21]. Mooney [6] considered seven supervised learning algorithms namely Naive Bayes, decision tree, k-nearest neighbor, perceptron, slogic-based conjunctive and disjunctive normal form learners and a decision list learner. Subsequently, they were tested on the ambiguous word ‘line’ having six senses and a comparison is drawn among them, which showed favorable results on Naive Bayes classifier and perceptron. The words surrounding the ambiguous word were used as features for the classifiers. Leacock et. al. [3] used Naive Bayes classifier and merged topic and local context to achieve comparable accuracy for the combination in order to disambiguate among nouns, a verbs and an adjectives. Ped-erson [12] took an ensemble of several simple Naïve Bayes classifiers to improve WSD accuracy using nine different window sizes of left and right of context: 0-5,10,20 and 50. A measure of 88% and 89% accuracy was achieved on the two datasets. Le and
Shimazu [19] studied Naive Bayes classifier for performing WSD task utilizing en-riched features. They combined features represented by sequence of words in a local context and collocations using features derived from a forward sequential selection al-gorithm. The accuracy obtained was 92.3% for four common test words and 72.7% for nouns and 66.4% for verbs when tested on large DSO corpus. A decomposable model proposed by Bruce and Wiebe [11] is complex model which considered dependence graph of various dependent and independent features. The primary weakness of this approach is estimation of large number of parameters, which are related to the number of different mix of the interdependent characteristics. Therefore, this technique required a large size of dataset as well as computational cost to significantly estimate the param-eters. In contrast to this, Pederson and Rebecca [9] proposed an unsupervised method on unlabeled text for the identification of an optimal model, with high performance and low cost in parameter estimation. Zhong and Ng developed [27] a system based on a supervised learning approach, a flexible framework that achieved good results on sev-eral sensEval and semEval tasks. Lee et. al. [20] investigated support vector machines on multiple knowledge sources to obtain the results for a lexical task and reported that their method performed fairly well. Chaplot, et. al. [33] developed an unsupervised model using Maximum A Posteriori Inference Query built on a Markov Random Field using WordNet and Standford Parser. It is a graphical model which was evaluated on English all word dataset and showed better and fast results compared to the existing best unsupervised models [25]. The entropy maximization approach estimates proba-bilities on the basis of making a small but necessary number of assumptions. This prop-erty is satisfied only by the probability distribution with the highest entropy. The ad-vantage of such models is that even poor features may be applied accurately, therefore, such models are also found relevant in case of WSD [5,17]. Singh, et. al. [29] have put forward the initial efforts on Hindi WSD task using su-pervised classification, principally by the application of Naive Bayes algorithm in sense tagged Hindi corpus. They considered eleven feature vectors based on local context, collocations, bag of words and vibhakti . The precision obtained from their model is 77.52% for bag of words, and 86.11% for nouns words in feature vector after applying morphology. A precision of 56.49%, by including vibhakti in the feature vector is also reported. The task for Hindi WSD supervised approach was further extended using sense annotated Hindi corpus, dictionary definitions and semantic relationships to as-sign weightage to words present within the context of polysemous words . The context of the polysemous word is a sequence of words appearing in window with the polyse-mous word at the center. For evaluation, sense tagged corpus consisting of sixty Hindi words (nouns) using sense definitions and semantic relations obtained from Hindi WordNet, were utilized. The results reported show that overall average precision and recall values were 78.98% and 73.41% respectively.
Table 2.
Excerpts of Hindi text containing the polysemous word ‘ हार ’ in two senses. Sense 1 Paragraph1 . निमााता - निदेशक करण जौहर िे नविटर पर नलखा है के के आर की हार से बहुत दुख हुआ। खेल को खेल भाििा से देखिा चानहए और हार स्िीकार करिी चानहए। Sense 2 Paragraph . न्यूयॉका । हीरे का हार पहिी एक बाबी गुनिया न्यूयॉका में रेकॉिा कीमत में िीलाम हुई है । अपिी तरह की ये अके ली बाबी िॉल काला नलबास पहिे हुई है और उसके गले में एक कै रेट का चौकोर गुलाबी हीरे का हार है । ये गुनिया में बिाया गया था और तबसे लेकर आज तक इसका रूप कई बार बदला है । सबसे बिी िीलामी का रेकॉिा बिािे िाली बाबी गुनिया को ऑस्रेनलया के एक गहिों के निजायिर स्टीफािो कै न्टुरी िे बिाया है । Hadni et. al. [34] reported WSD task for Arabic text categorization using feature selec-tion methods implemented on SVM and NB classifier and achieved a precision of 72% and 71% respectively. Naseer and Sharmad reported the work on NB classifier for Urdu WSD. They selected four ambiguous Urdu words (1 noun and 3 verbs) having high frequencies appearing in the corpus used with varying window sizes of ±3, ±5, ±7 around the target word. The maximum accuracy achieved was 98.33% for window size 7x7 when tested on three verbs and one noun. They observed that an increase in window size resulted in a enhanced performance [26]. Word Sense Disambiguation for Hindi Language
The application of Word Sense Disambiguation on polysemous words present in Hindi language text written in Devanagari script is called Hindi Word Sense Disambiguation. A Hindi polysemous word does also have a different meaning in different contexts. The text excerpts in Table 2. is the text in the Hindi language which illustrates one such word ‘ हार ’ . It can be clearly observed that there are two senses of the word ‘ हार ’ , in the first paragraph it means ‘ पराजय ’ (defeat) and in the second paragraph, it means ‘ माला ’ (necklace). The more details of senses of word ‘ हार ’ is presented in Table 3. Table 3.
Senses of ‘ हार ’ Sense Synonyms Meaning Use in sentence Sense 1 पराजय , आपजय (defeat) पराजित होने की अवस्था या भाव (defeated state or ex-pressions) इस चुनाव में उसकी हार जनजित है (in this elec-tion his defeat is cer-tain) जवघात , असफलता (failure) मात (checkmate) अजभभव (disgrace) Sense 2 माला (garland) गले में पहनने का एक प्रकार का सोने , चााँदी आजद का गहना (a type of gold or silver jewelry used for wearing around the neck) उसने हीरे का हार पहन रखा है (she is wearing a diamond necklace) नेकलेस (necklace) अवतंस , अवतन्स (garland) आभूषण (jewelry) Lexical Attributes for Hindi WSD task
The basic lexical attributes used in sense disambiguation are local context, collocation, bag of words, bag of words after stop words removal and vibhakti which are defined below. Illustrative examples of each method are given throughout this section.
Definition 1.
Local context ( l j ) is defined as the collection of words surrounding the ambiguous word in a given piece of text with a window size of j. The local context feature set denoted by l , contain words of local context in a window size ±2, i.e. two words from the left and two words from the right of the ambiguous word. The example may be given as l = [‘ हीरे ’, ‘ का ’, ‘ हार ’, ‘ पहिी ’, ‘ एक ’] . Definition 2.
Collocation ( c j ) is defined as the group of those sequence of words that include the target word. In a window size of j, the collocation feature set, c j , is the group of those sequences of size 2 to j+1 words which contain the ambiguous word. The col-location feature set denoted by c , contain collocation in a window size ±2, i.e. word sequence of size 2 and 3 which include the ambiguous word. The example may be given as c = [‘ हीरे का हार ’, ‘ का हार ’, ‘ हार पहिी ’, ‘ हार पहिी एक ’, ‘ का हार पहिी ’] . Definition 3.
Bag of words ( b j ) is the simple bag of j words in the left and right of the ambiguous words in a window size ±j, but without removing the stop words from the text. The example of window size ±2 may be given as b = [‘ हीरे ’, ‘ का ’, ‘ हार ’, ‘ पहिी ’, ‘ एक ’] . Definition 4.
Bag of words after stop word removal ( b *j ) is the bag of j words in the left and right of the ambiguous word in a window size ±j after removing the stop words from the text. The example may be given as b *2 = [‘’, ‘ हीरे ’, ‘ हार ’, ‘ पहिी ’] . Definition 5.
Vibhakti ( v j ) are set of words considered important constituent of Hindi grammar, referring to the relationship between verbs and other constituents typically nouns or pronouns in a sentence. The vibhakti involved in this work are, v = [ िे (ne) , को (ko) , से (se) , के (ke) , नलए (liye) , का (ka) , की (ki) , में (mein) , पर (par) , हे (hey) , अरे (arey)] [27]. The feature set is defined by making bag of words with only vibhakti in the left and the right of the target word with a window size ±j. The example feature set may be given as, v = [‘’, ‘ का ’, ‘’,‘’] . Table 2. Paragraph 2.
Table 4.
Dataset description for Hindi WSD task. Number of senses Hindi Polysemous words 2 अशोक , काांि , कोटा , क्रिया , गल्ला , गुिा , गुरु , ग्राम , घटिा , चांदा , चारा , जीिा , जेठ , िब्बा , िाक , , सोिा , हल , ढाल , ताि , ताि , नतल , तीर , तुलसी , दक्ष , दर , दाद , दाम , धि , धुि , मााँग , लाल , निनध , शेर , सीमा , हार अांग , अांश , अचल , उत्तर , कमाि , कुां भ , क्वाटार , खाि , चरण , तेल , थाि , फल , बाल , मत , मात्रा , िचि , िगा , सांिमण , सांबांध कदम , कलम , धारा , मूल चाल , टीका Lexical Attribute Associations for Hindi WSD task
In order to construct a novel set of lexical attribute associations, we defined lexical association feature set on a range of window size. This set is obtained by a unique combination of basic attributes, viz. local context, collocation, bag of words after stop words removal and vibhakti . We proposed two novel attribute associations and tested them on window size range from ±2 to ±5. These are defined as follows
Collocation and bag of words after stop words removal ([c+b*]j).
It is the combi-nation of local context and bag of words after stop words removal of window size j. The feature set [ c + b * ] j is obtained by merging the basic features of collocation c j and bag of words after stop word removal b * j of window size ±j. Therefore, the feature set [ c + b * ] is defined as the combination of c and b * of window size j=±2. Using the same example text as used in previous examples, the feature set of [ c + b * ] is obtained as [‘ हीरे का हार ’, ‘ का हार ’, ‘ हार पहिी ’, ‘ हार पहिी एक ’, ‘ का हार पहिी ’, ‘ हीरे ’, ‘ हार ’, ‘ पहिी ’ ] . Similarly, the feature sets of window size ±3, ±4, ±5 are also obtained. Local context, collocation and vibhakti ([l+c+v]j).
It is the combination of three basic features, local context, collocation and vibhakti . The feature [ l + c + v ] j is obtained by merging the basic features of local context l j , collocation c j and vibhakti v j of window size ±j. Therefore, the feature [ l + c + v ] is defined as the combination of feature l , c and v of window size j=±2. Using the same example text as used in previous examples, the feature set of [ l + c + v ] is obtained as [‘ हीरे ’, ‘ का ’, ‘ हार ’, ‘ पहिी ’, ‘ एक ’, ‘ हीरे का हार ’, ‘ का हार ’, ‘ हार पहिी ’, ‘ हार पहिी एक ’, ‘ का हार पहिी ’, ‘’, ‘ का ’, ‘’, ‘’ ] . Similarly, the feature sets of different window size of ±3, ±4, ±5 are also constructed. Table 5.
Precision, Recall and F-Score Method Window P R F Collocation + bow 5
Collocation + local context + vibhakti 0.74 0.80 0.77 Singh et. al. 0.77 0.82 0.79 Bow after stop word removed 0.76 0.81 0.78 Collocation + bow 4 0.79 0.84 0.82 Collocation + local context + vibhakti 0.74 0.80 0.77 Singh et. al. 0.76 0.82 0.79 Bow after stop word removed 0.75 0.81 0.78 Collocation + bow 3 0.78 0.83 0.80 Collocation + local context + vibhakti 0.74 0.80 0.77 Singh et. al. 0.76 0.81 0.78 Bow after stop word removed 0.74 0.80 0.77 Collocation + bow 2 0.77 0.82 0.79 Collocation + local context + vibhakti 0.76 0.81 0.78 Singh et. al. 0.76 0.81 0.78 Bow after stop word removed 0.70 0.76 0.73
The attribute association suggested by Singh et. al. (2015) [29] for Hindi Word Sense Disambiguation is local context with collocation ([ l + c ] j ). It is a combination of local context and collocation of window size j. This set is obtained by merging the basic feature sets l j and c j as defined at the beginning of this section. The feature set [ l + c ] consists of words in local context and collocation in window size j=±2, which can also be obtained by merging the basic features l and c . The example may be given as [ l + c ] = [ ‘ हीरे ’, ‘ का ’, ‘ हार ’, ‘ पहिी ’, ‘ एक ’, ‘ हीरे का हार ’, ‘ का हार ’, ‘ हार पहिी ’, ‘ हार पहिी एक ’, ‘ का हार पहिी ’ ] . This feature combination [ l + c ] j was analyzed for different window size having a range from j = ±5 to ±25 [27]. Analysis of Lexical Relations for Hindi WSD
Experimental Setup
The dataset is generated using the sense tagged Hindi corpus, also used by Singh et. al. [29] which consists of sixty polysemous Hindi nouns with their senses. Table 4. presents the complete set of all such words along with the number of senses for each word. To this corpus, we added one more sense of two words, ‘ बाल ’ and ‘ कदम ’ . It was identified that the word ‘ बाल has one more sense ‘ भुट्टे का बाल ’ (corn silk) which is not present in the corpus. Similarly, it was also identified word ‘ कदम ’ has one more sense ‘ कदम का पेड़ ’ (bur flower) which is also not present in the corpus. Subsequently, the in-stances of both words were collected and merged with the corpus leading to the final dataset having additional 64 instances of 1133 words more, than original corpus. Although the corpus is organized in a defined format, we further preprocess it for our experimental settings. Finally, we divide the dataset into training and testing data according to ratio 3:1. We performed a series of experiments using the proposed as well as previously existing attribute associations and also using the basic attributes alone. All the methods are examined on the window size of range from ±2 to ±5. Analysis of results
By the application of the Naïve Bayes classification algorithm, we predicted the appro-priate sense of each ambiguous word. We predicted it for each of the test samples, and reported the sense accuracy as the number of correct sense predictions divided by total number of test sample.
We computed the metrics such as precision, recall and F1-meas-ure for the two proposed attribute associations as well as methods of Singh et. al. [29] which are presented in Table 5. It is observed from the analysis that the proposed at-tribute associations for collocation and bag of words after stop word removal with win-dow size ±5, i.e. [ c + b* ] , performs best and achieves the highest precision, recall and accuracy values of 0.80, 0.85 and 0.85 respectively. Moreover, this combination method with window size in the range from ±3 to ±5 outperforms the other methods. We deduce that this combination is best suited for our problem and on increasing the window size greater than ±5 much higher performance can be achieved. The second proposed combination of local context, collocation and vibhakti shows poor perfor-mance than other methods because of the introduction of vibhakti . The reason being that there is no sense related words in it. This can be endorsed by the observation that it alone shows the worst performance. Therefore, we deduce that the vibhakti do not make any contribution in the performance improvement of the model when used alone as well as when used in combination, moreover it degrades the performance when used in combination. The performance of basic attributes without combination show reduced performance as compared to the proposed and existing attribute associations. The bag of words after stop words removal of window size ±5, i.e. b* is the best among the basic attributes. It is, therefore, considered most appropriate for the comparison in the analysis throughout, although, the results of local context and collocation have also been obtained from the experiments. Conclusion
The present work explores the lexical semantics for the Hindi language in Devanagari script. A contribution is made by the addition one new sense each in case of two poly-semous words which contains 1133 Hindi Devanagari words under 64 instances. The investigation is carried out to examine the effect of several lexical attribute associations over Hindi sense disambiguation. Realizing that the neighbouring words in the context of the ambiguous word play a vital role in feature vector formation, two novel attribute associations are proposed. These methods are successfully tested along with the exist-ing and basic methods on the corpus of 60 polysemous words. The precision, recall and F1- measure are computed for each feature vector. One of our proposed combination method performs best as compared to rest of the methods which is observed from the figures of outcomes obtained from the experiments. The second proposed method val-idates the fact that vibhakti do not make any contribution in disambiguation of senses. A scope of examining the proposed methods on a higher range of window size is left for future. References Duda, R. O. and Hart, P.E. (1973): Pattern Classification and Scene Analysis, John Wiley & Sons. 2.
Gale, W. A., Church, K., and Yarowsky, D. (1992b): A method for disambiguating word senses in a corpus, Comput. Human, 26, 415–439. 3.
Leacock, C., Geoff T. and Ellen V. (1993): Corpus based statistical sense resolution, Pro-ceedings of the ARPA Workshop on Human Language Technology, Plainsboro, U.S.A., 260–265. 4.
Yarowsky, D. (1995): Decision lists for lexical ambiguity resolution: Application to accent resolution in Spanish and French, Proceedings of the 32nd Annual Meeting of the Associa-tion for Computational Linguistics (Las Cruces, NM), 88-95. 5.
Berger, A., Pietra, S. D., and Pietra, V. D. (1996): A Maximum Entropy Approach to Natural Language, Processing Computational Linguistics, 22(1). 6.
Mooney, R. J. (1996): Comparative experiments on disambiguating word senses: An illus-tration of the role of bias in machine learning, Proceedings Conference on Empirical Meth-ods in Natural Language Processing (EMNLP), 82–91. 7.
Ng, T. H. (1997): Getting serious about word sense disambiguation, Proceedings of the ACL SIGLEX Workshop on Tagging Text with Lexical Semantics: Why, What, and How? (Washington D.C.), 1–7. 8.
Resnik, P. and Yarowsky, D, (1997): A perspective on word sense disambiguation methods and their evaluation. In Proceedings of the ACL SIGLEX Workshop on Tagging Text with Lexical Semantics: Why, What, and How? (Washington, D.C.). 79–86. 9.
Pederson, T. and Rebecca, B. (1997): Distinguishing word senses in untagged text, Pro-ceedings of the Second Conference on Empirical Methods in Natural Language Processing, Providence, U.S.A., 197–207. 10.
Leacock, C., Miller, G. A., and Chodorow, M. (1998): Using Corpus Statistics and WordNet Relations for Sense Identification, Journal of Computational Linguistics, Special issue on word sense disambiguation, 24, I, 147-165. 11.
Bruce, R. F. and Wiebe, J. M. (1999): Decomposable Modeling in Natural Language Pro-cessing, Computational Linguistics, 25(2). 12.
Pederson, T. (2000): A Simple Approach to Building Ensembles of Naïve Bayesian Classi-fier for word sense disambiguation, Proceeding of the North American Chapter of the Asso-ciation for Computational Linguistics, NAACL, 63-69. 13.
Manning, C.D. and Schuze, H. (2000): Introduction to Statistical Natural Language Procees-ing, The MIT Press, Massachusetts. 1 14.
Kilgarriff, A. and Rosenzweig, J. (2000): English SENSEVAL: Report and Results, Pro-ceedings of the International Conference on Language Resources and Evaluation, LREC. 15.
Escudero, G., Mµarquez, L. and Rigau, G. (2000a): A Comparison between Supervised Learning Algorithms for Word Sense Disambiguation, Proceedings of the Computational Natural Language Learning Workshop, CoNLL. 16.
Kilgarriff, A. (2001): English Lexical Sample Task Description, Proceedings of the Interna-tional Workshop on Evaluating Word Sense Disambiguation Systems, Senseval-2. 17.
Suarez, A. and Palomar, M. (2002): A Maximum Entropy based Word Sense Disambigua-tion System, Proceedings International Conference on Computational Linguistics, COLING. 18.
Banerjee, S., and Pedersen, T., (2003): Extended gloss overlaps as a measure of semantic relatedness; In Proceedings of the 18th International Joint Conference on Artificial Intelli-gence (IJCAI, Acapulco, Mexico). 805-810. 19.
Le C. A. and Shimazu, A. (2004): High WSD accuracy using Naïve Bayesian classifier with rich features, PACLIC 18, Waseda University, Tokyo, 105, 113. 20.
Lee, Y. K., Ng, H. T., and Chia, T. K. (2004): Supervised Word Sense Disambiguation with Support Vector Machine and Multiple Knowledge Sources, Proceeding of SENSEVAL-3 : Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, Association for Computational Linguistics,137-140. 21.
Yuret, D. (2004): Some Experiments with a Naive Bayes WSD System, Proceedings of the International Workshop on Evaluating Word Sense Disambiguation Systems, Senseval3. 22.
Suarez, A. (2004): Resolucion de la ambigauedad semantica de las palabras mediante modelos de probabilidad de maxima entrop³a, PhD thesis, Departamento de Lenguajes y Sistemas Informaticos, Universidad de Alicante. 23.
Mihalcea, R. Chklovski, T. and Kilgarriff, A. (2004): The Senseval-3 English lexical sam-ple task, Proceedings of SENSEVAL-3, the third international workshop on the evaluation of systems for the semantic analysis of text. 24.
Rada Mihalcea, Courtney Corley, Carlo Strapparava (2006): Corpus-based and Knowledge-based measures of text semantic similarity; AAAI, Volume 6, 775-780. 25.
Navigli, R. and Lapata, M. (2007): Graph Connectivity Measures for Unsupervised Word Sense Disambiguation, Proceedings of the 20th International Joint Conference on Artificial Intelligence Hyderabad, India, 1683–1688. 26.
Naseer, A. and Sharmad, H. (2010): Supervised Word Sense Disambiguation for Urdu using Bayesian Classification, Proceeding of Conference on Language & Technology (CLT10). 27.
Zhong, Z. and Ng, H. T., (2010): It Makes Sense: A Wide-Coverage Word Sense Disambig-uation System for Free Text, Proceeding of the ACL, System Demonstration, pp8-83, Upp-sala, Sweden. 28.
Singh, S., Siddiqui, T. J., (2013): A supervised algorithm for Hindi Word Sense Disambig-uation, International Journal of Systems, Algorithms & Algorithms, Volume 3. 29.
Singh, S., Siddiqui, T. J., Sharma, S. K., (2014): Naïve Bayes (NB) classifier for Hindi Word Sense Disambiguation (WSD), Proceedings of the 7th ACM India Computing Conference (Compute 14) , http://dx.doi.org/10. 30.
Sense Annotated Hindi Corpus: Indian Language Technology Proliferation & Develop-mentCenter,tdil-de.in/index.php 31.
Francisco, V. J., (2014): Word sense disambiguation through associative dictionaries, PhD thesis, Instituto Politecnico Nacional, Mexico, D. F. 32.
Ansari, M.Z., Ahmad, T., Ali, M. A., (2018): Cross Script Hindi English NER Corpus from Wikipedia in Proceedings of International Conference on Intelligent Data and Internet of Things ICICI 2018. 2 33.
Chaplot, D. S., Bhattacharyya, P and Paranjape, A. (2015): Unsupervised Word Sense Dis-ambiguation Using Markov Random Field and Dependency Parser; Proceedings of the 29 AAAI Conference on Artificial Intelligence. 34.