Ruket Cakici

Automatic description generation from images: a survey of models, datasets, and evaluation measures

Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language processing communities. In this survey, we classify the existing approaches based on how they conceptualize this problem, viz., models that cast description as either generation problem or as a retrieval problem over a visual or multimodal representational space. We provide a detailed review of existing models, highlighting their advantages and disadvantages. Moreover, we give an overview of the benchmark image datasets and the evaluation measures that have been developed to assess the quality of machine-generated image descriptions. Finally we extrapolate future directions in the area of automatic image description generation.

Annotating Subordinators in the Turkish Discourse Bank

In this paper we explain how we annotated subordinators in the Turkish Discourse Bank (TDB), an effort that started in 2007 and is still continuing. We introduce the project and describe some of the issues that were important in annotating three subordinators, namely karsin, ragmen and halde, all of which encode the coherence relation Contrast-Concession. We also describe the annotation tool.

Multi-lingual Dependency Parsing with Incremental Integer Linear Programming

Our approach to dependency parsing is based on the linear model of McDonald et al.(McDonald et al., 2005b). Instead of solving the linear model using the Maximum Spanning Tree algorithm we propose an incremental Integer Linear Programming formulation of the problem that allows us to enforce linguistic constraints. Our results show only marginal improvements over the non-constrained parser. In addition to the fact that many parses did not violate any constraints in the first place this can be attributed to three reasons: 1) the next best solution that fulfils the constraints yields equal or less accuracy, 2) noisy POS tags and 3) occasionally our inference algorithm was too slow and decoding timed out.

A Distributed Representation Based Query Expansion Approach for Image Captioning

In this paper, we propose a novel query expansion approach for improving transferbased automatic image captioning. The core idea of our method is to translate the given visual query into a distributional semantics based form, which is generated by the average of the sentence vectors extracted from the captions of images visually similar to the input image. Using three image captioning benchmark datasets, we show that our approach provides more accurate results compared to the state-of-theart data-driven methods in terms of both automatic metrics and subjective evaluation.

TasvirEt: A benchmark dataset for automatic Turkish description generation from images

Automatically describing images with natural sentences is considered to be a challenging research problem that has recently been explored. Although the number of methods proposed to solve this problem increases over time, since the datasets used commonly in this field contain only English descriptions, the studies have mostly been limited to single language, namely English. In this study, for the first time in the literature, a new dataset is proposed which enables generating Turkish descriptions from images, which can be used as a benchmark for this purpose. Furthermore, two approaches are proposed, again for the first time in the literature, for image captioning in Turkish with the dataset we named as TasvirEt. Our findings indicate that the new Turkish dataset and the approaches used here can be successfully used for automatically describing images in Turkish.

Initial Explorations of CCG Supertagging for Universal Dependency Parsing.

In this paper we describe the system by METU team for universal dependency parsing of multilingual text. We use a neu- ral network-based dependency parser that has a greedy transition approach to dependency parsing. CCG supertags contain rich structural information that proves useful in certain NLP tasks. We experiment with CCG supertags as additional features in our experiments. The neural network parser is trained together with dependencies and simplified CCG tags as well as other features provided.

Data-driven image captioning via salient region discovery

In the past few years, automatically generating descriptions for images has attracted a lot of attention in computer vision and natural language processing research. Among the existing approaches, data-driven methods have been proven to be highly effective. These methods compare the given image against a large set of training images to determine a set of relevant images, then generate a description using the associated captions. In this study, the authors propose to integrate an object-based semantic image representation into a deep features-based retrieval framework to select the relevant images. Moreover, they present a novel phrase selection paradigm and a sentence generation model which depends on a joint analysis of salient regions in the input and retrieved images within a clustering framework. The authors demonstrate the effectiveness of their proposed approach on Flickr8K and Flickr30K benchmark datasets and show that their model gives highly competitive results compared with the state-of-the-art models.

Toponym recognition on Turkish tweets

In recent years, Twitter has become a popular platform for following and spreading trends, news and ideas all over the world. Geographical scope of tweets is crucial to many tasks like disaster management, event tracking and information retrieval. First step for assigning a geographical location to a tweet is toponym recognition. Toponym Recognition (Geoparsing) is identification of toponyms (place names) in a text. In this study, we investigated performance of three existing approaches for toponym recognition on Turkish tweets. We conducted experiments for measuring performance of the existing approaches on a sample data set. Best results have been obtained with the NER algorithm by Küçük et.al. However, we observed that existing NER algorithms for Turkish neglect the syntactic and semantic features of text.

Contrastive Max-Sum Opinion Summarization

People can reach all kinds of information online incuding reviews and comments on products, movies, holiday destinations and so on. However, one usually need to go through the reviews to have an objective opinion the positive and the negative aspects of the item reviewed. We aim to provie a method that will extract positive and negative opinions on a specific aspect and compare them in an attempt to ease on the information overflow. Contrastive opinion summarization (COS) aims to solve this issue. COS methods extract representative and comparative sentences in terms of specific aspects of a product. In this paper, we propose a new COS method, namely Contrastive Max-Sum Opinion Summarization (CMSOS). This method considers representativeness and contrastiveness at the same time. For the evaluation, we use an English dataset which was specifically created for COS studies. In addition, we created a new dataset in Turkish and shared it publicly. We provide the results on both datasets with our method.

Wide-Coverage Parsing, Semantics, and Morphology

Wide-coverage parsing poses three demands: broad coverage over preferably free text, depth in semantic representation for purposes such as inference in question answering, and computational efficiency. We show for Turkish that these goals are not inherently contradictory when we assign categories to sub-lexical elements in the lexicon. The presumed computational burden of processing such lexicons does not arise when we work with automata-constrained formalisms that are trainable on word-meaning correspondences at the level of predicate-argument structures for any string, which is characteristic of radically lexicalizable grammars. This is helpful in morphologically simpler languages too, where word-based parsing has been shown to benefit from sub-lexical training.