Melita Hajdinjak

The PARADISE Evaluation Framework: Issues and Findings

There has been a great deal of interest over the past 20 years in developing metrics and frameworks for evaluating and comparing the performance of spoken-language dialogue systems. One of the results of this interest is a potential general methodology, known as the PARADISE framework. This squib highlights some important issues concerning the application of PARADISE that have, up to now, not been sufficiently emphasized or have even been neglected by the dialogue-system community. These include considerations regarding the selection of appropriate regression parameters, normalization effects on the accuracy of the prediction, the influence of speech-recognition errors on the performance function, and the selection of an appropriate user-satisfaction measure. In addition, it gives the results of an evaluation of data from two Wizard-of-Oz experiments. These evaluations include different dependent variables and examination of individual user-satisfaction measures.

Visual re-identification across large, distributed camera networks

We propose a holistic approach to the problem of re-identification in an environment of distributed smart cameras. We model the re-identification process in a distributed camera network as a distributed multi-class classifier, composed of spatially distributed binary classifiers. We treat the problem of re-identification as an open-world problem, and address novelty detection and forgetting. As there are many tradeoffs in design and operation of such a system, we propose a set of evaluation measures to be used in addition to the recognition performance. The proposed concept is illustrated and evaluated on a new many-camera surveillance dataset and SAIVT-SoftBio dataset. Display Omitted Formalization of object re-identification problem in a distributed environmentRe-identification treated as an open-world problemNovelty detection and forgetting included in the schemeA set of performance measures, geared towards open-world, distributed surveillanceExperiments on a many-camera (36) surveillance dataset and publicly available source code

Information-Providing Dialogue Management

The central module of any natural language dialogue system is the dialogue manager, which plays the role of an intermediate agent between the user and the information source. Its cooperativity and portability highly determines the efficiency of the dialogue system. Therefore, as the basis for cooperativity of information-providing dialogue systems we propose a knowledge representation of the information source based on intuitionistic modal logic. For modeling of the dialogue flow we use conversational game theory, which, on the other hand, significantly increases the portability.

The Wizard of Oz System for Weather Information Retrieval

We introduce first steps of constructing a natural language dialogue system for weather information retrieval. Data for speech recognition, speech understanding and construction of the dialogue manager is being collected with a Wizard-of-Oz (WOZ) system. The hearth of the WOZ system is a graphical interface which is connected to the system’s database, the language generation module and the Text-to-Speech module. The WOZ simulation results are presented and a brief description of our spontaneous speech database is given. ...

Vehicle interconnection metric and clustering protocol for improved connectivity in vehicular ad hoc networks

Communication is the main driving force behind the emerging intelligent transportation systems, which are expected to make traveling safer, more ecological, and faster. The most challenging among all the different communication technologies that will be used are the direct vehicle-to-vehicle communications, because vehicles move with high speeds and in different directions. Apart from that, it is expected that vehicles should be able to organize themselves in an ad hoc network without any assistance from outside entities, such as road side units. To make the ad hoc network less dynamic and communications more reliable, vehicles with similar movement patterns can be grouped together in clusters. Clustering is a well-known method for organizing ad hoc networks and is used in mobile and wireless sensor networks, but with different constraints and goals, so new clustering solutions for vehicular ad hoc networks (VANET) have to be developed. In recent years, this has been a hot topic among researchers and many different clustering algorithms for VANET have been proposed. In this paper, we propose a new clustering metric for VANET, named vehicle interconnection metric, which is based on sending periodic beacons among vehicles and reflects the communication abilities between them. We also propose a new clustering algorithm whose primary goal is increased connectivity and lower number of disconnects. The working principle of this algorithm is also inverted compared to others and uses unneeded cluster head elimination instead of cluster head election. Mathematical analysis of memory usage and communication overhead are provided, predicting low-resource usage. Simulation results, obtained with the ns-3 network simulator and the SUMO vehicle movement simulator, have confirmed the analysis and expected performance in terms of cluster head duration, number of connectivity losses and role switches.

K-Relations and Beyond

Although the theory of relational databases is highly developed and proves its usefulness in practice every day Garcia-Molina et al. (2008), there are situations where the relational model fails to offer adequate formal support. For instance, when querying approximate dataHjaltason & Brooks (2003); Minker (1998) or data within a given range of distance or similarityHjaltason & Brooks (2003); Patella & Ciaccia (2009). Examples of such similarity-search applications are databases storing images, fingerprints, audio clips or time sequences, text databases with typographical or spelling errors, and text databases where we look for documents that are similar to a given document. A core component of such cooperative systems is a treatment of imprecise data Hajdinjak & Mihelic (2006); Minker (1998).

SIMILARITY-BASED RELATIONS IN DATALOG PROGRAMS

We consider similarity-based relational databases that allow to retrieve approximate data, find data within a given range of distance or similarity, and support imprecise queries. We focus on the recently introduced relational algebra with similarities on -relations, which are annotated with multi-dimensional similarity values with each dimension referring to a single attribute. The codomains of the annotated relations are De Morgan frames, and the annotations express the relevance of the tuples as answers to a similarity-based query. In this paper, we study Datalog programs on -relations, with and without negation. We describe the least-fixpoint algorithm for safe and rectified Datalog programs on -relations with finite support but without negative literals in the body. We further describe the perfect-minimal-fixpoint algorithm of a Datalog program on -relations with finite support and negative literals in the body when rules are safe, rectified and stratified. We introduce the idea of controlling the calculation of the annotations such that the tuples that enter an IDB relation last will be announced less desirable than those that enter first. For this we define a damping function that augments/diminishes the individual annotations that contribute to the final annotations of tuples. With a damping function, for instance, long chains of inferences may be made significantly less desirable or even totally undesirable.

A wizard-of-Oz system evaluation study

In order to evaluate the performance of the dialogue-manager component of a developing, Slovenian and Croatian spoken dialogue system, two Wizard-of-Oz experiments were performed. The only difference between the two experiment settings was in the dialogue-management manner, i.e., while in the first experiment dialogue management was performed by a human, the wizard, in the second experiment it was performed by the newly-implemented dialogue-manager component. The data from both Wizard-of-Oz experiments was evaluated with the PARADISE evaluation framework, a potential general methodology for evaluating and comparing different versions of spoken-language dialogue systems. The study ascertains a remarkable difference in the performance functions when taking different satisfaction-measure sums or even individual scores as the target to be predicted, it proves the indispensableness of the recently introduced database parameters when evaluating information-providing dialogue systems, and it confirms the dialogue managers cooperativity subject to the incorporated knowledge representation.