Siddika Parlak

Turkish Broadcast News Transcription and Retrieval

This paper summarizes our recent efforts for building a Turkish Broadcast News transcription and retrieval system. The agglutinative nature of Turkish leads to a high number of out-of-vocabulary (OOV) words which in turn lower automatic speech recognition (ASR) accuracy. This situation compromises the performance of speech retrieval systems based on ASR output. Therefore using a word-based ASR is not adequate for transcribing speech in Turkish. To alleviate this problem, various sub-word-based recognition units are utilized. These units solve the OOV problem with moderate size vocabularies and perform even better than a 500 K word vocabulary as far as recognition accuracy is concerned. As a novel approach, the interaction between recognition units, words and sub-words, and discriminative training is explored. Sub-word models benefit from discriminative training more than word models do, especially in the discriminative language modeling framework. For speech retrieval, a spoken term detection system based on automata indexation is utilized. As with transcription, retrieval performance is measured under various schemes incorporating words and sub-words. Best results are obtained using a cascade of word and sub-word indexes together with term-specific thresholding.

Spoken term detection for Turkish Broadcast News

In this paper, we present a baseline spoken term detection (STD) system for Turkish broadcast news. The agglutinative structure of Turkish causes a high out-of-vocabulary (OOV) rate and increases word error rate (WER) in automatic speech recognition. Several approaches are attempted to reduce this negative effect on the STD system. Sub-word units are used to handle the OOV queries and lattice-based indexing is used to obtain different operating points and handle high WER cases. A recently proposed method for setting term specific thresholds is also evaluated and extended to allow us to choose an operating point suitable for our needs. Best results are obtained by using a cascade of word and sub-word lattice indices with term-thresholding.

Introducing RFID technology in dynamic and time-critical medical settings

We describe the process of introducing RFID technology in the trauma bay of a trauma center to support fast-paced and complex teamwork during resuscitation. We analyzed trauma resuscitation tasks, photographs of medical tools, and videos of simulated resuscitations to gain insight into resuscitation tasks, work practices and procedures. Based on these data, we discuss strategies for placing RFID tags on medical tools and for placing antennas in the environment for optimal tracking and activity recognition. Results from our preliminary RFID deployment in the trauma bay show the feasibility of our approach for tracking tools and for recognizing trauma team activities. We conclude by discussing implications for and challenges to introducing RFID technology in other similar settings characterized by dynamic and collocated collaboration.

Performance Analysis and Improvement of Turkish Broadcast News Retrieval

This paper presents our work on the retrieval of spoken information in Turkish. Traditional speech retrieval systems perform indexing and retrieval over automatic speech recognition (ASR) transcripts, which include errors either because of out-of-vocabulary (OOV) words or ASR inaccuracy. We use subword units as recognition and indexing units to reduce the OOV rate and index alternative recognition hypotheses to handle ASR errors. Performance of such methods is evaluated on our Turkish Broadcast News Corpus with two types of speech retrieval systems: a spoken term detection (STD) and a spoken document retrieval (SDR) system. To evaluate the SDR system, we also build a spoken information retrieval (IR) collection, which is the first for Turkish. Experiments showed that word segmentation algorithms are quite useful for both tasks. SDR performance is observed to be less dependent on the ASR component, whereas any performance change in ASR directly affects STD. We also present extensive analysis of retrieval performance depending on query length, and propose length-based index combination and thresholding strategies for the STD task. Finally, a new approach, which depends on the detection of stems instead of complete terms, is tried for STD and observed to give promising results. Although evaluations were performed in Turkish, we expect the proposed methods to be effective for similar languages as well.

SPEECH AND SLIDING TEXT AIDED SIGN RETRIEVAL FROM HEARING IMPAIRED SIGN NEWS VIDEOS

The objective of this study is to automatically extract annotated sign data from the broadcast news recordings for the hearing impaired. These recordings present an excellent source for automatically generating annotated data: In news for the hearing impaired, the speaker also signs with the hands as she talks. On top of this, there is also corresponding sliding text superimposed on the video. The video of the signer can be segmented via the help of either the speech or both the speech and the text, generating segmented, and annotated sign videos. We call this application as Signiary, and aim to use it as a sign dictionary where the users enter a word as text and retrieve sign videos of the related sign. This application can also be used to automatically create annotated sign databases that can be used for training recognizers.

Non-intrusive localization of passive RFID tagged objects in an indoor workplace

This paper presents our work on localizing a passive UHF RFID tagged object in an indoor workplace. We focus on uncontrolled settings with random orientations of the target object, dynamically moving people in the environment and cluttered rooms with many furniture items. Multiple fixed antennas are used to handle random tag orientations and human body effects. The antennas are placed in a way to minimize the obstruction for human activities and the effect of human presence and movement on the localization system. We use zone-based and exact localization methods incorporating probabilistic and deterministic machine learning techniques. We also propose a combined coarse-to-fine approach to improve accuracy and increase speed. Experimental results show that our system is able to localize an object with an error of 37 cm for exact localization and with an accuracy of 92% for zone-based classification. Experiments in challenging conditions showed that our overall design is robust to human body effects, even exploits the destructive effects of human body on UHF RFID sensing.

Detecting Object Motion Using Passive RFID: A Trauma Resuscitation Case Study

We studied object motion detection in an indoor environment using RFID technology. Unlike prior work, we focus on dynamic scenarios, such as emergency medical situations, subject to signal interference by people and many RFID tags. We build a realistic trauma resuscitation setting and record a dataset of around 14000 detection instances. We find that factors affecting radio signal, such as tag motion, have different statistical fingerprints, making them discernible using statistical methods. Our method for object motion detection extracts descriptive features of the received signal strength and classifies them using machine-learning techniques. We report experimental results obtained with several statistical features and classifiers, and provide guidelines for feature and classifier selection in different environments. Experimental results show that object motion could be detected with an accuracy of 80% in complex scenarios and 90% on average. The motion type, on the other hand, could not be identified with such high accuracy using currently available passive RFID technology.

Comparison of language modeling approaches for Turkish Broadcast News

In this paper, we investigate the performance of several language modeling approaches on a speech recognition system for Turkish broadcast news. The agglutinative structure of Turkish introduces a high out-of-vocabulary rate and hence increases word error rate. To eliminate out-of-vocabulary problem, we utilize various sub-word models. In addition, we experiment with high vocabulary sizes. Since the models are statistical, we expect an improvement in performance as the amount of training data increases. We build word and sub-word language models using various amounts of corpora and compare their recognition performance.

Passive RFID for Object and Use Detection during Trauma Resuscitation

We evaluated passive radio-frequency identification (RFID) technology for detecting the use of objects and related activities during trauma resuscitation. Our system consists of RFID tags and antennas, optimally placed for object detection, as well as algorithms for processing RFID data to infer object use. To evaluate our approach, we tagged 81 objects in the resuscitation room and recorded RFID signal strength during 32 simulated resuscitations performed by trauma teams. We then analyzed RFID data to identify cues for recognizing resuscitation activities. Using these cues, we extracted descriptive features and applied machine-learning techniques to monitor interactions with objects. Our results show that an instance of a used object can be detected with accuracy rates greater than 90 percent in a crowded and fast-paced medical setting using off-the-shelf RFID equipment, and the time and duration of use can be identified with up to 83 percent accuracy. We conclude with insights into the limitations of passive RFID and areas in which RFID needs to be complemented with other sensing technologies.

Design and Evaluation of RFID Deployments in a Trauma Resuscitation Bay

We examined configuring a radio frequency identification (RFID) equipment for the best object use detection in a trauma bay. Unlike prior work on RFID, we 1) optimized the accuracy of object use detection rather than just object detection; and 2) quantitatively assessed antenna placement while addressing issues specific to tag placement likely to occur in a trauma bay. Our design started with an analysis of the environment requirements and constraints. We designed several antenna setups with different number of components (RFID tags or antennas) and their orientations. Setups were evaluated under scenarios simulating a dynamic medical setting. We used three metrics with increasing complexity and bias: read rate, received signal strength indication distribution distance, and target application performance. Our experiments showed that antennas above the regions with high object density are most suitable for detecting object use. We explored tagging strategies for challenging objects so that sufficient readout rates are obtained for computing evaluation metrics. Among the metrics, distribution distance was correlated with target application performance, and also less biased and simpler to calculate, which made it an excellent metric for context-aware applications. We present experimental results obtained in the real trauma bay to validate our findings.