Hsin-Yu Ha

Data-Driven Techniques in Disaster Information Management

Improving disaster management and recovery techniques is one of national priorities given the huge toll caused by man-made and nature calamities. Data-driven disaster management aims at applying advanced data collection and analysis technologies to achieve more effective and responsive disaster management, and has undergone considerable progress in the last decade. However, to the best of our knowledge, there is currently no work that both summarizes recent progress and suggests future directions for this emerging research area. To remedy this situation, we provide a systematic treatment of the recent developments in data-driven disaster management. Specifically, we first present a general overview of the requirements and system architectures of disaster management systems and then summarize state-of-the-art data-driven techniques that have been applied on improving situation awareness as well as in addressing users’ information needs in disaster management. We also discuss and categorize general data-mining and machine-learning techniques in disaster management. Finally, we recommend several research directions for further investigations.

Hierarchical disaster image classification for situation report enhancement

In this paper, a hierarchical disaster image classification (HDIC) framework based on multi-source data fusion (MSDF) and multiple correspondence analysis (MCA) is proposed to aid emergency managers in disaster response situations. The HDIC framework classifies images into different disaster categories and sub-categories using a pre-defined semantic hierarchy. In order to effectively fuse different sources (visual and text) of information, a weighting scheme is presented to assign different weights to each data resource depending on the hierarchical structure. The experimental analysis demonstrates that the proposed approach can effectively classify disaster images at each logical layer. In addition, the paper also presents an iPad application developed for situation report management using the proposed HDIC framework.

A Multimedia Semantic Retrieval Mobile System Based on HCFGs

A multimedia semantic retrieval system based on hidden coherent feature groups (HCFGs) can support multimedia semantic retrieval on mobile applications. The system can capture the correlation between features and partition the original feature set into HCFGs, which have strong intragroup correlation while maintaining low intercorrelation. The authors present a novel, multimodel fusion scheme to effectively fuse the multimodel results and generate the final ranked retrieval results. In addition, to incorporate user interaction for effective retrieval, the proposed system also features a user feedback mechanism that helps refine the retrieval results.

Correlation-based Feature Analysis and Multi-Modality Fusion framework for multimedia semantic retrieval

In this paper, we propose a Correlation based Feature Analysis (CFA) and Multi-Modality Fusion (CFA-MMF) framework for multimedia semantic concept retrieval. The CFA method is able to reduce the feature space and capture the correlation between features, separating the feature set into different feature groups, called Hidden Coherent Feature Groups (HCFGs), based on Maximum Spanning Tree (MaxST) algorithm. A correlation matrix is built upon feature pair correlations, and then a MaxST is constructed based on the correlation matrix. By performing a graph cut procedure on the MaxST, a set of feature groups are obtained, where the intra-group correlation is maximized and the inter-group correlation is minimized. Finally, one classifier is trained for each of the feature groups, and the generated scores from different classifiers are fused for the final retrieval. The proposed framework is effective because it reduces the dimensionality of the feature space. The experimental results on the NUSWIDE-Lite data set demonstrate the effectiveness of the proposed CFA-MMF framework.

Content-Based Multimedia Retrieval Using Feature Correlation Clustering and Fusion

Nowadays, only processing visual features is not enough for multimedia semantic retrieval due to the complexity of multimedia data, which usually involve a variety of modalities, e.g. graphics, text, speech, video, etc. It becomes crucial to fully utilize the correlation between each feature and the target concept, the feature correlation within modalities, and the feature correlation across modalities. In this paper, the authors propose a Feature Correlation Clustering-based Multi-Modality Fusion Framework FCC-MMF for multimedia semantic retrieval. Features from different modalities are combined into one feature set with the same representation via a normalization and discretization process. Within and across modalities, multiple correspondence analysis is utilized to obtain the correlation between feature-value pairs, which are then projected onto the two principal components. K-medoids algorithm, which is a widely used partitioned clustering algorithm, is selected to minimize the Euclidean distance within the resulted clusters and produce high intra-correlated feature-value pair clusters. Majority vote is applied to subsequently decide which cluster each feature belongs to. Once the feature clusters are formed, one classifier is built and trained for each cluster. The correlation and confidence of each classifier are considered while fusing the classification scores, and mean average precision is used to evaluate the final ranked classification scores. Finally, the proposed framework is applied on NUS-wide Lite data set to demonstrate the effectiveness in multimedia semantic retrieval.

A Visual Analytics Multimedia Mobile System for Emergency Response

We present a novel visual analytics system and multimedia enabled mobile application that allows emergency management (EM) personnel access to timely and relevant disaster situation information. The system is able to semantically integrate text-based emergency management disaster situation reports with related disaster imagery taken in the field by EM responders and community residents. In addition, through an intuitive and seamless Apple iPad application, users are able to interact with the system in diverse places and conditions and thus provide a more effective response. The system is demonstrated via its iPad application which aims at providing relevant and actionable information.

Correlation-Based Deep Learning for Multimedia Semantic Concept Detection

Nowadays, concept detection from multimedia data is considered as an emerging topic due to its applicability to various applications in both academia and industry. However, there are some inevitable challenges including the high volume and variety of multimedia data as well as its skewed distribution. To cope with these challenges, in this paper, a novel framework is proposed to integrate two correlation-based methods, Feature-Correlation Maximum Spanning Tree (FC-MST) and Negative-based Sampling (NS), with a well-known deep learning algorithm called Convolutional Neural Network (CNN). First, FC-MST is introduced to select the most relevant low-level features, which are extracted from multiple modalities, and to decide the input layer dimension of the CNN. Second, NS is adopted to improve the batch sampling in the CNN. Using NUS-WIDE image data set as a web-based application, the experimental results demonstrate the effectiveness of the proposed framework for semantic concept detection, comparing to other well-known classifiers.

FC-MST: Feature correlation maximum spanning tree for multimedia concept classification

Feature selection is an actively researched topic in varies domains, mainly owing to its ability in greatly reducing feature space and associated computational time. Given the explosive growth of high-dimensional multimedia data, a well-designed feature selection method can be leveraged in classifying multimedia contents into high-level semantic concepts. In this paper we present a multi-phase feature selection method using maximum spanning tree built from feature correlation among multiple modalities (FC-MST). The method aims to first thoroughly explore not only the correlation between features within and across modalities, but also the association of features towards semantic concepts. Secondly, with the correlations, we identify important features and exclude redundant or irrelevant ones. The proposed method is tested on a well-known benchmark multimedia data set called NUS-WIDE and the experimental results show that it outperforms four well-known feature selection methods in all three important measurement metrics.

Correlation-based re-ranking for semantic concept detection

Semantic concept detection is among the most important and challenging topics in multimedia research. Its objective is to effectively identify high-level semantic concepts from low-level features for multimedia data analysis and management. In this paper, a novel re-ranking method is proposed based on correlation among concepts to automatically refine detection results and improve detection accuracy. Specifically, multiple correspondence analysis (MCA) is utilized to capture the relationship between a targeted concept and all other semantic concepts. Such relationship is then used as a transaction weight to refine detection ranking scores. To demonstrate its effectiveness in refining semantic concept detection, the proposed re-ranking method is applied to the detection scores of TRECVID 2011 benchmark data set, and its performance is compared with other state-of-the-art re-ranking approaches.

Constraint driven model using correlation and collaborative filtering for sustainable building

Sustainable building has emerged as an important topic due to the fact that it can significantly reduce the impact of buildings and their operation on the natural environment and more efficiently utilize resources throughout a buildings life-cycle. When compared with a traditional buildingdesign process, integrated project delivery has proven to be more efficient, and is thus gaining wider acceptance for many sustainable building projects. However, managing design and construction from different disciplines is still challenging. Conflicts among constraints are often not identified at the right design stage, which results in multiple iterations of the design process. In this paper, a novel constrain-driven model that enhances design processes through better management of constraints and thus delivers optimal design solutions with higher energy performance is proposed. Multiple Correspondence Analysis was applied to capture the correlations between different items (parameter-value pairs) and classes (constraints). Meanwhile, it integrated Collaborative Filtering methods and Constraint Satisfaction Problem to train and refine the proposed model. Finally, we have applied our model to a synthetic data sets to demonstrate its performance.