Siripen Pongpaichet

EventShop: recognizing situations in web data streams

Web Observatories must address fundamental societal challenges using enormous volumes of data being created due to the significant progress in technology. The proliferation of heterogeneous data streams generated by social media, sensor networks, internet of things, and digitalization of transactions in all aspect of humans? life presents an opportunity to establish a new era of networks called Social Life Networks (SLN). The main goal of SLN is to connect People to Resources effectively, efficiently, and promptly in given Situations. Towards this goal, we present a computing framework, called EventShop, to recognize evolving situations from massive web streams in real-time. These web streams can be fundamentally considered as spatio-temporal-thematic streams and can be combined using a set of generic spatio-temporal analysis operators to recognize evolving situations. Based on the detected situations, the relevant information and alerts can be provided to both individuals and organizations. Several examples from the real world problems have been developed to test the efficacy of EventShop framework.

Situation fencing: making geo-fencing personal and dynamic

Geo-fencing has recently been applied to multiple applications including media recommendation, advertisements, wildlife monitoring, and recreational activities. However current geo-fencing systems work with static geographical boundaries. Situation Fencing allows for these boundaries to vary automatically based on situations derived by a combination of global and personal data streams. We present a generic approach for situation fencing, and demonstrate how it can be operationalized in practice. The results obtained in a personalized allergy alert application are encouraging and open door for building thousands of similar applications using the same framework in near future.

Using Photos as Micro-Reports of Events

Photos serve dual role. Photos are important for capturing, saving, sharing, and reminiscing memories of events and people. Modern photos, however, are becoming more spontaneous, objective, compelling, and universal reports of a moment in an event also. In this paper our focus is on millions of photos being captured as informative reports and using them for emerging applications including situation recognition, trend analysis, and cultural dynamics. EventShop is an open source platform for situation recognition. Utilizing this platform and using a stream of photo reports from various sources as one of the data streams in this platform, we build a visual analytics system to understand the information that could be gleaned from such photo report streams. Our early experiments are based on the Yahoo Flickr Creative Commons 100 Million photos set released recently. We are also using other sources to import and understand the efficacy of these reports for various important applications.

Geospatial interpolation analytics for data streams in eventshop

EventShop is an open-source software which provides a generic infrastructure for the analysis of heterogeneous spatio-temporal data streams. Efficient interpolation of data from spatially sparse sources is critical but currently missing in EventShop. To address this challenge, we implement a Spatial Gaussian Process based statistical operator into the EventShop framework. Spectral analysis is employed to generate features at higher spatial resolution and to improve interpolation accuracy at unsampled locations. Further, we test this operator by interpolating air pollution levels in California. The evaluations of multiple metrics demonstrate that our operators outperform earlier EventShop operators, chemical transportation models, and state-of-the-art methods.

A graph based multimodal geospatial interpolation framework

Recent multimedia research has increasingly focused on large scale multimodal data from disparate geospatial sensors. In addition to the volume of the data, the diversity and granularity of the data poses a major challenge in extracting meaningful and actionable information. To address this, we present a novel spatial interpolation framework, capable of incorporating multimodal data sources and modeling the spatial processes comprehensively at multiple resolutions. The framework transforms the spatial interpolation problem into a graph structure learning problem, based on the latent structure of the data. This enables more efficient and accurate predictions at unobserved locations. We demonstrate the effectiveness of our approach by testing it on air pollution interpolation.

Integration of Diverse Data Sources for Spatial PM2.5 Data Interpolation

Heterogeneous data fusion from disparate geospatial sensors has drawn increasing attention in multimedia. Unfortunately, environmental sensors are usually sparsely and preferentially located, which restricts situation recognition of geographical regions and results in uncertainty in derived inferences. Spatial interpolation is an effective way to solve the problem of data sparsity, which demands the availability of related data sources. However, these data sources are usually in different resolutions, distributions, scales, and densities, which poses a major challenge in data integration. To address this problem, we present a novel spatial interpolation framework to incorporate diverse data sources and model the spatial processes explicitly at multiple resolutions. Spectral analysis is deployed to generate features at multiple spatial resolutions and to improve the interpolation accuracy at unobserved locations. A statistical operator based on the spatial Gaussian process is implemented and integrated into a geospatial situation recognition system, which can analyze heterogeneous spatio-temporal data streams derived from sensors. To verify the effectiveness and efficiency of the proposed framework, this framework is applied to the PM2.5 air pollution application. Experiments conducted in California, USA, demonstrate that the proposed method outperforms state-of-the-art approaches.

Research Challenges in Developing Multimedia Systems for Managing Emergency Situations

With an increasing amount of diverse heterogeneous data and information, the methodology of multimedia analysis has become increasingly relevant in solving challenging societal problems such as managing emergency situations during disasters. Using cybernetic principles combined with multimedia technology, researchers can develop effective frameworks for using diverse multimedia (including traditional multimedia as well as diverse multimodal) data for situation recognition, and determining and communicating appropriate actions to people stranded during disasters. We present known issues in disaster management and then focus on emergency situations. We show that an emergency management problem is fundamentally a multimedia information assimilation problem for situation recognition and for connecting peoples needs to available resources effectively, efficiently, and promptly. Major research challenges for managing emergency situations are identified and discussed. We also present a intelligently detecting evolving environmental situations, and discuss the role of multimedia micro-reports as spontaneous participatory sensing data streams in emergency responses. Given enormous progress in concept recognition using machine learning in the last few years, situation recognition may be the next major challenge for learning approaches in multimedia contextual big data. The data needed for developing such approaches is now easily available on the Web and many challenging research problems in this area are ripe for exploration in order to positively impact our society during its most difficult times.

Situation Recognition from Multimodal Data

Situation recognition is the problem of deriving actionable insights from heterogeneous, real-time, big multimedia data to benefit human lives and resources in different applications. This tutorial will discuss the recent developments towards converting multitudes of data streams including weather patterns, stock prices, social media, traffic information, and disease incidents into actionable insights.

Exploring spatio-temporal-theme correlation between physical and social streaming data for event detection and pattern interpretation from heterogeneous sensors

In this paper, we introduce a new method that explores spatio-temporal-theme correlations between physical and social streaming data for event detection and pattern interpretation from heterogeneous sensors. Particularly, we employ a basic two-phase framework in pattern recognition (i.e. feature extraction and detection) with the novel improvement that concerns the use of semantic information acquired from social sensors to automatically label the low-level features extracted from physical sensors. Moreover, by symbolizing the trend component of time-series data, the proposed method has an ability to interpret events patterns to help users get insights of how events happen. Differentiating from conventional supervised learning methods whose training data are labeled manually and in an off-line mode, the proposed method can collect labels for training data automatically and in an on-line mode. Moreover, after running for a certain time, a training stage can run parallel with the detecting stage when an event model is totally built. After that, the training stage continues learning to increase the accuracy of the event model by nonstop collecting new samples with labels from streaming data. The problem of environmental factors and particularly air pollution impacts on asthma exacerbation is considered for evaluating the proposed method. The experimental results show that the proposed method can probably detect the prevalence of asthma risks in a specific spatio-temporal context as well as help users understand how a change in the surrounding environment (e.g. weather condition and air pollution) can influence their health (e.g. asthma attack) by interpreting detected events patterns.