Laleh Jalali

Using Photos as Micro-Reports of Events

Photos serve dual role. Photos are important for capturing, saving, sharing, and reminiscing memories of events and people. Modern photos, however, are becoming more spontaneous, objective, compelling, and universal reports of a moment in an event also. In this paper our focus is on millions of photos being captured as informative reports and using them for emerging applications including situation recognition, trend analysis, and cultural dynamics. EventShop is an open source platform for situation recognition. Utilizing this platform and using a stream of photo reports from various sources as one of the data streams in this platform, we build a visual analytics system to understand the information that could be gleaned from such photo report streams. Our early experiments are based on the Yahoo Flickr Creative Commons 100 Million photos set released recently. We are also using other sources to import and understand the efficacy of these reports for various important applications.

An intelligent notification system using context from real-time personal activity monitoring

People can now receive custom-made information through smartphones, tablets or wearable devices. However, people often tend to miss vital information, even reminders, in the flood of notifications. The problem of finding convenient moments for need-to-know information should be investigated. Because each persons message awareness pattern on a smart medium might be different, the necessity of personalized notification time should be emphasized. We believe that tracking changes in a users physical activity and other contextual factors will reveal the most convenient moments. We propose a mobile framework, smartNoti, to carefully examine the user environment. The main contributions of our framework are: 1) developing an architecture to provide crucial information in a timely manner at a recognizable moment; 2) integrating, processing, training, and storing personalized latent features from heterogeneous data streams; 3) detecting user context transitions that might provide recognizable and available moments; and 4) predicting these moment and providing a notification message. The experimental validation on Intelligent Callback Reminder, which we implemented on an android application to notify a user missed or rejected call, demonstrates that our approach is effective. We believe that our findings can lead to intelligent strategies to issue unobtrusive notifications on todays smart phones at no extra cost, by using sensors and contextual factors.

Building health persona from personal data streams

Most people already use phones with myriad sensors that continuously generate data streams related to most aspects of their life. By detecting events in basic data streams and correlating and reasoning among them, it is possible to create a chronicle of personal life. We call it Personicle and use this to build individual Health Persona. Such Health Persona may then be used for understanding societal health as well as making decisions in emerging Social Life Networks. In this paper, we present a framework that collects, manages, and correlates personal data from heterogeneous data sources and detects events happening at personal level to build health persona. We use several data streams such as motion tracking, location tracking, activity level, and personal calendar data. We illustrate how two recognition algorithms based on Formal Concept Analysis and Decision Trees can be applied to Life Event detection problem. Also, we demonstrate the applicability of this framework on simulated data from Moves app, GPS, Nike fuel band, and Google calendar. We expect to soon have results for several individuals using real data streams from disparate wearable and smart phone sensors.

Bringing Deep Causality to Multimedia Data Streams

We live in a data abundance era. Availability of large volume of diverse multimedia data streams (ranging from video, to tweets, to activity, and to PM2.5) can now be used to solve many critical societal problems. Causal modeling across multimedia data streams is essential to reap the potential of this data. However, effective frameworks combining formal abstract approaches with practical computational algorithms for causal inference from such data are needed to utilize available data from diverse sensors. We propose a causal modeling framework that builds on data-driven techniques while emphasizing and including the appropriate human knowledge in causal inference. We show that this formal framework can help in designing a causal model with a systematic approach that facilitates framing sharper scientific questions, incorporating experts knowledge as causal assumptions, and evaluating the plausibility of these assumptions. We show the applicability of the framework in a an important Asthma management application using meteorological and pollution data streams.

From health-persona to societal health

In this position paper, we propose an approach for Web Observatories that builds on using social media, personal data, and sensors to build Persona for an individual, but also use this data and the concept of Focused Micro Blogs (FMB) for situation detection, helping individual using situation action rules, and finally gaining insights for obtaining insights about society. We demonstrate this in a concrete use case of fitness and health care related sensors for building health persona and using this for understanding societal health issues.

Multimedia for personal health and health care

Ever since the emergence of digitization, the term multimedia has been used to represent a combination of different kinds of media types, such images, audio, and videos. As new sensing technologies emerge and are now becoming omnipresent in daily lives, the definition, role and significance of multimedia is changing. Multimedia now represents the means for communicating, cooperating, and also for monitoring numerous aspects of daily life, at various levels of granularity and application, ranging from personal to societal. With this shift, we have since moved from comprehending single media and its state toward comprehending media in terms of its use context. Multimedia is thus no longer confined to documentation and preservation, entertainment or personal media collections; rather, it has become an integral part of the tools and systems that are providing solutions to todays societal challenges-including challenges related to health care and personal health , aging, education, societal participation, sustainable energy, and intelligent transportation. Multimedia has thus evolved into a core enabler for future interactive and cooperative applications at the heart of society. In this workshop we explore the relevance and contribution of multimedia to health care and personal media.

Exploring spatio-temporal-theme correlation between physical and social streaming data for event detection and pattern interpretation from heterogeneous sensors

In this paper, we introduce a new method that explores spatio-temporal-theme correlations between physical and social streaming data for event detection and pattern interpretation from heterogeneous sensors. Particularly, we employ a basic two-phase framework in pattern recognition (i.e. feature extraction and detection) with the novel improvement that concerns the use of semantic information acquired from social sensors to automatically label the low-level features extracted from physical sensors. Moreover, by symbolizing the trend component of time-series data, the proposed method has an ability to interpret events patterns to help users get insights of how events happen. Differentiating from conventional supervised learning methods whose training data are labeled manually and in an off-line mode, the proposed method can collect labels for training data automatically and in an on-line mode. Moreover, after running for a certain time, a training stage can run parallel with the detecting stage when an event model is totally built. After that, the training stage continues learning to increase the accuracy of the event model by nonstop collecting new samples with labels from streaming data. The problem of environmental factors and particularly air pollution impacts on asthma exacerbation is considered for evaluating the proposed method. The experimental results show that the proposed method can probably detect the prevalence of asthma risks in a specific spatio-temporal context as well as help users understand how a change in the surrounding environment (e.g. weather condition and air pollution) can influence their health (e.g. asthma attack) by interpreting detected events patterns.

Complex asthma risk factor recognition from heterogeneous data streams

There are many studies regarding the relationships between environmental factors, particularly air pollution, and asthma exacerbation. Most of these studies ignore the potential confounding effects of a sequence of these factors with a specific time lag between them and asthma outbreaks. In this paper we present a new method for identifying consequential relations in the form of complex patterns between environmental factors and asthma attacks. Temporal structure and order relation between these data and their effect on asthma exacerbation comprise complex patterns called asthma risk factors. By extracting such patterns we create a risk prediction model that is important both for an asthmatic patient and public health. For experimental evaluations, we have collected pollution and meteorological data in Tokyo city and found 32 complex risk factor patterns that might result in asthma outbreaks. The experimental results show that extracted model has 71.15% precision.