Zhengwei Qiu

The smartphone as a platform for wearable cameras in health research.

BACKGROUND The Microsoft SenseCam, a small camera that is worn on the chest via a lanyard, increasingly is being deployed in health research. However, the SenseCam and other wearable cameras are not yet in widespread use because of a variety of factors. It is proposed that the ubiquitous smartphones can provide a more accessible alternative to SenseCam and similar devices. PURPOSE To perform an initial evaluation of the potential of smartphones to become an alternative to a wearable camera such as the SenseCam. METHODS In 2012, adults were supplied with a smartphone, which they wore on a lanyard, that ran life-logging software. Participants wore the smartphone for up to 1 day and the resulting life-log data were both manually annotated and automatically analyzed for the presence of visual concepts. The results were compared to prior work using the SenseCam. RESULTS In total, 166,000 smartphone photos were gathered from 47 individuals, along with associated sensor readings. The average time spent wearing the device across all users was 5 hours 39 minutes (SD=4 hours 11 minutes). A subset of 36,698 photos was selected for manual annotation by five researchers. Software analysis of these photos supports the automatic identification of activities to a similar level of accuracy as for SenseCam images in a previous study. CONCLUSIONS Many aspects of the functionality of a SenseCam largely can be replicated, and in some cases enhanced, by the ubiquitous smartphone platform. This makes smartphones good candidates for a new generation of wearable sensing devices in health research, because of their widespread use across many populations. It is envisioned that smartphones will provide a compelling alternative to the dedicated SenseCam hardware for a number of users and application areas. This will be achieved by integrating new types of sensor data, leveraging the smartphones real-time connectivity and rich user interface, and providing support for a range of relatively sophisticated applications.

Remote Real-Time Monitoring of Subsurface Landfill Gas Migration

The cost of monitoring greenhouse gas emissions from landfill sites is of major concern for regulatory authorities. The current monitoring procedure is recognised as labour intensive, requiring agency inspectors to physically travel to perimeter borehole wells in rough terrain and manually measure gas concentration levels with expensive hand-held instrumentation. In this article we present a cost-effective and efficient system for remotely monitoring landfill subsurface migration of methane and carbon dioxide concentration levels. Based purely on an autonomous sensing architecture, the proposed sensing platform was capable of performing complex analytical measurements in situ and successfully communicating the data remotely to a cloud database. A web tool was developed to present the sensed data to relevant stakeholders. We report our experiences in deploying such an approach in the field over a period of approximately 16 months.

A real-time life experience logging tool

E-memories attempt to digitally encode all life experiences in an archive for later search and real-time recommendation. In this paper we describe a prototype real-time e-memory gathering infrastructure and system, that uses smartphones to gather and organise semantically rich e-memory.

Mining user activity as a context source for search and retrieval

Nowadays in information retrieval it is generally accepted that if we can better understand the context of searchers then this could help the search process, either at indexing time by including more metadata or at retrieval time by better modelling the user needs. In this work we explore how activity recognition from tri-axial accelerometers can be employed to model a users activity as a means of enabling context-aware information retrieval. In this paper we discuss how we can gather user activity automatically as a context source from a wearable mobile device and we evaluate the accuracy of our proposed user activity recognition algorithm. Our technique can recognise four kinds of activities which can be used to model part of an individuals current context. We discuss promising experimental results, possible approaches to improve our algorithms, and the impact of this work in modelling user context toward enhanced search and retrieval.

Green multimedia: informing people of their carbon footprint through two simple sensors

In this work we discuss a new, but highly relevant, topic to the multimedia community; systems to inform individuals of their carbon footprint, which could ultimately effect change in community carbon footprint-related activities. The reduction of carbon emissions is now an important policy driver of many governments, and one of the major areas of focus is in reducing the energy demand from the consumers i.e. all of us individually. In terms of CO2 generated from energy consumption, there are three predominant factors, namely electricity usage, thermal related costs, and transport usage. Standard home electricity and heating sensors can be used to measure the former two aspects, and in this paper we evaluate a novel technique to estimate an individuals transport-related carbon emissions through the use of a simple wearable accelerometer. We investigate how providing this novel estimation of transport-related carbon emissions through an interactive web site and mobile phone app engages a set of users in becoming more aware of their carbon emissions. Our evaluations involve a group of 6 users collecting 25 million accelerometer readings and 12.5 million power readings vs. a control group of 16 users collecting 29.7 million power readings.

Term Weighting Approaches for Mining Significant Locations from Personal Location Logs

In this paper, we describe experiments into the application of term weighting techniques from text retrieval to support the automatic identification of significant locations from a large location log, which we consider to be important for supporting many location-based social network applications. We identify the fact that the distribution of locations follows a similar shaped distribution to that of terms in a language and in so doing motivate our use of term weighting techniques. Using this information we then show that these proven techniques can be used to automatically identify social visits and “pass through” locations, as well as standard home and work locations. We also suggest that it is possible to classify whether an extended segment of personal location data may be a tourist trip, business trip or a typical working (at home) period of time.

ZhiWo: activity tagging and recognition system for personal lifelogs

With the increasing use of mobile devices as personal recording, communication and sensing tools, extracting the semantics of life activities through sensed data (photos, accelerometer, GPS etc.) is gaining widespread public awareness. A person who engages in long-term personal sensing is engaging in a process of lifelogging. Lifelogging typically involves using a range of (wearable) sensors to capture raw data, to segment into discrete activities, to annotate and subsequently to make accessible by search or browsing tools. In this paper, we present an intuitive lifelog activity recording and management system called ZhiWo. By using a supervised machine learning approach, sensed data collected by mobile devices are automatically classified into different types of daily human activities and these activities are interpreted as life activity retrieval units for personal archives.

Exploring the optimal visual vocabulary sizes for semantic concept detection

The framework based on the Bag-of-Visual-Words (BoVW) feature representation and SVM classification is popularly used for generic content-based concept detection or visual categorization. However, visual vocabulary (VV) size, one important factor in this framework, is always chosen differently and arbitrarily in previous work. In this paper, we focus on investigating the optimal VV sizes depending on other components of this framework which also govern the performance. This is useful as a default VV size for reducing the computation cost. By unsupervised clustering, a series of VVs covering a wide range of sizes are evaluated under two popular local features, three assignment modes, and four kernels on two different-scale benchmarking datasets respectively. These factors are also evaluated. Experimental results show that best VV sizes vary as these factors change. However, the concept detection performance usually improves as the VV size increases initially, and then gains less, or even deteriorates if larger VVs are used since overfitting occurs. Overall, VVs with sizes ranging from 1024 to 4096 achieve best performance with higher probability when compared with other-size VVs. With regard to the other factors, experimental results show that the OpponentSIFT descriptor outperforms the SURF feature, and soft assignment mode yields better performance than binary and hard assignment. In addition, generalized RBF kernels such as X2 and Laplace RBF kernels are more appropriate for semantic concept detection with SVM classification.

Evaluating Access Mechanisms for Multimodal Representations of Lifelogs

Lifelogging, the automatic and ambient capture of daily life activities into a digital archive called a lifelog, is an increasingly popular activity with a wide range of applications areas including medical memory support, behavioural science analysis of quality of life, work-related auto-recording of tasks and more. In this paper we focus on lifelogging where there is sometimes a need to re-find something from ones past, recent or distant, from the lifelog. To be effective, a lifelog should be accessible across a variety of access devices. In the work reported here we create eight lifelogging interfaces and evaluate their effectiveness on three access devices; laptop, smartphone and e-book reader, for a searching task. Based on tests with 16 users, we identify which of the eight interfaces are most effective for each access device in a known-item search task through the lifelog, for both the lifelog owner, and for other searchers. Our results are important in suggesting ways in which personal lifelogs can be most effectively used and accessed.

Exploring the technical challenges of large-scale lifelogging

Ambiently and automatically maintaining a lifelog is an activity that may help individuals track their lifestyle, learning, health and productivity. In this paper we motivate and discuss the technical challenges of developing real-world lifelogging solutions, based on seven years of experience. The gathering, organisation, retrieval and presentation challenges of large-scale lifelogging are discussed and we show how this can be achieved and the benefits that may accrue.