Longqi Yang

Inferring occupancy from opportunistically available sensor data

Commercial and residential buildings are usually instrumented with meters and sensors that are deployed as part of a utility infrastructure installed by companies that provide services such as electricity, water, gas, security, phone, etc. As part of their normal operation, these service providers have direct access to information from the sensors and meters. A concern arises that the sensory information collected by the providers, although coarse-grained, can be subject to analysis that reveals private information about the users of the building. Oftentimes, multiple services are provided by the same company, in which case the potential for leakage of private information increases. Our research seeks to investigate the extent to which easily available sensory information may be used by external service providers to make occupancy-related inferences. Particularly, we focus on inferences from two different sources: motion sensors, which are installed and monitored by security companies, and smart electric meters, which are deployed by electric companies for billing and demand-response management. We explore the motion sensor scenario in a three-person single-family home and the electric meter scenario in a twelve-person university lab. Our exploration with various inference methods shows that sensory information available to service providers can enable them to make undesired occupancy related inferences, such as levels of occupancy or even the identities of current occupants, significantly better than naive prediction strategies that do not make use of sensor information.

Yum-Me: A Personalized Nutrient-Based Meal Recommender System

Nutrient-based meal recommendations have the potential to help individuals prevent or manage conditions such as diabetes and obesity. However, learning people’s food preferences and making recommendations that simultaneously appeal to their palate and satisfy nutritional expectations are challenging. Existing approaches either only learn high-level preferences or require a prolonged learning period. We propose Yum-me, a personalized nutrient-based meal recommender system designed to meet individuals’ nutritional expectations, dietary restrictions, and fine-grained food preferences. Yum-me enables a simple and accurate food preference profiling procedure via a visual quiz-based user interface and projects the learned profile into the domain of nutritionally appropriate food options to find ones that will appeal to the user. We present the design and implementation of Yum-me and further describe and evaluate two innovative contributions. The first contriution is an open source state-of-the-art food image analysis model, named FoodDist. We demonstrate FoodDist’s superior performance through careful benchmarking and discuss its applicability across a wide array of dietary applications. The second contribution is a novel online learning framework that learns food preference from itemwise and pairwise image comparisons. We evaluate the framework in a field study of 227 anonymous users and demonstrate that it outperforms other baselines by a significant margin. We further conducted an end-to-end validation of the feasibility and effectiveness of Yum-me through a 60-person user study, in which Yum-me improves the recommendation acceptance rate by 42.63%.

Personalizing Software and Web Services by Integrating Unstructured Application Usage Traces

Users of software applications generate vast amounts of unstructured log-trace data. These traces contain clues to the intentions and interests of those users, but service providers may find it difficult to uncover and exploit those clues. In this paper, we propose a framework for personalizing software and web services by leveraging such unstructured traces. We use 6 months of Photoshop usage history and 7 years of interaction records from 67K Behance users to design, develop, and validate a user-modeling technique that discovers highly discriminative representations of Photoshop users; we refer to the model as cutilization-to-vector, util2vec. We demonstrate the promise of this approach for three sample applications: (1) a practical user-tagging system that automatically predicts areas of focus for millions of Photoshop users; (2) a two-phase recommendation model that enables cold-start personalized recommendations for many new Behance users who have Photoshop usage data, improving recommendation quality (Recall@100) by 21.2% over a popularity-based recommender; and (3) a novel inspiration engine that provides real-time personalized inspirations to artists. We believe that this work demonstrates the potential impact of unstructured usage-log data for personalization.

OpenRec: A Modular Framework for Extensible and Adaptable Recommendation Algorithms

With the increasing demand for deeper understanding of users» preferences, recommender systems have gone beyond simple user-item filtering and are increasingly sophisticated, comprised of multiple components for analyzing and fusing diverse information. Unfortunately, existing frameworks do not adequately support extensibility and adaptability and consequently pose significant challenges to rapid, iterative, and systematic, experimentation. In this work, we propose OpenRec, an open and modular Python framework that supports extensible and adaptable research in recommender systems. Each recommender is modeled as a computational graph that consists of a structured ensemble of reusable modules connected through a set of well-defined interfaces. We present the architecture of OpenRec and demonstrate that OpenRec provides adaptability, modularity and reusability while maintaining training efficiency and recommendation accuracy. Our case study illustrates how OpenRec can support an efficient design process to prototype and benchmark alternative approaches with inter-changeable modules and enable development and evaluation of new algorithms.

Understanding user interactions with podcast recommendations delivered via voice

Voice interfaces introduced by smart speakers present new opportunities and challenges for podcast content recommendations. Understanding how users interact with voice-based recommendations has the potential to inform better design of vocal recommenders. However, existing knowledge about user behavior is mostly for visual interfaces, such as the web, and is not directly transferable to voice interfaces, which rely on user listening and do not support skimming and browsing. To fill in the gap, we conducted a controlled study to compare user interactions with recommendations delivered visually to those with recommendations delivered vocally. Through an online A/B testing with 100 participants, we found that when recommendations are vocally conveyed, users consume more slowly, explore less, and choose fewer long-tail items. The study also reveals the correlation between user choices and exploration via voice interfaces. Our findings pose challenges to the design of voice interfaces, such as adaptively recommending diverse content and designing better navigation mechanisms.

Modularizing deep neural network-inspired recommendation algorithms

This tutorial reviews recent developments of deep neural network-based recommendation algorithms and demonstrates how to extend and adapt such algorithms for diverse application scenarios. The customization is supported by OpenRec framework that modularizes neural recommenders. The tutorial consists of a lecture and two hands-on sessions. It targets intermediate and advanced audiences who already possess knowledge of deep neural networks and are interested in applying those knowledge to the domain of recommendation. Materials are available at: http://openrec.ai/

Unbiased offline recommender evaluation for missing-not-at-random implicit feedback

Implicit-feedback Recommenders (ImplicitRec) leverage positive only user-item interactions, such as clicks, to learn personalized user preferences. Recommenders are often evaluated and compared offline using datasets collected from online platforms. These platforms are subject to popularity bias (i.e., popular items are more likely to be presented and interacted with), and therefore logged ground truth data are Missing-Not-At-Random (MNAR). As a result, the widely used Average-Over-All (AOA) evaluator is biased toward accurately recommending trendy items. In this paper, we (a) investigate evaluation bias of AOA and (b) develop an unbiased and practical offline evaluator for implicit MNAR datasets using the Inverse-Propensity-Scoring (IPS) technique. Through extensive experiments using four real-world datasets and four widely used algorithms, we show that (a) popularity bias is widely manifested in item presentation and interaction; (b) evaluation bias due to MNAR data pervasively exists in most cases where AOA is used to evaluate ImplicitRec; and (c) the unbiased estimator significantly reduces the AOA evaluation bias by more than 30% in the Yahoo! music dataset in terms of the Mean Absolute Error (MAE).