Shiwali Mohan

Object-Oriented Representation and Hierarchical Reinforcement Learning in Infinite Mario

In this work, we analyze and improve upon reinforcement learning techniques used to build agents that can learn to play Infinite Mario, an action game. We extend the object-oriented representation by introducing the concept of object classes which can be effectively used to constrain state spaces. We then use this representation combined with the hierarchical reinforcement learning model as a learning framework. We also extend the idea of hierarchical RL by designing a hierarchy in action selection using domain specific knowledge. With the help of experimental results, we show that this approach facilitates faster and efficient learning for this domain.

Leveraging Self-Affirmation to Improve Behavior Change: A Mobile Health App Experiment

Background mHealth interventions can help to improve the physical well-being of participants. Unfortunately, mHealth interventions often have low adherence and high attrition. One possible way to increase adherence is instructing participants to complete self-affirmation exercises. Self-affirmation exercises have been effective in increasing many types of positive behaviors. However, self-affirmation exercises often involve extensive essay writing, a task that is not easy to complete on mobile platforms. Objective This study aimed to adapt a self-affirmation exercise to a form better suited for delivery through a mobile app targeting healthy eating behaviors, and to test the effect of differing self-affirmation doses on adherence to behavior change goals over time. Methods We examined how varied self-affirmation doses affected behavior change in an mHealth app targeting healthy eating that participants used for 28 days. We divided participants into the 4 total conditions using a 2×2 factorial design. The first independent variable was whether the participant received an initial self-affirmation exercise. The second independent variable was whether the participant received ongoing booster self-affirmations throughout the 28-day study. To examine possible mechanisms through which self-affirmation may cause positive behavior change, we analyzed three aspects of self-affirmation effects in our research. First, we analyzed how adherence was affected by self-affirmation exercises. Second, we analyzed whether self-affirmation exercises reduced attrition rates from the app. Third, we examined a model for self-affirmation behavior change. Results Analysis of 3556 observations from 127 participants indicated that higher doses of self-affirmation resulted in improved adherence to mHealth intervention goals (coefficient 1.42, SE 0.71, P=.04). This increased adherence did not seem to translate to a decrease in participant attrition (P value range .61-.96), although our definition of attrition was conservative. Finally, we examined the mechanisms by which self-affirmation may have affected intentions of behavior change; we built a model of intention (R2=.39, P<.001), but self-affirmation did not directly affect final intentions (P value range .09-.93). Conclusions Self-affirmations can successfully increase adherence to recommended diet and health goals in the context of an mHealth app. However, this increase in adherence does not seem to reduce overall attrition. The self-affirmation exercises we developed were simple to implement and had a low cost for both users and developers. While this study focused on an mHealth app for healthy eating, we recommend that other mHealth apps integrate similar self-affirmation exercises to examine effectiveness in other behaviors and contexts.

Interactive Task Learning

This article presents a new research area called interactive task learning (ITL), in which an agent actively tries to learn not just how to perform a task better but the actual definition of a task through natural interaction with a human instructor while attempting to perform the task. The authors provide an analysis of desiderata for ITL systems, a review of related work, and a discussion of possible application areas for ITL systems.

Acceptability of a team-based mobile health (mHealth) application for lifestyle self-management in individuals with chronic illnesses

With increased incidence of chronic illnesses arising due to unhealthy lifestyle habits, it is increasingly important to leverage technology applications to promote and sustain health behavior change. We developed a smartphone-based application, NutriWalking (NW), which recommends personalized daily exercise goals and promotes healthy nutritional habits in small peer teams. Here, we demonstrate an early study of usability and acceptability of this app in patients with type 2 Diabetes Mellitus and Depression. Our goal was to evaluate the potential of NW as a self-management support tool. Findings point to design considerations for team-based self-management tools delivered via mHealth platforms.With increased incidence of chronic illnesses arising due to unhealthy lifestyle habits, it is increasingly important to leverage technology applications to promote and sustain health behavior change. We developed a smartphone-based application, NutriWalking (NW), which recommends personalized daily exercise goals and promotes healthy nutritional habits in small peer teams. Here, we demonstrate an early study of usability and acceptability of this app in patients with type 2 Diabetes Mellitus and Depression. Our goal was to evaluate the potential of NW as a self-management support tool. Findings point to design considerations for team-based self-management tools delivered via mHealth platforms.