Usability and Acceptability of Two Smartphone Apps for Smoking Cessation Among Young Adults With Serious Mental Illness: Mixed Methods Study

Abstract

Background Young adults with serious mental illness (SMI) have higher smoking rates and lower cessation rates than young adults without SMI. Scalable interventions such as smartphone apps with evidence-based content (eg, the National Cancer Institute’s [NCI’s] QuitGuide and quitSTART) could increase access to potentially appealing and effective treatment for this group but have yet to be tested in this population. Objective The goal of this user-centered design study is to determine the user experience (including usability and acceptability) of 2 widely available apps developed by the NCI—QuitGuide and quitSTART—among young adult tobacco users with SMI. Methods We conducted usability and acceptability testing of QuitGuide and quitSTART among participants with SMI aged between 18 and 35 years who were stable in community mental health treatment between 2019 and 2020. Participants were randomly assigned to use QuitGuide or quitSTART on their smartphones. App usability was evaluated at baseline and following a 2-week field test of independent use via a video-recorded task completion protocol. Using a mixed method approach, we triangulated 4 data sources: nonparticipant observation, open-ended interviews, structured interviews (including the System Usability Scale [SUS]), and backend app use data obtained from the NCI. Quantitative data were analyzed using descriptive statistics, and qualitative data were analyzed using thematic analysis. Results Participants were 17 smokers who were not interested in quitting, with a mean age of 29 (SD 4) years; 41% (n=7) presented with psychotic disorders. Participants smoked an average of 15 (SD 7) cigarettes per day. The mean SUS scores for QuitGuide were similar at visits one and two (mean 64, SD 18 and mean 66, SD 18, respectively). The mean SUS scores for quitSTART numerically increased from visit one (mean 55, SD 20) to visit two (mean 64, SD 16). Acceptability scores followed the same pattern. Observed task completion rates were at least 75% (7/9 for QuitGuide, 6/8 for quitSTART) for both apps at both visits for all but 2 tasks. During the 13-day trial period, QuitGuide and quitSTART users interacted with their assigned app on an average of 4.6 (SD 2.8) days versus 10.8 (SD 3.5) days, for a mean total of 5.6 (SD 3.8) interactions versus 41 (SD 26) interactions, and responded to a median of 1 notification (range 0-8) versus 18.5 notifications (range 0-37), respectively. Qualitative comments indicated moderate to high satisfaction overall but also included concerns about the accuracy of the apps’ feedback. Conclusions Both QuitGuide and quitSTART had acceptable levels of usability and mixed levels of acceptability among young adults with SMI. The higher level of engagement with quitSTART suggests that quitSTART may be a favorable tool for young adult smokers with SMI. However, clinical support or coaching may be needed to overcome initial usability issues.