Jill A. Dever

Estimating Propensity Adjustments for Volunteer Web Surveys

Panels of persons who volunteer to participate in Web surveys are used to make estimates for entire populations, including persons who have no access to the Internet. One method of adjusting a volunteer sample to attempt to make it representative of a larger population involves randomly selecting a reference sample from the larger population. The act of volunteering is treated as a quasi-random process where each person has some probability of volunteering. One option for computing weights for the volunteers is to combine the reference sample and Web volunteers and estimate probabilities of being a Web volunteer via propensity modeling. There are several options for using the estimated propensities to estimate population quantities. Careful analysis to justify these methods is lacking. The goals of this article are (a) to identify the assumptions and techniques of estimation that will lead to correct inference under the quasi-random approach, (b) to explore whether methods used in practice are biased, and (c) to illustrate the performance of some estimators that use estimated propensities. Two of our main findings are (a) that estimators of means based on estimates of propensity models that do not use the weights associated with the reference sample are biased even when the probability of volunteering is correctly modeled and (b) if the probability of volunteering is associated with analysis variables collected in the volunteer survey, propensity modeling does not correct bias.

Validation of a Research Case Definition of Gulf War Illness in the 1991 US Military Population

Background: A case definition of Gulf War illness with 3 primary variants, previously developed by factor analysis of symptoms in a US Navy construction battalion and validated in clinic veterans, identified ill veterans with objective abnormalities of brain function. This study tests prestated hypotheses of its external validity. Methods: A stratified probability sample (n = 8,020), selected from a sampling frame of the 3.5 million Gulf War era US military veterans, completed a computer-assisted telephone interview survey. Application of the prior factor weights to the subjects’ responses generated the case definition. Results: The structural equation model of the case definition fit both random halves of the population sample well (root mean-square error of approximation = 0.015). The overall case definition was 3.87 times (95% confidence interval, 2.61–5.74) more prevalent in the deployed than the deployable nondeployed veterans: 3.33 (1.10–10.10) for syndrome variant 1; 5.11 (2.43–10.75) for variant 2, and 4.25 (2.33–7.74) for variant 3. Functional status on SF-12 was greatly reduced (effect sizes, 1.0–2.0) in veterans meeting the overall and variant case definitions. Conclusions: The factor case definition applies to the full Gulf War veteran population and has good characteristics for research.

The SmokefreeTXT (SFTXT) Study: Web and Mobile Data Collection to Evaluate Smoking Cessation for Young Adults

Background Text messaging (short message service, SMS) has been shown to be effective in delivering interventions for various diseases and health conditions, including smoking cessation. While there are many published studies regarding smoking cessation text messaging interventions, most do not provide details about the study’s operational methods. As a result, there is a gap in our understanding of how best to design studies of smoking cessation text messaging programs. Objective The purpose of this paper is to detail the operational methods used to conduct a randomized trial comparing three different versions of the National Cancer Institute’s SmokefreeText (SFTXT) program, designed for smokers 18 to 29 years of age. We detail our methods for recruiting participants from the Internet, reducing fraud, conducting online data collection, and retaining panel study participants. Methods Participants were recruited through website advertisements and market research online panels. Screening questions established eligibility for the study (eg, 18 to 29 years of age, current smoker). Antifraud measures screened out participants who could not meet the study requirements. After completing a baseline survey, participants were randomized to one of three study arms, which varied by type and timing of text message delivery. The study offered US

Refinements of the Medicare Outpatient Therapy Annual Expenditure Limit Policy

20 gift cards as incentives to complete each of four follow-up surveys. Automated email reminders were sent at designated intervals to increase response rates. Researchers also provided telephone reminders to those who had not completed the survey after multiple email reminders. We calculated participation rates across study arms and compared the final sample characteristics to the Current Population Survey to examine generalizability. Results Recruitment methods drove 153,936 unique visitors to the SFTXT Study landing page and 27,360 began the screener. Based on the screening questions, 15,462 out of 27,360 responders (56.51%) were eligible to participate. Of the 15,462 who were eligible, 9486 passed the antifraud measures that were implemented; however, 3882 failed to verify their email addresses or cell phone numbers, leaving 5604 who were invited to complete the baseline survey. Of the 5604 who were invited, 4432 completed the baseline survey, but only 4027 were retained for analysis because 405 did not receive the intervention. Conclusions Although antifraud measures helped to catch participants who failed study requirements and could have biased the data collected, it is possible that the email and cell phone verification check excluded some potentially eligible participants from the study. Future research should explore ways to implement verification methods without risking the loss of so many potential participants. ClinicalTrial Clinical Trials.gov NCT01885052; https://clinicaltrials.gov/ct2/show/NCT01885052; (Archived by WebCite at http://www.webcitation.org/6iWzcmFdw)

Preventive behaviors adults report using to avoid catching or spreading influenza, United States, 2015-16 influenza season

Background A Medicare beneficiarys annual outpatient therapy expenditures that exceed congressionally established caps are subject to extra documentation and review requirements. In 2011, these caps were

Effects of Cluster Sizes on Variance Components in Two-Stage Sampling

1,870 for physical therapy and speech-language pathology combined and

An Overview of Sample Design and Weighting

1,870 for occupational therapy separately. Objective This article considers the distributional effects of replacing current cap policy with equal caps by therapy discipline (physical therapy, occupational therapy, and speech-language pathology) or a single combined cap, and risk adjusting the physical therapy cap using beneficiary characteristics and functional status. Methods Alternative therapy cap policies are simulated with 100% Medicare claims for 2011 therapy users (N=4.9 million). A risk-adjusted cap for annual physical therapy expenditures is calculated from a quantile regression estimated on a sample of physical therapy users with diagnoses and clinician assessments of functional ability merged to their claims (n=4,210). Results Equal discipline-specific caps of

The Area Sample Design: One Solution

1,710 each for physical therapy, occupational therapy, and speech-language pathology result in the same aggregate Medicare expenditures above the caps as 2011 cap policy. A single combined-disciplines cap of

Weighting the Personnel Survey: One Solution

2,485 also results in the same aggregate expenditures above the cap. Risk adjustment varies the physical therapy cap by as much as 5 to 1 across beneficiaries and equalizes the probability of exceeding the physical therapy cap across diagnosis and functional status groups. Limitations One limitation of the study was the assumption of no behavioral response on the part of beneficiaries or providers to a change in cap policy. Additionally, analysis of risk adjusting the therapy caps was limited by sample size. Conclusions Equal discipline-specific caps for physical therapy, occupational therapy, and speech-language pathology are more equitable to high users of both physical therapy and speech-language pathology than current cap policy. Separating the physical therapy and speech-language pathology caps is a change that policy makers could consider. Risk adjustment of the therapy caps is a first step in incorporating beneficiary need for services into Medicare outpatient therapy payment policy.

Calibration and Other Uses of Auxiliary Data in Weighting

Introduction Influenza vaccination can prevent influenza and potentially serious influenza-related complications. Although the single best way to prevent influenza is annual vaccination, everyday preventive actions, including good hygiene, health, dietary, and social habits, might help, too. Several preventive measures are recommended, including: avoiding close contact with people who are sick; staying home when sick; covering your mouth and nose when coughing or sneezing; washing your hands often; avoiding touching your eyes, nose, and mouth; and practicing other good health habits like cleaning and disinfecting frequently touched surfaces, getting plenty of sleep, and drinking plenty of fluids. Understanding public acceptance and current usage of these preventive behaviors can be useful for planning both seasonal and pandemic influenza prevention campaigns. This study estimated the percentage of adults in the United States who reported practicing preventive behaviors to avoid catching or spreading influenza, and explored associations of reported behaviors with sociodemographic factors. Methods We analyzed data from 2015 National Internet Flu Survey, a nationally representative probability-based Internet panel survey of the non-institutionalized U.S. population ≥18 years. The self-reported behaviors used to avoid catching or spreading influenza were grouped into four and three non-mutually exclusive subgroups, respectively. Weighted proportions were calculated. Multivariable logistic regression models were used to calculate adjusted prevalence differences and to determine independent associations between sociodemographic characteristics and preventive behavior subgroups. Results Common preventive behaviors reported were: 83.2% wash hands often, 80.0% cover coughs and sneezes, 78.2% stay home if sick with a respiratory illness, 64.4% avoid people sick with a respiratory illness, 51.7% use hand sanitizers, 50.2% get treatment as soon as possible, and 49.8% report getting the influenza vaccination. Race/ethnicity, gender, age, education, income, region, receipt of influenza vaccination, and household size were associated with use of preventive behaviors after controlling for other factors. Conclusion Many adults in the United States reported using preventive behaviors to avoid catching or spreading influenza. Though vaccination is the most important tool available to prevent influenza, the addition of preventive behaviors might play an effective role in reducing or slowing transmission of influenza and complement prevention efforts.