Gordon Willis

The Automated Self-Administered 24-Hour Dietary Recall (ASA24): A Resource for Researchers, Clinicians, and Educators from the National Cancer Institute

Extensive evidence has demonstrated that 24-hour dietary recalls (24HDRs) provide high-quality dietary intake data with minimal bias, making them the preferred tool for monitoring the diets of populations and, increasingly, for studying diet and disease associations (1-3). Traditional 24HDRs, however, are expensive and impractical for large-scale research because they rely on trained interviewers, and multiple administrations are needed to estimate usual intakes. To address these challenges, the National Cancer Institute (NCI), in collaboration with the research firm Westat (Rockville, MD), and with the support of other institutes and offices at the National Institutes of Health, developed the Automated Self-Administered 24-hour dietary recall (ASA24) (4-6). ASA24 is a public-access, freely available, web-based tool for researchers, clinicians and educators, modeled on the Automated Multiple Pass Method (AMPM) (7). Development of an automated self-administered 24HDR for adults began in 2006 and was informed by input from stakeholders participating in an External Working Group and small-scale cognitive and usability testing (4,5). A Beta version released in August 2009 has been used by over 175 researchers to collect over 40,000 recalls; Version 1, which offers improved usability and new features, was released in September 2011 (8). A modified version intended for self-administration by children is under development by researchers at the Baylor College of Medicine (Houston, TX) and is expected to be available mid-2012 (9,10). This paper describes the features of ASA24 and planned evaluations.

Assessment of the Accuracy of Portion Size Reports Using Computer-Based Food Photographs Aids in the Development of an Automated Self-Administered 24-Hour Recall

OBJECTIVE To assess the accuracy of portion-size estimates and participant preferences using various presentations of digital images. DESIGN Two observational feeding studies were conducted. In both, each participant selected and consumed foods for breakfast and lunch, buffet style, serving themselves portions of nine foods representing five forms (eg, amorphous, pieces). Serving containers were weighed unobtrusively before and after selection as was plate waste. The next day, participants used a computer software program to select photographs representing portion sizes of foods consumed the previous day. Preference information was also collected. In Study 1 (n=29), participants were presented with four different types of images (aerial photographs, angled photographs, images of mounds, and household measures) and two types of screen presentations (simultaneous images vs an empty plate that filled with images of food portions when clicked). In Study 2 (n=20), images were presented in two ways that varied by size (large vs small) and number (4 vs 8). SUBJECTS/SETTING Convenience sample of volunteers of varying background in an office setting. STATISTICAL ANALYSES PERFORMED Repeated-measures analysis of variance of absolute differences between actual and reported portions sizes by presentation methods. RESULTS Accuracy results were largely not statistically significant, indicating that no one image type was most accurate. Accuracy results indicated the use of eight vs four images was more accurate. Strong participant preferences supported presenting simultaneous vs sequential images. CONCLUSIONS These findings support the use of aerial photographs in the automated self-administered 24-hour recall. For some food forms, images of mounds or household measures are as accurate as images of food and, therefore, are a cost-effective alternative to photographs of foods.

Health Care Provider Surveys in the United States, 2000–2010 A Review

Surveys of health care providers (e.g., physicians and other health care professionals) are an important tool for assessing health care practices and the settings in which care is delivered. Although multiple methods are used to increase survey data quality, little is known about which methods are most commonly implemented. We reviewed 117 large surveys described in literature published between 2000 and 2010, examining descriptions of survey design features, survey implementation, and response rates. Despite wide variation, the typical provider survey selected practicing physicians as respondents, used the American Medical Association Masterfile as sample frame, included mail as both mode of initial contact and questionnaire administration mode, and offered monetary incentives to respondents. Our review revealed inconsistency of documentation concerning procedures used, and a variety of response rate calculation methods, such that it was difficult to determine practices that maximize response rate. We recommend that reports provide more comprehensive documentation concerning key methodological features to improve assessment of survey data quality.

Confirmation of Family Cancer History Reported in a Population-Based Survey

BACKGROUND Knowledge of family cancer history is essential for estimating an individuals cancer risk and making clinical recommendations regarding screening and referral to a specialty cancer genetics clinic. However, it is not clear if reported family cancer history is sufficiently accurate for this purpose. METHODS In the population-based 2001 Connecticut Family Health Study, 1019 participants reported on 20 578 first-degree relatives (FDR) and second-degree relatives (SDR). Of those, 2605 relatives were sampled for confirmation of cancer reports on breast, colorectal, prostate, and lung cancer. Confirmation sources included state cancer registries, Medicare databases, the National Death Index, death certificates, and health-care facility records. Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for reports on lung, colorectal, breast, and prostate cancer and after stratification by sex, age, education, and degree of relatedness and used to estimate report accuracy. Pairwise t tests were used to evaluate differences between the two strata in each stratified analysis. All statistical tests were two-sided. RESULTS Overall, sensitivity and positive predictive value were low to moderate and varied by cancer type: 60.2% and 40.0%, respectively, for lung cancer reports, 27.3% and 53.5% for colorectal cancer reports, 61.1% and 61.3% for breast cancer reports, and 32.0% and 53.4% for prostate cancer reports. Specificity and negative predictive value were more than 95% for all four cancer types. Cancer history reports on FDR were more accurate than reports on SDR, with reports on FDR having statistically significantly higher sensitivity for prostate cancer than reports on SDR (58.9% vs 21.5%, P = .002) and higher positive predictive value for lung (78.1% vs 31.7%, P < .001), colorectal (85.8% vs 43.5%, P = .004), and breast cancer (79.9% vs 53.6%, P = .02). CONCLUSIONS General population reports on family history for the four major adult cancers were not highly accurate. Efforts to improve accuracy are needed in primary care and other health-care settings in which family history is collected to ensure appropriate risk assessment and clinical care recommendations.

Improving the Quality of Surveys of Physicians and Medical Groups A Research Agenda

Because health care providers have a central role in implementing guidelines, health care reform, and new standards of care and technologies, surveying them about their practices and perspectives is vital for health services and policy research. In November 2010, the National Cancer Institute convened a workshop to review and discuss current methodologies in designing and fielding large-scale surveys of physicians and medical groups. This report summarizes key issues and future directions for four topic areas addressed in the workshop: sample frames for surveying physicians and medical groups; points of contact and response modes; response incentives; and questionnaire design and burden. Recommendations were made for improving sample frame databases, optimizing mixed-mode surveys, and studying use of incentives with gatekeepers and in medical group settings. There is particular need for empirical assessment of factors that motivate or impede participation of physicians, other types of clinicians, and medical groups in survey research.

MEASURING EVERYDAY RACIAL/ETHNIC DISCRIMINATION IN HEALTH SURVEYS

While it is clear that self-reported racial/ethnic discrimination is related to illness, there are challenges in measuring self-reported discrimination or unfair treatment. In the present study, we evaluate the psychometric properties of a self-reported instrument across racial/ ethnic groups in a population-based sample, and we test and interpret findings from applying two different widely-used approaches to asking about discrimination and unfair treatment. Even though we found that the subset of items we tested tap into a single underlying concept, we also found that different groups are more likely to report on different aspects of discrimination. Whether race is mentioned in the survey question affects both frequency and mean scores of reports of racial/ethnic discrimination. Our findings suggest caution to researchers when comparing studies that have used different approaches to measure racial/ethnic discrimination and allow us to suggest practical empirical guidelines for measuring and analyzing racial/ethnic discrimination. No less important, we have developed a self-reported measure of recent racial/ethnic discrimination that functions well in a range of different racial/ethnic groups and makes it possible to compare how racial/ethnic discrimination is associated with health disparities among multiple racial/ethnic groups.

Methods for Translating an English-Language Survey Questionnaire on Tobacco Use into Mandarin, Cantonese, Korean, and Vietnamese:

This article reports research on procedures for translating a survey questionnaire on tobacco use from English into Mandarin Chinese, Cantonese Chinese, Korean, and Vietnamese. The goal is to offer practical guidelines for researchers involved in translating questionnaires. The authors operationalize a five-step process for translation and evaluation based on the frameworks presented in Harkness, Van de Vijver, and Mohler (2003) and the U.S. Census Bureau (2004). Based on qualitative observations, the five-step process produced effective questionnaire translations. The iterative nature of the process and the team-based approach the process encourages were particularly important to the success. Based on documented experiences, the authors identify lessons learned and make recommendations to other researchers who need to translate questionnaires.

Cognitive Interviewing as a Tool for Improving the Informed Consent Process

Consent materials often contain complex information, legalese, and other features that render them difficult to comprehend in such a way that consent is truly informed. I propose that researchers adapt cognitive interviewing, normally used for the pretesting of survey questionnaires, to evaluate the understandability of consent materials and the way in which subjects use this information to make decisions regarding participation. Cognitive interviewing involves the intensive probing of small samples of volunteer subjects to elucidate thought processes that otherwise remain hidden. Cognitive interviewing can be applied: (a) to further the basic science of informed consent; (b) to pretest materials for a specific study; and (c) as an embedded procedure for assessing subject thought processes in the course of obtaining consent.

Cross-Cultural Cognitive Interviewing Seeking Comparability and Enhancing Understanding

Cognitive interviewing (CI) has emerged as a key qualitative method for the pretesting and evaluation of self-report survey questionnaires. This article defines CI, describes its key features, and outlines the data analysis techniques that are commonly used. The authors then consider recent extensions of cognitive testing to the cross-cultural survey research realm, where the major practical objectives are: (1) to facilitate inclusion of a range of cultural and linguistic groups and (2) for purposes of comparative analysis, to produce survey questionnaire items that exhibit comparability of measurement, across groups. Challenges presented by this extension to the cross-cultural and multilingual areas are discussed. Finally, the authors introduce the articles contained within the current special issue of Field Methods (2011), which endeavor to apply cognitive testing in specific cross-cultural survey projects, and to both identify and suggest solutions to the unique problems that face questionnaire designers and researchers more generally, in the practice of survey pretesting and evaluation methods as these endeavor to cover the sociocultural spectrum.

Using cognitive interviews to evaluate items for measuring sexual functioning across cancer populations: improvements and remaining challenges

PurposeOne goal of the Patient-Reported Outcomes Measurement Information System™ (PROMIS™) is to develop a measure of sexual functioning that broadens the definition of sexual activity and incorporates items that reflect constructs identified as important by patients with cancer. We describe how cognitive interviews improved the quality of the items and discuss remaining challenges to assessing sexual functioning in research with cancer populations.MethodsWe conducted 39 cognitive interviews of patients with cancer and survivors on the topic of sexual experience. Each of the 83 candidate items was seen by 5–24 participants. Participants included both men and women and varied by cancer type, treatment trajectory, race, and literacy level. Significantly revised items were retested in subsequent interviews.ResultsCognitive interviews provided useful feedback about the relevance, sensitivity, appropriateness, and clarity of the items. Participants identified broad terms (e.g., “sex life”) to assess sexual experience and exposed the challenges of measuring sexual functioning consistently, considering both adjusted and unadjusted sexual experiences.ConclusionsCognitive interviews were critical for item refinement in the development of the PROMIS measure of sexual function. Efforts are underway to validate the measure in larger cancer populations.