Jeffrey J. Hyman

The limitations of using insurance data for research

Jeffrey Hyman, DDS, PhD I n the March issue of The Journal, there was a reanalysis 1 of an earlier study by Giannobile and colleagues and responses by Braun and colleagues and Ioannidis. These studies use data from Delta Dental of Michigan, a dental insurance claims database that includes data on 60 million people. Insurance data have become very popular for researchers and have been used in a number of oral health–related studies, often looking at oral health treatment and possible reductions in the incidence of systemic diseases. Insurance data have many advantages for researchers. The data are already collected and the sample sizes are very large,making it easy to achieve statistically significant results. This makes it more likely that the study will be published, due to the well-known issue of publication bias that favors publication of significant results. Given high research costs, it is extremely unlikely that studies involving collection of new data with the very large sample sizes available in insurance databases will be ever be done. Moreover, the large number of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes combined with large sample sizes of insurance databases allows researchers to address a wide range of questions. However, studies that are based on insurance claims or other third-party data are often misinterpreted or flawed because the information contained in insurance data is often limited, making it unsuitable for exploring important questions about disease and risk factors, causation, and treatment. Prominent issues that must be considered in evaluating studies based on third-party data include external validity, selection bias, confounding, misclassification bias, and causality. External validity—or the ability to generalize the results of a study to a wider population—is a major consideration when interpreting studies that are based on data from third-party payers. In 2012, approximately 60% of the US population had dental insurance, and these people were healthier than those who did not have insurance. They also had higher incomes and probably more education. Results based on these studies likely only apply to those with insurance. Most importantly, studies that are based on insurance data may be flawed or misinterpreted because they are often subject to selection bias or confounding. Insurance data are primarily collected for financial purposes and not to answer important questions about disease such as prevalence, etiology, risk, and treatment. Selection bias occurs when a sample of study participants is not representative of the population that is of interest. Confounding is similar and occurs when other variables associated with a disease are unevenly distributed

The genesis of a catalog of oral health-related surveys: locating oral health-related datasets.

Abstract The National Institute of Dental and Craniofacial Research (NIDCR), in collaboration with the Division of Oral Health, Centers for Disease Control and Prevention (DOH, CDC), has established a Dental, Oral and Craniofacial Data Resource Center (DRC). One element of the DRC is the Catalog of Surveys Related to Oral Health. The Catalog is a searchable electronic database that includes federal, state, international, and privately sponsored surveys and other datasets. Its purpose is to make researchers aware of surveys that have been conducted and to highlight features of complex surveys that relate to oral health. Other components of the DRC include an Archive of Procedures and Methods, Archive of Procedures and Methods Used in Oral Health Surveys, which is linked to the Catalog; an Annual Report, Oral Health U.S., 2002;and a data warehouse of acquired datasets. A Web-based statistical query system related to oral health is also under development. It is the intention of the DRC to meet the needs of NIDCR and DOH, CDC staff as well as other researchers interested in the status of oral health. The Catalog is available on CD-ROM at no cost and in the future will be made available through the NIDCR Web site.