Almut G. Winterstein

Validity and reliability of measurement instruments used in research

PURPOSE Issues related to the validity and reliability of measurement instruments used in research are reviewed. SUMMARY Key indicators of the quality of a measuring instrument are the reliability and validity of the measures. The process of developing and validating an instrument is in large part focused on reducing error in the measurement process. Reliability estimates evaluate the stability of measures, internal consistency of measurement instruments, and interrater reliability of instrument scores. Validity is the extent to which the interpretations of the results of a test are warranted, which depends on the particular use the test is intended to serve. The responsiveness of the measure to change is of interest in many of the applications in health care where improvement in outcomes as a result of treatment is a primary goal of research. Several issues may affect the accuracy of data collected, such as those related to self-report and secondary data sources. Self-report of patients or subjects is required for many of the measurements conducted in health care, but self-reports of behavior are particularly subject to problems with social desirability biases. Data that were originally gathered for a different purpose are often used to answer a research question, which can affect the applicability to the study at hand. CONCLUSION In health care and social science research, many of the variables of interest and outcomes that are important are abstract concepts known as theoretical constructs. Using tests or instruments that are valid and reliable to measure such constructs is a crucial component of research quality.

Systematic Review of the Incidence and Characteristics of Preventable Adverse Drug Events in Ambulatory Care

Objective: To estimate the incidence and describe characteristics of preventable adverse drug events (pADEs) in ambulatory care. Data Sources: Studies were searched in PubMed (1966–March 2007), International Pharmaceutical Abstracts (1970–December 2006), the Cochrane database of systematic reviews (1993–March 2007), EMBASE (1980–February 2007), and Web of Science (1945–March 2007). Key words included medication error, adverse drug reaction, iatrogenic disease, outpatient, ambulatory care, primary health care, general practice, patient admission, hospitalization, observational study, retrospective studies, health services research, and follow-up studies. Additional articles were found in the reference sections of retrieved articles. Study Selection and Data Extraction: Peer-reviewed articles assessing pADEs in ambulatory care, with detailed descriptions/frequency distributions of (1) ADE/pADE incidence, (2) clinical outcomes, (3) associated drug groups, and/or (4) underlying medication errors were included. Study country, year and design, sample size, follow-up time, ADE/pADE identification method, proportion of ADEs/pADEs and ADEs/pADEs requiring hospital admission, and frequency distribution of adverse outcome, associated drug groups, or medication errors were extracted. Data Synthesis: Twenty-nine studies met inclusion criteria: 14 were ambulatory-based and 15 were hospital-based. Seven studies enrolled only elderly patients. The median ADE incidence was 14.9 (range 4.0–91.3) per 1000 person-months, and the pADE incidence was 5.6 per 1000 person-months (1.1–10.1). The median ADE preventability rate was 21% (11–38%). The median incidence of ADEs requiring hospital admission was 0.45 (0.10–13.1) per 1000 person-months, and the median incidence of pADEs requiring hospital admission was 4.5 per 1000 person-months. Cardiovascular drugs, analgesics, and hypoglycemic agents together accounted for 86.5% of pADEs, and 77.2% of pADEs resulted in symptoms of the central nervous system, electrolyte/renal system, and gastrointestinal tract. Medication errors resulting in pADEs occurred in the prescribing and monitoring stages. The most frequent drug therapy problem and error of commission reported in ambulatory-based studies on pADEs was the use of inappropriate drugs (42.7%; 40.4–45%). For pADEs requiring hospital admission, the most frequent drug therapy problem and error of omission reported was inadequate monitoring (45.4%; range 22.2–69.8%). Failure to prescribe prophylaxis to patients taking nonsteroidal antiinflammatory drugs or antiplatelet drugs frequently caused gastrointestinal toxicity, whereas lack of monitoring of diuretic, hypoglycemic, and anticoagulant use caused over- or under-diuresis, hyper- or hypoglycemia, and bleeding. Conclusions: ADEs in ambulatory care are common, with many being preventable and many resulting in hospitalization. Quality improvement programs should target errors in prescribing and monitoring, especially for patients using cardiovascular, analgesic, and hypoglycemic agents.

Preventable Drug-Related Hospital Admissions

OBJECTIVE: To estimate the prevalence of preventable drug-related hospital admissions (PDRAs) and to explore if selected study characteristics affect prevalence estimates. METHODS: Keyword search of MEDLINE (1966–December 1999), International Pharmaceutical Abstracts (1970–December 1999), and hand search. Two reviewers independently selected studies published in peer-reviewed journals and extracted crude prevalence estimates and study characteristics. Trials had to specifically address consequences of drug therapy requiring hospital admission and include a quantitative preventability assessment. Stratified analysis and meta-regression were used to explore the association between study characteristics and prevalence estimates. DATA SYNTHESIS: Fifteen studies reported a median PDRA prevalence of 4.3% (interquartile range [IQR] 3.1–9.5%). The median preventability rate of drug-related admissions was 59% (IQR 50–73%). No evidence of publication bias related to study size could be determined. Because the individual study results were highly heterogeneous (Cochrans Q = 176, df = 14; p < 0.001), no meta-analytic summary estimate was computed. Stratified analysis suggested an association between prevalence estimates and 3 study characteristics: exclusion of first admissions (readmission studies: average PDRA prevalence of 14.0 %, estimated prevalence OR = 3.7); mean age of admissions >70 (OR = 2.1); and inclusion of “indirect” drug-related morbidity, such as omission errors or therapeutic failure (OR = 1.9). There was little evidence of other associations with prevalence estimates, such as selection of specific hospital units, exclusion/inclusion of planned admissions, country, and specified methods of PDRA case ascertainment. CONCLUSIONS: Drug-related morbidity is a significant healthcare problem, and a great proportion is preventable. Study methods in prevalence reports vary and should be considered when interpreting findings or planning future research.

Utilization of Pharmacologic Treatment in Youths with Attention Deficit/Hyperactivity Disorder in Medicaid Database:

Background: Little is known about longitudinal changes in drug utilization in attention-deficit/hyperactivity disorder (ADHD). Objective: To describe longitudinal trends in ADHD drug utilization and explore demographic differences among youths eligible for a large Southern state Medicaid program. Methods: A cross-sectional and longitudinal analysis of 10 years of claims data for all Medicaid beneficiaries younger than 20 years of age with 6 months or more of continuous insurance (N = 2,131,953) was conducted. Annual prevalence, incidence, and persistence in ADHD medication use (stimulants and atomoxetine) were estimated based on pharmacy claims and clinician-reported ADHD diagnosis. Results: ADHD prevalence increased 1.70-fold (95% CI 1.67 to 1.73) from 3.10% (21,904 of 705,573 beneficiaries) in fiscal year 1995–1996 to 5.27% (41,661 of 790,338) in 2003–2004, paralleled by a 1.84-fold (95% CI 1.81 to 1.87) increase in drug use to 4.63%. In 2003–2004, 0.69% of youths were diagnosed and newly started on drugs, reflecting a 1.38-fold (95% CI 1.33 to 1.43) increase over 1995–1996. One in five white males between the ages of 10 and 14 years (19.24%; 95% CI 18.81 to 19.67) received ADHD medication in 2003–2004. Males continued to be more likely diagnosed and treated than females (prevalence ratio [PR] in 2003–2004 = 2.96:95% CI 2.90 to 3.03 vs 3.82; 95% CI 3.69 to 3.96 in 1995–1996), as were whites when compared with Hispanics (PR in 2003–2004 = 2.65; 95% CI 2.57 to 2.73 vs 3.78; 95% CI 3.57 to 3.99 in 1995–1996) and blacks (PR in 2003–2004 = 1.81; 95% CI 1.76 to 1.85 vs 2.00; 95% CI 1.93 to 2.07 in 1995–1996). The most common starting age throughout the study period was 5–9 years, with 2.45% (95% CI 2.37 to 2.52) new ADHD drug users in 2003–2004, but largest increases in prevalence were observed in adolescents 15–19 years of age, with 2.47% (95% CI 2.38 to 2.55) in 2003–2004 compared with 0.45% (95% CI 0.41 to 0.49) in 1995–1996. Medication persistence varied, with only 49.9% (95% CI 49.4 to 50.5) of new users receiving drugs after 1 year, with yet another 17.2% (95% CI 16.4 to 18.0) continuing for 5 years or more. Conclusions: ADHD drug utilization continues to increase due to steady increases in diagnosis and chronic use of the drugs over several years. While racial, ethnic, and sex differences persist, the age distribution of drug users has shifted toward older children. These findings emphasize the need for studies that analyze determinants of treatment as well as outcomes, both benefits and risks, associated with long-term medication use.

Cardiac Safety of Central Nervous System Stimulants in Children and Adolescents With Attention-Deficit/Hyperactivity Disorder

OBJECTIVES. Case reports have raised concerns about the risk of cardiac events associated with central nervous system stimulants for the treatment of attention-deficit/hyperactivity disorder. PATIENTS AND METHODS. This was a retrospective cohort study that used 10 years (July 1994 to June 2004) of Florida Medicaid claims data cross-linked to Vital Statistics Death Registry data. The cohort was composed of all youth 3 to 20 years old who were newly diagnosed with attention-deficit/hyperactivity disorder. Each month of follow-up was classified according to stimulant claims (methylphenidate, amphetamines, and pemoline) as current use (active stimulant claim), former use (time after periods of current use), or nonuse (time preceding the first stimulant claim, including follow-up of youth who were never exposed to stimulants). The studys end points were (1) cardiac death, (2) first hospital admission for cardiac causes or (3) first emergency department visit for cardiac causes. Risks were compared with time-dependent Cox regression analysis adjusting for various cardiac risk factors. RESULTS. During 124932 person-years of observation (n = 55383), 73 youth died, 5 because of cardiac causes. No cardiac death occurred during 42612 person-years of stimulant use. Hospital admissions for cardiac cause occurred for 27 children (8 during stimulant use, 11 during 35671 person-years of former use, and 8 during 46649 person-years of nonuse); and 1091 children visited the emergency department for cardiac causes (8.7 per 1000 person-years). Current stimulant use was associated with a 20% increase in the hazard for emergency department visits when compared with nonuse. No increased risk was found for periods of former use when compared with nonuse. CONCLUSIONS. Incidence rates of cardiac events requiring hospitalization were small and similar to national background rates. Stimulants were associated with an increase in cardiac emergency department visits. More evidence is needed that addresses the long-term risk/benefit of the various treatment options and the effect of other cardiac risk factors and comedications.

Cost-effectiveness of Respiratory Syncytial Virus Prophylaxis in Various Indications

OBJECTIVES To evaluate the cost-effectiveness of immunoprophylaxis against respiratory syncytial virus (RSV) infections with palivizumab based on actual cost and observed incidence rates in various pediatric risk groups. DESIGN Decision tree analysis comparing children with various combinations of the following indications: chronic lung disease, congenital heart disease, or prematurity (≤32 weeks gestation), and children with none of these indications. One-way sensitivity analyses and Monte Carlo simulations were used to quantify parameter uncertainty. SETTING Florida during the 2004-2005 RSV season. PARTICIPANTS A total of 159,790 Medicaid-eligible children aged 0 to 2 years. INTERVENTION Palivizumab prophylaxis compared with no prophylaxis. OUTCOMES MEASURE Incremental cost (2010 US dollars) per hospitalization for RSV infection avoided. RESULTS The mean cost of palivizumab per dose ranged from

Stimulants and Cardiovascular Events in Youth with Attention-Deficit/Hyperactivity Disorder

1661 for infants younger than 6 months of age to

Cardiac Safety of Methylphenidate Versus Amphetamine Salts in the Treatment of ADHD

2584 for children in their second year of life. Among preterm infants younger than 6 months of age without other indications, immunoprophylaxis with palivizumab cost

Activated Partial Thromboplastin Time Versus Antifactor Xa Heparin Assay in Monitoring Unfractionated Heparin by Continuous Intravenous Infusion

302,103 (95% confidence interval,

Cardiovascular safety of central nervous system stimulants in children and adolescents: population based cohort study.

141,850-