Edgar C. Merkle

The problem of model selection uncertainty in structural equation modeling.

Model selection in structural equation modeling (SEM) involves using selection criteria to declare one model superior and treating it as a best working hypothesis until a better model is proposed. A limitation of this approach is that sampling variability in selection criteria usually is not considered, leading to assertions of model superiority that may not withstand replication. We illustrate that selection decisions using information criteria can be highly unstable over repeated sampling and that this uncertainty does not necessarily decrease with increases in sample size. Methods for addressing model selection uncertainty in SEM are evaluated, and implications for practice are discussed.

An application of the poisson race model to confidence calibration.

In tasks as diverse as stock market predictions and jury deliberations, a persons feelings of confidence in the appropriateness of different choices often impact that persons final choice. The current study examines the mathematical modeling of confidence calibration in a simple dual-choice task. Experiments are motivated by an accumulator model, which proposes that information supporting each alternative accrues on separate counters. The observer responds in favor of whichever alternatives counter first hits a designated threshold. Confidence can then be scaled from the difference between the counters at the time that the observer makes a response. The authors examine the overconfidence result in general and present new findings dealing with the effect of response bias on confidence calibration.

Understanding How Participation in a Consumer-Run Organization Relates to Recovery

The goal of this study was to examine how different types of participation in a consumer-run organization (CRO) are related to recovery. More specifically, this study uses structural equation modeling to examine the relative impact of empowering and socially supportive participation experiences on progress towards recovery among 250 CRO members from 20 CROs. An empowering participation experience refers to involvement in leadership roles and contribution to organizational functioning. A socially supportive participation experience refers to social involvement in mutually supportive friendships with intimacy and sharing. Results indicate that both types of participation are associated with recovery, although a socially supportive participation experience maintains a stronger relationship with recovery than an empowering participation experience. Findings are consistent with the idea that CROs should encourage both types of participation. Drawing from over ten years of experience supporting CROs, the discussion section explores several strategies CROs can use to foster empowering and socially supportive participation experiences.

Informed Decision Making: Assessment of the Quality of Physician Communication about Prostate Cancer Diagnosis and Treatment.

Introduction. Little is known about how physicians present diagnosis and treatment planning in routine practice in preference-sensitive treatment decisions. We evaluated completeness and quality of informed decision making in localized prostate cancer post biopsy encounters. Methods. We analyzed audio-recorded office visits of 252 men with presumed localized prostate cancer (Gleason 6 and Gleason 7 scores) who were seeing 45 physicians at 4 Veterans Affairs Medical Centers. Data were collected between September 2008 and May 2012 in a trial of 2 decision aids (DAs). Braddock’s previously validated Informed Decision Making (IDM) system was used to measure quality. Latent variable models for ordinal data examined the relationship of IDM score to treatment received. Results. Mean IDM score showed modest quality (7.61±2.45 out of 18) and high variability. Treatment choice and risks and benefits were discussed in approximately 95% of encounters. However, in more than one-third of encounters, physicians provided a partial set of treatment options and omitted surveillance as a choice. Informing quality was greater in patients treated with surveillance (β = 1.1, p = .04). Gleason score (7 vs 6) and lower age were often cited as reasons to exclude surveillance. Patient preferences were elicited in the majority of cases, but not used to guide treatment planning. Encounter time was modestly correlated with IDM score (r = 0.237, p = .01). DA type was not associated with IDM score. Discussion. Physicians informed patients of options and risks and benefits, but infrequently engaged patients in core shared decision-making processes. Despite patients having received DAs, physicians rarely provided an opportunity for preference-driven decision making. More attention to the underused patient decision-making and engagement elements could result in improved shared decision making.

Generalized Linear Models for Categorical and Continuous Limited Dependent Variables

Introduction and Overview The Nature of Limited Dependent Variables Overview of GLMs Estimation Methods and Model Evaluation Organization of This Book Discrete Variables Binary Variables Logistic Regression The Binomial GLM Estimation Methods and Issues Analyses in R and Stata Exercises Nominal Polytomous Variables Multinomial Logit Model Conditional Logit and Choice Models Multinomial Processing Tree Models Estimation Methods and Model Evaluation Analyses in R and Stata Exercises Ordinal Categorical Variables Modeling Ordinal Variables: Common Practice versus Best Practice Ordinal Model Alternatives Cumulative Models Adjacent Models Stage Models Estimation Methods and Issues Analyses in R and Stata Exercises Count Variables Distributions for Count Data Poisson Regression Models Negative Binomial Models Truncated and Censored Models Zero-Inflated and Hurdle Models Estimation Methods and Issues Analyses in R and Stata Exercises Continuous Variables Doubly Bounded Continuous Variables Doubly Bounded versus Censored The beta GLM Modeling Location and Dispersion Estimation Methods and Issues Zero- and One-Inflated Models Finite Mixture Models Analyses in R and Stata Exercises Censoring and Truncation Models for Censored and Truncated Variables Non-Gaussian Censored Regression Estimation Methods, Model Comparison, and Diagnostics Extensions of Censored Regression Models Analyses in R and Stata Exercises Extensions Extensions and Generalizations Multilevel Models Bayesian Estimation Evaluating Relative Importance of Predictors in GLMs

Testing for Measurement Invariance with Respect to an Ordinal Variable

Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.

Why Do Patients Derogate Physicians Who Use a Computer-Based Diagnostic Support System?

Objective. To better understand 1) why patients have a negative perception of the use of computerized clinical decision support systems (CDSSs) and 2) what contributes to the documented heterogeneity in the evaluations of physicians who use a CDSS. Methods. Three vignette-based studies examined whether negative perceptions stemmed directly from the use of a computerized decision aid or the need to seek external advice more broadly (experiment 1) and investigated the contributing role of 2 individual difference measures, attitudes toward statistics (ATS; experiment 2) and the Multidimensional Health Locus of Control Scale (MHLC; experiment 3), to these findings. Results. A physician described as making an unaided diagnosis was rated significantly more positively on a number of attributes than a physician using a computerized decision aid but not a physician who sought the advice of an expert colleague (experiment 1). ATS were unrelated to perceptions of decision aid use (experiment 2); however, greater internal locus of control was associated with more positive feelings about unaided care and more negative feelings about care when a decision aid was used (experiment 3). Conclusion. Negative perceptions of computerized decision aid use may not be a product of the need to seek external advice more generally but may instead be specific to the use of a nonhuman tool and may be associated with individual differences in locus of control. Together, these 3 studies may be used to guide education efforts for patients.

Choosing a Strictly Proper Scoring Rule

Strictly proper scoring rules, including the Brier score and the logarithmic score, are standard metrics by which probability forecasters are assessed and compared. Researchers often find that ones choice of strictly proper scoring rule has minimal impact on ones conclusions, but this conclusion is typically drawn from a small set of popular rules. In the context of forecasting world events, we use a recently proposed family of proper scoring rules to study the properties of a wide variety of strictly proper rules. The results indicate that conclusions vary greatly across different scoring rules, so that ones choice of scoring rule should be informed by the forecasting domain. We then describe strategies for choosing a scoring rule that meets the needs of the forecast consumer, considering three unique families of proper scoring rules.

The disutility of the hard-easy effect in choice confidence

A common finding in confidence research is the hard-easy effect, in which judges exhibit greater overconfidence for more difficult sets of questions. Many explanations have been advanced for the hard-easy effect, including systematic cognitive mechanisms, experimenter bias, random error, and statistical artifact. In this article, I mathematically derive necessary and sufficient conditions for observing a hard-easy effect, and I relate these conditions to previous explanations for the effect. I conclude that all types of judges exhibit the hard-easy effect in almost all realistic situations. Thus, the effect’s presence cannot be used to distinguish between judges or to draw support for specific models of confidence elicitation.

Chemotherapy was not associated with cognitive decline in older adults with breast and colorectal cancer: Findings from a prospective cohort study

Objectives:This study tested 2 hypotheses: (1) chemotherapy increases the rate of cognitive decline in breast and colorectal cancer patients beyond what is typical of normal aging and (2) chemotherapy results in systematic cognitive declines when compared with breast and colorectal cancer patients who did not receive chemotherapy. Subjects:Data came from personal interviews with a prospective cohort of patients with breast (n=141) or colorectal cancer (n=224) with incident disease drawn from the nationally representative Health and Retirement Study (1998–2006) with linked Medicare claims. Measures:The 27-point modified Telephone Interview for Cognitive Status was used to assess cognitive functioning, focusing on memory and attention. We defined the smallest clinically significant change as 0.4 points per year. Results:We used Bayesian hierarchical linear models to test the hypotheses, adjusting for multiple possible confounders. Eighty-eight patients were treated with chemotherapy; 277 were not. The mean age at diagnosis was 75.5. Patients were followed for a median of 3.1 years after diagnosis, with a range of 0 to 8.3 years. We found no differences in the rates of cognitive decline before and after diagnosis for patients who received chemotherapy in adjusted models (P=0.86, one-sided 95% posterior intervals lower bound: 0.09 worse after chemotherapy), where patients served as their own controls. Moreover, the rate of cognitive decline after diagnosis did not differ between patients who had chemotherapy and those who did not (P=0.84, one-sided 95% posterior intervals lower bound: 0.11 worse for chemotherapy group in adjusted model). Conclusions:There was no evidence of cognitive decline associated with chemotherapy in this sample of older adults with breast and colorectal cancer.