Eveline Schollaert

Situational judgment tests: a review of recent research

Purpose – The purpose of this paper is to give an empirically‐based review of the strengths and weaknesses of situational judgment tests (SJTs).Design/methodology/approach – The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990‐2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion‐related validity, incremental validity, construct‐related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects.Findings – Strengths of SJTs are that they show criterion‐related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs ...

A closer look at the frame-of-reference effect in personality scale scores and validity

This article contributes to the understanding of why the use of a frame-of-reference leads to increased criterion-related validity of personality inventories. Two competing explanations are described and tested. A between-subjects (N = 337) and a within-subject (N = 105) study are conducted to test the hypothesized effects of use of a frame of reference on reliability and validity. Regarding the effects on reliability, use of a frame of reference reduces within-person inconsistency (instead of between-person variability) in responding to generic items. Use of a frame of reference further leads to higher validity as a result of the reduction of between-person variability and within-person inconsistency. Yet, reducing these inconsistencies is not enough. It is also important to use a frame of reference that is conceptually relevant to the criterion. Besides implications for contextualized personality inventories, these results provide an explanation for the moderate validities of generic personality inventories.

Measurement Equivalence of the Wong and Law Emotional Intelligence Scale Across Self and Other Ratings

There exist a variety of measurement instruments for assessing emotional intelligence (EI). One approach is the use of other reports wherein knowledgeable informants indicate how well the scale items describe the assessed person’s behavior. In other reports, the same EI scales are typically used as in self-reports. However, it is not known whether the measurement structure underlying EI ratings is equivalent across self and other ratings. In this study, the measurement equivalence of an extant EI measure (Wong and Law Emotional Intelligence Scale [WLEIS]) across self and other ratings was tested. Using multiple group confirmatory factor analysis, the authors conducted a sequence of increasingly more restrictive tests of equivalence across self and other ratings. The WLEIS was found to be configurally and metrically invariant across self and other ratings. However, there was no evidence for structural invariance between rater groups. Future research is needed to test the equivalence of other EI measures across self and other ratings.

Building Situational Stimuli in Assessment Center Exercises: Do Specific Exercise Instructions and Role-Player Prompts Increase the Observability of Behavior?

Little is known about how assessment center exercises might be designed to better elicit job-relevant behavior. This study uses trait activation theory as a theoretical lens for increasing the number of behaviors that can be observed in assessment centers. Two standardized exercise stimuli (specific exercise instructions and role-player prompts) are proposed, and their effects on the observability of candidate behavior are examined. Results showed a significant effect of role-player prompts in increasing both the general number of behavioral observations and the number of behavioral observations related to three out of four dimensions. Specific exercise instructions did not have effects on observability. Implications for trait activation theory and assessment center practice are discussed.

The Interplay of Elicitation and Evaluation of Trait-Expressive Behavior: Evidence in Assessment Center Exercises

In assessment centers (ACs), research on eliciting candidate behavior and evaluating candidate behavior have largely followed independent paths. This study integrates trait activation and trait rating models to posit hypotheses about the effects of behavior elicitation via situational cues on key assessor observation and rating variables. To test the hypotheses, a series of experimental and field studies are conducted. Only when trait-expressive behavior activation and evaluation models work in conjunction, increases in observability are coupled with increases in the interrater reliability, convergent validity, discriminant validity, and accuracy of AC ratings. Implications of these findings for AC theory and practice are formulated.

The image of psychology programs: the value of the instrumental-symbolic framework.

ABSTRACT As competition for funding and students intensifies, it becomes increasingly important for psychology programs to have an image that is attractive and makes them stand out from other programs. The current study uses the instrumental–symbolic framework from the marketing domain to determine the image of different masters programs in psychology and examines how these image dimensions relate to student attraction and competitor differentiation. The samples consist of both potential students (N = 114) and current students (N = 68) of three psychology programs at a Belgian university: industrial and organizational psychology, clinical psychology, and experimental psychology. The results demonstrate that both instrumental attributes (e.g., interpersonal activities) and symbolic trait inferences (e.g., sincerity) are key components of the image of psychology programs and predict attractiveness as well as differentiation. In addition, symbolic image dimensions seem more important for current students of psychology programs than for potential students.