John W. Kirk

The base rate of suboptimal effort in a pediatric mild TBI sample: Performance on the Medical Symptom Validity Test

Performance on the Medical Symptom Validity Test (MSVT) was examined in 193 consecutively referred patients aged 8 through 17 years who had sustained a mild traumatic brain injury. A total of 33 participants failed to meet actuarial criteria for valid effort on the MSVT. After accounting for possible false positives and false negatives, the base rate of suboptimal effort in this clinical sample was 17%. Only one MSVT failure was thought to be influenced by litigation. The present results suggest that a sizable minority of children is capable of putting forth suboptimal effort during neuropsychological exam, even when external incentives are not readily apparent. The MSVT appears to have good potential value as an objective measure for detecting symptom invalidity in school-age youth.

The Value of the WISC-IV Digit Span Subtest in Detecting Noncredible Performance during Pediatric Neuropsychological Examinations

In adult populations, research on methodologies to identify negative response bias has grown exponentially in the last two decades. Far less work has focused on methods appropriate for children. Although several recent studies have demonstrated the appropriateness of using stand-alone symptom validity tests with younger populations, a near absence of pediatric work has investigated embedded validity indicators. The present study examined the classification value of several scores derived from the WISC-IV Digit Span subtest. The sample consisted of 274 clinically referred mild traumatic brain injury patients aged 8 through 16 years. Fourteen percent of the participants failed both the Medical Symptom Validity Test and Test of Memory Malingering, which was used as the criterion for noncredible effort. For age-corrected scaled scores, a score of ≤5 resulted in the optimal cut-score, yielding sensitivity of 51% and specificity of 96%. For Reliable Digit Span, the optimal cut-score was ≤6, with sensitivity of 51% and specificity of 92%. Although only moderately sensitive, Digit Span scores are likely to have good utility in identifying noncredible performance in relatively high-functioning older children and adolescents. Indeed, classification statistics produced in this pediatric sample compare favorably with those produced in many real-world adult patients.

The implications of symptom validity test failure for ability-based test performance in a pediatric sample.

If an examinee exerts inadequate effort to perform well during a psychological or neuropsychological exam, the resulting data will represent an inaccurate representation of the individuals true abilities and difficulties. In adult populations, methodologies to identify noncredible effort have grown exponentially in the last 2 decades. Though a comparatively modest amount of work has focused on tools to identify noncredible effort in pediatric populations, recent research has demonstrated that children can consistently pass several stand-alone symptom validity tests (SVTs) using cutoffs established with adults. However, no identified studies have examined the implications of pediatric SVT failure for ability-based test performance. The current sample consisted of 276 children aged 8-16 years referred consecutively for outpatient clinical neuropsychological consultation following mild traumatic brain injury (TBI). An earlier subgroup of this same case series that also included 17-year-olds was presented in Kirkwood and Kirk (2010). Nineteen percent of the current sample performed below the actuarial cutoff on the Medical Symptom Validity Test (MSVT). No background or injury-related variable differentiated those who passed from those who failed the MSVT. Performance on the MSVT was correlated significantly with performance on all ability-based tests and explained 38% of the total ability-based test variance. Participants failing the MSVT performed significantly worse on nearly all neuropsychological tests, with large effect sizes apparent across most tests. The results provide compelling evidence that practitioners should add objective SVTs to the evaluation of school-aged youth, even when secondary gain issues might not be readily apparent and particularly following mild TBI.

Performance on the Test of Memory Malingering (TOMM) among a large clinic-referred pediatric sample

Growing concerns with suboptimal effort in pediatric populations have led clinicians to investigate the utility of symptom validity tests (SVT) among children and adolescents. Performance on the Test of Memory Malingering (TOMM) was analyzed among a clinical sample of individuals ranging in age from 5 through 16 years. The 101 patients were referred for a variety of learning, developmental, psychiatric, and neurological concerns. All children were administered the TOMM as part of a clinical neuropsychological evaluation. Within the sample, 4 patients did not meet the adult cutoff criteria for passing the TOMM. Three of the 4 patients also demonstrated suboptimal effort on another SVT. Results revealed statistically significant correlations between TOMM performance and age, intelligence, and memory. Despite these correlations, 97 out of the 101 performed at or above the adult cutoff score. The findings suggest that children perform similarly to adults on the TOMM and that the TOMM is appropriate for use with pediatric clinical populations as young as 5 years.

Noncredible Effort during Pediatric Neuropsychological Exam: A Case Series and Literature Review

Neuropsychological test interpretation rests upon the assumption that the examinee has exerted full effort. If an individual provides inadequate effort during exam, the resulting data will be invalid and represent an underestimate of the persons true abilities. Although youth have been assumed historically to be less capable of deception than adults, acts of deception in childhood are not uncommon, even in normative populations. Yet, very few cases of children who have provided suboptimal effort during neuropsychological exam have appeared in the scientific literature. We present six clinical cases illustrating that children down to at least age 8 years can present with noncredible performance. The cases include those in which clear external incentives could be identified to those in which intrinsic or psychological factors were presumed to predominate. The fairly diverse nature of the presented cases, along with other recent work, suggests that suboptimal effort in children is apt to occur more frequently than previously recognized, even if it might occur less often than in comparable adult samples. In most of the presented cases, noncredible performance would not have been detected definitively by clinical judgment alone, reinforcing the value of routinely incorporating symptom validity tests into the neuropsychological assessment of school-aged children. The number of effort tests that have demonstrated utility in children pales in comparison to those available to the adult practitioner, although recent research now supports the use of several standalone measures with pediatric patients.

Detecting performance invalidity in children: Not quite as easy as A, B, C, 1, 2, 3 but automatized sequences appears promising

In adult populations, embedded performance validity indicators are well established, as they are time efficient, resistant to coaching, and allow for more continuous monitoring of effort than standalone measures. Although several recent studies have demonstrated the appropriateness of using standalone validity tests with school-age children, a paucity of pediatric work has examined embedded indicators. The present study investigated the value of a simple automatized sequences task to detect performance invalidity in 439 clinically referred patients with mild head injury aged 8 through 17 years. Sixteen percent of the participants failed the Medical Symptom Validity Test (MSVT). Thirteen percent failed the MSVT and also performed below established cutoffs on either the Test of Memory Malingering or Wechsler Digit Span subtest. The group classified as providing invalid data performed significantly worse than the group passing the MSVT across all sequencing conditions. Sensitivity and specificity for the total time on the sequencing task compared favorably to data produced for many respected adult-based embedded indicators (i.e., sensitivity around 50% when specificity ≥ 90%). Classification statistics for any embedded performance validity test can be expected to be worse in more severely affected populations; however, the current sequencing task appears to have value in detecting invalid performance in relatively high-functioning older children and adolescents. The fact that the task takes less than a couple of minutes to administer makes it especially appealing.

Embedded Performance Validity Indicators Within the California Verbal Learning Test, Children’s Version

To date, few studies have examined the use of embedded performance validity indicators in pediatric populations. The present study examined the utility of variables within the California Verbal Learning Test, Children’s Version (CVLT-C) in detecting noncredible effort among a pediatric mild traumatic brain injury sample. The sample consisted of 411 clinically referred patients aged 8–16 years. A total of 13% of the participants failed both the Medical Symptom Validity Test and at least one other performance validity measure. No demographic or injury-related variables differentiated the noncredible and adequate effort groups. The noncredible group performed significantly worse than the adequate effort group across a majority of CVLT-C variables. Logistic regression analysis revealed that the Recognition Discriminability (RD) score was the most robust in predicting noncredible effort. Among this relatively high-functioning sample, an RD cutoff z-score of –0.5 resulted in sensitivity of 55% and specificity of 91%. A more conservative RD cutoff z-score of –1.0 resulted in sensitivity of 41% and specificity of 97%. These findings are comparable to the classification statistics found for many embedded indicators in the adult literature. Although only moderately sensitive, the RD score on the CVLT-C appears to have good utility in identifying noncredible effort in a relatively high-functioning pediatric mTBI population.

The use of the Rey 15-Item Test and recognition trial to evaluate noncredible effort after pediatric mild traumatic brain injury

The Rey 15-Item Test (FIT) is a performance validity test commonly used in adult neuropsychological assessment. FIT classification statistics across studies have been variable, so a recognition trial was created to enhance the measure (Boone, K. B., Salazar, X., Lu, P., Warner-Chacon, K., & Razani, J. (2002). The Rey 15-Item recognition trial: A technique to enhance sensitivity of the Rey 15-Item Memorization Test. Journal of Clinical and Experimental Neuropsychology, 24(5), 561–573.). The current study assessed the utility of the FIT and recognition trial in a pediatric mild traumatic brain injury sample (N = 319, M = 14.57 years). All participants were administered the FIT and recognition trial as part of an abbreviated clinical neuropsychological evaluation. Failure on the Medical Symptom Validity Test was used as the criterion for noncredible effort. Fifteen percent of the sample met the criterion. The traditional adult cutoff score of <9 on the FIT recall trial yielded excellent specificity (98%), but very poor sensitivity (12%). When the recognition trial was utilized, a total score of <26 resulted in the best combined cutoff score (sensitivity = 55%, specificity = 91%). Results indicate that the FIT with recognition trial may be useful in the assessment of noncredible effort with children and adolescents, at least among relatively high-functioning populations.

Utility of the test of memory malingering (TOMM) in children ages 4-7 years with and without ADHD.

There is growing consensus that assessment for non-credible performance is a necessary component of pediatric neuropsychological examination. The current study examined the utility and validity of the Test of Memory Malingering (TOMM) in children ages 4–7 years with and without Attention-deficit/Hyperactivity Disorder (ADHD); 66 children (30 controls, 36 ADHD) completed all three TOMM trials. There were no significant group differences in total score on any trial, or passing rate for Trial 2 or Retention. Four-year-olds with ADHD achieved “passing” score on Trial 1 less often than controls. Across groups, performance on Trial 2 and Retention improved with age, such that 85% of the sample achieved a passing score. Four-year-olds had greater difficulty and achieved a passing score significantly less often than children 5–7 years. Moreover, half of the 4-year-olds performed worse on Retention than Trial 2, calling into question the utility of the Retention trial at this age. Performance was associated with IQ only within the ADHD group on the Retention trial. Results suggest that the TOMM can be used with confidence in clinical groups as young as 5 years. Among 4-year-olds, performance appears dependent on severity of ADHD or disruptive behaviors, and may be associated with factors other than effort.

The Relationship Between the Self-Report BASC-2 Validity Indicators and Performance Validity Test Failure After Pediatric Mild Traumatic Brain Injury

In adult populations, research on methodologies to identify noncredible performance and exaggerated symptoms during neuropsychological evaluations has grown exponentially in the past two decades. Far less work has focused on methods appropriate for children. Although several recent studies have used stand-alone performance validity tests with younger populations, a near absence of pediatric work has investigated other indices to identify response bias. The present study examined the relationship between the validity scales from the self-report Behavior Assessment System for Children, Second Edition (BASC-2) and performance on the Medical Symptom Validity Test (MSVT), a stand-alone performance validity test. The sample consisted of 274 clinically referred patients with mild traumatic brain injuries aged 8 through 17 years. Fifty patients failed the MSVT based on actuarial criteria. The majority of these patients (92%) provided valid self-report BASC-2 profiles, with only three patients (6%) producing an invalid profile due to an elevated F index. Analysis of valid/invalid self-report BASC-2 profiles and MSVT pass/fail did not reveal a significant relationship (p = 0.471, two-tailed Fisher’s exact test). These findings suggest that performance validity tests like the MSVT provide substantively different information about the validity of a neuropsychological profile than that provided by the self-report validity scales of the BASC-2.