Sharmila Vaz

The Case for Using the Repeatability Coefficient When Calculating Test-Retest Reliability

The use of standardised tools is an essential component of evidence-based practice. Reliance on standardised tools places demands on clinicians to understand their properties, strengths, and weaknesses, in order to interpret results and make clinical decisions. This paper makes a case for clinicians to consider measurement error (ME) indices Coefficient of Repeatability (CR) or the Smallest Real Difference (SRD) over relative reliability coefficients like the Pearson’s (r) and the Intraclass Correlation Coefficient (ICC), while selecting tools to measure change and inferring change as true. The authors present statistical methods that are part of the current approach to evaluate test–retest reliability of assessment tools and outcome measurements. Selected examples from a previous test–retest study are used to elucidate the added advantages of knowledge of the ME of an assessment tool in clinical decision making. The CR is computed in the same units as the assessment tool and sets the boundary of the minimal detectable true change that can be measured by the tool.

Face Recognition and Visual Search Strategies in Autism Spectrum Disorders: Amending and Extending a Recent Review by Weigelt et al.

The purpose of this review was to build upon a recent review by Weigelt et al. which examined visual search strategies and face identification between individuals with autism spectrum disorders (ASD) and typically developing peers. Seven databases, CINAHL Plus, EMBASE, ERIC, Medline, Proquest, PsychInfo and PubMed were used to locate published scientific studies matching our inclusion criteria. A total of 28 articles not included in Weigelt et al. met criteria for inclusion into this systematic review. Of these 28 studies, 16 were available and met criteria at the time of the previous review, but were mistakenly excluded; and twelve were recently published. Weigelt et al. found quantitative, but not qualitative, differences in face identification in individuals with ASD. In contrast, the current systematic review found both qualitative and quantitative differences in face identification between individuals with and without ASD. There is a large inconsistency in findings across the eye tracking and neurobiological studies reviewed. Recommendations for future research in face recognition in ASD were discussed.

The impact of personal background and school contextual factors on academic competence and mental health functioning across the primary-secondary school transition

Students negotiate the transition to secondary school in different ways. While some thrive on the opportunity, others are challenged. A prospective longitudinal design was used to determine the contribution of personal background and school contextual factors on academic competence (AC) and mental health functioning (MHF) of 266 students, 6-months before and after the transition to secondary school. Data from 197 typically developing students and 69 students with a disability were analysed using hierarchical linear regression modelling. Both in primary and secondary school, students with a disability and from socially disadvantaged backgrounds gained poorer scores for AC and MHF than their typically developing and more affluent counterparts. Students who attended independent and mid-range sized primary schools had the highest concurrent AC. Those from independent primary schools had the lowest MHF. The primary school organisational model significantly influenced post-transition AC scores; with students from Kindergarten - Year 7 schools reporting the lowest scores, while those from the Kindergarten - Year 12 structure without middle school having the highest scores. Attending a school which used the Kindergarten - Year 12 with middle school structure was associated with a reduction in AC scores across the transition. Personal background factors accounted for the majority of the variability in post-transition AC and MHF. The contribution of school contextual factors was relatively minor. There is a potential opportunity for schools to provide support to disadvantaged students before the transition to secondary school, as they continue to be at a disadvantage after the transition.

Experiences of facilitators or barriers in driving education from learner and novice drivers with ADHD or ASD and their driving instructors

Abstract Background: Little is known about whether individuals with autism spectrum disorder (ASD) or attention deficit hyperactive disorder (ADHD) experience any specific facilitators or barriers to driving education. Objective: To explore the facilitators or barriers to driving education experienced by individuals with ASD or ADHD who obtained a learner’s permit, from the perspective of the learner drivers and their driving instructors. Methods: Data were collected from 33 participants with ASD or ADHD, and nine of their driving instructors. Results: Participants with ASD required twice as many driving lessons and more on-road tests than those with ADHD. Participants with ADHD repeated the written tests more than those with ASD. Driving license theory was more challenging for individuals with ADHD, whilst individuals with ASD found translating theory into practice and adjusting to “unfamiliar” driving situations to be the greatest challenges. Conclusion: Obtaining a driving license was associated with stressful training experience.

Should Schools Expect Poor Physical and Mental Health, Social Adjustment, and Participation Outcomes in Students with Disability?

The literature on whether students with disabilities have worse physical and mental health, social adjustment, and participation outcomes when compared to their peers without disabilities is largely inconclusive. While the majority of case control studies showed significantly worse outcomes for students with disabilities; the proportion of variance accounted for is rarely reported. The current study used a population cross-sectional approach to determine the classification ability of commonly used screening and outcome measures in determining the disability status. Furthermore, the study aimed to identify the variables, if any, that best predicted the presence of disability. Results of univariate discriminant function analyses suggest that across the board, the sensitivity of the outcome/screening tools to correctly identify students with a disability was 31.9% higher than the related Positive Predictive Value (PPV). The lower PPV and Positive Likelihood Ratio (LR+) scores suggest that the included measures had limited discriminant ability (17.6% to 40.3%) in accurately identifying students at-risk for further assessment. Results of multivariate analyses suggested that poor health and hyperactivity increased the odds of having a disability about two to three times, while poor close perceived friendship and academic competences predicted disability with roughly the same magnitude. Overall, the findings of the current study highlight the need for researchers and clinicians to familiarize themselves with the psychometric properties of measures, and be cautious in matching the function of the measures with their research and clinical needs.

School Belongingness and Mental Health Functioning across the Primary-Secondary Transition in a Mainstream Sample : Multi-Group Cross-Lagged Analyses

The relationship between school belongingness and mental health functioning before and after the primary-secondary school transition has not been previously investigated in students with and without disabilities. This study used a prospective longitudinal design to test the bi-directional relationships between these constructs, by surveying 266 students with and without disabilities and their parents, 6-months before and after the transition to secondary school. Cross-lagged multi-group analyses found student perception of belongingness in the final year of primary school to contribute to change in their mental health functioning a year later. The beneficial longitudinal effects of school belongingness on subsequent mental health functioning were evident in all student subgroups; even after accounting for prior mental health scores and the cross-time stability in mental health functioning and school belongingness scores. Findings of the current study substantiate the role of school contextual influences on early adolescent mental health functioning. They highlight the importance for primary and secondary schools to assess students’ school belongingness and mental health functioning and transfer these records as part of the transition process, so that appropriate scaffolds are in place to support those in need. Longer term longitudinal studies are needed to increase the understanding of the temporal sequencing between school belongingness and mental health functioning of all mainstream students.

Internal Consistency, Test–Retest Reliability and Measurement Error of the Self-Report Version of the Social Skills Rating System in a Sample of Australian Adolescents

The social skills rating system (SSRS) is used to assess social skills and competence in children and adolescents. While its characteristics based on United States samples (US) are published, corresponding Australian figures are unavailable. Using a 4-week retest design, we examined the internal consistency, retest reliability and measurement error (ME) of the SSRS secondary student form (SSF) in a sample of Year 7 students (N = 187), from five randomly selected public schools in Perth, western Australia. Internal consistency (IC) of the total scale and most subscale scores (except empathy) on the frequency rating scale was adequate to permit independent use. On the importance rating scale, most IC estimates for girls fell below the benchmark. Test–retest estimates of the total scale and subscales were insufficient to permit reliable use. ME of the total scale score (frequency rating) for boys was equivalent to the US estimate, while that for girls was lower than the US error. ME of the total scale score (importance rating) was larger than the error using the frequency rating scale. The study finding supports the idea of using multiple informants (e.g. teacher and parent reports), not just student as recommended in the manual. Future research needs to substantiate the clinical meaningfulness of the MEs calculated in this study by corroborating them against the respective Minimum Clinically Important Difference (MCID).

Is using the strengths and difficulties questionnaire in a community sample the optimal way to assess mental health functioning

An important characteristic of a screening tool is its discriminant ability or the measure’s accuracy to distinguish between those with and without mental health problems. The current study examined the inter-rater agreement and screening concordance of the parent and teacher versions of SDQ at scale, subscale and item-levels, with the view of identifying the items that have the most informant discrepancies; and determining whether the concordance between parent and teacher reports on some items has the potential to influence decision making. Cross-sectional data from parent and teacher reports of the mental health functioning of a community sample of 299 students with and without disabilities from 75 different primary schools in Perth, Western Australia were analysed. The study found that: a) Intraclass correlations between parent and teacher ratings of children’s mental health using the SDQ at person level was fair on individual child level; b) The SDQ only demonstrated clinical utility when there was agreement between teacher and parent reports using the possible or 90% dichotomisation system; and c) Three individual items had positive likelihood ratio scores indicating clinical utility. Of note was the finding that the negative likelihood ratio or likelihood of disregarding the absence of a condition when both parents and teachers rate the item as absent was not significant. Taken together, these findings suggest that the SDQ is not optimised for use in community samples and that further psychometric evaluation of the SDQ in this context is clearly warranted.

Self-efficacy and pain acceptance as mediators of the relationship between pain and performance of valued life activities in women and men with rheumatoid arthritis

Objective: To study whether personal factors (self-efficacy and pain acceptance) mediate the relationship between pain and performance of valued life activities in persons with rheumatoid arthritis. Methods: Persons with rheumatoid arthritis for at least four years (n = 737; 73% women) answered a questionnaire measuring self-efficacy, pain acceptance, performance of valued life activities, and self-rated pain. Relationships among these constructs were explored using univariate and multivariate analyses. Structural equation modelling was then used to examine the mediational role of personal factors on the relationship between pain and performance of valued life activities. Results: A direct negative association between pain and performance of valued life activities was identified (Beta = .34, P < .001). This suggests that people with rheumatoid arthritis who had higher levels of pain has increased difficulties in performing valued life activities. Self-efficacy and activity engagement component of pain acceptance mediated the relationship between pain and performance of valued life activities, however the pain willingness component of pain acceptance did not influence participation in valued life activities. Conclusion: These findings highlight the importance of considering personal factors, such as pain acceptance and self-efficacy, in facilitating participation in valued life activities.

Managing children's postural risk when using mobile technology at home: Challenges and strategies

Maintaining the musculoskeletal health of children using mobile information and communication technologies (ICT) at home presents a challenge. The physical environment influences postures during ICT use and can contribute to musculoskeletal complaints. Few studies have assessed postures of children using ICT in home environments. The present study investigated the Rapid Upper Limb Assessment (RULA) scores determined by 16 novice and 16 experienced raters. Each rater viewed 11 videotaped scenarios of a child using two types of mobile ICT at home. The Grand Scores and Action Levels determined by study participants were compared to those of an ergonomist experienced in postural assessment. All postures assessed were rated with an Action Level of 2 or above; representing a postural risk that required further investigation and/or intervention. The sensitivity of RULA to assess some of the unconventional postures adopted by children in the home is questioned.