Janette Rose

Proceed with caution: Using verbal protocol analysis to measure situation awareness

Abstract Verbal protocol analysis (VPA) is often used to elicit information about the cognitive processes of operators as it provides rich data and can be used in naturalistic settings. Recently VPA has been used to investigate the acquisition and maintenance of situation awareness (SA), and to make comparisons between groups despite a lack of research regarding the efficacy of using VPA for this purpose. This train simulator experiment investigated whether VPA can effectively measure SA. Novice and expert participants were recorded on an audio device while talking aloud throughout the trials and their verbalisations were transcribed verbatim. A coding scheme developed from the transcripts was used to code the verbalisations. Results did not support the use of VPA as a measure of SA but did show that VPA detected differences in SA errors. Potential reasons for the conflicting findings between this experiment and those conducted by other researchers are discussed. Practitioner summary: This paper examined the validity of verbal protocol analysis (VPA) as a situation awareness (SA) measure. A repeated measures experiment was conducted using a train simulator. Normal VPA did not detect changes in SA but a measure of errors did. Caution should be used when using VPA to measure SA. Abbreviations: LETSSA: low-event task subjective situation awareness technique; SA: situation awareness; SAGAT: situation awareness global assessment technique; TPD: train performance display; VPA: verbal protocol analysis

The Low-Event Task Subjective Situation Awareness (LETSSA) technique: Development and evaluation of a new subjective measure of situation awareness

Situation awareness (SA) is an important component of an individuals ability to function in a complex environment. As such, it is essential to have effective measures of an individuals SA. The most widely used subjective measure of SA is the Situation Awareness Rating Technique [SART]. However, SART has been criticised for not predicting performance or objective SA, and being highly correlated with workload. This paper describes the development and testing of a new subjective measure of SA, the Low-Event Task Subjective Situation Awareness (LETSSA) measure. To evaluate LETSSA a train simulator study was conducted with 23 novice and 26 expert freight train drivers. LETSSA was able to detect differences in manipulated SA and was comparable to an established objective SA measure (SAGAT). LETSSA was significantly associated with performance but not significantly associated with workload. While further validation is required, LETSSA shows promise as an effective subjective measure of SA.

Risk Factors for Driver Distraction and Inattention in Tram Drivers

Tram driving is a complex task requiring high levels of workload, route knowledge, and attention. Metropolitan tram networks typically contain many routes and share roads with other road users. Collision potential is highest at road intersections and areas where the track runs along the road with no segregation and when collisions occur they can cause serious injury and disruption. A study was conducted on an Australian tram network to identify collision risk factors. The approach included focus groups and discussions with 22 drivers, and observations at two high-risk locations. Data were coded thematically using a recently published taxonomy for driver distraction and inattention. The majority of factors fell into the Driver Cursory Attention category, with a large representation also in the Misprioritised and Neglected Attention categories and instances of Diverted Attention were mainly driving-related. Findings are discussed in terms of potential mitigation strategies and their implications for further refinements to driver distraction and inattention taxonomies.