Tjerk de Greef

Eye Movement as Indicators of Mental Workload to Trigger Adaptive Automation

This research describes an approach to objective assessment of mental workload, by analyzing differences in pupil diameter and several aspects of eye movement (fixation time, saccade distance, and saccade speed) under different levels of mental workload. In an experiment, these aspects were measured by an eye-tracking device to examine whether these are indeed indicators for mental workload. Pupil diameter and fixation time both show a general significant increase if the mental workload increases while saccade distance and saccade speed do not show any significant differences. This assessment of mental workload could be a trigger for aiding the operator of an information system, in order to meet operational requirements.

Distilling Support Opportunities to Improve Urban Search and Rescue Missions

Current USAR missions are challenged by many factors leading to a study on how human computer interaction can provide support in this domain. Using data from a two-day observation in combination with mission reports, we applied a situated cognitive engineering design methodology to distill the operational demands, the human factors challenges, and the current and future technological design space. The operational demands result in a set of core functions that were explained in various parts of the USAR mission organization. Furthermore, an exemplary support scenario and prototype was provided in combination with claims on the envisioned effect.

Network-aware support for mobile distributed teams

An experiment evaluated network-aware support to increase understanding of the factors that are important for successful teamwork in mobile geographically dispersed teams of first responders. Participants performed a simulated search and rescue team task and were equipped with a digitized map and real-time situation updates on the location of other participants in a simulated disaster area. The connection to a server, however, was made deliberately error-prone, leading to occasional losses of network connections. Consequently, participants were not provided with real-time situation updates. To deal with this problem we equipped team members with a network-aware application that signaled network loss to them and adapted the graphical representation of the location of fellow team members accordingly to the quality of location information present. The experiment revealed that presenting complete and reliable geospatial information improves teamwork. Teams connected to a server over a fast and reliable link showed superior performance over teams with no network connection whatsoever to a server. The present study failed, however, to demonstrate the added value of network-aware support when teams had to collaborate in the presence of an unreliable communications infrastructure. Although participants indicated a slight preference for the network-aware application over a condition without support signaling network loss, no differences were observed in team process and outcome measures.

sCEthics: embedding ethical values in cognitive engineering

Contemporary and future technologies are getting more intelligent and connect easily to one another, potentially leading to conflicts with human ethical values. Value Sensitive Design (VSD) is promising for its focus on ethical values but lacks an explicit and systematical elicitation of requirements, which is targeted explicitly in the situated Cognitive Engineering (sCE) methodology. Therefore, a sCE tool was combined with VSD leading to the sCEthics design methodology that accounts for ethical values in an explicit and systematical way. This entailed implementing five requirements and these were evaluated with eleven participants. The results reveal that the implementation of requirements relating to values, policies and design patterns are seen as useful, but require minor revisions. The extended scenario system was not seen as very useful while both visualization overviews were perceived as extremely useful. These results show that sCEthics serves as a structured and guiding user-centered approach towards the design of ethical intelligent systems.

Combining adaptive automation and adaptive teams in a naval command centre

Motivation -- Adaptive teams and adaptive automation promote a flexible work division between humans mutually and between humans and machine in order to overcome limitations in human information processing under highly variable workloads. We wanted to construct a framework that enables both system-supported adaptability in a team and adaptive automation using the same concept. Research approach -- A situated iterative cognitive engineering approach was used to establish a set of requirements for the system support of adaptive teams. Findings/Design -- A framework that supports a number of different work allocation schemes can be constructed. The schemes are compared using informal narrative descriptions. The framework accommodates the teams currently used in command centres of naval ships and could aid in the adaptability of such teams. Research Limitations/Implications -- This study accommodates the hierarchical teams that are common in defence organisations. A re-evaluation is necessary when the model is applied to other organisations. Originality/Value -- The research extends the research on adaptive automation with a team-centred design that uses the same concepts for system and team adaptation. Take away message -- Adaptive teams and adaptive automation can be combined into a single framework that uses the same concepts for cooperation between users mutually and between users and system that does not unduly complicate the design of the system algorithms.

Using eye tracker data in air traffic control

Motivation/Research approach -- An exploratory study was conducted to investigate whether eye movement metrics discriminate between different air traffic control tasks. Findings/Design -- The results show the three tasks elicit different eye movement, as Yarbus (1967) also showed in static pictures, and that a number of eye tracking metrics demonstrate the differences. Research limitations/Implications -- The effect was demonstrated using only one participant. The results can be used to further study various eye movement metrics. Originality/Value -- The research demonstrates that different calculus distinguishes between tasks allowing targeting specific support given the type of task. Take away message -- A combination of eye tracker metrics discriminates between tasks helping to provide flexible task support.

An ethical boundary agent to prevent the abdication of responsibility in combat systems

Remote controlled combat systems and future autonomous systems create unprecedented capabilities to control the delivery of military force. However, there is a growing concern, which is only starting to be addressed now, that ethical values are violated as a result of high levels of autonomy and remote control. Before combat technologies can be deployed, we need to ensure that their usage enhances, rather than undermines, human decision-making capacities. To do this, we propose combining the idea of an ethical boundary agent with a partnership approach. The partnership approach is seen as a promising area for improved efficiency in interactive systems. In this paper, we claim that the ethical boundary agent safeguards compliance with implemented legal and moral boundaries. Hypothetically, such an agent prevents human operators from abdicating from their responsibilities. It does this by challenging operators to think critically about whether actions meet relevant ethical standards.

Editorial for special issue on cognitive ergonomics for work, education and everyday life

This special issue presents five papers organized by the theme: ‘‘Cognitive Ergonomics for Work, Education and everyday life’’. As a central theme of the 31st European Conference on Cognitive Ergonomics (2013), the theme is organized around important issues related to the evolution of work and education and to technical evolutions concerning health and mobility. On the one hand, it is related to the evolution of work and education, such as learning at work and learning at school. Indeed, the twenty-first century has witnessed the increasing inter-penetration of the worlds of work and education in the society. New technologies at school, such as mobile technology, question the complex ecology of school, in particular the role of teachers. Education also infiltrates work—given needs for flexibility and lifelong learning—supported by training or by enabling work environments. Furthermore, learning and development cross the boundaries of established educational and work institutions as evidenced by online communities of practice. Such evolution questions the inter-relation between productive and constructive activities, as well as the potential of technical/social/organizational environments to enable cognitive development of individuals and teams, and to support their motivation and engagement, autonomy, potential for action, and control of tasks and activities (Falzon 2014). Two papers are related to this concern, exploring the introduction of digital technology at school (Rubegni and Landoni 2016), and the role of online communities in a professional domain (Barcellini et al. 2016). On the other hand, it is related to the societal evolutions of everyday life in two particular domains, health and mobility. Here again, the introduction of technologies aims at enhancing people’s capacity in self-monitoring and in decision making as well as at enabling people’s autonomy. In the health domain, home-based self-monitoring technology aims to enhance people’s capacity for self-care and promote preventive self-monitoring activities at home. One paper is related to this issue, exploring design and use of self-monitoring technologies (Verdezoto and Gronvall 2016) in the ‘quantified self’ movement. In the mobility domain, new tools and services are designed to support journey decision making. Two papers are related to this issue either for public transportation (Grison et al. 2016) or driving (Sharples et al. 2016).

Design for responsibility: safeguarding moral perception via a partnership architecture

Advanced warfare technologies (AWT) create unprecedented capabilities to control the delivery of military force up to the point, some argue, that we are loosing humanity. But dependence on them generates difficult moral challenges impacting the decision-making process, which are only beginning to be addressed. In order to arrive at an informed opinion about the impact of AWT on decision-making, we need to know more about what AWTs are and how they operate. We provide a short overview of the different types of AWTs and discuss the key principles that underlie Humanitarian Law. We also discuss the impact of physical distance and increased levels of autonomy on AWT and discuss the challenges posed to moral perception. Before such systems can be deployed, we need to rest assured that their usage enhances, rather than undermines, human decision-making capacities. There are important choices to be made, and sound design is ‘design for responsibility’. As a solution, we therefore propose the partnership architecture that embeds concurrent views of the world and working agreements, ensuring that operators use appropriate information in the decision-making process.

Eye Metrics for Task-Dependent Automation

Future air traffic is expected to grow increasingly, opening up a gap for task dependent automation and adaptive interfaces, helping the Air Traffic Controller to cope with fluctuating workloads. One of the challenging factors in the application of such intelligent systems concerns the question what the operator is doing in order to optimize support and minimize automation surprises. This study questions whether eye metrics are able to determine what task the operator is engages in. We therefore examined A) if the eye-path would differ for three different ATC tasks and B) whether this effect can be quantified with six eye-metrics. In an experiment, the six eye-metrics were calculated and used as a dependent variable. The results show that some tasks can be inferred by eye movement metrics and other metrics infer workload, although none inferred by both task and workload.