Allan Oliveira

SAW-Oriented User Interfaces for Emergency Dispatch Systems

Situational awareness (SAW) is a concept widely spread in application areas that require critical decision-making, such as in emergency dispatching systems. SAW is related to the level of consciousness that an individual or team has to a situation. SAW-oriented UI for critical systems require specialized user interfaces to provide operators a dynamic understanding of what is happening in an environment. The information to be managed by such interfaces affects the way operators in an emergency dispatch system acquire, maintain and recover SAW. A challenging issue on the design of SAW-oriented interfaces is how the human-system interaction process can be redesigned for the enhancement of SAW considering environments with potential large scale heterogeneous multi sensors data in complex, ever-changing situations. The problem is increased when such information is subject to uncertainty, which may compromise the acquisition of the situational awareness. Also, humans are expected to make decisions based on their own understanding of what is going on, which allied to experience and expertise can be valuable assets to be used to process refinement during the construction of an incremental knowledge. The goal of this paper is to introduce a conceptual framework to create specialized interfaces that support the participation of operators in the process of SAW acquisition. Such SAW-oriented interface presents a tight integration between the operator and the other phases of an assessment process, such as information quality assessment, information fusion and information visualization. A robbery event report, in an emergency dispatch system, is used as a case study to demonstrate practical and promising results of the applicability of our solution.

Model-Based Design of Interactions That can Bridge Realities - The Augmented "Drag-and-Drop"

The development of Augmented Reality (AR) applications is still driven by development at source-code level. Although recent approaches focus to standardize AR functionality, to our knowledge a declarative and model-driven design (MDD) has not been applied for AR development so far. MDD approaches have been successfully applied to model user interfaces for a wide spectrum of modes (such as speech-command interfaces, or remote controls) and media such as HTML, 3D, and smart phone interfaces for instance. In this paper we propose a MDD approach for modeling seamless interaction between Web and AR interfaces. Therefore we implemented a prototype of a web furniture shop that supports arranging furniture in an augmented reality to prove our approach. We then present our approach of modeling a reality spanning Drag-and-Drop interaction between a 2D browser and an AR scene. Finally, we discuss the issues that we were confronted with to support reality spanning interaction like switching interaction modes and coordinate systems and present limitations that we have experienced with using other control modes, like a Wii-Controller or glove-based gesture detection.

Augmented reality and tangible user interfaces integration for enhancing the user experience

The integration of post-wimp computer interfaces arises as an alternative to meet individual limitations of each one, considering both interaction components and feedbacks to users. Tangible interfaces can present restrictions referring to physical space on tabletop architectures, which limits the manipulation of objects and deprecates the interactive process. Hence, this paper proposes the integration of techniques of mobile Augmented Reality with tabletop tangible architecture for blending real and virtual components on its surface, aiming to make the interactive process richer, seamless and more complete.

Quality-aware human-driven information fusion model

Situational Awareness (SAW) is a widespread concept in areas that require critical decision-making and refers to the level of consciousness that an individual or team has about a situation. A poor SAW can induce humans to failures in the decision-making process, leading to losses of lives and property damage. Data fusion processes present opportunities to enrich the knowledge about situations by integrating heterogeneous and synergistic data from different sources and transforming them into more meaningful subsidies for decision-making. However, a problem arises when information is subject to problems concerning its quality, especially when humans are the main sources of data (HUMINT). Motivated by the informational demand from the emergency management domain and by the limitations and challenges of the state of the art, this work proposes and describes a new information fusion model, called Quantify (Quality-aware Human-Driven Information Fusion Model), whose main contribution is the exhaustive use of the quality information management throughout the fusion process to parameterize and to guide the work of humans and systems. To validate the model, an emergency situation assessment system prototype was developed, called ESAS (Emergency Situation Assessment Systems). Then, experts from the Sao Paulo State Police (PMESP) tested the prototypes and the system was evaluated using SART (Situation Awareness Rating Technique), which showed higher rates of SAW using the Quantify model, compared to the model from the state-of-the-art, especially in questions relating to the components of resource supply and situational understanding.

A Head-up Display with Augmented Reality and Gamification for an E-Maintenance System: Using Interfaces and Gamification to Motivate Workers in Procedural Tasks

A current challenge in industrial systems, such as E-maintenance, responsible for gathering all maintenance-related data in a single system, is how to display information to users. This same challenge is present when considering new devices for visualization, such as Head Mounted Displays -HDM (e.g. Google Glass). Advanced interfaces such as Augmented Reality (AR) and Head-Up Display (HUD) provide a means to display these information for users. However, there is still a lack of theoretical design studies (studies that do not consider current technologies limitations) of Augmented Reality and HUD. Another problem in industrial scenarios is motivation, especially considering repetitive procedural works. Therefore, this paper present and discuss a high fidelity theoretical prototype of an interface for maintenance, that uses Gamification techniques to motivate the user, and AR and HUD to display information in a HMD device.

Model-based design of multimodal interaction for augmented reality web applications

Despite the increasing use of Augmented Reality (AR) in many different application areas, implementation support is limited and still driven by development at source-code level. Although efforts have been made to overcome these limitations, there is a clear gap between authoring environments and source code level framework approaches for creating AR interfaces for the web with multimodal control. Model-based design for interaction can offer support to fill this gap between authoring environments and frameworks. However, to the best of our knowledge, a declarative and model-driven design (MDD) has not yet been applied to model AR interfaces for a wide spectrum of modes. Thus, this paper presents an extension of the model-driven design to cope with interactors, whose novelty lies on the introduction of a modeling approach targeted at AR developers and designers in their task to design new forms of interactions that can be later used in authoring environments. To validate our approach, we demonstrate how a reality spanning Drag-and-Drop interaction can be modeled for an online furniture shop. And we implemented a gesture based control to show how new control modes can be added to an existing MDD-based design to extend the interaction capabilities.

Integration Framework of Augmented Reality and Tangible Interfaces for Enhancing the User Interaction

The integration of post-wimp computer interfaces arises as an alternative to meet the individual limitations of each modality, considering both interaction components and the feedbacks to users. Tangible interfaces can present restrictions referring to physical space on tabletop architectures, which limits the manipulation of objects and deprecates the interactive process. Hence, this paper proposes the integration of techniques of mobile Augmented Reality with tabletop tangible architecture for blending real and virtual components on its surface, aiming to make the interactive process richer, seamless and more complete.

Desenvolvimento de Interface de Usuário para a Melhoria da Consciência Situacional em Gerenciamento de Emergências

ConscienciaSituacional(SituationalAwareness-SAW)eumprocesso cognitivo amplamente difundido em areas que demandam a tomada de decisao critica e se refere ao nivel de consciencia que um individuo ou equipe detem sobre uma situa- cao. No dominio de gerenciamento de emergencias, a informacao gerida por sistemas de apoio a avaliacao de situacoes afeta a forma como os operadores humanos adqui- rem, mantem e recuperam SAW. Tais rotinas, cruciais para a efetividade das atividades decisorias, sao desafiadoras, nao somente pela natureza critica das situacoes, mas tam- bem devido a dinamicidade das informacoes que a compoem. Falhas de SAW podem induzir operadores humanos a erros no processo decisorio e acarretar riscos a vida, ao patrimonio ou ao meio ambiente. Alem disso, dominios criticos tambem exigem uma preocupacao constante com a qualidade da informacao, principalmente quando humanos sao tambem fontes de informacao, o que pode gerar incertezas. Neste con- texto, interfaces orientadas a SAW exercem um papel fundamental, nao somente para a representacao de informacoes que podem ser incertas, mas tambem para promover uma gestao participativa da situacao que evolui com o tempo. Este trabalho apresenta uma interface orientada por principios de SAW e demonstra como inclui-la em um processo de avaliacao de situacoes de emergencias, composto por rotinas inovadoras de fusao, avaliacao, representacao e visualizacao de informacoes situacionais. Em es- tudo de caso, foram utilizados eventos de roubo relatados ao servico de atendimento a emergencias da Policia Militar do Estado de Sao Paulo (PMESP). Uma avaliacao de SAW foi conduzida com operadores da PMESP, que testaram e avaliaram a interface desenvolvida, revelando indices satisfatorios quanto ao entendimento de uma situacao.

The electronic maintenance situational awareness interface

The new technologies for maintenance can vastly improve safety and efficiency. E-Maintenance is a technology that can aggregate data from multiple databases. Internet of Things (IoT) is defined as a novel paradigms to enable “things” (any object with a computational device able of communicating) to communicate, exchanging data with each other. IoT can detect and read the environment and equipments in real time. These advances in technology can result in work practice improvements for maintenance technicians if they are well applied. To bring these technologies to technicians, a maintenance system needs to consider the data they actually need to perform better in their current context. The systems User Interface needs to be goal-driven, providing the right information at the right time. This paper explores the challenge of delivering knowledge to technicians in a way that improves their efficiency and safety. It introduces a conceptual framework to Situation Awareness (SA) that can be applied in designing systems for maintenance technicians. This framework defines seven entities of importance (personal, procedure, equipments, system, environment, team and enterprise) in maintenance work and how they affect the users SA. The objective is to fill the current knowledge gap in studies of SA and assist the process of understanding data and situations of maintenance activities.

Human-Centered Interfaces for Situation Awareness in Maintenance

In Maintenance, a vital activity in industry, ineffective Situation Awareness (SAW) is responsible for 13-17% of accidents. Therefore, a successful SAW has the potential of leveraging safety and efficiency, especially in field work. Bibliographic revision shows that the main gap to develop this idea is the conflict between the structure of Maintenance (procedural) and that of SAW (dynamic). Using a holistic study to elaborate a conclusive definition of SAW in the field, our work was able to solve this dichotomy of structures by developing a conceptual framework which maintains the task-oriented nature of field operation while clustering SAW inputs into entities to outline their potential effects. This Conceptual Framework of Situation Awareness in Maintenance (CFSAM) acknowledges the link between SAW and UIs, and creates a definitive list of multiple factors/entities that supports responsiveness and improves reliability and resilience. The seven entities identified are: task, equipment, system, environment, team, enterprise and personal. To each of these entities, CFSAM assigns a role in the UI design, and analyses challenges and solutions. This new approach of considering SAW-oriented design is a multidisciplinary effort and so far the results are promising: it facilitates an efficient design of SAW in maintenance field work, increases the focus on safety and efficiency and leverages the potential of developing a coherent system with high level of adherence.