Jan Joris Roessingh

Transfer of Manual Flying Skills From PC-Based Simulation to Actual Flight-Comparison of In-Flight Measured Data and Instructor Ratings

Three groups of novice pilots received training to fly aerobatic maneuvers in a light aircraft. Trainees in the control group received in-flight instruction and were given the usual briefings before each flight. Trainees in the 2 experimental groups received extra training: Each in-flight lesson was preceded by PC-based simulated flight. A total of 2,053 maneuvers were analyzed on the basis of both flight-data recordings and instructor ratings. I hypothesized that complex manual flying skills, learned on the ground, transfer to the aircraft. The results provide no objective support for this hypothesis. There were no significant differences in flying skills between the three groups as measured by the flight-data recordings. However, both experimental (PC) groups managed to fly significantly more maneuvers in the same amount of flight time in the aircraft and received better instructor ratings. I analyze these differences in detail. In the discussion, I compare the findings with published transfer experiments with PC-based simulation.

Dynamic Scripting with Team Coordination in Air Combat Simulation

Traditionally, behavior of Computer Generated Forces CGFs is controlled through scripts. Building such scripts requires time and expertise, and becomes harder as the domain becomes richer and more life-like. These downsides can be reduced by automatically generating behavior for CGFs using machine learning techniques. This paper focuses on Dynamic Scripting DS, a technique tailored to generating agent behavior. DS searches for an optimal combination of rules from a rule base. Under the assumption that intra-team coordination leads to more effective learning, we propose an extension of DS, called DS+C, with explicit coordination. In a comparison with regular DS we find that the addition of team coordination results in earlier convergence to optimal behavior. In addition, we achieved a performance increase of 20% against an unpredictable opponent. With DS+C, behavior for CGFs can be generated that is more effective since the CGFs act on knowledge achieved by coordination and the behavior converges more efficiently than with regular DS.

Tailoring a cognitive model for situation awareness using machine learning

Using a pure machine learning approach to enable the generation of behavior for agents in serious gaming applications can be problematic, because such applications often require human-like behavior for agents that interact with human players. Such human-like behavior is not guaranteed with e.g. basic reinforcement learning schemes. Cognitive models can be very useful to establish human-like behavior in an agent. However, they require ample domain knowledge that might be difficult to obtain. In this paper, a cognitive model is taken as a basis, and the addition of scenario specific information is for a large part automated by means of machine learning techniques. The performance of the approach of automatically adding scenario specific information is rigorously evaluated using a case study in the domain of fighter air combat. An evolutionary algorithm is proposed for automatically tailoring a cognitive model for situation awareness of fighter pilots. The standard algorithm and several extensions are evaluated with respect to performance in air combat. The results show that it is possible to apply the algorithm to optimize belief networks for cognitive models of intelligent agents (adversarial fighters) in the aforementioned domain, thereby reducing the effort required to elicit knowledge from experts, while retaining the required ‘human-like’ behavior.

Rewarding Air Combat Behavior in Training Simulations

Computer generated forces (CGFs) inhabiting air combat training simulations must show realistic and adaptive behavior to effectively perform their roles as allies and adversaries. In earlier work, behavior for these CGFs was successfully generated using reinforcement learning. However, due to missile hits being subject to chance (a.k.a. The probability of-kill), the CGFs have in certain cases been improperly rewarded and punished. We surmise that taking this probability of-kill into account in the reward function will improve performance. To remedy the false rewards and punishments, a new reward function is proposed that rewards agents based on the expected outcome of their actions. Tests show that the use of this function significantly increases the performance of the CGFs in various scenarios, compared to the previous reward function and a naïve baseline. Based on the results, the new reward function allows the CGFs to generate more intelligent behavior, which enables better training simulations.

Learning parameters for a cognitive model on situation awareness

Cognitive models are very useful to establish human-like behavior in an agent. Such humanlike behavior can be essential in for instance serious games in which humans have to learn a certain task, and are either faced with automated teammates or opponents. To tailor these cognitive models towards a certain scenario can however be a time-consuming task requiring a lot of domain expertise. In this paper, a cognitive model is taken as a basis, and the addition of scenario specific information is for a large part automated. The performance of the approach of automatically adding scenario specific information is rigorously evaluated using a case study in the domain of fighter pilots.

Modeling behavior of Computer Generated Forces with Machine Learning Techniques, the NATO Task Group approach

Commercial/Military-Off-The-Shelf (COTS/MOTS) Computer Generated Forces (CGF) packages are widely used in modeling and simulation for training purposes. Conventional CGF packages often include artificial intelligence (AI) interfaces, but lack behavior generation and other adaptive capabilities. We believe Machine Learning (ML) techniques can be beneficial to the behavior modeling process, yet such techniques seem to be underused and perhaps under-appreciated. This paper aims at bridging the gap between users in academia and the military/industry at a high level when it comes to ML and AI. We address specific requirements and desired capabilities for applying machine learning to CGF behavior modeling applications. The paper is based on the work of the NATO Research Task Group IST-121 RTG-060 Machine Learning Techniques for Autonomous Computer Generated Entities.

Transfer Learning of Air Combat Behavior

Machine learning techniques can help to automatically generate behavior for computer generated forces inhabiting air combat training simulations. However, as the complexity of scenarios increases, so does the time to learn optimal behavior. Transfer learning has the potential to significantly shorten the learning time between domains that are sufficiently similar. In this paper, we transfer air combat agents with experience fighting in 2-versus-1 scenarios to various 2-versus-2 scenarios. The performance of the transferred agents is compared to that of agents that learn from scratch in the 2v2 scenarios. The experiments show that the experience gained in the 2v1 scenarios is very beneficial in the plain 2v2 scenarios, where further learning is minimal. In difficult 2v2 scenarios transfer also occurs, and further learning ensues. The results pave the way for fast generation of behavior rules for air combat agents for new, complex scenarios using existing behavior models.

Co-evolutionary Learning for Cognitive Computer Generated Entities

In this paper, an approach is advocated to use a hybrid approach towards learning behavior for computer generated entities CGEs in a serious gaming setting. Hereby, an agent equipped with cognitive model is used but this agent is enhanced with Machine Learning ML capabilities. This facilitates the agent to exhibit human like behavior but avoid an expert having to define all parameters explicitly. More in particular, the ML approach utilizes co-evolution as a learning paradigm. An evaluation in the domain of one-versus-one air combat shows promising results.

A Model of Team Decision Making Using Situation Awareness

In order for agents to successfully make decisions about task allocations within a team, two elements are essential: (1) a good judgement of the situation, and (2) once the situation is known have a good decision making process to derive and assign tasks that should be performed. Within research on agent systems, little work has been done on the combination of the two. In this paper, a human-based situation awareness model is combined with a decision making procedure (which incorporates task identification and task allocation).