Amanda Lampton

Reinforcement Learning of a Morphing Airfoil-Policy and Discrete Learning Analysis

An episodic unsupervised learning algorithm using the Q-Learning method is developed to learn the optimal shape and shape change policy of a morphing airfoil. Optimality is addressed by reward functions based on airfoil properties such as lift coecient, drag coecient, and moment coecient about the leading edge representing optimal shapes for specified flight conditions. The reinforcement learning as it is applied to morphing is integrated with a computational model of an airfoil. The methodology is demonstrated with numerical examples of a NACA type airfoil that autonomously morphs in two degrees of freedom, thickness and camber, to a shape that corresponds to specified goal requirements. Due to the continuous nature of the thickness and camber of the airfoil, this paper addresses the convergence of the learning algorithm given several action step sizes. Convergence is also analyzed with three candidate policies: 1) a fully random exploration policy, 2) a policy annealing from random exploration to exploitation, and 3) an annealing discount factor in addition to the annealing policy. The results presented in this paper show the inherent dierences in the learned action-value function when the state space discretization, policy, and learning parameters dier. It was found that a policy annealing from fully explorative to almost fully exploitative yielded the highest rate of convergence as compared to the other policies. Also, the coarsest discretization of the state space resulted in convergence of the action-value function in as little as 200 episodes.

Prediction of icing effects on the dynamic response of light airplanes

The accumulation of ice on an airplane in flight is one of the leading contributing factors to general aviation accidents. To date, only relatively sophisticated methods based on detailed empirical data and flight data exist for its analysis. This paper develops a methodology and simulation tool for preliminary safety and performance evaluations of airplane dynamic response and climb performance in icing conditions. The important aspect of dynamic response sensitivity to pilot control input with the autopilot disengaged is also highlighted. Using only basic mass properties, configuration, propulsion data, and known icing data from a similar configuration, icing effects are applied to the dynamics of a non-real-time, six degree-of-freedom simulation model of a different, but similar, light airplane. Besides evaluating various levels of icing severity, the paper addresses distributed icing which consists of wing alone, horizontal tail alone, and unequal distributions of combined wing and horizontal tail icing. Results presented in the paper for a series of simulated climb maneuvers with various levels and distributions of ice accretion show that the methodology captures the basic effects of ice accretion on pitch response and climb performance, and the sensitivity of the dynamic response to pilot control inputs.

Flight Test Evaluation of the SAFE-Cue System for Loss of Control Mitigation

*‡ § ** For more than a decade, researchers have been developing novel adaptive flight control systems that provide a means to safely operate an air vehicle in the presence of damage or failures. While these systems show great promise, integration of exemplar systems in flight test aircraft at NASA have revealed the potential for unfavorable pilot-vehicle coupling including pilot-induced oscillations. To address this issue directly, the Smart Adaptive Flight Effective Cue or SAFE-Cue system was developed. The SAFE-Cue system features an adaptive command path gain to mitigate oscillation tendencies and an inceptor force feedback cue to alert the pilot that the system is active. To evaluate the system, a representative adaptive controller that had been successfully evaluated in previous flight test programs and the SAFE-Cue system were implemented in a Calspan Learjet In-Flight Simulator. Three test pilots conducted two sorties each to evaluate the effectiveness of the combined system in the presence of pitch and roll axis failures. The resulting pilot opinion ratings and quantitative assessments of performance indicated that the SAFE-Cue system successfully eliminated the oscillation tendencies that can lead to loss of control. Furthermore, the system restored a more linear vehicle response that allowed the pilots to successfully complete the evaluation task with performance that approaches that of the baseline healthy aircraft. While the SAFE-Cue system was integrated with an adaptive controller in this application, the system is completely general and can be applied to any flyby-wire design.

Morphing Airfoils with Four Morphing Parameters

An episodic unsupervised learning simulation using the Q-Learning method is developed to learn the optimal shape and shape change policy for a problem with four state variables. Optimality is addressed by reward functions based on airfoil properties such as lift coecient, drag coecient, and moment coecient about the leading edge representing optimal shapes for specified flight conditions. The reinforcement learning as it is applied to morphing is integrated with a computational model of an airfoil. The methodology is demonstrated with numerical examples of a NACA type airfoil that autonomously morphs in four degrees-of-freedom, thickness, camber, location of maximum camber, and airfoil angle-of-attack, to a shape that corresponds to specified goal requirements. Although nonunique shapes can satisfy the aerodynamic requirements, the results presented in this paper show that this methodology is capable of learning the range of acceptable shapes for a given set of requirements. Also shown is that the agent can use its knowledge to change from one shape to another to satisfy a series of requirements with a probability of success between 92% 96%. This ability is analogous to an aircraft transitioning from one flight phase to another.

Multiresolution state-space discretization method for Q-Learning

For large scale problems Q-Learning often suffers from the Curse of Dimensionality due to large numbers of possible state-action pairs. This paper develops a multiresolution state-space discretization method for the episodic unsupervised learning method of Q-Learning, in which a state-space is adaptively discretized by progressively finer grids around the areas of interest within the state or learning space. Optimality of the learning algorithm is addressed by a cost function. Applied to a morphing airfoil with two morphing parameters (two state variables), it is shown that by setting the multiresolution method to define the area of interest by the goal the agent seeks, this method can learn a specific goal within ±0.002, while reducing the total number of state-action pairs need to achieve this level of specificity by almost 90%.

Vehicle Upset Detection and Recovery for Onboard Guidance and Control

This paper discusses the development and testing of an upset recovery architecture that is applicable for both piloted and autonomous recoveries. The architecture was first developed for unmanned vehicles and intended for fully automated implementation. The architecture was extended for use in manned aircraft and, in particular, for situations in which recoveries are being manually flown by a pilot. This extension required development of display technology for presenting recommended recovery guidance to the pilot as well as modification of recovery strategies to make them suitable for execution by a human pilot. Most recently, the architecture has been extended to accommodate off-nominal vehicle dynamics (e.g., due to actuator failures) and has been structured specifically to facilitate implementation without modification and recertification of existing flight control software. The approaches have been tested in multiple pilot-in-the-loop simulation experiments, which have shown both favorable pilot opini...

Development and Pilot-In-The-Loop Evaluation of Robust Upset-Recovery Guidance

Aircraft Loss-Of-Control (LOC) has been a longstanding contributor to fatal aviation accidents. The research presented herein is structured to directly address several known contributing and causal factors associated with vehicle upset and LOC. This paper discusses the development and evaluation of an approach to improve flight safety by visually providing closed-loop guidance for upset recovery that is robust to pilot behavior variation and is able to accommodate vehicle failures and impairment. The Damage Adaptive Guidance for piloted Upset Recovery (DAGUR) system provides continuous closed-loop recovery guidance via visual cues to reduce instances of inappropriate pilot reaction and pilot inaction. Adaptation enables the recovery module to provide appropriate guidance even in cases of vehicle damage or impairment. The recovery guidance system is also specifically designed to be robust to variations in pilot dynamic behavior (including behavior associated with high-stress situations). The adaptive recovery guidance is implemented “upstream” of the pilot and provided via visual cues; therefore it does not require modifications to existing flight control software (for fly-by-wire aircraft) and is equally applicable to non-fly-by-wire aircraft. Included desktop simulation and pilot-in-the-loop evaluation results show that the upset recovery guidance system is able to provide effective guidance for recovery from a variety of post-stall and unusual attitude upsets including cases of hardover control surface failures and that the recovery guidance is robust to large variations in pilot dynamic behavior. Additionally, pilots who evaluated the system indicated that they found the guidance to be useful and intuitive, and that it provided timely and measured recovery guidance. Quantitatively, the pilot-in-the-loop evaluation revealed that the recovery guidance significantly reduced subject pilot inceptor frequency content magnitude (energy) and the associated vehicle response.

Flight-Test Evaluation of a Loss-of-Control Mitigation System

For more than a decade, researchers have been developing novel adaptive flight control systems that provide a means to safely operate an air vehicle in the presence of damage or failures. Although these systems show great promise, integration of exemplar systems in flight‐test aircraft at NASA has revealed the potential for unfavorable pilot–vehicle coupling, including pilot‐induced oscillations. To address this issue directly, the Smart Adaptive Flight Effective Cue system was developed that features an adaptive command path gain to mitigate oscillation tendencies and an inceptor force‐feedback cue to alert the pilot that the system is active. To evaluate the system, a representative adaptive controller that had been successfully evaluated in previous flight‐test programs and the Smart Adaptive Flight Effective Cue system were implemented in a Calspan Learjet in‐flight simulator. Flight‐test evaluations consisted of three test pilots conducting two sorties each to evaluate the effectiveness of the combin...

Loss-of-Control Mitigation via Predictive Cuing

Flying near the edge of the safe operating envelope is an inherently unsafe proposition. The edge of the envelope here implies that small changes or disturbances in the system state or system dynamics can take the system out of the safe envelope in a short time and could result in loss-of-control events. This study evaluates approaches to pilot cueing based on predicting loss-of-control safety margins as the aircraft gets closer to the edge of the safe operating envelope. The goal is to provide the pilot aural, visual, and tactile cues focused on maintaining the pilot’s control action within predicted loss-of-control boundaries. The predictive architecture presented in this paper combines quantitative loss-of-control boundaries, an adaptive prediction, and data-based predictive control algorithms. The combined architecture is applied to a nonlinear transport-class aircraft. Evaluations of various loss-of-control cues using both test and commercial pilots in the NASA Ames Research Center’s vertical motion-...

Development of Spatial Disorientation Demonstration Scenarios for Commercial Pilot Training

A study of world-wide commercial jet transport accidents by The Boeing Company found the most common events to be loss of control associated with an inability of pilots to recover from upsets and unusual attitudes. A key component in some of these events was pilot spatial disorientation. In terms of non-visual illusion cases, the spatial disorientation most commonly took the form of somatogravic and/or somatogyral illusions. Somatogravic illusions are those associated with the false sensation of body tilt, while somatogyral illusions are those associated with the inability of the human to perceive extended rotations. Improved pilot training in these abnormal flight conditions, including the ability of commercial pilot training simulators to replicate spatial disorientation, is needed to reduce loss of control accidents. A Federal Aviation Association-sponsored program led by Systems Technology, Inc. is currently developing spatial disorientation demonstration training scenarios using the B747-400 flight simulator at NASA Ames Research Center. Scenarios for a missed approach/go around and a steep bank, constant altitude turn have evolved from evaluations in the simulator conducted over several weeks. Evolutions in the scenarios included selection of out-of-the-cockpit visual conditions that effectively diminished a discernible horizon and adaptations of the hexapod motion system response to create the spatial disorientation sensations. Limited results from guest pilot evaluations are presented to illustrate the potential effectiveness of the approach.