Kenton Kirkpatrick

Active Length Control of Shape Memory Alloy Wires Using Reinforcement Learning

Actively controlled shape memory alloy actuators are useful for a variety of applications that require accurate shape control. For shape memory alloy wires, strain is modulated with temperature, usually by an applied voltage difference across the length. Numerical simulation using reinforcement learning has previously been used for determining the temperature–strain relationship of a shape memory alloy wire and for synthesizing a limited control policy that relates applied temperature to desired strain. However, learning the voltage–strain relationship is of more practical interest in synthesizing feedback control laws for shape memory alloy wires since the control input in practical applications will be an applied voltage that modulates temperature. This article implements a Sarsa-based algorithm for determining a feedback control law in voltage–strain space and validates it experimentally. Experimental results presented in this article demonstrate the ability to control a shape memory alloy specimen from arbitrary initial strains ranging from zero to maximum, including intermediate strains, to an arbitrary intermediate strain. The results also demonstrate theability to control the specimen from similar arbitrary initial values of strain to zero strain. The voltage–strain learning algorithm developed in this article is a promising candidate for synthesizing practical shape memory alloy actuator feedback control laws.

Intelligent Motion Video Guidance for Unmanned Air System Ground Target Surveillance

Unmanned air systems with video capturing systems for surveillance and visual tracking of ground targets have worked relatively well when employing gimbaled cameras controlled by two or more operators: one to fly the vehicle, and one to orient the camera and visually track ground targets. However, autonomous operation to reduce operator workload and crew levels is more challenging when the camera is strapdown, or fixed to the airframe without a pan-and-tilt capability, rather than gimbaled, so that the vehicle must be steered to orient the camera field of view. Visual tracking becomes even more difficult when the target follows an unpredictable path. This paper investigates a machine learning algorithm for visual tracking of stationary and moving ground targets by unmanned air systems with nongimbaling, fixed pan-and-tilt cameras. The algorithm is based on Q learning, and the learning agent initially determines an offline control policy for vehicle orientation and flight path such that a target can be tra...

APPLICATION OF SMA ACTUATORS TO SPACESUIT GLOVE MOBILITY

A common problem with space suit systems is overly rigid gloves that resist astronaut finger motion and gripping. In this paper, the design of a mechanism is considered that exploits the small-scale nature of shape memory alloys for actuating glove fingers to assist hand gripping. The design problem is introduced by considering the objectives, constraints, and variables relevant to a spacesuit glove actuator design. Advantages and disadvantages of shape memory alloys for this application are presented. The selection of a specific shape memory alloy composition is considered, as well as specimen dimensions and connection configuration, based on ergonomic considerations. Mechanism actuation was then simulated using finite element analysis of a simulated glove-finger-actuator assembly. The results of this paper show that the designed shape memory alloy mechanism is feasible for assisting astronaut hand gripping.Copyright

Characterization of Shape Memory Alloys Using Artificial Neural Networks

Shape memory alloys are capable of delivering advantageous solutions to a wide range of engineering-based problems. Implementation of these solutions, however, is often complicated by the hysteretic, non-linear, thermomechanical behavior of the material. Existing constitutive models are largely capable of accurately describing this unique behavior, but they require prior characterization of material parameters. Current characterization procedures necessitate extensive data collection and data processing, creating a high barrier of entry for shape memory alloy application. This thesis develops a novel approach in which a form of computational intelligence is applied to the task of shape memory alloy material parameter characterization. Specifically, this work develops a methodology in which an artificial neural network is trained to identify transformation temperatures and stress influence coefficients of shape memory alloy specimens using strain-temperature coordinates as inputs. Training data is generated through the use of an existing shape memory alloy constitutive model. Factorial and Taguchi-based methods of generating training data are implemented and compared. Results show that trained artificial neural networks are capable of identifying shape memory alloy material parameters with satisfactory accuracy. Comparison of the implemented training data generation methods indicates that the Taguchi-based approach yields an artificial neural network that outperforms that of the factorial-based approach despite requiring significantly fewer training data specimens.

Rapid Characterization of Shape Memory Alloy Material Parameters Using Computational Intelligence Methods

Shape memory alloys are capable of delivering advantageous solutions to a wide range of engineering-based problems. Implementation of these solutions, however, is often complicated by the hysteretic, non-linear, thermo-mechanical behavior of the material. Although existing shape memory alloy constitutive models are largely accurate in describing this unique behavior, they require prior characterization of the material parameters. Consequently, before thorough modeling and simulation can occur for a shape memory alloy-based project, one must first go through the process of identifying several material parameters unique to shape memory alloys. Current characterization procedures necessitate extensive experimentation, data collection, and data processing. As a result, these methods simultaneously create a high barrier of entry for engineers new to active materials and impede the advanced study of shape memory alloy material parameter evolution. This paper develops a novel method in which computational intelligence methods are used to rapidly identify shape memory alloy material parameters. Specifically, an artificial neural network is trained to identify transformation temperatures and stress influence coefficients of given shape memory alloy specimens using strain-temperature coordinates as inputs. After generating training data through the use of a constitutive model, the resulting trained artificial neural network was used to identify parameters for a number of randomly generated theoretical shape memory alloys. Results presented in the paper show that the artificial neural network was able to rapidly identify both transformation temperatures and stress influence coefficients with satisfactory accuracy. The generation of training data was then repeated using Taguchi methods. Further results presented in the paper show that the artificial neural network trained with the Taguchi-based training data yielded improved characterization accuracy while using less training data.Copyright

Characterization and Control of Hysteretic Dynamics Using Online Reinforcement Learning

Hysteretic dynamical systems are challenging to control due to their hard nonlinearity and difficulty in modeling. One type of system with hysteretic dynamics that is gaining use in aerospace systems is the shape-memory alloy-based actuator. These actuators provide aircraft and spacecraft systems with the ability to achieve component-level or vehicle-level geometry or shape changes. Characterization of the material dynamics and properties of these actuators is usually accomplished with empirical testing of physical specimens, in which the hysteresis dynamics are often abstracted to very simplified models or ignored entirely. Machine learning techniques have the potential to learn hysteretic dynamics, but they routinely encounter difficulties that make them unsuitable. This paper proposes and develops a reinforcement learning-based approach that directly learns an input–output mapping characterization of hysteretic dynamics, which is then used as a control policy. A hyperbolic tangent-based model is used t...

Approximation of Agent Dynamics Using Reinforcement Learning

Reinforcement Learning for control of dynamical systems is popular due to the ability to learn control policies without requiring a model of the system being controlled. It can be dicult to learn ideal control policies because it is common to abstract out or ignore completely the dynamics of the agents in the system. In this paper, Reinforcement Learning-based algorithms are developed for learning agents’ time dependent dynamics while also learning to control them. Three algorithms are introduced. Sampled-Data Q-learning is an algorithm that learns the optimal sample time for controlling an agent without a prior model. First-Order Dynamics Learning is an algorithm that determines the proper time constants for agents known to have rst-order dynamics, while Second-Order Dynamics Learning is an algorithm for learning natural frequencies and damping ratios of second-order systems. The algorithms are demonstrated with numerical simulation. Results presented in this paper show that the algorithms are able to determine information about the system dynamics without resorting to traditional system identication.

Dimensionality effects on the Markov property in Shape Memory Alloy hysteretic environment

Shape Memory Alloy actuators can be used for morphing, or shape change, by controlling their temperature, which is effectively done by applying a voltage difference across their length. Control of these actuators requires determination of the relationship between voltage and strain so that an input-output map can be developed. To determine this policy and map the hysteretic region, a Reinforcement Learning algorithm called Sarsa was used. Proper use of Reinforcement Learning requires that the learning environment have the Markov Property. However, hysteresis spaces are commonly referenced as non-Markovian due to the fact that state history is needed to properly predict future states and rewards. This paper reveals that this formerly non-Markovian learning environment of Shape Memory Alloy hysteresis can become Markovian by means of increasing the dimensionality of the measured states. The paper compares learning attempts in both versions of the environment and will show that Reinforcement Learning is successful in the modified learning environment by learning a near-optimal policy for controlling the length of a Shape Memory Alloy wire. This is then validated by using the modified Reinforcement Learning agent to learn a near-optimal control policy in an experimental setting.

Active Length Control of Shape Memory Alloy Wires Via Reinforcement Learning

The ability to actively control the shape of aerospace structures has initiated research regarding the use of Shape Memory Alloy actuators. These actuators can be used for morphing or shape change by controlling their temperature, which is effectively done by applying a voltage difference across their length. The ability to characterize this temperaturestrain relationship using Reinforcement Learning has been previously accomplished, but in order to control Shape Memory Alloy wires it is more beneficial to learn the voltage-position relationship. Numerical simulation using Reinforcement Learning has been used for determining the temperature-strain relationship for characterizing the major and minor hysteresis loops, and determining a limited control policy relating applied temperature to desired strain. Since Reinforcement Learning creates a non-parametric control policy, and there is not currently a general parametric model for this control policy, determining the voltage-position relationship for a Shape Memory Alloy is done separately. This paper extends earlier numerical simulation results and experimental results in temperature-strain space by applying a similar Reinforcement Learning algorithm to voltage-position space using an experimental hardware apparatus. Results presented in the paper show the ability to converge on a near-optimal control policy for Shape Memory Alloy length control by means of an improved Reinforcement Learning algorithm. These results demonstrate the power of Reinforcement Learning as a method of constructing a policy capable of controlling Shape Memory Alloy wire length.