Zachary D. Asher

Development of an Autonomous Vehicle Control Strategy Using a Single Camera and Deep Neural Networks

Autonomous vehicle development has benefited from sanctioned competitions dating back to the original 2004 DARPA Grand Challenge. Since these competitions, fully autonomous vehicles have become much closer to significant real-world use with the majority of research focused on reliability, safety and cost reduction. Our research details the recent challenges experienced at the 2017 Self Racing Cars event where a team of international Udacity students worked together over a 6 week period, from team selection to race day. The team’s goal was to provide real-time vehicle control of steering, braking, and throttle through an end-to-end deep neural network. Multiple architectures were tested and used including convolutional neural networks (CNN) and recurrent neural networks (RNN). We began our work by modifying a Udacity driving simulator to collect data and develop training models which we implemented and trained on a laptop GPU. Then, in the two days between car delivery and the start of the competition, a customized neural network using Keras and Tensorflow was developed. The deep learning network algorithm predicted car steering angles using a single front-facing camera. Training and deployment on the vehicle was completed using two GTX 1070s since a cloud GPU computing instance was neither available nor feasible. Using the proposed methods and working within the competition’s strict requirements, we completed several semi-autonomous laps and the team remained competitive. The results of the competition indicated that autonomous vehicle command and control can be achieved in a limited form using a single-camera with a short engineering development timeline. This approach lacks robustness and reliability and therefore, a semantic segmentation network was developed using feature extraction from the YOLOv2 network and the CamVid dataset with a correction for the unbalanced occurrence of the different classes. Currently 31 classes can be reliably detected and classified allowing for a more complex and robust decision making architecture.

Prediction Error Applied to Hybrid Electric Vehicle Optimal Fuel Economy

Fuel economy (FE) improvements for hybrid electric vehicles using a predictive Optimal Energy Management Strategy (Optimal EMS) is an active subject of research. Recent developments have focused on real-time prediction-based control strategies despite the lack of research demonstrating the aspects of prediction that are most important for FE improvements. In this paper, driving-derived nonstochastic prediction errors are applied to a globally optimal control strategy implemented on a validated model of a 2010 Toyota Prius, and the FE results are reported for each type of prediction error. This paper first outlines the real-world drive cycle development, then the baseline model development that simulates a 2010 Toyota Prius, followed by an implementation of dynamic programming (DP) to derive the globally optimal control, and finally the use of the DP solution to evaluate prediction errors. FE comparisons are reported for perfect prediction, prediction errors from 14 alternate drive cycles, and prediction errors from 6 alternate vehicle parameters. The results show that FE improvements from the Optimal EMS are maintained under mispredicted stops, traffic, and vehicle parameters, while route changes and compounded drive cycle mispredictions may result in FE improvements being lost. Taken together, these results demonstrate that implementation of an Optimal EMS can result in a reliable FE improvement.

Enabling Prediction for Optimal Fuel Economy Vehicle Control

Vehicle control using prediction based optimal energy management has been demonstrated to achieve better fuel economy resulting in economic, environmental, and societal benefits. However, research focusing on prediction derivation for use in optimal energy management is limited despite the existence of hundreds of optimal energy management research papers published in the last decade. In this work, multiple data sources are used as inputs to derive a prediction for use in optimal energy management. Data sources include previous drive cycle information, current vehicle state, the global positioning system, travel time data, and an advanced driver assistance system (ADAS) that can identify vehicles, signs, and traffic lights. To derive the prediction, the data inputs are used in a nonlinear autoregressive artificial neural network with external inputs (NARX). Two real world drive cycles were developed for analysis in the Denver, Colorado region: a city-focused drive cycle that passes through downtown as well as a highway-focused drive cycle that transitions across multiple interstates. A validated model of a 2010 Toyota Prius in Autonomie is used to determine the vehicle control fuel economy improvements that are possible from the NARX prediction. The optimal energy management control strategy is determined using dynamic programming due to its ease of use and that the solution produced is the globally optimal solution. The control strategies compared include the existing 2010 Toyota Prius control strategy as a baseline, the neural network prediction optimal energy management control strategy, and a 100% accurate prediction optimal energy management control strategy. Results show that inclusion of various sensors and signals enables a significant amount of the fuel economy improvement with respect to 100% accurate prediction. The conclusion is that prediction based optimal energy management enabled fuel economy improvements can be realized with currently available sensors and signals.

Vehicle Electrification in Chile: A Life Cycle Assessment and Techno-Economic Analysis Using Data Generated by Autonomie Vehicle Modeling Software

The environmental implications of converting vehicles powered by Internal Combustion Engines (ICE) to battery powered and hybrid battery/ICE powered are evaluated for the case of Chile, one of the worldwide leaders in the production of lithium (Li) required for manufacturing of Li-ion batteries. The economic and environmental metrics were evaluated by techno-economic analysis (TEA) and Life Cycle Assessment (LCA) tools SuperPro Designer and Gabi®/ GREET® models. The system boundary includes both the renewable and nonrenewable energy sources available in Chile and well-to-pump energy consumptions and GHG emissions due to Li mining and Li-ion battery manufacturing. All the major input data required for TEA and LCA were generated using Autonomie vehicle modeling software. This study compares economic and environmental indicators of three vehicle models for the case of Chile including compact, mid-size, and a light duty truck. Autonomie was utilized to predict the fuel economy for the hybrid electric vehicle (HEV) and electric vehicle (EV) for each of the three vehicle types. The baseline fuel economy without vehicle electrification for each case was 44, 29, and 19 mpg, respectively. The LCA and TEA results suggest that vehicle electrification for the case of Chile would improve the metrics of sustainability and economic impacts at the nationwide level. The electrification of compact, mid-size, and a light duty truck, reduce the nationwide GHG emissions by 27%, 47%, and 37%, respectively, for the HEV scenario. Use of renewable energies in vehicle electrification, including hydroelectric and photovoltaic energies, currently 39% of the generation mix, and gasoline usage reduction reduces GHG emissions of the country. The EV scenario; however, increases the GHG emissions of the subcompact vehicle by 22%, whereas this scenario reduces the emissions of the mid-size and the light-duty truck by 25% and 47%, respectively. Use of crude oil, natural gas, and coal in Chile, currently 61% of the generation mix, contributes to increase the life cycle emissions for the EV scenario. The results of this research demonstrate that vehicle electrification has a significant impact not only in the reduction of GHG emissions but also in the economy of the country. Overall, this research will help policymakers and scientific communities to develop strategies to promote and research HEV and EV.

V2V Communication Based Real-World Velocity Predictions for Improved HEV Fuel Economy

Studies have shown that obtaining and utilizing information about the future state of vehicles can improve vehicle fuel economy (FE). However, there has been a lack of research into whether near-term technologies can be utilized to improve FE and the impact of real-world prediction error on potential FE improvements. In this study, a speed prediction method utilizing simulated vehicle-to-vehicle (V2V) communication with real-world driving data and a drive cycle database was developed to understand if incorporating near-term technologies could be utilized in a predictive energy management strategy to improve vehicle FE. This speed prediction method informs a predictive powertrain controller to determine the optimal engine operation for various prediction durations. The optimal engine operation is input into a validated high-fidelity fuel economy model of a Toyota Prius. A tradeoff analysis between prediction duration and prediction fidelity was completed to determine what duration of prediction resulted in the largest FE improvement. This study concludes that speed prediction and prediction-informed optimal vehicle energy management can produce FE improvements with real-world prediction error and drive cycle variability. This Optimal Energy Management Strategy (EMS) achieved up to a 6% FE improvement over the Baseline EMS and up to 85% of the FE benefit of perfect speed prediction. Additionally, the results from this prediction method are compared to the results of a previous study that incorporates only loca l vehicle information in speed predictions.

Towards Improving Vehicle Fuel Economy with ADAS

Modern vehicles have incorporated numerous safetyfocused Advanced Driver Assistance Systems (ADAS) in the last decade including smart cruise control and object avoidance. In this paper, we aim to go beyond using ADAS for safety and propose to use ADAS technology to enable predictive optimal energy management and improve vehicle fuel economy. We combine ADAS sensor data with a previously developed prediction model, dynamic programming optimal energy management control, and a validated model of a 2010 Toyota Prius to explore fuel economy. First, a unique ADAS detection scope is defined based on optimal vehicle control prediction aspects demonstrated to be relevant from the literature. Next, during real-world city and highway drive cycles in Denver, Colorado, a camera is used to record video footage of the vehicle environment and define ADAS detection ground truth. Then, various ADAS algorithms are combined, modified, and compared to the ground truth results. Lastly, the impact of four vehicle control strategies on fuel economy is evaluated: 1) the existing vehicle control, 2) actual ADAS detection for prediction and optimal energy management (we consider two variants ADAS1 and ADAS2 for this strategy), 3) ground truth ADAS detection for prediction and optimal energy management, and 4) 100% accurate prediction and optimal energy management. Results show that the defined ADAS scope and algorithms provide close correlation with ADAS ground truth and can enable fuel economy improvements as part of a prediction based optimal energy management strategy. Our proposed approach can leverage existing ADAS technology in modern vehicles to realize prediction based optimal energy management, thus obtaining fuel economy improvements with minor modifications.

Economic and Efficient Hybrid Vehicle Fuel Economy and Emissions Modeling Using an Artificial Neural Network

High accuracy hybrid vehicle fuel consumption (FC) and emissions models used in practice today are the product of years of research, are physics based, and bear a large computational cost. However, it may be possible to replace these models with a non-physics based, higher accuracy, and computationally efficient versions. In this research, an alternative method is developed by training and testing a time series artificial neural network (ANN) using real world, on-road data for a hydraulic hybrid truck to predict instantaneous FC and emissions. Parameters affecting model fidelity were investigated including the number of neurons in the hidden layer, specific training inputs, dataset length, and hybrid system status. The results show that the ANN model was computationally faster and predicted FC within a mean absolute error of 0-0.1%. For emissions prediction the ANN model had a mean absolute error of 0-3% across CO2, CO, and NOx aggregate predicted concentrations. Overall, these results indicate that ANN models could be used for a variety of research applications due to their economic and computational benefits such as derivation of vehicle control strategies to reduce FC and emissions in modern vehicles.

Application of Pre-Computed Acceleration Event Control to Improve Fuel Economy in Hybrid Electric Vehicles

Application of predictive optimal energy management strategies to improve fuel economy in hybrid electric vehicles is an active subject of research. Acceleration events during a drive cycle provide particularly attractive opportunities for predictive optimal energy management because of their high energy cost and limited variability, which enables optimal control trajectories to be computed in advance. In this research, dynamic-programming derived optimal control matrices are implemented during a drive cycle on a validated model of a 2010 Toyota Prius to simulate application of pre-computed control to improve fuel economy over a baseline model. This article begins by describing the development of the vehicle model and the formulation of optimal control, both of which are simulated over the New York City drive cycle to establish baseline and upper-limit fuel economies. Then, optimal control strategies are computed for acceleration events in the drive cycle. The model is first simulated with optimal control during acceleration events, then with optimal control matrices applied to different acceleration events than originally derived, so as to represent mis-prediction and mis-application of optimal control. The results show that drive cycle fuel economy can be robustly improved by applying pre-computed control matrices to acceleration events. Overall, this article demonstrates that fuel economy improvements with predictive optimal energy management are achievable without precise prediction capabilities or realtime, on-vehicle computation of optimal control.

Increasing the Fuel Economy of Connected and Autonomous Lithium-Ion Electrified Vehicles

When the sensors and signals that enable connected and autonomous vehicle (CAV) technology are combined with vehicle electrification, new vehicle control strategies that improve fuel economy (FE) are possible through perception, planning, and a control request issued to the vehicle plant. In this chapter, each CAV technology that could contribute to planning is introduced and discussed. Next, the techniques for modeling and validating a vehicle plant and running controller are discussed. Then, three planning-based control strategies are developed: (1) an Optimal Energy Management Strategy (Optimal EMS), (2) Eco-Driving strategies, and (3) an Optimal EMS combined with Eco-Driving strategies. Each of these planning-based control strategies is evaluated using a validated model of a 2010 Toyota Prius in Autonomie so that engine power, battery state of charge, and FE results can be compared. The results indicate that a 40% + FE improvement is possible when an Optimal EMS is combined with Eco-Driving for city drive cycles. Overall, as more vehicles incorporate CAV technologies and electrification, these FE improvements will be easier to achieve and will have a greater impact on transportation sustainability.