Deep Neural Network-Based Controllers with Guaranteed Exponential Convergence: A Lyapunov- and Concurrent Learning-Based Approach

Abstract

With the increase in computational power, there have been significant developments in deep neural network (DNN)-based control approaches due their function approximation property. DNNs are capable of approximating complex dynamic functions by learning complex relationships be tween the input-output data through training and optimization. Classical DNN training methods utilize numerical optimization tools to improve the function approximation performance. How ever, in most cases, DNN-based controllers are implemented in an open-loop manner, meaning that the DNN model in the controller does not adapt its weights online and therefore fails to take into consideration any unexpected behaviors that could hinder system stability and robustness. To address these issues, control designers have developed Lyapunov- and DNN-based control approaches that adjust the weights of the DNN during controller implementation. Lyapunov-based stability analyses have been performed for two distinct DNN update approaches: a multi-timescale approach and a real-time update approach, each guaranteeing safe trajectory tracking. However, these adaptation laws do not ensure convergence of the DNN parameters. To ensure the parameter convergence, a concurrent learning (CL) approach is introduced. CL is a data-based approach that leverages prior data to modify an adaptive parameter update law such that the parameters converge towards their ideal values. In this dissertation, DNN-based update laws are augmented with a CL framework, and the stability and convergence of the resulting error system are analyzed for various dynamic systems, including a hybrid exoskeleton. Chapter 1 presents the motivation for developing a CL-augmented DNN weight update law. It begins by introducing DNNs and their function approximation capabilities, followed by an overview of their application in control systems. After that, the limitations of traditional DNN approaches are discussed followed by a discussion on the importance of DNN weight update laws for improving function approximation and tracking performance. Two existing DNN parameter update strategies are then discussed, highlighting their advantages and limitations. The motivation for incorporating the CL framework to improve parameter convergence is subsequently presented. Finally, the chapter provides an overview of the dissertation, summarizing the development of CL-augmented DNN update laws for various classes of dynamic systems. In Chapter 2, a CL-based approach to improve the parameter convergence of the output layer weights of a DNN-based controller for an uncertain control-affine system is presented. The dynamic model in this chapter includes unknown disturbances and an unknown control effectiveness matrix, requiring an additional adaptation law to estimate the unknown parameters in the effectiveness matrix, modifications to the DNN adaptation law, and a distinctive approach to the stability analysis. Additionally, a rigorous Lyapunov-based stability analysis is performed to prove that the tracking errors and the output-layer weight estimation errors exponentially converge to an ultimate bound and that the proposed control law and all associated adaptation laws are bounded. Moreover, this chapter includes simulations and experiments to validate the control development. For example, Monte Carlo simulations were performed on a two-state Van der Pol oscillator model to evaluate the robustness and performance of the proposed control framework. Specifically, Monte Carlo simulations were conducted for varying DNN architectures to determine the architecture (i.e., the number of hidden layers and neurons per layer) that achieved the best tracking performance. The best DNN architecture was then used in subsequent simulation and experimental studies to validate the effectiveness of the proposed multi-timescale and CL-based DNN approach. In the later simulation study, the DNN weights and biases were initialized using three widely utilized DNN initialization techniques, including randomized initial weights. The results obtained from the initialization methods were later evaluated and compared to demonstrate the function approximation performance and the overall robustness of the proposed method. In the experimental study, the control framework was used to track a desired yaw rate for a Quanser Aero 2 testbed, which is a reconfigurable and fully integrated dual-rotor experimental aerodynamic system that is designed for use in control system research and education. In Chapter 3, CL is integrated into a DNN weight adaptation law for an uncertain, nonlinear Euler-Lagrange (EL) system. The control development only uses a single DNN, for the given EL system, unlike a prior DNN method by the author, to reduce the computational complexity and improve implementation efficiency, while maintaining an effective approximation and control performance. A control effectiveness term has also been incorporated into the input torque of the system to better reflect real systems, thereby broadening the control applicability. Furthermore, a Lyapunov-based stability analysis of the proposed control method was performed to ensure that the trajectory tracking and output-layer weight estimation errors aren’t just bounded, but that they also globally exponentially converge to an ultimate bound that can be arbitrarily small, provided that a finite excitation (FE) learning condition is satisfied. Monte Carlo simulations were also conducted on an EL dynamic model to verify the effectiveness and robustness of the proposed control methodology across different DNN architectures. The proposed control law with the best DNN architecture, as determined by the Monte Carlo simulations, was then applied to a two–link robotic system to track a desired cycling trajectory, which demonstrated the controller’s real world effectiveness in achieving an accurate tracking performance. Moreover, the experiments compared the tracking performance of the proposed controller with other controllers to highlight the superiority of the proposed control method. In Chapter 4, a DNN- and admittance-based control framework for a hybrid exoskeleton is developed. Specifically, DNNs were used to approximate the complex dynamics of a hybrid exoskeleton and were integrated into the function electrical stimulation (FES) control input. Moreover, an admittance-based motor controller is developed for the hybrid exoskeleton to indirectly track a desired torque trajectory. An advantage of DNN-based FES control is that it is an adaptive control approach and can be adjusted according to a user’s dynamics, which can help to lower the FES inputs to slow the onset of fatigue and to improve rehabilitation. In this chapter, the inner-layer weights and biases of the DNN are updated offline; whereas, the output-layer weights are updated online using a Lyapunov-based adaptive update law. In an effort to improve the DNN performance, this chapter also augments the real-time output-layer DNN update law with CL in spired terms for the hybrid exoskeleton (i.e, a dynamic model with two inputs). Lyapunov-based stability and passivity analyses were performed to ensure global uniformly ultimately bounded position tracking of the FES controller and passivity of the admittance controllers in addition to global tracking of the admittance errors. Finally, a preliminary simulation was done to evaluate the performance of the proposed control approach. In Chapter 5, a DNN-based controller is developed for an EL dynamic system. Specifically, the controller includes an adaptive feedforward DNN term that approximates the uncertain dynamics, and robust terms to provide robustness to disturbances. Unlike Chapter 3, in this chapter a Lyapunov-based DNN weight adaptation law is designed to enable real-time training of all the DNN’s weights, where for the first time, the adaptation law is augmented with a newly developed, stability-backed, and CL inspired term to ensure that the estimated DNN weights converge towards their optimal values, establishing a new standard for adaptive control. Importantly, this approach 1) improves the tracking and learning performance of DNN-based controllers, 2) ex tends the traditional neural network (NN) control approach (those with a single-hidden-layer) to DNNs with a selectable number of hidden-layers, and 3) ensures that the DNN weights converge toward their optimal values. Moreover, a rigorous Lyapunov-based stability analysis is performed to prove exponential convergence of both the trajectory tracking errors and the DNN weight estimation errors to an ultimate bound. Additionally, a number of simulations and experiments were conducted on the proposed control architecture to further validate the proposed controller’s performance and effectiveness. In Chapter 6 the dissertation is concluded by highlighting the contributions and findings of each individual chapter. This chapter also discusses the limitations of the the present study and suggests further areas for research.