Robustly Invertible Nonlinear Dynamics and the BiLipREN: Contracting Neural Models with Contracting Inverses

IntroductionDeep brain stimulation (DBS) is a promising therapy for treatment-resistant major depressive disorder (MDD). MDD involves the dysfunction of a brain network that can exhibit complex nonlinear neural dynamics in multiple frequency bands. However, current open-loop and responsive DBS methods cannot track the complex multiband neural dynamics in MDD, leading to imprecise regulation of symptoms, variable treatment effects among patients, and high battery power consumption.MethodsHere, we develop a closed-loop brain-computer interface (BCI) system of predictive neuromodulation for treating MDD. We first use a biophysically plausible ventral anterior cingulate cortex (vACC)-dorsolateral prefrontal cortex (dlPFC) neural mass model of MDD to simulate nonlinear and multiband neural dynamics in response to DBS. We then use offline system identification to build a dynamic model that predicts the DBS effect on neural activity. We next use the offline identified model to design an online BCI system of predictive neuromodulation. The online BCI system consists of a dynamic brain state estimator and a model predictive controller. The brain state estimator estimates the MDD brain state from the history of neural activity and previously delivered DBS patterns. The predictive controller takes the estimated MDD brain state as the feedback signal and optimally adjusts DBS to regulate the MDD neural dynamics to therapeutic targets. We use the vACC-dlPFC neural mass model as a simulation testbed to test the BCI system and compare it with state-of-the-art open-loop and responsive DBS treatments of MDD.ResultsWe demonstrate that our dynamic model accurately predicts nonlinear and multiband neural activity. Consequently, the predictive neuromodulation system accurately regulates the neural dynamics in MDD, resulting in significantly smaller control errors and lower DBS battery power consumption than open-loop and responsive DBS.DiscussionOur results have implications for developing future precisely-tailored clinical closed-loop DBS treatments for MDD.

Non-holonomic vehicle motion has been studied extensively using physics-based models. Common approaches when using these models interpret the wheel/ground interactions using a linear tire model and thus may not fully capture the nonlinear and complex dynamics under various environments. On the other hand, neural network models have been widely employed in this domain, demonstrating powerful function approximation capabilities. However, these black-box learning strategies completely abandon the existing knowledge of well-known physics. In this paper, we seamlessly combine deep learning with a fully differentiable physics model to endow the neural network with available prior knowledge. The proposed model shows better generalization performance than the vanilla neural network model by a large margin. We also show that the latent features of our model can accurately represent lateral tire forces without the need for any additional training. Lastly, We develop a risk-aware model predictive controller using proprioceptive information derived from the latent features. We validate our idea in two autonomous driving tasks under unknown friction, outperforming the baseline control framework.

Nonlinear System Identification

Deep learning methods achieve remarkable predictive performance in modeling complex, large-scale data. However, assessing the quality of derived models has become increasingly challenging, as more classical statistical assumptions may no longer apply. These difficulties are particularly pronounced for spatio-temporal data, which exhibit dependencies across both space and time and are often characterized by nonlinear dynamics, time variance, and missing observations, hence calling for new accuracy assessment methodologies. This paper introduces a residual correlation analysis framework for assessing the optimality of spatio-temporal relational-enabled neural predictive models, notably in settings with incomplete and heterogeneous data. By leveraging the principle that residual correlation indicates information not captured by the model, enabling the identification and localization of regions in space and time where predictive performance can be improved. A strength of the proposed approach is that it operates under minimal assumptions, allowing also for robust evaluation of deep learning models applied to multivariate time series, even in the presence of missing and heterogeneous data. In detail, the methodology constructs tailored spatio-temporal graphs to encode sparse spatial and temporal dependencies and employs asymptotically distribution-free summary statistics to detect time intervals and spatial regions where the model underperforms. The effectiveness of what proposed is demonstrated through experiments on both synthetic and real-world datasets using state-of-the-art predictive models. • Proposes a novel residual correlation analysis to assess the quality of deep spatio-temporal models and identify regions where the predictions can be improved. • Complements traditional accuracy-based evaluations by offering an independent, metric-agnostic assessment of model quality. • Operates under minimal assumptions and remains effective even with missing or heterogeneous data, making it broadly applicable to real-world scenarios and deep learning models.

With an unprecedented increase in the global aging population and with it, the age-related neuromuscular dysfunction diseases, there is an exorbitant and escalating need for physical rehabilitation. Delivering these services - especially to those that are most vulnerable - under the current COVID-19 pandemic restriction for physical-distancing, is an even greater challenge. Interest in telerehabilitation is spiking, and robotic tel-erehabilitation could drastically improve patients' access to care. Some of the major challenges in developing the control methods for these robots are identifying, estimating, and overcoming the effects of dynamic modeling uncertainties, nonlinearities, and disturbances. Having humans in the loop creates the additional need for safety and compliance. Telerehabilitation control methods have the added requirement of delivering telepresence and addressing communication delays which, if not managed, could result in ineffective therapy, destabilize the system, and even cause injury. In this paper, we present a novel adaptive robust integral Radial Basis Function Neural Network Impedance model (RBFNN-I) control method for telerehabilitation with robotic exoskeletons which compensates for dynamic modeling uncertainties in the presence of external human torques and time delays. One of the salient features of the proposed control system is the implementation of a new human torque regulator which improves telepresence. Stability proof using Lyapunov stability theory is shown for the proposed control method. An exoskeleton was designed and used for unilateral and bilateral telerehabilitation simulations. Excellent tracking performance, telepresence and stability was achieved in the presence of large, variable and asymmetric time delays and human torques.

Predicting service request in support centers based on nonlinear dynamics, ARMA modeling and neural networks

A neurological abnormality in the brain that manifests as a seizure is the prime risk of epilepsy. The earlier and accurate detection of the epileptic seizure is the foremost task for the diagnosis of epilepsy. In this chapter, a nonlinear deep neural network is used for seizure classification. The proposed network is based on the autoencoder that significantly explores the non-linear dynamics of the electroencephalogram (EEG) signals. It involves the traditional deep neural domain expertise to extract the features from the raw data in order to fit a deep neural network-based learning model and predicts the class of the unknown seizures. The EEG signals are subjected to an autoencoder-based neural network that unintendedly extracts the significant attributes that are applied to the softmax classifier. The achieved classification accuracy is up to 100% on different publicly available Bonn University database classes. The proposed algorithm is suitable for real-time implementation.

In this work, Lagrangian Neural Networks (LNNs) were applied to determine the Lagrangian of a mechanical system and to predict its trajectory in the state space. Neural networks, Lagrangian mechanics, Newtonian mechanics, dynamic models, numerical integration, parametric optimization, and evaluation criteria were applied for the case study of a non-linear mass-spring-damper system. The trajectories predicted by means of the Lagrangian Neural Networks were compared to the data generated by means of computer simulations of Newton’s second law. The main conclusion of this work is that it is feasible to predict the trajectory of this system by means of Lagrangian Neural Networks. Furthermore, the flexibility of the LNN allowed the modeling of non-linear dynamics in the system.

Liquid tank systems play important role in industrial application such as in food processing, beverage, dairy, filtration, effluent treatment, pharmaceutical industry, water purification system, industrial chemical processing and spray coating. Recently, classical PID controller based on the linear model is widely used in controlling industrial liquid level control application. However aggressive performance requirements may not be achievable with this controller due to the presence of nonlinear dynamics inherent in the liquid control system and system parameter variations caused by for example corrosive build-up in liquid level systems creates variation of cross section areas of the tank and discharge orifice. In order to allow advanced controller which can deal with system nonlinearities, a nonlinear model of the tank system is required. This paper describes a comparison study of mathematical-based and artificial neural network-based model of a coupled industrial tank system including its nonlinearity. First, a non-linear mathematical model is developed and its parameters are identified using extended Kalman filter (EKF) based on the experimental data. Next, a multi layer feedforward neural network trained by using backpropagation learning algorithm is used to develop the system model based on the experimental data. A series of validation test is carried out to evaluate the effectiveness of the both models. The results confirm that ANN-based model is more suitable model of the lab-scale industrial tank than EKF-based system.

Modeling dynamical systems is crucial for a wide range of tasks, but it remains challenging due to complex nonlinear dynamics, limited observations, or lack of prior knowledge. Recently, data-driven approaches such as Neural Ordinary Differential Equations (NODE) have shown promising results by leveraging the expressive power of neural networks to model unknown dynamics. However, these approaches often suffer from limited labeled training data, leading to poor generalization and suboptimal predictions. On the other hand, semi-supervised algorithms can utilize abundant unlabeled data and have demonstrated good performance in classification and regression tasks. We propose TS-NODE, the first semi-supervised approach to modeling dynamical systems with NODE. TS-NODE explores cheaply generated synthetic pseudo rollouts to broaden exploration in the state space and to tackle the challenges brought by lack of ground-truth system data under a teacher-student model. TS-NODE employs an unified optimization framework that corrects the teacher model based on the student's feedback while mitigating the potential false system dynamics present in pseudo rollouts. TS-NODE demonstrates significant performance improvements over a baseline Neural ODE model on multiple dynamical system modeling tasks.

Supervised learning is mainly used to optimize Adaptive Neural Fuzzy Inference System (ANFIS) controllers. In order to generate data for supervised learning, a controller is designed and optimized using Particle Swarm Optimization (PSO) or any other algorithms. This paper proposes and compares reinforcement learning based ANFIS and Approximate Reasoning Intelligent controller (ARIC) controllers. Reinforcement learning based ANFIS reduces the work flow required to train it by directly optimizing the membership functions using Proximal Policy Optimization (PPO) algorithm. ANFIS and ARIC neuro fuzzy controllers are designed for nonlinear dynamics of coffee roasting process using Schwartzberg’s model. A custom layer is designed for every membership function and fuzzy inference operations using MATLAB’s Deep Learning Toolbox. This neural connectionist model of ANFIS and ARIC is used as actor. The critic which evaluates the goodness of action taken is a two-layer neural network with sigmoidal activation function. Simulink environment is also created to represent the dynamics of coffee roasting process. The agent is trained to track roast profile for 50 episodes. The training converged at 50th iteration. After training, the Root Mean Square Error (RMSE) for ARIC architecture reduced from 0.5134 to 0.08122. Similarly, the RMSE of ANFIS improved from 0.2026 to 0.0624.

This paper focuses on the design of passive controller with adaptive neural compensation for uncertain strict-feedback nonlinear systems with input-delay. For local linearization model, the delay-dependent γ-passive control is presented. Then, γ-passive control law of local linear model is decomposed as the virtual control of sub-systems by backstepping. In order to compensate the nonlinear dynamics, the adaptive neural model is proposed. The NN weights are turned on-line by Lyapunov stability theory with no prior training. The design procedure of whole systems is a combination of local γ-passive control and adaptive neural network compensation techniques.