LUCID: Learning-Enabled Uncertainty-Aware Certification of Stochastic Dynamical Systems

We propose a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of model predictive shielding (MPS), where a backup controller is used to override the learned policy as needed to ensure safety. The key challenge is how to compute a backup policy in the context of stochastic dynamics. We propose to use a tube-based robust nonlinear model predictive controller (NMPC) as the backup controller. We estimate the tubes using sampled trajectories, leveraging ideas from statistical learning theory to obtain high-probability guarantees. We empirically demonstrate that our approach can ensure safety in stochastic systems, including cart-pole and a non-holonomic particle with random obstacles.

A fundamental way to analyze complex multidimensional stochastic dynamics is to describe it as diffusion on a free energy landscape-free energy as a function of reaction coordinates (RCs). For such a description to be quantitatively accurate, the RC should be chosen in an optimal way. The committor function is a primary example of an optimal RC for the description of equilibrium reaction dynamics between two states. Here, additive eigenvectors (addevs) are considered as optimal RCs to address the limitations of the committor. An addev master equation for a Markov chain is derived. A stationary solution of the equation describes a sub-ensemble of trajectories conditioned on having the same optimal RC for the forward and time-reversed dynamics in the sub-ensemble. A collection of such sub-ensembles of trajectories, called stochastic eigenmodes, can be used to describe/approximate the stochastic dynamics. A non-stationary solution describes the evolution of the probability distribution. However, in contrast to the standard master equation, it provides a time-reversible description of stochastic dynamics. It can be integrated forward and backward in time. The developed framework is illustrated on two model systems-unidirectional random walk and diffusion.

This study designed a safe combined certification system by approaching the matter of certification upon recent expansion of the cyber world and the market size of mobile shopping through a newer plan. Recently, mobile field has drawn much attention and it is used for most means of living for users like purchase of commodity, payment, and user certification. The mobile economy also grows by this trend, but hacking accidents or financial crimes are also in full swing. It is expected that the service for mobile in the future will increase more than ever, but in security aspects we still use problematic certification and security system. Data control service in the future is expected to do a lot with the service, which basically stores all services in phones as well as service providers through cloud service. Data control service will lead to the expansion of IoT (Internet of Things) service, as it substitutes existing off-line backup and safely provide user data regardless of location or kinds of devices. That is, users can use data from the devices connected to the Internet without copying or moving to devices users want to use. To use this data safely, establishing a certification system of users will be an ever more important factor. Therefore, this treatise is going to suggest the plan that we can conduct the certification in a safer way among several threats. In particular, mobile hacking is a very serious and important issue, as it could directly lead to monetary loss. When certification is insufficient, the possibility of a third party reaping benefits through hacking is greater. This treatise is going to suggest a model that service providers can provide safe certification service by performing verification of transaction and data through electronic certification. Building of a combined and safe certification system upon recent FinTech boom is a very important factor and this study is going to suggest various structures in aspects of security and convenience. Keyword: Certificate, ISP, Authentication, Mobile, Payment

An empirical evaluation of a formal approach versus ad hoc implementations in robot behavior planning

State-based confidence bounds for data-driven stochastic reachability using Hilbert space embeddings

With the development of swarm intelligence in last decades, consensus problem of multi-agent systems has been attracting much attention. To reveal the inherent mechanism of leader-following consensus in multi-agent systems with stochastic dynamics, PI control protocols are designed in this paper. Owing to the stochastic analysis techniques, algebraic graph theory, and constructing appropriate Lyapunov function, it is proved that the leader-following consensus of nonlinear stochastic dynamical multi-agent systems can be reached in mean square. Sufficient condition are deduced to select the PI control protocol parameters. Finally, the theoretical results are demonstrated through a simulation example.

Earth and Space Science Open Archive PosterOpen AccessYou are viewing the latest version by default [v1]A Variational Approach to Small-Scale Parameterization for Nonlinear and Stochastic Dynamical SystemsAuthorsHonghuLiuiDMickaelChekrounJamesMcWilliamsiDSee all authors Honghu LiuiDCorresponding Author• Submitting AuthorVirginia TechiDhttps://orcid.org/0000-0002-9226-0744view email addressThe email was not providedcopy email addressMickael ChekrounUniversity of California Los Angelesview email addressThe email was not providedcopy email addressJames McWilliamsiDUniversity of California in Los AngelesiDhttps://orcid.org/0000-0002-1237-5008view email addressThe email was not providedcopy email address

Autonomous train operation (defined by the on - board part of the ATO - Automatic Train Operation) is a key innovation in rail transport, bringing increased efficiency and safety. Ensuring ATO safety is a key challenge that can be addressed by integrat ion wi th the European Train Control System (ETCS). This paper focuses on using ETCS safety mechanisms to ensure the safety of autonomous train control. The following paper analyses the possibility of taking over the ETCS safety guarantees into the ATO . It focuse s on the different types of data provided by the ETCS, including static train data, dynamic data (movement authority, speed profile, gradient, localization, ETCS modes and traction) and information about the braking capabilities of the train. The ke y quest ion is whether this information can be relied upon to implement autonomous control without additional redundant mechanisms. According to the RAMS (Reliability, Availability, Maintainability, and Safety) analysis, existing safety features can be adop ted if the risks corresponding to the subsystem are sufficiently reduced and there is a secure link between the ATO and the ETCS systems . The specific physical interconnection via Ethernet plays an important role here. This link is described directly in the Europ ean Railway Agency specifications. Twelve packets of information are transmitted through this link. The aim is to verify whether it is po ssible to take over the safety guarantees of the ETCS for the ATO and to find the interface requirements between these systems. The main requirements of the ATO functions that process the ETCS information are to stop safely at a defined location, to keep to the schedule within the conditions specified by the ETCS, to optimize the speed for efficient traction energy consump tion and to start the station check - in sequence only after all safety conditions have been met. The methodology includes the analysis of the common interface between the ATO and the ETCS and applying RAMS procedures to identify and mitigate risks. The foll owing paper describes that the ETCS provides robust safety barriers during standard train operation at GoA 1 (Grade of Automation), allowing the ATO sys tem to rely on its safety features and focus on the additional risks associated with autonomous control at higher GoA levels. However, the key factor is to ensure a secure connection between the ATO and the ETCS in accordance with the applicable standards. Thi s paper should contribute to a better understanding of the links between the ATO and the ETCS. It al so provides important information for the further development of autonomous railway systems.

Understanding the evolution of pollutants via hierarchical complexity of space-time deterministic and stochastic dynamical systems

This study aims to implement AIoT-based safety module algorithm and platform service technology to build a safety environment in shipyards. To this end, platform service technology is implemented through safety module algorithms and killer content implementation through artificial intelligence learning. In particular, accident prevention and prediction are very important for the continuous ESG management of exporting companies, and safety mode service technology is tested so that managers can quickly evaluate risks through the application of platform service technology and safety modules to prevent serious accidents. The Enforcement Decree of the Act on the Punishment of Serious Disasters, etc., which applies to workplaces with 50 or fewer employees from 2024, suggests the necessity of preparing a minimum safety framework to prevent serious accidents in the establishment and implementation of a safety and health management system. In particular, standardization and algorithm development for risk assessment in accordance with the enactment of notices and guidelines for each field are required. Based on the implementation of these laws, we present decision methods and guidelines for safety diagnosis determination through the implementation of analysis algorithms based on advanced technologies (sensor IoT, network, wearable devices, AI analysis technology, etc.) for major safety accident factors. Until recently, the focus was on the importance of the technical area by focusing on the development, installation, and operation of technology-oriented systems, and if the safety grade was worked on according to the institutional impact on risk diagnosis, the risk factors and harmful factors of the workplace should be identified in advance in accordance with the enforcement of the Serious Disaster Act to estimate and determine the likelihood of injury or disease caused by the factor and the intensity of the disaster, and to establish and implement reduction measures on their own. As Article 36 of the Occupational Safety and Health Act mandates employers to conduct risk assessments on their own, attention to disaster and safety has become commonplace. Accordingly, it is possible to prevent accidents in advance through response solutions based on AIoT technology, and to minimize workers' safety guarantees and accidents through rapid and effective responses. Therefore, by detecting and analyzing risk factors in the field in real time using sensor AIoT technology for small and medium-sized shipyards in poor environments, potential risks can be identified early and serious accidents can be handled quickly. AIoT service technology can warn of the possibility of an accident and the high probability of an accident by establishing a proactive system using big data analysis and prediction models. In addition, virtual space utilization technology based on artificial intelligence can inform field workers of risks in real time and take immediate safety measures through risk factor analysis. Recently, due to the aging society and the MZ generation's avoidance of field work, a large number of foreign workers have flowed in, and the responsibility for accidents at small and medium-sized shipyards due to lack of communication and unskilled workers is increasing. Moreover, from 2024, SMEs in poor environments must respond through safety environment solutions by applying the Major Disaster Act to small and medium-sized workplaces with 50 or fewer employees.

Previously, we showed that computational mechanic’s causal states—predictively equivalent trajectory classes for a stochastic dynamical system—can be cast into a reproducing kernel Hilbert space. The result is a widely applicable method that infers causal structure directly from very different kinds of observations and systems. Here, we expand this method to explicitly introduce the causal diffusion components it produces. These encode the kernel causal state estimates as a set of coordinates in a reduced dimension space. We show how each component extracts predictive features from data and demonstrate their application on four examples: first, a simple pendulum—an exactly solvable system; second, a molecular-dynamic trajectory of n-butane—a high-dimensional system with a well-studied energy landscape; third, the monthly sunspot sequence—the longest-running available time series of direct observations; and fourth, multi-year observations of an active crop field—a set of heterogeneous observations of the same ecosystem taken for over a decade. In this way, we demonstrate that the empirical kernel causal state algorithm robustly discovers predictive structures for systems with widely varying dimensionality and stochasticity.

This Paper deals with the recursive identification problem of stochastic linear dynamical systems , Important Algorithms are explained in the identification system domains that are time-varying, and using a recursive Least Square method with a famous approach to estimate the model parameter, that a forgetting factor, and a Kalman filter approach with different values for a best linear dynamic models, that are identified from the two type of stochastic linear dynamic systems: that the equation error models which contain ARX models and ARMAX models. Output error models which consist of OE and Box-Jenkins models, that are reached by using the suggested instrument in Off-Line Identification, where the exact Linear models reached their parameter stable with Time, Moreover, the Statistic terms are verified from a point of random errors and insignificant cross-correlation between inputs and outputs residual.

In this paper, a data-driven conflict-aware safe reinforcement learning (CAS-RL) algorithm is presented for control of autonomous systems. Existing safe RL results with predefined performance functions and safe sets can only provide safety and performance guarantees for a single environment or circumstance. By contrast, the presented CAS-RL algorithm provides safety and performance guarantees across a variety of circumstances that the system might encounter. This is achieved by utilizing a bilevel learning control architecture: A higher meta-cognitive layer leverages a data-driven receding-horizon attentional controller (RHAC) to adapt relative attention to different system's safety and performance requirements, and, a lower-layer RL controller designs control actuation signals for the system. The presented RHAC makes its meta decisions based on the reaction curve of the lower-layer RL controller using a meta-model or knowledge. More specifically, it leverages a prediction meta-model (PMM) which spans the space of all future meta trajectories using a given finite number of past meta trajectories. RHAC will adapt the system's aspiration towards performance metrics (e.g., performance weights) as well as safety boundaries to resolve conflicts that arise as mission scenarios develop. This will guarantee safety and feasibility (i.e., performance boundness) of the lower-layer RL-based control solution. It is shown that the interplay between the RHAC and the lower-layer RL controller is a bilevel optimization problem for which the leader (RHAC) operates at a lower rate than the follower (RL-based controller) and its solution guarantees feasibility and safety of the control solution. The effectiveness of the proposed framework is verified through a simulation example.

We present the first supermartingale certificate for quantitative $$\omega $$ ω -regular properties of discrete-time infinite-state stochastic systems. Our certificate is defined on the product of the stochastic system and a limit-deterministic Büchi automaton that specifies the property of interest; hence we call it a limit-deterministic Büchi supermartingale (LDBSM). Previously known supermartingale certificates applied only to quantitative reachability, safety, or reach-avoid properties, and to qualitative (i.e., probability 1) $$\omega $$ ω -regular properties.We also present fully automated algorithms for the template-based synthesis of LDBSMs, for the case when the stochastic system dynamics and the controller can be represented in terms of polynomial inequalities. Our experiments demonstrate the ability of our method to solve verification and control tasks for stochastic systems that were beyond the reach of previous supermartingale-based approaches.

A stochastic dynamical system is a system composed of many interacting components which includes stochastic behavior. Feedback control theory is designed to use information from the current state of the system in order to temper the final state toward the desired outcome. In this dissertation, I perform studies on two choices of classic and quantum dynamical systems to cover vast variety of statistical tools and methods. For classical model, I perform a study on finite-size effects at the phase transition in the Nagel-Schreckenberg traffic model as an example of non-equilibrium many body stochastic dynamical system. For quantum mechanical model, I introduce an optimal measurement-based feedback control protocol for cooling a single qubit as an example of complex system with one cell and many degrees of freedom. I examine the Nagel-Schreckenberg traffic model for a variety of maximum speeds. I show that the low density limit can be described as a dilute gas of vehicles with a repulsive core. At the transition to jamming, I observe finite-size effects in a variety of quantities describing the flow and the density correlations, but only if the maximum speed Vmax is larger than a certain value. A finite-size scaling analysis of several order parameters shows universal behavior, with scaling exponents that depend on Vmax The jamming transition at large Vmax can be viewed as the nucleation of jams in a background of freely flowing vehicles. Feedback control of quantum systems via continuous measurements involves complex nonlinear dynamics. As a result, even for a single qubit the optimal measurement for feedback control is known only in very special cases. I show here that for a broad class of noise processes, a series of compelling arguments can be applied to greatly simplify the problem of steady-state preparation of the ground-state, while loosing little in the way of optimality. Using numerical optimization to solve this simplified control problem, I obtain for the first time a non-trivial feedback protocol valid for all feedback strengths in the regime of good control. The protocol can be described relatively simply, and contains a discontinuity as a function of feedback strength.