Robust learning from sparse and noisy data for model discovery of nonlinear systems

The success of deep learning models in fluid dynamics applications will depend on their ability to handle sparse and noisy data accurately. This paper concerns the development of a deep learning model for reconstructing turbulent flow images from low-resolution counterparts encompassing noise. The flow is incompressible through a symmetric, sudden expansion featuring bifurcation, instabilities, and turbulence. The deep learning model is based on convolutional neural networks, in a high-performance, lightweight architecture. The training is performed by finding correlations between high- and low-resolution two-dimensional images. The study also investigates how to remove noise from flow images after training the model with high-resolution and noisy images. In such flow images, the turbulent velocity field is represented by significant color variations. The model's peak signal-to-noise ratio is 45, one of the largest achieved for such problems. Fine-grained resolution can be achieved using sparse data at a fraction of the time required by large-eddy and direct numerical simulation methods. Considering its accuracy and lightweight architecture, the proposed model provides an alternative when repetitive experiments are complex and only a small amount of noisy data is available.

In supervised machine learning, specifically in classification tasks, selecting and analyzing the feature vector to achieve better results is one of the most important tasks. Traditional methods such as comparing the features’ cosine similarity and exploring the datasets manually to check which feature vector is suitable is relatively time consuming. Many classification tasks failed to achieve better classification results because of poor feature vector selection and sparseness of data. In this paper, we proposed a novel framework, topic2features (T2F), to deal with short and sparse data using the topic distributions of hidden topics gathered from dataset and converting into feature vectors to build supervised classifier. For this we leveraged the unsupervised topic modelling LDA (latent dirichlet allocation) approach to retrieve the topic distributions employed in supervised learning algorithms. We made use of labelled data and topic distributions of hidden topics that were generated from that data. We explored how the representation based on topics affect the classification performance by applying supervised classification algorithms. Additionally, we did careful evaluation on two types of datasets and compared them with baseline approaches without topic distributions and other comparable methods. The results show that our framework performs significantly better in terms of classification performance compared to the baseline(without T2F) approaches and also yields improvement in terms of F1 score compared to other compared approaches.

We propose a robust control system which has robustness for disturbance and can deal with a nonlinear system with partially unknown structure by fusing reinforcement learning and robust control theory. First, we solved an optimal control problem without using unknown part of functions of the system, using neural network and the repetition learning of reinforcement learning algorithm. Second, we built the robust reinforcement learning control system which permits uncertainty and has robustness for disturbance by fusing the idea of H infinity control theory with above system.

Parameter estimation for nonlinear dynamic system models, represented by ordinary differential equations (ODEs), using noisy and sparse data, is a vital task in many fields. We propose a fast and accurate method, manifold-constrained Gaussian process inference (MAGI), for this task. MAGI uses a Gaussian process model over time series data, explicitly conditioned on the manifold constraint that derivatives of the Gaussian process must satisfy the ODE system. By doing so, we completely bypass the need for numerical integration and achieve substantial savings in computational time. MAGI is also suitable for inference with unobserved system components, which often occur in real experiments. MAGI is distinct from existing approaches as we provide a principled statistical construction under a Bayesian framework, which incorporates the ODE system through the manifold constraint. We demonstrate the accuracy and speed of MAGI using realistic examples based on physical experiments.

Enhancing multiview synergy: Robust learning by exploiting the wave loss function with consensus and complementarity principles.

In this paper is proposed a robust higher-order iterative learning control (ILC) algorithm for discrete-time systems. In contrast to conventional discrete-time learning methods, the proposed learning algorithm is constructed based on both time-domain performance and iteration-domain performance. Also, the proposed learning algorithm use more than one past error in the iteration-domain. It is proved that the proposed method has robustness in the presence of external disturbances and, in absence of all disturbances, the convergence of the proposed learning algorithm is guaranteed. A numerical example is given to show the robustness in the presence of state disturbance and convergence property according to parameters change.

Fuzzy partition based soft subspace clustering and its applications in high dimensional data

1-D coupled surface flow and transport equations revisited via the physics-informed neural network approach

Predicting biomolecular interactions via a dual-stream graph neural network with motif constraint and diffusion-based regularization.

Gait recognition is a newly developing bio-metric which has potential to recognize people at a distance when application of other biometrics might not be feasible. We propose a new technique to represent and learn various gait component relationships using a recently developing statistical relational learning technique called Markov Logic Networks. Markov Logic Network is a robust statistical learning technique that fuses expressive first-order logic with probabilistic graphical models and prove to be efficient in handling noisy and uncertain data. Initially we derive component based pattern classifiers in the imaging domain using an automatic segmentation scheme and represent gait components and their relationships using firstorder logic. Then we model and learn their characteristics using undirected graphs to finally classify gaits based on standard inference techniques. The proposed approach enables automatic gait recognition from low resolution videos and differs from conventional techniques which rely on manual markings on videos. We show that the proposed representation provide intuitive means to reason gait component relationships. Our results show that the proposed approach competes well with other state-of-the-art techniques.

Social scientists are frequently interested in how populations evolve over time. Creating poststratification weights for surveys, for example, requires information on the weighting variables’ joint distribution in the target population. Typically, however, population data are sparsely available across time periods. Even when population data are observed, the content and structure of the data—which variables are observed and whether their marginal or joint distributions are known—differ across time, in ways that preclude straightforward interpolation. As a consequence, survey weights are often based only on the small subset of auxiliary variables whose joint population distribution is observed regularly over time, and thus fail to take full advantage of auxiliary information. To address this problem, we develop a dynamic Bayesian ecological inference model for estimating multivariate categorical distributions from sparse, irregular, and noisy data on their marginal (or partially joint) distributions. Our approach combines (1) a Dirichlet sampling model for the observed margins conditional on the unobserved cell proportions; (2) a set of equations encoding the logical relationships among different population quantities; and (3) a Dirichlet transition model for the period-specific proportions that pools information across time periods. We illustrate this method by estimating annual U.S. phone-ownership rates by race and region based on population data irregularly available between 1930 and 1960. This approach may be useful in a wide variety of contexts where scholars wish to make dynamic ecological inferences about interior cells from marginal data. A new R package estsubpop implements the method.

A fast and robust algorithm for training feedforward neural networks (FNNs) by using a variable forgetting factor and U-D factorization-based recursive prediction error (RPE) method is proposed. In comparison with the backpropagation (BP) and RPE based learning algorithms, the proposed algorithm, called UD-RPE, can provide much more accurate learning results in fewer iterations with fewer hidden nodes and improve convergence rate and numerical stability (robustness). In addition, it is less sensitive to start-up parameters, such as initial weights and initial covariance matrix, and the randomness in the observed data. It also has good generalization ability and needs less learning time. Simulation results of nonlinear dynamic system modeling and identification show that the algorithm proposed here is an effective and efficient learning algorithm for FNNs.

<strong class="journal-contentHeaderColor">Abstract.</strong> Accurate river streamflow forecasts are a vital tool in the fields of water security, flood preparation and agriculture, as well as in industry more generally. Over the last century, physics-based models traditionally used to produce streamflow forecasts have become increasingly sophisticated, with forecasts improving accordingly. However, the development of such models is often bound by two soft limits: empiricism &ndash; many physical relationships are represented by computationally efficient empirical formulae; and data sparsity &ndash; the calibration of these models often requires long time-series of high-resolution observational data at both surface and subsurface levels. Artificial neural networks have previously been shown to be highly effective at simulating nonlinear systems where knowledge of the underlying physical relationships is incomplete. However, they also suffer from issues related to data sparsity. Recently, hybrid forecasting systems, which combine the traditional physics-based approach with statistical forecasting techniques, have been investigated for use in hydrological applications. In this study, we test the efficacy of a type of neural network, the long-short term memory (LSTM), at predicting streamflow at ten river gauge stations across various climatic regions of the western United States. The LSTM is trained on the catchment-mean meteorological and hydrological variables from the ERA5 and GloFAS-ERA5 reanalysis as well as historical streamflow observations. The performance of these hybrid forecasts is evaluated and compared to the performance of both raw and bias-corrected output from the Copernicus Emergency Management Service (CEMS) physics-based Global Flood Awareness System (GloFAS). Two periods are considered, a testing phase (June 2019 to June 2020), during which the models were fed with ERA5 data to investigate how well they simulated streamflow at the ten stations; and an operational phase (September 2020 to October 2021), during which the models were fed forecast variables from ECMWF's Integrated Forecast System (IFS), to investigate how well they could predict streamflow at lead times of up to ten days. All three models performed well in the testing phase, with the LSTM performing the best (skilful at nine stations, of which six were highly skilful). Similarly, the LSTM forecasts beat the raw and bias-corrected GloFAS forecasts during the operational phase, with skilful 5-day forecasts at nine stations, of which five were highly skilful. Implications and potential improvements to this work are discussed. In summary, this is the first time an LSTM has been used in a hybrid system to create a medium-range streamflow forecast, and in beating established physics-based models, shows promise for the future of neural networks in hydrological forecasting.

Accurate river streamflow forecasts are a vital tool in the fields of water security, flood preparation and agriculture, as well as in industry more generally. Over the last century, physics-based models traditionally used to produce streamflow forecasts have become increasingly sophisticated, with forecasts improving accordingly. However, the development of such models is often bound by two soft limits: empiricism – many physical relationships are represented by computationally efficient empirical formulae; and data sparsity – the calibration of these models often requires long time-series of high-resolution observational data at both surface and subsurface levels. Artificial neural networks have previously been shown to be highly effective at simulating nonlinear systems where knowledge of the underlying physical relationships is incomplete. However, they also suffer from issues related to data sparsity. Recently, hybrid forecasting systems, which combine the traditional physics-based approach with statistical forecasting techniques, have been investigated for use in hydrological applications. In this study, we test the efficacy of a type of neural network, the long-short term memory (LSTM), at predicting streamflow at ten river gauge stations across various climatic regions of the western United States. The LSTM is trained on the catchment-mean meteorological and hydrological variables from the ERA5 and GloFAS-ERA5 reanalysis as well as historical streamflow observations. The performance of these hybrid forecasts is evaluated and compared to the performance of both raw and bias-corrected output from the Copernicus Emergency Management Service (CEMS) physics-based Global Flood Awareness System (GloFAS). Two periods are considered, a testing phase (June 2019 to June 2020), during which the models were fed with ERA5 data to investigate how well they simulated streamflow at the ten stations; and an operational phase (September 2020 to October 2021), during which the models were fed forecast variables from ECMWF's Integrated Forecast System (IFS), to investigate how well they could predict streamflow at lead times of up to ten days. All three models performed well in the testing phase, with the LSTM performing the best (skilful at nine stations, of which six were highly skilful). Similarly, the LSTM forecasts beat the raw and bias-corrected GloFAS forecasts during the operational phase, with skilful 5-day forecasts at nine stations, of which five were highly skilful. Implications and potential improvements to this work are discussed. In summary, this is the first time an LSTM has been used in a hybrid system to create a medium-range streamflow forecast, and in beating established physics-based models, shows promise for the future of neural networks in hydrological forecasting.

Accurate river streamflow forecasts are a vital tool in the fields of water security, flood preparation and agriculture, as well as in industry more generally. Over the last century, physics-based models traditionally used to produce streamflow forecasts have become increasingly sophisticated, with forecasts improving accordingly. However, the development of such models is often bound by two soft limits: empiricism – many physical relationships are represented by computationally efficient empirical formulae; and data sparsity – the calibration of these models often requires long time-series of high-resolution observational data at both surface and subsurface levels. Artificial neural networks have previously been shown to be highly effective at simulating nonlinear systems where knowledge of the underlying physical relationships is incomplete. However, they also suffer from issues related to data sparsity. Recently, hybrid forecasting systems, which combine the traditional physics-based approach with statistical forecasting techniques, have been investigated for use in hydrological applications. In this study, we test the efficacy of a type of neural network, the long-short term memory (LSTM), at predicting streamflow at ten river gauge stations across various climatic regions of the western United States. The LSTM is trained on the catchment-mean meteorological and hydrological variables from the ERA5 and GloFAS-ERA5 reanalysis as well as historical streamflow observations. The performance of these hybrid forecasts is evaluated and compared to the performance of both raw and bias-corrected output from the Copernicus Emergency Management Service (CEMS) physics-based Global Flood Awareness System (GloFAS). Two periods are considered, a testing phase (June 2019 to June 2020), during which the models were fed with ERA5 data to investigate how well they simulated streamflow at the ten stations; and an operational phase (September 2020 to October 2021), during which the models were fed forecast variables from ECMWF's Integrated Forecast System (IFS), to investigate how well they could predict streamflow at lead times of up to ten days. All three models performed well in the testing phase, with the LSTM performing the best (skilful at nine stations, of which six were highly skilful). Similarly, the LSTM forecasts beat the raw and bias-corrected GloFAS forecasts during the operational phase, with skilful 5-day forecasts at nine stations, of which five were highly skilful. Implications and potential improvements to this work are discussed. In summary, this is the first time an LSTM has been used in a hybrid system to create a medium-range streamflow forecast, and in beating established physics-based models, shows promise for the future of neural networks in hydrological forecasting.