A learning method for vector field approximation by neural networks

This paper addresses the problem of global optimization by means of a monotonic transformation. With an observation on global optimality of functions under such a transformation, we show that a simple and effective algorithm can be derived to search within possible regions containing the global optima. Numerical experiments are performed to compare this algorithm with one that does not incorporate transformed information using several benchmark problems. These results are also compared to best known global search algorithms in the literature. In addition, the algorithm is shown to be useful for several neural network learning problems, which possess much larger parameter spaces.

We propose a novel global minimization method, called NOVEL (Nonlinear Optimization via External Lead), and demonstrate its superior performance on neural network learning problems. The goal is improved learning of application problems that achieves either smaller networks or less error prone networks of the same size. This training method combines global and local searches to find a good local minimum. In benchmark comparisons against the best global optimization algorithms, it demonstrates superior performance improvement.

Robust and fast stochastic 4D Flow Vector-field signature technique for quantifying composite flow dynamics from 4D Flow MRI: Application to left atrial flow in atrial fibrillation

In this work a theorical framework to apply the Poincare compactification technique to locally Lipschitz continuous vector fields is developed. It is proved that these vectors fields are compactifiable in the n-dimensional sphere, though the compactified vector field can be identically null in the equator. Moreover, for a fixed projection to the hemisphere, all the compactifications of a vector field, which are not identically null on the equator are equivalent. Also, the conditions determining the invariance of the equator for the compactified vector field are obtained. Up to the knowledge of the authors, this is the first time that the Poincare compactification of locally Lipschitz continuous vector fields is studied. These results are illustrated applying them to some families of vector fields, like polynomial vector fields, vector fields defined as a sum of homogeneous functions and vector fields defined by piecewise linear functions.

Robust feature extraction is an integral part of scientific visualization. In unsteady vector field analysis, researchers recently directed their attention towards the computation of near‐steady reference frames for vortex extraction, which is a numerically challenging endeavor. In this paper, we utilize a convolutional neural network to combine two steps of the visualization pipeline in an end‐to‐end manner: the filtering and the feature extraction. We use neural networks for the extraction of a steady reference frame for a given unsteady 2D vector field. By conditioning the neural network to noisy inputs and resampling artifacts, we obtain numerically stabler results than existing optimization‐based approaches. Supervised deep learning typically requires a large amount of training data. Thus, our second contribution is the creation of a vector field benchmark data set, which is generally useful for any local deep learning‐based feature extraction. Based on Vatistas velocity profile, we formulate a parametric vector field mixture model that we parameterize based on numerically‐computed example vector fields in near‐steady reference frames. Given the parametric model, we can efficiently synthesize thousands of vector fields that serve as input to our deep learning architecture. The proposed network is evaluated on an unseen numerical fluid flow simulation.

Real-time motion planning and control of multi-joint robot manipulators are studied using neural networks. The proposed neural network approach consists of two modules: one for real-time collision-free motion planning, the other for real-time fine control of the robot manipulators. The motion planning module is a biologically inspired, parallel connected, topologically organised neural network, where each neuron is characterised by a shunting equation, whose state space is the robot configuration joint space. This module is capable of planning a real-time optimal path for robot manipulators through the dynamic activity landscape of the neural network. The motion control module involves of a feedforward neural network together with a traditional PD feedback loop. It is capable of achieving real-time fine motion control of robot manipulators under significant uncertainties and without any prior knowledge of the robot dynamics. This module can quickly compensate sudden changes in the robot dynamics. The real-time fine control of robot manipulators is achieved through the online learning of the neural network. Both the motion planning and motion control modules are computationally efficient, and their global stabilities are proved using Lyapunov stability theory.

This thesis deals with recurrent neural networks, a particular class of artificial neural networks which can learn a generative model of input sequences. The input is mapped, through a feedback loop and a non-linear activation function, into a hidden state, which is then projected into the output space, obtaining either a probability distribution or the new input for the next time-step. This work consists mainly of two parts: a theoretical study for helping the understanding of recurrent neural networks framework, which is not yet deeply investigated, and their application to non-linear prediction problems, since recurrent neural networks are really powerful models suitable for solving several practical tasks in different fields. For what concerns the theoretical part, we analyse the weaknesses of state-of-the-art models and tackle them in order to improve the performance of a recurrent neural network. Firstly, we contribute in the understanding of the dynamical properties of a recurrent neural network, highlighting the close relation between the definition of stable limit cycles and the echo state property of an echo state network. We provide sufficient conditions for the convergence of the hidden state to a trajectory, which is uniquely determined by the input signal, independently of the initial states. This may help extend the memory of the network and increase the design options for the network. Moreover, we develop a novel approach to address the main problem in training recurrent neural networks, the so-called vanishing gradient problem. Our new method allows us to train a very simple recurrent neural network, making the gradient not to vanish even after many time-steps. Exploiting the singular value decomposition of the vanishing factors in the gradient and random matrices theory, we find that the singular values have to be confined in a narrow interval and derive conditions about their root mean square value. Then, we also improve the efficiency of the training of a recurrent neural network, defining a new method for speeding up this process. Thanks to a least square regularization, we can initialize the parameters of the network, in order to set them closer to the minimum and running fewer epochs of classical training algorithms. Moreover, it is also possible to completely train the network with our initialization method, running more iterations of it without losing in performance with respect to classical training algorithms. Finally, it is also possible to use it as a real-time learning algorithm, adjusting the parameters to the new data through one iteration of our initialization. In the last part of this thesis, we apply recurrent neural networks to non-linear prediction problems. We consider prediction of numerical sequences, estimating the following input choosing it from a probability distribution. We study an automatic text generation problem, where we need to predict the following character in order to compose words and sentences, and a path prediction of walking mobile users in the central area of a city, as a sequence of crossroads. Then, we analyse the prediction of video frames, discovering a wide range of applications related to the prediction of movements. We study the collision problem of bouncing balls, taking into account only the sequence of video frames without any knowledge about the physical characteristics of the problem, and the distribution over days of mobile user in a city and in a whole region. Finally, we address the state-of-the-art problem of missing data imputation, analysing the incomplete spectrogram of audio signals. We restore audio signals with missing time-frequency data, demonstrating via numerical experiments that a performance improvement can be achieved involving recurrent neural networks.

With the development of economic science and technology, the development of computer vision has undergone rapid changes, and various products relying on computer vision are also more and more, such as smart home, robot technology, and so on. At present, robot technology has become a very important part of the development of human science and technology, and in the field of industrial robots, the most rapid development is the robot with robot arm adaptive motion. It is very necessary to study the adaptive motion control of the manipulator based on machine learning. The robot with the adaptive motion of the manipulator can carry out logistics express sorting, operate in the doors and windows outside the building, and pick fruits in the orchard, which can ensure the effective implementation of hard work. Therefore, this paper proposes a mechanical adaptive control method based on a neural network. According to the motion model of the manipulator, the RBF neural network model is used to judge the stability of the system according to the Lyapunov function. The related algorithms of machine learning and multi-degree of freedom manipulator are studied and improved. The RBF neural network model approximates the unknown function infinitely and then establishes the complex motion model. Aiming at the adaptive neural network of a manipulator, a network adaptive terminal control method is proposed. Firstly, a stable manipulator motion system is designed by using a neural network, and then the terminal synovial controller is designed by using backstepping control technology. The stability of the method is proved by using the approximation virtual control technology of the neural network. The adaptive control is realized by using the learning and self-adaptability of the neural network; thus, the stability analysis of the closed-loop system is realized.

The Sixth International Symposium on Neural Networks (ISNN 2009)

Neural implicit fields have recently shown increasing success in representing, learning and analysis of 3D shapes. Signed distance fields and occupancy fields are still the preferred choice of implicit representations with well-studied properties, despite their restriction to closed surfaces. With neural networks, unsigned distance fields as well as several other variations and training principles have been proposed with the goal to represent all classes of shapes. In this paper, we develop a novel and yet a fundamental representation considering unit vectors in 3D space and call it Vector Field (VF). At each point in R3, VF is directed to the closest point on the surface. We theoretically demonstrate that VF can be easily transformed to surface density by computing the flux density. Unlike other standard representations, VF directly encodes an important physical property of the surface, its normal. We further show the advantages of VF representation, in learning open, closed, or multi-layered surfaces. We show that, thanks to the continuity property of the neural optimization with VF, a separate distance field becomes unnecessary for extracting surfaces from the implicit field via Marching Cubes. We compare our method on several datasets including ShapeNet where the proposed new neural implicit field shows superior accuracy in representing any type of shape, outperforming other standard methods. Codes are available at https://github.com/edomel/ImplicitVF.

It is a known fact that companies strive to cut their production costs. One of the most effective ways to reduce costs is to automate processes, for example, replacing employees with various robots or information systems. Often, at a construction site or a large-scale production, it is necessary to monitor employees' work. In its simplest form, monitoring can be carried out with the help of video cameras and people who view recordings from these cameras. However, this entails labor costs, taxes and other expenses. So, what one can do instead is introduce a system that tracks movements of workers, which will likely reduce the costs described above. To this day, machine learning and computer vision algorithms have developed quite strongly and allow the implementation of such tracking systems. There are already ready-made versions of tracking systems based on computer vision algorithms, but they are expensive, which makes them suitable only for large-scale industries. Another option is to develop your own tracking system, taking into account the nature of the business in which you plan to implement it. Aim. The authors aim at analyzing technologies that can be used to develop a system for tracking workers’ movements and proposing an implementation of such a system using machine learning algorithms and neural networks. Materials and methods. The authors give an overview of current solutions of monitoring systems based on RFID techno-logies, computer vision, GPS, and motion sensors, highlighting their pros and cons. The authors consider existing methods for motion detection based on computer vision and review known algorithms for detecting workers in special clothing. The YOLOv5 algorithm, consisting of backbone, neck and head, is considered in detail. Results. In the article, the authors provide a description of algorithms used for machine learning, computer vision, and neural networks. They propose an implementation of a system monitoring workers’ movements. They describe the modules that make up the tracking system and a diagram of the interaction of the modules, as well as the development of a motion detection algorithm and an algorithm for determining workers in special clothing.

Neural networks have been developed for more than 60 years since the model was proposed in the 1950s. At present, they have been widely used in various fields and have achieved great success, especially in speech recognition, computer vision, industrial inspection, and national defense. This chapter focuses on artificial neural networks, convolution neural networks, and graph neural networks, deeply analyzing the basic principles of each type of neural network algorithm and their advantages and disadvantages. An artificial neuron, referred to just as neuron for short, is the basic unit of a neural network. It mainly simulates the structure and characteristics of biological neurons, receiving a group of input signals and generating outputs. A convolutional neural networks is a kind of feedforward neural network with depth structure including convolution calculation. It is one of the representative algorithms of deep learning.

Artificial Intelligence in the American Healthcare Industry: Looking Forward to 2030

Computer Vision and Machine Learning are two branches of Artificial Intelligence which in combination have granted computational systems the ability of artificial vision and perception. Nowadays, human-machine interaction and the analysis of human behavior can be achieved using cameras. The subject of the current thesis is the implementation of a system for automatic recognition of static hand gestures in real-time videos with cameras, using modern techniques and algorithms of Machine Learning and Deep Learning. To begin with, we make an introduction into Computer Vision and its applications and we describe the problem of hand gesture recognition. Next, we refer to the field of Machine Learning and the main categories of the problems it includes. We describe the methodology of training a model and ways to measure its performance, focusing on Classification problems. Also, we introduce Deep Learning and Artificial Neural Networks. We present the most common network architectures used nowadays and the procedure of training a Neural Network and methods for optimization. Also, we mention the use of Convolutional Neural Networks for solving Computer Vision problems and we present some of the most known algorithms used for this task. Finally, we present our implementation of the hand gesture recognition tool, using Python programming language. We apply innovative algorithms of Convolutional Neural Networks for the detection of several key points of the hand and we train a Support Vector Machine and a Feedforward Neural Network model for the classification of those points into several predetermined static gestures. The implementation is targeted for practical real-time applications, runs using the CPU of the computer and requires a common RGB camera.

The artificial neural network (ANN), one of the machine learning (ML) algorithms, inspired by the human brain system, was developed by connecting layers with artificial neurons. However, due to the low computing power and insufficient learnable data, ANN has suffered from overfitting and vanishing gradient problems for training deep networks. The advancement of computing power with graphics processing units and the availability of large data acquisition, deep neural network outperforms human or other ML capabilities in computer vision and speech recognition tasks. These potentials are recently applied to healthcare problems, including computer-aided detection/diagnosis, disease prediction, image segmentation, image generation, etc. In this review article, we will explain the history, development, and applications in medical imaging