Pedestrian trajectory prediction with convolutional neural networks

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Recent experience in applying recurrent neural networks (RNNs) to interpreting Permannt Downhole Gauge (PDG) records has highlighted the potential utility of machine learning algorithms to learn reservoir behavior from data. The power of the RNN resides in its ability to retain information in a form of memory of previous patterns and information contained in the previous behavior of phenomena being modeled. This memory plays a role of informing the decision at the present time by using what happened in the past. This property suggests RNNs as a suitable choice to model sequences of reservoir information, even when the reservoir modeler is faced with incomplete knowledge of the underlying physical system. Convolutional Neural Networks (CNNs) are another variant of machine learning algorithms that have shown promise in sequence modeling domains, such as audio synthesis and machine translation. In this study, RNN and CNN were applied to tasks that traditionally would be modeled by a reservoir simulator. This was achieved by formulating the relationship between physical quantities of interest from subsurface reservoirs as a sequence mapping problem. In addition, the performance of a CNN layer as compared to RNN was evaluated systematically to investigate their capabilities in a variety of tasks of interest to the reservoir engineer. Preliminary results suggest that CNNs, with specific design modifications, are as capable as RNNs in modeling sequences of information and as reliable when making inferences to cases that has not been seen by the algorithm during training. Design details and reasons pertaining to the way these two seemingly different architectures process information, and handle memory are also discussed.

Pedestrian trajectory prediction is an important research area with significant applications in autonomous driving and intelligent surveillance. However, existing studies on pedestrian trajectory prediction often suffer from a noticeable discrepancy between predicted and actual trajectories, due to incomplete extraction of pedestrian trajectory features and the randomness of the pedestrian walking process. The key objective of this article is to address this issue by proposing a method that can reasonably simulate the randomness of pedestrian walking and comprehensively extract pedestrian trajectory features. To achieve this, a novel social informer model built upon the informer model is proposed in this article. The social informer utilizes a transformer encoder-based interaction module to comprehensively extract pedestrian trajectory features, which are input into the informer model for further processing. Additionally, an adaptive variance mechanism is proposed to determine the optimal variance and accurately simulate the random nature of pedestrian walking. Finally, the proposed model is evaluated in a comparative experiment on ETH and UCY datasets, with results demonstrating that the proposed model outperforms other models, exhibiting improved accuracy and performance.

A hybrid model based on neural networks for biomedical relation extraction

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

Pedestrian trajectory prediction can provide decision support for self-driving cars and robots to avoid potential collision risks. Modelling of social relationships and multimodal prediction are core challenges in pedestrian trajectory prediction. However, there are two problems with existing methods: (1) these methods usually focus on modelling pairwise or group interactions, ignoring the importance of high-order interactions; and (2) existing methods struggle to balance the performance between deterministic and multimodal predictions. To address these issues, we propose a higher-order graph attention network for pedestrian trajectory prediction. Specifically, we first construct spatial and temporal graphs to model social interactions and temporal dependencies. To achieve high-order interaction modelling, we design a higher-order graph attention module to capture pedestrian social and temporal interactions of different orders by stacking multiple layers of graph attention. Subsequently, we design a deformable spatial-temporal sparse module to adaptively eliminate social and temporal redundant interactions. To balance deterministic and multimodal prediction performance, we design a two-stage prediction strategy. The first stage predicts pedestrian deterministic trajectories, and the second stage refines pedestrian deterministic trajectories to complete both deterministic and multimodal predictions. Quantitative analysis on the ETH, UCY, and SDD datasets demonstrates that our method outperforms existing methods in both deterministic and multimodal predictions. Qualitative analysis shows that our method can effectively describe complex social relationships and accomplish reliable trajectory prediction.

The study intended to explore the effect of different neural network algorithms in the electrocardiogram (ECG) classification of patients with congenital heart disease (CHD). Based on the single convolutional neural network (CNN) ECG algorithm and the recurrent neural network (RNN) ECG algorithm, a multimodal neural network (MNN) ECG algorithm was constructed utilizing the MIT-BIH database as training set and test set. Furthermore, the MNN ECG algorithm was optimized to establish an improved MNN (IMNN) algorithm, which was applied to the diagnosis of CHD patients. The CHD patients admitted between August 2016 and August 2019 were selected for analysis to compare the classification effect and accuracy rate of IMNN, MNN, CNN ECG, and RNN ECG algorithms. It was found that the RNN ECG algorithm had higher classification sensitivity and true positive rate in terms of normal or bundle (NB) branch block beat, supraventricular abnormal (SA) rhythm, abnormal ventricular (AV) beat, and fusion beat (FB) than the CNN ECG algorithm ( P < 0.05 ), and the classification sensitivity and true positive rate of IMNN algorithm in the four aspects were significantly higher than those of MNN algorithm ( P < 0.05 ). The classification accuracy of CNN ECG algorithm and RNN ECG algorithm was above 98%, while that of MNN algorithm and IMNN algorithm was better than that of CNN ECG algorithm and RNN ECG algorithm, and the accuracy rate can reach 98.5% or more. Moreover, the accuracy rate of the IMNN algorithm can reach more than 98%. In conclusion, IMNN not only has a good classification ability in the simulated environment but also performs well in the actual environment, which is worthy of clinical promotion.

Deep learning methodologies have had a major impact on performance across a wide variety of machine learning tasks, and speech recognition is no exception. We describe a set of deep learning techniques that proved to be particularly successful in achieving performance gains in word error rate on a popular large vocabulary conversational speech recognition benchmark task (“Switchboard”). We found that the best performance is achieved by combining features from both recurrent and convolutional neural networks. We compare two recurrent architectures: partially unfolded nets with max-out activations and bidirectional long short-term memory nets. In addition, inspired by the success of convolutional networks for image classification, we designed a convolutional net with many convolutional layers and small kernels that create a receptive field with more nonlinearity and fewer parameters than standard configurations. When combined, these neural networks achieve a word error rate of 6.2% on this difficult task; this was the best reported rate at the time of this writing and is even more remarkable given that human performance itself is estimated to be 4% on this data.

In recent years, convolutional neural networks have been widely used in image processing and have shown good results. Particularly useful was their ability to automatically extract image features (textures and shapes of objects). The article proposes a method that improves the accuracy and speed of recognition of an ultra-precise neural network based on image recognition of people’s faces. At first, a recurrent neural network is introduced into the convolutional neural network, thereby studying the characteristics of the image more deeply. Deep image characteristics are studied in parallel using a convolutional and recurrent neural network. In line with the idea of skipping the ResNet convolution layer, a new ShortCut3- ResNet residual module is built. A double optimization model is created to fully optimize the convolution process. A study of the influence of various parameters of a convolutional neural network on network performance is demonstrated, also analyzed using simulation experiments. As a result, the optimal parameters of the convolutional neural network are established. Ex- periments show that the method presented in this paper can study various images of people’s faces regardless of age, gender, and also improves the accuracy of feature extraction and image recognition ability.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

The Sixth International Symposium on Neural Networks (ISNN 2009)

проведено исследованию методов оценки состояния канала связи (CSI) в ортогональном частотно-разделённом мультиплексировании (OFDM), применяемом в современных стандартах беспроводной связи, таких как LTE, Wi-Fi и 5G. Рассматриваются традиционные методы оценки канала, такие как Least Squares (LS) и Minimum Mean Square Error (MMSE), отмечаются их недостатки в сложных условиях, связанных с многолучевым распространением сигнала и высокой мобильностью абонентов. Рассмотрено перспективное направление оценки канала связи — глубокое обучение (DL). Доказано, что применение нейронных сетей позволяет существенно повысить точность оценки канала. Приводятся примеры исследований, использующих свёрточные нейронные сети (CNN) и другие архитектуры нейронных сетей для улучшения качества оценки CSI. Отмечено, что ряд аспектов применения нейронных сетей требует дальнейшего изучения, включая оптимизацию моделей под изменяющиеся условия среды и снижение вычислительной сложности. Предлагаются новые подходы для повышения эффективности оценки канала, среди которых использование рекуррентных нейронных сетей (RNN) и методов сжатия моделей (квантование и прунинг). Результаты экспериментов продемонстрировали потенциал использования RNN для эффективной оценки канала связи и восстановления сигналов в OFDM-системах. Моделирование подтвердило преимущество рекуррентных нейронных сетей над CNN в плане потребления вычислительных ресурсов, при сохранении сопоставимой производительности. Описана важность внедрения глубоких нейронных сетей для совершенствования технологии OFDM и предложены направления дальнейших исследований и разработок we conducted a study of methods for assessing the state of a communication channel (CSI) in orthogonal frequency-divided multiplexing (OFDM), used in modern wireless communication standards such as LTE, Wi-Fi and 5G. We considered traditional channel estimation methods such as Least Squares (LS) and Minimum Mean Square Error (MMSE), and their disadvantages are noted in difficult conditions associated with multipath signal propagation and high subscriber mobility. We considered a promising area of communication channel assessment, deep learning (DL). We proved that the use of neural networks can significantly improve the accuracy of channel estimation. We give examples of studies using convolutional neural networks (CNNs) and other neural network architectures to improve the quality of CSI assessment. We note that a number of aspects of the use of neural networks require further study, including optimizing models for changing environmental conditions and reducing computational complexity. We proposed new approaches to improve the efficiency of channel estimation, including the use of recurrent neural networks (RNNs) and model compression methods (quantization and pruning). The simulation confirmed the advantage of recurrent neural networks over CNNs in terms of computing resource consumption, while maintaining comparable performance. The experimental results demonstrated the potential of using RNN for effective assessment of communication quality and signal recovery in OFDM systems. We proposed the importance of the introduction of deep neural networks for the improvement of OFDM technology is described and the directions of further research and development

In autonomous driving scenarios, pedestrian trajectory prediction is an important research direction. Based on the spatio-temporal graph convolutional neural network, we propose a new pedestrian trajectory prediction algorithm. The new algorithm constructs a series of new models around pedestrian intention estimation. The construction of the estimation algorithm considers the following aspects: the contextual information of pedestrians and the surrounding environment, the “pedestrian ego-vehicle” interaction combined with the vehicle speed estimation, the pedestrian’s own skeletal structure information and body language estimation, which includes head joints and the relative structural relationship of the torso joints, including whether it is out of the same plane, is rotated, and so on. Skeleton information feature extraction and construction adopts the method of graph convolutional neural network to structure pedestrians into joints in the form of graphs in non-Euclidean space, and further adopts spatial temporal graph convolutional network for feature extraction and learning. The new method is named a “head-torso”-based spatial temporal graph convolutional network (HT-STGCN). On the dataset PID, the novel method achieves substantial improvements over mainstream methods. Experimental results show that combining HT-STGCN with observed action can improve trajectory prediction.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep neural networks (DNNs) have shown a great achievement in acoustic modeling for speech recognition task. Of these networks, convolutional neural network (CNN) is an effective network for representing the local properties of the speech formants. However, CNN is not suitable for modeling the long-term context dependencies between speech signal frames. Recently, the recurrent neural networks (RNNs) have shown great abilities for modeling long-term context dependencies. However, the performance of RNNs is not good for low-resource speech recognition tasks, and is even worse than the conventional feed-forward neural networks. Moreover, these networks often overfit severely on the training corpus in the low-resource speech recognition tasks. This paper presents the results of our contributions to combine CNN and conventional RNN with gate, highway, and residual networks to reduce the above problems. The optimal neural network structures and training strategies for the proposed neural network models are explored. Experiments were conducted on the Amharic and Chaha datasets, as well as on the limited language packages (10-h) of the benchmark datasets released under the Intelligence Advanced Research Projects Activity (IARPA) Babel Program. The proposed neural network models achieve 0.1–42.79% relative performance improvements over their corresponding feed-forward DNN, CNN, bidirectional RNN (BRNN), or bidirectional gated recurrent unit (BGRU) baselines across six language collections. These approaches are promising candidates for developing better performance acoustic models for low-resource speech recognition tasks.