МЕТОДЫ СНИЖЕНИЯ ВЫЧИСЛИТЕЛЬНОЙ СЛОЖНОСТИ В ЗАДАЧЕ ОЦЕНКИ КАНАЛА СВЯЗИ ДЛЯ СИСТЕМ OFDM С ИСПОЛЬЗОВАНИЕМ НЕЙРОННЫХ СЕТЕЙ

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

In this article, we explore the use of neural networks (NNs) to enhance the performance evaluation of Universal Filtered Multi-Carrier (UFMC) systems, a key technology for modern communication networks such as 5G and beyond. Traditional methods of evaluating the Bit Error Rate (BER) and system efficiency in UFMC system can be computationally intensive and less accurate under dynamic conditions. To address these challenges, we propose a NN based approach that not only improves the accuracy of BER prediction but also significantly optimizes the system's overall efficiency. Channel state estimation (CSE) plays a major role for UFMC system to address the phenomenon of multipath channel fading. In order to achieve a high data rate using UFMC technology, it is necessary to have an effective CSE and very accurate signal detection. Recently, there has been significant interest in utilizing deep learning (DL) to enhance channel estimations. This article introduces a new method for channel estimation (CE) in UFMC system. The suggested approach utilizes DL models to improve the CE. For the UFMC system, we propose a detector based on bidirectional long short-term memory (Bi-LSTM). To identify the transmitted symbols, the suggested detector uses DL training data directly. Currently, a significant drawback of UFMC systems is the presence of a high peak-to-average power ratio (PAPR). The approach aims to reduce the BER and enhances the efficiency of the UFMC system. This is achieved by dynamically setting the constellation mapping and symbol damping on each subcarrier and sub-symbol. The results illustrate that the proposed model can accurately and efficiently recognize UFMC signals. The suggested model is being compared to Least Square (LS), LSTM, and Minimum Mean Square Error (MMSE) channel estimators. Through extensive simulations, our results demonstrate that the NN model reduces BER and enhances efficiency. The proposed model gives more effective performance in terms of enhanced efficiency and reduction of BER. The findings offer valuable insights for the design and optimization of next-generation communication systems, where accurate and efficient performance evaluation is critical.

This paper studies the effect of channel estimation in WiMAX system. The channel estimation is based on the principle of pilot-symbol-aided channel estimation (PACE) in frequency domain. The algorithms used here are based on the least squares (LS) and the minimum mean squares error (MMSE) approaches. Furthermore we propose a simple method to reduce the computation complexity of MMSE algorithm significantly which has a desirable performance. In the end, our result will quantitatively demonstrate the effect of LS, MMSE and proposed low complexity MMSE algorithms for uncoded IEEE802.16d standard of WiMAX system.

Recent dominant approaches for scene text recognition are mainly based on convolutional neural network (CNN) and recurrent neural network (RNN), where the CNN processes images and the RNN generates character sequences. Different from these methods, we propose an attention-based architecture1 which is completely based on CNNs. The distinctive characteristics of our method include: (1) the method follows encoder-decoder architecture, in which the encoder is a two-dimensional residual CNN and the decoder is a deep one-dimensional CNN. (2) An attention module that captures visual cues, and a language module that models linguistic rules are designed equally in the decoder. Therefore the attention and language can be viewed as an ensemble to boost predictions jointly. (3) Instead of using a single loss from language aspect, multiple losses from attention and language are accumulated for training the networks in an end-to-end way. We conduct experiments on standard datasets for scene text recognition, including Street View Text, IIIT5K and ICDAR datasets. The experimental results show our CNN-based method has achieved state-of-the-art performance on several benchmark datasets, even without the use of RNN.

In recent years, Deep Learning (DL), such as the algorithms of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Generative Adversarial Networks (GAN), has been widely studied and applied in various fields including agriculture. Researchers in the fields of agriculture often use software frameworks without sufficiently examining the ideas and mechanisms of a technique. This article provides a concise summary of major DL algorithms, including concepts, limitations, implementation, training processes, and example codes, to help researchers in agriculture to gain a holistic picture of major DL techniques quickly. Research on DL applications in agriculture is summarized and analyzed, and future opportunities are discussed in this paper, which is expected to help researchers in agriculture to better understand DL algorithms and learn major DL techniques quickly, and further to facilitate data analysis, enhance related research in agriculture, and thus promote DL applications effectively. Keywords: deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition DOI: 10.25165/j.ijabe.20181104.4475 Citation: Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

Deep learning methodologies have had a major impact on performance across a wide variety of machine learning tasks, and speech recognition is no exception. We describe a set of deep learning techniques that proved to be particularly successful in achieving performance gains in word error rate on a popular large vocabulary conversational speech recognition benchmark task (“Switchboard”). We found that the best performance is achieved by combining features from both recurrent and convolutional neural networks. We compare two recurrent architectures: partially unfolded nets with max-out activations and bidirectional long short-term memory nets. In addition, inspired by the success of convolutional networks for image classification, we designed a convolutional net with many convolutional layers and small kernels that create a receptive field with more nonlinearity and fewer parameters than standard configurations. When combined, these neural networks achieve a word error rate of 6.2% on this difficult task; this was the best reported rate at the time of this writing and is even more remarkable given that human performance itself is estimated to be 4% on this data.

For a 5G wireless communication system, a convolutional deep neural network (CNN) is employed to synthesize a robust channel state estimator (CSE). The proposed CSE extracts channel information from transmit-and-receive pairs through offline training to estimate the channel state information. Also, it utilizes pilots to offer more helpful information about the communication channel. The proposed CNN-CSE performance is compared with previously published results for Bidirectional/long short-term memory (BiLSTM/LSTM) NNs-based CSEs. The CNN-CSE achieves outstanding performance using sufficient pilots only and loses its functionality at limited pilots compared with BiLSTM and LSTM-based estimators. Using three different loss function-based classification layers and the Adam optimization algorithm, a comparative study was conducted to assess the performance of the presented DNNs-based CSEs. The BiLSTM-CSE outperforms LSTM, CNN, conventional least squares (LS), and minimum mean square error (MMSE) CSEs. In addition, the computational and learning time complexities for DNN-CSEs are provided. These estimators are promising for 5G and future communication systems because they can analyze large amounts of data, discover statistical dependencies, learn correlations between features, and generalize the gotten knowledge.

Deep Learning and Beyond: AI’s New Horizons” is an expansive monograph that delves into the dynamic and rapidly evolving field of deep learning, a key area of artificial intelligence. This comprehensive work provides an insightful exploration of the revolutionary developments, foundational concepts, and future directions of deep learning, positioning itself as a crucial resource for understanding this transformative technology. The monograph commences with an introductory chapter that lays the groundwork by elucidating the background of machine learning and chronicling the ascent of deep learning. This foundational perspective is vital for comprehending the underlying principles and methodologies that have propelled deep learning to the forefront of technological innovation. Subsequent chapters delve into the “Historical Evolution” of AI, tracing the origins of neural networks from their early inception to the Renaissance period of deep learning. This historical analysis is pivotal in understanding the field’s trajectory and significant milestones. The “Foundations of Deep Learning” chapter presents an in-depth analysis of neural networks, covering their basic architecture, activation functions, layers, and the critical process of backpropagation and gradient descent. This is followed by examining “Advanced Neural Network Architectures,” where the focus shifts to state-of-the-art models such as convolutional neural networks, recurrent neural networks, and groundbreaking transformers and attention mechanisms. In addressing the “Training Deep Models,” the monograph explores regularization techniques, optimization strategies, and the increasingly significant role of transfer learning and pre-trained models. The application-oriented chapter, “Applications of Deep Learning,” showcases real-world implementations in domains like image recognition, natural language viii processing, and autonomous systems, illustrating deep learning technologies’ vast potential and versatility. The monograph also tackles the “Challenges in Deep Learning,” including overfitting, interpretability, and the ethical considerations surrounding AI deployment. It then ventures into the “Future of Deep Learning,” contemplating emerging trends such as integrating quantum computing, neural architecture search, and the convergence of neuroscience and deep learning. “Case Studies” provide practical insights and empirical evidence of the impact of deep learning. At the same time, the concluding chapter synthesizes the current state of deep learning, offers predictions for the future, and provides reflective thoughts on this transformative field. “Deep Learning and Beyond: AI’s New Horizons” educates and informs readers about the vast potential and future possibilities of deep learning and artificial intelligence. Keywords: Deep Learning, Artificial Intelligence, Neural Networks, Machine Learning, Convolutional Neural Networks, Recurrent Neural Networks, Transformers, Quantum Computing, Ethical AI, Neural Architecture Search, Natural Language Processing, Autonomous Systems, AI in Healthcare, Future of AI, AI Applications, AI Challenges.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Signal detection is crucial in multi-input multi-output orthogonal frequency division multiplexing (MIMO-OFDM) systems, yet classical detection methods often struggle with nonlinear issues in wireless channels. To handle this challenge, we propose a novel signal detection method for MIMO-OFDM system based on the fractional Fourier transform (FrFT), leveraging the robust time series processing capabilities of long short-term memory (LSTM) networks. Our innovative approach, termed IM-LSTMNet, integrates LSTM with convolutional neural networks (CNNs) and incorporates a Squeeze and Excitation Network to emphasize critical information, enhancing neural network performance. The proposed IM-LSTMNet is applied to the FrFT-based MIMO-OFDM system to improve signal detection performance. We compare the detection results of IM-LSTMNet with zero forcing (ZF), minimum mean square error (MMSE), simple LSTM neural network, and CNN–LSTM network by evaluating the bit error rate. Experimental results demonstrate that IM-LSTMNet outperforms ZF, MMSE, LSTM, and other methods, significantly enhancing system signal detection performance. This work offers a promising advancement in MIMO-OFDM signal detection, presenting a deep learning-based solution that effectively improves the system signal detection performance.

Recent experience in applying recurrent neural networks (RNNs) to interpreting Permannt Downhole Gauge (PDG) records has highlighted the potential utility of machine learning algorithms to learn reservoir behavior from data. The power of the RNN resides in its ability to retain information in a form of memory of previous patterns and information contained in the previous behavior of phenomena being modeled. This memory plays a role of informing the decision at the present time by using what happened in the past. This property suggests RNNs as a suitable choice to model sequences of reservoir information, even when the reservoir modeler is faced with incomplete knowledge of the underlying physical system. Convolutional Neural Networks (CNNs) are another variant of machine learning algorithms that have shown promise in sequence modeling domains, such as audio synthesis and machine translation. In this study, RNN and CNN were applied to tasks that traditionally would be modeled by a reservoir simulator. This was achieved by formulating the relationship between physical quantities of interest from subsurface reservoirs as a sequence mapping problem. In addition, the performance of a CNN layer as compared to RNN was evaluated systematically to investigate their capabilities in a variety of tasks of interest to the reservoir engineer. Preliminary results suggest that CNNs, with specific design modifications, are as capable as RNNs in modeling sequences of information and as reliable when making inferences to cases that has not been seen by the algorithm during training. Design details and reasons pertaining to the way these two seemingly different architectures process information, and handle memory are also discussed.

A hybrid model based on neural networks for biomedical relation extraction

Orthogonal frequency division multiplexing (OFDM) is a multicarrier modulation used in a variety of broadband systems, such as asymmetric digital subscriber lines, very-high-speed digital subscriber lines, digital video and audio broadcasting, and wide local area network standards. OFDM is a promising solution achieving high data rates in a mobile environment. In wireless communication systems, signal distortion and attenuation in transmission caused by multipath effects makes it necessary to obtain knowledge related to the channel impulse response using channel estimation to provide compensation. This study combined a back propagation neural network for the estimation of channel and compensation signals with a genetic algorithm to improve performance and the convergence rate. We compared bit error rates and the mean square error of the proposed neural network with that of the conventional neural network, the least square (LS) algorithm, and the minimum mean square error (MMSE) algorithm in existing OFDM channel environments. Our results demonstrate that the proposed algorithm outperforms the LS algorithm and is on par with the MMSE algorithm.