Pass Strategy of Robocup Robot System Based on Deep Learning Network

Accurate segmentation of the retinal microvasculature is a critical step in the quantitative analysis of the retinal circulation, which can be an important marker in evaluating the severity of retinal diseases. As manual segmentation remains the gold standard for segmentation of optical coherence tomography angiography (OCT-A) images, we present a method for automating the segmentation of OCT-A images using deep neural networks (DNNs). Eighty OCT-A images of the foveal region in 12 eyes from 6 healthy volunteers were acquired using a prototype OCT-A system and subsequently manually segmented. The automated segmentation of the blood vessels in the OCT-A images was then performed by classifying each pixel into vessel or nonvessel class using deep convolutional neural networks. When the automated results were compared against the manual segmentation results, a maximum mean accuracy of 0.83 was obtained. When the automated results were compared with inter and intrarater accuracies, the automated results were shown to be comparable to the human raters suggesting that segmentation using DNNs is comparable to a second manual rater. As manually segmenting the retinal microvasculature is a tedious task, having a reliable automated output such as automated segmentation by DNNs, is an important step in creating an automated output.

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

Combination of the evolutionary optimization algorithm and traditional neural network, one new kind of neural network which is called evolutionary neural network can be generated. To overcome the demerits of previously proposed evolutionary neural networks, combining the immune continuous ant colony algorithm proposed by author and BP neural network, a new evolutionary neural network is proposed. In this new evolutionary neural network, the architecture and connection weights of BP neural network are all optimized simultaneously by immune continuous ant colony algorithm. At last, to verify this new evolutionary neural network, the typical XOR problem is used. And also, the new evolutionary neural network is compared and analyzed with BP neural network, traditional evolutionary neural network based on genetic algorithm and evolutionary neural network based on evolutionary programming. The computing results show that the precision and efficiency of the new evolutionary neural network are all the best.

Deep learning neural network approach to thermal-wave imaging of damage in solids with application to diffusivity measurements of a green (unsintered) metal powder compact slab

Land cover and its dynamic information is the basis for characterizing surface conditions, supporting land resource management and optimization, and assessing the impacts of climate change and human activities. In land cover information extraction, the traditional convolutional neural network (CNN) method has several problems, such as the inability to be applied to multispectral and hyperspectral satellite imagery, the weak generalization ability of the model and the difficulty of automating the construction of a training database. To solve these problems, this study proposes a new type of deep convolutional neural network based on Landsat-8 Operational Land Imager (OLI) imagery. The network integrates cascaded cross-channel parametric pooling and average pooling layer, applies a hierarchical sampling strategy to realize the automatic construction of the training dataset, determines the technical scheme of model-related parameters, and finally performs the automatic classification of remote sensing images. This study used the new type of deep convolutional neural network to extract land cover information from Qinhuangdao City, Hebei Province, and compared the experimental results with those obtained by traditional methods. The results show that: (1) The proposed deep convolutional neural network (DCNN) model can automatically construct the training dataset and classify images. This model performs the classification of multispectral and hyperspectral satellite images using deep neural networks, which improves the generalization ability of the model and simplifies the application of the model. (2) The proposed DCNN model provides the best classification results in the Qinhuangdao area. The overall accuracy of the land cover data obtained is 82.0%, and the kappa coefficient is 0.76. The overall accuracy is improved by 5% and 14% compared to the support vector machine method and the maximum likelihood classification method, respectively.

Convolutional neural network-based wind pressure prediction on low-rise buildings

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Music is an indispensable part of our life and study and is one of the most important forms of multimedia applications. With the development of deep learning and neural network in recent years, how to use cutting-edge technology to study and apply music has become a research hotspot. Music waveform is not only the main form of music frequency but also the basis of music feature extraction. This paper first designs a method of note extraction based on the fast Fourier transform principle of the audio signal packet route under the self-organizing map (SOM neural network) which can accurately extract the musical features of the note, such as amplitude, loudness, period, and so on. Secondly, the audio segments are divided into summary by adding window moving matching method, and the music features such as amplitude, loudness, and period of each bar are obtained according to the performance of audio signal in each bar. Finally, according to the similarity of the audio music theory of the adjacent summary of each bar, the audio segments are divided, and the music features of each segment are obtained. The traditional recurrent neural network (RNN) is improved, and the SOM neural network is used to recognize the audio emotion features. The final experimental results show that the proposed method based on SOM neural network and big data can effectively extract and analyze music waveform features. Compared with previous studies, this paper creatively proposed a new algorithm, which can more accurately and quickly extract and analyze the data sound waveform, and used SOM neural network to analyze the emotion model contained in music for the first time.

In recent years, passive motion paradigms (PMPs), derived from the equilibrium point hypothesis and impedance control, have been utilised as manipulation methods for humanoid robots and robotic manipulators. These paradigms are typically achieved by creating a kinematic chain that enables the manipulator to perform goal-directed actions without explicitly solving the inverse kinematics. This approach leverages a kinematic model constructed through the training of artificial neural networks, aligning well with principles of cybernetics and cognitive computation by enabling adaptive and flexible control. Specifically, these networks model the relationship between joint angles and end-effector positions, facilitating the computation of the Jacobian matrix. Although this method does not require an accurate robot model, traditional neural networks often suffer from drawbacks such as overfitting and inefficient training, which can compromise the accuracy of the final PMP model. In this paper, we implement the method using a deep neural network and investigate the impact of activation functions and network depth on the performance of the kinematic model. Additionally, we propose a transfer learning approach to fine-tune the pre-trained model, enabling it to be transferred to other manipulator arms with different kinematic properties. Finally, we implement and evaluate the deep neural network-based PMP on the Universal Robots, comparing it with traditional kinematic controllers and assessing its physical interaction capabilities and accuracy.

The speech enhancement effect of traditional deep learning algorithms is not ideal under low signal-to-noise ratios (SNR). Skip connections-deep neural network (Skip-DNN) improves the traditional deep neural network (DNN) by adding skip connections between each layer of the neural network to solve the degradation problem of DNN. In this paper, the Multiresolution Cochleagram (MRCG) features in the gammachirp transform domain are denoised to obtain the improved MRCG (I-MRCG). The noise reduction method adopts the Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator (MMSE-STSA) and takes I-MRCG as the input feature and Skip-DNN as the training network to improve the speech enhancement effect of the model. This paper also proposes an improved source-to-distortion ratio (SDR) loss function. When the loss function uses the improved SDR, it will improve the performance of Skip-DNN speech enhancement model. The experiments in this paper are performed on the Edinburgh dataset. When using I-MRCG as the input feature of Skip-DNN, the average perceptual evaluation of speech quality (PESQ) is 2.9137, and the average short-time objective intelligibility (STOI) is 0.8515. Compared with MRCG as Skip-DNN input features, the improvements are 0.91% and 0.71%, respectively. When the improved SDR is used as the loss function of the speech model, the average PESQ is 2.9699 and the average STOI is 0.8547. Compared with other loss functions, the improved SDR has a better enhancement effect when used as the loss function of the speech enhancement model.

Artificial Intelligence (AI) has transformed a variety of areas, and education is no exception. With the development of deep learning and neural network, AI is poised to change the way people teach and learn. This paper explores the future prospects of deep learning and neural networks in education, highlighting the potential benefits and challenges they may bring. AI technologies, like deep learning algorithms and neural networks, have the potential to transform education through customized learning experiences, intelligent tutoring, streamlining administrative duties, and facilitating data-based decision making. Enhanced personalized learning helps students to learn at their own pace and in their preferred style, smart tutoring systems offer personalized guidance and support. Automation of administrative tasks increases efficiency and accuracy, while data-driven decision making helps educators make informed choices about students' outcomes. However, the implementation of AI in education poses challenges such as data privacy, equity, and the preservation of the teacher-student relationship. Efforts should be made to address these challenges and fully harness the potential of deep learning and neural networks in education.

Loan approval prediction is a critical task in the financial sector, enabling institutions to assess credit risk and make informed lending decisions. This research presents a comprehensive comparative study of various machine learning approaches for loan approval prediction, evaluating both traditional algorithms and deep learning architectures. We implemented and compared five traditional machine learning models (Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, and SVM) against three neural network architectures (Simple Feed-Forward Neural Network, Deep Neural Network with Batch Normalization, and Wide & Deep Network). Using a synthetic dataset of 1,000 loan applications with features including income, credit score, and employment status, we achieved the highest accuracy of 93.5% with Gradient Boosting, closely followed by the Wide & Deep Neural Network at 93.5%. Feature importance analysis revealed credit score (45%) as the most influential factor, followed by income (40%) and employment status (15%). Additionally, we developed a professional Tkinter-based graphical user interface that enables real-time predictions, batch processing, and interactive visualizations. This system demonstrates the practical application of machine learning in financial decision-making and provides a framework for deploying predictive models in production environments. Keywords — Loan Approval, Machine Learning, Deep Learning, Neural Networks, Gradient Boosting, Tkinter GUI, Credit Risk Assessment

Electrochemical machining is a promising non-traditional manufacturing process to make high-quality parts. The benefits of minimal thermally and mechanically induced stresses, free of burr, and a low surface roughness are appealing for industry and research institutes. However, the combined chemical reaction, electric field, fluid mechanics, and material properties involve a significant number of independent parameters which are difficult to analyze in order to draw comprehensive conclusions. To our current knowledge, process responses such as the material removal rate, optimal feed rate, and cutting profile cannot be represented accurately by analytical solutions. In recent years, deep learning has had tremendous success in analyzing sophisticated systems. The improved computation efficiency and reduced size of the training dataset required for deep learning have enabled various prediction models in the manufacturing industry. In this paper, a new approach is developed using the deep convolutional network with the Bayesian optimization algorithm to predict the diameters of the drilled hole from an electrochemical machining process. The Keras application programming interface (API) was used to build the deep convolutional network; the feed rate, pulse-on time, and voltage were used as input parameters to provide a fair comparison with a neural network from previous research. Random dropout layers were added to prevent overfitting of the network. Instead of tuning the network parameter by trial and error, the Bayesian parameter optimization algorithm was implemented to find the optimal set of parameters of the deep convolutional network that yields the minimum mean square error. The proposed algorithm was compared with a previously developed neural network with partially embedded physical knowledge. Improved training speed and accuracy were observed in comparison with the traditional neural network. The prediction model using the proposed deep learning algorithm demonstrated better prediction accuracy and provided a more systematic way to select the hyperparameter for the deep convolutional network.

Purpose: The aim of the article is to develop an algorithm for classifying cracks in the analyzed images using modern methods of deep machine learning and transfer learning based on pretrained convolutional neural network - Inception-ResNet-v2. Design/Methodology/Approach: Transfer learning based on the pretrained convolutional neural network was used to categorize the images. The fully conected layer of the Inception-ResNet-v2 network has been modified. The last layer was trained using a two-class (binary) linear SVM (Support Vector Machine). In total, 20,000 training cases (images) were used to train the fully connected layer within transfer learning process. The research analyzed the possibility of using the deep neural networks for quick and fully automatic identification of cracks / defects on the surface of analyzed parts. Findings: The results indicate that pretrained convolutional neural network using SVM to train a fully connected layer is a very effective solution for visual crack / fault detection. In the analyzed model, a positive classification was obtained at the level of 99.89%. Practical Implications: The model presented in the article can be used in quality control carried out by process monitoring. An effective model for identifying defective parts can be used in both logistics and production processes. Originality/Value: A novelty is the use of a freely available, deep neural network trained to classify 1000 categories of various images for binary categorization of faults (cracks). The algorithm was adjusted by replacing the primary, 1000-output fully connected layer in the Inception-ResNet-v2 network with a binary layer (2 categories). The fully connected layer has been trained using the classification version of the popular SVM learner, but thanks to the combination of this layer with the sophisticated fearure extraction ability of the pre-trained Inception-ResNet-v2 deep network, the resulting predictive model enables the classification of defects with a very high level of accuracy.

The article explores the development of language generation technologies, from machine learning models to fluid neural networks for text generation. English is one of the most widespread languages in the world: it is the official language of more than 60 countries. The events of recent years have led to the development of the popularity of the Ukrainian language not only in the country but also abroad. The article analyses scientific sources on this topic, the results of which formed a base for creating a genealogical tree of the development of language synthesis technology. The study pays particular attention to the problem of automating the generation of texts in the Ukrainian language, which includes the absence of strict grammatical constructions of sentences, numerous rules of word formation, presentation of words, placement of accents, and exceptions to almost all rules. The article presents several different transformative architectures that one can use to generate the Ukrainian language. We considered the problems of training such networks and ways to solve them for machine learning models and neural networks. As a result, we determined the stages of development of a neural network for the generation of texts in the Ukrainian language. Even though neural networks are a powerful tool of artificial intelligence, they have certain limitations that the new architecture – liquid neural networks (LNN) – is devoid of. The article also discusses their advantages and disadvantages. The LNN architecture differs from traditional neural networks in its ability to efficiently process continuous or time series data. If new data is available, LNNs can change the number of neurons and connections in a layer. Due to these two main features of LNN, the network expresses the system’s state over time as opposed to the traditional neural network (NN) approach, so it does not require large amounts of observed training data to obtain accurate results. In conclusion, we should note that, despite their shortcomings, LNN networks are more dynamic, adaptive, efficient, and reliable than traditional neural networks and have many potential uses. Keywords: neural network, Ukrainian language, machine learning, speech synthesis, text-to-speech.