Compressed Domain Image Classification Using a Dynamic-Rate Neural Network

Summary Advances in machine learning and data science have given rise to a new set of tools and methods for developing efficient fit-for-purpose proxy models. Two major difficulties in adopting these methods are (i) the lack of physical constraints and domain insight in the design and construction of these models, and (ii) their limited interpretability, which can seriously impede their broader application to scientific domains, including reservoir engineering. In this paper, neural network architectures are designed based on simple reservoir engineering insights to characterize inter-well connectivity and well production performance under different operational conditions. Two different approaches are presented for construction of a neural network proxy model. The first approach takes a global view by representing all well connections in a fully connected neural network to allow any injector to be connected to a producer. The resulting network is a highly redundant description as many of the included connections are not plausible. To systematically eliminate unrealistic connections, l1-norm regularization is adopted to sparsify the network topology (connections) during the training stage. The resulting sparse structure characterizes the inter-well connectivity and defines a network topology that is supported by the training data. In the second approach, a local view is taken in building the proxy model. In this case, each producer is assumed to be supported by very few surrounding injection wells and likely to have weak connections with distant wells. However, inter-well connectivity in complex large-scale reservoirs is not just a function of distance and rather difficult to determine. Therefore, we use a series of randomly sized neighborhoods around each producer to include different number of injectors in the local networks. The variability in the neighborhood size reflects the prior uncertainty about the potential connectivity between wells at different distances. This approach results in many local neural networks (several local networks per each producer) that can be aggregated into a single large neural network model with pre-defined topological structure to represent possible connections. The training is then used to estimate the weights in the resulting architecture. The methods are applied to predict inter-well connectivity and oil production in a large-scale mature field that undergoes water flooding. Examination of the estimated connectivity parameters inside the neural network shows that the retrieved inter-well connectivity map is consistent with the existing geological features. The results suggest that even simple engineering insights can lead to noticeable improvement in the performance of neural networks.

This research study examines the use of two models of artificial intelligence based on a single neural network (SNN) and bootstrap aggregated neural networks (BANN) for the prediction value of hourly global horizontal irradiance (GHI) received over one year in Tamanrasset City (Southern Algeria). The SNN and BANN were created using overall data points. To improve the accuracy and durability of neural network models generated with a limited amount of training data, stacked neural networks are developed. To create many subsets of training data, the training dataset is re-sampled using bootstrap re-sampling with replacement. A neural network model is created for each set of training datasets. A stacked neural network is created by combining multiple individual neural networks (INN). For the testing phase, higher correlation coefficients (R = 0.9580) were discovered when experimental global horizontal irradiance (GHI) was compared to predicted global horizontal irradiance (GHI). The performance of the models (INN, BANN, and SNN) demonstrates that models generated with BANN are more accurate and robust than models built with individual neural networks (INN) and (SNN).

Previous approaches for scene text detection have already achieved promising performances across various benchmarks. However, they usually fall short when dealing with challenging scenarios, even when equipped with deep neural network models, because the overall performance is determined by the interplay of multiple stages and components in the pipelines. In this work, we propose a simple yet powerful pipeline that yields fast and accurate text detection in natural scenes. The pipeline directly predicts words or text lines of arbitrary orientations and quadrilateral shapes in full images, eliminating unnecessary intermediate steps (e.g., candidate aggregation and word partitioning), with a single neural network. The simplicity of our pipeline allows concentrating efforts on designing loss functions and neural network architecture. Experiments on standard datasets including ICDAR 2015, COCO-Text and MSRA-TD500 demonstrate that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both accuracy and efficiency. On the ICDAR 2015 dataset, the proposed algorithm achieves an F-score of 0.7820 at 13.2fps at 720p resolution.

Introduction. The article considers the solution of the problem of detecting fires at technological facilities in automatic mode. To solve this problem, an approach was chosen to create a convolutional neural network capable of operating on a real-time video stream. Aims and Purposes. The aim of this work is to create a neural network capable of detecting flames and smoke in the image from CCTV cameras. The purposes of the work include: selection of the optimal architecture of the neural network in accordance with the latest research in this field; speeding up the chosen architecture using quantization and filter thinning techniques. Methods. Different architectures of convolutional neural networks performing the task of detecting objects in an image are considered. Their performance and quality of work are compared. The YOLOv5 architecture, its target function, training methods and ways of speeding up work are considered. Results and discu ssion. The paper shows the results of training a convolutional neural network of the YOLOv5 architecture for the task of flame and smoke detection, as well as how the results change when applying neural network acceleration methods. It was determined that the use of such acceleration methods such as quantization and filter cleaning can significantly increase up the speed of the neural network, while almost no loss in accuracy of operation. Conclusions. As a result of the conducted work, the architecture of the neural network was determined for performing the task of detecting a fire source. Based on the chosen architecture, a neural network was trained to detect flames and smoke in the image. The speed of its work allows to process video stream in real-time without using graphic accelerator.

This study examines the high temperature deformation and constitutive modeling of flow stress for 310s stainless steel. To this end, hot tensile experiments were conducted under the temperatures of 700° and 800° at the strain rates of 0.0002/s, 0.002/s, and 0.02/s. Flow stress was modeled using the Arrhenius type constitutive equation and neural network approach. Specifically, Regime-Switched Neural Networks (RSNN), a set of neural networks with switching, has been newly proposed as a better predictive model for flow stress. The modeling performance of the RSNN model was evaluated through a comparison with traditional Arrhenius-type constitutive equations and the single neural network model. The results showed that the accuracy of the proposed RSNN was substantially higher than those of the existing Arrhenius-type equations, and the prediction performance was therefore significantly improved. In addition, the accuracy of the proposed RSNN was improved by approximately more than 24% in comparison with the existing global model—a single neural network—thus confirming the superiority of the proposed switching model.

The design of neural network models involves numerous complexities, including the determination of input vectors, choosing the number of hidden layers and their computational units, and specifying activation functions for the latter. The combinatoric possibilities are daunting, yet experience has yielded informal guidelines that can be useful. Alternatively, current research on genetic algorithms (GA) suggests that they might be of practical use as a formal method of determining ‘good’ architectures for neural networks. In this paper, we use a genetic algorithm to find effective architectures for backpropagation neural networks (BP). We compare the performance of heuristically designed BP networks with that of GA-designed BP networks. Our test domains are sets of problems having compensatory, conjunctive, and mixed-decision structures. The results of our experiment suggest that heuristic methods produce architectures that are simpler and yet perform comparatively well. © 1998 John Wiley & Sons, Ltd.

Bio-informatics data sets may be large in the number of examples and/or the number of features. Predicting the secondary structure of proteins from amino acid sequences is one example of high dimensional data for which large training sets exist. The data from the KDD Cup 2001 on the binding of compounds to thrombin is another example of a very high dimensional data set. This type of data set can require significant computing resources to train a neural network. In general, decision trees will require much less training time than neural networks. There have been a number of studies on the advantages of decision trees relative to neural networks for specific data sets. There are often statistically significant, though typically not very large, differences. Here, we examine one case in which a neural network greatly outperforms a decision tree; predicting the secondary structure of proteins. The hypothesis that the neural network learns important features of the data through its hidden units is explored by a using a neural network to transform data for decision tree training. Experiments show that this explains some of the performance difference, but not all. Ensembles of decision trees are compared with a single neural network. It is our conclusion that the problem of protein secondary structure prediction exhibits some characteristics that are fundamentally better exploited by a neural network model.

Forecasting stock prices has always been a complex and challenging task due to its high volatility, noise in data, and abrupt price fluctuations caused by different factors. Traditional approaches, such as quantitative, fundamental, and technical analysis, often struggle to accurately predict stock price movements due to their difficulty in processing non-linear data. The evolution of machine learning, particularly neural network models, opened new opportunities for improving prediction accuracy by handling non-linear and multidimensional data with high effectiveness. This study focuses on evaluating and comparing three neural network architectures, in which the last is a combination of the first two - Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and a hybrid CNN-LSTM model, for forecasting stock prices. Using ten years of historical stock data, taken for Apple Inc. (AAPL) stocks as an example, ranging from 2015 to 2025, the models were trained and tested on key performance metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and R². Results show that while both CNN and LSTM models show good performance individually, the hybrid CNN-LSTM model can significantly outperform them, achieving the highest accuracy with the lowest error rates. By combining the strengths of CNN for feature extraction and LSTM for temporal sequence processing, the hybrid model effectively captures complex patterns in stock price movements. By iterating over the types of layers to use, we came to the conclusion, that by combining both models into one hybrid model, some layers, like one CNN layer, and one LSTM layer should be removed. This research demonstrates the potential of hybrid neural network models in financial time-series forecasting and comprehensively analyzes their advantages over non-hybrid models.

This paper presents a novel and customized hardware architecture for the realization of artificial neural networks on reconfigurable computing platforms like FPGAs. The proposed architecture employs only one single-hardware-computing layer (namely SHL-ANN) to perform the whole computing fabric of multi-layer feed-forward neural networks. The 16-bit half-precision floating-point number format is used to represent the weights of the designed network. We investigate the scalability and hardware resource utilization of the proposed neural network architecture on the Xilinx Virtex-5 XC5VLX-110T FPGA. For performance evaluation, the handwritten digit recognition application with MNIST database is performed, which reported the best recognition rate of 97.20% when using a neural network architecture of size 784-40-40-10 with two hidden layers, occupying 91.8% FPGA hardware resource. Experimental results show that the proposed neural network architecture is a very promising design choice for high-performance embedded recognition applications.

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

In this paper, in order to solve the existing problems of the low recognition rate and poor real-time performance in limb motor imagery, the integrated back-propagation neural network (IBPNN) was applied to the pattern recognition research of motor imagery EEG signals (imagining left-hand movement, imagining right-hand movement and imagining no movement). According to the motor imagery EEG data categories to be recognized, the IBPNN was designed to consist of 3 single three-layer back-propagation neural networks (BPNN), and every single neural network was dedicated to recognizing one kind of motor imagery. It simplified the complicated classification problems into three mutually independent two-class classifications by the IBPNN. The parallel computing characteristic of IBPNN not only improved the generation ability for network, but also shortened the operation time. The experimental results showed that, while comparing the single BPNN and Elman neural network, IBPNN was more competent in recognizing limb motor imagery EEG signals. Also among these three networks, IBPNN had the least number of iterations, the shortest operation time and the best consistency of actual output and expected output, and had lifted the success recognition rate above 97 percent while other single network is around 93 percent.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Dynamical analysis and vibration estimation of a flexible plate with enhanced active constrained layer damping treatment by combinatorial neural networks of surrogates

Determining the best model for the neural network architecture and how to optimize the architecture with the metaheuristic Protis Approach is a subject of the study. A comprehensive investigation and utilization of metaheuristic methods are necessary. These methods aim to solve problems and adapt from the lifestyle of the amoeba protis. In this study, the proposed method modifies the life cycle of the amoeba, which consists of four phases: prophase, metaphase, anaphase, and telophase. These four phases are modified in the neural network architecture to optimize the appropriate number of hidden layers and produce an efficient architecture model. The results show that the protis approach optimized the neural network architecture, especially in generating hidden layers to improve the neural network model. Distinctive features of the results obtained are that the average range of degenerate neurons in the hidden layer is 0 to 35 neurons in each layer. The standard number of neurons makes it possible to solve the problem of determining the best model on the neural network architecture. The protis algorithm embedded in the protis recurrent neural network for categorical data measurements produces an average RMSE value, representing the difference between actual measurements and predictions, equal to 0.066. Consequently, the developed model surpasses the current classical neural network model in terms of performance. Regarding accuracy, the protis algorithm embedded in the neural network for categorical and time series data achieves an average precision of 0.952 and a recall of 0.950. The protis convolutional neural network achieves an accuracy of 95.9 %. Therefore, from the three tested datasets, the protis convolutional neural network exhibits the highest accuracy value

A clustering algorithm based selective neural networks ensemble (CLUSEN) is proposed to predict the degree of malignancy in brain glioma. Since the degree prediction of malignancy is critical before brain surgery, many learning methods are used like rule induction algorithm, single neural networks, support vector machines, etc. Ensemble learning methods can improve the generalization of single learning machine, and are becoming popular in the machine learning and medical data processing communities. The procedure of CLUSEN can efficiently remove redundancy learning individuals and help improve the diversity of ensemble methods. CLUSEN is used to predict the degree of malignancy in brain glioma. Experimental results on a set of brain glioma data show that, compared to support vector machines, rule induction and single neural networks, the classification accuracy of CLUSEN is higher.