Placing Feedforward Neural Networks Among Several Circuit Complexity Classes

On the Circuit Complexity of Sigmoid Feedforward Neural Networks

An artificial neural network(ANN) is a type of computing system that mimics the functioning of the neural networks in a biological brain. In this paper, an ANN has been simulated on an Field Programmable Gate Array (FPGA) for the application of handwritten digit recognition using the sigmoid and hyperbolic tangent(tanh) activation functions. Both software and hardware simulations have been carried out, using python and Verilog HDL respectively. The two activation functions were implemented on an Artix-7 FPGA. A comparison has been made between the sigmoid and tanh activation functions based on speed, accuracy, and hardware required, and it has been inferred that the tanh function is best for the application of handwritten digit recognition as it has 3% higher accuracy and uses 5 less LUTs than the sigmoid activation function.

Application and performance analysis of neural networks for decision support in conceptual design

Single-hidden-layer feed-forward quantum neural network based on Grover learning

Artificial intelligent provides diverse solutions for the complex problems in agriculture research. The study aimed to use three models of artificial neural networks (Feed Forward Neural Network (FFNN), Generalized Regression Neural Network (GRNN) and Radial-Basis Neural Network (RBNN)) in the field of wheat yield prediction. 27-year data for the period (1986-2012) were utilized to improve the models and four-year data (2013 and 2016) were used to estimate the models, to compare their outputs with the measured data. Prediction data was not entered in the process of building neural network models. The results showed that the optimal configuration of the FFNN model consists of 40 neurons in the hidden layer (8-40-1). The Tan Sigmoid activation function was used in both the hidden layer and the output layer using all of these models (anterior neural feeding network and the regression neural network and radial base neural network) in the 4-year wheat yield forecast field for production (2013-2016) by applying 8 input parameters that were result of NMMS (8.6%, 7.6% and 15.7% resp.), To find that FFNN and GRNN provide the best result from BRNN because while the information set was large or in a wide range, then the range data ranges from -1 to +1 (normalization data) , GRNN gives better outcomes after the information or sample data were in large range.

This chapter is dedicated to studying the impact of the curriculum learning process and the Swish activation function for finding the solution of Differential Equations (DEs) with initial conditions. Then we have compared the result of the proposed training algorithm and the usual training algorithm. Also, we have compared the neural result using Swish, Tanh, and Sigmoid activation functions. The artificial neural network (ANN) trial solution of the differential equation is written as a sum of two terms. Here the first term satisfies boundary conditions and the second term involves containing adjustable parameters so as the trial solution can solve the differential equations. In this investigation, first we have trained our neural network in a small domain and gradually expanded the domain. Feedforward neural network (FFNN) and error backpropagation algorithm have been used to minimize the error function and modification of weights and biases. Finally, several problems have been solved to illustrate the proposed training method, and analytical results have been compared with neural results.

In this paper, a combined approach of Principal Component Analysis (PCA)-based Extreme Learning Machine (ELM) for boiler output forecasting in a thermal power plant is presented. The input used for this prediction model is taken from the boiler unit of the Yermarus Thermal Power Station (YTPS), India. Calculation of the accurate electrical output of a boiler in an operating system requires the knowledge of hundreds of operating parameters. The dimensionality of the input dataset is reduced by applying principal component analysis using IBM@SPSS Software. In the process of principal component analysis, a dataset of 232 parameters is standardized into 16 principal components. The total dataset collected is divided into training and testing datasets. The extreme learning machine is designed for various activation functions and the number of neurons. Sigmoid and hyperbolic tangent activation functions are studied here. Its generalization performance is examined in terms of the Mean Square Error (MSE), Mean Absolute Error (MAE), Root Mean Square (RMSE), and Mean Absolute Percentage Error (MAPE). ELM and PCA–ELM are compared. In both the ELM and PCA–ELM models, when the extreme learning machine was designed with a sigmoid activation function with 100 nodes in the hidden layer, RMSE was 5.026 and 4.730, respectively. Therefore, the developed combined approach of PCA–ELM proved as a promising technique in forecasting with reduced errors and reduced time.

Chronic wounds are a heavy burden on medical facilities, so any help in treating them is most welcome. Current research focuses on wound analysis, especially wound tissue classification, wound measurement, and wound healing prediction to assist medical personnel in wound treatment, with the main goal of reducing wound healing time. The first phase of wound analysis is wound segmentation, where the task is to extract wounds from the healthy tissue and image background. In this work, a standard feedforward neural network was developed for the purpose of wound segmentation using data from the MICCAI 2021 Foot Ulcer Segmentation (FUSeg) Challenge. It proved to be a simple yet efficient method for extracting wounds from images. The proposed algorithm is part of a compact system that analyzes chronic wounds using a robotic manipulator, RGB-D camera and 3D scanner. The feedforward neural network consists of only five fully connected layers, the first four with Rectified Linear Unit (ReLU) activation functions and the last with sigmoid activation functions. Three separate models were trained and tested using images provided as part of the challenge. The predicted images were post-processed and merged to improve the final segmentation performance.The accuracy metrics observed during model training and selection were Precision, Recall and F1 score. The experimental results of the proposed network provided a recall value of 0.77, precision value of 0.72, and an F1 score (Dice score) of 0.74.

A feedforward neural network with sigmoidal activation functions is proposed to perform the wavelet decomposition of a discrete time signals. The proposed network is made of two parts, the main network and the auxiliary network. The learning of the auxiliary network is achieved off-line, in a prior phase, in order to identify the desired wavelet. This identification is possible due to the properties of a neural network with one hidden layer to approximate any continuous function with a desired accuracy. >

This research mainly based on multilayer perceptron (MLP) neural networks technique of data mining to forecast the wheat crop yield at the district level. There are many statistical and simulation models available, but the proposed algorithm with new activation function provides promising results in a shorter time with more accuracy. Sigmoid and hyperbolic tangent activation functions are widely used in the neural network. The activation functions play an important role in the neural network learning algorithm. The main objective of the proposed work is to develop an amended MLP neural network with new activation function and revised random weights and bias values for crop yield estimation by using the different weather parameter datasets. MLP model has been tested by existing activation functions and newly created activation functions with different cases including weights and bias values. In this research study, we evaluate the result of different activation functions and recommend some new simple activation functions, named DharaSig, DharaSigm and SHBSig, to improve the performance of neural networks and accurate results. Also, three new activation functions created with little variations in the DharaSig function named DharaSig1, DharaSig2 and DharaSig3. In this research study, variable numbers of hidden layers are tested with the variable number of neurons per hidden layer for the agriculture dataset. Variable values of momentum, seed and learning rate are also used in this study. Experiments show that newly created activation functions provide better results compared to ‘sigmoid’ default neural network activation function for agriculture datasets.

We investigate the intersection problem for finite semigroups, which asks for a given set of regular languages, represented by recognizing morphisms to finite semigroups, whether there exists a word contained in their intersection. We introduce compressibility measures as a useful tool to classify the intersection problem for certain classes of finite semigroups into circuit complexity classes and Turing machine complexity classes. Using this framework, we obtain a new and simple proof that for groups and commutative semigroups, the problem is contained in \(\mathsf {NP}\). We uncover certain structural and non-structural properties determining the complexity of the intersection problem for varieties of semigroups containing only trivial submonoids. More specifically, we prove \(\mathsf {NP}\)-hardness for classes of semigroups having a property called unbounded order and for the class of all nilpotent semigroups of bounded order. On the contrary, we show that bounded order and commutativity imply containment in the circuit complexity class \(\mathsf {qAC}^k\) (for some \(k \in \mathbb {N}\)) and decidability in quasi-polynomial time. We also establish connections to the monoid variant of the problem.

Circuit Complexity and Feedforward Neural Networks: Ian Parberry

We demonstrate that it is possible to implement a quantum perceptron with a sigmoid activation function as an efficient, reversible many-body unitary operation. When inserted in a neural network, the perceptron's response is parameterized by the potential exerted by other neurons. We prove that such a quantum neural network is a universal approximator of continuous functions, with at least the same power as classical neural networks. While engineering general perceptrons is a challenging control problem —also defined in this work— the ubiquitous sigmoid-response neuron can be implemented as a quasi-adiabatic passage with an Ising model. In this construct, the scaling of resources is favorable with respect to the total network size and is dominated by the number of layers. We expect that our sigmoid perceptron will have applications also in quantum sensing or variational estimation of many-body Hamiltonians.

Ultrasound (US) imaging is the initial phase in the preliminary diagnosis for the treatment of kidney diseases, particularly to estimate kidney size, shape and position, to give information about kidney function, and to help in diagnosis of abnormalities like cysts, stones, junctional parenchyma and tumors which is shown in Figs. 7–9. This study proposes Grey Level Co-occurrence Matrix (GLCM)-based Probabilistic Principal Component Analysis (PPCA) and Artificial Neural Network (ANN) method for the classification of kidney images. Grey Wolf Optimization (GWO) is used to update the current positions of abnormal kidney images in the discrete searching space, thus getting the optimal feature subset for better classification purposes based on Feed Forward Neural Network (FFNN). The scanned image is pre-processed and the required features are extracted by GLCM, among those, some features are selected by PPCA. Feed Forward Back propagation Neural Network (FFBN) is used to classify the normalities and abnormalities in the part of kidney images. The proposed methodology is implemented in MATLAB platform and the analyzed result produces 98% accuracy using GWO-FFBN technique.

The adoption of wind energy has grown significantly in recent years. New, cost-effective technologies have been developed, led by customer awareness of green technologies and a legal framework proposed at the European Union level. The stochastic nature of wind speed is transferred to wind turbine output, making wind energy difficult to predict. The main scope of predicting wind energy production is to be proactive in balancing and reserving energy to meet demand. When the prediction identifies a potential gap between supply and demand, additional energy from other sources must be generated and supplied. Creating a synergy of physical devices through advanced sensing capabilities, software, storage and analytics capabilities, the Industrial Internet of Things is enabling the effective transition to wind energy through automation by removing many of the disadvantages in a way that has recently become accessible. This research focuses on the data analytics, proposing a fast univariate network-based approach for wind energy prediction, using Feed Forward Neural Networks, Recurrent Neural Networks, Long-Short Term Memory, Gated Recurrent Unit, and Convolutional Neural Networks. Moreover, by introducing the theoretical fundamentals, the implementation method and the hyperparameters of the final models, this article becomes unique in the context of wind energy. At the time of this study, no prior research studies have presented a direct comparison between feedforward, recurrent, and convolutional neural networks ? these being the most important in the field of supervised learning.