General Multi-Composite Sigmoid Relied Banach Space Valued Univariate Neural Network Approximation

<p style='text-indent:20px;'>Here we study the univariate fuzzy fractional quantitative approximation of fuzzy real valued functions on a compact interval by quasi-interpolation arctangent-algebraic-Gudermannian-generalized symmetrical activation function relied fuzzy neural network operators. These approximations are derived by establishing fuzzy Jackson type inequalities involving the fuzzy moduli of continuity of the right and left Caputo fuzzy fractional derivatives of the involved function. The approximations are fuzzy pointwise and fuzzy uniform. The related feed-forward fuzzy neural networks are with one hidden layer. We study also the fuzzy integer derivative and just fuzzy continuous cases. Our fuzzy fractional approximation result using higher order fuzzy differentiation converges better than in the fuzzy just continuous case.</p>

Artificial intelligent provides diverse solutions for the complex problems in agriculture research. The study aimed to use three models of artificial neural networks (Feed Forward Neural Network (FFNN), Generalized Regression Neural Network (GRNN) and Radial-Basis Neural Network (RBNN)) in the field of wheat yield prediction. 27-year data for the period (1986-2012) were utilized to improve the models and four-year data (2013 and 2016) were used to estimate the models, to compare their outputs with the measured data. Prediction data was not entered in the process of building neural network models. The results showed that the optimal configuration of the FFNN model consists of 40 neurons in the hidden layer (8-40-1). The Tan Sigmoid activation function was used in both the hidden layer and the output layer using all of these models (anterior neural feeding network and the regression neural network and radial base neural network) in the 4-year wheat yield forecast field for production (2013-2016) by applying 8 input parameters that were result of NMMS (8.6%, 7.6% and 15.7% resp.), To find that FFNN and GRNN provide the best result from BRNN because while the information set was large or in a wide range, then the range data ranges from -1 to +1 (normalization data) , GRNN gives better outcomes after the information or sample data were in large range.

Purpose In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Design/methodology/approach In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Findings In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Originality/value In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing.

Fractional calculus has been adopted in the modelling of many scientific processes and systems. Due to the inherent feature of long term memory of fractional derivatives, it has been used in the learning process of neural networks. A fractional order derivative based back propagation learning algorithm in neural networks is proposed in this paper. Specifically, Riemann-Liouville (R-L), Caputo (C) and Caputo Fabrizio (CF) fractional Derivative based on the back propagation algorithms in a three layer feed-forward neural network employed. To get a faster learning rate without oscillation, momentum factor is incorporated. The effect of fractional order and momentum factor is investigated and compared. The performance of these fractional derivatives based algorithms with integer derivatives based algorithm in terms of mean square error (MSE), particularly the salary based on years of experience is predicted. Results demonstrate that fractional derivative based learning algorithms outperform the integer derivatives.

Development of controlled environment in greenhouse is of prim importance for out of season production, increasing yield and enhancing the quality of produce.Due to high cost and impossibility of continuous human attendance in greenhouse, it is desirable to control the greenhouse environment by employing automatic control devices.In This study, the greenhouse conditions were controlled by using artificial neural network (ANN).First, an experimental greenhouse was built and equipped with control instruments.Then by using electronic sensors, some climatic parameter data (temperature, humidity, carbon dioxide and light index) were measured and saved during five minute periods.In the next stage, three types of ANN including feed forward neural networks with multiple delays in the input, two-layer neural network with a feedback from hidden layer and input delay and three-layer neural network with two feedbacks from hidden layers and input delay were trained by 66% of the recorded data, and were evaluated by using the remaining data.The three-layer neural network with two feedbacks from hidden layers and input delay was able to better predict humidity and light index of the greenhouse with MSE,s of 0.025 and 0.032, respectively.Temperature and infrared index were better predicted by using the feed forward neural networks with multiple delays in the input with MSE,s of 0.016 and 0.017, respectively.In all cases, training time was less than 14 minutes and simulation time being always less than 0.2 second, makes using neural network feasible for automatic control of greenhouse.

This paper develops and presents a new neural network approach to model trip distribution, which is one of the important phases of conventional four-step travel demand modelling. The trip distribution problem has been investigated using back-propagation artificial neural networks in a number of studies and it was concluded that back-propagation artificial neural networks underperform when compared to traditional models. Such underperformance is due to the thresholding of the linearly combined inputs by utilising a non-linear function and carrying out this operation both in hidden and output layers. The proposed neural trip distribution model does not threshold the linearly combined outputs from the hidden layer. This makes it different from back-propagation artificial neural networks where combined inputs from the hidden layer are activated once more in the output layer. In addition, the neuron in the output layer is used as a summation unit in contrast to the methodologies cited in the neural network applications literature. At the same time, the bias neuron is not connected to the output neuron in the output layer. When this model is compared with various approaches such as the gravity model, modular neural networks and back-propagation neural networks, it was concluded that this new model provides better prediction of trip distribution and therefore, outperforms all the existing approaches.

In theory, recent results in nonparametric regression show that neural network estimates are able to achieve good rates of convergence provided suitable assumptions on the structure of the regression function are imposed. However, these theoretical analyses cannot explain the practical success of neural networks since the theoretically studied estimates are defined by minimizing the empirical L_2 risk over a class of neural networks and in practice, solving this kind of minimization problem is not feasible. Consequently, the neural networks examined in theory cannot be implemented as they are defined. This means that neural network in applications differ from the ones that are analyzed theoretically. In this thesis we narrow the gap between theory and practice. We deal with neural network regression estimates for (p,C)-smooth regression functions m that satisfy a projection pursuit model. We construct three implementable neural network estimates and show that each of them achieve up to a logarithmic factor the optimal univariate rate of convergence. Firstly, for univariate regression functions with p contained in [-1/2,1] we construct a neural network estimate with one hidden layer where the weights are learned via gradient descent. The starting weights are randomly chosen from an interval independently of the data. The interval is large enough to guarantee that the estimate is close to a piecewise constant approximation. Secondly, for multivariate regression functions with p contained in (0,1] we construct a neural network estimate with one hidden layer where the weights are learned via gradient descent. The initial weights are chosen from specific intervals dependently on the data and the projection directions. This choice guarantees that the estimate is close to a piecewise constant approximation. The projection directions are repeatedly chosen randomly. Lastly, for multivariate regression functions with p>0 we construct a multilayer neural network estimate. The value of the inner weights are prescribed dependently on the projection directions by a new approximation result for a projection pursuit model by piecewise polynomials. The outer weights are chosen by solving a linear equation system. The projection directions are repeatedly chosen randomly. Since we are able to show a rate of convergence that is independent of the dimension of the data our second and third estimates are able to circumvent the curse of dimensionality.

ABSTRACTWe investigate an isotonic parameterization for monotone polynomials previously unconsidered in the statistical literature. We show that this parameterization is more flexible than its alternatives through enabling polynomials to be constrained to be monotone over either a compact interval or a semi-compact interval of the form , in addition to over the whole real line. Furthermore, algorithms based on our new parameterization estimate the fitted monotone polynomials much faster than algorithms based on previous isotonic parameterizations which in turn makes the use of standard bootstrap methodology feasible. We investigate the use of the bootstrap under monotonicity constraints to obtain confidence bands for the fitted curves and show that an adjustment by using either the ‘m out of n’ bootstrap or a post hoc symmetrization of the confidence bands is necessary to achieve more uniform coverage probabilities. We illustrate our new methodology with two real world examples which demonstrate not only the need for such techniques, but how restricting the monotonicity constraints to be over either a compact or semi-compact interval allows the fitting of even degree monotone polynomials. We also describe methods for using the ‘m out of n’ bootstrap to select the degree of the fitted monotone polynomial. All algorithms discussed in this paper are available in the R package MonoPoly (version 0.3-6 or later).

Space-Division Multiple Access-OFDM based wireless communication has the potential to significantly increase the spectral efficiency, system performance and number of users. Research in the development of efficient multiuser signal detection algorithms for such systems has generated much interest in recent years. This research proposes MUD schemes utilizing three neural network (NN) models like Feed Forward NN (FNN) without any hidden layer, FNN with a single hidden layer and Recurrent Neural Network (RNN) as possible alternatives to existing Genetic Algorithm (GA) based Minimum Bit Error Rate (MBER) MUD. Further, these techniques offer low complexity. Extensive simulation based performance study is carried out to prove the efficiency of the proposed techniques. Better performance of FNN with hidden layer and RNN compared to FNN without hidden layer is clearly observed in rank deficient MIMO scenario, where number of users exceeds the number of receiving antennas. The Bit Error Rate (BER) and complexity analysis of the proposed neural MUD schemes are close to optimal ML and show improvement over the previous implemented technique.

IntroductionSimulating thermal convection in Earth and planetary interiors is crucial for various applications, including benchmarking numerical codes (e.g. [1], [2]), deriving scaling laws for heat transfer in complex flows (e.g. [3], [4]), determining mixing efficiency (e.g. [5], [6]) and the characteristic spatial wavelengths of convection (e.g. [7]). However, reaching a statistically-steady state in these simulations, even in 2D, can be computationally expensive. While choosing "close-enough" initial conditions can significantly speed simulations up, this selection process can be challenging, especially for systems with multiple controlling parameters. This work explores how machine learning can be leveraged to identify optimal initial conditions, ultimately accelerating numerical convection simulations on their path to a statistically-steady state.Convection model and neural network architectureWe have compiled a comprehensive dataset comprising 128 simulations of Rayleigh-Bénard convection within a rectangular domain of aspect ratio 4, characterized by variable viscosity and internal heating. A randomized parameter sweep was employed to vary three key factors: the ratio of internal heat generation to thermal driving forces (RaQ/Ra), the temperature-dependent viscosity contrast (FKT), and the pressure-dependent viscosity contrast (FKV). Each simulation was executed until the root mean square velocity and mean temperature attained a statistically stationary state. Subsequently, we extracted the time-averaged one-dimensional temperature profile, horizontally averaged across the domain, over the final 200 time steps of each simulation.Figure 1: Schematic of the neural network used to predict statistically-steady temperature profiles.We used a feedforward neural network (NN) to predict the 1D temperature profiles as a function of the control parameters (Fig. 1). We modified the formulation of the NN from the one we used in [8] in the following ways. First, instead of each training example consisting of the full temperature profile, we now predict temperature at a given spatial point in depth (y). This allows the network to learn and predict profiles on variable computational grids, making it more adaptable. This formulation also avoids the "wiggles" encountered when predicting high-dimensional profiles. Second, we use skip-connections. For a given hidden layer, the features from all the previous hidden layers are added to it before applying the activation function. Third, we use SELU instead of tanh for faster convergence. Fourth, we concatenate the parameters to the last hidden layer, a technique shown to improve learning in some cases  (e.g., [9]). Fifth, as a consequence of conditioning the NN on spatial points, we augment the data with copies of points from the bottom and top thermal boundary layers to ensure that each batch is likely to be more representative of relevant temperature features occurring at the outer boundaries of the domain. We split the dataset into training, cross-validation, and test sets with 96, 15, and 16 simulations, respectively. To test the extrapolation capacity of the network, we pick simulations for the test set only where at least one of the simulation parameters exceeds a certain threshold. The cross-validation dataset consists of interpolated values only and is used additionally to choose the best performing NN architecture and save network weights only when the cross-validation loss improves. ResultsWe test the NN against the following baselines: linear regression, kernel ridge regression, and nearest neighbor. As shown in Table 1, when predicting unseen profiles from interpolated cases in the cross-validation set or even the extrapolated cases (Test set), the NN predictions are the most accurate among the tested possibilities. We visualize some of the profiles predicted by these algorithms in the cross-validation set in Fig. 2. Linear regression results are left out for ease of visualization.  Algorithm                  Training set Cross-validation set Test set Linear Regression  0.0385 0.0388 0.0676 Kernel Ridge Regression  0.0148 0.0147 0.0371 Neural Network 0.0071  0.0071  0.0187 Nearest Neighbor   0.0 0.0282  0.0495 Table 1: Mean absolute error of the prediction of 1D temperature profiles of statistically-steady simulations based on different tested algorithms. The lowest error per dataset is highlighted in bold.The plots in Fig. 2 corroborate the obtained statistics about the accuracy of different methods. When the simulation parameters we are predicting happen to be close to a simulation in the training set, the nearest neighbor profile already provides a good estimate. Otherwise, the NN predictions appear to be the most accurate. This is encouraging because it shows that even for small datasets such as the one we considered, machine learning can already start delivering useful results. Figure 2: Four temperature profiles from the cross-validation dataset (solid red lines) and the corresponding predictions from different models (dashed lines).OutlookWe are currently exploring how initial conditions impact the time a system takes to reach a statistically-steady state in terms of temperature and RMS velocity. We're comparing four starting points: hot, cold, linear temperature profiles, and predictions from our neural network. This will quantify the efficiency gain achieved by using the NN predictions to accelerate simulations towards equilibrium. In addition, we also plan to share the dataset and trained neural network with the community.

While recent advances have successfully integrated neural networks with physical models to derive numerical solutions, there remains a compelling need to obtain exact analytical solutions. The ability to extract closed-form expressions from these models would provide deeper theoretical insights and enhanced predictive capabilities, complementing existing computational techniques. In this paper, we study the nonlinear Gardner equation and the (2+1)-dimensional Zabolotskaya–Khokhlov model, both of which are fundamental nonlinear wave equations with broad applications in various physical contexts. The proposed models have applications in fluid dynamics, describing shallow water waves, internal waves in stratified fluids, and the propagation of nonlinear acoustic beams. This study integrates a modified generalized Riccati equation mapping approach and a novel generalized G′G-expansion method with neural networks for obtaining exact solutions for the suggested nonlinear models. Researchers are currently investigating potential applications of these neural networks to enhance our understanding of complex physical processes and to develop new analytical techniques. The proposed strategies incorporate the solutions of the Riccati problem into neural networks. Neural networks are multi-layer computing approaches including activation and weight functions among neurons in input, hidden, and output layers. Here, the solutions of the Riccati equation are allocated to each neuron in the first hidden layer; thus, new trial functions are established. We evaluate the suggested models, which lead to the construction of exact solutions in different forms, such as kink, dark, bright, singular, and combined solitons, as well as hyperbolic and periodic solutions, in order to verify the mathematical framework of the applied methods. The dynamic properties of certain wave-related solutions have been shown using various three-dimensional, two-dimensional, and contour visualizations. This paper introduces a novel framework for addressing nonlinear partial differential equations, with significant potential applications in various scientific and engineering domains.

The Sixth International Symposium on Neural Networks (ISNN 2009)

The aim of this paper is to estimate the approximation error which results from the method of feedforward neural networks (FNNs) with logarithmic sigmoidal function s(x) = (1 + e <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-x</sup> ) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> . By means of an extending function approach, a class of FNNs with single hidden layer and the active function s(x) is constructed to approximate the continuous function defined on a compact interval. By using the modulus of continuity of function as metric, the rate of convergence of the FNNs is estimated. Also, a numerical examples for illustrating the theoretical results is given.

Deals with the approximation of continuous functions by feedforward neural networks. After presenting one of the main results of Ito, the paper tries to get a universal approximator implementable as a (1+1/2) layer neural network using Heaviside functions as univariate functions. The paper presents an explicit formula for function approximation implementable as a three-layer feedforward neural network instead of a four-layer neural network. These three-layer feedforward neural networks have the same number of neurons in the hidden layer as the equivalent four-layer neural networks have in the second hidden layer.

A review on neural networks with random weights