Real-time scheduling on heterogeneous system-on-chip architectures using an optimised artificial neural network

A neural network composed of a finite number of continuous value output neurons is known to be able to simulate any deterministic Turing machine. Several simulation models have been proposed, but there remains the problem that a massive number of neurons is needed to simulate the tape part in real time. In this paper the authors demonstrate that a recurrent higher‐order neural network can simulate any Turing machine, then specify the lower limit for the number of neurons required and the time needed for the simulation. First, the authors show that a recurrent higher‐order neural network consisting of a limited number of neurons using a threshold input/output function and a linear input/output function can simulate any deterministic Turing machine in real time, and that the number of neurons required to simulate the tape can be reduced to six. Then the authors show that if a saw‐type function is used, the number of neurons required for the simulation can be reduced to four. A simplified simulation model is then offered as a viable framework for learning formal language using a neural network. © 2001 Scripta Technica, Syst Comp Jpn, 32(10): 42–50, 2001

The Hopfield neural network is expected to act as the engine to solve the association problem or the optimization problem. In the associative processing using the Hopfield neural network, it is necessary that the precalculated weights should be memorized in the neural network. In the Hopfield neural network, however, there must be m2 weights, where m is the number of neurons. In other words, a tremendous memory capacity is required when m is increased. the weight can be stored either by an analog memory or a digital memory. the analog meomory, however, has problems of error and the decay of the memorized content. Then, digital memory must be used. In this case, the required number of memory elements is m2 × (data bits) and the hardware complexity increases in proportion to the number of data bits.Another point is that the hardware, such as the multiplier of the synapse, becomes more complicated with the increase of the number of bits. A remedy for those problems may be to reduce the number of bits for the weight. When a digital code of some 2 bits is used to represent the weight, for example, the memory capacity for storage can be reduced and the multiplication at the synapse can be simplified. One of the important issues then is how to determine the weight when the number of bits for the wieght is reduced. Since it is difficult to approximate the weight by a continuous variable, it is desirable to realize an efficint method which is differrent from the existing method of weight determination.With the foregoing as background, this paper discusses the weight determination method for the case where the weight determination method for the case where the weight can take only a few number of discrete values. In the proposed method, the evaluation function for the weight is examined and the weights are determined by optimization using an algorithm, which is an application of the branch‐and‐bound method. It is shown by simulation that a high associative ability is realized by the association processing using the weights obtained by the proposed method. Future applications can be expected.

This paper investigates the error of determining the coordinates of an acoustic signal source depending on various parameters of the developed software-mathematical models for training a neural network. The authors determined the optimal values of these parameters according to the criterion of minimum error. The study found that the optimal amount of data for training a neural network is 25,000 training pairs, and the number of hidden layers should not exceed 5. Among several forms of sensor arrangement, the best is the square, as it provides a smaller error and uniform filling of the microphone’s perimeter. Their number should not exceed 9, since the best algorithm for training the neural network trainlm requires significant computing resources of the system on which training is performed. An increase in the number of sensors leads to an increase in the training time of the neural network but does not lead to an improvement in accuracy. When studying the effect of the distance to the acoustic signal source and the algorithm for selecting the base sensor for calculating the time difference, it was found that changing these parameters does not affect the accuracy of the neural network. The best structure of the number of hidden neurons in each hidden layer is an expanding structure, in which the number of neurons in each layer increases from the input layer to the middle-hidden layer, then decreases to the output layer. Using the optimized parameters, the neural network was trained and its accuracy was evaluated on two data sets: those used for training and those unknown to the neural network. In both cases, the neural network demonstrated good results. According to the results of the study, it was found that the error of the optimised system is 6.7681 × 10-5 m in the Y coordinate and 6.6120 × 10-5 m in the X coordinate.

Research on artificial neural networks (ANNs) has been carried out for more than five decades. A renewed interest appeared in the 80's with the finding of powerful models like J. Hopfield's recurrent networks, T. Kohonen's self-organizing feature maps, and the back-propagation rule. At that time, there was no platform that was at the same time versatile enough for any ANN model to be implemented and fast enough to solve large problems. Super-computers were the sole exception to this rule, but were prohibitively expensive for most applications. However, both research scientists and application engineers clearly identified the need for such a computing power. This triggered many projects in the field. In parallel, research on multi-processor systems started during the 60's. Systolic arrays have been proposed in 1979 as a means to fully exploit the possibilities of VLSI. Two previous theses, by F. Blayo and C. Lehmann, have studied the use of bi-dimensional systolic arrays for neural computation. At first, the presented system, called GENES, has been designed for the Hopfield model. Extensions to other ANNs have also been proposed. The goal of the present thesis is to study, design, build, and analyze an efficient accelerator for neural computation. In a first step, the GENES architecture has been extended towards generality and efficiency. This includes a thorough analysis of ANN models, of other neural computers, and of previous GENES implementations. The result of this work is the GENES IV integrated circuit, whose architecture has been co-designed by P. Ienne and the author. The main part of this thesis discusses the architecture, the design and the analysis of the MANTRA I machine, a neural computer based on a GENES IV array with up to 40 × 40 processing elements (PEs). The delta rule (and hence the Perceptron and Adaline rules), the back-propagation rule, the Hopfield model, and the Kohonen model can be implemented on this system. Although not a generic system, such a machine may be regarded as a multi-model neural computer. A prototype has been running for a year and is used daily by software designers. Several novel features distinguish the MANTRA I machine from other neural systems. First, it belongs to the few existing neural computers, contrary to the majority of implementations, which are specific to an application or to an algorithm. The machine does not hard-wire any algorithm, but provides the necessary primitives to implement the target models. This is a key feature for research, since several algorithms or versions of an algorithm can be tested on a problem. It is an important aspect for applications as well, because different ANN models are often cascaded to solve a problem. The GENES IV array — that is, the computing core of the MANTRA I machine — features synapse-level parallelism (i.e., one real or virtual PE is allocated per synapse or neural connection), while most other systems exploit only neuron-level parallelism (i.e., one PE per neuron). Hence, this system aims at a much finer parallelism grain and is well suited for massively parallel architectures. The problem size that can be computed by a neural accelerator should not be limited by the hardware (except for memory size). Therefore, it is essential to support time-sharing of PEs. On the MANTRA I machine, this is achieved by the concept of virtual arrays. Matrices are divided into sub-matrices that can be mapped onto the physical array, which is then time-shared among them. An efficient mechanism has been implemented to swap sub-matrices in background, while some other computation is performed. Since systolic arrays are pipelined systems, it is important to avoid emptying and re-filling them too often, in order to keep the hardware utilization rate high. Therefore, a systolic instruction flow has been implemented, so that each instruction follows the data for which it has been issued. Like any SIMD system, the MANTRA I machine is composed of a parallel or SIMD module and a control module. The SIMD module consists of the GENES IV array and a set of dedicated units designed for the computation that scales with the number of neurons and would poorly fit on a bi-dimensional array. A complex system of FIFOs and memories sustains the required input/output streams for the systolic array. The control module is a complete SISD system. Its tasks are (1) to control the SIMD module by dispatching instructions, (2) to manage data input and output, (3) to communicate with the external world, and (4) to perform data pre- and post-processing. The SIMD instructions are of the very long instruction word (VLIW) type. Synchronization between the two modules is achieved by an instruction FIFO. The performance of the MANTRA I machine has been analyzed using the delta rule. Measurements show that the sustained performance is very close to its peak value, as long as the problem fits in the memory banks connected to the GENES IV array. Experiments have also been run to investigate the impact of the constraints imposed by the hardware on the convergence of algorithms. Finally, the use of systolic arrays as neural accelerators is discussed in the light of the experience acquired with the GENES IV array and the MANTRA I machine. The weaknesses of the machine are analyzed, and several solutions are proposed to avoid them in a future design. A general discussion of the future of neural computers concludes this thesis.

An optimized neural network modeling approach with genetic algorithm for short-term load forecasting based on only multiple delayed historical power load data is proposed. Genetic algorithm is used to globally optimize the number of delayed input neurons and the number of neurons in the hidden layer of the neural network architecture. Modification of Levenberg-Marquardt algorithm with Bayesian regularization is used to improve the generalization ability of the neural network. The performance of our proposed approach has been compared using actual power load data sets. Numerical results show that our proposed power load forecasting approach is comparable to the existing approaches that use multiple input variables such as power load data, day type load patterns and weather conditions

A neural network was applied to land cover classification for the purpose of utilizing spatial information. Co-occurrence matrices, which are extracted from single band image data, were used for the input feature matrix to the neural network. The adopted model of the neural network has multi layered architecture with the back propagation algorithm as the training method. Firstly, various models and parameters were evaluated by classification experiments which were threshold models of each neuron, learning parameters at back propagation, the number of hidden layers, the number of neurons in the hidden layers and the size of co-occurrence matrix. Through the experiments, the best performance in landcover classification was achieved with sigmoid function for the threshold model, large learning rate and small momentum as the learning parameters, single hidden layer, two times of output neurons as the number of the hidden neurons and 16×16 to 32×32 as the size of co occurrence matrix. Secondly, landcover classification using the proposed method and three kinds of conventional methods were conducted with Landsat TM and SPOT HRV images. The conventional methods were a pixel wise maximum likelihood classifier, a pixel wise neural network classifier and a 3× 3 pixel wise neural network classifier. As a result, the proposed neural network classifier with the aid of co-occurrence matrix showed 5% to 17% higher classification accuracies than those of the conventional classifiers.

Drilling is one of the most common and fundamental machining processes. Since approximately 40% of all the cutting operations are drilling in industry. It is most frequently performed, material removing process and is used as a preliminary step for many operations, such as reaming, tapping and boring. Because of their importance in nearly all production operations twist drills have been the subject of numerous investigations. Surface finish quality of a machined work piece is an issue of main concern to the manufacturing industry. The aim of the present work is to identify suitable parameters for the prediction of surface roughness. Back propagation neural networks were used for detection of surface roughness. Drill diameter, cutting speed, feed, and machining time were given as inputs to the neural network structure and surface roughness was estimated. Drilling experiments with 10 mm drill size were performed at three cutting speeds and feeds. The number of neurons were selected from 1,2,3,…, 20. The learning rate was selected as 0.01 and no smoothing factor was used. The best structure of neural networks were selected based on the criteria as the minimum of summation of square with the actual value of surface roughness. For statistical analysis, it was assumed that surface roughness depends on cutting speed, feed and machining time. For the mathematical analysis inverse coefficient matrix method was used for calculating the estimated values of surface roughness. Comparative analysis has been done between the actual values and the estimated values obtained by statistical analysis, mathematical analysis and neural network structure.

The performance of neural network as aclassifier depends on several factors such as initializationof weights, its architecture, between class imbalance in thedataset, activation function etc. Though a three layeredneural network is able to approximate any non-linearfunction, number of neurons in the hidden layer plays asignificant roll in the performance of the classifier. In thisstudy, the importance of the number of hidden layerneurons of neural network is analyzed for the classificationof ECG signals. Five different arrhythmias and thenormal beat are classified for different number of hiddenlayer neurons to examine the performance of theclassifier. In this study we get the best number as 35. Afterthe training of the neural network with the optimizednumber of neurons in the hidden layer, we have tested theperformance with three different datasets. The averagesensitivity, specificity and accuracy achieved is 94.91%,99.69% and 99.46% respectively.

This paper proposes a step-by-step technique for combining basic models that forecast electricity consumption in an artificial neural network by the method of preliminary selection and further hybridization. The reported experiments were conducted using data on hourly electricity consumption at the metallurgical plant AO ArcelorMittal Temirtau in the period from January 1, 2019, to November 30, 2021. The current research is related to the planned introduction of a balancing electricity market. 96 combinations of basic models were compiled, differing in the type of neural network, the set of initial data, the order of lag, the learning algorithm, and the number of neurons in the hidden layer. It has been determined that the NARX-type network is the most optimal architecture to forecast electricity consumption. Based on experimental studies, the number of hidden neurons needed to form a planned daily profile should equal 3 or 4; it is recommended to use the conjugate gradient method as a learning algorithm. When selecting models from three groups, it was revealed that the conjugate gradient method produces better results compared to the Levenberg-Marquardt algorithm. It is determined that the values of the selected RMSE error indicator take values of 23.17, 22.54, and 22.56, respectively, for the first, second, and third data groups. The adaptive hybridization method has been shown to reduce the RMSE error rate to 21.73. However, the weights of the best models with values of 0.327 for the first group of data, and 0.336 for the second and third ones, show that the individual use of a separate combination of models is also applicable. The devised forecasting electricity consumption model can be integrated into an automated electricity metering system

The purpose of the studyis to examine the theory and practice ofusing neural networks in education, to develop the concept of aneural network adaptive learning environment, and to implementa neural network model of one of the subsystems of this environment (using a model example of creating an adaptive educationaltrajectory).Materials and methods.The study includes a review of bibliographicsources on the application of neural networks in the field of education.It also includes modeling the structure of a neural network adaptivelearning environment. The PyTorch library was used to programmatically implement the neural network.Results.The prospects for the use of neural networks in the field ofeducation are considered, including various tasks of recognition,diagnostics, classification, clustering, forecasting, optimization,etc. A structural model of an adaptive learning NeuroSmartenvironment is created. This environment includes a number ofsubsystems for solving the following tasks: biometric identificationof the student; determining starting level; choosing an adaptivelearning path; determining the current level of competency development; automated verification of students’ work using recognitiontechnology; analysis of the final result of training; monitoringinformation security incidents in an electronic course, etc. Inorder to study the possibilities and problems of applying neuralnetwork models to the tasks of constructing student adaptivelearning trajectories, a model example of a neural network wascreated. This example illustrates the possibility of using a neural network to select further nodes of the educational trajectorybased on the available data on the current learning parametersin an electronic course. To implement the neural network, thePyTorch deep learning library and Pandas library modules wereused. SGD, Adam, Rprop were used as an optimizer to performgradient descent steps. For each of the optimizers, a computerstudy of the stability of the neural network was carried out byvarying the following parameters: the learning rate coefficient,the number of neurons in the hidden layer, and the number oftraining epochs.Conclusion.It can be assumed that the next stage in the evolutionof the use of neural network technologies in the field of educationwill be their integration into complex multi-component Smart systemscapable of automatically supporting student learning at all stagesof the implementation of their personal educational trajectory. Obviously, the practical implementation of complex neural networksystems of this level is a very difficult task and can be solved so faronly at the level of individual subsystems. There are a number of problems associated with computer simulation of the educationalenvironment based on neural network models: the question of theoptimal structure of the neural network (for example, the numberof neurons and layers in the network) has not been studied enough,there are no clear criteria for the optimality of the adaptive educational trajectory. Nevertheless, it should be noted that the task ofdeveloping new forms and technologies of personalized e-learning isin great demand, which makes modeling based on neural networksespecially relevant.

The complexity of the financial environment and the international community makes the capital flow face various challenges, and it is difficult to obtain accurate credit prediction results in the actual application environment. Considering the complex non-linear characteristics of customer information, the Analytic Hierarchy Process is studied to meet the needs of bank credit risk assessment. On this basis, a depth neural network with different complexities was selected for the three indicators built to classify the features. The composition of the neural network module and the number of neurons were determined by experiment, and Dropout was used to prevent overfitting of the test dataset. Stability and ablation experiments showed that the model can control the error between datasets to 0.021. The ablation experiment showed that the numbers of hidden layers and neurons were the best. Simulation tests showed that the sensitivity and accuracy of this method were 85.25% and 92.55%, respectively, which were superior to other classification methods. The real data of banks in the past four years were tested. The results could accurately classify the risks of enterprises and individual customers, and the results of stress test showed that the model is stable. It is found that traditional credit risk assessment models rely on statistical means and rule decisions, and these methods may not fully reveal the complex non-linear relationship and the internal relationship of financial indicators in high-dimensional data. The combination of deep learning technology and hierarchical analysis can better deal with and explain the complex non-linear problems in bank risk assessment.

In this study, an artificial neural network (ANN) has been developed to predict the adsorption amount of dye (methylene blue) onto multiwalled carbon nanotubes. Batch experiments have been carried out to obtain experimental data. Important parameters in the adsorption system such as initial dye concentration, adsorbent dosage, temperature, pH and contact time have been used as the inputs of the network, while the output is the final concentration of dye in aqueous solution after adsorption. The neural network structure has been optimized by testing various training algorithms and different number of neurons in a hidden layer. An empirical equation for determination of final dye concentration in aqueous solutions after adsorption has been developed by using the weights of the optimized network. The results of the optimized ANN have been compared with conventional models in equilibrium and kinetic fields. According to error analysis and determination coefficient, the ANN was found to be the most appropriate model to describe this adsorption process. Sensitivity analysis showed that initial dye concentration, pH and contact time are the most effective parameters in this process. The influence percentages of these parameters on the output were 28, 24 and 24 %, respectively.

Most neural network models for solving the job-shop scheduling problem (JSP) are energy based and the networks usually take a long time to converge to solutions. We previously proposed (1994) a new neural model called the job-shop scheduling neural networks (JSSNNs), which need no special convergence procedure to be performed and can find optimal or near-optimal solutions of the problem at a much faster speed. However, in this model as well as other energy-based models, the number of neurons are proportional to the batch sizes of the jobs. This may complicate the implementation. In this paper the model is extended to solve this problem. In this extended model, the number of neurons are fixed for different batch sizes. This approach can obtain solutions that are better than, or as good as those in prior work. Furthermore, in this new model, mn(n+7) number of neurons are needed to solve an n-job m-machine problem with an arbitrary batch size for each job. We presents the simulation results of this new type of neural network and compare it with some heuristic dispatching rules for appraising its quality.

Electrical load forecasting is one of the important parts for smart grid system. The reliable prediction of the load demand contributes to the efficient and economical operations and planning. The artificial neural network is used extensively in load demand forecasting. The nonlinear nature of the electrical load demand conforms to the ability of the artificial neural network in calculating the nonlinear relationship of inputs and outputs. Among many models of neural networks, radial basis neural networks yield superior performance in small error and fast simulation time. However, it is challenge to design the radial basis neural networks. The excessive numbers of hidden neurons lead to lacking of generalization or so called overfitting problems. This paper proposes an approach to design the radial basis neural networks that use as least numbers of hidden neurons as possible. The error criterion is optimized based on modified genetic algorithm as the numbers of hidden neurons are incrementally increased. Simulation results of short term load forecasting are calculated in Matlab, and compared to the orthogonal least square error method. The proposed approach gives better results with the same numbers of hidden neurons.

Back propagation is one of the supervised learning and multi-layered training program and uses errors during the process of changing the weight value in the backward process as well as the forward propagation. In the method for predicting cognitive abilities backpropagation the first step is to set the input neuron number, the number of neurons that are hidden, and the number of output neurons. The number of neurons used in the program is 6 neurons consisting of cognitive criteria, 6 hidden neuron layers, and 2 neuron outputs. The highest level of accuracy is in the binary sigmoid and bipolar sigmoid activation functions at the 64th epoch with the accuracy of each function of 82.93% +/- 37.63% and 85.37% +/- 35.34%. The smallest root mean squared error value was found in binary sigmoid of 0.266 with a tolerance of +/- 0.258 on the 100th epoch with an accuracy of 80.49% while for the sigmoid bipolar activation function the smallest root mean squared error value was obtained at the epoch 500 of 0.282 with tolerance +/- 0.353.