Multi-Model Inference Accelerator for Binary Convolutional Neural Networks

1. In this study, neural network (NN) and response surface (RS) models were developed to investigate the response [average daily gain (ADG) and feed efficiency (FE)] of young broiler chickens to dietary protein and lysine. For this purpose, data on their responses to dietary protein and lysine were extracted from the literature and separate NN and RS models were constructed. 2. Comparison between the NN and RS models revealed higher accuracy of prediction with the NN models compared to the RS models. In terms of R 2 values, the NN models developed for both ADG (R 2 = 0.923) and FE (R 2 = 0.904) were far superior to the RS models (R 2 for ADG = 0.511; R 2 for FE = 0.67). This suggests that the NN models can serve as an alternative option to conventional regression approaches including use of RS models. 3. Optimisation of the NN models developed for response to protein and lysine showed that diets containing 220.7 (g/kg of diet) protein and 12.85 (g/kg of diet) lysine maximise ADG, whereas maximum FE is achieved with diets containing 241.3 and 13.12 (g/kg) protein and lysine, respectively. Based on the optimisation results, optimal dietary protein and lysine concentrations for maximum FE in broiler chickens during the starting period are higher than for ADG.

The molecular regulatory network (MRN) within a cell determines cellular states and transitions between them. Thus, modeling of MRNs is crucial, but this usually requires extensive analysis of time-series measurements, which is extremely difficult to obtain from biological experiments. However, single-cell measurement data such as single-cell RNA-sequencing databases have recently provided a new insight into resolving this problem by ordering thousands of cells in pseudo-time according to their differential gene expressions. Neural network modeling can be employed by using temporal data as learning data. In contrast, Boolean network modeling of MRNs has a growing interest, as it is a parameter-free logical modeling and thereby robust to noisy data while still capturing essential dynamics of biological networks. In this study, we propose a Boolean feedforward neural network (FFN) modeling by combining neural network and Boolean network modeling approach to reconstruct a practical and useful MRN model from large temporal data. Furthermore, analyzing the reconstructed MRN model can enable us to identify control targets for potential cellular state conversion. Here, we show the usefulness of Boolean FFN modeling by demonstrating its applicability through a toy model and biological networks.

Performance comparison of artificial neural network and multiple regression models for wire electrical discharge machining of haste alloy

Background: Data mining (DM) is an approach used in extracting valuable information from environmental processes. This research depicts a DM approach used in extracting some information from influent and effluent wastewater characteristic data of a waste stabilization pond (WSP) in Birjand, a city in Eastern Iran. Methods: Multiple regression (MR) and neural network (NN) models were examined using influent characteristics (pH, Biochemical oxygen demand [BOD 5 ], temperature, chemical oxygen demand [COD], total suspended solids [TSS], total dissolved solid [TDS], electrical conductivity [EC] and turbidity) as the regression input vectors. Models were adjusted to input attributes, effluent BOD 5 (BODout) and COD (CODout). The models performances were estimated by 10-fold external cross-validation. An internal 5-fold cross-validation was also used for the training data set in NN model. The models were compared using regression error characteristic (REC) plot and other statistical measures such as relative absolute error (RAE). Sensitivity analysis was also applied to extract useful knowledge from NN model. Results: NN models (with RAE = 78.71 ± 1.16 for BODout and 83.67 ± 1.35 for CODout) and MR models (with RAE = 84.40% ± 1.07 for BODout and 88.07 ± 0.80 for CODout) indicate different performances and the former was better (P < 0.05) for the prediction of both effluent BOD 5 and COD parameters. For the prediction of CODout the NN model with hidden layer size (H) = 4 and decay factor = 0.75 ± 0.03 presented the best predictive results. For BODout the H and decay factor were found to be 4 and 0.73 ± 0.03, respectively. TDS was found as the most descriptive influent wastewater characteristics for the prediction of the WSP performance. The REC plots confirmed the NN model performance superiority for both BOD and COD effluent prediction. Conclusion: Modeling the performance of WSP systems using NN models along with sensitivity analysis can offer better understanding on exploring the most significant parameters for the prediction of system performance. The findings of this study could build the foundation for prospective work on the characterization of WSP operations and optimization of their performances with a view to conducting statistical approaches.

Computer-aided Prediction of the ZrO2 Nanoparticles' Effects on Tensile Strength and Percentage of Water Absorption of Concrete Specimens

Artificial neural network and neuron models have made significant contributions to the area of neurodynamics. Investigating the dynamics of artificial neurons and neural networks is vital in developing brain-like systems and understanding how the brain functions. Neural network models and memristive neurons are currently demonstrating a lot of promise in the study of neurodynamics. In order to model the dynamics of biological synapses, this study explores the complex dynamical behavior of a discrete fractional Hopfield-type neural network using a flux-controlled memristive element with periodic memductance. Hyperbolic tangent and sine are the heterogeneous activation functions that are implemented in the proposed system to improve nonlinearity and replicate various forms of brain activity. Stability and bifurcation analyses are used to illustrate the nonlinear dynamical nature of the constructed network model. We examine how the fractional order (ν) and periodical memductance aspects influence the dynamics of the system to emphasize the emerging complex phenomena like multi-layered dynamics and the presence of several distinct dynamical states throughout the system variables. Randomness and complexity of the time series data for the proposed system are illustrated with the help of approximate entropy analysis. These findings could help researchers better understand brain-like memory networks, neuromorphic computers, and the theoretical study of neurological and mental abilities. The study of multi-layer attractors can be useful in advanced sensory devices, neuromorphic devices, and secure communication.

Particulate atmospheric pollution in urban areas is considered to have significant impact on human health. Therefore, the ability to make accurate predictions of particulate ambient concentrations is important to improve public awareness and air quality management. This study examines the possibility of using neural network methods as tools for daily average particulate matter with aerodynamic diameter <10 µm (PM10) concentration forecasting, providing an alternative to statistical models widely used up to this day. Based on a data inventory, in a fixed central site in Athens, Greece, ranging over a two-year period, and using mainly meteorological variables as inputs, neural network models and multiple linear regression models were developed and evaluated. Comparison statistics used indicate that the neural network approach has an edge over regression models, expressed both in terms of prediction error (root mean square error values lower by 8.2–9.4%) and of episodic prediction ability (false alarm rate values lower by 7–13%). The results demonstrate that artificial neural networks (ANNs), if properly trained and formed, can provide adequate solutions to particulate pollution prognostic demands.

Machine learning applications on distributed in-memory computing architecture (XIMA), including single-layer feedforward neural network (SLFN)-based learning, binary convolutional neural network (BCNN)-based inferences on passive array as well as One Selector One ReRAM (1s1R) array. In this network, matrix-vector multiplication, the major operation in machine learning on ReRAM crossbar is implemented by three steps with detailed mapping strategy for each step. The process of mapping of X&apos;A to ReRAM. The proposed 3D CMOS-ReRAM accelerator can greatly improve the energy efficiency and neural network processing speed. The matrix-vector multiplication can be intrinsically implemented by ReRAM crossbar. Compared to CMOS, a multibit tensor-core weight can be represented by a single ReRAM, and the addition can be realized by current merging. Moreover, as all the tensor cores for the neural network weights are stored in the ReRAM devices, better area efficiency can be achieved. The power consumption of the proposed CMOS-ReRAM is much smaller than the CMOS implementation due to nonvolatile property of ReRAM. For energy efficiency, our proposed accelerator can achieve 1,055.95 GOPS/W, which is equivalent to 7.661 TOP S/W for uncompressed neural network. Our pro-posed TNN accelerator can also achieve 2.39 x better energy efficiency comparing to 3D CMOS-ASIC result (441.36 GOPS/W), and 227.24x better energy efficiency comparing to Nvidia Tesla K40, which can achieve 1,092 GFLOPS and consume 235W

Background: Nowadays, forecasting models based on nonparametric models have been developed in many branches of science such as mathematics and economics. However, the relatively complicated structure of these models has made them less practical in medical sciences. Objectives: In this study, we investigated the application of a nonparametric regression model to predict the psychological symptoms appearing six months after mild traumatic brain injury. We also made a comparison between the performance of nonparametric regression and artificial neural network models in predicting the psychological symptoms. Methods: In a six-month period during 2015 - 2017, information of 100 mild traumatic patients was included in a prospective cohort study. The data were then divided randomly into two groups of training (n = 50) and testing (n = 50) for 100 times. In the training group, the focus was on 100 artificial neural network and nonparametric regression models. However, in the testing group, a comparison was made between the values obtained using the two final models. To compare the models, the ROC curve and the accuracy rate were finally applied. Results: According to the obtained results, the nonparametric regression model showed an accuracy rate of 91.25% while the neural network model had an accuracy rate of 85.34%. In the experimental set for both neural network and nonparametric regression models, the area under the ROC curve appeared to be 81.51% and 85.73 %, respectively. Conclusions: The nonparametric regression model appeared to be more powerful than the neural network model in predicting psychological symptoms.

An attempt has been made to develop neural network and neuro-fuzzy based models to predict the green compressive strength of clay bonded moulding sand mixtures and to analyse the properties of the mixed sand. Data for the models were generated following standard experimental procedure in the sand laboratory. Process parameters such as clay content, moisture, coal dust and mulling time were varied. The collected data were used in developing these models. The predicted green compressive strengths by neural network and neuro-fuzzy based models have been found to be in good agreement with the experimental values. Furthermore, it has been observed from the results that the predicted values from the neuro-fuzzy model are more accurate than those predicted from the neural network model.

A pre-trained convolutional neural network (CNN) is a feed-forward computation perspective, which is widely used in the embedded systems requiring highly power-and-area efficiency. This paper realizes a binarized CNN which treats only binarized values (+1/-1) for the weights and the activation value. In this case, the multiplier is replaced by an XNOR circuit instead of a dedicated DSP block. Binarization for both weights and activation are more suitable for hardware implementation. However, the first convolutional layer still calculates in integer precision, since the input value is 8 bit RGB pixel, and not binarized. In this paper, we decompose the input value into maps of which each pixel is in 1-bit precision. The proposed method enables a binarized CNN to use bitwise operation in all layers, and shares a binarized convolutional circuit among all convolutional layers. We call this all binarized CNN. We compared our proposal with conventional ones. Since all binarized CNN do not require a dedicated DSP block, our proposal is smaller and 1.2 times faster than the typical CNNs, and almost maintains baseline classification accuracy. In addition, pipelined all binarized CNN achieved 1840 FPS, consumed 0.3 watts and its accuracy was 82.8%.

Pulse coupled neural network (PCNN) is a new neural network model known as the third generation of artificial neural network. It was proposed through the study of the mammalian visual cortex pulse oscillation phenomenon, so PCNN has its biological basis. Over the past 20 years, PCNN has been a hot research field in digital image processing for its unique advantages. And there were also many achievements in other fields of the intelligent information processing. In this paper, we make a review of PCNN latest research progress about the algorithms theory and applications. It describes the basic pulse coupled neural network theory and models firstly, then analyse its features. Next you will see some improved network models and new applications of PCNN raised by researchers in recent years. Finally it pointed out the research and development prospects of PCNN in the future.

In this paper, a speech-to-text translation model has been developed for Malaysian speakers based on 41 classes of Phonemes. A simple data acquisition algorithm has been used to develop a MATLAB graphical user interface (GUI) for recording the isolated word speech signals from 35 non-native Malaysian speakers. The collected database consists of 86 words with 41 classes of phoneme based on Affricatives, Diphthongs, Fricatives, Liquid, Nasals, Semivowels and Glides, Stop and Vowels. The speech samples are preprocessed to eliminate the undesirable artifacts and the fuzzy voice classifier has been employed to classify the samples into voiced sequence and unvoiced sequence. The voiced sequences are divided into frame segments and for each frame, the Linear Predictive co-efficients features are obtained from the voiced sequence. Then the feature sets are formed by deriving the LPC features from all the extracted voiced sequences, and used for classification. The isolated words chosen based on the phonemes are associated with the extracted features to establish classification system input-output mapping. The data are then normalized and randomized to rearrange the values into definite range. The Multilayer Neural Network (MLNN) model has been developed with four combinations of input and hidden activation functions. The neural network models are trained with 60%, 70% and 80% of the total data samples. The neural network architecture was aimed at creating a robust model with 60%, 70%, and 80% of the feature set with 25 trials. The trained network model is validated by simulating the network with the remaining 40%, 30%, and 20% of the set. The reliability of trained network models were compared by measuring true-positive, false-negative, and network classification accuracy. The LPC features show better discrimination and the MLNN neural network models trained using the LPC spectral band features gives better recognition.

Evaluation of neural network modeling to predict non-water-stressed leaf temperature in wine grape for calculation of crop water stress index

Evaluation of insulin sensitivity temporal prediction by using quantile regression combined with neural network model.