Wound Detection by Simple Feedforward Neural Network

This paper proposes an automatic sleep apnea monitoring device for adults employing of single ECG patch and a pulse oximeter. The device is designed to automatically detect sleep apneic (SA) events with the inference of feedforward neural network (FNN) model embedded in digital hardware. The three-layer (8-6-4) FNN model was trained over several epochs with a 5-fold cross validation technique where the training set had a mini-batch size of 10. Open-source Apnea ECG dataset collected from the PhysioNET bank was used in training, validating, and testing the model. Rectified Linear Unit (ReLU) activation function was used in the input and hidden layers of the network and sigmoid function was used as the output classifier. ADAM optimizer was used for optimization of the model while mean-squared-error (MSE) was used for calculating model loss. The final trained and validated model was implemented onto re-programmable digital hardware called Field-Programmable Gate Array (FPGA). The hardware implementation of the model yielded an accuracy of over 87 percent with a power consumption rate of around 52 W, which is 5x times lower than that of commercially available machine learning hardware accelerators. The proposed system design will be realized in integrated circuits on CMOS platform for developing energy-efficient, smart, wearable, and automated sleep apnea detection and screening device for adults.

A novel method based on a physics informed data driven model is developed to design an origami composite crash tube. The structure consists of two axially stacked basic components, called modules. Each module presents lower and upper square sections with an octagonal section in the middle. The parameters of the octagonal cross-section and the height of each module are optimized to maximize the energy absorption of the tube when subjected to an axial impact. In contrast to standard surrogate modelling techniques, whose accuracy only depends on the amount of available data, a Physics-informed Neural Network (PINN) scheme is adopted to correlate the crushing response of the single modules to that of the whole origami tube, constraining the data driven method to physically consistent predictions. The PINN is first trained on the results obtained with an experimentally validated Finite Element model and then used to optimize the structure. Results show that the PINN can accurately predict the crushing response of the origami tube, while consistently reducing the computational effort required to explore the whole design domain. Also, the comparison with a standard Feed Forward Neural Network (FFNN) shows that the PINN scheme leads to more accurate results.

A simple Feed Forward Neural Network (FFNN) model with a learning back-propagation algorithm was applied to forecast rainfall data from 1997-2016 of Juba County, South Sudan. Annual rainfall data were aggregated into three seasons MAMJ, JAS and OND and later trained for best predictions for the period 2017-2034 using the Alyuda Forecaster XL software. Best training was attained once the minimum error or cost function of the weight was attained during gradient descent and expressed as Mean Square Error (MSE) and AE of the input variable. The results showed that for MAMJ and JAS months, the number good forecasts were over 97% whereas this was between 60-80% for OND months. The Seasonal Kendal (SK) test on future rainfall forecasts as well as the Theil-Sen slope showed a declining monotonic trend in the mean amounts in all three seasons with MAMJ, JAS at OND at 100, 150 and 80 mm respectively towards the end of 2034. Declining onset of MAMJ rains is expected to significantly affect the timing for land preparation and crop planting. The forecast accuracy of the FFNN can be used as a vital tool for decision makers in projecting future rainfall events.

The design of the neural network model and its adaptive wavelets (wavelet networks and wavenets) was used to estimate the wave-induced hydrodynamic inline force acting on a vertical cylinder. The data used to calibrate and validate the models were obtained from an experiment. In the brain, wavelet neural networks (WNNs) use wavelets to activate their hidden layers of neurons. In WNNs, both the position and dilation of the wavelets are optimized along with the weights. In one special approach to this kind of network construction, the position and dilation of the wavelets are fixed and only the weights of the network are optimized. In the present study, the neural network procedure and the above mentioned approach were employed to design a WNN, a so-called wavenet, using feed-forward neural network topology and its training method. Then, a comparison of these two methods was made. Numerical results demonstrate that both networks are capable of predicting hydrodynamic inline force. Furthermore, the combination of the neural network concept and the wavelet theory i.e. wavenet provides a more robust tool rather than standard feed-forward neural network, considering its more appropriate ability to predict any other data which the network had not experienced before. The results of this study can contribute to reducing the errors in future efforts to predict hydrodynamic inline force using WNNs, and thus improve the reliability of that prediction in comparison to the ANN and other methods. Therefore, this method can be applied to relevant engineering projects with satisfactory results.

Spectral graph convolutional neural networks (GCN) are proposed to incorporate important information contained in graphs such as gene networks. In a standard spectral GCN, there is only one gene network to describe the relationships among genes. However, for genomic applications, due to condition- or tissue-specific gene function and regulation, multiple gene networks may be available; it is unclear how to apply GCNs to disease classification with multiple networks. Besides, which gene networks may provide more effective prior information for a given learning task is unknown a priori and is not straightforward to discover in many cases. A deep multiple graph convolutional neural network is therefore developed here to meet the challenge. The new approach not only computes a feature of a gene as the weighted average of those of itself and its neighbors through spectral GCNs, but also extracts features from gene-specific expression (or other feature) profiles via a feed-forward neural networks (FNN). We also provide two measures, the importance of a given gene and the relative importance score of each gene network, for the genes' and gene networks' contributions, respectively, to the learning task. To evaluate the new method, we conduct real data analyses using several breast cancer and diffuse large B-cell lymphoma datasets and incorporating multiple gene networks obtained from "GIANT 2.0" Compared with the standard FNN, GCN, and random forest, the new method not only yields high classification accuracy but also prioritizes the most important genes confirmed to be highly associated with cancer, strongly suggesting the usefulness of the new method in incorporating multiple gene networks.

To develop and evaluate a model for obstructive sleep apnea (OSA) detection using an artificial neural network (ANN) based on the combined features of body mass index (BMI), electrocardiogram (ECG), and pulse oxygen saturation (SpO2). Polysomnography (PSG) data for 148 patients withOSA and 33 unaffected individuals were included. A multi-layer feed-forward neural network (FNN) was used based on the features obtained from ECG, SpO2, and BMI. The receiver operating characteristic (ROC) curve and the metrics of accuracy, sensitivity, and specificity were used to evaluate the performance of the overall classification. Some other machine learning methods including linear discriminant, linear Support Vector Machine (SVM), Complex Tree, RUSBoosted Trees, and Logistic Regression were also used to compare their performance with the FNN. The accuracy, sensitivity, and specificity of the proposed multi-layer FNN were 97.8%, 98.6%, and 93.9%, respectively, and the area under the ROC curve was 97.0%. Compared with the other machine learning methods mentioned above, the FNN achieved the highest performance. The satisfactory performance of the proposed FNN model for OSA detection indicated that it is reliable to screen potential patients withOSA using the combined channels of ECG and SpO2 and also taking into account BMI. This strategy might be a viable alternative method for OSA diagnosis.

In order to solve the problem that the traditional long-term high-speed traffic forecasting algorithm is affected by the approximation ability of the function and easy to fall into the local mass value, we wrote a multivariate-based highway traffic forecasting algorithm scaling and convolutional networks. Because the feedforward wavelet neural network algorithm predicts the short-term traffic flow in different areas, it is necessary to examine the ability to predict the difference between different models. From the standard feedforward wavelet neural network algorithm using global optimization capabilities, we improve the wolf pack algorithm, improve the search accuracy of the algorithm, get the best solution of the estimated value of the work according to the search results when completing the research objectives, and get the ability to predict the work of the model. Feedforward neural network algorithm: we develop and obtain the best short-term high-speed traffic forecast values. The results are as follows: after using the author’s algorithm, the processing time increases by 1.5 seconds, but the average percentage of errors decreases by more than 50%, in fact the error and the root mean square error decreased by about 30%, and the smoothing coefficient increased by about 1%. The prediction of the author’s algorithm for short-term high-speed traffic is better than the wavelet neural network prediction algorithm, and the prediction accuracy and stability of the author’s algorithm are higher.

This paper examines the circuit complexity of feedforward neural networks having sigmoid activation function. The starting point is the complexity class NN k defined in [18]. First two additional complexity classes N N Δ k https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781315784076/1ea703af-0c66-41e4-8ff3-9e9c700ac445/content/eq1772.tif/> and N N Δ , e k https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781315784076/1ea703af-0c66-41e4-8ff3-9e9c700ac445/content/eq1773.tif/> having less restrictive conditions (than NN k ) concerning fan-in and accuracy are defined. We then prove several relations among these three classes and well established circuit complexity classes. All the proofs are constructive as being built around binary adders. By relaxing the fan-in condition from logarithmic [18] to (almost) polynomial, the results show that substantially larger classes of sigmoid activation feedforward neural networks can be implemented in polynomial size Boolean circuits. This is done on the expense of a logarithmic factor increase in the number of layers, but having constant fan-in (of 2). The main conclusion is that there are interesting fan-in dependent depth-size tradeoffs when trying to digitally implement sigmoid activation neural networks.

Making Artificial Brains: Components, Topology, and Optimization.

Most log-based anomaly detectors assume logs are stable, though in reality they are often unstable due to software or environmental changes. Anomaly detection on unstable logs (ULAD) is therefore a more realistic, yet under-investigated challenge. Current approaches predominantly employ machine learning (ML) models, which often require extensive labeled data for training. To mitigate data insufficiency, we propose FlexLog , a novel hybrid approach for ULAD that combines ML models — decision tree, k-nearest neighbors, and a feedforward neural network — with a Large Language Model (Mistral) through ensemble learning. FlexLog also incorporates a cache and retrieval-augmented generation (RAG) to further enhance efficiency and effectiveness. To evaluate FlexLog , we configured four datasets for ULAD, namely ADFA-U, LOGEVOL-U, SynHDFS-U, and SYNEVOL-U. FlexLog outperforms all baselines by at least 1.2 percentage points (pp) in F1 score while using much less labeled data (62.87 pp reduction). When trained on the same amount of data as the baselines, FlexLog achieves up to a 13 pp increase in F1 score on ADFA-U across varying training dataset sizes. Additionally, FlexLog maintains inference time under one second per log sequence, making it suitable for most applications, except latency-sensitive systems. Further analysis reveals the positive impact of FlexLog ’s key components: cache, RAG, and ensemble learning.

Recently, Deep learning has made a great deal of success in processing images, audios, and natural languages and so on. The activation function is one of the key factors in Deep learning. In this paper, according to characteristics of biological neurons, an improved Leaky Single-Peaked Triangle Linear Unit (LSPTLU) activation function is presented for the right-hand response unbounded of Rectified Linear Unit (ReLU) and Leaky ReLU (LReLU). LSPTLU is more in line with the biological neuron essence and achieves the excellent performance of equivalent or beyond ReLU and LReLU on different datsets, e.g., MNIST, Fashion-MNIST, SVHN, IMAGENET, CALTECH101 and CIFAR10 datasets.

The efficacy of deep learning models has been called into question by the presence of adversarial examples. Addressing the vulnerability of deep learning models to adversarial examples is crucial for ensuring their continued development and deployment. In this work, we focus on the role of rectified linear unit (ReLU) activation functions in the generation of adversarial examples. ReLU functions are commonly used in deep learning models because they facilitate the training process. However, our empirical analysis demonstrates that ReLU functions are not robust against adversarial examples. We propose a modified version of the ReLU function, which improves robustness against adversarial examples. Our results are supported by an experiment, which confirms the effectiveness of our proposed modification. Additionally, we demonstrate that applying adversarial training to our customized model further enhances its robustness compared to a general model.

Focusing on the problems of complex structure and low feature extraction efficiency that exist in some traditional neural network algorithms, an improved convolutional neural network, combined with a convolutional attention mechanism, is proposed to im- prove the network feature extraction efficiency and reduce the network complexity. First, the convolutional block attention module (CBAM) was introduced, which can strengthen the extraction of useful features from the data, thereby improving the network feature extraction efficiency. Second, the residual structure was optimized, and the original multi-layer convolution was simplified to a single-layer convolution, which reduces the complexity of the model and improves the diagnostic efficiency. Then, the Rectified Linear Unit (ReLU) activation function in the model was replaced with the better-performing Exponential Linear Unit (ELU) activation function. Finally, the overall structure of the ResNet model was optimized. The CBAM-ResNet network model was based on the traditional ResNet18 model and consists of a CBAM module and the optimized ResNet18 model. Finally, experiments were performed using sensor data collected from rolling bearing test benches. The experimental results show that the feature extraction capability and efficiency of the new model are better than those of several other classic deep network models and the original network model, and this reduces the time complexity and model size while also maintaining a high accuracy rate.

The scale of deer breeding has gradually increased in recent years and better information management is necessary, which requires the identification of individual deer. In this paper, a deer face dataset is produced using face images obtained from different angles, and an improved residual neural network (ResNet)-based recognition model is proposed to extract the features of deer faces, which have high similarity. The model is based on ResNet-50, which reduces the depth of the model, and the network depth is only 29 layers; the model connects Squeeze-and-Excitation (SE) modules at each of the four layers where the channel changes to improve the quality of features by compressing the feature information extracted through the entire layer. A maximum pooling layer is used in the ResBlock shortcut connection to reduce the information loss caused by messages passing through the ResBlock. The Rectified Linear Unit (ReLU) activation function in the network is replaced by the Exponential Linear Unit (ELU) activation function to reduce information loss during forward propagation of the network. The preprocessed 6864 sika deer face dataset was used to train the recognition model based on SE-Resnet, which is demonstrated to identify individuals accurately. By setting up comparative experiments under different structures, the model reduces the amount of parameters, ensures the accuracy of the model, and improves the calculation speed of the model. Using the improved method in this paper to compare with the classical model and facial recognition models of different animals, the results show that the recognition effect of this research method is the best, with an average recognition accuracy of 97.48%. The sika deer face recognition model proposed in this study is effective. The results contribute to the practical application of animal facial recognition technology in the breeding of sika deer and other animals with few distinct facial features.

Transformer architectures have achieved state-of-the-art performance in various applications. However, deploying transformer models on resource-constrained platforms is still challenging due to its dynamic workloads, intensive computations, and substantial memory access. In this article, we propose a two-stage pipelined compute-in-memory (CIM) macro for effectively deploying and accelerating the feed-forward network (FFN) layers of transformer models. Two independent CIM arrays are designed to execute the two distinct linear projections in FFN layers, which are interconnected by co-designed analog rectified linear unit (ReLU) circuits to realize the nonlinear activation function. The analog multiply-and-add (MAC) results from the first CIM array are streamed directly to the analog ReLU circuits, and subsequently to the next CIM array for performing another linear projection. This architecture eliminates the need for analog-to-digital converters (ADCs) and digital-to-analog converters (DACs) for internal results’ staging, thereby enhancing overall macro efficiency and reducing computing latency. A proof-of-concept macro is fabricated using TSMC 65-nm process and achieves 4.096 TOPS peak throughput, 4.39 TOPS/mm2 area efficiency, and 49.83 TOPS/W energy efficiency. To map transformer models onto the proposed macro, we quantize the FFN layers of BERTMINI model under per-token granularity for activations and per-tensor granularity for weights using quantization-aware training (QAT), which exhibits excellent accuracy across multiple benchmarks.