All binarized convolutional neural network and its implementation on an FPGA

In order to overcome the effects of posture, illumination, expression and other factors on face recognition, this paper proposes an algorithm which is based on local binary pattern (LBP) and convolutional neural network (CNN). LBP is a texture description method which describes the local texture features of an image. It has good robustness of illumination and posture. CNN can effectively extract the spatial features of images and reduce the dimensions of features. This paper combines the advantages of LBP and CNN to improve the accuracy of face recognition. The CNN in this algorithm has four convolution layers, two max-pooling layers, one activation layer, one fully connected layer, and one output layer. In order to optimize the network structure, batch normalization layer is added after the convolution layer. We get the local binary pattern coded images and put the images as the input of the CNN and train the network. Hence, we can use the well-trained CNN for classification and identification. The Experiments on the CMU-PIE face database show that our algorithm can effectively improve the rate of face recognition.

Deep Convolutional Neural Networks (CNNs) are widely employed in modern AI systems due to their unprecedented accuracy in object recognition and detection. However, it has been proven that the main bottleneck to improve large scale deep CNN based hardware implementation performance is massive data communication between processing units and off-chip memory. In this paper, we pave a way towards novel concept of in-memory convolver (IMC) that could implement the dominant convolution computation within main memory based on our proposed Spin Orbit Torque Magnetic Random Access Memory (SOT-MRAM) array architecture to greatly reduce data communication and thus accelerate Binary CNN (BCNN). The proposed architecture could simultaneously work as non-volatile memory and a reconfigurable in-memory logic (AND, OR) without add-on logic circuits to memory chip as in conventional logic-in-memory designs. The computed logic output could be also simply read out like a normal MRAM bit-cell using the shared memory peripheral circuits. We employ such intrinsic in-memory processing architecture to efficiently process data within memory to greatly reduce power-hungry and long distance data communication concerning state-of-the-art BCNN hardware. The hardware mapping results show that IMC can process the Binarized AlexNet on ImageNet data-set favorably with 134.27 μJ/img where ∼ 16x and 9x lower energy and area are achieved, respectively, compared to RRAM-based BCNN. Furthermore, 21.5% reduction in data movement in term of main memory accesses is observed compared to CPU/DRAM baseline.

The proposed deep learning algorithm will be integrated as a binary classifier under the umbrella of a multi-class classification tool to facilitate the automated detection of non-healthy deformities, anatomical landmarks, pathological findings, other anomalies and normal cases, by examining medical endoscopic images of GI tract. Each binary classifier is trained to detect one specific non-healthy condition. The algorithm analyzed in the present work expands the ability of detection of this tool by classifying GI tract image snapshots into two classes, depicting haemorrhage and non-haemorrhage state. The proposed algorithm is the result of the collaboration between interdisciplinary specialists on AI and Data Analysis, Computer Vision, Gastroenterologists of four University Gastroenterology Departments of Greek Medical Schools. The data used are 195 videos (177 from non-healthy cases and 18 from healthy cases) videos captured from the PillCam(R) Medronics device, originated from 195 patients, all diagnosed with different forms of angioectasia, haemorrhages and other diseases from different sites of the gastrointestinal (GI), mainly including difficult cases of diagnosis. Our AI algorithm is based on convolutional neural network (CNN) trained on annotated images at image level, using a semantic tag indicating whether the image contains angioectasia and haemorrhage traces or not. At least 22 CNN architectures were created and evaluated some of which pre-trained applying transfer learning on ImageNet data. All the CNN variations were introduced, trained to a prevalence dataset of 50%, and evaluated of unseen data. On test data, the best results were obtained from our CNN architectures which do not utilize backbone of transfer learning. Across a balanced dataset from no-healthy images and healthy images from 39 videos from different patients, identified correct diagnosis with sensitivity 90%, specificity 92%, precision 91.8%, FPR 8%, FNR 10%. Besides, we compared the performance of our best CNN algorithm versus our same goal algorithm based on HSV colorimetric lesions features extracted of pixel-level annotations, both algorithms trained and tested on the same data. It is evaluated that the CNN trained on image level annotated images, is 9% less sensitive, achieves 2.6% less precision, 1.2% less FPR, and 7% less FNR, than that based on HSV filters, extracted from on pixel-level annotated training data.

A pre-trained convolutional deep neural network (CNN) is widely used for embedded systems, which requires highly power-and-area efficiency. In that case, the CPU is too slow, the embedded GPU dissipates much power, and the ASIC cannot keep up with the rapidly progress of the CNN variations. This paper uses a binarized CNN which treats only binary 2-values for the inputs and the weights. Since the multiplier is replaced into an XNOR circuit, we can realize a high-performance MAC circuit by using many XNOR circuits. In the paper, we eliminate internal FC layers excluding the last one, then, insert a binarized average pooling layer, which can be realized by a majority circuit for binarized (1/0) values. In that case, since the weight memory is replaced into the 1's counter, we can realize a compact and faster CNN than the conventional ones. We implemented the VGG-11 benchmark CNN for the CIFAR10 image classification task on the Xilinx Inc. Zedboard. Compared with the conventional binarized implementations on an FPGA, the classification accuracy was almost the same, the performance per power efficiency is 5.1 better, as for the performance per area efficiency, it is 8.0 times better, and as for the performance per memory, it is 8.2 times better.

Although deep learning algrithms have outstanding performance in biometrics and been paid more and more attention, triditional features for should not be ignored. In this paper, fusion of partition local binary patterns (PLBP) and convolutional neural networks (CNNs) is investigated in three schemes. In serial fusion (SF) method, PLBP feature is extracted and reshaped as the input of CNNS. Decision fusion (DF) carries out the PLBP with nearest neighour classifer and CNNs seperatelly and weighted fuses the results. For feature fusion (FF), PLBP feature is reshaped and weighted fused with the feature map of CNNs. To examine the proposed methods, NCUT data set with 2040 images from 204 hands is augmented using PCA. The results indicate that when the PLBP and CNNs are merged in FF scheme with weights of 0.2 and 0.8, our fusion method reaches a state-of-the-art recognition rate of 99.95%.

Artificial intelligence for the assessment of bowel preparation

A pre-trained convolutional deep neural network (CNN) is a feed-forward computation perspective, which is widely used for the embedded systems, requires highly power and area efficiency. Also, with a rapid growth of the CNN techniques, a short time-implementation is a mandatory. In this talk, I introduce a binarized CNN on an FPGA which treats only binary 2-values (+1/−1) for the inputs and the weights. In this case, the multiplier is replaced into an XNOR circuit instead of a dedicated DSP block. For hardware implementation, using binarized inputs and weights is more suitable. Additionally, I show the optimization techniques for the binarized one for the FPGA implementation. Next, I propose a framework for an FPGA realization of the binarized CNN by using a high-level synthesis. Our developed framework supports a modern CNN frameworks, such as a Caffe, Theano, TensorFlow, and Chainer. Thus, we can implement the high-performance CNN on the FPGA with a short time. Finally, I show the comparison of the conventional implementations, then, introduce the applications using the binarized CNN on the FPGA.

Here we investigate the application of convolutional neural networks (CNNs) to predict the survival time of patients with Glioblastoma Multiforme (GBM) brain tumor. Our dataset consists of T1-weighted high-resolution MRI images of just 68 GBM patients. We compare two analytic methods for predicting survival time. The first consists of training a small convolutional neural network (CNN) and the second uses extracted deep features from a pre-trained CNN. Our method is completely automated, except for tumor region segmentation. In addition, we utilize a snapshot ensemble approach to boost test accuracy when dealing with limited availability of medical images for CNN training purposes. Our approach achieves an accuracy of 72.06% using a trained small network and 66.18% using a pre-trained deep CNN. Our results compare favorably with the accuracy of 54.41% using histogram of oriented gradients (HOG) features and a non-neural network classifier.

Sleep arousal is defined as a shift from deep sleep to light sleep or complete awakening. Arousals cause sleep deprivation by fragmenting sleep, and ultimately, many health problems. Arousals can be induced by well-studied apneas and hypopneas or other sleep orders such as hypoventilation, bruxism, respiratory effort-related arousals. Thus, detection of less-studied non-apnea/hypopnea arousals is important for diagnosis and treatment of sleep disorders. Traditionally, polysomnography (PSG) test that is recording and inspecting overnight physiological signals is used for sleep studies. In this work, a novel method based on one dimensional local binary pattern (1D-LBP) and convolutional neural network (CNN) for automatic arousal detection from polysomnography recordings is proposed. 25 recordings from PhysioNet Challenge 2018 PSG dataset are used for experiments. Each signal in PSG recordings is transformed to a new signal using 1D-LBP, and then segmented using 10-s-long sliding window. The segments are fed to a CNN model formed by stacking 25 layers for classification of non-apnea/hypopnea arousal regions from non-arousal regions. Area under precision-recall curve (AUPRC) and area under receiver operating characteristic curve (AUROC) metrics are used for performance measurement. Experimental results reflect that the proposed method shows a great promise and obtains an AUPRC of 0.934 and an AUROC of 0.866.

Recently, as technology has advanced, the use of in-vehicle infotainment systems has increased, providing many functions. However, if the driver’s attention is diverted to control these systems, it can cause a fatal accident, and thus human–vehicle interaction is becoming more important. Therefore, in this paper, we propose a human–vehicle interaction system to reduce driver distraction during driving. We used voice and continuous-wave radar sensors that require low complexity for application to vehicle environments as resource-constrained platforms. The proposed system applies sensor fusion techniques to improve the limit of single-sensor monitoring. In addition, we used a binarized convolutional neural network algorithm, which significantly reduces the computational workload of the convolutional neural network in command classification. As a result of performance evaluation in noisy and cluttered environments, the proposed system showed a recognition accuracy of 96.4%, an improvement of 7.6% compared to a single voice sensor-based system, and 9.0% compared to a single radar sensor-based system.

Early and accurate identification of brain tumors is essential for determining effective treatment strategies and improving patient outcomes. Artificial intelligence (AI) and deep learning (DL) techniques have shown promise in automating diagnostic tasks based on magnetic resonance imaging (MRI). This study evaluates the performance of four pre-trained deep convolutional neural network (CNN) architectures for the automatic multi-class classification of brain tumors into four categories: Glioma, Meningioma, Pituitary, and No Tumor. The proposed approach utilizes the publicly accessible Brain Tumor MRI Msoud dataset, consisting of 7023 images, with 5712 provided for training and 1311 for testing. To assess the impact of data availability, subsets containing 25%, 50%, 75%, and 100% of the training data were used. A stratified five-fold cross-validation technique was applied. The CNN architectures evaluated include DeiT3_base_patch16_224, Xception41, Inception_v4, and Swin_Tiny_Patch4_Window7_224, all fine-tuned using transfer learning. The training pipeline incorporated advanced preprocessing and image data augmentation techniques to enhance robustness and mitigate overfitting. Among the models tested, Swin_Tiny_Patch4_Window7_224 achieved the highest classification Accuracy of 99.24% on the test set using 75% of the training data. This model demonstrated superior generalization across all tumor classes and effectively addressed class imbalance issues. Furthermore, we deployed and benchmarked the best-performing DL model on embedded AI platforms (Jetson AGX Xavier and Orin Nano), demonstrating their capability for real-time inference and highlighting their feasibility for edge-based clinical deployment. The results highlight the strong potential of pre-trained deep CNN and transformer-based architectures in medical image analysis. The proposed approach provides a scalable and energy-efficient solution for automated brain tumor diagnosis, facilitating the integration of AI into clinical workflows.

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Recently Pretrained Convolutional Neural Networks (CNNs) have proven its effectiveness in image extraction and classification. This powerful feature of CNNs in image processing is facilitated by machine learning to train and classify big data. Image capturing and security transformation are considered as a central necessity of remote sensing imagery of unmanned aerial vehicles (UAVs) and drones. This paper presents a novel artificial intelligent drone-based encrypted machine learning of image classification using a pertained CNN and image encryption-decryption by utilizing singular value decomposition (SVD) and XOR-Secret-Key block cipher cryptology. Initially, pretrained convolutional neural networks (CNN) are extensively used to extract and classify image features making advantage of machine learning training tools features. Training of partial set of image data can be performed to test, classify and label the untrained image data. Pretrained CNN can classify images into object categories. Afterward, the CNN the classified image output is transformed into a digital matrix using SVD and identifies its associated eigenvalues. These eigenvalues are then converted into a binary code. The image data encryption is implemented according to suggested keys. The first part applies the exclusive OR (XOR) operation of the eigenvalues with a selected cipher key. Meanwhile, the second part implements the XOR operation of the output of part one with a randomly generated key using Poisson distribution. The last step in the encryption will be obtained by generating a non-real SVD decomposition matrix; according to which a non-readable image will be resulted. The original image-matrix can be constructed by reversing the process using the security key-cipher block (Poisson Distribution Key and Stand-alone Cipher Code). Finally, SVD image processing results are demonstrated to verify the effectiveness and security of the applied approach that can be implemented for different images.

Signature representations that are extracted by convolutional neural networks (CNN) can achieve low error rates. However, a trade-off exists between such models’ complexities and hand-crafted features’ slightly higher error rates. A novel writer-dependent (WD) recurrent binary pattern (RBP) network, and a novel signer identification CNN is proposed. RBP network is a recurrent neural network (RNN) to learn the sequential relation between binary pattern histograms over image windows. A novel histogram selection method is introduced to remove the stop-word codes. Dimensionality is reduced by more than 25% while improving the results. This work is the first to combine binary patterns and RNNs for static signature verification. Several test sets, derived from large-scale and popular databases (GPDS-960 and GPDS-Synthetic-10000) are used. Without training any global classifier, RBP network provides competitive equal error rates (EER). The proposed architectures are compared and integrated with other recent CNN models. Score-level integration of WD classifiers trained with different representations are investigated. Cross-validation tests demonstrate the EERs reduced compared to the best single classifier. A state-of-the-art EER of 1.11% is reported with a global decision threshold (0.57% EER with user-based thresholds) on GPDS-160 database.

Convolutional neural network (CNN) is the primary technique that has greatly promoted the development of computer vision technologies. However, there is little research on how to allocate parameters in different convolution layers when designing CNNs. We research mainly on revealing the relationship between CNN parameter distribution, i.e., the allocation of parameters in convolution layers, and the discriminative performance of CNN. Unlike previous works, we do not append more elements into the network, such as more convolution layers or denser short connections. We focus on enhancing the discriminative performance of CNN through varying its parameter distribution under strict size constraint. We propose an energy function to represent the CNN parameter distribution, which establishes the connection between the allocation of parameters and the discriminative performance of CNN. Extensive experiments with shallow CNNs on three public image classification data sets demonstrate that the CNN parameter distribution with a higher energy value will promote the model to obtain better performance. According to the motivated observation, the problem of finding the optimal parameter distribution can be transformed into an optimization problem of finding the biggest energy value. We present a simple yet effective guideline that uses balanced parameter distribution to design CNNs. Extensive experiments on ImageNet with three popular backbones, i.e., AlexNet, ResNet34, and ResNet101, demonstrate that the proposed guideline can make consistent improvements upon different baselines under strict size constraint.