Implementation of Convolutional Neural Network on Farming Robots for Detecting Broccoli

Over the last years, Convolutional Neural Networks (CNNs) have been widely used in remote sensing applications, such as marine surveillance, traffic management or road networks detection. However, since CNNs have extremely high computation, bandwith and memory requirements, the hardware implementation of a CNN on space-grade devices like FPGAs for the on-board processing of the acquired images has brought many challenges. In fact, FPGAs show great advantages in terms of reconfigurability and performance-power ratio, whichmakes them a suitable election to deploy CNNs, although the computational capabilities of this type of hardware devices available on-board is limited, so implementations have to be carefully planned. In this paper, the authors present their work towards the implementation of an efficient CNN onto a spacegrade FPGA in order to achieve the on-board processing of very-high resolution remotely sensed images as soon as the data are provided by the sensor. All this work has been conducted within the EU-funded VIDEO project. As it will be presented in this paper, the work includes the introduction of a methodology based on the project constraints, the evaluation of different state-of-the-art CNN architectures by means of a new efficiency measurement also proposed in this work, the introduction of a new efficient CNN architecture, and finally, its optimized hardware implementation by means of high-level synthesis tools.

The hardware implementation of convolutional neural network has the problem of resource limitation, which can be solved by the design of convolutional neural network accelerator based on FPGA dynamic reconstruction. The whole parallel strategy and architecture of CNN accelerator are designed, and the functional modules of CNN are designed based on pipeline. The dynamic reconstruction technology is used to redesign the convolutional neural network accelerator, and the dynamic reconstruction region and division are established; BPI flash is selected to store the configuration file, and the configuration file is read internally to dynamically configure the dynamic reconstruction area. Finally, for lenet. 5 handwriting recognition, compared with the corresponding static design, the use of slice LUTS, slice registers and DSP resources of the accelerator based on dynamic reconstruction design is reduced by 46%, 25% and 68% respectively. Compared with the implementation based on software platform. The system execution time is greatly reduced. However, due to the bandwidth limitation of the internal configuration port, the reconfiguration configuration time prolongs the execution time of the whole convolutional network.

Automatic target recognition (ATR) is a challenging task for several computer vision applications. It requires efficient, accurate, and robust methods for target detection and target identification. Deep learning has shown great success in many computer vision applications involving color RGB images. However, the performance of these networks in ATR with infrared sensor data needs further investigation. In this paper, we propose a multistage automatic target detection and recognition (ATDR) system that performs both target detection and target classification on infrared (IR) imagery using deep learning. Our system processes large IR image frames where targets take <1 % of the total number of pixels. First, we train a state-of-the-art object detector you only look once (YOLO) to localize all potential targets in the input image frame. Then, we train a convolutional neural network (CNN) to identify these detections as targets or false alarms. In this second phase, we adapt and analyze the performance of three CNN architectures: a compact and fully connected CNN, VGG16 with batch normalization, and a wide residual neural network (WRN). We also explore the use of a loss function that optimizes directly the area under the receiver operating characteristic (ROC) curve (AUC), and adapt it to our ATR application. To enhance the robustness of the proposed ATR to perturbation and variations introduced during the detection stage, we train our CNN classifiers on automatically detected targets using YOLO, in addition to ground truth bounding boxes and apply selected data augmentation techniques. To simulate real testing environments, where the spatial location of the targets within the image frame is unknown, only YOLO-detected boxes are used during validation. We evaluate our ATDR on a real benchmark dataset that includes different vehicles captured at different resolutions. Our experiments have shown that YOLO can detect most of the targets at the expense of generating a high number of false alarms. We show that the VGG-16 network with batch normalization, which is the best performing model, can correctly identify the classes of the targets, as well as classify the majority of YOLO’s false detections into an additional nontarget class. We also show that the proposed training modification to optimize an AUC-based loss function for ATR proved to be advantageous mainly in identifying difficult targets.

The use of plastic bottles for various needs is increasingly massive, especially in consumption needs such as mineral water bottles. The use of plastic bottles is used to reduce costs and be effective in maintaining the quality of mineral water, but its impact can affect natural conditions if not managed properly. Plastic bottle waste if left buried in the ground will have difficulty expanding, which can cause environmental pollution. Therefore, we can take advantage of technology to sort plastic bottle waste using a camera based on the size of plastic bottles. Differentiating the size of bottles aims to distinguish the economic value when exchanged at the waste bank. This technology utilizes object detection and recognition functions such as the YOLO (You Only Look Once) method. YOLO is a detection method that is a development of the CNN (Convolutional Neural Network) algorithm. By using YOLOv5, we can detect objects in the form of plastic bottle waste of various different sizes. To maximize object detection according to size, data annotation is done by creating a Bounding Box on each dataset according to its size. The test was carried out with several different distance configurations including 40cm, 80cm and 1m. Detection results using YOLOv5 produce up to 84% accuracy in real-time.

This research develops a system for recognizing finger spelling gestures in Indonesian Sign Language (BISINDO) using Convolutional Neural Network (CNN). The objective of this research is to apply the Convolutional Neural Network (CNN) method to the BISINDO finger spelling gesture recognition system to improve its accuracy. The method employed is Convolutional Neural Network (CNN), an effective method for processing image data for pattern recognition. Based on the test results, the system demonstrates that the developed CNN model is capable of recognizing BISINDO finger spelling gestures with an accuracy of 97.5%. This indicates that the BISINDO finger spelling gesture recognition system performs well in pattern recognition. The implementation of the system for real-time prediction via a web interface using Flask also enhances its accessibility. However, there is still room for improvement, particularly in recognizing one of the 26 letters that has not been predicted accurately. For further development, it is recommended to consider collecting a larger dataset and incorporating more complex gesture variations to improve recognition accuracy.

Convolutional Neural Network (CNN) is a deep learning algorithm that is widely used to identify and classify a digital image object. In this study the Convolutional Neural Network (CNN) is used as an algorithm that functions to identify leaf types (certain plants) based on images obtained from a public dataset provider named Daun Jamu Indonesia. The existence of image characteristics causes the assistance process to require a more detailed feature selection process. Therefore the CNN method is used in order to solve the problem. The Convolutional Neural Network (CNN) method is capable of performing image recognition by minimizing feature extraction. CNN is also reliable in processing unstructured data because it uses a multi-layered structure of artificial reasoning networks. The image recognition process is carried out by looking for the shape of the model that matches the processed data in order to get the best results. In this study, the augmentation process was carried out on the training data and validation data so that overfitting does not occur in the Convolutional Neural Network (CNN). The results obtained in this study indicate that the Convolutional Neural Network (CNN) method can identify leaf types with a measured accuracy rate of 92% using the Confusion Matrix evaluation method. It is hoped that this research can be used as a reference for the use of the Convolutional Neural Network (CNN) method for image data, especially plant leaf types.

Convolutional neural networks (CNNs) have demonstrated remarkable performance in many areas but require significant computation and storage resources. Quantization is an effective method to reduce CNN complexity and implementation. The main research objective is to develop a scalable quantization algorithm for CNN hardware design and model the performance metrics for the purpose of CNN implementation in resource-constrained devices (RCDs) and optimizing layers in deep neural networks (DNNs). The algorithm novelty is based on blending two quantization techniques to perform full model quantization with optimum accuracy, and without additional neurons. The algorithm is applied to a selected CNN model and implemented on an FPGA. Implementing CNN using broad data is not possible due to capacity issues. With the proposed quantization algorithm, we succeeded in implementing the model on the FPGA using 16-, 12-, and 8-bit quantization. Compared to the 16-bit design, the 8-bit design offers a 44% decrease in resource utilization, and achieves power and energy reductions of 41% and 42%, respectively. Models show that trading off one quantization bit yields savings of approximately 5.4K LUTs, 4% logic utilization, 46.9 mW power, and 147 μJ energy. The models were also used to estimate performance metrics for a sample DNN design.

Technological advancements have had a significant impact on the education sector, including the application of Convolutional Neural Networks (CNN) for facial image analysis. This research aims to implement CNN to measure students' learning concentration levels. The FER2013 dataset, which includes seven emotion classifications and comprises 28,709 images for training data, is used as the database. The data is processed through rescaling and augmentation to prepare the CNN model. The model consists of several convolutional layers, pooling layers, and fully connected layers designed to extract crucial features from facial images. Evaluation results demonstrate a very high accuracy of 94.95% on training data, indicating that the model effectively recognizes complex patterns within the data. Although there is a higher loss value of 157% and a decreased accuracy of 62.75% on validation data, this suggests that the model possesses a strong foundational capability and can still be improved through further adjustments. With high accuracy in training and promising validation results, the model shows substantial potential for real-world application, where it can assist teachers in understanding students' emotional responses in real-time. The implementation of CNN aids educators in comprehending students' emotional responses and adapting their teaching methods more effectively, thereby creating a more conducive learning environment and enhancing students' academic and social development. These findings also open opportunities for further research to improve the performance and generalization of the model on unseen data, making this technology an increasingly reliable tool in education

and AI-assisted image manipulation tools, which The emergence of advanced image manipulation techniques in the digital age has made it extremely difficult to detect and categorize forged images. In order to identify and classify image forgeries, this study proposes a deep learning-based method that combines a Convolutional Neural Network (CNN) classifier with the You Only Look Once (YOLO) object detection model. The suggested system uses sophisticated feature extraction techniques to categorize images into authentic, copy-move, spliced, and deepfake forgeries after being trained on a custom dataset of manipulated images. While CNN-based classification offers accurate categorization, our experimental results show how reliable and effective YOLO is at locating forgery regions. Because of its high accuracy, the suggested model is a useful tool for security and digital forensics applications. Keywords: Image Forgery, YOLO, CNN, Deep Learning, Classification, Digital Forensics

YOLO Based Breast Masses Detection and Classification in Full-Field Digital Mammograms

Randomly switching neurons ON/OFF while training and inference process is an interesting characteristic of biological neural networks, that potentially results in inherent adaptability and creativity expressed by human mind. Dropouts inspire from this random switching behaviour and in the artificial neural network they are used as a regularization techniques to reduce the impact of over-fitting during the training. The energy-efficient digital implementations of convolutional neural networks (CNN) have been on the rise for edge computing IoT applications. Pruning larger networks and optimization for performance accuracy has been the main direction of work in this field. As opposed to this approach, we propose to build a near-sensor analogue CNN with high-density memristor crossbar arrays. Since several active elements such as amplifiers are used in analogue designs, energy efficiency becomes a main challenge. To address this, we extend the idea of using dropouts in training to also the inference stage. The CNN implementations require a subsampling layer, which is implemented as a mean pooling layer in the design to ensure lower energy consumption. Along with the dropouts, we also investigate the effect of non-idealities of memristor and that of the network.

Background: Memristor-based implementations of Convolutional Neural Networks (CNNs) offer the potential to reduce power consumption and improve computational efficiency significantly. However, the non-ideal characteristics of memristive devices, such as variability and limited switching behavior, can introduce nonlinearity and degrade network performance. It is therefore essential to simulate memristor-based deep learning systems and evaluate the impact of these imperfections before proceeding with hardware-level deployment. Objective: This study aims to design a lightweight memristor-based MobileNetV2 model for smoking behavior detection and analyze the influence of memristor non-idealities on the network’s detection accuracy. Methods: A lightweight MobileNetV2 network was constructed and trained on a self-developed dataset for smoking behavior detection, achieving a peak accuracy of 92.25%. Based on the trained model, a memristor-based MobileNetV2 architecture was implemented using the Mem- Torch simulation framework. Two memristor models, VTEAM and Data-Driven Verilog-A RRAM, were employed to replace the convolutional and fully connected layers with memristor crossbar arrays. The impact of non-ideal memristor properties, including conductance variability and resistance state faults, was systematically evaluated. Results: Simulation results demonstrated that the memristor-based MobileNetV2 network achieved detection accuracies of 92.11% with the VTEAM model and 91.82% with the Verilog-A RRAM model. As device non-idealities increased, the detection accuracy declined significantly, indicating a strong correlation between memristor imperfections and the performance of CNN- based target detection tasks. Conclusion: Memristor non-idealities substantially affect the performance of neuromorphic CNNs. This study provides targeted analyses and proposes mitigation strategies, such as differential encoding and improved fabrication techniques, to support the practical deployment of memristor- based neural networks in energy-efficient AI applications.

Effective disease control in crops is essential for ensuring food security, as numerous diseases can significantly reduce agricultural productivity. Leaf disease is a crucial determinant of crop yield and quality. Potatoes are widely regarded as a vital staple in diets around the world. Potato blight, often known as late blight, is a severe disease impacting Solanaceae plants, particularly the potato. The oomycete Phytophthora infestans serves as the causal agent, posing a significant threat to potato crops by reducing yields and leading to financial losses. Identifying potato diseases is essential for ensuring food security and minimizing economic losses in agriculture. Numerous studies have explored the applications of machine learning and deep learning technologies for detecting blight diseases in affected crops. The proposed methodology outlined in our study offers a promising and effective means to enhance the accuracy of potato late blight classification. Advancements in technology can significantly improve the identification and treatment of diseases. Deep learning algorithms have been effectively employed in a range of practical applications. In our investigation, convolutional neural network models are used to analyze images of leaves. The primary aim of this initiative is to gain insights into and enhance the capacity to recognize diseases in their early phases. After the input images undergo preprocessing, the target region is distinctly isolated. CNN examines images to identify unique features, including edges, colours, and textures. This study evaluates the effectiveness of the proposed model through a diverse dataset of potato leaf images, making essential contributions to the domain of automated disease detection systems. The results of this study could enhance techniques for agricultural management and strengthen the sustainability of agriculture. This study introduces an improved CNN model designed to identify the blight disease state in potato leaf images, providing a valuable tool for farmers. The findings may aid in the evaluation and implementation of Convolutional Neural Network (CNN) models for classifying proximal sensing images of potato plant leaves. Additional investigations could leverage this modelling framework and have the opportunity to examine various other factors to identify leaf infections at an earlier stage for crop protection.

Deep convolutional neural network (DCNN) is an advanced technology in image recognition. Because of extreme computing resource requirements, DCNN implementation with software alone cannot achieve real-time requirement. Therefore, the need to implement DCNN accelerator hardware is increasing. In this paper, we present a field programmable gate array (FPGA)-based hardware accelerator design of DCNN targeting handwritten Hangul character recognition application. Also, we present design optimization techniques in SDAccel environments for searching the optimal FPGA design space. The techniques we used include memory access optimization and computing unit parallelism, and data conversion. We achieved about 11.19 ms recognition time per character with Xilinx FPGA accelerator. Our design optimization was performed with Xilinx HLS and SDAccel environment targeting Kintex XCKU115 FPGA from Xilinx. Our design outperforms CPU in terms of energy efficiency (the number of samples per unit energy) by 5.88 times, and GPGPU in terms of energy efficiency by 5 times. We expect the research results will be an alternative to GPGPU solution for real-time applications, especially in data centers or server farms where energy consumption is a critical problem.

An operator feels a burden when he/she controls a rescue robot remotely because he/she has to keep watching camera images to find the target object. We think that this burden can be reduced by the combination of Head Mounted Display (HMD) and object recognition by deep learning. In the first half part of this study, we examine the effect that how presentation method by You Only Look Once (YOLO), a deep learning algorithm, and its recognition results to an operator wearing HMD. In the experiment, three methods of presentation were set: no display of object recognition, display only one object recognition result, and display 80 kinds of object recognition results. Under each presentation method, we measured the time it took for the operator to operate the robot and complete the given task. Additionally, we ask a questionnaire for each experiment. The results of the questionnaire showed that the method to present only one object recognition result was useful. In the second half part of this study, we develop a system to present 3D images with YOLO added, to further ease the burden of object search. Furthermore, we numerically prove that this system represents depth. In the experiment, two methods of displaying were set up: 2D images with Bounding Box (BB) by YOLO and 3D images with BB by YOLO. For each method of presentation, the operator operated the robot and recorded the number of objects found within a time limit. Additionally, we asked a questionnaire at the end of the search in each condition and at the end of all the experiments. The results of the questionnaire suggested points that need to be improved. Furthermore, we consider the flicker of the image found in the experiment.