Phenology-based learning framework for yield estimation and harvest forecasting of raspberry fruits

Sea turtles exhibit high migratory rates and occupy a broad range of habitats, which in turn makes monitoring these taxa challenging. Applying deep learning (DL) models to vast image datasets collected from citizen science programs can offer promising solutions to overcome the challenge of monitoring the wide habitats of wildlife, particularly sea turtles. Among DL models, object detection models, such as the You Only Look Once (YOLO) series, have been extensively employed for wildlife classification. Despite their successful application in this domain, detecting objects in images with complex backgrounds, including underwater environments, remains a significant challenge. Recently, instance segmentation models have been developed to address this issue by providing more accurate classification of complex images compared to traditional object detection models. This study compared the performance of two state-of-the-art DL methods namely; the object detection model (YOLOv5) and instance segmentation model (YOLOv5-seg), to detect and classify sea turtles. The images were collected from iNaturalist and Google and then divided into 64% for training, 16% for validation, and 20% for test sets. Model performance during and after finishing training was evaluated by loss functions and various indexes, respectively. Based on loss functions, YOLOv5-seg demonstrated a lower error rate in detecting rather than classifying sea turtles than the YOLOv5. According to mean Average Precision (mAP) values, which reflect precision and recall, the YOLOv5-seg model showed superior performance than YOLOv5. The mAP0.5 and mAP0.5:0.95 for the YOLOv5 model were 0.885 and 0.795, respectively, whereas for the YOLOv5-seg, these values were 0.918 and 0.831, respectively. In particular, based on the loss functions and classification results, the YOLOv5-seg showed improved performance for detecting rather than classifying sea turtles compared to the YOLOv5. The results of this study may help improve sea turtle monitoring in the future.

Deep Learning models are automating many daily routine tasks, indicating that in the future, even high-risk tasks will be automated, such as healthcare and automated driving areas. However, due to the complexity of such deep learning models, it is challenging to understand their reasoning. Furthermore, the black box nature of the designed deep learning models may undermine public confidence in critical areas. Current efforts on intrinsically interpretable models focus only on classification tasks, leaving a gap in models for object detection. Therefore, this paper proposes a deep learning model that is intrinsically explainable for the object detection task. The chosen design for such a model is a combination of the well-known Faster-RCNN model with the ProtoPNet model. For the Explainable AI experiments, the chosen performance metric was the similarity score from the ProtoPNet model. Our experiments show that this combination leads to a deep learning model that is able to explain its classifications, with similarity scores, using a visual “bag of words”, which are called prototypes, that are learned during the training process. Furthermore, the adoption of such an explainable method does not seem to hinder the performance of the proposed model, which achieved a mAP of 69% in the KITTI dataset and a mAP of 66% in the GRAZPEDWRI-DX dataset. Moreover, our explanations have shown a high reliability on the similarity score.

In the "blind review" mode for technical bid evaluation within the power industry, review experts must make impartial judgments based on anonymized technical response documents. However, these documents often contain a large amount of complex graphical content, which existing general Optical Character Recognition (OCR) technology struggles to accurately identify. To address this, this study first constructs a specialized image dataset covering typical power industry response document scenarios. Subsequently, an improved You Only Look Once (YOLO) object detection model is developed, incorporating an anchor box clustering strategy optimized for small, densely packed power symbols and a multi-scale feature fusion mechanism. Then, a local contextual relationship inference module based on Graph Neural Networks is designed and integrated. Finally, an end-to-end recognition pipeline from image input to structured information output is constructed. Experimental results indicate that the improved model achieves a mean Average Precision (mAP) of 90.0% in key equipment detection tasks; in topology relationship restoration experiments, the average Graph Edit Distance (GED) is 0.172; in end-to-end efficiency experiments simulating real blind review scenarios, the average processing time per document is only 8.7 seconds, with an information completeness rate of 98.2%. The research demonstrates that the YOLO-based object detection and relationship modeling method, customized for the blind review mode, can effectively address the challenge of recognizing complex image content in power response documents, providing reliable technical support for the efficiency and fairness of blind reviews.

Backdoor Attacks with Wavelet Embedding: Revealing and enhancing the insights of vulnerabilities in visual object detection models on transformers within digital twin systems

Recently, edge computing has emerged as a prevailing paradigm in applying deep learning-based object detection models, offering a promising solution for time-sensitive tasks. However, existing edge object detection faces several challenges: 1) These methods struggle to balance detection precision and model lightweightness. 2) Existing generalized edge-deployment designs offer limited adaptability for object detection. 3) Current works lack real-world evaluation and validation. To address these challenges, we propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><u>E</u>dge <u>D</u>etection <u>Toolbox</u></i> (ED-TOOLBOX), which leverages generalizable plug-and-play components to enable edge-site adaptation of object detection models. Specifically, we propose a lightweight <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Reparameterized Dynamic Convolutional Network</i> (Rep-DConvNet) that employs a weighted multi-shape convolutional branch structure to enhance detection performance. Furthermore, ED-TOOLBOX includes a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sparse Cross-Attention</i> (SC-A) network that adopts a localized-mapping-assisted self-attention mechanism to facilitate a well-crafted <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Joint Module</i> in adaptively transferring features for further performance improvement. Moreover, we propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Efficient Head</i> for the classification and location modules to achieve more efficient prediction. Additionally, in practical industrial scenarios, we identify that helmet detection-one of the most representative edge object detection tasks-overlooks band fastening, which introduces potential safety hazards. To address this, we build a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Helmet Band Detection Dataset</i> (HBDD) and apply an edge object detection model optimized by the ED-TOOLBOX to tackle this real-world task. Extensive experiments validate the effectiveness of components in ED-TOOLBOX. In visual surveillance simulations, ED-TOOLBOX-assisted edge detection models outperform six <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">state-of-the-art</i> methods, enabling real-time and accurate detection. These results demonstrate that our approach offers a superior solution for edge object detection.

Global warming and pollution are huge problems today. One of the main factors behind these two problems is plastic pollution. As plastic takes millennia to decompose, an estimated 270,000 tons of plastic are floating around our oceans, which is too much to be considered “safe.” This problem is mainly caused by the fact that people improperly dispose of plastic, whether that be through littering or putting it into the trash instead of a recycling bin. To better identify and correct plastic that was inappropriately disposed of, a deep-learning model (YOLOv5) that uses object detection and classification was implemented to detect which bin someone’s trash should go in. The YOLOv5 uses the PyTorch framework for the object detection model. Using this model helped solve this problem, as a custom object-detection model would have needed to be developed, which would not have been efficient. The model was tested by trying to run the model on pieces of trash placed on a tabletop and analyzing the code output on which trash bin the waste will have to be thrown away. After conducting multiple tests, the model exhibited a commendable accuracy rate of 90%, which is noteworthy given the substantial amount of data leveraged. To further improve its efficacy and real-world value, future research could explore augmenting the training data, refining the object detection model for greater precision, and expanding the dataset to encompass a broader range of use cases.

Leaf counting has numerous applications in plant phenotyping, such as plant growth analysis, yield prediction, and disease detection. However, manual counting is labor-intensive and time-consuming, posing a significant limitation. To address this issue, deep learning-based object detection models are implemented for automated leaf counting. Existing works that explore automated leaf counting are limited due to the inherent structure of plants, e.g., plants are small and occur in overlapping clusters, causing the models to perform sub-optimally. Furthermore, the numerous amount of object detection models introduces a problem of choosing the proper model. Currently, there is limited understanding in determining which object detection architectures perform well for automated leaf counting, particularly when dealing with complex plant anatomy. To address the gap in the literature, three foundational object detection models with different characteristics are compared: YOLOv8, YOLOv12, and Faster R-CNN. The three models chosen display key distinctions such as one-stage vs. two-stage detections and convolution-based vs. attention-based. To provide comprehensive results, multiple optimization techniques were applied to each model. Our experimental results showed that YOLOv8 had the best performance on all performance metrics (mAP50, mAP5095, IOU, MAE, and MAPE). Specifically, YOLOv8 achieved an IOU of 0.6412, while YOLOv12 and Faster R-CNN only achieved an IOU of 0.567 and 0.606, respectively.

The chapter discusses the design and development of a novel object detection model keeping the motive that anyone and everyone with little to no prior knowledge can easily use it. The object detection is a part of computer vision which deals with detection and localization of objects in images or videos. These systems can be implemented for surveillance, keeping track of goods, robotics, medical field, self-driving cars, and much more. A lot of organizations are using it for various purposes such as it is used by NASA for analysis for deep space analysis, self-driving cars use it for making accurate decisions, and airports use it for security and surveillance. Although it has numerous use cases, but because of complex implementation and need of expertise in the field, it is not being widely used currently. In this chapter, we have made an effort to automate most of the process that a user should do in order to train and use models for object detection. The most complex part of using these technologies is training custom models, and only people having expertise in field are doing it. With the system designed by us, anyone would be able to train custom models on their own and they don’t have to do much for it. Normally, first users have to gather large amount of images and then annotate them 124one by one, which is a tedious and time taking process. One has to do this for thousands of images and each of those images can have multiple instances of various objects to train a model to give a good accuracy. We are automating the whole process with the help of AWS recognition module of AWS. You only look once (YOLO) object detection model is used for automation, as it is fast and accurate at the same time which gives it edge over the other models available for use. This model is based on deep learning to learn through the images provided by user in training phase and then later on can be used to perform detection on images or real-time videos. A graphic user interface (GUI) is also developed, so the users can easily interact with the system and train their own custom models and use as they want it to be used. Data are sent over the AWS servers and retrieve the results using a block of python code, then the response collected from AWS is manipulated and the data are extracted that is required to annotate the images and then processing is done according to the need of our target model. This format can be different for various models which can be used for object detection.

An adversarial attack on DNN-based black-box object detectors

Object detection with deep learning model has achieved good results in many fields, but in some fields that think highly of data privacy, such as medical care, its applications is greatly limited by data. And Federated Learning allows clients to train a model together, while leaving their data in the local, without sharing with the server or other clients. Using the methods of Federated Learning, such as Federated Averaging(FedAvg), to train models can provide privacy, security benefits. Nonetheless, there is little experiment applying Federated Learning algorithms to train the model with a large number of parameters, such as deep learning object detection model. With non-IID data, the accuracy of object detection model trained by FedAvg reduces significantly, and need more rounds to coverage. In this work, we use Kullback-Leibler divergence(KLD) measure the weights divergence between different model trained with non-IID data. And we propose a useful scheme to improve FedAvg based Abnormal Weights Supression, reducing the influence of the weights divergence caused by non-IID and unbalanced data. As a representative of object detection, we choose Single Shot MultiBox Detector(SSD) as the base model. The results of the experiments show that the Mean Average Precision(mAP) get obvious improvement in Pascal VOC 2007 test dataset.

This study developed a rapid rice yield estimation workflow and customized yield prediction model by integrating remote sensing and meteorological data with machine learning (ML). Several issues need to be addressed while developing a crop yield estimation model, including data quality issues, data processing issues, selecting a suitable machine learning model that can learn from few available time-series data, and understanding the non-linear relationship between historical crop yield and remote sensing and meteorological factors. This study applied a series of data processing techniques and a customized ML model to improve the accuracy of crop yield estimation at the district level in Nepal. It was found that remote sensing-derived NDVI product alone was not sufficient for accurate estimation of crop yield. After incorporating other meteorological variables into the ML models, estimation accuracy improved dramatically. Along with NDVI, the meteorological variables of rainfall, soil moisture, and evapotranspiration also exhibited a strong association with rice yield. This study also found that stacking multiple tree-based regression models together could achieve better accuracy than benchmark linear regression or standalone ML models. Due to the unique and distinct physio-geographical setting of each district, a variation in estimation accuracy from district to district could be observed. Our data processing and ML model workflow achieved an average of 92% accuracy of yield estimation with RMSE 328.06 kg/ha and MAE 317.21 kg/ha. This methodological workflow can be replicated in other study areas and the results can help the local authorities and stakeholders understand the factors affecting crop yields as well as estimating crop yield before harvesting season to ensure food security and sustainability.

Deep neural networks (DNNs) play an important role in our daily lives, from aiding us in menial tasks to solving world issues such as cancer cell detection. However, few pieces of research have been conducted using DNNs and deep learning models as a medium to help classify and detect trash, in efforts to solve our global waste crisis. This is because current DNNs struggle to be both efficient and accurate while detecting indistinct objects such as waste. To address this issue, this work focuses on YOLO-Green, a novel real-time object detection model designed specifically for trash detection. The model is trained on a dataset gathered from real-world trash divided into seven of the most common types of solid waste. With only 100 epochs of training, YOLO-Green achieves an outstanding mAP of 78.04%, FPS of 2.72, while retaining a model size of only 117 MB. Based on the original object detection of YOLOv4, YOLO-Green exceeds YOLOv4 and other popular deep learning models in both its accuracy and efficiency, while maintaining a relatively small model size. Ultimately, this study sheds a positive light on the potential of using deep learning models as an alternative to manual waste management.

This paper proposed an innovative mechanical design using the Rocker-bogie mechanism for resilient Trash-Collecting Robots. Mask-RCNN, YOLOV4, and YOLOv4-tiny were experimented on and analyzed for trash detection. The Trash-Collecting Robot was developed to be completely autonomous as it was able to detect trash, move towards it, and pick it up while avoiding any obstacles along the way. Sensors including a camera, ultrasonic sensor, and GPS module played an imperative role in automation. The brain of the Robot, namely, Raspberry Pi and Arduino, processed the data from the sensors and performed path-planning and consequent motion of the robot through actuation of motors. Three models for object detection were tested for potential use in the robot: Mask-RCNN, YOLOv4, and YOLOv4-tiny. Mask-RCNN achieved an average precision (mAP) of over 83% and detection time (DT) of 3973.29 ms, YOLOv4 achieved 97.1% (mAP) and 32.76 DT, and YOLOv4-tiny achieved 95.2% and 5.21 ms DT. The YOLOv4-tiny was selected as it offered a very similar mAP to YOLOv4, but with a much lower DT. The design was simulated on different terrains and behaved as expected.

Object detection and semantic segmentation are two fundamental techniques of various applications in the fields of Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Early studies separately handle these two problems. In this paper, inspired by some recent works, we propose a deep neural network model for joint object detection and semantic segmentation. Given an image, an encoder-decoder convolution network extracts a set of feature maps, these feature maps are shared by the detection branch and the segmentation branch to jointly carry out the object detection and semantic segmentation. In the detection branch, we design a PriorBox initialization mechanism to propose more object candidates. In the segmentation branch, we use the multi-scale atrous convolution to explore the global and local semantic information in traffic scenes. Benefiting from the PriorBox Initialization Mechanism (PBIM) and Multi-Scale Atrous Convolution (MSAC), our model presents the competitive performance. In the experiments, we widely compare with several recently-proposed methods on the public Cityscapes dataset, achieving the highest accuracy. In addition, to verify the robustness and generalization of our model, the extension experiments are also conducted on the well-known VOC2012 dataset.

Foreign bodies such as a surgical gauze can be retained in the body after surgery and in some cases cannot be detected by postoperative radiography. The aim of this study was to develop an object detection model capable of postsurgical detection of retained gauze in the body. The object detection model used deep learning using abdominal radiographs, and a phantom study was performed to evaluate the ability of the model to automatically detect retained surgical gauze. The object detection model was constructed using a Single Shot MultiBox Detector (SSD) 300. In total, 268 abdominal phantom images were used: 180 gauze images were used as training data, 20 gauze images were used as validation data, and an additional 34 gauze images and 34 nongauze images were used as test data. To evaluate the performance of the object detection model, a confusion matrix was created and the accuracy and sensitivity were calculated. True-positive (TP) rate, true-negative (TN) rate, false-positive (FP) rate, and false-negative (FN) rate were 0.92, 1.00, 0.00, and 0.08, respectively. Accuracy was 0.96, and sensitivity was 0.92. The object detection model could detect surgical gauze on abdominal phantom images with a high accuracy and sensitivity.