Automatic “Ground Truth” Annotation and Industrial Workpiece Dataset Generation for Deep Learning

X-ray baggage inspection is an essential task to detect threat objects at important controlled access places, which can guard personal safety and prevent crime. Generally, it is carried out by screeners to visually determine whether or not a bag contains threat objects. Whereas, manual detection exhibits distinct shortcomings, from high detection errors to different detection results produced by screeners. These limitations can be addressed by introducing automated detection model of threat objects for X-ray baggage inspection. However, existing automated detection methods cannot realize end-to-end detection and the detection results include only classification without location. In this paper, we propose an automated detection model of threat objects based on depthwise separable convolution. Our model is able to not only categorize the threat object but also locate it simultaneously. The network model has the advantage of high detection accuracy, fast computational speed, and a few parameters. Meanwhile, the precision of threat object regions is enhanced with the help of multi-scale prediction. A deformation layer is added in our model, which can provide invariance to affine warping. The experiments on the GDXray database (Mery et al. in J Nondestr Eval 34(4):42, 2015) demonstrate that the overall performance of our proposed model is superior to YOLOv3 (Redmon J and Farhadi A in YOLOv3: an incremental improvement, 2018) model, SSD (Liu et al. in SSD: single shot multibox detector. In: European Conference on Computer Vision (ECCV), pp. 21–37, 2016) model, and Tiny_YOLO (Redmon et al. in You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788, 2015) model.

Nursery plays an important role on starting chili cultivation, determining the crop health, fertility from disease, and growth performance. Early-stage germination detection is necessary to minimize nursery failure and improve plant health, but manual detection is challenging for large scale nursery in the greenhouse. The aim of this research was to develop an automatic detection model integrated with a You Only Look Once (YOLO) based deep learning algorithm using RGB camera to monitor the chili germination stages. Method to detect germination was YOLO with several steps, included: (1) early stages chili germination images acquisition, (2) datasets preparations, (3) dataset annotation and labeling, (4) model development using deep learning YOLO algorithms, and (5) model testing and validation. The training of 11,423 images was conducted utilizing the YOLOv5 and YOLOv8 algorithms, which categorized into, three classes (germinated, not germinated, and cotyledon appearance). The model was evaluated using mean Average Precision (mAP), precision, accuracy, and recall with the respective values of 0.697, 73%, 75%, and 73% for YOLOv8, and 0.664, 70%, 73%, and 70% for YOLOv5. Both model achieved high accuracy, but YOLOv8 was better to detect and classify chili seedling growth stages than YOLOv5. This study also demonstrated that model can be implemented in real applications integrated with automatic monitoring system included in the model. Keywords: Chili seedling, Deep learning, Detection system, Germination.

The natural phenomenon of harmful algae bloom (HAB) has a bad impact on the quality of pure and freshwater. It increases the risk to human health, water bodies and overall aquatic ecosystem. It is necessary to continuously monitor and perform proper action against HAB. The inspection of algae blooms by using conventional methods, like algae detection under microscopes, is a difficult, expensive, and time-consuming task, however, computer vision-based deep learning models play a vital role in identifying and detecting harmful algae growth in aquatic ecosystems and water reservoirs. Many studies have been conducted to address harmful algae growth by using a CNN based model, however, the YOLO model is considered more accurate in identifying the algae. This advanced deep learning method is extensively used to detect algae and classify them according to their corresponding category. In this study, we used various versions of the convolution neural network (CNN) based on the You Only Look Once (YOLO) model. Recently YOLOv5 has been getting more attention due to its performance in real-time object detection. We performed a series of experiments on our custom microscopic images dataset by using YOLOv3, YOLOv4, and YOLOv5 to detect and classify the harmful algae bloom (HAB) of four classes. We used pre-processing techniques to enhance the quantity of data. The mean average precision (mAP) of YOLOv3, YOLOv4, and YOLO v5 is 75.3%, 83.0%, and 91.0% respectively. For the monitoring of algae bloom in freshwater, computer-aided based systems are very helpful and effective. To the best of our knowledge, this work is pioneering in the AI community for applying the YOLO models to detect algae and classify from microscopic images.

Vision-based structural displacement monitoring of rotor has attracted extensive attention from scholars due to its non-contact, non-damage, and multi-point synchronous measurement advantages. However, there is a lack of one comprehensive dataset and benchmark for rotor detection since the problem of limited acquisition equipment or on-site environment in the industrial field. For that, a large benchmark for rotor’s vibration displacement detection is introduced in this paper. Firstly, a rotor vibration displacement detection network (RDNet) for rotating structures is proposed and state-of-the-art performance is achieved. Secondly, the evaluation criteria are designed to evaluate the rotor image detection accuracy using the mean Average Precision@0.5:0.95 (mAP@0.5:0.95), the Frames Per Second (FPS), and the Normalized Root Mean Squared Error (NRMSE) as performance indicators. Thirdly, a dataset namely Rotor V1.0 is built, which contains three frames (high, middle, and low) of 13,500 rotor images. It is the rotor dataset for vibration displacement monitoring of rotating structures. Lastly, the baseline performance resulted from traditional methods (i.e., match template and support vector machines) and deep learning methods (i.e., CenterNet, Faster Region-convolutional neural networks (Faster R-CNN), RetinaNet, Single Shot MultiBox Detector (SSD), and You only look once (YOLO)) are reported and analyzed. It is hoped that RDNet, evaluation criteria, and the novel Rotor V1.0 dataset can promote the implementation of visual vibration measurement tasks in industrial sites.

Hand gesture recognition has great applications in human-computer interaction (HCI), human-robot interaction (HRI), and supporting the deaf and mute. To build a hand gesture recognition model using deep learning (DL) with high results then needs to be trained on many data and in many different conditions and contexts. In this paper, we publish the TQU-HG dataset of large RGB images with low resolution (640×480) pixels, low light conditions, and fast speed (16 fps). TQU-HG dataset includes 60,000 images collected from 20 people (10 male, 10 female) with 15 gestures of both left and right hands. A comparative study with two branches: i) based on Mediapipe TML and ii) Based on convolutional neural networks (CNNs) (you only look once (YOLO); YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLO-Nas, single shot multiBox detector (SSD) VGG16, residual network (ResNet)18, ResNext50, ResNet152, ResNext50, MobileNet V3 small, and MobileNet V3 large), the architecture and operation of CNNs models are also introduced in detail. We especially fine-tune the model and evaluate it on TQU-HG and HaGRID datasets. The quantitative results of the training and testing are presented (F1-score of YOLOv8, YOLO-Nas, MobileNet V3 small, ResNet50 is 98.99%, 98.98%, 99.27%, 99.36%, respectively on the TQU-HG dataset and is 99.21%, 99.37%, 99.36%, 86.4%, 98.3%, respectively on the HaGRID dataset). The computation time of YOLOv8 is 6.19 fps on the CPU and 18.28 fps on the GPU.

Object detection is challenging in the computer vision area and is crucial in autonomous driving systems. The largest number of traditional techniques or the use of deep learning are evaluated in the urban area, but in rural areas, there is little research carried out. The goal of this work is object detection in rural roads, this paper presents the use of deep learning frameworks used as You Only Look Once (YOLO) and another that belongs to the same category of one-stage is the well-known Single Shot Multi-Box Detector (SSD), in the state of the literature, produces excellent results in detecting objects in real-time. These models behave differently in network architecture, for this reason, we use images of rural roads with different environments to achieve an optimal balance between precision and precision in the detection of objects. Furthermore, as there is no dataset in these environments, we created our own data set to perform the experiments due to the difficulty of this problem. The result of both detectors has produced acceptable results under certain conditions like lighting conditions, viewing perspectives, partial occlusion of the object.

The timely and accurate mapping and monitoring of mine tailings dams is crucial to the improvement of management practices by decision makers and to the prevention of disasters caused by failures of these dams. Due to the complex topography, varying geomorphological characteristics, and the diversity of ore types and mining activities, as well as the range of scales and production processes involved, as they appear in remote sensing imagery, tailings dams vary in terms of their scale, color, shape, and surrounding background. The application of high-resolution satellite imagery for automatic detection of tailings dams at large spatial scales has been barely reported. In this study, a target detection method based on deep learning was developed for identifying the locations of tailings ponds and obtaining their geographical distribution from high-resolution satellite imagery automatically. Training samples were produced based on the characteristics of tailings ponds in satellite images. According to the sample characteristics, the Single Shot Multibox Detector (SSD) model was fine-tuned during model training. The results showed that a detection accuracy of 90.2% and a recall rate of 88.7% could be obtained. Based on the optimized SSD model, 2221 tailing ponds were extracted from Gaofen-1 high resolution imagery in the Jing–Jin–Ji region in northern China. In this region, the majority of tailings ponds are located at high altitudes in remote mountainous areas. At the city level, the tailings ponds were found to be located mainly in Chengde, Tangshan, and Zhangjiakou. The results prove that the deep learning method is very effective at detecting complex land-cover features from remote sensing images.

Object detection is a fundamental problem in computer vision, with applications spanning self-driving cars, surveillance systems, medical imaging, robotics, and smart cities. Among the myriad of algorithms developed for this task, the You Only Look Once (YOLO) family stands out for its ability to perform real-time and accurate object detection. This article provides a comprehensive analysis of the YOLO algorithm series, from YOLOv1 to YOLOv8, evaluating them across key performance metrics, including precision, recall, mean Average Precision (mAP), frames per second (FPS), and overall effectiveness. Unlike traditional two-stage detectors such as R-CNN, YOLO formulates object detection as a single regression problem: a single pass over the image simultaneously predicts bounding boxes and class probabilities. This end-to-end design enables YOLO to achieve high speed while maintaining a competitive accuracy-efficiency trade-off. We examine architectural innovations across YOLO versions, including batch normalization, anchor boxes, residual blocks, feature pyramid networks, and attention mechanisms, and discuss their impact on performance. Lightweight models (e.g., YOLOv5-Nano, YOLOv8-Small) are explored with a focus on their suitability for mobile and embedded systems, highlighting YOLO’s adaptability to resource-constrained environments. Challenges such as small object detection, occlusion, and domain-specific tuning are also addressed. This article serves as a practical guide for researchers, developers, and practitioners aiming to leverage YOLO for real-world object detection tasks.

The development of robotic solutions for agriculture requires advanced perception capabilities that can work reliably in any crop stage. For example, to automatise the tomato harvesting process in greenhouses, the visual perception system needs to detect the tomato in any life cycle stage (flower to the ripe tomato). The state-of-the-art for visual tomato detection focuses mainly on ripe tomato, which has a distinctive colour from the background. This paper contributes with an annotated visual dataset of green and reddish tomatoes. This kind of dataset is uncommon and not available for research purposes. This will enable further developments in edge artificial intelligence for in situ and in real-time visual tomato detection required for the development of harvesting robots. Considering this dataset, five deep learning models were selected, trained and benchmarked to detect green and reddish tomatoes grown in greenhouses. Considering our robotic platform specifications, only the Single-Shot MultiBox Detector (SSD) and YOLO architectures were considered. The results proved that the system can detect green and reddish tomatoes, even those occluded by leaves. SSD MobileNet v2 had the best performance when compared against SSD Inception v2, SSD ResNet 50, SSD ResNet 101 and YOLOv4 Tiny, reaching an F1-score of %, an mAP of % and an inference time of with the NVIDIA Turing Architecture platform, an NVIDIA Tesla T4, with 12 GB. YOLOv4 Tiny also had impressive results, mainly concerning inferring times of about 5 .

Ensuring compliance with complex industry standards and regulations during the design and implementation phases of construction projects is a significant challenge in the building information modeling (BIM) domain. Traditional manual compliance checking methods are inefficient and error-prone, failing to meet modern engineering demands. Natural language processing (NLP) and deep learning methods have improved efficiency and accuracy in rule interpretation and compliance checking. However, these methods still require extensive manual feature engineering, large, annotated datasets, and significant computational resources. Large language models (LLMs) provide robust language understanding with minimal labeled data due to their pre-training and few-shot learning capabilities. However, their application in the AEC field is still limited by the need for fine-tuning for specific tasks, handling complex texts with nested clauses and conditional statements. This study introduces an innovative automated compliance checking framework that integrates LLM, deep learning models, and ontology knowledge models. The use of LLM is motivated by its few-shot learning capability, which significantly reduces the need for large, annotated datasets required by previous methods. Deep learning is employed to preliminarily classify regulatory texts, which further enhances the accuracy of structured information extraction by the LLM compared to directly feeding raw data into the LLM. This novel combination of deep learning and LLM significantly enhances the efficiency and accuracy of compliance checks by automating the processing of regulatory texts and reducing manual intervention. This approach is crucial for architects, engineers, project managers, and regulators, providing a scalable and adaptable solution for automated compliance in the construction industry with broad application prospects.

Opium poppies are a major source of traditional drugs, which are not only harmful to physical and mental health, but also threaten the economy and society. Monitoring poppy cultivation in key regions through remote sensing is therefore a crucial task; the location coordinates of poppy parcels represent particularly important information for their eradication by local governments. We propose a new methodology based on deep learning target detection to identify the location of poppy parcels and map their spatial distribution. We first make six training datasets with different band combinations and slide window sizes using two ZiYuan3 (ZY3) remote sensing images and separately train the single shot multibox detector (SSD) model. Then, we choose the best model and test its performance using 225 km2 verification images from Lao People’s Democratic Republic (Lao PDR), which exhibits a precision of 95% for a recall of 85%. The speed of our method is 4.5 km2/s on 1080TI Graphics Processing Unit (GPU). This study is the first attempt to monitor opium poppies with the deep learning method and achieve a high recognition rate. Our method does not require manual feature extraction and provides an alternative way to rapidly obtain the exact location coordinates of opium poppy cultivation patches.

Deep learning methods have become popular among researchers in the field of fault detection. However, their performance depends on the availability of big datasets. To overcome this problem researchers started applying transfer learning to achieve good performance from small available datasets, by leveraging multiple prediction models over similar machines and working conditions. However, the influence of negative transfer limits their application. Negative transfer among prediction models increases when the environment and working conditions are changing continuously. To overcome the effect of negative transfer, we propose a novel deep transfer learning method, coined deep boosted transfer learning, for wind turbine gearbox fault detection that prevents negative transfer and only focuses on relevant information from the source machine. The proposed method is an instance-based deep transfer learning method that updates the weights of the source and the target machine training samples separately. The weights of different source training samples are gradually decreased to reduce the impact on the final model. The proposed method is verified by the Case Western Reserve University bearing and real field wind farm datasets. The results show that the proposed method ignores negative transfer and achieves higher accuracy compared to standard deep learning and deep transfer learning methods.

Wild birds are monitored with the important objectives of identifying their habitats and estimating the size of their populations. Especially in the case of migratory bird, they are significantly recorded during specific periods of time to forecast any possible spread of animal disease such as avian influenza. This study led to the construction of deep-learning-based object-detection models with the aid of aerial photographs collected by an unmanned aerial vehicle (UAV). The dataset containing the aerial photographs includes diverse images of birds in various bird habitats and in the vicinity of lakes and on farmland. In addition, aerial images of bird decoys are captured to achieve various bird patterns and more accurate bird information. Bird detection models such as Faster Region-based Convolutional Neural Network (R-CNN), Region-based Fully Convolutional Network (R-FCN), Single Shot MultiBox Detector (SSD), Retinanet, and You Only Look Once (YOLO) were created and the performance of all models was estimated by comparing their computing speed and average precision. The test results show Faster R-CNN to be the most accurate and YOLO to be the fastest among the models. The combined results demonstrate that the use of deep-learning-based detection methods in combination with UAV aerial imagery is fairly suitable for bird detection in various environments.

This paper explores and evaluates the potential of deep neural network (DNN)-based machine learning algorithms on embed-ded many-core processors in cyber-physical systems, such as self-driving systems. To run applications in embedded systems, a plat-form characterized by low power consumption with high accuracy and real-time performance is required. Furthermore, a platform is required that allows the coexistence of DNN applications and other applications, including conventional real-time control soft-ware, to enable advanced embedded systems, such as self-driving systems. Clustered many-core processors, such as Kalray MPPA3-80 Coolidge, can run multiple applications on a single platform because each cluster can run applications independently. Moreover, MPPA3-80 integrates multiple arithmetic elements that operate at low frequencies, thereby enabling high performance and low power consumption comparable to that of embedded graphics processing units. Furthermore, the Kalray Neural Network (KaNN) code generator, a deep learning inference compiler for the MPPA3-80 platform, can efficiently perform DNN inference on MPPA3-80. This paper evaluates DNN models, including You Only Look Once (YOLO)-based and Single Shot MultiBox Detector (SSD)-based mod-els, on MPPA3-80. The evaluation examines the frame rate and power consumption in relation to the size of the input image, the computational accuracy, and the number of clusters.

Image set annotation is an important task in the supervised training of the deep neural network. Manual and data-driven dataset annotation methods are commonly used approaches. Both of them have shortcomings, especially in the case of the dataset requiring professional knowledge, which leads to high cost with manual annotation methods and poorly diversified annotation samples with data-driven annotation methods. Although the recommendation annotation method based on cosine similarity using deep neural network features takes advantages of manual annotation and data-driven method, there are still problems such as low accuracy and click-through rate. In order to improve the recommendation accuracy and click-through rate, we propose a confusion graph recommendation annotation method, which builds a confusion graph based on the Largest Margin Nearest Neighbor (LMNN) distance among deep neural network features, to recommend the most confusing images to annotators. In this paper, we made ablation studies on the self-built child face dataset in terms of Precision, mAP (mean Average Precision), and CTR (click-through-rate). The experimental results show that the proposed method achieves superior performance, compared with the cosine similarity recommendation annotation method and the manual annotation method.