OpenCL-Darknet: implementation and optimization of OpenCL-based deep learning object detection framework

Application of Deep Learning for Object Detection

Air-to-ground object detection has important application value in many fields such as military reconnaissance, air strike, personnel rescue, and natural exploration. Object detection algorithm based on deep learning solves the problems of poor generality and low robustness of traditional manual detection algorithms. It has great advantages in air-to-ground detection tasks, but for small unmanned platforms, currently deep learning object detection algorithms have high hardware requirements and are difficult to deploy. In this paper, a deep learning object detection simulation system is built based on the TX2 embedded development board with the features of low power consumption, light weight and fast calculation speed. This object detection simulation experiment is performed for two mainstream deep learning object detection algorithms. The experimental results showed that the regression-based SSD algorithm combined with MobileNets feature extraction network could perform real-time in the embedded system, which laid the foundation for the actual construction of the next unmanned aerial platform.

Vegetation cover classification using mixed or low-resolution scalar images is challenging. Fortunately, recently deep learning object detection methods have emerged as a replacement to the conventional machine learning methods for the detection and classification of land use and land cover. This paper presents a deep learning object detection approach for land use and land cover detection using low/mixed resolution satellite images acquired from Google Earth satellite images. Google Earth images are accessible freely using the Google Earth Pro desktop application. Our dataset consists of two (02) classes (vegetation and non-vegetation) with a total of 450 labeled images captured from different parts of Pakistan. We present a comparison of the recent anchor-free object detection model YOLOX with the anchor-based object detection model YOLOR for solving real-time problems. The end-to-end differentiability, efficient GPU utilization, and absence of hand-crafted parameters make anchor-free models a compelling choice in object detection, and yet not been explored on Land cover classification using satellite images. Our experimental study shows that YOLOX delivers an overall accuracy of 83.50% on Vegetation and 86% on Non-Vegetation classes, which outperformed YOLOR by 30% on Vegetation classes and 34% on non-Vegetation classes for our dataset. We also show how an object detection system can be used for Vegetation and Non-Vegetation classification tasks, which can then be used for change monitoring and assisting in developing geographical maps using low/mixed resolution freely available satellite images.

Object counting is an indispensable task in manufacturing and management. Recently, the development of image-processing techniques and deep learning object detection has achieved excellent performance in object-counting tasks. Accordingly, we propose a novel small-size smart counting system composed of a low-cost hardware device and a cloud-based object-counting software server to implement an accurate counting function and overcome the trade-off presented by the computing power of local hardware. The cloud-based object-counting software consists of a model adapted to the object-counting task through a novel DBC-NMS (our own technique) and hyperparameter tuning of deep-learning-based object-detection methods. With the power of DBC-NMS and hyperparameter tuning, the performance of the cloud-based object-counting software is competitive over commonly used public datasets (CARPK and SKU110K) and our custom dataset of small pills. Our cloud-based object-counting software achieves an mean absolute error (MAE) of 1.03 and a root mean squared error (RMSE) of 1.20 on the Pill dataset. These results demonstrate that the proposed smart counting system accurately detects and counts densely distributed object scenes. In addition, the proposed system shows a reasonable and efficient cost–performance ratio by converging low-cost hardware and cloud-based software.

With the rapid advancement of rail transit, railway intrusion detection has become a crucial and indispensable technology for ensuring the safe operation of trains. Current mainstream railway intrusion detection methods are based on deep learning and general object detection frameworks. However, they rely on large-scale, high-cost annotated datasets, leading to expensive data collection and poor performance in data-scarce railway scenarios. Meanwhile, few-shot object detection methods often generalize poorly to novel classes and suffer from catastrophic forgetting of base classes. To address these issues, we leverage a visible-light camera as the vision sensor and propose a few-shot railway intrusion detection method based on Dual Detector, Contrastive Learning within Novel Classes (CLNC), and an Efficient Fine-Tuning Framework. The Dual Detector design decouples the detection of base and novel classes, mitigating catastrophic forgetting, while the CLNC module enhances intra-class compactness and inter-class separability, improving generalization to novel classes. Additionally, the Efficient Fine-Tuning Framework optimizes module collaboration, further enhancing detection accuracy. Extensive experiments on the self-constructed few-shot railway intrusion dataset (FSRI2024), collected using a visible-light camera, demonstrate that the proposed G-FSRD achieves better performance compared to state-of-the-art few-shot object detection methods. It effectively preserves common base intrusions detection performance while efficiently adapting to rare novel intrusions, making it well-suited for railway intrusion detection.

Nowadays pathologists have to analyse blood cells manually with the aim to diagnose diseases. In order to perform this manual task blood samples must be collected from the patient and then placed on a microscope slide. This slide is studied with the aim to detect the abnormality presence. The manual method used to identify abnormality of blood cells is tedious, prone to human errors and time consuming. Hence there is a need for computer aided system which can analyze blood cells automatically at faster rate with accuracy. Such systems can be designed using image processing techniques. To automatise this process by helping pathologists, in this paper we propose the adoption of deep learning to automatically count and localise red blood cells, white blood cells and platelets by analysing blood microscopic images. We resort to the YOLO object detection model, able to look at the whole image so its predictions are informed by global context in the image. To show the effectiveness of the proposed method we evaluate our model on a dataset composed by 874 microscopic blood images, obtaining interesting results. Furthermore we show several examples related to how the proposed method can be helpful for the pathologists in their real-world work.

Lung cancer is a leading cause of death on a global scale. Professor I-Fang Chung leads a team of researchers at the Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taiwan, investigating how machine learning can help predict the risk of tumour recurrence following lung cancer surgery. The standard treatment for early-stage lung cancer patients is complete surgical resection of the tumour but disease recurrence within the first five years following surgery is common. The route of this recurrence is usually tumour spread through the air spaces (STAS) and the presence of STAS has recently been identified as a risk factor for recurrence of tumours. The researchers are exploring the importance of STAS and how targeting this phenomenon and using machine learning methods to aid in the analysis of medical imaging of diseased tissues can help in predicting the recurrence risk of post-surgery lung cancer patients. Dr Yi-Chen Yeh, a pathologist from the Taipei Veterans General Hospital, is working with Chung to employ a variety of deep-learning object detection methodologies to detect STAS in pathology images. Regions of interest (ROI) images extracted from pathology whole slide images (WSI) are marked and annotated, which provides location information for STAS. Deep learning object detection methods are used to train a model which can find STAS, then additional techniques including using pre-trained model parameters, augmenting random image data and modifying loss function are used to improve the detection rates for the model. Their model helps pathologists identify STAS and accurately predicts patient outcomes.

Objective: To design a deep transfer learning framework for modeling fluence map predictions for stereotactic body radiation therapy (SBRT) of adrenal cancer and similar sites that usually have a small number of cases. Approach: We developed a transfer learning framework for adrenal SBRT planning that leverages knowledge in a pancreas SBRT planning model. Treatment plans from the two sites had different dose prescriptions and beam settings but both prioritized gastrointestinal sparing. A base framework was first trained with 100 pancreas cases. This framework consists of two convolutional neural networks (CNN), which predict individual beam doses (BD-CNN) and fluence maps (FM-CNN) sequentially for 9-beam intensity-modulated radiation therapy (IMRT) plans. Forty-five adrenal plans were split into training/validation/test sets with the ratio of 20/10/15. The base BD-CNN was re-trained with transfer learning using 5/10/15/20 adrenal training cases to produce multiple candidate adrenal BD-CNN models. The base FM-CNN was directly used for adrenal cases. The deep learning (DL) plans were evaluated by several clinically relevant dosimetric endpoints, producing a percentage score relative to the clinical plans. Main results: Transfer learning significantly reduced the number of training cases and training time needed to train such a DL framework. The adrenal transfer learning model trained with 5/10/15/20 cases achieved validation plan scores of 85.4/91.2/90.7/89.4, suggesting that model performance saturated with 10 training cases. Meanwhile, a model using all 20 adrenal training cases without transfer learning only scored 80.5. For the final test set, the 5/10/15/20-case models achieved scores of 73.5/75.3/78.9/83.3. Significance: It is feasible to use deep transfer learning to train an IMRT fluence prediction framework. This technique could adapt to different dose prescriptions and beam configurations. This framework potentially enables DL modeling for clinical sites that have a limited dataset, either due to few cases or due to rapid technology evolution.

With the increase in urban rail transit construction, instances of tunnel disease are on the rise, and cracks have become the focus of tunnel maintenance and management. Therefore, it is essential to carry out crack detection in a timely and efficient manner to not only prolong the service life of the tunnel but also reduce the incidence of accidents. In this paper, the design and structure of a tunnel crack detection system are analyzed. On this basis, this paper proposes a new method for crack identification and feature detection using image processing technology. This method fully considers the characteristics of tunnel images and the combination of these characteristics with deep learning, while a deep convolutional network (Single-Shot MultiBox Detector (SSD)) is proposed based on deep learning for object detection in complex images. The experimental results show that the test set accuracy and training set accuracy of the support vector machine (SVM) in the classification comparison test are up to 88% and 87.8%, respectively; while the test accuracy of Alexnet’s deep convolutional neural network-based classification and identification is up to 96.7%, and the training set accuracy is up to 97.5%. It can be seen that this deep convolutional network recognition algorithm based on deep learning and image processing is better and more suitable for the detection of cracks in subway tunnels.

A novel data augmentation approach for mask detection using deep transfer learning

In the field of construction, the safety of construction workers on construction sites has been a major problem over the years. According to statistic of previous study, approximately 88% of these accidents are caused by unsafe behaviors. In recent years, the development and application of deep learning have attracted considerable research interest such as object detection, human pose estimation, etc. Meanwhile, the object detection techniques have been applied to detect whether workers wear or use proper equipment. However, using proper equipment doesn't mean performing correct operation. Therefore, this paper proposes a deep learning framework, DeepSafty, to detect unsafe behaviors of construction workers through their postures to reduce their mortality rate. In our proposed DeepSafty, the object detection model YOLOv3 is used to locate construction workers precisely. Then, human pose estimation technology is used to determine various postures of construction workers. There are 17 joints in the poses. In other words, a 51-dimensional vector will be produced from the neural network, as unsafe behaviors are continual movements in the field of construction. Finally, a model that deals with time series in deep learning, i.e., long short-term memory, is used to solve any classification problems of time-dependent joint vectors. Finally, we conduct a comprehensive experimental study based on a dataset collected through a CCTV of a real construction site. The results demonstrate the effectiveness of our deep learning approach and show the strength of taking both object detection and human pose estimation into account in unsafe behaviors of construction workers.

Deep learning models require vast amounts of annotated training data. Gathering and annotating the data from the real world is an expensive and time-consuming process. Thus, synthetically generated data is being researched more and more. This paper tries to answer the question of whether and to what extent synthetically generated data can help in developing the advanced driver-assistance system (ADAS) algorithms for autonomous vehicles, for the object detection task based on deep learning. Two state-of-the-art deep learning object detectors were trained on various combinations of real-world and synthetic data. A total of 12 detectors were tested on real-world test images. Results show that synthetic data can contribute to better detector performance until a certain ratio between real-world and synthetic data is reached.

Deep learning-based object detection in low-altitude UAV datasets: A survey

A comprehensive empirical study on bug characteristics of deep learning frameworks

Deep learning (DL) frameworks serve as the backbone for a wide range of artificial intelligence applications. However, bugs within DL frameworks can cascade into critical issues in higher-level applications, jeopardizing reliability and security. While numerous techniques have been proposed to detect bugs in DL frameworks, research exploring common API patterns across frameworks and the potential risks they entail remains limited. Notably, many DL frameworks expose similar APIs with overlapping input parameters and functionalities, rendering them vulnerable to shared bugs, where a flaw in one API may extend to analogous APIs in other frameworks. To address this challenge, we propose MirrorFuzz, an automated API fuzzing solution to discover shared bugs in DL frameworks. MirrorFuzz operates in three stages: First, MirrorFuzz collects historical bug data for each API within a DL framework to identify potentially buggy APIs. Second, it matches each buggy API in a specific framework with similar APIs within and across other DL frameworks. Third, it employs large language models (LLMs) to synthesize code for the API under test, leveraging the historical bug data of similar APIs to trigger analogous bugs across APIs. We implement MirrorFuzz and evaluate it on four popular DL frameworks (TensorFlow, PyTorch, OneFlow, and Jittor). Extensive evaluation demonstrates that MirrorFuzz improves code coverage by 39.92% and 98.20% compared to state-of-the-art methods on TensorFlow and PyTorch, respectively. Moreover, MirrorFuzz discovers 315 bugs, 262 of which are newly found, and 80 bugs are fixed, with 52 of these bugs assigned CNVD IDs.