- Research Article
25
- 10.1016/j.heliyon.2024.e38141
Application of mask R-CNN for building detection in UAV remote sensing images
- Sep 19, 2024
- Heliyon
- Tao Hou + 1 more +1
Application of mask R-CNN for building detection in UAV remote sensing images
Infrared thermography (IRT), is one of the most interesting techniques to identify different kinds of defects, such as delamination and damage existing for quality management of material. Objective detection and segmentation algorithms in deep learning have been widely applied in image processing, although very rarely in the IRT field. In this paper, spatial deep-learning image processing methods for defect detection and identification were discussed and investigated. The aim in this work is to integrate such deep-learning (DL) models to enable interpretations of thermal images automatically for quality management (QM). That requires achieving a high enough accuracy for each deep-learning method so that they can be used to assist human inspectors based on the training. There are several alternatives of deep Convolutional Neural Networks for detecting the images that were employed in this work. These included: 1. The instance segmentation methods Mask-RCNN (Mask Region-based Convolutional Neural Networks) and Center-Mask; 2. The independent semantic segmentation methods: U-net and Resnet-U-net; 3. The objective localization methods: You Only Look Once (YOLO-v3) and Faster Region-based Convolutional Neural Networks (Fast-er-RCNN). In addition, a regular infrared image segmentation processing combination method (Absolute thermal contrast (ATC) and global threshold) was introduced for comparison. A series of academic samples composed of different materials and containing artificial defects of different shapes and nature (flat-bottom holes, Teflon inserts) were evaluated, and all results were studied to evaluate the efficacy and performance of the proposed algorithms.
Loading PDF
Application of mask R-CNN for building detection in UAV remote sensing images
Application of mask R-CNN for building detection in UAV remote sensing images
Deep Learning Techniques—R-CNN to Mask R-CNN: A Survey
With the advances in the field of machine learning, statistics, and computer vision, the advanced deep learning techniques have attracted increasing research interests over the last decade. This is because of their inherent capabilities of overcoming the drawback of traditional techniques. The main contribution of this work is to provide a comprehensive description of region-based convolutional neural network (R-CNN) and its recent improvement like fast R-CNN, faster R-CNN, region-based fully convolutional networks, single shot detector, deconvolutional single shot detector, R-CNN minus R, you only look once (YOLO), mask R-CNN, etc., with brief details. This survey paper presents an overview of the last update in this field and their practical applications and its classification for ease of understanding. The performances and challenges of these techniques in terms of speed, accuracy, or simplicity are also compared. In general, the speed performance of YOLO is approximately 21 ~ 155 fps which is the fastest and the average precision of Mask R-CNN is ~47.3 which outperforms all other techniques.
Read moreInfluence of Image Quality and Light Consistency on the Performance of Convolutional Neural Networks for Weed Mapping
Recent computer vision techniques based on convolutional neural networks (CNNs) are considered state-of-the-art tools in weed mapping. However, their performance has been shown to be sensitive to image quality degradation. Variation in lighting conditions adds another level of complexity to weed mapping. We focus on determining the influence of image quality and light consistency on the performance of CNNs in weed mapping by simulating the image formation pipeline. Faster Region-based CNN (R-CNN) and Mask R-CNN were used as CNN examples for object detection and instance segmentation, respectively, while semantic segmentation was represented by Deeplab-v3. The degradations simulated in this study included resolution reduction, overexposure, Gaussian blur, motion blur, and noise. The results showed that the CNN performance was most impacted by resolution, regardless of plant size. When the training and testing images had the same quality, Faster R-CNN and Mask R-CNN were moderately tolerant to low levels of overexposure, Gaussian blur, motion blur, and noise. Deeplab-v3, on the other hand, tolerated overexposure, motion blur, and noise at all tested levels. In most cases, quality inconsistency between the training and testing images reduced CNN performance. However, CNN models trained on low-quality images were more tolerant against quality inconsistency than those trained by high-quality images. Light inconsistency also reduced CNN performance. Increasing the diversity of lighting conditions in the training images may alleviate the performance reduction but does not provide the same benefit from the number increase of images with the same lighting condition. These results provide insights into the impact of image quality and light consistency on CNN performance. The quality threshold established in this study can be used to guide the selection of camera parameters in future weed mapping applications.
Read moreEvaluation of Non-Classical Decision-Making Methods in Self Driving Cars: Pedestrian Detection Testing on Cluster of Images with Different Luminance Conditions
Self-driving cars, i.e., fully automated cars, will spread in the upcoming two decades, according to the representatives of automotive industries; owing to technological breakthroughs in the fourth industrial revolution, as the introduction of deep learning has completely changed the concept of automation. There is considerable research being conducted regarding object detection systems, for instance, lane, pedestrian, or signal detection. This paper specifically focuses on pedestrian detection while the car is moving on the road, where speed and environmental conditions affect visibility. To explore the environmental conditions, a pedestrian custom dataset based on Common Object in Context (COCO) is used. The images are manipulated with the inverse gamma correction method, in which pixel values are changed to make a sequence of bright and dark images. The gamma correction method is directly related to luminance intensity. This paper presents a flexible, simple detection system called Mask R-CNN, which works on top of the Faster R-CNN (Region Based Convolutional Neural Network) model. Mask R-CNN uses one extra feature instance segmentation in addition to two available features in the Faster R-CNN, called object recognition. The performance of the Mask R-CNN models is checked by using different Convolutional Neural Network (CNN) models as a backbone. This approach might help future work, especially when dealing with different lighting conditions.
Read moreAutomatic Image Segmentation of Monocytes and Index Computation Using Deep Learning.
The classification of white cells plays an important part in medical diagnosis. The counts may suggest the presence of infection, inflammation, anemia, bleeding, and other blood-associated issues. More specifically, the counting in our study is the calculation of the Monocyte Index (MI). The purpose of MI is to determine whether the patient can receive units of blood by analyzing the assay. In case of incompatible blood transfusion, monocytes may ingest or adhere red cells. The index is the percentage of red cells adhered, ingested, or both, versus free monocytes. Manual methods for blood cell counting may take several hours and are highly prone to different sources of errors. Automatic methods, such as Linear Discriminant Analysis, Quadratic Discriminant Analysis, K-Nearest Neighbors, Naïve Bayes, Support Vector Machine, Convolutional Neural Network (CNN), Fast Region-based CNN, Faster Region-based CNN, Spatial Pyramidal Pooling network, Single Shot Detector and Mask Region-based CNN, exist for classification. However, these methods currently do not perform automatic counting and calculation of MI. The dataset is our own collection of images using ZEISS Axiocam 208 color/202 mono microscope camera. For the labels in our own collection, we performed polygonal annotation using the VGG Annotator tool. We trained the Mask R-CNN deep neural network model for automatic segmentation at the pixel-level, using COCO pre-trained weights. Our results look promising, as the Mask R-CNN can perform automatic segmentation with 72% accuracy. Compared to a medical laboratory scientist, the model can process large amount of data simultaneously, quickly and efficiently, with approximately the same judgment accuracy as a human eye. This may significantly reduce the burden of the laboratory scientist and provide a useful reference for doctors to identify a potential blood candidate to be transfused.
Read moreA novel directional object detection method for piled objects using a hybrid region-based convolutional neural network
A novel directional object detection method for piled objects using a hybrid region-based convolutional neural network
UAV remote sensing monitoring of pine forest diseases based on improved Mask R-CNN
Timely and accurate monitoring of pine forest diseases is greatly important to pine forest disease prevention and forest protection. A method for unmanned aerial vehicle (UAV) remote sensing monitoring of pine forest diseases is proposed based on improved Mask region-based convolutional neural network (R-CNN) to solve the problem of low accuracy of the existing methods due to complex background and nonevident characteristics of pine forest diseases. The Mask R-CNN with multitask capabilities is utilized as backbone network to build the monitoring model. An improved multiscale receptive field block module is added to the Mask R-CNN to extract the detailed features of pine forest diseases and reduce the missed detection of pine forest diseases. Meanwhile, a multilevel fusion residual pyramid structure is adopted to integrate low-level and high-level features to obtain distinguishable features of pine forest diseases to reduce the misdetection and misidentification of complex backgrounds. In addition, the proposed method uses cutting mixing splicing method to construct training data set to increase the number of background objects in the training images and reduce the impact of complex backgrounds on the monitoring results. Experimental results show that the proposed method can monitor pine forest diseases more accurately than methods of SSD, Faster R-CNN, MaskScoring R-CNN, and the original Mask R-CNN. Compared with the method of backbone network, the precision of the proposed method has increased by 22.4%, the recall has increased by 3.5%, and the F1-score has increased by 14.4%.
Read moreSatellite Image-Based Field Compost Detection Using Deep Learning
With the development of agriculture, the illegal management of field compost, a non-point source of pollution, has become a growing concern as a source of water and environmental pollution.However, detection of field compost through field surveys is difficult and costly.Therefore, following the recent increase in research on the detection and management of field compost, this study aims to detect field compost using high-resolution satellite imagery.We collected satellite image data in the blue, green, red, and near-infrared (NIR) bands over agricultural fields in Gyeongsangnam-do.We labeled unmanaged field compost and evaluated the performance of field compost detection using deep learning models.A total of four models for field compost detection were presented: semantic segmentation detection using U-Net, object segmentation detection using Mask Region-based Convolutional Neural Network (R-CNN), object detection using Faster R-CNN, and a hybrid model combining Faster R-CNN and U-Net for semantic segmentation detection.In the accuracy evaluation based on pixel accuracy and mean Intersection-over-Union (mIoU), the object-based model was more reliable than other models, and the combined model proposed in this paper showed the highest mIoU of 0.68.Based on these results, it is expected that the cost advantage of satellite imagery and the high reliability of field compost detection through unmanned aerial vehicles can be utilized to solve the current problems of field compost detection.In future studies, if the methodology and the quality of satellite images can be improved, accurate field compost detection will be possible.
Read moreFibeR-CNN: Expanding Mask R-CNN to improve image-based fiber analysis
FibeR-CNN: Expanding Mask R-CNN to improve image-based fiber analysis
A comparison of deep learning algorithms on image data for detecting floodwater on roadways
Object detection and segmentation algorithms evolved significantly in the last decade. Simultaneous object detection and segmentation paved the way for real-time applications such as autonomous driving. Detection and segmentation of (partially) flooded roadways are essential inputs for vehicle routing and traffic management systems. This paper proposes an automatic floodwater detection and segmentation method utilizing the Mask Region-Based Convolutional Neural Networks (Mask-R-CNN) and Generative Adversarial Networks (GAN) algorithms. To train the model, manually labeled images with urban, suburban, and natural settings are used. The performances of the algorithms are assessed in accurately detecting the floodwater captured in images. The results show that the proposed Mask-R-CNN-based floodwater detection and segmentation outperform previous studies, whereas the GAN-based model has a straightforward implementation compared to other models.
Read moreGlaucoma Detection using Mask Region-Based Convolutional Neural Networks
Glaucoma is a disease that impairs the vision of the human eyes, and it has grown increasingly common in recent years due to increased pressure in the eyes. It is the result of this disease, which is fatal once it has shown itself, that vision loss occurs. The identification of glaucoma disease has already been achieved via several deep learning (DL) algorithms, as previously stated. The findings of our research on the detection of glaucoma sickness in this paper, in which model used a deep learning technique known as a Convolutional neural network to detect the illness (CNN). The Mask Region-Based convolutional neural network (Mask-RCNN) presents a unique pattern for glaucoma and non-glaucoma affected eyes, which work may utilize to detect glaucoma using machine learning techniques. In this study, the usage of Mask-RCNN is used to create a hierarchical structure for differentiating between images of glaucoma affected and non-affected eyes, allowing for a more accurate categorization of glaucoma patients. With our proposed method may evaluate at six different degrees of complexity and 33 convolutional layers. When detecting glaucoma disease, the proposed research uses the dropout mechanism to enhance the overall efficacy of the performance, as described above. The SCES and ORIGA datasets were used in this research to analyze the intended work, and the results were presented in this paper. The Accuracy of RCNN on the ORIGA dataset was 92.32 percent, Faster-RCNN was 93.89 percent, and Mask RCNN was 95.72 percent, according to the values obtained. When applied to the SCES dataset, the results showed that the Accuracy of the RCNN was 91.56 percent, that of the Faster-RCNN was 94.32 percent, and that of Mask RCNN was 97.56 percent. The mAP of RCNN was 89 percent, Faster-RCNN was 92 percent, and Mask RCNN was 90 percent. mAP of RCNN is 90 percent, Faster-RCNN is 91 percent, and Mask RCNN is 93 percent, according to the results.
Read moreDetection of Electronic Devices in real images using Deep Learning Techniques
Object Detection from real world scenario is a subset of Computer Vision, that uses state-of-the-art algorithms and techniques in deep learning to identify and locate the objects in an image or video. Latest advancements in Deep Learning, especially Convolutional Neural Networks (CNN) and in the field of image processing has further improved the process of object detection. Deep learning algorithms that are developed over the years aim to solve several challenges associated with object detection which includes localizing the object in an image, classifying the object correctly with a high confidence score and realtime detection of objects. The performance of the existing algorithms involves a tradeoff between accuracy and detection speed. Algorithms like Faster Region based Convolutional Neural Networks (R-CNN) and Single Shot Detector (SSD) that achieved high accuracy in classifying objects were slow in detecting the objects. Such algorithms were not able to keep up with the pace of detection with video input in realtime and thus were not suitable for implementation in critical applications. The drawbacks associated with these algorithms can be eliminated by following a unified one-state approach. The approach is to fully identify and classify the required objects of interest by passing the image only once through the network. This approach thus decreases detection time considerably. You Only Look Once (YOLO) family of algorithms is one such single shot detector that uses CNNs to detect objects. In our work, we have used the YOLOv3 algorithm to develop a model that detects electronic devices. The model was also tested against realtime input from webcam and mean Average Precision (mAP) of YOLOv3 has been computed and compared with another model developed using Faster R-CNN.
Read moreObject Detection for Healthcare Data Using Deep Convolutional Neural Networks
Gallstone disease is a prevalent chronic condition impacting individuals worldwide, posing significant challenges to healthcare systems globally. It ranks among the most common ailments encountered by individuals seeking emergency care due to abdominal discomfort. The complexity of gallbladder ultrasound scans arises from numerous factors, including variations in gallbladder anatomy. In this study, we propose a healthcare informatics system aimed at identifying and analyzing gallstones. We conduct a thorough examination of several state-of-the-art object detection algorithms, including Faster Region-based Convolutional Neural Network (Faster RCNN), Mask Region-based Convolutional Neural Network (Mask R-CNN), and Single Shot Detector (SSD) Our approach, which combines elements of Mask R-CNN, SSD, and Faster R-CNN, facilitates the precise detection of gallstones within the gallbladder by leveraging region-based proposals. We specifically focus on training the Mask RCNN model with various backbone networks. Ultrasound images utilized in our experiments were sourced from medical professionals, encompassing diverse demographic characteristics such as gender, age, and urban/rural residence. Our findings demonstrate that the Mask R-CNN model, with a Resnet-101-FPN backbone network, excels in gallstone detection, surpassing alternative techniques in object localization accuracy.
Read moreThe Development of a Skin Cancer Classification System for Pigmented Skin Lesions Using Deep Learning.
Recent studies have demonstrated the usefulness of convolutional neural networks (CNNs) to classify images of melanoma, with accuracies comparable to those achieved by dermatologists. However, the performance of a CNN trained with only clinical images of a pigmented skin lesion in a clinical image classification task, in competition with dermatologists, has not been reported to date. In this study, we extracted 5846 clinical images of pigmented skin lesions from 3551 patients. Pigmented skin lesions included malignant tumors (malignant melanoma and basal cell carcinoma) and benign tumors (nevus, seborrhoeic keratosis, senile lentigo, and hematoma/hemangioma). We created the test dataset by randomly selecting 666 patients out of them and picking one image per patient, and created the training dataset by giving bounding-box annotations to the rest of the images (4732 images, 2885 patients). Subsequently, we trained a faster, region-based CNN (FRCNN) with the training dataset and checked the performance of the model on the test dataset. In addition, ten board-certified dermatologists (BCDs) and ten dermatologic trainees (TRNs) took the same tests, and we compared their diagnostic accuracy with FRCNN. For six-class classification, the accuracy of FRCNN was 86.2%, and that of the BCDs and TRNs was 79.5% (p = 0.0081) and 75.1% (p < 0.00001), respectively. For two-class classification (benign or malignant), the accuracy, sensitivity, and specificity were 91.5%, 83.3%, and 94.5% by FRCNN; 86.6%, 86.3%, and 86.6% by BCD; and 85.3%, 83.5%, and 85.9% by TRN, respectively. False positive rates and positive predictive values were 5.5% and 84.7% by FRCNN, 13.4% and 70.5% by BCD, and 14.1% and 68.5% by TRN, respectively. We compared the classification performance of FRCNN with 20 dermatologists. As a result, the classification accuracy of FRCNN was better than that of the dermatologists. In the future, we plan to implement this system in society and have it used by the general public, in order to improve the prognosis of skin cancer.
Read morePavement Surface Defect Detection Using Mask Region-Based Convolutional Neural Networks and Transfer Learning
Pavement defect detection is critical for pavement maintenance and management. Meanwhile, the accurate and timely detection of pavement defects in complex backgrounds is a huge challenge for maintenance work. Therefore, this paper used a mask region-based convolutional neural network (Mask R-CNN) and transfer learning to detect pavement defects in complex backgrounds. Twelve hundred pavement images were collected, and a dataset containing corresponding instance labels of the defects was established. Based on this dataset, the performance of the Mask R-CNN was compared with faster region-based convolutional neural networks (Faster R-CNNs) under the transfer of six well-known backbone networks. The results confirmed that the classification accuracy of the two algorithms (Mask R-CNN and Faster R-CNN) was consistent and reached 100%; however, the average precision (AP) of the Mask R-CNN was higher than that of Faster R-CNNs. Meanwhile, the testing time of the models using a feature pyramid network (FPN) was lower than that of other models, which reached 0.21 s per frame (SPF). On this basis, the segmentation performance of the Mask R-CNN was further analyzed at three learning rates (LRs). The Mask R-CNN performed best with ResNet101 plus FPN as its backbone structure, and its AP reached 92.1%. The error rate of defect quantification was between 4% and 16%. It has an ideal detection effect on multi-object and multi-class defects on pavement surfaces, and the quantitative results of the defects can provide a reference for pavement maintenance personnel.
Read more