Research for Defect Detection based on Few-Shot Learning of Siamese Network

Internal defects detection and classification in hollow cylindrical surfaces using single shot detection and MobileNet

We propose the Multi-Dropout Regression Network (MDRN) for real-time facial landmark localization across extreme poses. Different from most landmark localization methods only work for -45° to 45° in yaw, the proposed MDRN works for the full coverage of -90° to 90°. It employs the Single Shot Multibox Detector (SSD) [1] as a preprocessor for fast and accurate face detection. Given an SSD detected face, the MDRN locates the landmarks. The MDRN is composed of 2 double-layer convolution blocks, 1 triple-layer convolution block and 3 fully-connected layers. Unlike most networks with only one dropout layer connected to the last convolution layer, in the MDRN each convolution block is followed by a max-pooling layer and a dropout layer ahead of connecting to the next processing layer. Experiments reveal that multiple dropouts better stabilize the regression and improve the accuracy of landmark localization. To locate the landmarks on profile faces and other extreme poses, the MDRN is trained on an augmented database composed of imagery of synthesized poses. A comparison study shows that the proposed solution delivers a comparable performance to the state of the art for wide-angle landmark localization.

Vision-based structural displacement monitoring of rotor has attracted extensive attention from scholars due to its non-contact, non-damage, and multi-point synchronous measurement advantages. However, there is a lack of one comprehensive dataset and benchmark for rotor detection since the problem of limited acquisition equipment or on-site environment in the industrial field. For that, a large benchmark for rotor’s vibration displacement detection is introduced in this paper. Firstly, a rotor vibration displacement detection network (RDNet) for rotating structures is proposed and state-of-the-art performance is achieved. Secondly, the evaluation criteria are designed to evaluate the rotor image detection accuracy using the mean Average Precision@0.5:0.95 (mAP@0.5:0.95), the Frames Per Second (FPS), and the Normalized Root Mean Squared Error (NRMSE) as performance indicators. Thirdly, a dataset namely Rotor V1.0 is built, which contains three frames (high, middle, and low) of 13,500 rotor images. It is the rotor dataset for vibration displacement monitoring of rotating structures. Lastly, the baseline performance resulted from traditional methods (i.e., match template and support vector machines) and deep learning methods (i.e., CenterNet, Faster Region-convolutional neural networks (Faster R-CNN), RetinaNet, Single Shot MultiBox Detector (SSD), and You only look once (YOLO)) are reported and analyzed. It is hoped that RDNet, evaluation criteria, and the novel Rotor V1.0 dataset can promote the implementation of visual vibration measurement tasks in industrial sites.

The automatic parking system based on vision is greatly affected by uneven lighting, which is difficult to make an accurate judgment on parking spaces in the case of complex image information. To solve this problem, this paper proposes a parking space visual detection and image processing method based on deep learning. Firstly, a 360-degree panoramic system was designed to photograph the vehicle environment. The image has been processed to obtain a panoramic aerial view, which was input as the original image of the parking space detection system. Secondly, the Faster R-CNN (Region-Convolutional Neural Network) parking detection model was established based on deep learning. It was aimed to detect and extract the parking space from the input image. Thirdly, the problems of uneven illumination and complex background were solved effectively by removing the background light from the image. Finally, a parking space extraction method based on the connected region has been designed, which further simplified the parking space extraction and image processing. The experiment results show that the mAP (mean Average Precision) value of the Faster R-CNN model using 101-Floor ResNet as the feature extraction network is 89.30%, which is 2.28% higher than that of the Faster R-CNN model using 50-Floor ResNet as the feature extraction network. The model built in this paper can detect most parking spaces well. The position of the output target box is accurate. In some test scenarios, the confidence of parking space recognition can even reach 100%. In summary, the proposed method can realize the effective identification and accurate positioning of parking spaces.

The aim of this study was to evaluate the performance of a clinical decision support system (CDSS) for therapeutic plans in geriatric dentistry. The information that needs to be considered in a therapeutic plan includes not only the patient's oral health status obtained from an oral examination, but also other related factors such as underlying diseases, socioeconomic characteristics, and functional dependency. A Bayesian network (BN) was used as a framework to construct a model of contributing factors and their causal relationships based on clinical knowledge and data. The faster R-CNN (regional convolutional neural network) algorithm was used to detect oral health status, which was part of the BN structure. The study was conducted using retrospective data from 400 patients receiving geriatric dental care at a university hospital between January 2020 and June 2021. The model showed an F1-score of 89.31%, precision of 86.69%, and recall of 82.14% for the detection of periodontally compromised teeth. A receiver operating characteristic curve analysis showed that the BN model was highly accurate for recommending therapeutic plans (area under the curve = 0.902). The model performance was compared to that of experts in geriatric dentistry, and the experts and the system strongly agreed on the recommended therapeutic plans (kappa value = 0.905). This research was the first phase of the development of a CDSS to recommend geriatric dental treatment. The proposed system, when integrated into the clinical workflow, is expected to provide general practitioners with expert-level decision support in geriatric dental care.

In modern days, increasing weapon-related threats in public places have created an immediate need for intelligent surveillance systems to detect crime in real-time. Traditional surveillance systems have struggles with recognizing small objects, occlusion, and the time it takes to respond, which makes them ineffective in crowded and fast-changing situations. To overcome these challenges, the suggested system combines closed-circuit television (CCTV) surveillance cameras with advanced deep learning methods, image processing, and computer vision techniques for real-time crime prediction and prevention. This study proposes a hybrid deep learning framework that merges a Faster region convolutional neural network and Mask Region Convolutional Neural Network, named FMR-CNN. The novel approach FMR-CNN represents a significant advancement towards improving object recognition and segmentation of images and videos. It has been combined with YOLOv8 to increase the real-time detection speed and localization accuracy significantly. Such a combination enables the concurrent utilization of high-resolution spatial context information and rapid frame-wise predictions, thus making it well-suited for continuous video surveillance tasks. The model was trained and tested on a five labeled class annotated dataset, where MobileNetV3 features are extracted to simulate real-world surveillance conditions. Experimental results show the hybrid model attains detection accuracy of 98.7%, average precision (AP) of 90.1, and speed of 9.2 frames per second (FPS), and generalizes to varied lighting, occlusion, object scales, and reduced computational complexity, making it highly effective for crime prevention. Using these models benefits police departments and law enforcement agencies, as it allows them to detect criminal offenses earlier and avoid untoward situations.

Bus passengers' data are crucial for the bus agents, especially in Indonesia. With this data, bus agents could identify the traffic for each route of the bus. To handle this problem, many researchers have made a system to count and detect the public transportation passengers with different algorithms. Many researchers defined that You Look Only Once (YOLO) has best performance to overcome the object detection problem that has similarity in this research. The Convolutional Neural Network algorithm is also not inferior in implementing object detection either. In this research, it will investigate these three algorithms, You Only Look Once version 8 (YOLOv8), Faster Region Convolutional Neural Network (Faster R-CNN), and Mask Region Convolutional Neural Network (Mask R-CNN), in counting bus passengers and detecting the bus passenger’s gender. To find the best performance of these two algorithms, they will use a dataset that contains 408 photos of bus passengers. This research aims to analyze the result of the bus passengers data that could reduce the misalignment and determine the best algorithm to use in this case.

PurposeTo apply a deep learning approach to automatically detect implanted seeds on a fluoroscopy image in prostate brachytherapy.Material and methodsForty-eight fluoroscopy images of patients, who underwent permanent seed implant (PSI) were used for this study after our Institutional Review Boards approval. Pre-processing procedures that were used to prepare for the training data, included encapsulating each seed in a bounding box, re-normalizing seed dimension, cropping to a region of prostate, and converting fluoroscopy image to PNG format. We employed a pre-trained faster region convolutional neural network (R-CNN) from PyTorch library for automatic seed detection, and leave-one-out cross-validation (LOOCV) procedure was applied to evaluate the performance of the model.ResultsAlmost all cases had mean average precision (mAP) greater than 0.91, with most cases (83.3%) having a mean average recall (mAR) above 0.9. All cases achieved F1-scores exceeding 0.91. The averaged results for all the cases were 0.979, 0.937, and 0.957 for mAP, mAR, and F1-score, respectively.ConclusionsAlthough there are limitations shown in interpreting overlapping seeds, our model is reasonably accurate and shows potential for further applications.

SummaryLung cancer is the most deadly type of cancer and it is caused by genetic variations in lung tissues. Other causes of lung cancer are alcohol, smoking, and threatening gas exposures. The diagnosis of lung cancer is an intricate task and early detection of lung cancer can help to get the exact treatment in advance. The application of a computer‐aided diagnosis process helps to predict lung cancer earlier, nonetheless, it does not provide better accuracy. The overfitting nature of features and dimensionality of lung cancer can prevent it from obtaining maximum accuracy. Hence, we proposed a novel faster region convolutional neural network (RCNN) based fuzzy butterfly optimization algorithm (FBOA) to achieve better prediction accuracy and effectiveness. The proposed Faster RCNN can provide better positioning of lung cancer swiftly and effectively and the FBOA approach can be used to perform two‐stage classification. The fuzzy rules used in the FBOA can be utilized to find the severity of the lung cancer and differentiate the Benign stage and Malignant stage effectively. The experimental analyses are performed in MATLAB simulation. The preprocessing of images is performed by different tools from MATLAB and is to format the images as required. The cancer imaging archive (TCIA) dataset is utilized to analyze the performance of the proposed method and compared with various state‐of‐art works. The performance of the proposed method is evaluated by using different evaluation metrics such as precision, recall, F‐measure, and accuracy and attained 99, 98, 99, and 97% respectively. Thus, our proposed method outperforms all the other approaches.

Recognizing objects and estimating their poses have a wide range of application in robotics. For instance, to grasp objects, robots need the position and orientation of objects in 3D. The task becomes challenging in a cluttered environment with different types of objects. A popular approach to tackle this problem is to utilize a deep neural network for object recognition. However, deep learning-based object detection in cluttered environments requires a substantial amount of data. Collection of these data requires time and extensive human labor for manual labeling. In this study, our objective was the development and validation of a deep object recognition framework using a synthetic depth image dataset. We synthetically generated a depth image dataset of 22 objects randomly placed in a 0.5 m × 0.5 m × 0.1 m box, and automatically labeled all objects with an occlusion rate below 70%. Faster Region Convolutional Neural Network (R-CNN) architecture was adopted for training using a dataset of 800,000 synthetic depth images, and its performance was tested on a real-world depth image dataset consisting of 2000 samples. Deep object recognizer has 40.96% detection accuracy on the real depth images and 93.5% on the synthetic depth images. Training the deep learning model with noise-added synthetic images improves the recognition accuracy for real images to 46.3%. The object detection framework can be trained on synthetically generated depth data, and then employed for object recognition on the real depth data in a cluttered environment. Synthetic depth data-based deep object detection has the potential to substantially decrease the time and human effort required for the extensive data collection and labeling.

The fabric defect detection algorithm based on object detection has become a research hotspot. The method based on the Single Shot MultiBox Detector (SSD) model has a fast detection speed, but the detection accuracy is insufficient. To balance the detection speed and accuracy of the model and meet the actual needs of the industry, an improved fabric defect detection algorithm based on SSD is proposed in this study. The Fully Convolutional Squeeze-and-Excitation (FCSE) block is added into the traditional SSD to improve the detection accuracy of the model. The number of default boxes was adjusted to accommodate the detection of long strip defects on fabric surface. Experimental results on the TILDA and Xuelang dataset confirm that our detection method based on SSD efficiently detected various fabric defects.

Electrolytic capacitor surface defect detection based on deep convolution neural network

When the wearer of Wearable Human-Machine Interaction System (WHMIS) is talking a strenuous walk, the Inertial Measurement Units (IMUs) installed on the extremities are likely to be beyond the range of design specification of the IMU sensors. This overrange leads to poor navigation performances of the system. To overcome the problem, this paper proposes a deep learning-based method to realize robust autonomous navigation. Specifically, the Faster Region-Convolutional Neural Network (R-CNN) is built to recognize the gait types of the wearer. For each gait type, the inertial data of the wearer's thigh and the foot are collected at the same frequency as the training samples, and then used to construct a correspondent Gated Recurrent Unit-Support Vector Regression (GRU-SVR) deep hybrid neural network model for virtual IMU simulation. Then, the virtual foot-mounted IMU will be constructed in real-time by the physical thigh-mounted IMU based on the neural network models mentioned above. Once the over-range of the physical foot-mounted IMU is detected, the virtual foot inertial information will be used to reconstruct the WHMIS navigation system. Experimental results show that the proposed navigation method has the ability of fault tolerance in complex gaits, and the 3D positioning performance of the reconstructed navigation system is approximately equivalent to that of the fault-free WHMIS navigation system.

Pedestrians, motorist, and cyclist remain the victims of poor vision and negligence of human drivers, especially in the night. Millions of people die or sustain physical injury yearly as a result of traffic accidents. Detection and recognition of road markings play a vital role in many applications such as traffic surveillance and autonomous driving. In this study, we have trained a nighttime road-marking detection model using NIR camera images. We have modified the VGG-16 base network of the state-of-the-art faster R-CNN algorithm by using a multilayer feature fusion technique. We have demonstrated another promising feature fusion technique of concatenating all the convolutional layers within a stage to extract image features. The modification boosts the overall detection performance of the model by utilizing the advantages of the shallow layers and the deep layers of the VGG-16 network. The training samples were augmented using random rotation and translation to enhance the heterogeneity of the detection algorithm. We have achieved a mean average precision (mAP) of 89.48% and 92.83% for the baseline faster R-CNN and our modified method, respectively.

Metallic surface defect detection is of great significance in quality control for production. However, this task is very challenging due to the noise disturbance, large appearance variation, and the ambiguous definition of the defect individual. Traditional image processing methods are unable to detect the damaged region effectively and efficiently. In this paper, we propose a new defect detection method, Attention Cascade R- CNN with Mix-NMS (ACRM), to classify and locate defects robustly. Three submodules are developed to achieve this goal: 1) a lightweight attention block is introduced, which can improve the ability in capture global and local feature both in the spatial and channel dimension; 2) we firstly apply the cascade R-CNN to our task, which exploits multiple detectors to sequentially refine the detection result robustly; 3) we introduce a new method named Mix Non-Maximum Suppression (Mix- NMS), which can significantly improve its ability in filtering the redundant detection result in our task. Extensive experiments on a real industrial dataset show that ACRM achieves state-of-the- art results compared to the existing methods, demonstrating the effectiveness and robustness of our detection method.