Datasets and Methods for Boosting Infrastructure Inspection: A Survey on Defect Classification

Traditional image object classification and detection algorithms and strategies cannot meet the problem of video image acquisition and processing. Deep learning deliberately simulates the hierarchical structure of human brain, and establishes the mapping from low-level signals to high-level semantics, so as to achieve hierarchical feature representation of data. Deep learning technology has powerful visual information processing ability, which has become the forefront technology and domestic and international research hotspots to deal with this challenge. In order to solve the problem of target space location in video surveillance system, time-consuming and other problems, in this paper, we propose the algorithm based on RNN-LSTM deep learning. At the same time, according to the principle of OpenGL perspective imaging and photogrammetry consistency, we use 3D scene simulation imaging technology, relying on the corresponding relationship between video images and simulation images we locate the target object. In the 3D virtual scene, we set up the virtual camera to simulate the imaging processing of the actual camera, and the pixel coordinates in the video image of the surveillance target are substituted into the simulation image, next, the spatial coordinates of the target are inverted by the inverse process of the virtual imaging. The experimental results show that the detection of target objects has high accuracy, which has an important reference value for outdoor target localization through video surveillance images.

The reliability increase of a photovoltaic (PV) module, thus PV systems, passes through an efficient diagnosis process. This can be achieved by the use of two main techniques such as fault detection and fault classification. Identifying faults by using classification algorithms gives us a better knowledge about a fault nature. However, faults may have the same numeric signature by using conventional methods that consider only a presence or absence of symptoms or residues. This paper proposes a fault detection and classification algorithm using a neuro-fuzzy classifier. The faults being in concerns are relating to series losses, defect by-pass diode and blocking diode faults. The fault diagnosis based parameters are maximum power and, opencircuit voltage deviations, that are used as defect PV system symptoms. The method has been found proficient for fault detection and shows good discrimination for the most frequently occurring PV module faults.

In this study, a machine vision system is developed to achieve fabric inspection and defect classification processes automatically. The system consists of an image acquisition hardware and an image processing software. A simple and portable system was designed so that it can be adapted easily to all types of the fabric inspection machines. The software of the system consists of defect detection and classification algorithms. The defect detection algorithm is based on wavelet transform, double thresholding binarization, and morphological operations. It was applied real time via a user interface prepared by using MATLAB® program. The defect classification approach is based on gray level co-occurrence matrix and feed forward neural network. Five commonly occurring defect types, warp lacking, weft lacking, soiled yarn hole, and yarn flow, were detected and classified. The defective and defect-free regions of the fabric were detected with an accuracy of 93.4% and the defects are classified with 96.3% accuracy rate.

Predicting software defects is an important task during software testing phase, especially for allocating appropriate resources and prioritizing testing tasks. Typically, classification algorithms are used to accomplish this task by using previously collected datasets. However, these datasets suffer from imbalanced label distribution where clean modules outnumber defective modules. Traditional classification algorithms cannot handle this nature in defect datasets because they assume the datasets are balanced. Failing to address this problem, the classification algorithm will produce a prediction biased towards the majority label. In the literature, there are several techniques designed to address this problem and most of them focus on data re-balancing. Recently, ensemble class imbalance techniques have emerged as an opposing approach to data rebalancing approaches. Regarding the software defect prediction, there are no studies examining the performance of ensemble class imbalance learning against data re-balancing approaches. This paper investigates the efficiency of ensemble class imbalance learning for software defect prediction. We conducted a comprehensive experiment that involved 12 datasets, six classifiers, nine class imbalance techniques, and 10 evaluation metrics. The experiments showed that ensemble approaches, particularly the Under Bagging technique, outperform traditional data re-balancing approaches, particularly when dealing with datasets that have high defect ratios.

The availability of advanced sensors and computing enables early detection of faults and failures in large-scale solar photovoltaic (PV) systems. However, small variations in current and voltage caused by arcing or other faults/failures are often difficult to detect, potentially leading to a permanent reduction in power output and can potentially cause safety hazard. This work develops novel algorithm for detection and classification of DC arc and AC faults using Conditional Generative Adversarial Networks (CGANs). Three distinct CGAN architectures are developed: two for DC arc fault detection and classification, and another for AC faults. The CGAN networks generate output signals representing probability distributions based on voltage and current measurements at the inverter location of the PV system. The performance of the proposed CGAN architectures is evaluated using rolling precision scores on backcasted datasets and compared against a k-Nearest Neighbors (kNN) model, and also validated using real field data. Results show that the proposed architectures achieve high precision in detecting and classifying both types of faults in a 1 MW solar PV system modeled in a real-time simulator.

To prevent fatal vehicle accidents under various driving conditions, it is essential to develop a fail-safe system that integrates hardware and software. In this paper, fault detection, classification, and fault-tolerant steering algorithms are proposed based on the sliding mode approach and relative error method to deal with driving system faults in four-wheel independent steering and driving (4WISD) vehicles. First, a sliding mode observer (SMO) is designed to estimate the uncertainty of driving systems using an injection term, and the relative errors are computed using the injection term for fault detection. The injection term is used for recursive least squares (RLS) to estimate the longitudinal tire force and the differences in the damping coefficient and wheel torque. Second, a fault classification algorithm is designed to identify driving system faults based on the differences in the calculated relative errors. Third, the sliding mode control (SMC) is designed with the finite-time stability condition to compute the steering control inputs of each wheel for fault-tolerant steering. A control allocation rule consisting of four proportional constants is designed based on Lyapunov stability conditions. The performance was evaluated via co-simulation using the MATLAB/Simulink and CarMaker software to verify the performance of the proposed algorithms under a step steering scenario with two fault conditions.

Composite materials are widely used in aerospace industries due to their light weight, strength, and various other desired properties. However, they are susceptible to various defects occurring during the manufacturing process or in service. Typical defects include porosity, wrinkles, and delamination. Nondestructive means of detection of the defects at any stage are of great importance to ensure quality and safety of composite structures. A nonintrusive removable Lamb wave system and accompanied methodology that is not material dependent are presented in this paper to detect various types of typical defects in laminated composite plates, flat or curved. Through multidimensional data acquisition and processing, abnormality in waves caused by defects is captured and presented in inspection images. The methodologies are demonstrated in two cases: delamination in a curved plate and wrinkles in a flat plate. Overall, the results show that Lamb waves using the piezoelectric transducer and laser vibrometer system can be used for various types of defects inspection in flat or curved composite plates.

The method based on deep learning shows excellent performance in the recognition and classification of surface defects of some industrial products. The method based on deep learning has high efficiency in the identification and classification of surface defects of industrial products, and the false detection rate and missed detection rate are relatively low. However, the recognition accuracy of defect detection and classification of most industrial products needs to be improved, especially for those with similar contours and relatively large structural different casting. This paper takes casting defect detection as the goal and proposes a convolutional neural network casting defect detection and classification (RCNN-DC) algorithm based on the recursive attention model. Through this model, the casting can be better identified and detected, and casting defects can be avoided as much as possible, which is of great significance to the technological development of the industry. First, use a large amount of readily available defect-free sample data to detect anomalous defects. Next, we compare the accuracy and performance of the detection model and the general recognition model. The research results show that the test effect of the RCNN-DC casting defect detection network model is significantly better than the traditional detection model, with a classification accuracy of 96.67%. Then, we compare the RCNN-DC network with three classic popular networks, GooGleNet, ResNet-50, and AlexNet. Among them, AlexNet and ResNet-50 achieved 95.00% and 95.56% classification accuracy, respectively, while GooGleNet achieved slightly better results of 96.38%. In contrast, the accuracy of RCNN-DC is 1.67% higher than that of AlexNet, while the number of FLOPs is reduced by 17.2 times, and the accuracy is 1.09% higher than that of ResNet-50, while the number of FLOPs is reduced by 99.7 times, and the accuracy is higher than GooGleNet 0.29% while FLOPs whose number has been reduced by 36.5 times.

Defect inspection is indispensable process in manufacturing, and automatic optical inspection (AOI) has been rapidly applied to various areas. In AOI, artificial intelligence (AI) based deep learning methods are more and more advantageous in many fields. However, obtainment of high-performance deep learning algorithms always requires a large amount of training data, while defect samples are often scarce. So small sample has become one of the key problems in the industrial application of deep learning algorithms. Transfer learning enable us to utilize the knowledge of source domains to improve performance on target domain, which could be used to tackle the small sample problem intuitively. Therefore, this paper proposes a defect inspection network which is based on one of the transfer learning techniques: domain adaptation. We name the network as multi-source and multi-scale weighted domain adaptation network which is based on adversarial learning. Firstly, three adversarial domain adaptation modules are proposed to align feature distributions between multi-source domains and target domain under three scales, which make the backbone extract domain-invariant features. Simultaneously, the weights of domain adaptation module under each scale are set reasonably. Secondly, in order to reduce the effect of negative transfer, a novel similarity weight is proposed, which is applied on domain adaptation modules. Finally, experiments are carried out to prove the effectiveness of our method. The results show that our method can improve the mean average precision(mAP) from 62.3 to 78.5 in the case of 40 samples available for 4 defect categories, which surpasses other counterparts.

This project focuses on developing an autonomous self-driving car by integrating advanced algorithms with cutting-edge hardware. The vehicle is designed to perceive its environment, make informed decisions, and navigate complex traffic scenarios autonomously by leveraging machine learning and computer vision technologies. Essential methods include using deep learning algorithms for object detection and classification, sensor fusion techniques to combine data from cameras, LiDAR, and radar, and reinforcement learning for decision-making in dynamic environments. The project emphasizes the critical role of AI in enabling the car to interpret and respond to dynamic road conditions while ensuring reliable performance through rigorous testing and validation in simulated and real-world scenarios. Initial results from testing indicate that the car can successfully navigate complex traffic situations, such as merging onto highways and responding to pedestrians, with a high degree of accuracy. By pushing the boundaries of autonomous vehicle technology, this research aspires to contribute to the ongoing evolution of self-driving cars and their potential to revolutionize mobility. With a focus on safety, efficiency, and innovation, this endeavor represents a significant step forward in developing autonomous vehicles, offering new possibilities for the future of transportation. The project promises to improve travel convenience and significantly reduce traffic accidents, ultimately contributing to a safer and more efficient road system.

The segmentation, detection and extraction of malignant tumor regions from magnetic resonance (MR) images are challenging tasks in medical image analysis. Approaches based on machine and deep learning have been introduced, which performed better than traditional image processing methods. However, many approaches still show limited ability due to the complex dataset and image modalities. This study evaluated the deep learning approach's performance and traditional image processing algorithms for medical imaging segmentation, detection, and classification. The proposed system comprises multiple stages. The Median filters are used in the pre-processing step, and morphological operation and Otsu thresholding are used to segment MR images. Discrete Wavelet Transform (DWT) algorithm is considered in the extraction features, and their classification is executed by a convolutional neural network (CNN) and support vector machine (SVM) algorithms. The Mat lab has been used for simulation and experimental findings to evaluate the suggested method's performance on the brain's complex and highly 2D structures. The results show that the methodology is reliable and efficient, with 93.5% accuracy.

Solar cells defects inspection plays an important role to ensure the efficiency and lifespan of photovoltaic modules. However, it is still an arduous task because of the diverse attributes of electroluminescence images, such as indiscriminative complex background with extremely unbalanced defects and various types of defects. In order to deal with these problems, this paper proposes a new precise and accurate defect inspection method for photovoltaic electroluminescence (EL) images. The proposed algorithm leverages the advantage of multi attention network to efficiently extract the most important features and neglect the nonessential features during training. Firstly, we designed a channel attention to exploit contextual representations and spatial attention to effectively suppress background noise. Secondly, we incorporate both attention networks into modified U-net architecture and named it multi attention U-net (MAU-net) to extract effective multiscale features for defects inspection. Finally, we propose a hybrid loss which combines focal loss and dice loss aiming to solve two problems: a) overcome the class imbalance problem, and b) allowing the network to train with irregular image labels for some complex defects. The proposed multi attention U-net is evaluated on real photovoltaic EL images datasets using 5-fold cross validation technique. Experimental results demonstrate that the proposed network can segment and detect various complex defects correctly. The proposed method achieved the mean intersection over-union (m-IOU) of 0.699 and F-measure of 0.799 which outperforms the previous methods.

Simple SummaryArtificial Intelligence (AI) algorithms can assist clinicians in their daily tasks by automatically detecting and/or classifying nodules in chest CT scans. Bias of such algorithms is one of the reasons why implementation of them in clinical practice is still not widely adopted. There is no published review on the bias that these algorithms may contain. This review aims to present different types of bias in such algorithms and present possible ways to mitigate them. Only then it would be possible to ensure that these algorithms work as intended under many different clinical settings.Artificial Intelligence (AI) algorithms for automatic lung nodule detection and classification can assist radiologists in their daily routine of chest CT evaluation. Even though many AI algorithms for these tasks have already been developed, their implementation in the clinical workflow is still largely lacking. Apart from the significant number of false-positive findings, one of the reasons for that is the bias that these algorithms may contain. In this review, different types of biases that may exist in chest CT AI nodule detection and classification algorithms are listed and discussed. Examples from the literature in which each type of bias occurs are presented, along with ways to mitigate these biases. Different types of biases can occur in chest CT AI algorithms for lung nodule detection and classification. Mitigation of them can be very difficult, if not impossible to achieve completely.

Currently, the radiologist can more accurately identify brain tumours through the development of Computer-Assisted Diagnosis (CAD), Machine Learning and Deep Learning. Recently, Deep Learning (DL) strategies have gained traction as a means to rapidly and accurately construct automated systems for diagnosing and segmenting the image. The standard approach to this issue is to create a custom feature for classification. Most neurological diseases originate from abnormal growth of brain cells, which can compromise brain architecture and even lead to malignant brain tumours. Brain tumour detection and classification algorithms that are both quick and accurate have been the subject of extensive study. This facilitates the straight forward diagnosis of brain tumours using Magnetic Resonance Image (MRI) images. Through Deep Learning (DL) model the diagnosis of brain malignancies in MRI images using Convolutional Neural Network (CNN) is possible by training the data. So, in this paper the brain tumouris predicted byproposing a Hybridfeature extraction technique i.e., tuned CNN model with ResNet150 and U-net.

A complete blood cell count is an important test in medical diagnosis to evaluate overall health condition. Traditionally blood cells are counted manually using haemocytometer along with other laboratory equipment's and chemical compounds, which is a time-consuming and tedious task. In this work, the authors present a machine learning approach for automatic identification and counting of three types of blood cells using ‘you only look once’ (YOLO) object detection and classification algorithm. YOLO framework has been trained with a modified configuration BCCD Dataset of blood smear images to automatically identify and count red blood cells, white blood cells, and platelets. Moreover, this study with other convolutional neural network architectures considering architecture complexity, reported accuracy, and running time with this framework and compare the accuracy of the models for blood cells detection. They also tested the trained model on smear images from a different dataset and found that the learned models are generalised. Overall the computer-aided system of detection and counting enables us to count blood cells from smear images in less than a second, which is useful for practical applications.