Surface Defect Detection for Automated Inspection Systems using Convolutional Neural Networks

The use of Convolutional Neural Networks (CNNs) for image classification is wellestablished in defect detection within manufacturing environments. However, their application in assembly processes remains underexplored. This study investigates the effectiveness of using relatively smaller CNNs for detecting assembly defects by testing multiple existing pre-trained CNNs and a custom CNN on a dataset of model train seat assembly images. The dataset includes common assembly defects such as missing, rotated, and swapped parts. Despite implementing various data augmentation and image processing techniques, such as cropping, normalization, and image transforms, and making several modifications to the models, the existing techniques struggled to accurately predict defects while providing the correct rationale for their predictions. The Grad-CAM analysis revealed that the models often focused on irrelevant features, highlighting the challenges of defect detection in complex assembly environments. These findings indicate the need for more robust machine learning approaches capable of handling high levels of noise and variations typical of realworld assembly conditions. This study underscores the limitations of current CNN-based defect detection methods in uncontrolled assembly settings and the necessity for further research to develop more reliable solutions.

Leaf disease detection is a crucial task in modern agriculture, aiding in early diagnosis and prevention of crop infections. In this research paper, authors present a comprehensive study comparing nine widely used pre-trained models, namely DenseNet201, EfficientNetB3, EfficientNetB4, InceptionResNetV2, MobileNetV2, ResNet50, ResNet152, VGG16, and Xception, with our newly developed custom CNN (Convolutional Neural Network) for leaf disease detection. The objective is to determine if our custom CNN can match the performance of these established pre-trained models while maintaining superior efficiency. The authors trained and fine-tuned each pre-trained model and our custom CNN on a large dataset of labeled leaf images, covering various diseases and healthy states. Subsequently, the authors evaluated the models using standard metrics, including accuracy, precision, recall, and F1-score, to gauge their overall performance. Additionally, the authors analyzed computational efficiency regarding training time and memory consumption. Surprisingly, our results indicate that the custom CNN performs comparable to the pre-trained models, despite their sophisticated architectures and extensive pre-training on massive datasets. Moreover, our custom CNN demonstrates superior efficiency, outperforming the pre-trained models regarding training speed and memory requirements. These findings highlight the potential of custom CNN architectures for leaf disease detection tasks, offering a compelling alternative to the commonly used pre-trained models. The efficiency gains achieved by our custom CNN can be beneficial in resource-constrained environments, enabling faster inference and deployment of leaf disease detection systems. Overall, our research contributes to the advancement of agricultural technology by presenting a robust and efficient solution for the early detection of leaf diseases, thereby aiding in crop protection and yield enhancement.

Content filtering is gaining popularity due to easy exposure of explicit visual contents to the public. Excessive exposure of inappropriate visual contents can cause devastating effects such as the growth of improper mindset and rise of societal issues such as free sex, child abandonment and rape cases. At present, most of the broadcasting media sites are hiring censorship editors to label graphic contents manually. Nevertheless, the efficiency is limited by factors such as the attention span of humans and the training required for the editors. This paper proposes to study the effect of usage of Convolutional Neural Network (CNN) as feature extractor coupled with Support Vector Machine (SVM) as classifier in an automated pornographic detection system. Three CNN architectures: Mobile Net, Visual Geometry Group-19 (VGG-19) and Residual Network-50 Version 2 (ResNet50_V2), and two classifiers: CNN and SVM were utilized to explore the combination that produce the best result. Frames of films fed as input into the CNN were classified into two groups: porn or non-porn. The best accuracy was 92.80% obtained using fine-tuned ResNet50_V2 as feature extractor and SVM as classifier. Transfer learning and SVM have improved the CNN model by approximately 10%.

With the rise of crimes associated with Automated Teller Machines (ATMs), security reinforcement by surveillance techniques has been a hot topic on the security agenda. As a result, cameras are frequently installed with ATMs, so as to capture the facial images of users. The main objective is to support follow-up criminal investigations in the event of an incident. However, in the case of miss-use, the user’s face is often occluded. Therefore, face occlusion detection has become very important to prevent crimes connected with ATM usage. Traditional approaches to solving the problem typically comprise a succession of steps: localization, segmentation, feature extraction and recognition. This paper proposes an end-to-end facial occlusion detection framework, which is robust and effective by combining region proposal algorithm and Convolutional Neural Networks (CNN). The framework utilizes a coarse-to-fine strategy, which consists of two CNNs. The first CNN detects the head element within an upper body image while the second distinguishes which facial part is occluded from the head image. In comparison with previous approaches, the usage of CNN is optimal from a system point of view as the design is based on the end-to-end principle and the model operates directly on image pixels. For evaluation purposes, a face occlusion database consisting of over 50[Formula: see text]000 images, with annotated facial parts, was used. Experimental results revealed that the proposed framework is very effective. Using the bespoke face occlusion dataset, Aleix and Robert (AR) face dataset and the Labeled Face in the Wild (LFW) database, we achieved over 85.61%, 97.58% and 100% accuracies for head detection when the Intersection over Union-section (IoU) is larger than 0.5, and 94.55%, 98.58% and 95.41% accuracies for occlusion discrimination, respectively.

Although many machine learning methods have been widely used to predict PM2.5 concentrations, these single or hybrid methods still have some shortcomings. This study integrated the advantages of convolutional neural network (CNN) feature extraction and the regression ability of random forest (RF) to propose a novel CNN-RF ensemble framework for PM2.5 concentration modeling. The observational data from 13 monitoring stations in Kaohsiung in 2021 were selected for model training and testing. First, CNN was implemented to extract key meteorological and pollution data. Subsequently, the RF algorithm was employed to train the model with five input factors, namely the extracted features from the CNN and spatiotemporal factors, including the day of the year, the hour of the day, latitude, and longitude. Independent observations from two stations were used to evaluate the models. The findings demonstrated that the proposed CNN-RF model had better modeling capability compared with the independent CNN and RF models: the average improvements in root mean square error (RMSE) and mean absolute error (MAE) ranged from 8.10% to 11.11%, respectively. In addition, the proposed CNN-RF hybrid model has fewer excess residuals at thresholds of 10 μg/m3, 20 μg/m3, and 30 μg/m3. The results revealed that the proposed CNN-RF ensemble framework is a stable, reliable, and accurate method that can generate superior results compared with the single CNN and RF methods. The proposed method could be a valuable reference for readers and may inspire researchers to develop even more effective methods for air pollution modeling. This research has important implications for air pollution research, data analysis, model estimation, and machine learning.

Pest infestation is a significant challenge in agriculture, causing substantial economic losses and impacting crop yields. Manual pest identification is time-consuming and error-prone, especially in large-scale farming. This study presents a hybrid deep learning approach to automate pest detection and classification using image data. It employs a pre-trained VGG16 model for robust feature extraction, followed by a custom Convolutional Neural Network (CNN) for classification. The VGG16 model leverages transfer learning to capture high-level visual patterns from pest images, while the custom CNN is trained on these features to distinguish between multiple pest classes. The proposed pipeline is optimized using data augmentation, early stopping, and adaptive learning techniques. Performance is evaluated using metrics such as accuracy, precision, recall, and F1-score. The model demonstrates high accuracy and robustness on agricultural image datasets, making it a scalable solution for smart farming systems. Index Terms: Pest Detection, Deep Learning, VGG16, Convolutional Neural Network, Transfer Learning, Image Classification, Precision Agriculture, Feature Extraction.

It is observed that pneumonia is the largest cause of death in toddlers. Early detection would facilitate in early treatment, hence, saving many lives. This paper intend to develop a method for automating the detection of pneumonia using chest X-rays and comparing convolutional neural network and multilevel perceptron. Custom Convolutional neural network and multi-layer perceptron were implemented on a chest X-ray dataset by Kaggle. A GUI was created that accepts the chest X-ray and predicts the presence of pneumonia and gives the congestion percentage. The model accuracy was 92.63% and 77.56% for convolutional neural network and multilayer perceptron respectively. The GUI developed gives favourable outcomes. Convolutional neural network showed better results as compared to multi-layer perceptron. Thus, the custom convolutional neural network was used for the GUI.

Palm tree detection using remote sensing images has received increasing attention in recent years, concerning the issues of sustainability, productivity and profitability. There has been significant progress in the research of using machine learning techniques, especially convolutional neural networks (CNNs) for automatic palm tree detection. However, whether CNNs can actually outperform traditional human-engineered approaches in terms of classification accuracy and detection speed is yet unknown. In the present study, we have compared human-engineered features namely histogram of oriented gradients (HOG), local binary pattern (LBP) and scale-invariant feature transform (SIFT) with features extracted using pre-trained AlexNet model for detecting palm trees in high resolution images obtained via unmanned aerial vehicle (UAV). Support vector machines (SVM) with linear and non-linear kernels were used to classify feature vectors obtained by different feature extractors. Haar-like features used in Viola-Jones framework was also tested in the study. Results showed that features extracted from the fifth convolutional layer of AlexNet achieved the highest accuracy of 96.1% using SVM with RBF kernel as classifier, which surpassed the accuracy obtained by fully-connected CNN, namely 95.6%. The results also suggest that SVM classifier with radial basis function (RBF) kernel using LBP as features is the optimal combination as it achieved comparable accuracy but higher detection speed than CNN approaches.

The widespread and growing use of flying unmanned aerial vehicles (UAVs) is attributed to their high spatial mobility, autonomous control, and lower cost compared to usual manned flying vehicles. Applications, such as surveying, searching, or scanning the environment with application-specific sensors, have made extensive use of UAVs in fields like agriculture, geography, forestry, and biology. However, due to the large number of applications and types of UAVs, limited power has to be taken into account when designing task-specific software for a target UAV. In particular, the power constraints of smaller UAVs will generally necessitate reducing power consumption by limiting functionality, decreasing their movement radius, or increasing their level of autonomy. Reducing the overhead of control and decision-making software onboard is one approach to increasing the autonomy of UAVs. Specifically, we can make the onboard control software more efficient and focused on specific tasks, which means it will need less computing power than a general-purpose algorithm. In this work, we focus on reducing the size of the computer vision object classification algorithm. We define different tasks by specifying which objects the UAV must recognize, and we construct a convolutional neural network (CNN) for each specific classification. However, rather than creating a custom CNN that requires its dataset, we begin with a pre-trained general-purpose classifier. We then choose specific groups of objects to recognize, and by using response-based pruning (RBP), we simplify the general-purpose CNN to fit our specific needs. We evaluate the pruned models in various scenarios. The results indicate that the evaluated task-specific pruning can reduce the size of the neural model and increase the accuracy of the classification tasks. For small UAVs intended for tasks with reduced visual content, the proposed method solves both the size reduction and individual model training problems.

This study investigates the use of convolutional neural networks to automatically detect malaria-infected cells in blood smear images, offering an alternative to manual diagnosis, which depends on specialized professionals and adequate infrastructure. Manual diagnosis is time-consuming and prone to human errors, especially in malaria-endemic regions with limited resources. Automated approaches based on convolutional neural networks provide a promising solution to optimize the diagnostic process and improve access to rapid treatment in remote areas. The research evaluates the performance of different convolutional neural network architectures for malaria diagnosis, following the Cross Industry Standard Process for Data Mining methodology to structure preprocessing, modeling, and model evaluation. Preprocessing involved normalization and data augmentation techniques to enhance sample quality and diversity. Two architectures were compared: a customized convolutional neural network designed to balance computational efficiency and accuracy, and an adapted VGG16, recognized for its advanced image feature extraction capabilities. Both models were trained and evaluated using a robust dataset of blood cell images. The customized network achieved 95.6% accuracy, outperforming similar models in the literature and demonstrating its practicality for low-resource settings. It also exhibited high precision, recall, and F1-score, ensuring reliable and balanced classification of infected and healthy cells. This approach reduces reliance on specialists and advanced equipment, making malaria diagnosis more accessible and efficient. The findings position the customized convolutional neural network as a viable solution for automated malaria diagnosis, combining simplicity with high performance and offering significant potential for improving healthcare in endemic regions.

The Sentinel-2 (S2) constellation provides multispectral images at 10 m, 20 m, and 60 m resolution bands. Obtaining all bands at 10 m resolution would benefit many applications. Recently, many model-based and deep learning (DL)-based sharpening methods have been proposed. However, the downside of those methods is that the DL-based methods need to be trained separately for the 20 m and the 60 m bands in a supervised manner at reduced resolution, while the model-based methods heavily depend on the hand-crafted image priors. To break the gap, this article proposes a novel unsupervised DL-based S2 sharpening method using a single convolutional neural network (CNN) to sharpen the 20 and 60 m bands at the same time at full resolution. The proposed method replaces the hand-crafted image prior by the deep image prior (DIP) provided by a CNN structure whose parameters are easily optimized using a DL optimizer. We also incorporate the modulation transfer function-based degradation model as a network layer, and add all bands to both network input and output. This setting improves the DIP and exploits the advantage of multitask learning since all S2 bands are highly correlated. Extensive experiments with real S2 data show that our proposed method outperforms competitive methods for reduced-resolution evaluation and yields very high quality sharpened image for full-resolution evaluation.

In the domain of chemometrics, multiblock data analysis is widely performed for exploring or fusing data from multiple sources. Commonly used methods for multiblock predictive analysis are the extensions of latent space modelling approaches. However, recently, deep learning (DL) approaches such as convolutional neural networks (CNNs) have outperformed the single block traditional latent space modelling chemometric approaches such as partial least-square (PLS) regression. The CNNs based DL modelling can also be performed to simultaneously deal with the multiblock data but was never explored until this study. Hence, this study for the first time presents the concept of parallel input CNNs based DL modelling for multiblock predictive chemometric analysis. The parallel input CNNs based DL modelling utilizes individual convolutional layers for each data block to extract key features that are later combined and passed to a regression module composed of fully connected layers. The method was tested on a real visible and near-infrared (Vis-NIR) large data set related to dry matter prediction in mango fruit. To have the multiblock data, the visible (Vis) and near-infrared (NIR) parts were treated as two separate blocks. The performance of the parallel input CNN was compared with the traditional single block CNNs based DL modelling, as well as with a commonly used multiblock chemometric approach called sequentially orthogonalized partial least-square (SO-PLS) regression. The results showed that the proposed parallel input CNNs based deep multiblock analysis outperformed the single block CNNs based DL modelling and the SO-PLS regression analysis. The root means squared errors of prediction obtained with deep multiblock analysis was 0.818%, relatively lower by 4 and 20% than single block CNNs and SO-PLS regression, respectively. Furthermore, the deep multiblock approach attained ∼3% lower RMSE compared to the best known on the mango data set used for this study. The deep multiblock analysis approach based on parallel input CNNs could be considered as a useful tool for fusing data from multiple sources.

Automated evaluation of rheumatoid arthritis from hand radiographs using Machine Learning and deep learning techniques

The world has recently seen unprecedented growth of social networks that have enabled internet users to share and communicate their views in various areas of life. Analysis of opinion is one type of text mining that aims to discover human views and emotions about any particular event, or topic. In the experiment of this research, we perform document level sentiment analysis in the form of positive and negative sentiments using both the traditional models and recent deep learning models. Three traditional methods (i.e. Naive Bayes (NB), Logistic Regression (LR), and Voting Ensemble) and three deep learning layout (i.e. Single Convolutional Neural Network (CNN), Single CNN + FastText and Ensemble(2-Layer CNN + Gated Recurrent Unit (GRU))) have been employed for two different datasets to evaluate the opinion classification. In our experiment, accuracy, f1-score, and RMSE have been considered as evaluation parameters and the optimistic results show that for airline review dataset, Single CNN + FastText comes out with the highest performance whereas for hotel review dataset, 2-L CNN + GRU comes out with the highest performance.

<p>With consumer-grade unmanned aerial vehicles (UAVs) on the rise, which enable easy, time-flexible, and cost-effective acquisition of very high-resolution RGB data, the mapping of forest tree species using solely RGB imagery is of high interest, as it does not rely on sophisticated sensors, does not require extensive calibration and preprocessing and, therefore, enables the application by a wide audience. In combination with convolutional neural networks (CNNs), which particularly exploit spatial patterns and, therefore, highly benfit from very high-resolution remote sensing, this offers great potential for accurately mapping forest tree species.</p><p>Here, we present the findings of our recent study, in which we used very high-resolution RGB imagery from UAVs in combination with CNNs for the mapping of forest tree species. In this study, we used multicopter UAVs to obtain very high-resolution (<2 cm) RGB imagery over 51 ha of temperate forests in the Southern Black Forest region, and the Hainich National Park in Germany. To fully harness the end-to-end learning capabilities of CNNs, we used a semantic segmentation approach (U-net) that concurrently segments and classifies tree species from imagery. With a diverse dataset in terms of study areas, site conditions, illumination properties, and phenology, we accurately mapped nine tree species, three genus-level classes, deadwood, and forest floor (mean F1-score 0.73). We found that a coarser spatial resolution substantially reduced the model accuracy (mean F1-score of 0.26 at 32 cm resolution) and that larger tile sizes during CNN training negatively affected the model accuracies for underrepresented classes. Additional height information from normalized digital surface models slightly increased the model accuracy but simultaneously increased computational complexity and data requirements. Our results highlight the key role that UAVs can play in the mapping of forest tree species, given that air- and spaceborne remote sensing currently does not provide comparable spatial resolutions. Given the end-to-end learning capabilites of CNNs extensive preprocessing becomes partly obsolete, whereas the use of a large and diverse dataset facilitates a high degree of generalization of the CNN, thus fostering transferability. The synergy of high-resolution UAV imagery and CNN provide a fast and flexible yet accurate means of mapping forest tree species.</p><p>In this contribution, we will give an outlook on how the combination of UAV imagery and CNNs can be integrated with multitemporal satellite imagery (Sentinel-1 and Sentinel-2) in order to extrapolate the UAV-based tree species maps to larger areas.</p>