Visual Analysis of Agricultural Workers using Explainable Artificial Intelligence (XAI) on Class Activation Map (CAM) with Characteristic Point Data Output from OpenCV-based Analysis

This research presents a fuzzy controller technique in navigation with obstacle avoidance for a general purpose mobile robot in a given global environment with image processing technique using Open Source Computer Vision (OpenCV) library on Visual C++. Fuzzy Logic is used to control the navigation of the robot towards the goal while avoiding obstacles along the way by changing its direction of movement. The positions of the mobile robot, obstacle and the destination are taken into consideration and an overhead camera (above the robot’s environment) is used to gather these necessary information. The images captured are processed using different techniques to get the desired positions and is directly integrated with the fuzzy controller making the algorithm more efficient compared to other vision-guided navigation techniques.

Super-resolution face image acquisition system is indispensable in people's life. Under the condition of low illumination, the illumination environment difference is too big or the light is insufficient, which leads to the traditional image acquisition system can not collect the high quality face image, and the limitation is poor. Based on open source computer vision library (OpenCV) in the C++ environment configuration, the use of Three Dimension (3D) face recognition technology algorithm, design a set of low illumination conditions of the super resolution face image acquisition system. Experiments show that the design scheme with real-time focusing speed), fast (single acquisition 0.05 seconds), accurate (facial recognition rate of 99.3%) etc. characteristics, be able to fully meet the needs of low illumination conditions for super-resolution of face image acquisition.

Explainable AI (XAI) frameworks are becoming essential in many areas, including the medical field, as they help us to understand AI decisions, increasing clinical trust and improving patient care. This research presents a robust and comprehensive Explainable AI framework. To classify images from the BloodMNIST and Raabin‐WBC datasets, various pre‐trained convolutional neural network (CNN) architectures: the VGG, the ResNet, the DenseNet, the EfficientNet, the MobileNet variants, the SqueezeNet, and the Xception are implemented both individually and in combination with SpinalNet. For parameter analysis, four models, VGG16, VGG19, ResNet50, and ResNet101, were combined with SpinalNet. Notably, these SpinalNet hybrid models significantly reduced the model parameters while maintaining or even improving the model accuracy. For example, the VGG 16 + SpinalNet shows a 40.74% parameter reduction and accuracy of 98.92% (BloodMnist) and 98.32% (Raabin‐WBC). Similarly, the combinations of VGG19, ResNet50, and ResNet101 with SpinalNet resulted in weight parameter reductions by 36.36%, 65.33%, and 52.13%, respectively, with improved accuracy for both datasets. These hybrid SpinalNet models are highly efficient and well‐suited for resource‐limited environments. The authors have developed a dynamic model selection framework. This framework optimally selects the best models based on prediction scores, prioritizing lightweight models in cases of ties. This method guarantees that for every input, the most effective model is used, which results in higher accuracy as well as better outcomes. Explainable AI (XAI) techniques: Local Interpretable Model‐agnostic Explanations (LIME), SHapley Additive ExPlanations (SHAP), and Gradient‐weighted Class Activation Mapping (Grad‐CAM) are implemented. These help us to understand the key features that influence the model predictions. By combining these XAI methods with dynamic model selection, this research not only achieves excellent accuracy but also provides useful insights into the elements that influence model predictions.

In this system, it helps to convert the sign language to voice with hand gestures understanding and capture the motion of hands. Many sign languages are natural languages, but they are differing in construction from oral languages used in proximity to them, and are employed mainly by deaf people in order to communicate. It is based on a Raspberry Pi with a camera module and is programmed in Python with the Open-Source Computer Vision (Open CV) library as a backend. A built-in image processing algorithm on the Raspberry Pi is named gesture that tracks an object (a finger) with features pulled out. The main purpose of a gesture recognition system is to establish a connection between a human and a computer control system. Camera is used in this system and it captures the various gestures of hands. Various algorithms are taking place in processing of image. First preprocessing of the image takes place. Finally, the sign is identified by using Tensor flow algorithm and the outcome result as a voice using TTS algorithm. Open CV Python is implemented in this system. Various libraries are used in this system.

Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays and represent the state-of-the-art for image-based analysis. However, the under- standing of the accurate decision-making process of a CNN is rather unknown. The research area of explainable artificial intelligence (xAI) primarily revolves around understanding and interpreting this black-box behavior. One way of interpreting a CNN is the use of class ac- tivation maps (CAMs) that represent heatmaps to indicate the importance of image areas for the prediction of the CNN. For classification tasks, a variety of CAM algorithms exist. But for segmentation tasks, only one CAM algorithm for the interpretation of the output of a CNN exist. We propose a transfer between existing classification- and segmentation- based methods for more detailed, explainable, and consistent results which show salient pixels in semantic segmentation tasks. The resulting Seg-HiRes-Grad CAM is an exten- sion of the segmentation-based Seg-Grad CAM with the transfer to the classification-based HiRes CAM. Our method improves the previously-mentioned existing segmentation-based method by adjusting it to recently published classification-based methods. Especially for medical image segmentation, this transfer solves existing explainability disadvantages. The code is available at <a href='https://github.com/TillmannRheude/SegHiResGrad_CAM'>https://github.com/TillmannRheude/SegHiResGrad_CAM</a>

Trouble of moving Freight car Detection System (TFDS) constitutes an important part of the railway safety warning systems. With TFDS, dynamic images of freight cars are captured by high-speed cameras, timely transmitted to the train inspection center by special optical fiber network and finally observed for fault recognition. In this paper, an automatic fault recognition method for side frame key in TFDS is proposed based on open source computer vision library (OpenCV) to overcome the disadvantages of manual fault recognition. At first, image preprocessing and segmentation are applied to eliminate the impact of the surrounding environment and further simplify images. Then the axle and through-hole are calibrated through Hough circle transformation and the side frame key is located indirectly according to the geometric relationship among the axle, through-hole and side frame key. Finally, the difference of mean gray values in the region of interest (ROI) is analyzed to judge whether the side frame key is missing or not. Featured by the high efficiency, reliability and practicability, the proposed method lays the foundation for engineering applications in automatic fault recognition of freight cars.

This paper presents fast tracking of a mobile robots 2D pose in a plane using the open source computer vision library(OpenCV). This can be useful for setting up experiments to study mobile robot control, robot formation or conflict resolution. Here the feature detectors SIFT, AKAZE and ORB are tested for their speed and accuracy for tracking a robot on a plane of size 2.7m × 2.1m. To determine the accuracy that can be achieved they are compared against an edge-based template matching algorithm which has a known accuracy. First the accuracy vs detection time is studied on different size images. Then sensor fusion is studied by combining the extended Kalman filter (EKF) and unscented Kalman filter (UKF) with odometry to see what gains can be made. Root mean squared pose errors of less than 3mm in translation and less than 1 degree in heading are achieved at a object detection times of less than 50ms.

Physical interpretation of machine learning-based recognition of defects for the risk management of existing bridge heritage

Structured quantitative data, such as survey responses in human resource management research, are often analysed using machine learning methods, including logistic regression. Although these methods provide accurate statistical predictions, their results are frequently abstract and difficult for non-specialists to comprehend. This limits their usefulness in practice, particularly in contexts where eXplainable Artificial Intelligence (XAI) is essential. This study proposes a domain-independent approach for the autonomous classification and interpretation of quantitative data using visual processing. This method transforms individual responses based on rating scales into visual representations, which are subsequently processed by Convolutional Neural Networks (CNNs). In combination with Class Activation Maps (CAMs), image-based CNN models enable not only accurate and reproducible classification but also visual interpretability of the underlying decision-making process. Our evaluation found that CNN models with bar chart coding achieved an accuracy of between 93.05% and 93.16%, comparable to the 93.19% achieved by logistic regression. Compared with conventional numerical approaches, exemplified by logistic regression in this study, the approach achieves comparable classification accuracy while providing additional comprehensibility and transparency through graphical representations. Robustness is demonstrated by consistent results across different visualisations generated from the same underlying data. By converting abstract numerical information into visual explanations, this approach addresses a core challenge: bridging the gap between model performance and human understanding. Its transparency, domain-agnostic design, and straightforward interpretability make it particularly suitable for XAI-driven applications across diverse disciplines that use quantitative response data.

With the emergence of visual sensors and their widespread application in intelligent systems, precise and interpretable visual explanations have become essential for ensuring the reliability and effectiveness of these systems. Sensor data, such as that from cameras operating in different spectra, LiDAR, or other imaging modalities, are often processed using complex deep learning methods, whose decision-making processes can be unclear. Accurate interpretation of network decisions is particularly critical in domains such as autonomous vehicles, medical imaging, and security systems. Moreover, during the development and deployment of deep learning architectures, the ability to accurately interpret results is crucial for identifying and mitigating any sources of bias in the training data, thereby ensuring fairness and robustness in the model’s performance. Explainable AI (XAI) techniques have garnered significant interest for their ability to reveal the rationale behind network decisions. In this work, we propose leveraging entropy information to enhance Class Activation Maps (CAMs). We explore two novel approaches: the first replaces the traditional gradient averaging scheme with entropy values to generate feature map weights, while the second directly utilizes entropy to weigh and sum feature maps, thereby reducing reliance on gradient-based methods, which can sometimes be unreliable. Our results demonstrate that entropy-based CAMs offer significant improvements in highlighting relevant regions of the input across various scenarios.

The rapid and accurate detection of fire in architectural imagery is critical for safeguarding built heritage and ensuring public safety, presenting a compelling challenge at the intersection of computer vision and architectural studies. This paper introduces a robust deep-learning framework tailored for this task, with principal contributions spanning data, methodology, and model interpretability. First, we construct and publicly release a novel dataset of building exterior images encompassing fire, smoke, and normal scenes, providing a dedicated benchmark for scholarly and applied research. Second, we implement a fine-tuned ResNet-50 model, enhanced by strategic data augmentation and class-balancing techniques, which achieves perfect classification performance on a balanced test set. Finally, and most significantly for fostering trust, we employ Class Activation Mapping (CAM) to generate visual explanations. These heatmaps empirically verify that the model’s decisions are grounded in semantically relevant visual features—specifically, flames and smoke—rather than spurious correlations, thereby validating its reliability. Our work demonstrates the potent synergy of data-centric Artificial Intelligence (AI) and explainable AI for architectural image analysis. The findings offer substantial implications for interdisciplinary studies in architectural imagery, visual cognition, and the development of intelligent, reliable monitoring systems for the built environment.

Parkinson’s disease (PD), the second most prevalent neurodegenerative disorder globally, afflicting approximately 10 million individuals, necessitates early detection for optimal management. In this paper, we propose deep learning models to discern Parkinson’s disease through the nuanced analysis of handwriting with the overall objective of achieving transparency and trustworthiness through the integration of Explainable and Interpretable AI. Leveraging transfer learning from well-established VGG16 and VGG19 architectures and introducing two bespoke CNN models (PD-Detect1 and PD-Detect2), we meticulously scrutinize diverse datasets (HandPD, NewhandPD, Parkinson Drawing) to ascertain the efficacy of our approach. LIME and SHAP Explainable AI techniques are employed to pinpoint specific regions of the spiral drawings that significantly influence the predictions made by the VGG16 and PD-Detect2 models. Additionally, Convolutional Filter Visualization and Grad-CAM are utilized to illustrate how the convolutional layers of the PD-Detect2 model function. Finally, LIME is applied to the PD-Detect2 model to identify visual markers of handwriting symptoms in the spiral drawing. Remarkable results underscore our reliance on VGG16 and VGG19 for precise identification, achieving an outstanding 100% accuracy in the waves drawing dataset. PD-Detect1 exhibits commendable performance with an accuracy of 94.44% in the meander of NewhandPD dataset, while VGG16 achieves an accuracy of 95%. VGG16 records 95% accuracy while PD-Detect2 achieves 85% accuracy in the spiral drawing dataset. Both results further bolstered to 100% with the application of classic data augmentation techniques. The Positive/Negative superpixels from LIME and SHAP highlight the key regions used by VGG16 and PD-Detect2 for predictions, while PD-Detect2 places more emphasis on disease-related features. Visualizing convolutional filters provided insight into the functionality of each layer within the PD-Detect2 model, while the Class Activation Maps produced by Grad-CAM highlighted the image regions most influential to the model’s decision. Ultimately, LIME’s superpixels identified visual markers of handwriting symptoms associated with Parkinson’s disease. Explainable AI and Interpretable AI enhance the quality of CNN models and support the decision-making process, enabling healthcare professionals to more accurately assess disease probability and monitor treatment responses. This leads to a more effective system for the early diagnosis of Parkinson’s disease through prediction and visual monitoring of handwriting symptoms.

We report the successful integration of deep learning approach for autonomous robot KUKA YouBot navigation. The incorporation of deep learning approach was carried out through the combination among Robot Operating System, Python software and Open Source Computer Vision Library working environments. The Robot Operating System with Hydro medusa distribution provides the managing of odometry, kinematics and path planning nodes. The combination of all nodes allows the simulation of the autonomous robot navigation by using Gazebo and provides the implementation of the algorithms in simulated and real platforms. Python software improves the communication tasks taking advantage of data processing tools in the deep learning process. Then, Open Source Computer Vision Library allows the integration of the deep learning approach by using the Single Shot Detector algorithm which provides robustness in velocity and precision which mainly allows the human detection by using a trained neuronal network. The integration of Robot Operating System, Python software and Open Source Computer Vision Library is a promising architecture for providing smart autonomous navigation capability for different applications like: social robots, human rescue, precision farming, industry and so on.

We present the successful demonstration of autonomous robot KUKA YouBot navigation based on path planning and traffic signals recognition. The integration of both capabilities path planning and traffic signals recognition was carried out, thanks to the integration among Robot Operating System, MATrix LABoratory software and Open Source Computer Vision Library working environments. The Robot Operating System allows the simulation of the autonomous robot navigation by using Gazebo and provides the implementation of the algorithms in simulated and real platforms. MATrix LABoratory software improves the communication tasks taking advantage of data processing tools in the path planning process. Finally, Open Source Computer Vision Library allows the traffic signals recognition by using the Scale-Invariant Feature Transform and Speeded-Up Robust Features algorithm. The integration of Robot Operating System, MATrix LABoratory software and Open Source Computer Vision Library is a promising approach to provide autonomous navigation capability in any mobile robot and in uncontrolled environments.

An open-access fire detection image dataset for research in computer vision and safety monitoring.