Understanding Thermal Face Detection: Challenges and Evaluation

Object detection in thermal images is an important computer vision task and has many applications such as unmanned vehicles, robotics, surveillance, and night vision. Deep learning-based detectors have achieved major progress, which usually need large amount of labelled training data. However, labelled data for object detection in thermal images is scarce and expensive to collect. How to take advantage of the large number labelled visible images and adapt them into thermal image domain is expected to solve. This paper proposes an unsupervised image-generation enhanced adaptation method for object detection in thermal images. To reduce the gap between visible domain and thermal domain, the proposed method manages to generate simulated fake thermal images that are similar to the target images and preserves the annotation information of the visible source domain. The image generation includes a CycleGAN-based image-to-image translation and an intensity inversion transformation. Generated fake thermal images are used as renewed source domain, and then the off-the-shelf domain adaptive faster RCNN is utilized to reduce the gap between the generated intermediate domain and the thermal target domain. Experiments demonstrate the effectiveness and superiority of the proposed method.

The automotive industry is currently focusing on automation in their vehicles, and perceiving the surroundings of an automobile requires the ability to detect and identify objects, events and persons, not only from the outside of the vehicle but also from the inside of the cabin. This constitutes relevant information for defining intelligent responses to events happening on both environments. This work presents a new method for in-vehicle monitoring of passengers, specifically the task of real-time face detection in thermal images, by applying transfer learning with YOLOv3. Using this kind of imagery for this purpose brings some advantages, such as the possibility of detecting faces during the day and in the dark without being affected by illumination conditions, and also because it’s a completely passive sensing solution. Due to the lack of suitable datasets for this type of application, a database of in-vehicle images was created, containing images from 38 subjects performing different head poses and at varying ambient temperatures. The tests in our database show an AP50 of 99.7% and an AP of 78.5%.

A numerical simulation tool for predicting the impact of outdoor thermal environment on building energy performance

Football clubs use various methods such as thermal imaging which is a non-invasive and faster method to detect injuries and increase the success rate of the football club by reducing the injury rate. Studies have proven that with thermal imaging it is possible to detect inflammation caused by an injury. Therefore, it is possible to detect potential injury with infrared thermography. One of the biggest handicaps of injury detection with thermal imaging is that it is open to subjective interpretation, there are many points that can be missed, and it takes time to analyse them one by one. In order to avoid this problem, to increase the success of injury detection, a deep learning supported pipeline has been designed in this study to detect injuries from thermal images. In this pipeline, the hamstring muscle region from the football player thermal images was segmented using U-Net architecture. After that in order to detect injuries, segmented muscle region is classified by using Densenet, Resnet, VGG, Efficientnet architectures variations and feature pyramid added at the end of these architectures. Among the architectures used for classification, the EfficientnetB0 and EfficientnetB1+feature pyramid architectures are the most successful, with accuracies of 83.9% and 81%, respectively.

Accurate eye detection in thermal images is essential for diverse applications, including biometrics, healthcare, driver monitoring, and human-computer interaction. However, achieving this accuracy is often hindered by the inherent limitations of thermal data, such as low resolution and poor contrast. This work addresses these challenges by proposing a novel, multifaceted approach that combines both deep learning and image processing techniques. We first introduce a unique dataset of thermal facial images captured with meticulous eye location annotations. To improve image clarity, we employ Contrast Limited Adaptive Histogram Equalization (CLAHE). Subsequently, we explore the effectiveness of advanced YOLO models (YOLOv8 and YOLOv9) for accurate eye detection. Our experiments reveal that YOLOv8 with CLAHE-enhanced images achieved the highest accuracy (precision and recall of 1, mAP50 of 0.995, and mAP50-95 of 0.801), the YOLOv9 model also demonstrated excellent performance with a precision of 0.998, recall of 0.998, mAP-50 of 0.995, and mAP50-95 of 0.753. Furthermore, to enhance the resolution of detected eye regions, we investigate various super-resolution techniques, ranging from traditional methods like Bicubic interpolation to cutting-edge approaches like generative adversarial networks (BSRGAN, ESRGAN) and advanced models like Real-ESRGAN, SwinIR, and SwinIR-Large with ResShift. The performance of these techniques is evaluated using both objective and subjective quality measures. Overall, this work demonstrates the effectiveness of our proposed pipeline, which seamlessly integrates image enhancement, deep learning, and super-resolution techniques. This synergic fusion significantly improves the contrast, accuracy of eye detection, and overall resolution of thermal images, paving the way for potential applications across various fields.

Thermal video can be used as a privacy-preserving and non-contact sensor for long-term health monitoring including respiratory activity. Face detection from thermal images is required to define the region of interest for automated respiration monitoring. In this study, we focus on thermal face detection using deep learning-based methods and transfer learning. First, YOLOv7, YOLOv7-tiny, and Detector Transformer (DeTr) object detection models were trained on an open thermal image dataset of faces. The weights from the pretrained models were transferred to a new model that was trained on our own target dataset. Results showed that transfer learning resulted in improved intersection-over-union (IoU) face detection performance. Moving beyond face detection in a single frame, we evaluated the stability of the trained face detection model with regard to the time-consistency of the detected bounding boxes in thermal videos. The DeTr model showed higher performance with 0.812 IoU and more stable predicted bounding boxes compared to YOLOv7 and YOLOv7-tiny. The proposed methods were also evaluated with regard to model size, as it pertains to viable deployment using edge computing, as part of a complete respiration rate estimation pipeline.

<p class="Abstract">Face identification is amongst the most efficacious and extensive applications in biometrics involving extraction and locating facial features. With identification being monotonous task attributable to reliance on parameters like varied cameras, fluctuating backgrounds, and exposure to the environment in which an individual is present. Thermal imaging is endeavoring to resolve the accuracy issue of apparent imaging, such as lighting and brightness intensity, among all biometric variables. This paper presents a study of thermal imaging and effective methods involved in the feature extraction process for facial features with thermal imaging under the influence of varied noise. A novel face dataset is created TID comprising 27 thermal images and its corresponding visual band image using Fluke 480 Ti Pro camera. The study analyses detection efficiency of six feature extraction techniques in visible and thermal bands in facial features identification. Also, the influence of noise in the thermal band within the region of interest using feature points FIN, FOUT has been estimated. Throughout TID dataset, ORB extraction technique has been able to identify strongest inlier features FIN to a maximum extent with detection around the nose, eyes, and mouth. Further, results indicate feature detection in thermal images being invariant to effect of noise for detecting facial features.</p>

Providing high data rates and wide bandwidth applications are the most important challenges that mobile systems and radio networks are trying to accomplish. Radio over Fiber-RoF systems is a promoting research field since it benefits of Passive Optical Network (PON) advantages and unlicensed large bands in 60 GHz millimeter band (57-64GHz). In this study, we consider a complete RoF system from the central office to the end-user passing by the base station and we study the impact of propagation environment on the whole system performance and radio coverage by considering different scenarios for multipath propagation (Line of Sight-LOS, Non Line of Sight-NLOF). In particular, we have been interested in Rayleigh and Rice propagation models for both indoor and outdoor environments. Optisystem/ADS Co-simulation has been investigated to simulate RoF network coverage for indoor and outdoor environments while considering high data rates.

Tracking objects or humans using background subtraction is commonly used both in indoor and outdoor environment. But this method may not produce quality results due to illumination change, shadows, and occlusion occurred in indoor or outdoor environments. Song and Nevatia suggested a simultaneous tracking of face and body for human tracking in an indoor environment using Adaboost face detection and Camshift color tracking methods (2004). Changes in illumination are not considered in their work. In this research, an intelligent method for color tracking scheme using skin reflectance values is proposed to track the human in real-time video for varying illuminations both in indoor and outdoor environments. This research finds application in the live transmission of an indoor video conference or a live news cast from an outdoor as well as in the places where the illumination change occurs frequently

Language acquisition by children depends on the learning environment when they were in kindergarten to some extent. The goals of this study were to find out the impact of the indoor and outdoor environment on children's language development. Structural equation modeling was adopted. Children randomly selected from 270 preschool classes of five provinces in China were recruited in this study (1642 children aged between 36 and 72 months). We concluded that language achievement of children cannot be directly predicted by indoor and outdoor facilities and play materials, and an intermediate transmission pathway was needed. As an intermediary variable, the curriculum implementation partially mediated the influence of the indoor learning center on children's language achievement. The psychological atmosphere and curriculum implementation have a chain mediation effect between indoor and outdoor facilities and play material quality and childhood language achievement, and the influence effect of indoors was different from that of outdoors. In terms of the influence of indoor learning center planning and play materials, the effect on children's language achievement accounted for 61.5%, while when it came to that of outdoor learning center facilities and equipment, it was 60.7%. This indicated that the impact of the indoor and outdoor environment on children's language development could not be ignored. The indoor environment was more dependent than the outdoor environment on the chain role played by class curriculum implementation, psychological atmosphere and curriculum implementation quality. Therefore, teachers should create a positive psychological atmosphere, organically combine the curriculum implementation with environmental planning and material delivery, cultivate a prepared language environment for children and support their active learning and development.

Deep Neural Networks for object detection have become of significant interest with the substantial improvement in their efficiency and increase in their applications. However, training such a network requires a large annotated dataset, which is very expensive in term of time and human effort. In this context, several automatic image annotation solutions have been proposed which are dependent on rich visual features and thus suitable for color images only. On the other hand, recent experiments have proven that color cameras alone are not enough for pedestrian related applications and there is a need to use other visual sensors as well, such as thermal cameras. In this paper, we propose an automatic image annotation technique for pedestrian detection in thermal images using an adaptive background/foreground estimation model to train Faster-RCNN. The results presented in this paper, obtained through longterm experiments demonstrate the efficacy of our technique. The results also show that our proposed technique is be very useful to generate image annotations automatically and to train a deep neural network without having a manually annotated dataset for cameras with different modalities.

Human detection in thermal images has become an important topic that stands out in many critical areas such as search and rescue operations, security, and autonomous systems. The deep learning-based object detection model You Look Only Once is preferred due to its real-time detection capability, high accuracy and precision, and resilience against challenges encountered in thermal images such as adverse weather conditions, night vision complexity, and insufficient light. In this study, the human detection performance of the YOLOv5s, YOLOv8s, YOLOv8l, and YOLOv11s models was comparatively evaluated using a suitable thermal image dataset. The models were compared using precision, recall, mAP at 0.50 IoU, and processing time (time) metrics. YOLO11s and YOLOv8s achieved the highest precision values (88–89%), while YOLOv5s lagged behind with a lower precision rate of 87% and YOLOv8l with 87%. In terms of mAP at 0.50 IoU values, YOLOv8s and YOLO11s achieved high success at 89% levels, while YOLOv5s and YOLOv8l showed a lower performance at around 87%. In terms Relative Computation Complexity (RCC), YOLOv8l, selected as the baseline model, has the lowest computational load, while YOLOv5s and YOLOv8s show similar levels of complexity, and YOLOv11s has been determined to be the model with the highest computational complexity. In addition, hardware and computational costs were also considered using metrics such as the number of parameters, inference speed in frames per second (FPS), latency, and memory usage in video random-access memory (VRAM). The results obtained show that YOLOv11s and YOLOv8s are particularly suitable for systems requiring real-time performance.

Sensors capable of multispectral and thermal imaging beyond visible bands offer many analysis possibilities for environmental monitoring. Different sensor images constitute an important source of information especially in the fields of agriculture, forestry, geology and energy. Photogrammetric studies have been affected by this development in recent years and have been used in the production of multispectral and thermal models besides the RGB model. However, due to geometric and radiometric resolution differences, it is difficult to combine or evaluate models produced from different types of sensors. In this study, the three-dimensional test field images obtained with RGB, multispectral and thermal sensors were oriented and modeled photogrammetrically. The accuracies of the control points on the produced models were compared and discussed. When the results are examined, control point accuracy was obtained as almost similar as in the RGB model after the orientation based on automatic feature matching. Automatic feature detection and matching in thermal images were not robustly produced due to low geometric resolution. For this reason, manual measurements were performed in thermal images, and the photogrammetric orientation and adjustment process was done accordingly. The fused evaluation approach considering RGB, multispectral and thermal images in one photogrammetric model was also implemented and discussed.

Towards a systematic understanding of the dynamic fate of polychlorinated biphenyls in indoor, urban and rural environments

Micro-level energy sources from the ambient environment can be a viable alternative to increase the life of the sensor nodes from a WSN network. Harvesting energy from the environment - particularly solar energy, is a technique that can support the operation of the sensor nodes. The power extracted from the luminous radiation is extremely variable, depending on day/night alternation, atmospheric conditions, temperature, geographical location, indoor or outdoor environment - which makes the energy harvested insufficient for use as an exclusive power source to supply sensor nodes. A special case is the indoor environment, in which the solar radiation is strongly reduced, compared to the outside environment, and artificial lighting levels are usually below 1000 Lux. We made experiments on an indoor solar cell using illumination from three different sources: fluorescent, incandescent, LED. Detailed measurements, performance comparisons, and conclusions are presented.