Pose and Illumination Invariant Face Recognition in Video

In this paper, we propose an eye detection method that is robust to facial pose changes using gradient directional features of brightness and a particle filter. The rejection function of incorrect eye detection in the proposed method allows for the eye detection again even if the eye is incorrectly detected. In this method, to estimate the boundary between the iris and sclera or eyelid, the gradient intensities are calculated by four directional Prewitt filters in four regions. The likelihood used in the particle filter is obtained by averaging the gradient intensities for the specific direction in the four regions and the upper eyelid area. From experimental results, the average detection rates of both eyes for roll, yaw, and pitch angles of the face are more than 90% by using rejection function for incorrect eye detection. The rejection function produces the 4.4%, 4.5%, and 4.9% increases in average detection rates of both eyes for roll, yaw, and pitch facial angles, respectively. The proposed eye detection method can track both eye in real-time (about 20 ms) and is robust to the facial pose changes.

Over the recent years, a great deal of effort has been made to age estimation from face images. It has been reported that age can be accurately estimated under controlled environment such as frontal faces, no expression, and static lighting conditions. However, it is not straightforward to achieve the same accuracy level in real-world environment because of considerable variations in camera settings, facial poses, and illumination conditions. In this paper, we apply a recently-proposed machine learning technique called covariate shift adaptation to alleviating lighting condition change between laboratory and practical environment. Through real-world age estimation experiments, we demonstrate the usefulness of our proposed method.

This paper describes the iCare Interaction Assistant, an assistive device for helping the individuals who are visually impaired during social interactions. The research presented here addresses the problems encountered in implementing real-time face recognition algorithms on a wearable device. Face recognition is the initial step towards building a comprehensive social interaction assistant that will identify and interpret facial expressions, emotions and gestures. Experiments conducted for selecting a face recognition algorithm that works despite changes in facial pose and illumination angle are reported. Performance details of the face recognition algorithms tested on the device are presented along with the overall performance of the system. The specifics of the hardware components used in the wearable device are mentioned and the block diagram of the wearable system is explained in detail.

One of the persistent challenges in computer vision has been tracking objects under varying lighting conditions. In this paper we present a method for estimation of 3D motion of a rigid object from a monocular video sequence under arbitrary changes in the illumination conditions under which the video was captured. This is achieved by alternately estimating motion and illumination parameters using a generative model for integrating the effects of motion, illumination and structure within a unified mathematical framework. The motion is represented in terms of translation and rotation of the object centroid, and the illumination is represented using a spherical harmonics linear basis. The method does not assume any model for the variation of the illumination conditions - lighting can change slowly or drastically. For the multi-camera tracking scenario, we propose a new photometric constraint that is valid over the overlapping field of view between two cameras. This is similar in nature to the well-known epipolar constraint, except that it relates the photometric parameters, and can provide an additional constraint for illumination invariant multi-camera tracking. We demonstrate the effectiveness of our tracking algorithm on single and multi-camera video sequences under severe changes of lighting conditions.

Three-dimensional human face modeling is a very popular topic with many applications,such as facial animation, face recognition and model-based facial video communication. Therefore, how to model the facial geometry, texture intensity and illumination variation is important in computer vision and graphics. Previous works on 3D head modeling from a single face image utilized prior information on 3D head models. However, it is difficult to accurately reconstruct the 3D face model from a single face image with expression since the facial expression induces 3D face model deformation in a complex manner. The main challenge is the coupling of the neutral 3D face model and the 3D deformation due to expression, thus making the 3D model estimation from a single face image with expression very challenging. On the other hand, the illumination condition also makes the problem more difficult. In this thesis,we focus on developing a 3D face model reconstruction system including surface registration and training of 3D face models with expressional deformations as well as the estimation of the 3D neutral shape and the 3D expressional deformation from a single face image. The proposed reconstruction algorithm integrates the linear and non-linear subspace representations for a prior 3D neutral morphable model and the probabilistic manifold-based 3D expressional deformation. We incorporate the face geometry, expression deformation, texture and illumination information into the problem so that it is well constrained. The reconstructed 3D face models can also be further extended and applied to many real-world applications.

In today’s world, we have CCTV cameras everywhere and they have a huge value when we consider public safety in a public environment. A major problem in such systems is person re-identification, which is defined as “the problem of identifying people across images which are extracted from various surveillance cameras which don’t have any overlapping fields of view.” With the increasing need of people to identify persons in the videos, this work is also getting increased importance. It also covers many other applications such as tracking across cameras, multi-camera behavior analysis and search for criminal activities. But, this problem turns out to be quite challenging because of the huge amounts of changes in lighting conditions, point of viewing, change of pose and the background. Thus, the objective is to implement a system to identify a person again and again from multiple images that have been captured by different cameras, whose fields of view do not overlap.KeywordsInstance segmentationMask-RCNNReLUResNetSoftMax loss

This study evaluates the generalization capabilities of computer vision (CV) models for detecting cow locations in indoor free-stall barns, with a focus on quantifying the resources required for effective deployment across different farms where camera angles and lighting conditions vary. Given the increasing reliance of precision livestock farming (PLF) on object localization techniques to monitor livestock health and optimize resource management, understanding how to deploy CV models with minimal resources while maintaining high performance is critical. To assess generalization, we investigated three key hypotheses: (1) Model generalization is equally influenced by changes in lighting conditions and camera angles; (2) Higher model complexity guarantees better generalization performance; and (3) Fine-tuning with custom initial weights trained on related tasks enhances final performance compared to using pre-trained weights. The study utilized the newly released COws LOcalization (COLO) dataset, comprising 1,254 images and 11,818 cow instances captured from five distinct imaging settings with varying camera angles and lighting conditions. The first hypothesis was tested by training models on images from specific camera angles and evaluating performance on new angles. The second hypothesis was examined by comparing the performance of five different model sizes (e.g., YOLO11n with 2.6M parameters vs. YOLO11x with 56.9M parameters) on the same dataset. The third hypothesis was evaluated by pre-training a model on a subset of images from a specific barn and fine-tuning it on another subset from a similar setting to simulate transferring the training outcome to a different location within the same barn. Results indicate that changes in lighting conditions have a negligible impact on model generalization, whereas shifting from a top-view to a side-view perspective significantly reduces performance (P < 0.001). Model complexity does not always correlate with improved generalization, as optimal complexity depends on the dataset and task. Additionally, fine-tuning with custom initial weights benefits small, simple models but provides no significant (P > 0.05) advantage for larger models compared to training from scratch. As artificial intelligence adoption grows, alongside rising concerns about its environmental impact, this study provides practical guidelines for PLF researchers to optimize CV model deployment with minimal resource consumption. It also highlights key generalization challenges and leverages the COLO dataset to drive further research in resource-efficient livestock monitoring.

A comparative study on face detection and tracking algorithms

This paper describes a robust method for discriminating 3D face pose (face orientation) from a video sequence featuring views of a human head under variable lighting and facial expression conditions. The wavelet transform is used to decompose the image into multiresolution face images containing both spatial and spatial-frequency information. Principal component analysis (PCA) is used to project a low-resolution sub-band face pose onto a pose eigen-space where the first 3 eigen coefficients are found most sensitive to pose and follow a trajectory as the pose changes. Any unknown pose of an query image can then be estimated by finding the Euclidean distance of the first 3 eigen coefficients of the query image from the estimated trajectory. The wavelet transform reduces the computational load on the PCA and makes the algorithm robust against illumination changes and facial expression. An accuracy of 84% was obtained for test images unseen during training under different environment conditions, facial expressions, and even different human subjects.

Accuracy of skin segmentation algorithms is highly sensitive to changes in lighting conditions. When the lighting condition in a scene is different from that in the training examples, miss-classification rate of the skin segmentation algorithms is high. Using color constancy approach we aim to compensate for skin color variations to achieve accurate skin color segmentation. Skin color constancy is realized in an unsupervised manner by using the color changes observed on a face for different illuminations to drive the model. By training on few faces of different ethnicities, our model is able to generalize the color mapping for any unseen ethnicity. The color changes observed are used to learn the color mapping from one lighting condition to the other. These mappings are represented in a low dimensional subspace to obtain basis vector fields. Using these basis vector fields we can model the nonlinear color changes to transform skin colors in arbitrary lighting conditions to a reference lighting condition. We show the proof of concept of unsupervised skin color constancy on faces from the PIE database. Skin segmentation is performed on the color compensated faces using a Skin Distribution Map (SDM), which is trained on skin colors in reference lighting condition.

Enabling FIRA medium-sized soccer robots to recognize target objects according to color information requires that competing teams manually set the range of colors according to ambient lighting conditions prior to games. This color information is used to differentiate features of target objects, such as the ball, the goals, and the field. Constructing a color-feature model such as this is extremely time-consuming and the resulting model is unable to adapt dynamically to changes in lighting conditions. This study applied a look-up table method to execute RGB-HSV color space conversion to accelerate video processing. A particle swarm optimization (PSO) scheme was developed to detect the color-feature parameters of the target objects in the HSV color space. This enables the automatic completion of color-feature modeling and the construction of the knowledge model required by the robot for object recognition. Experiment results demonstrate that the proposed method is capable of enhancing the robustness of the robot vision system in determining changes in lighting conditions. In addition, the manpower and time required to set robot parameters prior to games were reduced significantly.

We introduce a method that localizes the direct manipulation of blendshape models for facial animation with a customized sketch-based interface. Direct manipulation methods address the problem of cumbersome weight editing process using the traditional tools with a practical pin-and-drag operation directly on the 3D facial models. However, most of the direct manipulation methods have a global deformation impact, which lead to unintuitive and unexpected results. To this end, we propose a new way to localize the theory of direct manipulation method, using geodesic circles for confining the edits to the local geometry. Inspired by artists' brush painting on canvas, we additionally introduce a sketch-based interface as an application that provides direct manipulation and produces expressive facial poses efficiently and intuitively. Our method allows the artists to simply sketch directly onto the 3D facial model and automatically produces the expeditious manipulation until the desired facial pose is obtained. We show that localized blendshape direct manipulation has the potential to reduce the time-consuming blendshape editing process to an easy freehand stroke drawing.

The performance of human face recognition algorithms is seriously affected by two important factors: head pose and lighting condition. The effective processing of the pose and illumination variations is a vital key for improving the recognition rate. This paper proposes a novel method that can synthesize images with different head poses and lighting conditions by using a modified 3D CANDIDE model, linear vertex interpolation and NURBS curve surface fitting method, as well as a mixed illumination model. A specific Eigenface method is also proposed to perform face recognition based on a pre-estimated head pose method. Experimental results show that the quality of the synthesized images and the recognition performance are good.

The saliency feature is a key factor in achieving vision-based tracking for multi-UAV control. However, due to the complex and variable environments encountered during multi-UAV operations—such as changes in lighting conditions and scale variations—the UAV’s visual features may degrade, especially under high-speed movement, ultimately resulting in failure of the vision tracking task and reducing the stability and robustness of swarm flight. Therefore, this paper proposes an adaptive active light source system based on light intensity matching to address the issue of visual feature loss caused by environmental light intensity and scale variations in multi-UAV collaborative navigation. The system consists of three components: an environment sensing and control module, a variable active light source module, and a light source power module. This paper first designs the overall framework of the active light source system, detailing the functions of each module and their collaborative working principles. Furthermore, optimization experiments are conducted on the variable active light source module. By comparing the recognition effects of the variable active light source module under different parameters, the best configuration is selected. In addition, to improve the robustness of the active light source system under different lighting conditions, this paper also constructs a light source color matching model based on light intensity matching. By collecting and comparing visible light images of different color light sources under various intensities and constructing the light intensity matching model using the comprehensive peak signal-to-noise ratio parameter, the model is optimized to ensure the best vision tracking performance under different lighting conditions. Finally, to validate the effectiveness of the proposed active light source system, quantitative and qualitative recognition comparison experiments were conducted in eight different scenarios with UAVs equipped with active light sources. The experimental results show that the UAV equipped with an active light source has improved the recall of yoloV7 and RT-DETR recognition algorithms by 30% and 29.6%, the mAP50 by 21% and 19.5%, and the recognition accuracy by 13.1% and 13.6, respectively. Qualitative experiments also demonstrated that the active light source effectively improved the recognition success rate under low lighting conditions. Extensive qualitative and quantitative experiments confirm that the UAV active light source system based on light intensity matching proposed in this paper effectively enhances the effectiveness and robustness of vision-based tracking for multi-UAVs, particularly in complex and variable environments. This research provides an efficient and computationally effective solution for vision-based multi-UAV systems, further enhancing the visual tracking capabilities of multi-UAVs under complex conditions.

The transformations in the design of Catholic church architecture, partly influenced by the Second Vatican Council, have led to notable changes in spatial and lighting conditions. This research endeavors to investigate user preferences concerning lighting and spatial elements in Catholic church buildings. The study adopts a mixed-methods approach, executed in two stages. Data collection involves the utilization of online questionnaires employing a non-random/non-probability sampling method. Subsequent data analysis incorporates principal component analysis, factor analysis, and correlation analysis. Each independent variable is systematically analyzed to discern user preferences. The findings of the study reveal that factors such as color, contrast, visual comfort, lighting hierarchy, visual richness, building age, perceived building shape, verticality, symmetry, and secluded space exert influence on the stimulation of positive and spiritual impressions. The research sheds light on the intricate interplay between architectural elements and user perceptions within the context of Catholic church design, providing valuable insights for further considerations in sustainable and meaningful architectural practices.