Robust visual localization in changing lighting conditions

Visual localization has been well studied in recent decades and applied in many fields as a fundamental capability in robotics. However, the success of the state of the arts usually builds on the assumption that the environment is static. In dynamic scenarios where moving objects are present, the performance of the existing visual localization systems degrades a lot due to the disturbance of the dynamic factors. To address this problem, we propose a novel sparse motion removal (SMR) model that detects the dynamic and static regions for an input frame based on a Bayesian framework. The similarity between the consecutive frames and the difference between the current frame and the reference frame are both considered to reduce the detection uncertainty. After the detection process is finished, the dynamic regions are eliminated while the static ones are fed into a feature-based visual simultaneous localization and mapping (SLAM) system for further visual localization. To verify the proposed method, both qualitative and quantitative experiments are performed and the experimental results have demonstrated that the proposed model can significantly improve the accuracy and robustness for visual localization in dynamic environments.<;/p><;p><;i>Note to Practitioners<;/i>-This article was motivated by the visual localization problem in dynamic environments. Visual localization is well applied in many robotic fields such as path planning and exploration as the basic capability for a mobile robot. In the GPS-denied environments, one robot needs to localize itself through perceiving the unknown environment based on a visual sensor. In real-world scenes, the existence of the moving objects will significantly degrade the localization accuracy, which makes the robot implementation unreliable. In this article, an SMR model is designed to handle this problem. Once receiving a frame, the proposed model divides it into dynamic and static regions through a Bayesian framework. The dynamic regions are eliminated, while the static ones are maintained and fed into a feature-based visual SLAM system for further visual localization. The proposed method greatly improves the localization accuracy in dynamic environments and guarantees the robustness for robotic implementation.

Abstract. In this paper, we present a method for visual localization and pose estimation based on 3D image spaces. The method works in indoor and outdoor environments and does not require the presence of control points or markers. The method is evaluated with different sensors in an outdoor and an indoor test field. The results of our research show the viability of single image localization with absolute position accuracies at the decimetre level for outdoor environments and 5 cm or better for indoor environments. However, the evaluation also revealed a number of limitations of single image visual localization in real-world environments. Some of them could be addressed by an alternative AR-based localization approach, which we also present and compare in this paper. We then discuss the strengths and weaknesses of the two approaches and show possibilities for combining them to obtain accurate and robust visual localization in an absolute coordinate frame.

Accurate and robust visual localization under changing environments remains a fundamental challenge in autonomous driving and mobile robotics. Traditional handcrafted features often degrade under long-term illumination and viewpoint variations, while recent CNN-based methods, although more robust, typically rely on coarse semantic cues and remain vulnerable to dynamic objects. In this paper, we propose a fine-grained semantics-guided feature extraction framework that adaptively selects stable keypoints while suppressing dynamic disturbances. A fine-grained semantic refinement module subdivides coarse semantic categories into stability-homogeneous sub-classes, and a dual-attention mechanism enhances local repeatability and semantic consistency. By integrating physical priors with self-supervised clustering, the proposed framework learns discriminative and reliable feature representations. Extensive experiments on the Aachen and RobotCar-Seasons benchmarks demonstrate that the proposed approach achieves state-of-the-art accuracy and robustness while maintaining real-time efficiency, effectively bridging coarse semantic guidance with fine-grained stability estimation. Quantitatively, our method achieves strong localization performance on Aachen (up to 88.1% at night under the (0.2°,0.25 m) threshold) and on RobotCar-Seasons (up to 57.2%/28.4% under the same threshold for day/night), demonstrating improved robustness to seasonal and illumination changes.

This study evaluates the generalization capabilities of computer vision (CV) models for detecting cow locations in indoor free-stall barns, with a focus on quantifying the resources required for effective deployment across different farms where camera angles and lighting conditions vary. Given the increasing reliance of precision livestock farming (PLF) on object localization techniques to monitor livestock health and optimize resource management, understanding how to deploy CV models with minimal resources while maintaining high performance is critical. To assess generalization, we investigated three key hypotheses: (1) Model generalization is equally influenced by changes in lighting conditions and camera angles; (2) Higher model complexity guarantees better generalization performance; and (3) Fine-tuning with custom initial weights trained on related tasks enhances final performance compared to using pre-trained weights. The study utilized the newly released COws LOcalization (COLO) dataset, comprising 1,254 images and 11,818 cow instances captured from five distinct imaging settings with varying camera angles and lighting conditions. The first hypothesis was tested by training models on images from specific camera angles and evaluating performance on new angles. The second hypothesis was examined by comparing the performance of five different model sizes (e.g., YOLO11n with 2.6M parameters vs. YOLO11x with 56.9M parameters) on the same dataset. The third hypothesis was evaluated by pre-training a model on a subset of images from a specific barn and fine-tuning it on another subset from a similar setting to simulate transferring the training outcome to a different location within the same barn. Results indicate that changes in lighting conditions have a negligible impact on model generalization, whereas shifting from a top-view to a side-view perspective significantly reduces performance (P &amp;lt; 0.001). Model complexity does not always correlate with improved generalization, as optimal complexity depends on the dataset and task. Additionally, fine-tuning with custom initial weights benefits small, simple models but provides no significant (P &amp;gt; 0.05) advantage for larger models compared to training from scratch. As artificial intelligence adoption grows, alongside rising concerns about its environmental impact, this study provides practical guidelines for PLF researchers to optimize CV model deployment with minimal resource consumption. It also highlights key generalization challenges and leverages the COLO dataset to drive further research in resource-efficient livestock monitoring.

Enabling FIRA medium-sized soccer robots to recognize target objects according to color information requires that competing teams manually set the range of colors according to ambient lighting conditions prior to games. This color information is used to differentiate features of target objects, such as the ball, the goals, and the field. Constructing a color-feature model such as this is extremely time-consuming and the resulting model is unable to adapt dynamically to changes in lighting conditions. This study applied a look-up table method to execute RGB-HSV color space conversion to accelerate video processing. A particle swarm optimization (PSO) scheme was developed to detect the color-feature parameters of the target objects in the HSV color space. This enables the automatic completion of color-feature modeling and the construction of the knowledge model required by the robot for object recognition. Experiment results demonstrate that the proposed method is capable of enhancing the robustness of the robot vision system in determining changes in lighting conditions. In addition, the manpower and time required to set robot parameters prior to games were reduced significantly.

It is important to control acoustic noise aboard the International Space Station (ISS) to provide a satisfactory environment for voice communications, crew productivity, alarm audibility, and restful sleep, and to minimize the risk for temporary and permanent hearing loss. Acoustic monitoring is an important part of the noise control process on ISS, providing critical data for trend analysis, noise exposure analysis, validation of acoustic analyses and predictions, and to provide strong evidence for ensuring crew health and safety, thus allowing Flight Certification. To this purpose, sound level meter (SLM) measurements and acoustic noise dosimetry are routinely performed. And since the primary noise sources on ISS include the environmental control and life support system (fans and airflow) and active thermal control system (pumps and water flow), acoustic monitoring will reveal changes in hardware noise emissions that may indicate system degradation or performance issues. This paper provides the current acoustic levels in the ISS modules and sleep stations and is an update to the status presented in 2011. Since this last status report, many payloads (science experiment hardware) have been added and a significant number of quiet ventilation fans have replaced noisier fans in the Russian Segment. Also, noise mitigation efforts are planned to reduce the noise levels of the T2 treadmill and levels in Node 3, in general. As a result, the acoustic levels on the ISS continue to improve.

Accuracy of skin segmentation algorithms is highly sensitive to changes in lighting conditions. When the lighting condition in a scene is different from that in the training examples, miss-classification rate of the skin segmentation algorithms is high. Using color constancy approach we aim to compensate for skin color variations to achieve accurate skin color segmentation. Skin color constancy is realized in an unsupervised manner by using the color changes observed on a face for different illuminations to drive the model. By training on few faces of different ethnicities, our model is able to generalize the color mapping for any unseen ethnicity. The color changes observed are used to learn the color mapping from one lighting condition to the other. These mappings are represented in a low dimensional subspace to obtain basis vector fields. Using these basis vector fields we can model the nonlinear color changes to transform skin colors in arbitrary lighting conditions to a reference lighting condition. We show the proof of concept of unsupervised skin color constancy on faces from the PIE database. Skin segmentation is performed on the color compensated faces using a Skin Distribution Map (SDM), which is trained on skin colors in reference lighting condition.

On the off-the-shelf navigational assistance devices, the localization precision is limited to the signal error of global navigation satellite system (GNSS). During travelling outdoors, the inaccurately localization perplexes visually impaired people, especially at key positions, such as gates, bus stations or intersections. The visual localization is a feasible approach to improving the positioning precision of assistive devices. Using multiple image descriptors, the paper proposes a robust and efficient visual localization algorithm, which takes advantage of priori GNSS signals and multi-modal images to achieve the accurate localization of key positions. In the experiments, we implement the approach on the wearable system and test the performance of visual localization under practical scenarios.

The saliency feature is a key factor in achieving vision-based tracking for multi-UAV control. However, due to the complex and variable environments encountered during multi-UAV operations—such as changes in lighting conditions and scale variations—the UAV’s visual features may degrade, especially under high-speed movement, ultimately resulting in failure of the vision tracking task and reducing the stability and robustness of swarm flight. Therefore, this paper proposes an adaptive active light source system based on light intensity matching to address the issue of visual feature loss caused by environmental light intensity and scale variations in multi-UAV collaborative navigation. The system consists of three components: an environment sensing and control module, a variable active light source module, and a light source power module. This paper first designs the overall framework of the active light source system, detailing the functions of each module and their collaborative working principles. Furthermore, optimization experiments are conducted on the variable active light source module. By comparing the recognition effects of the variable active light source module under different parameters, the best configuration is selected. In addition, to improve the robustness of the active light source system under different lighting conditions, this paper also constructs a light source color matching model based on light intensity matching. By collecting and comparing visible light images of different color light sources under various intensities and constructing the light intensity matching model using the comprehensive peak signal-to-noise ratio parameter, the model is optimized to ensure the best vision tracking performance under different lighting conditions. Finally, to validate the effectiveness of the proposed active light source system, quantitative and qualitative recognition comparison experiments were conducted in eight different scenarios with UAVs equipped with active light sources. The experimental results show that the UAV equipped with an active light source has improved the recall of yoloV7 and RT-DETR recognition algorithms by 30% and 29.6%, the mAP50 by 21% and 19.5%, and the recognition accuracy by 13.1% and 13.6, respectively. Qualitative experiments also demonstrated that the active light source effectively improved the recognition success rate under low lighting conditions. Extensive qualitative and quantitative experiments confirm that the UAV active light source system based on light intensity matching proposed in this paper effectively enhances the effectiveness and robustness of vision-based tracking for multi-UAVs, particularly in complex and variable environments. This research provides an efficient and computationally effective solution for vision-based multi-UAV systems, further enhancing the visual tracking capabilities of multi-UAVs under complex conditions.

Visual information extracted from RGB images has been successfully used for mobile robot localization. The main difficulty with localization using RGB images is that visual features from RGB images are not completely invariant against changes in viewpoints and lighting conditions. This problem can be overcome using features from RGB-D images. In this paper, we evaluate two depth features, depth patches and histograms of oriented normal vectors, extracted from RGB-D images for localization of a mobile robot and demonstrate that robust localization is possible under varying lighting conditions.

Dynamic non-uniform lighting conditions, prevalent in many field robot applications, cause drastic changes in the visual information captured by camera images, resulting in major difficulties for mobile robots attempting to localise visually. Most current solutions to the problem rely on extracting visual information from images that is decoupled from the effects of lighting. This is not possible in many situations. Chrominance information is often cited as having some invariance to lighting changes, which is confirmed by experiments in this thesis. However, in the bland application environments investigated,chrominance is not a pertinent metric, indicating that chrominance is not the complete solution to the lighting problem. Descriptions of the intensity gradient are also cited as having robustness to lighting changes. Many descriptions of image-point features are based on the intensity gradient and are commonly used as a basis for visual localisation. However, the non-uniform effects of lighting – shadows and shading – are tangled into the intensity gradient, making these descriptions sensitive to non-uniform lighting changes. Experiments are presented which reveal that image-point features recorded at one time of the day cannot be reliably matched with images captured only one or two hours later, after typical changes in sunlight. It appears that autonomously building visual maps which permit geometric localisation in many lighting conditions remains an unsolved problem. Therefore, this thesis develops systems based on manually generated maps which are created a priori and ensure that only permanent, invariant, information is included within the map and allows reliable localisation to be achieved in many conditions. The first proposed visual localisation system is for autonomous ground vehicles operating at outdoor industrial sites. The system avoids the problem of mapping from fluctuating visual information by using a professionally-surveyed 3D-edge map of the permanent buildings. The vehicles are fitted with fish-eye cameras that often have direct sunlight in the field of view, causing an issue of camera exposure – dealt with by using an intelligent exposure control algorithm. Results from the system show accurate localisation during the full range of lighting conditions experienced over a day. The second visual localisation framework discussed is for submarines navigating underwater structures, where the only light source is a spotlight mounted on the vehicle. The moving vehicle and hence changing incident angle of the light source cause major variations in the appearance of the structure. This makes it difficult to employ traditional tracking techniques. The proposed localisation system uses the novel idea of incorporating a light source model. The light model is used to render synthetic images of scene – which accurately recreate the non-uniform lighting effects. The synthetic images are compared with the real camera image to localise the vehicle. The idea of using a light model is partly motivated by the human visual system’s understanding of the light source within a scene and is also motivated by the limitations of the traditional approaches to factor-out lighting. Using a light model within a visual localisation system enables a more natural link between the internal environment representation and the image and is demonstrated to allow successful localisation in this difficult visual scenario. The results of the two proposed localisation systems are encouraging, given the extremely challenging dynamic non-uniform lighting in each environment. Both systems have attracted the interest of industry partners and the projects will continue to be developed into the future, with the goal of progressing them into fully functioning robotic systems.

Robust visual localization in traffic scenes is a fundamental problem for self-driving vehicles. However, it is still challenging to achieve accurate localization performance because of drastic viewpoint and illumination changes. To address the issues, we design a novel monocular localization framework based on a light-weight prior map, called BSP-MonoLoc, which leverages the 2D semantic primitives from the monocular images and the 3D basic semantic primitives from the prior map. These primitives are commonly available but lack of distinctive signature. To effectively make associations between the 2D and 3D primitives and refine the vehicle’s pose, we adopt an iterative optimization method, where an efficient hierarchical sample strategy is designed to give a good initial prediction for the associations and the pose. Experimental results on the KAIST dataset and our dataset demonstrate the proposed method can achieve high localization accuracy and run at a real-time performance.

This paper deals with the terrain classification problem for autonomous service robots in semi-structured outdoor environments. The aim is to recognize the drivable terrain in front of a robot that navigates on roads of different surfaces, avoiding areas that are considered non-drivable. Since the system should be robust to such factors as changing lighting conditions, mud and fallen leaves, we employ multi-sensor perception with a monocular camera and a 2D laser scanner. The labeling of the terrain obtained from a Random Trees classifier is refined by context-aware inference using the Conditional Random Field. We demonstrate that automatic learning of the parameters for Conditional Random Fields improves results in comparison to similar approaches without the context-aware inference or with parameters set by hand.

This paper presents an innovative method for skin detection under changing lighting conditions. The proposed algorithm employs a Gaussian skin color model to classify skin pixels. This Gaussian model is adapted beforehand to accommodate the current lighting condition. AT first, the color temperature of the light source is estimated from a skin sample. Then the skin sample is going through a color constancy process to produce a corrected skin sample. The means of the raw and corrected skin samples are utilized to transform the Gaussian model. Experiments, using different adaptive skin detection methods and the proposed algorithm, have been conducted. Test results assert the effectiveness of the proposed method for skin detection under changing lighting conditions.

Urine color (Uc) is used to asses urine concentration when laboratory techniques are not feasible. To compare the accuracy of Uc scoring using 4 light conditions and 2 scoring techniques with a 7-color Uc chart. Additionally, to assess the results' generalizability, a subsample was compared with scores obtained from fresh samples. Descriptive laboratory study. A total of 178 previously frozen urine samples were scored, and 78 samples were compared with their own fresh outcomes. Urine color and accuracy for classifying urine samples were calculated using receiver operating characteristics analysis, allowing us to compare the diagnostic capacity against a 1.020 urine specific gravity cutoff and defining optimal Uc cutoff value. Urine color was different among light conditions (P < .01), with the highest accuracy (80.3%) of correct classifications of low or high urine concentrations occurring at the brightest light condition. Lower light intensity scored 1.5 to 2 shades darker on the 7-color Uc scale than bright conditions (P < .001), but no further practical differences in accuracy occurred between scoring techniques. Frozen was 0.5 to 1 shade darker than freshly measured Uc (P < .004), but the values were moderately correlated (r = 0.64). A Bland-Altman plot showed that reporting bias mainly affected darker Uc without affecting the diagnostic ability of the method. Urine color scoring, accuracy, and Uc cutoff values were affected by lighting condition but not by scoring technique, with greater accuracy and a 1-shade-lower Uc cutoff value at the brightest light (ie, light-emitting diode flashlight).