Appearance change prediction for long-term navigation across seasons

Visual place recognition is a mature field in mobile robotics research. Recognizing places in datasets covering traversals of hundreds or thousands of kilometres and accurate localization in small and medium size environments have been successfully demonstrated. However, for real world long term operation, visual place recognition has to face severe environmental appearance changes due to day-night cycles, seasonal or weather changes. Existing approaches for recognizing places in such changing environments provide solutions for matching images from the exact same viewpoint using powerful holistic descriptors, using less sophisticated holistic descriptors in combinations with images sequences, and/or pose strong requirements on training data to learn systematic appearance changes. In this paper, we present a novel, training free, single image matching procedure that builds upon local region detectors for powerful Convolutional Neural Network (CNN) based descriptors. It can be used with a broad range of local region detectors including keypoints, segmentation based approaches and object proposals. We propose a novel evaluation criterion for selection of an appropriate local region detector for changing environments and compare several available detectors. The scale space extrema detector known from the SIFT keypoint detector in combination with appropriate magnification factors performs best. We present preliminary results of the proposed image matching procedure with several region detectors on the challenging Nordland dataset on place recognition between different seasons and a dataset including severe viewpoint changes. The proposed method outperforms the best existing holistic method for place recognition in such changing environments and can additionally handle severe viewpoint changes. Additionally, the combination of the best performing detectors with superpixel based spatial image support shows promising results.

Appearance-invariant place recognition by discriminatively training a convolutional neural network

Distilled representation using patch-based local-to-global similarity strategy for visual place recognition

Inferring ego position by recognizing previously seen places in the world is an essential capability for autonomous mobile systems. Recent advances have addressed increasingly challenging recognition problems, e.g. long-term vision-based localization despite severe appearance changes induced by changing illumination, weather or season. Since robots typically move continuously through an environment, there is high correlation within consecutive sensory inputs and across similar trajectories. Exploiting this sequential information is a key element of some of the most successful approaches for place recognition in changing environments. We present a novel, neurally inspired approach that uses sequences for mobile robot localization. It builds upon Hierarchical Temporal Memory (HTM), an established neuroscientific model of working principles of the human neocortex. HTM features two properties that are interesting for place recognition applications: (1) It relies on sparse distributed representations, which are known to have high representational capacity and high robustness towards noise. (2) It heavily exploits the sequential structure of incoming sensory data. In this paper, we discuss the importance of sequence information for mobile robot localization, we provide an introduction to HTM, and discuss theoretical analogies between the problem of place recognition and HTM. We then present a novel approach, applying a modified version of HTM’s higher order sequence memory to mobile robot localization. Finally we demonstrate the capabilities of the proposed approach on a set of simulation-based experiments.

Visual place recognition is the problem of camera-based localization of a robot given a database of images of known places, potentially under severe appearance changes (e.g., different weather or illumination). Typically, the current query images and the database images are sequences of consecutive images. In previous work, we proposed to adapt hierarchical temporal memory, a biologically plausible model of sequence processing in the human neocortex to address this task. The previous work described the algorithmic steps and showed synthetic experimental results from simulation. This letter extends the approach to application on real-world data based on a novel encoder for state-of-the-art image processing front ends. The neurologically inspired approach is compared with several state-of-the-art algorithms on a variety of datasets and shows preferable performance. Beyond the place recognition performance, the neurological roots of the algorithm result in appealing properties like potentially very energy efficient implementation due to the usage of sparse distributed representations and natural extendability like the integration of motion estimates similar to entorhinal grid cells. Finally, we underline its practical applicability by online, soft real-time application on a mobile robot.

Recently, the methods based on Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in visual place recognition. CNN is a class of multilayer perceptrons, but unlike common multilayer perceptrons that it is not usually fully connected networks. It can acquire more general image features and make the image processing computationally manageable through filtering the connections by proximity. In this paper, we utilize the deep features generated by CNNs and the dynamic time warping (DTW) algorithm for image sequence place recognition. We propose a novel image similarity measurement, which is derived from cosine distance and can better distinguish match and mismatch. Meanwhile, we improve the DTW algorithm to design a local matching method that can reduce time complexity from O(n3) to O(n). To test the proposed method, four datasets (Nordland, Gardens Point, St. Lucia, and UoA datasets) are used as benchmarks; using two traverses in each dataset with one for reference and the other for testing. The results show high precision-recall characteristics of our method in the cases of severe appearance changes. Besides, our method achieves substantial improvements over the methods using the deep feature representations of a single image for recognition, which reflects that the spatiotemporal information contained in the image sequence is significant for the task of visual place recognition. Moreover, the proposed method also shows to outperform the classical sequence-based method SeqSLAM.

This thesis addresses the problem of tracking under severe appearance changes using a combination of receding horizon and Incremental Locally Linear Embeddings (ILLE) techniques that provides adaptation to both time varying dynamics and abrupt appearance changes. In addition to enabling sustained tracking under these conditions, the proposed technique allows for efficiently detecting appearance changes by simply computing the distance between successive embeddings. These results are illustrated with several examples showing the ability of the proposed technique to flag appearance changes while not breaking track.

In this paper, we propose a visual place recognition (VPR) detection method which utilizes multi-level CNN features. High-level CNN features contain much semantic information and can deal with the change of viewpoint, middle-level CNN features contain much geometric information and have good robustness to the change of light and so on. Fully integrating the advantages of high-level and middle-level CNN features, the place recognition detection method will own good robustness to challenge the environment with appearance and viewpoint changes. Due to the high dimension of CNN feature vectors, we pre-process the feature vectors before they are used to the detection. And we introduce how to choose the image representation and compute the similarity score in detail. Finally we perform the experiments on three open datasets with viewpoint and appearance changes, which indicate that the performance of multi-level CNN features outperforms any other single-level CNN features and Fab-Map2.0.

Vision-based localization on robots and vehicles remains unsolved when extreme appearance change and viewpoint change are present simultaneously. The current state of the art approaches to this challenge either deal with only one of these two problems; for example FAB-MAP (viewpoint invariance) or SeqSLAM (appearance-invariance), or use extensive training within the test environment, an impractical requirement in many application scenarios. In this paper we significantly improve the viewpoint invariance of the SeqSLAM algorithm by using state-of-the-art deep learning techniques to generate synthetic viewpoints. Our approach is different to other deep learning approaches in that it does not rely on the ability of the CNN network to learn invariant features, but only to produce“good enough” depth images from day-time imagery only. We evaluate the system on a new multi-lane day-night car dataset specifically gathered to simultaneously test both appearance and viewpoint change. Results demonstrate that the use of synthetic viewpoints improves the maximum recall achieved at 100% precision by a factor of 2.2 and maximum recall by a factor of 2.7, enabling correct place recognition across multiple road lanes and significantly reducing the time between correct localizations.

Introduction Patients with neuroproliferative vestibulodynia (NPV), characterized by persistent entry dyspareunia, allodynia, and hyperalgesia, are often treated by complete vestibulectomy with vaginal advancement flap reconstruction. Vestibulectomy surgery involves subepithelial excision of the 1:00– 11:00 and, if appropriate, 12:00 endodermal vestibular regions, with ectodermal vulva anastomosed to mesodermal vagina. NPV patients considering surgery often express concerns pre-operatively regarding post-operative appearance of their genitalia. Counseling for this has been challenging as, to the best of our knowledge, there have been no reported objective assessments of post-operative changes in genital tissue appearance. Objective To objectively assess changes in vulvar genital tissue appearance in NPV patients who underwent complete vestibulectomy. Methods To assess for change in genital tissue appearance post-operatively, we reviewed charts of patients from a single site who i) underwent complete vestibulectomy between September 2022 and February 2024 and ii) returned to the clinic for a post-operative visit including vulvoscopy with photography at least 3 months after surgery. The primary outcome measure was the 8-item modified Vulvar Genital Tissue Appearance (mVGTA) scale. Vulvoscopic photographs were deidentified and blinded as to whether they were from pre-operative or post-operative visits and read by a trained research team member not involved in patient care. Each photograph was graded for appearance using the mVGTA scale. The 8 items assessed individually and as a total were appearance of: labia majora resorption, labia minora resorption, clitoral glans visualization, clitoral adhesions, introitus pallor, introitus erythema, loss of introital moisture, and posterior fourchette tightness; judged on a 0 to 3 ordinal scale (0 = none; 1 = mild; 2 = moderate; 3 = severe). After unblinding, data were analyzed using descriptive statistics and Wilcoxon matched-pairs signed rank tests. In addition, the vaginal advancement flap was examined for both color and scar tissue between the vagina and vulva using the unblinded post operative photographs. Results Eleven patients were included, age 29 years (range 22-35) who met inclusion criteria. The average time from surgery to post-operative vulvoscopy was 6 months (range 3-13). A total of 178 photographs were graded. (Fig. 1) There were statistically significant improvements from pre- to post-operative mVGTA scores for: clitoral glans visualization (p = 0.023), clitoral adhesions (p = 0.0078), introital pallor (p = 0.0078), introital erythema (p = 0.0117), and the total mVGTA score (p = 0.0029).(Figure 2) Of note, 8 patients underwent lysis of clitoral adhesions at the time of vestibulectomy surgery, all with improved scores in clitoral glans visualization and clitoral adhesions. In all 11 patients, the vaginal advancement flap was pink in color and with no scar tissue identified between the vagina and vulva. There were no statistically significant changes in appearance of the labia majora/minora resorption, labial moisture or posterior fourchette tightness (Fig. 2). Conclusions These findings suggest that complete vestibulectomy improves appearance of genital tissue color and clitoral visualization, and does not change the labia majora, labia minora or posterior fourchette tightness. This should reassure patients concerned about genital appearance considering complete vestibulectomy for NPV. Disclosure No.

Effects of winter and summer conditions on Cd fractionation and bioavailability, bacterial communities and Cd phytoextraction potential of Brachiaria decumbens and Panicum maximum grown in a tropical soil

Temperature management of fresh products begins with proper handling at harvest. Inadvertent abuses, such as lengthy delays to cool or exposure of harvested product to the sun, can detrimentally impact postharvest quality. We used mature-green bell peppers to determine the periods of sun exposure (midday during August in Davis, Calif.) necessary to affect quality attributes (visual appearance, gloss, weight loss, and firmness). Peppers were evaluated after cooling, storage at 7.5 °C for 3 or 7 days, and storage plus 2 days at 20 °C. The impacts of sun exposure, although sometimes barely detectable after cooling, became more noticeable once the peppers were stored. The additional transfer period to 20 °C after storage further accentuated the impact of the exposure. Depending on the experiment, sun-exposed areas reached 45 to 55 °C within 1 hour. Peppers typically lost 0.4% to 0.5% and 1.0% to 1.3% weight during 1- and 2-hour exposures, respectively. Changes in gloss and firmness (whole fruit compression and pulp penetration) were preceded by changes in visual appearance. Exposure to the sun for 0.5 hour did not impact postharvest quality of peppers. Exposures from 1 to 1.5 hours usually resulted in changes apparent only after the storage period. Such exposures are problematic in commercial situations because these peppers are unlikely to be eliminated during sorting on the packingline. Exposures of 1.5 to 2 hours usually resulted in an immediate change in appearance (pitting, blistering, color change).

Omnidirectional cameras are a suitable and cost-effective choice for Visual Place Recognition (VPR), as they provide comprehensive information from the scene regardless of the robot orientation. However, vision sensors are vulnerable to environmental appearance changes (e.g., illumination, weather, season or moving objects). While multi-modal sensing approaches can overcome these challenges, they introduce significant cost and system complexity. This paper introduces PDPR (Panoramic-Depth Place Recognition), a novel fusion framework that enhances the robustness of VPR methods by integrating visual data with geometric features derived from monocular depth estimation techniques, while using a single-camera setup. In the ablation study, both early and late fusion strategies are evaluated to optimally combine appearance-based and depth-derived features. The extensive evaluation on challenging, indoor and outdoor datasets demonstrates that PDPR consistently boosts retrieval performance across multiple state-of-the-art VPR models. Furthermore, this improvement is achieved without requiring any fine tuning, allowing our method to function as a pluggable module for pretrained models. Consequently, this work presents a powerful, practical and low-cost solution for robust VPR, with high potential to scale as monocular depth estimation and VPR models continue to improve. The project website can be found at https://marcosalfaro.github.io/projects-PDPR/ . • Monocular depth estimation is used to enhance place recognition. • A thorough evaluation of preprocessing techniques to enhance the depth maps. • Fusion techniques are designed to leverage visual and geometric data. • A model-agnostic approach that improves the performance even with no fine tuning. • A robust method across different scenarios and lighting conditions.

To help evaluate root distribution patterns, elongation rates of individual roots were measured as a function of soil temperature for Encelia farinosa (a C{sub 3} species), Pleuraphis rigida (C{sub 4}), and Agave deserti (CAM), sympatric codominants in the northwestern Donoran Desert. Measurements were made at current and doubled CO{sub 2} concentrations under winter and summer conditions of air temperature (day/night temperatures of 17 C/10 C and 33 C/22 C, respectively). The three species had different optimal temperatures for root elongation (T{sub opt}) under winter conditions (25 C for E. farinosa, 35 C for P. rigida, and 30 C for A. deserti); T{sub opt} increased by 2-3 C under summer conditions for all three species. The limiting temperatures for elongation also acclimated from winter to summer conditions. The rate of root elongation at T{sub opt} was higher under summer than winter conditions for E. farinosa and P. rigida reflecting conditions for maximum photosynthesis; no difference occurred for A. deserti. Decreased elongation rates at extreme temperatures were associated with less cell division and reduced cell extension. The doubled CO{sub 2} concentration increased average daily root elongation rates for A. deserti under both winter (7%) and summer (12%) conditions, but had no effectmore » for the other two species. Simulations of root elongation as a function of soil temperatures showed that maximum elongation would occur at different depths and during different seasons, contributing to their niche separation. Shading of the soil surface moderated daily variations in soil temperature, reducing seasonal root elongation for winter and spring and increasing elongation for summer. Shading also altered root distribution patterns, e.g., optimal rooting depth for A. deserti and especially P. rigida increased for a hot summer day. 34 refs., 3 figs., 1 tab.« less

To help evaluate root distribution patterns, elongation rates of individual roots were measured as a function of soil temperature for Encelia farinosa (a C3 species), Pleuraphis rigida (C4), and Agave deserti (CAM), sympatric codominants in the northwestern Sonoran Desert. Measurements were made at current and doubled CO2 concentrations under winter and summer conditions of air temperature (day/night temperatures of 17 C/10 C and 33 C/22 C, respectively). The three species had different optimal temperatures for root elongation (Topt) under winter conditions (25 C for E. farinosa, 35 C for P. rigida, and 30 C for A. deserti); Topt increased by 2‐3 C under summer conditions for all three species. The limiting temperatures for elongation also acclimated from winter to summer conditions. The rate of root elongation at Topt was higher under summer than winter conditions for E. farinosa (9 vs. 6 mm d−1) and P. rigida (20 vs. 14 mm d−1), reflecting conditions for maximum photosynthesis; no difference occurred for A. deserti (9 vs. 10 mm d−1). Decreased elongation rates at extreme temperatures were associated with less cell division and reduced cell extension. The doubled CO2 concentration increased average daily root elongation rates for A. deserti under both winter (7%) and summer (12%) conditions, reflecting increased cell extension, but had no effect for the other two species. Simulations of root elongation as a function of soil temperatures showed that maximum elongation would occur at different depths (16‐20 cm for E. farinosa, 4‐8 cm for P. rigida, and 0‐4 cm for A. deserti) and during different seasons (winter to spring for E. farinosa, spring to summer for P. rigida, and all year for A. deserti), contributing to their niche separation. Shading of the soil surface moderated daily variations in soil temperature, reducing seasonal root elongation for winter and spring and increasing elongation for summer. Shading also altered root distribution patterns, e.g., optimal rooting depth for A. deserti and especially P. rigida increased for a hot summer day.