Pairwise Comparison-Based Salient Object Ranking Using Multimodal Large Models.

Humans can easily find a specific category of objects in a complex scene, simulating this mechanism has both scientific and economic impact. The existing research usually focuses on where humans will look, overlooking the effects of the category of objects on visual attention. In this paper, we proposed a computational visual model aimed at mimicking human visual attention while they were looking for a specific category of objects. First, we proposed a machine learning-based Probabilistic Latent Semantic Analysis (PLSA) model to recognize the generic category of objects and find their locations. The PLSA model parameters are learned in a supervised manner. Then, the category and location information of objects are used as high-level information, together with low-level features of the same objects, to train a Support Vector Regression (SVR) model. With this SVR model, we can predict the visual attention for a specific category. To demonstrate the effectiveness and applicability of the proposed method, we conducted comprehensive experiments. The results show that the model can provide a reasonable estimate for human visual attention under the visual search task for a specific category. Besides, we evaluated the proposed model's ability to discover a specific category of objects and their locations on the natural image dataset. The results show that besides facilitating the human visual attention mechanism research, the proposed model can also be applied to find a specific category of objects from a set of images.

Pneumothorax requires rapid recognition and accurate interpretation of chest X-rays (CXRs), particularly in acute settings where delays can have serious consequences. With the emergence of advanced image interpretation models capable of visual analysis, their diagnostic reliability in radiology practice remains to be determined. This study aimed to assess the diagnostic performance of three state-of-the-art systems in detecting pneumothorax using a large, well-annotated dataset. A total of 10,675 CXRs from the publicly available SIIM-ACR Pneumothorax Segmentation dataset were analyzed. Three multimodal models (GPT-4o, Gemini 2 Pro, and Claude 4 Sonnet) were evaluated using a uniform, image-based approach. Each model's binary outputs (presence: 1, absence: 0) were compared with reference results to determine accuracy, sensitivity, specificity, precision, and F1 scores. Additional subgroup analyses were conducted across pneumothorax size categories: small, medium, and large. Pairwise statistical comparisons were performed using McNemar's test. Sensitivity, specificity, and overall accuracy are reported with corresponding 95% confidence intervals. The prevalence of pneumothorax in the dataset was 22.3% (n = 2,379). All models demonstrated high specificity (above 0.90) but consistently low sensitivity (0.16-0.36). The best overall performance was observed with Gemini 2, which achieved an accuracy of 0.79 and specificity of 0.95, whereas Claude 4 showed greater sensitivity (0.20-0.34) across lesion-size categories. Diagnostic performance improved with increasing pneumothorax size, but smaller lesions remained difficult to identify. Pairwise comparisons confirmed statistically significant differences among all evaluated systems (P < 0.050). In this large-scale evaluation, the tested models exhibited strong reliability in identifying normal examinations but limited ability to detect subtle or small pneumothoraxes. Despite high specificity, low sensitivity limits the use of current Multimodal large language models as rule-out tools for pneumothorax. With continued refinement, these models may eventually support radiologists by improving workflow efficiency and diagnostic confidence. Automated systems capable of high specificity but low sensitivity should not be relied upon to exclude pneumothorax. However, they may serve as valuable assistants for confirming positive findings and prioritizing urgent cases in busy clinical workflows.

Unconscious attention shift has been proven to be automatic, effortless, and unresisting; however, whether it is applicable to images of complex scene remains to be verified. We propose a novel method for directing human visual attention by flashing a short-duration visual stimulus before presenting the image. This method directs viewer's visual attention to an arbitrary location in the image without engaging the viewer's awareness. Unlike previous studies using simple stimuli (e.g. simple geometric figures, image of uniform color, or artificial white-noise), we use complex color images for guiding unconscious visual attention in real-world. To our best knowledge, this is the first study demonstrating that short-duration cues presented at arbitrary locations can attract human visual attention in a complex scene. The proposed method is useful for applications that require efficient and unresisting attention shift to the target image area.

We humans are easily able to instantaneously detect the regions in a visual scene that are most likely to contain something of interest. Exploiting this pre-selection mechanism called visual attention for image and video processing systems would make them more sophisticated and therefore more useful. This paper briefly describes various computational models of human visual attention and their development, as well as related psychophysical findings. In particular, our objective is to carefully distinguish several types of studies related to human visual attention and saliency as a measure of attentiveness, and to provide a taxonomy from several viewpoints such as the main objective, the use of additional cues and mathematical principles. This survey finally discusses possible future directions for research into human visual attention and saliency computation.

Semantic Similarity is one of the highest levels of NLP. STS has significant advantages in NLP applications like information retrieval, information extraction, text summarization, data mining, machine translation, and other tasks. This research aims to present a deep learning approach for capturing semantic textual similarity (STS) in the Guragigna. The methodology involves collecting a Guragigna language corpus and preprocessing the text data and text representation is done using the Universal Sentence Encoder (USE), along with word embedding techniques including Word2Vec and GloVe and mean Square Error (MSE) is used to measure the performance. In the experimentation phase, models like LSTM, Bidirectional RNN, GRU, and Stacked RNN are trained and evaluated using different embedding techniques. The results demonstrate the efficacy of the developed models in capturing semantic textual similarity in the Guragigna language. Across different embedding techniques, including Word2Vec, GloVe, and USE, the Bidirectional RNN model with USE embedding achieves the lowest MSE of 0.0950 and the highest accuracy of 0.9244. GloVe and Word2Vec embedding also show competitive performance with slightly higher MSE and lower accuracy. The Universal Sentence Encoder consistently emerges as the top-performing embedding across all RNN architectures. The research results demonstrate the effectiveness of LSTM, GRU, Bi RNN, and Stacked RNN models in measuring semantic textual similarity in the Guragigna language.

Human visual attention is an area of research that has a strong effect on the field of human-robot-interaction. It has various applications for human-care robots and service robots. The ability of human visual attention to acquire knowledge on informative objects through interaction with the environment is an important research issue in the field of human-computer interaction and augmented cognition. Such knowledge can be acquired by automatically detecting visually attentive regions using the fixation extracted from eye-tracking data. Therefore, the optimal fixation must be selected from human eye-tracking data to detect the region of interest. In this paper, eye movement tracking was accurately used to capture human eye movements and to characterize the location and extent of a user's interest as an input mechanism to drive the interaction system. Furthermore, both top-down and bottom-up processes of the human visual system were at work in the process of selective attention. The correlation between human eye movement information and the bottom-up saliency map was calculated and compared with the correlation that was calculated by combining the top-down and bottom-up processes. The experiment results showed that the human visual attention system cannot be constructed with the bottom-up process alone and requires the combination of the top-down and bottom-up processes together.

Entity representation for pairwise collaborative ranking using restricted Boltzmann machine

The mobile robots are equipped with sensitive audio visual sensors, usually microphone arrays and video cameras. They are the main sources of audio visual information to perform suitable mobile robots navigation tasks, modeling the human audio visual perception. The results from the audio and visual perception algorithms are widely used, separate or in conjunction (audio visual perception) in the mobile robots navigation, for example to control mobile robots motion in applications like people and objects tracking, surveillance systems, etc. The effectiveness and precision of the audio visual perception methods in the mobile robots navigation can be enhanced combining audio visual perception with audio visual attention. Sufficient relative knowledge exists, describing the phenomena of human audio and visual attention. Such approaches are usually based on a lot of physiological, psychological, medical and technical experimental investigations relating the human audio and visual attention, with the human audio and visual perception with the leading role of the brain activity. Of course, the results from these investigations are very important, but not sufficient for the mobile robots audio visual attention modeling, mainly because of brain missing in mobile robots audio visual perception systems. Therefore, in this chapter is proposed to use the existing definitions and models for human audio and visual attention, adapting them to the models of mobile robots audio and visual attention and combining with the results from the mobile robots audio and visual perception in the mobile robots navigation tasks.

Co-saliency detection aims at finding the salient regions from multiple images which capture the focus of human visual system. In this paper, a novel visual co-saliency algorithm is proposed, which adopts three human visual attention cues: contrast, center-bias and symmetry. In order to apply co-saliency to the detection of traffic signs, a traffic sign detection framework based on visual co-saliency in complex scenes is devised. The detection process involves two stages. In the first stage, a cluster-based co-saliency model is built to generate the final co-saliency map. In the second stage, a geometric structure constraint model is constructed to discriminate the detected salient objects and then accurately achieve location of traffic signs. The advantage of our approach lies in the integration of bottom-up and top-down visual processing, and no heavy learning tasks. Experiments on a variety of benchmark databases illustrate high precision, high recall and operation efficiency of the proposed approach. Besides, for traffic sign detection it overcomes the interference of complex urbanization backgrounds. Furthermore, the best trade-off between precision and recall on warning signs is achieved, reaching 93.30% and 89.06%, respectively.

Visual odometry is critical in visual simultaneous localization and mapping for robot navigation. However, the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly distributed features because dense features occupy excessive weight. Herein, a new human visual attention mechanism for point-and-line stereo visual odometry, which is called point-line-weight-mechanism visual odometry (PLWM-VO), is proposed to describe scene features in a global and balanced manner. A weight-adaptive model based on region partition and region growth is generated for the human visual attention mechanism, where sufficient attention is assigned to position-distinctive objects (sparse features in the environment). Furthermore, the sum of absolute differences algorithm is used to improve the accuracy of initialization for line features. Compared with the state-of-the-art method (ORB-VO), PLWM-VO show a 36.79% reduction in the absolute trajectory error on the Kitti and Euroc datasets. Although the time consumption of PLWM-VO is higher than that of ORB-VO, online test results indicate that PLWM-VO satisfies the real-time demand. The proposed algorithm not only significantly promotes the environmental adaptability of visual odometry, but also quantitatively demonstrates the superiority of the human visual attention mechanism.

We propose a method of predicting human egocentric visual attention using bottom-up visual saliency and egomotion information. Computational models of visual saliency are often employed to predict human attention; however, its mechanism and effectiveness have not been fully explored in egocentric vision. The purpose of our framework is to compute attention maps from an egocentric video that can be used to infer a person’s visual attention. In addition to a standard visual saliency model, two kinds of attention maps are computed based on a camera’s rotation velocity and direction of movement. These rotation-based and translation-based attention maps are aggregated with a bottom-up saliency map to enhance the accuracy with which the person’s gaze positions can be predicted. The efficiency of the proposed framework was examined in real environments by using a head-mounted gaze tracker, and we found that the egomotion-based attention maps contributed to accurately predicting human visual attention.

In a study of the neural processes that mediate visual attention in humans, 32-channel recordings of event-related potentials were obtained from 14 normal subjects while they performed a spatial attention task. The generator locations of the early C1, P1, and N1 components of the visual evoked response were estimated by means of topographic maps of voltage and current source density in conjunction with dipole modelling. The topography of the C1 component (ca. 85 ms post-stimulus) was consistent with a generator in striate cortex, and this component was unaffected by attention. In contrast, the P1 and N1 components (ca. 95 and 170 ms) exhibited current density foci at scalp sites overlying lateral extrastriate cortex and were larger for attended stimuli than for unattended stimuli. The voltage topographies in the 75-175 ms latency range were modeled with a 5-dipole configuration consisting of a single striate dipole and left-right pairs of dipoles located in lateral extrastriate and inferior occipito-temporal areas. This model was found to account for the voltage topographies produced by both attended and unattended stimuli with low residual variance. These results support the proposal that visual-spatial attention modulates neural activity in extrastriate visual cortex but does not affect the initial evoked response in striate cortex.

Chapter 92 - The Selective Tuning Model for Visual Attention

Electrophysiology of Visual Attention in Humans

Attention can dramatically enhance behavioral performance based on a visual stimulus, but the degree to which attention modulates activity in early visual cortex is unclear. Whereas single-unit studies of spatial attention in monkeys have repeatedly revealed relatively modest attentional modulations in V1, human functional magnetic resonance imaging studies demonstrate a large attentional enhancement of the blood oxygen level-dependent (BOLD) signal in V1. To explore this discrepancy, we used intracranial electrodes to directly measure the effect of spatial attention on the responses of neurons near the human occipital pole. We found that spatial attention does not robustly modulate stimulus-driven local field potentials in early human visual cortex, but instead produces modest modulations that are consistent with those seen in monkey neurophysiology experiments. This finding suggests that the neuronal activity that underlies visual attention in humans is similar to that found in other primates and that behavioral state may alter the linear relationship between neuronal activity and BOLD.