Calibration-free gaze sensing using saliency maps

Characteristics of the human visual field are well known to be different in central (fovea) and peripheral areas. Existing computational models of visual saliency, however, do not take into account this biological evidence. The existing models compute visual saliency uniformly over the retina and, thus, have difficulty in accurately predicting the next gaze (fixation) point. This paper proposes to incorporate human visual field characteristics into visual saliency, and presents a computational model for producing such a saliency map. Our model integrates image features obtained by bottom-up computation in such a way that weights for the integration depend on the distance from the current gaze point where the weights are optimally learned using actual saccade data. The experimental results using a large number of fixation/saccade data with wide viewing angles demonstrate the advantage of our saliency map, showing that it can accurately predict the point where one looks next.

Region of interest (ROI) detection plays an important role in object detection. It needs to be accurate and fast in some applications like real time disaster victim detection systems. ROI can reduce time and search space in detecting objects. In this paper visual saliency map is used for ROI detection. In most literature, most of ROI detection models only concentrate on reducing false positive (detecting wrong objects as intended ones) rate rather than false negative (missing intended object). In disaster victim detection, missing disaster victims is more important than detecting other objects like victim. So, the proposed method also focuses on reducing false negative error rate in object detection. In the proposed system, local entropy feature is added in Graph Based Visual Saliency (GBVS) map in addition to colour, orientation and shape feature maps.

Computational modeling of visual attention is an active area of research. These models have been successfully employed in applications such as robotics. However, most computational models of visual attention are developed in the context of natural scenes, and their role with medical images is not well investigated. As radiologists interpret a large number of clinical images in a limited time, an efficient strategy to deploy their visual attention is necessary. Visual saliency maps, highlighting image regions that differ dramatically from their surroundings, are expected to be predictive of where radiologists fixate their gaze. We compared 16 state-of-art saliency models over three medical imaging modalities. The estimated saliency maps were evaluated against radiologists' eye movements. The results show that the models achieved competitive accuracy using three metrics, but the rank order of the models varied significantly across the three modalities. Moreover, the model ranks on the medical images were all considerably different from the model ranks on the benchmark MIT300 dataset of natural images. Thus, modality-specific tuning of saliency models is necessary to make them valuable for applications in fields such as medical image compression and radiology education.

Modeling visual saliency map of an image provides important information for image semantic understanding in many applications. Most existing computational visual saliency models follow a bottom-up framework that generates independent saliency map in each selected visual feature space and combines them in a proper way. Two big challenges to be addressed explicitly in these methods are (1) which features should be extracted for all pixels of the input image and (2) how to dynamically determine importance of the saliency map generated in each feature space. In order to address these problems, we present a novel saliency map computational model based on tensor decomposition and reconstruction. Tensor representation and analysis not only explicitly represent image's color values but also imply two important relationships inherent to color image. One is reflecting spatial correlations between pixels and the other one is representing interplay between color channels. Therefore, saliency map generator based on the proposed model can adaptively find the most suitable features and their combinational coefficients for each pixel. Experiments on a synthetic image set and a real image set show that our method is superior or comparable to other prevailing saliency map models.

Visual saliency map has been proposed as a computational model for estimating the visual attention of a human. The saliency map is used to estimate the bottom-up visual attention for still and moving images. However, under the condition including the top-down visual attention, e.g. advertisement, the accuracy of visual attention estimated from the saliency map decreased. The character feature is considered as one of the factors that the deterioration of the saliency map accuracy is induced. In this study, we hypothesized that the features of characters have the saliency to induce the visual attention. Two types of experiments were performed to test this hypothesis. The still images inserted with both of characters (HIRAGANAs in Japanese or Thai strings) and simple symbols were used as visual stimuli in the experiments. The visual stimuli were presented to the subjects for a short period (2s) to exclude the effect of top-down attention. In result, the fixation probability of the character (both HIRAGANAs and Thais) region in the image was higher than that of the symbol region. The paired t-test provided the significant difference of the fixation ratio between HIRAGANAs and symbols (p < 0.001). The same is true for the paired t-test between Thais and symbols (p < 0.001). Thus, the present results indicate the visual saliency of characters.

A fusion algorithm of infrared and visible images based on visual saliency map (VSM) and nonsubsampled contourlet transform (NSCT) was proposed. Usually, the visual salient region of infrared image is directed towards the targets which interpret the most important information in the image. For the given registered infrared and visible images, firstly, the frequency-tuned (FT) saliency detection algorithm is used to calculate the visual saliency map of infrared and visible images. Then the size of each salient region is obtained by maximizing entropy. In order to capture the details of the infrared and visible images, the low and high frequency fusion coefficients of nonsubsampled contourlet transform (NSCT) are selected based on region saliency, region energy (RE) and region sharpness (RS). Four different data sets from TNO, Human Factors are employed, and experimental results indicate that the proposed method can achieve a superior performance compared with other fusion methods in both subjective and objective assessments.

Multimodal medical image fusion based on visual saliency map and multichannel dynamic threshold neural P systems in sub-window variance filter domain

Infrared and visible image fusion based on visual saliency map and weighted least square optimization

In this article, we propose a simple and efficient method for computing an image saliency map, which performs well on both salient region detection and as well as eye gaze prediction tasks. A large number of distinct sub-windows with random co-ordinates and scales are generated over an image. The saliency descriptor of a pixel within a random sub-window is given by the absolute difference of its intensity value to the mean intensity of the sub-window. The final saliency value of a given pixel is obtained as the sum of all saliency descriptors corresponding to this pixel. Any given pixel can be included by one or more random sub-windows. The recall-precision performance of the proposed saliency map is comparable to other existing saliency maps for the task of salient region detection. It also achieves state-of-the-art performance for the task of eye gaze prediction in terms of receiver operating characteristics.KeywordsBottom-up Visual AttentionSaliency MapSalient Region DetectionEye Fixation

In this paper, we address the problem of free head motion in appearance-based gaze estimation. This problem remains challenging because head motion changes eye appearance significantly, and thus, training images captured for an original head pose cannot handle test images captured for other head poses. To overcome this difficulty, we propose a novel gaze estimation method that handles free head motion via eye image synthesis based on a single camera. Compared with conventional fixed head pose methods with original training images, our method only captures four additional eye images under four reference head poses, and then, precisely synthesizes new training images for other unseen head poses in estimation. To this end, we propose a single-directional (SD) flow model to efficiently handle eye image variations due to head motion. We show how to estimate SD flows for reference head poses first, and then use them to produce new SD flows for training image synthesis. Finally, with synthetic training images, joint optimization is applied that simultaneously solves an eye image alignment and a gaze estimation. Evaluation of the method was conducted through experiments to assess its performance and demonstrate its effectiveness.

Mechanical property prediction plays a crucial role in the steel industry, enabling better materials selection and enhancing production efficiency. In this study, we propose a novel framework that facilitates optimal materials selection for steel’s alloying elements, based on accurate predictions of the Jominy hardness curve. Leveraging Gaussian Process (GP) regression, we provide probabilistic predictions of steel hardenability characteristics from alloy element composition. Taking it a step further, our framework incorporates these accurate predictions into a constrained optimization process, yielding optimal compositions that reduce overall spending while meeting performance specifications. Through data obtained from 1080 steel samples, our GP regression model exhibits high accuracy, achieving an RMSE of 1.37 and showcasing significant improvements in the field. Moreover, our constrained optimization utilizing the GP model and historical market data reveals an average cost reduction of 18% on alloying element expenses, highlighting the tangible cost-saving potential of this approach. By leveraging Gaussian Process (GP) regression, we not only achieve accurate predictions of the Jominy hardness curve based on alloy element composition, but we also introduce a crucial element of uncertainty quantification. This empowers us to place trust in the results of our optimization process, ensuring robust and reliable materials selection. The integration of GP regression and optimization provides a powerful tool for achieving cost-effective materials selection and marks a significant advancement compared to existing studies. This research underscores the promise of machine learning in the steel industry, demonstrating its ability to yield substantial cost savings and enhance decision-making in materials selection.

Data storage via multimedia technology is more preferred as the information in multimedia contain rich meanings and are concise when compared to the traditional textual information. However, efficient information retrieval is a crucial factor in such storage. This paper presents a cognitive classification based visual saliency guided model for the efficient retrieval of information from multimedia data storage. The Itti visual saliency model is described here for generation of an overall saliency map with the integration of color saliency, intensity and direction maps. Multi-feature fusion paradigms are used for providing clear description of the image pattern. The definition is based on two stages namely complexity based on cognitive load and classification of complexity at a cognitive level. The image retrieval system is finalized by integrating a group sparse logistic regression model. In complex scenarios, the baselines are overcome by the proposed system when tested on multiple databased as compared to other state-of-the-art models.

We propose a method for extracting a gaze region from an observed image by analyzing human's view directions and image information. The view direction of the user, which is represented as a 2D gaze point in the observed image, is obtained by an eye-mark recorder at every image-capturing timing. All gaze points are translated to one of the images for extracting the gaze region based on the history of the view directions. The system divides all gaze points into several groups by comparing color information etc., and then generates several convex hulls as initial regions. Each initial region is extended based on its color information and the spatial distribution of the gaze points. All regions are finally integrated and regarded as the gaze region.

We propose a method for extracting a gaze region from an observed image by analyzing human's view directions and image information. The view direction of the user, which is represented as a 2D gaze point in the observed image, is obtained by an eye-mark recorder at every image-capturing timing. All gaze points are translated to one of the images for extracting the gaze region based on the history of the view directions. The system divides all gaze points into several groups by comparing color information etc., and then generates several convex hulls as initial regions. Each initial region is extended based on its color information and the spatial distribution of the gaze points. All regions are finally integrated and regarded as the gaze region.

Visual saliency distribution maps for explaining time-series AI models used in continuous production of textile fibers