New stereo visual comfort assessment method based on scene mode classification

Knowledge and use of visual soil structure assessment methods in Brazil – A survey

In this paper, we propose a data-driven foreground object detection technique which can detect foreground objects from a moving camera. We propose to build a data-driven consensus foreground object template (CFOT) and then detect the foreground object region in each frame. The proposed foreground object detection technique is equipped with the following functions: (1) the ability to detect the foreground object captured by a fast moving camera ; (2) the ability to detect a low contrast (spatially/temporally) foreground object; and (3) the ability to detect a foreground object from a dynamic background. There are three contributions of our method: (1) a newly proposed data-driven foreground region decision process for generating the CFOT has been shown robust and efficient; (2) a foreground object probability is proposed for properly dealing with the imperfect initial foreground region estimations; and (3) a CFOT is generated for precise foreground object detection.

Color asymmetry of the left and right views is a common phenomenon in the stereoscopic three‐dimensional (S3D) display, which can lead to visual discomfort. Due to visual discomfort is a subjective sensation, we hypothesize that variations of pupil diameter while viewing S3D images are related to experienced visual comfort. To test this hypothesis, we conducted eye‐tracking experiments on humans viewing hue‐asymmetric stereoscopic images while simultaneously collecting their judgment scores of experienced visual discomfort. From the collected eye‐tracking data, pupil diameter variations were extracted by a normalization formula. Results show that change in hue asymmetry levels causes a significant change in pupil diameter variations. There was a strong negative correlation between the pupil diameter variation and the visual comfort assessment (VCA) score, and the Pearson's r = −0.8936. We conclude that as the hue asymmetric level of stereoscopic image increases, the pupil diameter of the participant becomes smaller and the visual comfort decreases. A visual comfort predication model was fitted well using pupil diameter variation. It predicts that when the pupil diameter variation exceeds 20.57%, the brain feels visual discomfort.

Visual discomfort significantly limits the broader application of stereoscopic display technology. Hence, the accurate assessment of stereoscopic visual discomfort is a crucial topic in this field. Electroencephalography (EEG) data, which can reflect changes in brain activity, have received increasing attention in objective assessment research. However, inaccurately labeled data, resulting from the presence of individual differences, restrict the effectiveness of the widely used supervised learning methods in visual discomfort assessment tasks. Simultaneously, visual discomfort assessment methods should pay greater attention to the information provided by the visual cortical areas of the brain. To tackle these challenges, we need to consider two key aspects: maximizing the utilization of inaccurately labeled data for enhanced learning and integrating information from the brain's visual cortex for feature representation purposes. Therefore, we propose the weakly supervised graph convolution neural network for visual discomfort (WSGCN-VD). In the classification part, a center correction loss serves as a weakly supervised loss, employing a progressive selection strategy to identify accurately labeled data while constraining the involvement of inaccurately labeled data that are influenced by individual differences during the model learning process. In the feature extraction part, a feature graph module pays particular attention to the construction of spatial connections among the channels in the visual regions of the brain and combines them with high-dimensional temporal features to obtain visually dependent spatio-temporal representations. Through extensive experiments conducted in various scenarios, we demonstrate the effectiveness of our proposed model. Further analysis reveals that the proposed model mitigates the impact of inaccurately labeled data on the accuracy of assessment.

This paper investigates the principles of embedding learning to tackle the challenging semi-supervised video object segmentation. Unlike previous practices that focus on exploring the embedding learning of foreground object (s), we consider background should be equally treated. Thus, we propose a Collaborative video object segmentation by Foreground-Background Integration (CFBI) approach. CFBI separates the feature embedding into the foreground object region and its corresponding background region, implicitly promoting them to be more contrastive and improving the segmentation results accordingly. Moreover, CFBI performs both pixel-level matching processes and instance-level attention mechanisms between the reference and the predicted sequence, making CFBI robust to various object scales. Based on CFBI, we introduce a multi-scale matching structure and propose an Atrous Matching strategy, resulting in a more robust and efficient framework, CFBI+. We conduct extensive experiments on two popular benchmarks, i.e., DAVIS and YouTube-VOS. Without applying any simulated data for pre-training, our CFBI+ achieves the performance ( J& F) of 82.9 and 82.8 percent, outperforming all the other state-of-the-art methods. Code: https://github.com/z-x-yang/CFBI.

DIBR(Depth Image Based Rendering)을 이용한 영상 합성 시에 발생하는 경계 잡음은 원래 전경 영역에 속하던 화소가 배경으로 흩어져 나와 생성된 잡음이며, 이는 주로 참조 영상과 깊이지도 간 경계 불일치나 참조 영상에서의 블러링 때문에 발생된다. 영상 합성 과정에서 발생된 홀 영역은 일반적으로 주위 화소를 이용하여 채워지게 되므로, 홀에 인접한 경계 잡음은 합성 영상의 화질을 저하시키는 주요 원인으로 작용한다. 이 문제를 해결하기 위해 본 논문은 깊이지도의 전처리를 이용한 새로운 경계 잡음 제거 알고리즘을 제안한다. 기존의 전처리 기법들은 경계 불일치에 따른 경계 잡음의 제거를 위해 참조 영상과 깊이지도의 경계가 일치되도록 깊이지도를 수정한다. 그러나 대부분의 기존 기법들이 신호 기울기 기반의 단순 경계 탐색 알고리즘에 기반을 두고 있어 블러링이 존재하는 경계에서는 탐색 성능의 저하가 나타난다. 제안 알고리즘은 이의 해결을 위해 2단계 경계 탐색을 이용함으로써 이행 영역과 배경 영역 간 경계를 보다 효과적으로 탐색할 수 있는 구조를 제안하였다. 실험 결과를 통해 제안 알고리즘이 기존 알고리즘에 비해 우수한 경계 잡음 제거 성능을 가짐을 보인다. The boundary noise in image syntheses using DIBR consists of noisy pixels that are separated from foreground objects into background region. It is generated mainly by edge misalignment between the reference image and depth map or blurred edge in the reference image. Since hole areas are generally filled with neighboring pixels, boundary noise adjacent to the hole is the main cause of quality degradation in synthesized images. To solve this problem, a new boundary noise removal algorithm using a preprocessing of the depth map is proposed in this paper. The most common way to eliminate boundary noise caused by boundary misalignment is to modify depth map so that the boundary of the depth map can be matched to that of the reference image. Most conventional methods, however, show poor performances of boundary detection especially in blurred edge, because they are based on a simple boundary search algorithm which exploits signal gradient. In the proposed method, a two-step hierarchical approach for boundary detection is adopted which enables effective boundary detection between the transition and background regions. Experimental results show that the proposed method outperforms conventional ones subjectively and objectively.

The design of campus landscape is expected to reflect the institution values, provide the character and the spirit of place to campus environment. It is highly influenced by visual experience and impression of campus environment. There are two methods in landscape visual quality assessment: descriptive assessment method and evaluative assessment method. Both assessment methods cannot be done simultaneously, but as a sequence phase. Descriptive assessment method should be done first to obtain a reference for evaluative assessment method. Landscape visual quality evaluative assessment method is used to measure the level of public assessment about visual quality and visual response. Information Processing Theory is used to develop the visual quality evaluative assessment method and to obtain unified integration between descriptive assessment and evaluative assessment. The development of evaluative assessment method includes the process of comparing, averaging, or determine the ranking of each environmental hue or landscape areas in campus, based on the public or college user community preferences.

In this paper, we try to solve the problem that it is difficult to distinguish between the background and foreground regions in vein images, and improve the feature extraction of finger veins by using maximum curvature method (MCM), which is easily affected by noise. In this paper, we use the improved maximum curvature method to extract finger vein features. Different from the maximum curvature method(MCM), we try an enhancement mechanism to improve the anti-noise ability. Firstly, the improved mask filtering method is used to locate the finger region, which can effectively separate the foreground region from the background region. When it comes to feature extraction, the Gabor wavelet transform is used to remove the noise of the image, and the basic gray-level grouping (GLG) method is used to increase the contrast of the image. Finally, the vein width is obtained by the maximum curvature method. The method is based on two datasets, namely, MMCBNU_6000 finger vein database and SDUMLA-HMT finger vein database. Experiments show that the recognition rates of the two databases are 99.27% and 98.13% respectively.

Alpha matting is known as a process of accurately separating a foreground object from an image and it has been extensively studied in both academic and industrial areas as an important part of multimedia technology. However, most existing methods utilize single image to get its alpha matte. In this paper, we propose a novel chroma keying method based on stereo images, which can be applied to post-process the alpha matte generated by any matting approach. Given a pair of stereo images, a new matting Laplacian matrix is constructed based on the affinities between matching pixels and their neighbors from two frames. Based on the new matting Laplacian matrix, a new cost function is also formulated to estimate alpha values of the reference image through the propagation between stereo images. The experimental results illustrate that this method can be applied to different categories of existing alpha matting approaches and can obtain alpha matte of higher quality.

The underwater color disparity is an important cue for enhancing an underwater image. Applying the underwater color disparity indiscriminately to the entire underwater image tends to give rise to foreground-background crosstalk with either excessive foreground or insufficient background enhancement. To address the discriminativeness between underwater color disparities in foreground and background regions, we develop a discriminative underwater image enhancement method empowered by large foundation model technology. We first utilize the Segment Anything Model to generate segmentation masks, dividing the underwater image into foreground and background regions. This enables accurate foreground-background separation. Then, we conduct adaptive color compensation and fusion to improve the color histogram similarity for foreground and background regions separately. This corrects color deviations and improves contrasts in a discriminative manner that avoids the foreground-background crosstalk. Finally, we propose high-frequency edge fusion to extract high-frequency components from both the original underwater image and the fused image, and then fuse these components to obtain the final enhanced image. This eliminates blurred details arising from the discriminative processing of foreground and background regions. Our method represents the pioneering application of large foundation model technology to empower underwater image enhancement. Experimental results indicate that our method outperforms nine state-of-the-art underwater image enhancement methods in visual quality, achieves superior results across five underwater image quality evaluation metrics on three underwater image datasets, and is beneficial for practical applications such as underwater feature matching. We release our code at https://gitee.com/wanghaoupc/UIE SAM.

We propose visual comfort assessment for stereoscopic 3D (S3D) images based on salient discomfort regions. Color-based saliency successfully represents visual attention because that the human visual system (HVS) focuses on the most salient region in an image. Disparity-based saliency effectively expresses visual comfort in S3D images. Based on two saliencies, we extract salient discomfort regions which mostly determine the overall visual comfort degree in S3D images. Then, we predict the visual comfort score from salient discomfort regions by a disparity feature vector which combines saliency-weighted disparity and maximum disparity. Finally, we provide visual comfort index maps of S3D images based on visual comfort scores, thus showing perceptually salient discomfort regions and their degrees. Experimental results demonstrate that the proposed method achieves significantly high accuracy in visual comfort assessment as compared to existing methods.

Video segmentation has drawn increasing interest in multimedia applications. This paper proposes a novel joint space-time-range domain adaptive mean shift filter for video segmentation. In the proposed method, segmentation of moving/static objects/background is obtained through inter-frame mode-matching in consecutive frames and motion vector mode estimation. Newly appearing objects/regions in the current frame due to new foreground objects or uncovered background regions are segmented by intra-frame mode estimation. Simulations have been conducted to several image sequences, and results have shown the effectiveness and robustness of the proposed method. Further study is continued to evaluate the results.

This research addresses the issue of automatically segmenting color images into foreground (F) and back-ground (B) regions with the assumption that background regions are relatively smooth but may have gradually varying colors or be slightly textured. A multi-level segmentation scheme is used that involves color clustering, unsupervised segmentation using MDL (minimum description length) principle, edge-based F/B separation, and integrated F/B segmentation. The approach has been tested on more than 100 images. Some of the experimental results are presented.

This article studies the perceptual quality evaluation method of humanistic images based on the characteristics of human vision. First, this study obtained the disparity map of the human visual cultural image and used the edge map and saliency map to correct the weight of the disparity map. The weighted disparity map was then integrated into a perceptible score using a Minkowski fused deep neural network. A multi-scale method was employed to process the image to obtain the visual effect score. Taking Polytechnic University of Lausanne (EPFL) as an example, an experimental study was conducted on the three dimensional visual quality assessment method proposed in this article. The test results with the EPFL library show that the target score obtained by this method has good consistency and monotonicity with the subjective score of the EPFL library. The article's results verify the effectiveness of the visual effects and aesthetics evaluation method proposed in this article.

A movie captured by a wearable camera affixed to an actor's body gives audiences the sense of "immerse in the movie". The raw movie captured by wearable camera needs stabilization with jitters due to ego-motion. However, conventional approaches often fail in accurate ego-motion estimation when there are moving objects in the image and no sufficient feature pairs provided by background region. To address this problem, we proposed a new approach that utilizes an additional synchronized video captured by the camera attached on the foreground object (another actor). Formally we configure above sensor system as two face-to-face moving cameras. Then we derived the relations between four views including two consecutive views from each camera. The proposed solution has two steps. Firstly we calibrate the extrinsic relationship of two cameras with an AX=XB formulation, and secondly estimate the motion using calibration matrix. Experiments verify that this approach can recover from failures of conventional approach and provide acceptable stabilization results for real data.