SASIC: Stereo Image Compression with Latent Shifts and Stereo Attention

In this paper, we propose HESIC, an end-to-end trainable deep network for stereo image compression (SIC). To fully explore the mutual information across two stereo images, we use a deep regression model to estimate the homography matrix, i.e., H matrix. Then, the left image is spatially transformed by the H matrix, and only the residual information between the left and right images is encoded to save bitrates. A two-branch auto-encoder architecture is adopted in HESIC, corresponding to the left and right images, respectively. For entropy coding, we use two conditional stereo entropy models, i.e., Gaussian mixture model (GMM) based and context based entropy models, to fully explore the correlation between the two images to reduce the coding bit-rates. In decoding, a cross quality enhancement module is proposed to enhance the image quality based on inverse H matrix. Experimental results show that our HESIC outperforms state-of-the-art SIC methods on InStereo2K and KITTI datasets both quantitatively and qualitatively. Code is available at https://github.com/ywz978020607/HESIC.

An emerging feature of multimedia and telepresence systems is stereo imagery. Stereo images provide an enhanced sense of presence, and have been found to be operationally useful in tasks requiring remote manipulation or judgment of spatial relationships. A conventional stereo system with a single left-right pair needs twice the raw data as a monoscopic imaging system. As a result there has been increasing attention given to image compression methods specialized to stereo pairs. In this paper, the mixed-resolution coding, is a psychophysically justified technique that exploits known facts about human stereovision to code stereopairs in a subjectively acceptable manner, is used to the stereo image compression. By combining both the mixed-resolution coding and SPT(subspace projection technique)-based disparity-compensation techniques, the left image can be compressed by a wavelet-transform-based scheme independent of the right image. By performing low-resolution SPT-based disparity-compensation technique, the disparity is able to predict the low-resolution right image from the left image at a lower resolution using the disparity relation. The low-resolution images are obtained using the wavelet decomposition. At the decoder, the low-resolution right subimage is estimated using the disparity from the low-resolution left subimage. A full-sized resolution is obtained by upsampling with a factor of 4 and resonstructing with the synthesis low pass filter. We provide experimental results, that show that our proposed scheme achieves a PSNR gain of about 0.98 dB as compared to a block-based disparity compensation coding scheme, encoded at the same bit rate.© (2001) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Methods for determining the three-dimensional (3-D) form of complex objects in a scene from a pair of images using the principles of stereo vision are described. A pair of stereo images are epipolar- or raster-scan-aligned to restrict the search to one dimension along the horizontal lines as a first step in location of image feature points. A correlation window of size 5/spl times/5 around a pixel on the left image is superimposed on a similar window centered around every pixel along the same scan line of the right image. A normalized cross-correlation value for every pair of windows is determined. The image points in the right image which give the maximum correlation value are chosen as the corresponding points for the associated points in the left image. Once the corresponding points are found, the disparity is determined. This initial disparity map is noisy. A smoothing or interpolation algorithm has been developed, greatly increasing the number of feature points that can be identified. Sobel Edge detection is employed to locate discontinuities. Disparity values at edge points are added back into the smoothed disparity map, and the modified smoothed disparity map is used for 3-D rendering.

This article proposes a stereo vision based digital watermarking scheme that uses regularized feature-based disparity estimation for copyright protection of stereo content. The watermark data are embedded into the right image of a stereo image using the discrete wavelet transform algorithm, and a disparity vector is extracted from the left and watermarked right images. The disparity vector and the left image are then transmitted to the recipient through a communication channel. At the receiver, the watermarked right image is reconstructed from the received left image and disparity vector by employing regularized feature-based disparity estimation. From the difference between the watermarked and original right images, the embedded watermark image can then be extracted. From experiments using the stereo image pair of "Fichier," "Tunnel," and watermark data of "3D," it was found that the peak signal to noise ratio (PSNR) of the reconstructed right image through the proposed algorithm can be improved by an average of 3.61 dB compared to those of a conventional algorithm. At the same time, the PSNR of the watermark image extracted from this reconstructed right image can also be improved by an average of 2.36 dB when the Quantizer Scale is given as 20.

Stereo image completion (SIC) is to fill holes existing in a pair of stereo images. SIC is more complicated than single image repairing, which needs to complete the pair of images while keeping their stereoscopic consistency. In recent years, deep learning has been introduced into single image repairing but seldom used for SIC. The authors present a novel deep learning‐based approach for SIC. In their method, an X‐shaped fully convolutional network (called SICNet) is proposed and designed to complete stereo images, which is composed of two branches of convolutional neural network layers to encode the context of the left and right images separately, a fusion module for stereo‐interactive completion, and two branches of decoders to produce completed left and right images, respectively. In consideration of both inter‐view and intra‐view cues, they introduce auxiliary networks and define comprehensive losses to train SICNet to perform single‐view coherent and cross‐view consistent completion simultaneously. Extensive experiments are conducted to show the state‐of‐the‐art performances of the proposed approach and its key components.

When stereo images are shown in three-dimensional (3D) display devices of different aspect ratios, the resizing algorithm for single image could lead to shape and depth distortion of the stereo image’s main content. This paper aims to propose a novel method for retargeting stereo image pairs without distorting important objects in the scene while still maintaining the consistency between the left and right images. We extended seam carving algorithm to stereo images. The novelty of our method is that important objects are determined by jointly considering the intensities of gradients and visual fusion area. The retargeted stereo pair has a feasible 3D interpretation that is similar to the original one. Our method protected the important content and reduced the visual distortion in each of the images as well as the depth distortion. Experimental results are presented to demonstrate that the proposed method effectively guaranteed the geometric consistency of resized stereo images.

A New Straight Line Matching Technique by Integration of Vision-based Image Processing

In this chapter, a robust image watermarking algorithm in discrete wavelet transform (DWT) domain for stereo image coding is presented. First, a disparityimage is computed from the pair of stereo images using a frequency domain based matching criteria. Later, this disparity-image is used as a watermark and embedded into the left stereo image based on a modifying singular values concept. The strength of watermark is optimized using a real coded genetic algorithm to achieve the task of invisibility and robustness. The proposed scheme can achieve the following three main advantages. Any illegal user can not extract any information from the watermarked image since the host image is degraded using the ZIG-ZAG sequence. The second is that a legal user can retrieve the embedded watermark (disparity-image) and so able to recover 3-D information and right image of the stereo-pair. The third advantage is its robustness to the various attacks. Experimental results are presented to evaluate the performance of proposed algorithm in terms of accuracy and robustness.KeywordsSingular Value DecomDiscrete Cosine TransformDiscrete Wavelet TransformWatermark ImageWatermark SchemeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

An adaptive connectionist model was implemented to estimate structure from stereo by associating patterns of activity caused by the covariation of the left and right eye images to the surface structure of the environment. Neurallike elements tuned to different disparities and to different resolutions formed the units of the system. Artificial surfaces were computer synthesized and then illuminated to create left and right eye images. Pairs of image and depth maps were taught to the system by linear association and Widrow-Hoff error correction techniques. Extracting depth information about the environment from 2-D stereo image pairs has generally been considered as a two-part process. First, points on the left and right retinas that correspond to a single point in the world are matched, and second, the depth of the point is computed by the disparity information given by this correspondence. Our implementation implicitly incorporates both steps. The system was tested with stereo pairs that had not previously been learned and was able to reconstruct the depth maps with reasonable accuracy.

Adaptation to Natural Binocular Disparities in Primate V1 Explained by a Generalized Energy Model

Summary form only given. Stereo image compression can be pursued in two ways depending on whether the two images are intended to be reconstructed or it is required to code the 3-D information based on the disparity between them. At this stage the authors concentrate on the first option where the coding scheme reproduces the original stereo pair. To increase the degree of compression, the redundancy among the data elements can be exploited by registering or overlaying the two stereo images. A filtering technique is applied to both images in order to emphasize the high frequency and fine structures. >

Almost all the existing stereo algorithms fall under a common assumption that corresponding color or intensity values will be similar like one another. On the other hand, it is also not that true in practice where the image color or intensity values are regularly affected by different radiometric factors like illumination direction, change in image device, illuminant color and so on. For this issue, the information about the raw color of the images which is recorded by the camera should not depend on it totally, and also the common assumptions on color consistency doesn’t influence good (great) between the stereo images in real scenario. Therefore, most of the conventional stereo algorithms can be seriously degraded in terms of performance under radiometric variations. In this work, we intend to develop a new stereo matching algorithm which will be insensitive to change in radiometric conditions between stereo pairs i.e. left image as well as right image. Unlike the other stereo algorithms, we propose a computationally efficient two stage sequential framework for stereo matching which can handle the various radiometric variations between the stereo pairs.Experimental results proves that the proposed method outperforms extremely well compare to other state of the art stereo methods under change in various radiometric conditions for a given stereo pair and it is also found from the results that the execution time is less compare to existing methods.

The digital images have been studied for image classification, enhancement, image compression and image segmentation purposes. In the present work, it is proposed to study the effects of feature selection algorithm on the predictive classification accuracy of algorithms used for discriminating the different plant leaf images. The process involves extracting the important texture features from the digital images and then subjecting them to feature selection and further classification process. The leaf image features have been extracted by using Gabor texture features and these Gabor features are subjected to Random Forest feature selection algorithm for extracting important texture features. The four classification algorithms like K-Nearest Neighbour, J48, Classification and Regression Trees and Random Forest have been used for classification purpose. This study shows that there is a net improvement in the predictive classification accuracy values, when classification algorithms have been applied on selected features over the complete set of features.

Automatic plant species classification has always been a great challenge. Classical machine learning methods have been used to classify leaves using handcrafted features from the morphology of plant leaves which has given promising results. However, we focus on using non-handcrafted features of plant leaves for classification. So, to achieve it, we utilize a deep learning approach for feature extraction and classification of features. Recently Deep Convolution Neural Networks have shown remarkable results in image classification and object detection-based problems. With the help of the transfer learning approach, we explore and compare a set of pre-trained networks and define the best classifier. That set consists of eleven different pre-trained networks loaded with ImageNet weights: AlexNet, EfficientNet BO to B7, ResNet50, and Xception. These models are trained on the plant leaf image data set, consisting of leaf images from eleven different unique plant species. It was found that EfficientNet-B5 performed better in classifying leaf images compared to other pre-trained models. Automatic plant species classification could be helpful for food engineers, people related to agriculture, researchers, and ordinary people.

Due to availability and popularity of stereoscopic displays in the recent years, research into stereo image retargeting is receiving considerable attention. In this paper, we extend the tearable image warping method for stereo image retargeting. Our method retargets both the left and right image of the stereo image pair simultaneously to preserve scene consistency, and minimize distortion using a global optimization algorithm. It is also able to preserve stereoscopic properties of the resulting stereo image. Experimental results show that our approach can preserve the global image context better than stereoscopic cropping, preserve structural details better than stereoscopic seam carving, and protect objects better than stereoscopic traditional warping. Besides, compared to scene warping, our approach can guarantee semantic connectedness.