Video coding and processing: A survey

A new multiview just-noticeable-depth-difference(MJNDD) Model is presented and applied to compress the joint multiview video plus depth. Many video coding algorithms remove spatial and temporal redundancies and statistical redundancies but they are not capable of removing the perceptual redundancies. Since the final receptor of video is the human eyes, we can remove the perception redundancy to gain higher compression efficiency according to the properties of human visual system (HVS). Traditional just-noticeable-distortion (JND) model in pixel domain contains luminance contrast and spatial-temporal masking effects, which describes the perception redundancy quantitatively. Whereas HVS is very sensitive to depth information, a new multiview-just-noticeable-depth-difference(MJNDD) model is proposed by combining traditional JND model with just-noticeable-depth-difference (JNDD) model. The texture video is divided into background and foreground areas using depth information. Then different JND threshold values are assigned to these two parts. Later the MJNDD model is utilized to encode the texture video on JMVC. When encoding the depth video, JNDD model is applied to remove the block artifacts and protect the edges. Then we use VSRS3.5 (View Synthesis Reference Software) to generate the intermediate views. Experimental results show that our model can endure more noise and the compression efficiency is improved by 25.29 percent at average and by 54.06 percent at most compared to JMVC while maintaining the subject quality. Hence it can gain high compress ratio and low bit rate.

This paper presents a method for localizing noise-corrupted areas in quality degraded video frames, and for reducing the additive noise by utilizing the temporal redundancy in the video sequence. In the proposed algorithm, the local variance of each pixel is computed to obtain the spatial distribution of noise. After adaptive thresholding, region clustering, and merging, the corrupted areas of highest energy are detected. Due to the high temporal redundancy in the video sequence, the corrupted information can be compensated by overlapping the corrupted regions with the appropriate regions from adjacent video frames. The corresponding pixel locations in the adjacent frames are computed by using image registration and warping techniques. New pixel values are calculated based upon multi-frame stacking. Pixel values in the adjacent frames are weighted according to registration errors, whereas the values in the noisy frame are evaluated according to local variance. Knowing the location of the local noise enables the denoising process to be much more specific and accurate. Moreover, since only a portion of the frame is processed, as compared to standard denoising methods that operate on the entire frame, the details and features in other areas of the frame are preserved. The proposed scheme is applied to UAV video sequences, where the outstanding noise localization and reduction properties are demonstrated.

In the clinic, myocardial perfusion computed tomography (MPCT) imaging is commonly used to detect and assess myocardial ischemia quantitatively. However, repeated scanning on the myocardial region in the cine mode will increase the radiation dose for patients. With lowering radiation dose, the quality of images are degraded by noise induced artifact, which hampers the diagnostic accuracy. Therefore, in this paper, we propose a redundancy information induced iterative reconstruction framework for high quality MPCT images at the case of low dose. MPCT images have redundant structural information within frames and highly similarity between adjacent frames. Inspired by the two properties, in this work we propose a penalized weighted least-squares (PWLS) model incorporating NLM and TV based hybrid constraints, which is referred to as PWLS-aviNLM-TV for simplicity. The proposed algorithm can effectively eliminate noise and artifacts by taking into account the similarity between adjacent frames and redundancy information within frames, which also can improve spatial resolution within frames and maintain temporal resolution. The experimental results on the 4D extended cardiac-torso (XCAT) phantom and preclinical porcine dataset demonstrates that the PWLS-aviNLM-TV algorithm obtains better performance in terms of noise reduction and artifacts suppression than the PWLS-TV and PWLSaviNLM algorithm. Moreover, the proposed algorithm can preserve the edges and detail information thereby efficiently differentiate ischemia from myocardium. The present redundancy information induced reconstruction algorithm can reconstruct high-quality images from low-dose MPCT for better clinical imaging diagnosis.

Picture encoding methods based on the iterated function system (IFS) have been reported. However, they are designed for still pictures and few reports have been made for moving pictures. This paper proposes a coding method for moving pictures based on the recurrent iterated function system (RIFS). In RIFS, the function systems of contraction mappings in multiple spaces are composed. In this study, it is assumed that by using a sequence of moving pictures for multiple spaces, moving pictures can be coded. In this proposed method, several frames are treated as a group and the search in coding is adaptively switched, using the block (range cell) of each frame. The search is adaptively switched depending on whether the group of cells belongs to the still range or to the moving range. Using this technique, the degrees of spatial and time redundancy are reduced. This method is simulated to prove its validity. © 1999 Scripta Technica, Electron Comm Jpn Pt 1, 82(8): 32–41, 1999

Networks of multiple video cameras are being deployed in several scenarios like surveillance, traffic enforcement, human motion analysis and sports telecast. The unmanageable size of the raw video sequences necessitates the use of compression schemes to efficiently encode these videos. Traditional video compression schemes account for spatial and temporal redundancy in a video sequence. In this paper, we present a compression technique that also leverages the information redundancy between video sequences across different cameras with overlapping fields of view. An algorithm based on homography for efficient compression of multiple video sequences is presented that performs significantly better than current schemes. We also derive an efficient distributed version of the algorithm that can be implemented on a large scale network of cameras. This distributed algorithm minimizes both the communication costs and the compression costs simultaneously.

This paper proposed a removable visible watermark system for video sequences using dual watermarking technique, embedding both visible and invisible watermarks into the video sequence. The visible watermark is embedded into every frame of the video sequence in the Discrete Cosine Transform (DCT) domain, considering the Human Visual System (HVS) model. The invisible watermark is embedded into every intra-frame during the MPEG encoding, using the Quantization Index Modulation-Dither Modulation (QIM-DM) technique. In the proposed scheme, two user's keys are introduced to ensure that only authorized users can remove the visible watermark to obtain clear video sequence. The experimental results show the desirable performance of the proposed scheme, in which unauthorized users cannot remove the visible watermark. The process is carried out through a totally blind manner without any extra information.

In this paper we propose an effective video dehazing technique to reduce flicker artifacts by using adaptive temporal average. When applying a dehazing method for still image to a video sequence, flicker artifacts may occur because atmospheric values calculated independently for each frame change frame by frame. In existing approaches flicker artifacts are reduced by yielding highly correlated transmission values between adjacent frames. However, since flickering artifacts can also be caused by atmospheric light values which frequently change in a video sequence, it is necessary to effectively calculate atmospheric light values considering the correlation between adjacent frames. In this paper, we propose an approach to calculating atmospheric light values based on adaptive temporal averaging between adjacent frames. Experimental results have shown that the proposed method achieves better performance of video dehazing with less flicker artifact than existing methods.

A new method is presented for visualizing data as they are generated from real-time applications. These techniques allow viewers to perform simple data analysis tasks such as detection of data groups and boundaries, target detection, and estimation. The goal is to do this rapidly and accurately on a dynamic sequence of data frames. Our techniques take advantage of an ability of the human visual system called preattentive processing. Preattentive processing refers to an initial organization of the visual system based on operations believed to be rapid, automatic, and spatially parallel. Examples of visual features that can be detected in this way include hue, orientation, intensity, size, curvature, and line length. We believe that studies from preattentive processing should be used to assist in the design of visualization tools, especially those for which high speed target, boundary, and region detection are important. Previous work has shown that results from research in preattentive processing can be used to build visualization tools that allow rapid and accurate analysis of individual, static data frames. We extend these techniques to a dynamic real-time environment. This allows users to perform similar tasks on dynamic sequences of frames, exactly like those generated by real-time systems such as visual interactive simulation. We studied two known preattentive features, hue and curvature. The primary question investigated was whether rapid and accurate target and boundary detection in dynamic sequences is possible using these features. Behavioral experiments were run that simulated displays from our preattentive visualization tools. Analysis of the results of the experiments showed that rapid and accurate target and boundary detection is possible with both hue and curvature. A second question, whether interactions occur between the two features in a real-time environment, was answered positively.

In this paper, the problem of increasing information transfer authenticity is formulated. And to reach a decision, the control methods and algorithms based on the use of statistical and structural information redundancy are presented. It is assumed that the controllable information is submitted as the text element images and it contains redundancy, caused by statistical relations and non-uniformity probability distribution of the transmitted data. The use of statistical redundancy allows to develop the adaptive rules of the authenticity control which take into account non-stationarity properties of image data while transferring the information. The structural redundancy peculiar to the container of image in a data transfer package is used for developing new rules to control the information authenticity on the basis of pattern recognition mechanisms. The techniques offered in this work are used to estimate the authenticity in structure of data transfer packages. The results of comparative analysis for developed methods and algorithms show that their parameters of efficiency are increased by criterion of probability of undetected mistakes, labour input and cost of realization.

Visual information is taking a predominant place in our society. With the advent of digital technologies, visual information will find new applications in domains ranging from communication, commerce to entertainment. New functionalities will be required that permit an extended interaction with the visual information. In particular, this is the case for digital television and the related problem of video sequence compression. The extent of possible interactions depends on the manner in which the visual information is represented. Up to now, the canonical representation is used. Also referred to as the waveform representation, it is a technical artifact of the image capture procedure. This representation severely restricts the functionalities available to the end-user. The latter is unable to freely manipulate or customize the received visual information. In this dissertation, we propose to describe the visual information through a semantically meaningful representation. This representation directly derives from the scene content, which is decomposed in terms of its constituting objects. Consequently, the viewing process is totally disconnected from the image capture procedure. This permits full interactivity with the visual information, leading to enhanced functionalities for the end-user. The essence of this dissertation is to automatically define the objects forming the scene arid to automatically track them through the video sequence. Two main issues are identified: the initial segmentation of the objects and their tracking. The first issue deals with segmenting the scene into its constituent objects. This has to be performed on the basis of the information available in two consecutive frames. In order to solve the problem, a split-and-merge approach is used. First, the image is segmented into small, spatio-temporally homogeneous regions. This is achieved through a top-down approach where spatial, temporal and change information is combined. These regions are used as a starting point to define the objects forming the scene. A bottom-up approach is used which iteratively merges the regions. The propensity of the regions to form an object is assessed in terms of both spatial and temporal information. The second issue deals with segmenting and tracking the objects in the successive frames. The coherence between the successive segmentations is ensured by using past and current information. Also, the different objects composing the scene are identified throughout the video sequence. This identification relies on temporal, spatial and spatio-temporal features of the objects. The proposed representation of the visual information finds a natural application in video coding. A second generation video coding scheme is presented which combines compression efficiency with extended functionalities. In particular, scalable coding is achieved in terms of both scene content and object coding quality.

Summary form only given. In order to realize the efficient and economical transmission/storage of video sequences, and also the object-based functionality of MPEG-4, we propose a new video sequence compression scheme. The CPM/NCIM fractal coding scheme is applied on each object independently by a prior image segmentation map (alpha plane) which is exactly the same as defined in MPEG-4. We encode the first frames of the video sequence as a "set" using circular prediction mapping (CPM) and encode the remaining frames using noncontractive interframe mapping (NCIM). The CPM and NCIM accomplish the motion estimation/compensation which can exploit the high temporal correlations between the adjacent frames of the video sequence. The tested results with a nature video sequence provides promising performances at low bit rate coding, such as applications in video conferencing. We believe it will be a powerful and efficient technique for object-based video sequence coding.

Visual redundancy effects were investigated in two experiments using nonalphabetic material. Experiment 1 had adult subjects search for the presence or absence of a predetermined target symbol through single six-symbol linear arrays. Reaction times were significantly faster in both a distributional redundancy and a spatial redundancy condition than in a no-redundancy condition. Experiment 2 had good and poor sixth-grade readers serve in the no-redundancy and spatial redundancy conditions of Experment 1. Poor readers were equivalent to good readers in the no-redundancy condition but were significantly slower in the spatial redundancy condition. Thus, spatial redundancy was shown to be operative in single symbol search time in a paradigm that avoided confounding by higher order linguistic variables. The present experiments, in conjunction with prior findings that good and poor readers differ in letter search time by their ability to use spatial redundancy, suggest that processes involved in the component skill of letter identification cannot be considered trivial to the reading process.

It has long been recognized that synthetic aperture radar (SAR) images suffer in many applications from the speckle noise. Video SAR has a high frame rate of imaging and contains redundant information among frames. The temporal redundancy in video SAR images has been found useful for suppressing the speckle noise. However, the motion and the local differences between frames make it difficult to employ the temporal redundancy to suppress speckle noise. This article presents a video SAR image despeckling framework based on a new unsupervised training strategy referred to as DualNoise2Noise. This developed framework consists of a registration network and a denoising network. The registration network first compensates for the motion between two adjacent frames in video SAR in real time. After image registration, two adjacent frames with random speckle noise can be considered as the observations of the same region with local differences. The denoising network adopts the DualNoise2Noise training strategy to suppress speckle noise by using the temporal redundancy and to remove the negative impact caused by the local differences. The proposed approach has been used to process the real video SAR data, and the experimental results are convincing.

In this paper, we will propose a novel pixelwise adaptive prediction method for lossless video compression. The proposed method adequately takes use of the temporal and spatial redundancy in the video sequences. Firstly, the pixel to be encoded is predicted in temporal domain and spatial domain respectively, and then the final prediction values are chosen by an adaptive mode selection. In addition, we introduce a forgetting factor α in the temporal prediction and the adaptive mode selection to improve the prediction accuracy. Experimental results show that our proposed method gains the minimum zero-entropy compared to other prediction methods.

Kinesin and cytoplasmic dynein are microtubule-based, force-generating ATPases thought to be involved in the bidirectional transport of vesicular organelles along microtubules. In vitro, kinesin and dynein promote movement of 0.1 - 0.2μm plastic beads toward the plus and minus ends, respectively, of microtubules. In these systems, the proteins are thought to remain fixed, by nonspecific absorption, to the bead surface. The motion of the bead on the microtubule is detected by video enhanced, DIC microscopy, and reflects the movement of the motor protein along the microtubule.To quantitate bead motion, a method was devised for tracking the position of kinesin or dynein coated beads moving along microtubules. The method involves 1) cross-correlation of a template bead image (the kernal) with each of the frames in a video sequence, followed by 2) calculation of the centroid of the cross-correlation function for each of the frames in the sequence. The centroid provides a measure of bead position. Application of this method to the measurement of the relative position of two stationary beads adsorbed nonspecifically to the glass surface indicates that the standard deviation of the measured horizontal and vertical coordinates can be less than 0.5 nm. More recently, we have determined that centroid calculation of the unprocessed bead image (i.e. without cross-correlation) can provide a measure of bead position with standard deviations in the horizontal and vertical directions of less than 2 nm. The latter method is faster, but more sensitive to the contrast and magnification of the video image.