An effective error concealment method based on abrupt scene change detection algorithm

This paper presents an efficient error concealment method for multiview video plus depth (MVD) delivery over error-prone channels based on the view synthesis information. Specifically, we propose an enhanced temporal error concealment (TEC) method by estimating the probable partition and missing MVs of a corrupted macroblock (MB) using the synthesized information. Furthermore, the view synthesis error concealment (VSEC) approach that directly uses the synthesized pixels to conceal the corrupted MBs is also adaptively utilized together with the TEC approach. Experimental results have shown the effectiveness of the proposed method compared with the existing methods in terms of both objective and subjective measures.

Error concealment at a decoder is an efficient method to reduce the degradation of visual quality caused by channel errors. In this paper, we propose a novel spatio-temporal error concealment algorithm based on the spatial-temporal fading (STF) scheme which has been recently introduced. Although STF achieves good performance for the error concealment, several drawbacks including blurring still remain in the concealed blocks. To alleviate these drawbacks, in the proposed method, hybrid approaches with adaptive weights are proposed. First, the boundary matching algorithm and the decoder motion vector estimation which are well-known temporal error concealment methods are adaptively combined to compensate for the defect of each other. Then, an edge preserved method is utilized to reduce the blurring effects caused by the bilinear interpolation for spatial error concealment. Finally, two concealed results obtained by the hybrid spatial and temporal error concealment are pixel-wisely blended with adaptive weights. Experimental results exhibit that the proposed method outperforms conventional methods including STF in terms of the PSNR performance as well as subjective visual quality, and the computational complexity of the proposed method is similar to that of STF.

내용기반 검색을 위한 비디오 데이터 장면전환 검출에서 점진적인 전환을 검출하는 것은 갑작스런 전환을 검출하는 것에 비해 일반적으로 어려운 문제로 알려져 있다. 본 논문에서는 가변형 동적 임계값과 가장 최근에 검출된 컷 프레임과 현재 프레임간의 특징값 차이인 컷 프레임 차를 이용하여 갑작스런 전환과 점진적인 전환을 찾아내는 기법을 제안한다. 이를 위하여 본 논문에서는 점진적인 전환이 갖는 특성과 수학적 모델을 제시하고 컷 프레임 차를 이용하여 점진적인 전환을 검출할 수 있음을 보인다. 그리고 이를 바탕으로 갑작스런 전환과 점진적인 전환을 함께 검출할 수 있는 방법을 제시한다. 실세계 동영상 데이터를 대상으로 한 실험을 통해 제안하는 기법이 점진적인 전환 효과의 종류에 종속적이지 않으며 적은 연산 비용으로 쉽게 점진적인 전환 유무를 검출 할 수 있음을 보인다. Gradual scene change detection is known as more difficult problem then abrupt scene change detection on video data analysis for contents based retrieval. In this paper, we present a new method for scene change detection far both abrupt and gradual change using the variable dynamic threshold arid cut frame difference (CFD). For this, We present the characteristics arid mathematical models of gradual transitions anti then, how can be detected by the CFD. And also we present new scene change detection algorithm based on cut frame difference. By the experimental result using real world video data indicate that the proposed method detect various scene changes both abrupt and gradual change efficiently without time-consuming computation and any dependency on a kind of gradual change effects.

The automatic video parser, a necessary tool for the development and maintenance of a video library, must accurately detect video scene changes so that the resulting video clips can be indexed in some fashion and stored in a video database. With the current existing algorithms, abrupt scene changes are detected fairly well; however, gradual scene changes, including fade-ins, fade-outs, and dissolves, are often missed. In this paper, we propose a new gradual scene change detection algorithm. In particular, we focus on fade- ins, fade-outs, and dissolves. The proposed algorithm is based on the chromatic video edit model. The video edit model indicates that, for sequences without motion, the second partial derivative with respect to time is zero during fade- ins, fade-outs, and dissolves. However, it is also zero for static scenes. Thus, the proposed algorithm computes the first (to disregard static scenes) and second partial derivatives, and if the norm of the second derivative is 'small' relative to the norm of the first derivative, the algorithm declares a gradual scene change. The efficacy of our algorithm is demonstrated using a number of video clips and some performance comparisons are made with other existing approaches.

A common framework for rapid scene analysis for detecting scene changes in compressed motion JPEG and MPEG videos is proposed. We develop algorithms to detect both abrupt and gradual scene changes. The algorithms operate directly on the DC sequence which can be easily extracted from motion JPEG and MPEG compressed video without decompression. The DC images capture most of the essential "global" information, but is of a small fraction of the original data size. Operating on these images offers significant computation savings. Experimental results show that the proposed algorithms are fast and effective in detecting abrupt scene changes and gradual transitions.

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian and Wolf (SW), and Wyner and Ziv (WZ). While conventional coding has a rigid complexity allocation as most of the complex tasks are performed at the encoder side, DVC enables a flexible complexity allocation between the encoder and the decoder. The most novel and interesting case is low complexity encoding and complex decoding, which is the opposite of conventional coding. While the latter is suitable for applications where the cost of the decoder is more critical than the encoder's one, DVC opens the door for a new range of applications where low complexity encoding is required and the decoder's complexity is not critical. This is interesting with the deployment of small and battery-powered multimedia mobile devices all around in our daily life. Further, since DVC operates as a reversed-complexity scheme when compared to conventional coding, DVC also enables the interesting scenario of low complexity encoding and decoding between two ends by transcoding between DVC and conventional coding. More specifically, low complexity encoding is possible by DVC at one end. Then, the resulting stream is decoded and conventionally re-encoded to enable low complexity decoding at the other end. Multiview video is attractive for a wide range of applications such as free viewpoint television, which is a system that allows viewing the scene from a viewpoint chosen by the viewer. Moreover, multiview can be beneficial for monitoring purposes in video surveillance. The increased use of multiview video systems is mainly due to the improvements in video technology and the reduced cost of cameras. While a multiview conventional codec will try to exploit the correlation among the different cameras at the encoder side, DVC allows for separate encoding of correlated video sources. Therefore, DVC requires no communication between the cameras in a multiview scenario. This is an advantage since communication is time consuming (i.e. more delay) and requires complex networking. Another appealing feature of DVC is the fact that it is based on a statistical framework. Moreover, DVC behaves as a natural joint source-channel coding solution. This results in an improved error resilience performance when compared to conventional coding. Further, DVC-based scalable codecs do not require a deterministic knowledge of the lower layers. In other words, the enhancement layers are completely independent from the base layer codec. This is called the codec-independent scalability feature, which offers a high flexibility in the way the various layers are distributed in a network. This thesis addresses the following topics: First, the theoretical foundations of DVC as well as the practical DVC scheme used in this research are presented. The potential applications for DVC are also outlined. DVC-based schemes use conventional coding to compress parts of the data, while the rest is compressed in a distributed fashion. Thus, different conventional codecs are studied in this research as they are compared in terms of compression efficiency for a rich set of sequences. This includes fine tuning the compression parameters such that the best performance is achieved for each codec. Further, DVC tools for improved Side Information (SI) and Error Concealment (EC) are introduced for monoview DVC using a partially decoded frame. The improved SI results in a significant gain in reconstruction quality for video with high activity and motion. This is done by re-estimating the erroneous motion vectors using the partially decoded frame to improve the SI quality. The latter is then used to enhance the reconstruction of the finally decoded frame. Further, the introduced spatio-temporal EC improves the quality of decoded video in the case of erroneously received packets, outperforming both spatial and temporal EC. Moreover, it also outperforms error-concealed conventional coding in different modes. Then, multiview DVC is studied in terms of SI generation, which differentiates it from the monoview case. More specifically, different multiview prediction techniques for SI generation are described and compared in terms of prediction quality, complexity and compression efficiency. Further, a technique for iterative multiview SI is introduced, where the final SI is used in an enhanced reconstruction process. The iterative SI outperforms the other SI generation techniques, especially for high motion video content. Finally, fusion techniques of temporal and inter-view side informations are introduced as well, which improves the performance of multiview DVC over monoview coding. DVC is also used to enable scalability for image and video coding. Since DVC is based on a statistical framework, the base and enhancement layers are completely independent, which is an interesting property called codec-independent scalability. Moreover, the introduced DVC scalable schemes show a good robustness to errors as the quality of decoded video steadily decreases with error rate increase. On the other hand, conventional coding exhibits a cliff effect as the performance drops dramatically after a certain error rate value. Further, the issue of privacy protection is addressed for DVC by transform domain scrambling, which is used to alter regions of interest in video such that the scene is still understood and privacy is preserved as well. The proposed scrambling techniques are shown to provide a good level of security without impairing the performance of the DVC scheme when compared to the one without scrambling. This is particularly attractive for video surveillance scenarios, which is one of the most promising applications for DVC. Finally, a practical DVC demonstrator built during this research is described, where the main requirements as well as the observed limitations are presented. Furthermore, it is defined in a setup being as close as possible to a complete real application scenario. This shows that it is actually possible to implement a complete end-to-end practical DVC system relying only on realistic assumptions. Even though DVC is inferior in terms of compression efficiency to the state of the art conventional coding for the moment, strengths of DVC reside in its good error resilience properties and the codec-independent scalability feature. Therefore, DVC offers promising possibilities for video compression with transmission over error-prone environments requirement as it significantly outperforms conventional coding in this case.

The effect of healthy aging on change detection and sensitivity to predictable structure in crowded acoustic scenes

Region-based error concealment of right-view frames for stereoscopic video transmission

Consecutive corrupted MBs or slice errors are commonly seen in modern video transmission systems. Temporal error concealment is an effective approach to reduce the impact of errors. Conventional temporal error concealment techniques recover slice errors on a MB basis. We propose a new novel temporal scheme for slice error concealment based on a size-adaptive region basis. Size-adaptive region boundary matching can significantly reduce the misalignments or side mismatches that are commonly seen in MB-based approaches. Instead of using MSE or MAE as the matching criteria, We propose a matching criterion based on structural similarity (SSIM). Our proposed matching criterion can achieve better structural alignments and better visual perceptual quality. We also propose a method to detect motion vector outliers at the last stage of the error concealment scheme, which can further improve the visual quality of the concealed videos. Our proposed scheme conceals slice errors with good accuracy as well as low computation complexity, and it can be applied for real-time applications.

In temporal error concealment (EC), the sum of absolute difference (SAD) is commonly used to identify the best replacement macroblock. Even though the use of SAD ensures spatial continuity and produces visually good results, it is insufficient to ensure edge alignment. Other distortion criteria based solely on structural alignment may also perform poorly in the absence of strong edges. In this paper, we propose a spatio-temporal EC search algorithm using an edge weighted SAD distortion criterion. This distortion criterion ensures both edge alignment and spatial continuity. We assume the loss of motion information and use zero motion vector as the starting search point. We show that the proposed algorithm outperforms the use of unweighted SAD in general. Most importantly, the perceptual quality of EC is improved due to edge alignment while ensuring spatial continuity.

A motion picture can be modeled as a composition of many scenes where each scene is comprised of multiple shots. Thus, a conventional movie is a sequential aggregation of a large number of disparate image sequences. Within each image sequence or shot, there is consistency in image content and dynamics. This consistency in dynamics can be used in identifying scene changes for video segment decomposition and for techniques to improve data compression. We have developed an algorithm to use these dynamics for scene change detection and the decomposition of video streams into constituent logical shots. The algorithm uses intraframe image complexity and identifies scene transitions by considering short-term temporal dynam- ics. The algorithm has shown to be effective for detecting both abrupt scene changes (cuts) as well as smooth scene changes (fades and dissolves). This algorithm is used in an application we have developed called the Motion Picture Parser (MPP). The MPP automates the process of tagging segments of motion-JPEG- compressed movies. Segments are also tagged for subsequent semantic content-based retrieval in units of shots and scenes. The MPP application consists of a graphical user interface with various editing controls.

As the newest video coding standard, H.264/AVC adopts the high-efficiently predictive coding and variable length entropy coding to achieve high compression efficiency. On the other side, transmission errors become the major problem faced by video broadcasting service providers. Error concealment (EC) here is adopted to handle slices with huge conjunctive corrupted areas inside. Considering error propagation from corrupted slice to succeeding ones is the key factor affecting the video quality, this paper proposes a novel temporal EC scheme including the bi-direction motion vector (MV) retrieval method and an adaptive EC ordering basing on it. Background and motional steady shift part of slice will be given top and second priority, respectively. Combined with our proposed improved boundary matching algorithm (IBMA) which provides more accurate distortion function, experiments results show that our proposal achieves better performance under different error rate channel, compared with EC algorithm adopted in H.264 reference software.

Video is an important and challenging medium and requires sophisticated indexing schemes for efficient retrieval from visual databases. An important step in video indexing is scene change detection. Recently, several scene change detection algorithms in the pixel and compressed (MPEG-2) domains have been reported in the literature. These algorithms are computationally complex and are not very robust in detecting gradual scene changes. The authors propose an efficient technique for detecting scene changes in the MPEG-2 compressed domain. The proposed algorithm has the advantage of fast scene change detection. In addition, this algorithm has the potential to detect gradual scene changes.

In this paper, a prescription-based error concealment (PEC) method is proposed. PEC relies on pre-analyses of the concealment error image (CEI) for I-frames and the optimal error concealment scheme for P-frames at encoder side. CEI is used to enhance the image quality at decoder side after error concealment (by spatial interpolation or zero motion) of the corrupted intra-coded MBs. A set of pre-selected error concealment methods is evaluated for each corrupted inter-coded MB to determine the optimal one for the decoder. Both the CEI and the scheme indices are considered as the prescriptions for decoder and transmitted along with the video bit stream based on a reversible data embedding technique. Experiments show that the proposed method is capable of achieving PSNR improvement of up to 1.48 dB, at a considerable bit-rate, when the packet loss rate is 20%

Smart CCTV (Closed-Circuit Television) technology has increasingly been developed in the last few years to judge the situation and notify the administrator or take immediate action for security and surveillance reasons. Currently the methods to detect object motion typically include the Frame Difference Method (FDM) which can detect moving objects and the Background Subtraction Method (BSM) which is able to detect motionless objects. Those results can be obtained only if there were some background images ready in advance. The Adaptive Background Subtraction Method (ABSM) also could not recognize an object very well if there are rapid scene changes or an object does not move relatively for a long time. To resolve such a problem, in this research, a filmed image has been divided into the regular sized blocks and then, only the necessary parts of the previous frame image are updated in real-time and a background image was generated so that it is insensible to surrounding environment changes such as object motion, noise or light variations. We proposed a novel moving object detection method which showed high performance with regard to the MSE (Mean Squared Error) and the accuracy of detecting the moving object contours compared to other existing methods. We also evaluated quantitatively the detectability for a moving object region by quickly creating a background image even if it is difficult to shoot a background image or we do not have the baseline image prepared in advance. The proposed method could be used for cases that any background image does not exist or hard to be generated. It is also good for observation of many places at the same time with only a single CCTV system since it is especially robust to abrupt scene changes.