<title>Robust approach to video segmentation using compressed data</title>

An algorithm is proposed to detect abrupt and gradual scene changes in H.264 compressed videos. In H.264, the inter prediction in the encoding process generates the DC component of the prediction residual. In this paper, we use the DC component of inter prediction residuals as well as motion vectors to detect scene changes. Using the proposed algorithm, we can detect scene changes for H.264 compressed videos more effectively, as demonstrated by simulations.

Video is an important and challenging medium and requires sophisticated indexing schemes for efficient retrieval from visual databases. An important step in video indexing is scene change detection. Recently, several scene change detection algorithms in the pixel and compressed (MPEG-2) domains have been reported in the literature. These algorithms are computationally complex and are not very robust in detecting gradual scene changes. The authors propose an efficient technique for detecting scene changes in the MPEG-2 compressed domain. The proposed algorithm has the advantage of fast scene change detection. In addition, this algorithm has the potential to detect gradual scene changes.

A common framework for rapid scene analysis for detecting scene changes in compressed motion JPEG and MPEG videos is proposed. We develop algorithms to detect both abrupt and gradual scene changes. The algorithms operate directly on the DC sequence which can be easily extracted from motion JPEG and MPEG compressed video without decompression. The DC images capture most of the essential "global" information, but is of a small fraction of the original data size. Operating on these images offers significant computation savings. Experimental results show that the proposed algorithms are fast and effective in detecting abrupt scene changes and gradual transitions.

The automatic video parser, a necessary tool for the development and maintenance of a video library, must accurately detect video scene changes so that the resulting video clips can be indexed in some fashion and stored in a video database. With the current existing algorithms, abrupt scene changes are detected fairly well; however, gradual scene changes, including fade-ins, fade-outs, and dissolves, are often missed. In this paper, we propose a new gradual scene change detection algorithm. In particular, we focus on fade- ins, fade-outs, and dissolves. The proposed algorithm is based on the chromatic video edit model. The video edit model indicates that, for sequences without motion, the second partial derivative with respect to time is zero during fade- ins, fade-outs, and dissolves. However, it is also zero for static scenes. Thus, the proposed algorithm computes the first (to disregard static scenes) and second partial derivatives, and if the norm of the second derivative is 'small' relative to the norm of the first derivative, the algorithm declares a gradual scene change. The efficacy of our algorithm is demonstrated using a number of video clips and some performance comparisons are made with other existing approaches.

내용기반 검색을 위한 비디오 데이터 장면전환 검출에서 점진적인 전환을 검출하는 것은 갑작스런 전환을 검출하는 것에 비해 일반적으로 어려운 문제로 알려져 있다. 본 논문에서는 가변형 동적 임계값과 가장 최근에 검출된 컷 프레임과 현재 프레임간의 특징값 차이인 컷 프레임 차를 이용하여 갑작스런 전환과 점진적인 전환을 찾아내는 기법을 제안한다. 이를 위하여 본 논문에서는 점진적인 전환이 갖는 특성과 수학적 모델을 제시하고 컷 프레임 차를 이용하여 점진적인 전환을 검출할 수 있음을 보인다. 그리고 이를 바탕으로 갑작스런 전환과 점진적인 전환을 함께 검출할 수 있는 방법을 제시한다. 실세계 동영상 데이터를 대상으로 한 실험을 통해 제안하는 기법이 점진적인 전환 효과의 종류에 종속적이지 않으며 적은 연산 비용으로 쉽게 점진적인 전환 유무를 검출 할 수 있음을 보인다. Gradual scene change detection is known as more difficult problem then abrupt scene change detection on video data analysis for contents based retrieval. In this paper, we present a new method for scene change detection far both abrupt and gradual change using the variable dynamic threshold arid cut frame difference (CFD). For this, We present the characteristics arid mathematical models of gradual transitions anti then, how can be detected by the CFD. And also we present new scene change detection algorithm based on cut frame difference. By the experimental result using real world video data indicate that the proposed method detect various scene changes both abrupt and gradual change efficiently without time-consuming computation and any dependency on a kind of gradual change effects.

Scene boundary detection is the essential step for video analysis such as retrieval, indexing, browsing and so on. There have been quite a lot of scene boundary detection methods for MPEG video streams and they work well. However, the variable size of macroblocks and more type information of macroblocks in H.264/AVC bitstreams make it different from MPEG bitstreams. Consequently, most of the algorithms can not be applied to the H.264/AVC directly. In this paper, an algorithm based on the percentage of blocks coded with the same mode in each frame is proposed. After analyzing the statistics of type information of the sequence contain gradual scene changes, the algorithm sets appropriate thresholds to locate the gradual scene boundaries. Experimental results show that the proposed algorithm performs well and can detect 90.9% of the boundaries and make few false alarms.

Video segmentation is an important computer vision research field being applied in many digital video analysis domains, such as: video compression, video indexing and retrieval, video scene detection, video content analysis, video object tracking in dynamic scenes, and many others. Video has temporal properties. The temporal segmentation process leads to the partition of a given video into a set of meaningful and individually manageable temporal segments. An effective segmentation technique is able to detect not only abrupt changes but also gradual scene changes, such as fade and dissolve transitions. The effectiveness of four methods was analyzed for five different categories of movie: TV talk-show, documentary movie, animal video, action & adventure, and pop music video. The tests have shown that the specific nature of videos has an important influence on the effectiveness of temporal segmentation methods. Furthermore, the knowledge on these specific features of the style of video editing has been used to reduce the number of faulty detected shot cuts and faulty detected cross dissolve effects.Keywordsdigital video indexingtemporal segmentationscene changescutscross dissolve effectsmovie categories

Scene change detection is the process of identifying the differences between the multi-temporal image scenes at the semantic level, which has significant potential in the application of urban development and land management. In this paper, we propose a novel deep convolution canonical correlation analysis neural network (DCCANet) architecture, which could consider the spectral-spatial-temporal correlation feature for scene change detection in remote sensing images. For this purpose, we put together the convolutional neural networks (CNNs) and the deep canonical correlation analysis (DCCA) into the end-to-end network. The CNN could get the spectral-spatial feature information for scene representation, while the later could enhance the temporal correlation by nonlinear high- dimensional transformations between the multi-temporal image scenes for scene change detection. Experiments with high-resolution remote sensing image scene datasets demonstrated that our proposed approach can get a better performance in scene classification and change detection.

Scene change detection plays an important role in a number of video applications, including video indexing, semantic features extraction, and, in general, pre- and post-processing operations. This paper deals with the design and performance evaluation of a dynamic scene change detector optimized for H.264/AVC encoded video sequences. The detector is based on a dynamic threshold that adaptively tracks different features of the video sequence, to increase the whole scheme accuracy in correctly locating true scene changes. The solution has been tested on suitable video sequences resembling real-world videos thanks to a number of different motion features, and has provided good performance without requiring an increase in decoder complexity. This is a valuable issue, considering the possible application of the proposed algorithm in post-processing operations, such as error concealment for video decoding in typical error prone video transmission environments, such as wireless networks.

In this paper we extend the shot transition detection component of the ViBE video database system to include gradual scene changes. ViBE, a browsable/searchable paradigm for organizing video data containing a large number of sequences, is being developed at Purdue as a testbed to explore ideas and concepts in video databases. We also present result on the performance of our cut detection algorithm using a large test set. The performance of two other techniques are compared against our method.

Increasing the demands of digital video by developing the Internet market and the multimedia technologies, video indexing technique such as scene change detection is required to manage the data efficiently. In conventional methods, scene change is detected by comparing the value of current detected measure with the fixed threshold induced from preceding experiments. However, this can not guarantee the best performances on all various video sequences, due to their own specific characteristics. To solve this problem, a novel adaptive threshold decision method is proposed. First, histogram of scene change detection measure induced from preceding experiments is derived. Then this is modeled with log-normal distributed pdf and the model parameters are estimated. Consequently, experimental results obtained from this pdf model and estimated parameters demonstrate better performance of the proposed method comparing with conventional methods.

Rate control plays a key role in a video encoder, and it has been a challenging research topic. In this paper, an adaptive variable length group of picture (GOP) level rate control algorithm for H.264 based on scene change detection is proposed. In the proposed algorithm, the length of current GOP changes adaptively with the occurrence of scene change, so that the entire sequence contains a fixed number of I frame to avoid the actual bit rate increasing. Furthermore, it adjusts the QP of the scene change frame to ease the pressure of the buffer. Simulation results show that the proposed algorithm improves the video quality significantly on the basis of meeting bit rate constraints accurately, which shows better adaptability for sequences with scene change occurrence frequently. When compared with existed schemes, the average peak signal-to-noise ratio (PSNR) improvement is more than 1.4 dB.

Scene change detection is an essential step to automatic and content-based video indexing, retrieval, and browsing. In this paper, a robust scene change detection method is presented, which analyzes both audio and visual information sources and accounts for their inter-relations and coincidence to semantically identify video scenes. Audio analysis focuses on the segmentation of audio source into four types of semantic data such as silence, speech, music, and environmental sound. Speech data are further decomposed into different elements according to different speakers. Meanwhile, visual analysis partitions video source into shots. Results from single source segmentation are in some cases suboptimal. By combining visual and audio features, the scene extraction accuracy is enhanced, and more semantic segmentations are developed. Experimental results are proven to be appropriate for content-based video indexing and retrieval.

대용량 컨텐츠 수요와 공급의 증가에 따라 UHD 비디오의 분석, 색인, 편집 등을 위한 장면전환을 검출하는 방법이 요구되고 있다. 이전까지 많은 연구자들이 다양한 장면전환 검출 방법을 연구해왔지만 카메라의 다양한 움직임과 장면의 변화를 정확하게 검출하기는 어려웠다. 또한, 4K (3820x2160) 해상도 이상의 UHD 비디오들은 데이터 량을 더욱더 증가시키기 때문에 이전의 장면전환 검출 방법은 UHD 비디오 컨텐츠에 적용하기에는 너무 많은 시간이 걸리는 문제점이 발생한다. 따라서, 압축률이 높은 차세대 고효율 코덱 HEVC를 이용하여 장면전환을 검출하는 방법이 요구되고 있다. 본 논문에서는 차세대 고효율 코덱 HEVC의 부호화 부가정보를 이용한 4가지 장면전환 검출 방법을 제안하며, 대용량 비디오의 장면전환 검출을 위한 픽셀 기반의 새로운 장면전환 검출 시스템 구조를 제안한다. 연산량을 줄이기 위해 프레임 특징에 따른 방법을 각각 제시하며, 실험 결과를 통해 HEVC로 부호화 된 UHD 컨텐츠들의 장면전환 검출 가능성을 확인한다. With the increase of mass contents data, a method of a scene change detection is required for analysis, indexing and editing. Although many researchers are studying a variety of scene change detection method, it is too difficult to accurately detect various movements of the cameras and scene changes. Also, earlier scene change detection methods take too much time to apply to UHD video contents. That is because the UHD video contents with 4K (3820x2160) resolution or higher have greater amount of data. Therefore a method for detecting a scene change by using the next-generation codec, HEVC, is required. In this paper, we propose four scene change detection methods using the coding additional information of HEVC, and a new pixel-based scene change detection system. Furthermore, through the experimental results, we check the possibility of detecting the scene changes of UHD videos encoded in HEVC format.

An efficient video de-interlacing technique with scene change detection is proposed and its performances are examined. Scene changes happen quite often in film broadcasting and they tend to destabilize the quality of performance such as jagged effect, blurred effect, and artifacts effect, while de-interlacing technique is utilized. Therefore, the issue of scene change needs to be addressed with de-interlacing process. In the proposed method, de-interlacing begins with scene change detection, which is to ensure that the interfield information is used correctly. To improve the quality of de-interlacing, the factors of scene change are taken into account when de-interlacing techniques are applied. The simulation results show that the proposed algorithm exhibits better performances than other interpolation algorithms