Multi-histogram based scene change detection for frame rate up-conversion

대용량 컨텐츠 수요와 공급의 증가에 따라 UHD 비디오의 분석, 색인, 편집 등을 위한 장면전환을 검출하는 방법이 요구되고 있다. 이전까지 많은 연구자들이 다양한 장면전환 검출 방법을 연구해왔지만 카메라의 다양한 움직임과 장면의 변화를 정확하게 검출하기는 어려웠다. 또한, 4K (3820x2160) 해상도 이상의 UHD 비디오들은 데이터 량을 더욱더 증가시키기 때문에 이전의 장면전환 검출 방법은 UHD 비디오 컨텐츠에 적용하기에는 너무 많은 시간이 걸리는 문제점이 발생한다. 따라서, 압축률이 높은 차세대 고효율 코덱 HEVC를 이용하여 장면전환을 검출하는 방법이 요구되고 있다. 본 논문에서는 차세대 고효율 코덱 HEVC의 부호화 부가정보를 이용한 4가지 장면전환 검출 방법을 제안하며, 대용량 비디오의 장면전환 검출을 위한 픽셀 기반의 새로운 장면전환 검출 시스템 구조를 제안한다. 연산량을 줄이기 위해 프레임 특징에 따른 방법을 각각 제시하며, 실험 결과를 통해 HEVC로 부호화 된 UHD 컨텐츠들의 장면전환 검출 가능성을 확인한다. With the increase of mass contents data, a method of a scene change detection is required for analysis, indexing and editing. Although many researchers are studying a variety of scene change detection method, it is too difficult to accurately detect various movements of the cameras and scene changes. Also, earlier scene change detection methods take too much time to apply to UHD video contents. That is because the UHD video contents with 4K (3820x2160) resolution or higher have greater amount of data. Therefore a method for detecting a scene change by using the next-generation codec, HEVC, is required. In this paper, we propose four scene change detection methods using the coding additional information of HEVC, and a new pixel-based scene change detection system. Furthermore, through the experimental results, we check the possibility of detecting the scene changes of UHD videos encoded in HEVC format.

An efficient video de-interlacing technique with scene change detection is proposed and its performances are examined. Scene changes happen quite often in film broadcasting and they tend to destabilize the quality of performance such as jagged effect, blurred effect, and artifacts effect, while de-interlacing technique is utilized. Therefore, the issue of scene change needs to be addressed with de-interlacing process. In the proposed method, de-interlacing begins with scene change detection, which is to ensure that the interfield information is used correctly. To improve the quality of de-interlacing, the factors of scene change are taken into account when de-interlacing techniques are applied. The simulation results show that the proposed algorithm exhibits better performances than other interpolation algorithms

Video scene change detection has great importance of managing and analyzing large amount of videos. Traditionally this technique used for indexing, segmenting and categorizing different types of videos. Very few works addressed to classify obscene using scene change detection method. In this research we proposed a simple approach for sensing objectionable videos by observing scene changes into different video genres. Video scenes are grouped into set of key frames. After analyzing duration of each scene and counting the number of key frames of designated scene, it has been shown that obscene videos have infrequent scene changing nature. While in sports, dramas, music and action films have large number of scene changes. We used six types of video genres and the decision has been made by setting a threshold based on extracted key frames. Experimental result showed that the accuracy is 83.33% and false positive rate is 16.67%.

Scene change detection is the process of identifying the differences between the multi-temporal image scenes at the semantic level, which has significant potential in the application of urban development and land management. In this paper, we propose a novel deep convolution canonical correlation analysis neural network (DCCANet) architecture, which could consider the spectral-spatial-temporal correlation feature for scene change detection in remote sensing images. For this purpose, we put together the convolutional neural networks (CNNs) and the deep canonical correlation analysis (DCCA) into the end-to-end network. The CNN could get the spectral-spatial feature information for scene representation, while the later could enhance the temporal correlation by nonlinear high- dimensional transformations between the multi-temporal image scenes for scene change detection. Experiments with high-resolution remote sensing image scene datasets demonstrated that our proposed approach can get a better performance in scene classification and change detection.

내용기반 검색을 위한 비디오 데이터 장면전환 검출에서 점진적인 전환을 검출하는 것은 갑작스런 전환을 검출하는 것에 비해 일반적으로 어려운 문제로 알려져 있다. 본 논문에서는 가변형 동적 임계값과 가장 최근에 검출된 컷 프레임과 현재 프레임간의 특징값 차이인 컷 프레임 차를 이용하여 갑작스런 전환과 점진적인 전환을 찾아내는 기법을 제안한다. 이를 위하여 본 논문에서는 점진적인 전환이 갖는 특성과 수학적 모델을 제시하고 컷 프레임 차를 이용하여 점진적인 전환을 검출할 수 있음을 보인다. 그리고 이를 바탕으로 갑작스런 전환과 점진적인 전환을 함께 검출할 수 있는 방법을 제시한다. 실세계 동영상 데이터를 대상으로 한 실험을 통해 제안하는 기법이 점진적인 전환 효과의 종류에 종속적이지 않으며 적은 연산 비용으로 쉽게 점진적인 전환 유무를 검출 할 수 있음을 보인다. Gradual scene change detection is known as more difficult problem then abrupt scene change detection on video data analysis for contents based retrieval. In this paper, we present a new method for scene change detection far both abrupt and gradual change using the variable dynamic threshold arid cut frame difference (CFD). For this, We present the characteristics arid mathematical models of gradual transitions anti then, how can be detected by the CFD. And also we present new scene change detection algorithm based on cut frame difference. By the experimental result using real world video data indicate that the proposed method detect various scene changes both abrupt and gradual change efficiently without time-consuming computation and any dependency on a kind of gradual change effects.

Video is an important and challenging medium and requires sophisticated indexing schemes for efficient retrieval from visual databases. An important step in video indexing is scene change detection. Recently, several scene change detection algorithms in the pixel and compressed (MPEG-2) domains have been reported in the literature. These algorithms are computationally complex and are not very robust in detecting gradual scene changes. The authors propose an efficient technique for detecting scene changes in the MPEG-2 compressed domain. The proposed algorithm has the advantage of fast scene change detection. In addition, this algorithm has the potential to detect gradual scene changes.

In order to fulfill the requirement of limited channel bandwidth and of growing video demand like streaming media delivery on internet, and digital library, video compression is necessary. In video compression, temporal redundancy between adjacent frames is removed with block based motion estimation algorithms. Video represents a sequence of frames captured from camera. Scene is a series of consecutive frames captured from narrative point of view. In this paper we present an effective scene change detection method for an uncompressed video. We have divided frames in to blocks and applied a canny edge detector in consecutive frames. Count no of pixels (ones) in each block and compare it with consecutive frames. If scene change happens then number of pixels per block will change, based on that change we can detect scene change in consecutive frame. Here we have presented a hybrid approach, in which we have used scene change detection along with block based motion estimation algorithms (BME) to compress video.

An algorithm is proposed to detect abrupt and gradual scene changes in H.264 compressed videos. In H.264, the inter prediction in the encoding process generates the DC component of the prediction residual. In this paper, we use the DC component of inter prediction residuals as well as motion vectors to detect scene changes. Using the proposed algorithm, we can detect scene changes for H.264 compressed videos more effectively, as demonstrated by simulations.

This study proposes a framework for describing a scene change using natural language text based on indoor scene observations conducted before and after a scene change. The recognition of scene changes plays an essential role in a variety of real-world applications, such as scene anomaly detection. Most scene understanding research has focused on static scenes. Most existing scene change captioning methods detect scene changes from single-view RGB images, neglecting the underlying three-dimensional structures. Previous three-dimensional scene change captioning methods use simulated scenes consisting of geometry primitives, making it unsuitable for real-world applications. To solve these problems, we automatically generated large-scale indoor scene change caption datasets. We propose an end-to-end framework for describing scene changes from various input modalities, namely, RGB images, depth images, and point cloud data, which are available in most robot applications. We conducted experiments with various input modalities and models and evaluated model performance using datasets with various levels of complexity. Experimental results show that the models that combine RGB images and point cloud data as input achieve high performance in sentence generation and caption correctness and are robust for change type understanding for datasets with high complexity. The developed datasets and models contribute to the study of indoor scene change understanding.

The properties and characteristics of a new model of primate spatial vision are described and compared with human vision. The model employs a multiple level stack of graduated receptive field sizes whose sampling densities progressively decrease with level while maintaining a constant space-bandwidth product. The effect of this structure is to increase receptive field size with eccentricity, whilst retaining a constant number of samples per level, coupled with a set of octave-related spatial frequency filters at the fovea. Such an architecture, which exploits the Heirarchical Discrete Correlator of Burt (1981)17, correctly mimics the visual cortical mapping function of Schwartz (1983) 14 yet it has the valuable property that it can produce invariant responses to local changes in the size and position of features in the image. This architecture requires only local connections at each level, so producing the type of uniformity that is commonl observed as a feature of neural processing in the striate cortex (Rubel & Wiesel, 1974)1-1. An intrinsic characteristic of the model is the concept of attention area, representing the spatial extent of each level in the stack, and this concept helps to explain the high efficiency of human visual search in terms of hierarchical scanning. Our model has been simulated on an INTELLECT image processor using many different natural scene inputs. Analysis of the results has revealed possible mechanisms for human visual accommodation and neural gain control, which have enabled us to program the simulator to control reliably the focus and gain of its own TV camera input. Simple mechanisms have also been programmed that allow rapid detection of scene changes and consequent shift of attention, together with smooth pursuit of targets through natural scenes. While the simulation is slow, being a serial manifestation of a parallel original system, it has demonstrated the outstanding value of the data compression that is inherent in this form of architecture. The advantages of such a homomorphic system for intelligent machines are, we feel, obvious.

Rate control plays a key role in a video encoder, and it has been a challenging research topic. In this paper, an adaptive variable length group of picture (GOP) level rate control algorithm for H.264 based on scene change detection is proposed. In the proposed algorithm, the length of current GOP changes adaptively with the occurrence of scene change, so that the entire sequence contains a fixed number of I frame to avoid the actual bit rate increasing. Furthermore, it adjusts the QP of the scene change frame to ease the pressure of the buffer. Simulation results show that the proposed algorithm improves the video quality significantly on the basis of meeting bit rate constraints accurately, which shows better adaptability for sequences with scene change occurrence frequently. When compared with existed schemes, the average peak signal-to-noise ratio (PSNR) improvement is more than 1.4 dB.

An efficient method is proposed to dynamically adapt the Group of Pictures (GOP) size in H.264/AVC encoding, by on-the-fly identification of scene changes in the input video signal. Intra coding picture decision is taken at the end of the pre-analysis phase of a fast motion estimation method based on spatial-temporal motion correlation. With negligible additional computation and memory requirements, the proposed method allows improving the compression of the whole coding process by up to 15%, without affecting the overall image quality.

A common framework for rapid scene analysis for detecting scene changes in compressed motion JPEG and MPEG videos is proposed. We develop algorithms to detect both abrupt and gradual scene changes. The algorithms operate directly on the DC sequence which can be easily extracted from motion JPEG and MPEG compressed video without decompression. The DC images capture most of the essential "global" information, but is of a small fraction of the original data size. Operating on these images offers significant computation savings. Experimental results show that the proposed algorithms are fast and effective in detecting abrupt scene changes and gradual transitions.

Increasing the demands of digital video by developing the Internet market and the multimedia technologies, video indexing technique such as scene change detection is required to manage the data efficiently. In conventional methods, scene change is detected by comparing the value of current detected measure with the fixed threshold induced from preceding experiments. However, this can not guarantee the best performances on all various video sequences, due to their own specific characteristics. To solve this problem, a novel adaptive threshold decision method is proposed. First, histogram of scene change detection measure induced from preceding experiments is derived. Then this is modeled with log-normal distributed pdf and the model parameters are estimated. Consequently, experimental results obtained from this pdf model and estimated parameters demonstrate better performance of the proposed method comparing with conventional methods.

본 논문에서는 스마트폰의 플랫폼으로 하여 시공간적 중요도 기반으로 비디오를 요약하는 효율적인 기법을 제안한다. 제안하는 기법은 주어진 비디오에서 카메라 및 물체의 움직임에 강건한 색상 히스토그램의 차분으로 장면 전환을 검출하고 연속적인 프레임간의 유사성, 얼굴의 영역, 개별 프레임(frame)의 중요도를 통해 시공간적 중요도를 분석한다. 그리고 검출된 장면 전환을 이용하여 과분할된 계층적 트리를 생성하고 비디오 분석 과정에서 계산한 병합 및 유지 에너지를 이용하여 반복적으로 갱신한다. 또한 갱신된 계층적 트리에서 사용자가 요구하는 재생 길이와 최소 구간 길이를 충족하고 동시에 높은 중요도를 가진 노드들로부터 탐욕 알고리즘(greedy algorithm)을 통해 프레임을 추출한다. 실험 결과 상용 스마트폰에서 2분길이 분량의 입력 비디오를 10초 내외의 수행시간으로 요약할 수 있었으며, 그 결과는 상용 비디오 편집 소프트웨어인 Muvee보다 우수함을 보였다. In this paper, we propose a video summarization technique on a smartphone, based on spatiotemporal saliency. The proposed technique detects scene changes by computing the difference of the color histogram, which is robust to camera and object motion. Then the similarity between adjacent frames, face region, and frame saliency are computed to analyze the spatiotemporal saliency in a video clip. Over-segmented hierarchical tree is created using scene changes and is updated iteratively using mergence and maintenance energies computed during the analysis procedure. In the updated hierarchical tree, segmented frames are extracted by applying a greedy algorithm on the node with high saliency when it satisfies the reduction ratio and the minimum interval requested by the user. Experimental result shows that the proposed method summaries a 2 minute-length video in about 10 seconds on a commercial smartphone. The summarization quality is superior to the commercial video editing software, Muvee.