Chapter two - Digital Image and Video Compression and Processing

Usually, it is difficult to perform compression in video files as it significantly affects the perceptible quality. This paper has reviewed various works of literature toward finding unique approaches to video compression to find that the existing techniques offer a sophisticated algorithm for implementation with less focus on the visual quality of the decoded contents. Hence, this paper presents a model that uses graph theory to perform video compression compatible with High-Efficiency Video Coding (HEVC) standards of compression. By adoption of the Lagrangian theorem, the proposed algorithm finds the best solution for compression and offers better compression standards. The outcomes are compared with existing H.264 and existing research work to find that the proposed system excels better with respect to PSNR and bit rate and thereby offers better compatibility to compress High Definition (HD) videos.

Context-based adaptive variable length coding (CAVLC) for the H.264/advanced video coding (AVC) standard was originally designed for lossy video coding, and as such does not yield adequate performance for lossless video coding. In this paper, we propose an improved CAVLC for lossless intra-coding by considering the statistical differences in residual data between lossy and lossless coding. From experimental results, we confirm that the proposed method provides approximately 9% bit saving in terms of a compression ratio compared with the current H.264/AVC fidelity range extensions high profile.

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

compressed domain processing, transcoding, video editing, MPEG, splicing, reverse play, frame rate conversion, interlace to progressive, motion vector resampling Video compression algorithms are being used to compress digital video for a wide variety of applications, including video delivery over the Internet, advanced television broadcasting, video streaming, video conferencing, and video storage and editing. The impressive performance of modern compression algorithms, combined with the growing availability of video encoders and decoders and low-cost computers, storage devices, and networking equipment, makes it evident that between video capture and video playback, video will be handled in compressed video form. The resulting end-toend compressed digital video systems motivate the need to develop efficient algorithms for handling compressed digital video.

In this paper, we introduce a new implementation of motion-compensated temporal filter (MCTF) based video coder which utilizes rate-distortion optimal wavelet packet transform (RDWP) for some low bit rate video coding application scenarios. Although enhanced MC-EZBC video coder provides good scalability and high coding efficiency at high bit rates, reconstructed video quality will drop rapidly with increased motion at low bit rates. The proposed video coder applies optimal wavelet packet transform to each temporal frame and rate-distortion optimization to each GOP. Experimental results show this codec provides better performance than MC-EZBC with PSNR improved 0.3–1.2dB at low bit rates.

The two-dimensional discrete cosine transform (DCT) is probably the most popular transform in image and video coding systems as it achieves very high compression ratio for fast implementation. Various still digital image and video compression algorithms using this block-based transform coding method cause the artifacts, called the blocking, in the reconstructed images of a video sequence. This artifact is more prominent at low-bit rates. A new eight-neighbour symmetric post-filtering technique is introduced in this paper to reduce the widely known blocking effect which is commonly found in the reconstructed images of block-based transform coding schemes. In this technique, the weights of the eight neighbouring pixels of the pixel of interest are determined by the modified Lagrange's interpolation algorithm. The proposed technique reduces the blocking effect while maintaining the sharpness of the reconstructed image. The results obtained by the proposed technique are compared in this paper with the results obtained by several well known eight neighbour low-pass filters such as Reeve and Lim's method, the Laplacian eight neighbour accounting filter (LENAF), the Laplacian four neighbour filter with + form, the Laplacian four neighbour filter with x form and a two neighbour filtering method. These comparisons will be conducted with the following measures: peak signal-to-noise ratio (PSNR), block-edge impairment metric and subjective quality evaluation.

CHAPTER 14 - Video Coding Technology

Visual information, in the form of images and video, comes from the interaction of light with objects. Illumination is a fundamental element of visual information. Detecting and interpreting illumination effects is part of our everyday life visual experience. Shading for instance allows us to perceive the three-dimensional nature of objects. Shadows are particularly salient cues for inferring depth information. However, we do not make any conscious or unconscious effort to avoid them as if they were an obstacle when we walk around. Moreover, when humans are asked to describe a picture, they generally omit the presence of illumination effects, such as shadows, shading, and highlights, to give a list of objects and their relative position in the scene. Processing visual information in a way that is close to what the human visual system does, thus being aware of illumination effects, represents a challenging task for computer vision systems. Illumination phenomena interfere in fact with fundamental tasks in image analysis and interpretation applications, such as object extraction and description. On the other hand, illumination conditions are an important element to be considered when creating new and richer visual content that combines objects from different sources, both natural and synthetic. When taken into account, illumination effects can play an important role in achieving realism. Among illumination effects, shadows are often integral part of natural scenes and one of the elements contributing to naturalness of synthetic scenes. In this thesis, the problem of extracting shadows from digital images is discussed. A new analysis method for the segmentation of cast shadows in still and moving images without the need of human supervision is proposed. The problem of separating moving cast shadows from moving objects in image sequences is particularly relevant for an always wider range of applications, ranging from video analysis to video coding, and from video manipulation to interactive environments. Therefore, particular attention has been dedicated to the segmentation of shadows in video. The validity of the proposed approach is however also demonstrated through its application to the detection of cast shadows in still color images. Shadows are a difficult phenomenon to model. Their appearance changes with changes in the appearance of the surface they are cast upon. It is therefore important to exploit multiple constraints derived from the analysis of the spectral, geometric and temporal properties of shadows to develop effective techniques for their extraction. The proposed method combines an analysis of color information and of photometric invariant features to a spatio-temporal verification process. With regards to the use of color information for shadow analysis, a complete picture of the existing solutions is provided, which points out the fundamental assumptions, the adopted color models and the link with research problems such as computational color constancy and color invariance. The proposed spatial verification does not make any assumption about scene geometry nor about object shape. The temporal analysis is based on a novel shadow tracking technique. On the basis of the tracking results, a temporal reliability estimation of shadows is proposed which allows to discard shadows which do not present time coherence. The proposed approach is general and can be applied to a wide class of applications and input data. The proposed cast shadow segmentation method has been evaluated on a number of different video data representing indoor and outdoor real-world environments. The obtained results have confirmed the validity of the approach, in particular its ability to deal with different types of content and its robustness to different physically important independent variables, and have demonstrated the improvement with respect to the state of the art. Examples of application of the proposed shadow segmentation tool to the enhancement of video object segmentation, tracking and description operations, and to video composition, have demonstrated the advantages of a shadow-aware video processing.

During acquisition, transmission, storage and retrieval, quite often digital images become noised and mask the important details of an image and also present themselves with an ugly look. Such images if not processed properly, become useless for subsequent image processing operations like segmentation, classification etc. Hence the quality improvement (through denoising) of such noisy images becomes an important operation in the field of image processing. This paper proposes a two stage detection-estimation based filtering system to suppress random valued impulse noise from digital grey and color images based on neighborhood statistics of pixels of the working window under consideration. An adaptive working window and the adaptive tolerance computed from the pixels belonging to the current filtering window facilitates an appropriate detection of noisy pixels in the first phase, followed by the simple and computationally efficient restoration strategy in the second stage. The Simulation results prove that the proposed scheme exhibits much superior performance in comparison with other state of art image filtering methods in suppressing the random valued impulse noise from the digital grey and color images.

Video communication requires sufficient resources to obtain the desired image quality. Availability of bandwidth and screen resolution from different devices on the client end, as well as compression techniques used for transmission also significantly influence to the video quality. Even though the H.264/AVC video compression standard and its scalable extension is known as Scalable Video Coding (SVC) was introduced and successfully addressed the challenge, some weaknesses persist especially for high-resolution video transmissions. The size of the bitstream generated by the SVC standard for ultra-high resolution video compression can only be met if transmitted on a network with a large bandwidth. The Scalable High-Efficiency Video Coding (SHVC) were introduced to overcome problems in the SVC standards. The new video compression standard is proven to reduce bandwidth requirements by up to 46.41 % compared to the SVC standards for the same video quality. This study introduces the performance analysis of SHVC standard video transmission as compared to SVC standard. Both compression standards are transmitted and emulated through network emulator called Mininet-Wifi to get the real transmission picture. The client-server video transmission scheme with different bandwidth and various client position are set in the experiment scenario. It is found that transmission time, packet loss, and video reconstruction quality on scalable video transmission with SHVC standard is better than scalable video transmissions with SVC standard.

The growing popularity of high resolution video and the continuously increasing demands for high quality video on mobile devices are producing stronger needs for more efficient video encoder. Concerning these desires, HEVC, a newest video coding standard, has been developed by a joint team formed by ISO/IEO MPEG and ITU/T VCEG. Its design goal is to achieve a 50% compression gain over its predecessor H.264 with an equal or even higher perceptual video quality. Motion Estimation (ME) being as one of the most critical module in video coding contributes almost 50%-70% of computational complexity in the video encoder. This high consumption of the computational resources puts a limit on the performance of encoders, especially for full HD or ultra HD videos, in terms of coding speed, bit-rate and video quality. Thus the major part of this work concentrates on the computational complexity reduction and improvement of timing performance of motion estimation algorithms for HEVC standard. First, a new strategy to calculate the SAD (Sum of Absolute Difference) for motion estimation is designed based on the statistics on property of pixel data of video sequences. This statistics demonstrates the size relationship between the sum of two sets of pixels has a determined connection with the distribution of the size relationship between individual pixels from the two sets. Taking the advantage of this observation, only a small proportion of pixels is necessary to be involved in the SAD calculation. Simulations show that the amount of computations required in the full search algorithm is reduced by about 58% on average and up to 70% in the best case. Secondly, from the scope of parallelization an enhanced TZ search for HEVC is proposed using novel schemes of multiple MVPs (motion vector predictor) and shared MVP. Specifically, resorting to multiple MVPs the initial search process is performed in parallel at multiple search centers, and the ME processing engine for PUs within one CU are parallelized based on the MVP sharing scheme on CU (coding unit) level. Moreover, the SAD module for ME engine is also parallelly implemented for PU size of 32×32. Experiments indicate it achieves an appreciable improvement on the throughput and coding efficiency of the HEVC video encoder. In addition, the other part of this thesis is contributed to the VLSI architecture design for finding the first W maximum/minimum values targeting towards high speed and low hardware cost. The architecture based on the novel bit-wise AND scheme has only half of the area of the best reference solution and its critical path delay is comparable with other implementations. While the FPCG (full parallel comparison grid) architecture, which utilizes the optimized comparator-based structure, achieves 3.6 times faster on average on the speed and even 5.2 times faster at best comparing with the reference architectures. Finally the architecture using the partial sorting strategy reaches a good balance on the timing performance and area, which has a slightly lower or comparable speed with FPCG architecture and a acceptable hardware cost

Multi-layer security of color image based on chaotic system combined with RP2DFRFT and Arnold Transform

Many image display applications and printing devices allow only limited number of colours. They have limited computational power and storage to produce high quality outputs on high bit-depth colour image. A dithering technique is called for here in order to improve the perceptual visual quality of the limited bit-depth images. A dithered image is represented by a natural colour in the low bit depth image colour for displaying and printing. This technique obtains low cost colour image in displaying the colour and printing image pixels. This study proposes the dithering technique based on Tchebichef Moment Transform (TMT) to produce high quality image at low-bit colour. Earlier, a 2×2 Discrete Wavelet Transform (DWT) has been proposed for better image quality on dithering. The 2×2 TMT has been chosen here since it performs better than the 2×2 DWT. TMT provides a compact support on 2×2 blocks. The result shows that 2×2 TMT gives perceptually better quality on colour image dithering in significantly efficient fashion.

In spite of the recent advances in telecommunication standards, communication networks still have limited bandwidths and storage capacity. Therefore, video compression has drawn increasing importance since high-resolution video contents have become more and more used in various fields. These requirements raise the need for high-performance video-compression technologies able to reduce the amount of data to be transmitted or stored by compressing the input video signal into a bitstream file. Improving the coding efficiency was always one of the crucial issues of various compression standards that aim to get the most compact representation of the reconstructed video, with a high subjective quality. The high-efficiency video coding (HEVC) comes to respond to these requirements. However, the increased consumption of high-quality multimedia content has pushed the international communication companies to put much effort to better enhance video-coding techniques. In this perspective, an upcoming video-coding standard to be known as versatile video coding (VVC) has emerged aiming to improve the coding efficiency of the current HEVC codec. Improvements on rate distortion (RD) performance that came with both HEVC and VVC have brought an increased complexity in the majority of the coding modules, which makes it difficult to implement on hardware systems with real-time encoding. This chapter focuses on the transform coding stage as one of the most computationally demanding modules. For HEVC, efficient approximation algorithms in addition to reconfigurable and scalable architecture have been developed in order to decrease the computational complexity of the transform module. The main objectives are to meet low power and real-time processing constraints while maintaining a compression gain and a satisfying video quality. Field programmable gate array (FPGA) implementation results and comparisons with existing works confirm the efficiency of the proposed approximations since they contribute in reducing time and power consumption, optimizing the hardware resources and bringing peak signal-to noise ratio (PSNR) improvement as well. Similarly, for the adaptive multiple transform (AMT) introduced in the transform module of the VVC, approximations were done for discrete cosine transform (DCT)-II and discrete sine transform (DST)-VII transforms since they are statistically the most used ones among the five predefined types. Bitrate (BR) reduction with a slightly degradation of video quality and a less use of hardware resources are the main contributions of the proposed approximations.

High-speed color digital imaging devices are desired for many applications, including civilian and military missions. To capture color images a color sensor has to divide the incidence light into three spectral bands: blue, green and red. Due to the limitation of the spectral bandwidth, the speed of a color digital sensor is significant slower than that of a black and white sensor covering the entire visible bandwidth or the entire bandwidth from visible to near infrared. This paper introduces the conception of a high-speed color imaging device which can significantly increase the imaging speed without losing image resolution and other image qualities. This device uses the concept of black and white imaging to increase the imaging speed while keeping the image resolution unchanged. It also uses the concept of color imaging to capture color information, but at a significantly lower resolution to allow more incidence light being received by each sensor detector (pixel) to increase the imaging speed. The final color image at the desired resolution is recovered by integrating the spatial information from the black and white image and the color information from the low-resolution color image. An automatic image fusion technique is used to integrate the spatial information and the color information. The quality of the image fusion technique plays a crucial role for quality of the recovered color image. The concept of this high-speed color imaging device has been tested with available low-resolution color images and high-resolution black and white images from the latest commercial satellites, Ikonos and QuickBird. The test results demonstrated that the concept of the high-speed digital color imaging device is promising for producing real high-speed color digital devices for practical applications.