基於使用者感興趣區域之H.264/AVC編碼器位元速率控制技術

Region of interest (ROI) determination is necessary when using functional near-infrared spectroscopy (fNIRS) data to detect brain activity. To extract ROIs from multiple fNIRS channels, we investigated the validity of applying decision mode analysis to the fNIRS dataset. This classifies a dataset into clusters with similar features. For each cluster, the dataset is decomposed into a mean vector and a linear combination of eigenvectors. Applying this to fNIRS signals, the mean vector can be used to represent change in hemoglobin (Hb), and the eigenvectors interpreted as a signal component constructing the arbitrary signal. Characterizing these vectors by correlating them with a theoretical model of brain function aids our understanding of where Hb changes occur and what type of Hb changes reflect brain activity in fNIRS data. Decision mode analysis of fNIRS data measured during viewing stereoscopic images identified ROIs around the right inferior frontal gyrus associated with attentional control, and frontal association area associated with decision on action and prediction. Our experimental results showed that information obtained from decision mode analysis can aid quantitative and qualitative ROI determination.

Rate control plays a key role in a video encoder, and it has been a challenging research topic. In this paper, an adaptive variable length group of picture (GOP) level rate control algorithm for H.264 based on scene change detection is proposed. In the proposed algorithm, the length of current GOP changes adaptively with the occurrence of scene change, so that the entire sequence contains a fixed number of I frame to avoid the actual bit rate increasing. Furthermore, it adjusts the QP of the scene change frame to ease the pressure of the buffer. Simulation results show that the proposed algorithm improves the video quality significantly on the basis of meeting bit rate constraints accurately, which shows better adaptability for sequences with scene change occurrence frequently. When compared with existed schemes, the average peak signal-to-noise ratio (PSNR) improvement is more than 1.4 dB.

Security is one of the important characteristics while transmitting the data particularly medical images and patient information over cyberspace without any loss of information. In the proposed method LWT, DCT and SHA-256 hashing techniques are used to provide security to the patient data and to detect tampers in ROI of the medical images. In this paper, medical image is segmented into three regions, region of interest (ROI), region of non interest (RONI) and BORDER regions. ROI is an area that has an important impact on diagnosis, whereas RONI has less or no significance in diagnosis. This paper proposed ROI based tamper detection that embeds hash value of ROI into RONI for authentication and also embed the patient information into RONI for providing security. The experimental results of the proposed method on various medical image databases proved to identify the tampers in medical images. Compared with the existing technique, the proposed method offered high Peak Signal to Noise Ratio (PSNR) approximately 59db for watermarked medical images.

Base on the observation that Cauchy distribution provides an accurate estimates of rate and distortion characteristics of video sequences. In this paper, we propose to use a new rate control scheme based on Cauchy R-D optimization model for bit allocation of H.264 for low delay constraint. One solution which has been proposed in this paper use the Lagrange Multiplier technique as the cost function to find rate and distortion model subject to target bit rate constraint resulting in optimum of quantization step sizes. Model parameters are estimated using linear regression analysis. We propose a simple rate control scheme that achieves for new Cauchy R-D model in basic unit layer under low delay constraint. The target bits for each frame are determined according to their buffer status, combined with number of bits use in the previous frame. The technique proposed has been implemented on H.264 video coder. Experimental showed that the proposed can be achieves an improvement of average PSNR with smooth video quality and less frame skipping compared to the JM8.6 rate control.

We present a region of interest (ROI) based rate control for H.263 compatible video conferencing. A face detection and tracking scheme with very low complexity is proposed for segmentation. By analyzing quadratic rate models in frame layer, video object plane (VOP) layer, and macroblock (MB) layer extracted from the test data, a quadratic rate model at the MB layer with a modified physical meaning is proposed to improve the model accuracy. The basic idea is to use a group of uncoded MBs in the current VOP instead of individual MBs to update model parameters. A joint VOP layer and MB layer rate control algorithm is proposed. The VOP layer rate control assigns target bit rate for each VOP based on the coding complexity and visual importance, and determines an average quantization parameter (QP) for each VOP. Some new features of MB layer rate control are designed to utilize both average statistics of a VOP and individual statistics of MB together. The performance is compared with conventional TMN8 and object-based VM8 rate control, better peak SNR (PSNR) for ROI, and more accurate rate control can be achieved for various video sequences. The proposed rate control algorithm can be extended for H.264 ROI-based scalable video coding.

Rate control plays an important role in video coding and transmission. In this paper, instead of using rate- distortion (R-D) model to perform rate control as many existing methods do, a rate-quality (R-Q) model of H.264/AVC has first been proposed to represent the coding characteristics of H.264, where PSNR (peak signal-to-noise ratio) is used to represent the reconstructed video quality. Then a H.264 rate control method at frame layer has been presented based on the proposed R-Q model. In this method, the quantization parameter (i.e. quantization step) of the current frame is determined by the proposed R-Q model according to the coding statistics of the previous coded frames. The experimental results show that, compared with the rate control algorithm of JM86 reference software, the output bit rate achieved by our proposed one is more close to the target bit rate while the reconstructed quality is comparable.

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

Effective use of telecommunication and information technology in telemedicine increases the medical services to the patients who are from far away locations. The doctors provide these services by evaluating the patient details & scans like CT Scan, MRI and Ultra Sound. The patient information is exchanged between doctors and patients on a public network which is not safe. In medical image, specific regions are very important to diagnosis known as Region of Interest (ROI) and the rest of the regions are not of much importance known as Region of Non-Interest (RONI). Providing security to the ROI is an important issue hence medical image watermarking is used to transmit the medical images by embedding the ROI into RONI. At the destination, if tampering is found in ROI then recovery of ROI is possible by extracting the ROI from RONI. In the proposed method, the medical image is divided into three parts: BORDER, ROI and RONI. Further the ROI and RONI are divided into blocks and each ROI block is mapped to RONI block by applying division hash function. Lossless block compression technique is applied to each ROI block and embedded the compressed ROI block into mapped RONI block. To provide authenticity to ROI, checksum is calculated for ROI and embed this checksum in BORDER. Again checksum is calculated for each ROI block and placed in mapped RONI blocks. Whether ROI is tampered or not, is to be identified by extracting the checksum from BORDER and if it is tampered then recover the ROI by mapped RONI. The efficiency of the proposed algorithm is estimated by the performance measures mainly Peak Signal to Noise Ratio (PSNR). The proposed method gives good results on average 55 dB of PSNR compared to the previous methods [21] by efficiently compressing the ROI and by checking the authenticity.

With the fast progress of the wireless networking technique and the extension of its application, lots of real-time services in addition to the transmission and downloading applications start to use the wireless network transmission. Those applications include video telephony, video conferencing, digital TV, streaming media application, etc, which should be considered with the trade-off between the real time and the image quality under the limited bandwidth. According to such demands, more and more novel techniques are proposed. There are five levels of transmission mechanism in IEEE 802.16-2005, including Unsolicited Grant Service (UGS), extended real time Polling-Variable Rate Service (ert-VR), real time Polling-Variable Rate Service (rt-VR), non real time Polling-Variable Rate Service (nrt-VR), and Best Effort (BE). With the presentation of these five service levels, the data will be transmitted according to corresponding levels, and will be more suitable in wireless transmission. This paper use H.264/AVC video transmission, which defines a set of three Profiles. Applications of the Baseline Profile include video telephony, video conferencing; applications of the Main Profile include television broadcasting and video storage; and the Extended Profile may be particularly useful for streaming media applications. H.264 proposes the idea of Network Abstraction Layer (NAL). The output of the encoding process of VCL data will be packeted into NAL units. The NAL will give the NAL units corresponding header information according the contents of the units. This main research is to make the data of VCL transmit in network better by the parameters of NAL headers. According to the designed levels, the real-time multimedia information will be assigned with the priority of the ranking before transmission, and then transmitted in RT-VR of IEEE 802.16-2005. In the circumstance with the finite bandwidth in wireless network, we try the best to assign the bandwidth to the transmission of data with higher priority for transmission ranking. We simulate the situations of the data in all ranks under different loss rate.

This paper presents a scheme aiming at transmitting real-time video to wireless channel with vigorously varying quality, which is in practice the norm rather than the exception. Region of interest (ROI) is an efficient approach to making the video more adaptive to the wireless channel because ROI is the region that human eyes tend to put more attention to than the remainder region (RM). In our proposed scheme, we adopted this feature. The real-time source video stream is divided into two regions, the ROI and the RM regions. The two regions were encoded using H.263 standard codec such that the video transmission is adaptive to the current channel state, which is characterized by the effective data rate that varies from tens of kilobits per second to hundreds of kilobits per second. Channel state parameters are fed back to the source coder to adjust the compression ratio as well as the intra/inter options of the encoders. Results including frame loss probability, compression characteristics, peak signal to noise ratio (PSNR) against channel states are given, indicating that the resulting adaptive video codec can respond judiciously to time-varying channel quality. Our scheme is evaluated together with a ROI-enabled moving picture coding standard JPEG2000. Using the features provided in JPEG2000, we have made the JPEG2000 codec adaptive to the vigorously varying wireless channel and then compared it with the H.263 scheme. Our technique is suitable for a broad area of applications including real-time news reporting and video conferencing.

The H.264 video coding standard has been widely applied to multimedia data transmission. To prevent the contents of video from being recognized by someone unauthorized, providing sufficient security to H.264 has become more and more important. In this thesis, an improved H.264/AVC video encryption method is proposed. As compared with the Shahid’s method, more data are encrypted. And this makes the outcome more desirable. In the proposed scheme, selective encryption (SE) is performed in the motion vector difference (MVD), delta quantization parameter (delta QP), and the context-based adaptive variable length coding (CAVLC) modules. Owing that the encrypted stream should be decoded correctly before decryption, a format-compliant method is presented. Furthermore, in order to keep the bitrate unchanged, keeping the same stream length is another important characteristic of our proposed scheme. Since there is no escalation in bitrate, the proposed encryption algorithm is suitable for real-time multimedia streaming over heterogeneous networks. The encrypting key is generated by using the advanced encryption standard (AES) algorithm, and every frame has its own unique key. Four benchmark video sequences with different combinations of motion, texture and objects were used for experimental evaluation of the proposed scheme. The structural similarity (SSIM) index, a reference value to image quality, of the proposed scheme is 0.202 which is lower than 0.233 of the Shahid’s scheme. The key security is shown reliable under the test of only 1 bit change. The extra computation needed by our scheme is less than 0.5%, so it is feasible for real-time transmission. The above performance on security and sensitivity demonstrates that our proposed scheme is a good candidate for practical applications.

This paper points out some problems inherent in H.264/AVC intra frame rate control (RC) and presents several new algorithms to solve them. Three main algorithms are proposed in this paper. Multiple quantization parameters (QPs) determination algorithm based on the statistics of the deviation measure is proposed. Thus, relatively accurate QP estimation can be achieved since it takes the features of the current frame content into consideration. Two slice-based RC schemes are also proposed to stabilize the output bit rate. The scene change is detected with the complexity of adjacent frames based on deviation measure, which not only generates stable bit rate output but also improves the video quality. The combination of all the above algorithms improve the average peak signal-to-noise ratio (PSNR) by 0.74 dB while keeping the mismatch of the bit rate output less than 7.18% compared to the H.264/AVC reference model, JM12.0, under various sequences.

Mobile devices have been transformed from voice communication tools to advanced tools for consuming multimedia contents. The extensive use of such mobile devices entails watching multimedia contents on small LCD panels. However, the most of video sequences are captured for normal viewing on standard TV or HDTV, but for cost reasons, merely resized and delivered without additional editing. This may give the small-display-viewers uncomfortable experiences in understanding what is happening in a scene. For instance, in a soccer video sequence taken by a long-shot camera technique, the tiny objects (e.g., soccer ball and players) may not be clearly viewed on the small LCD panel. Thus, an intelligent display technique needs to be developed to provide small-display-viewers with better experience. To this end, one of the key technologies is to determine region of interest (ROI) and display the magnified ROI on the screen, where ROI is a part of the scene that viewers pay more attention to than other regions. In this paper, which is an extension from our prior work, we focus on soccer video display for mobile devices, and a fully automatic and computationally efficient method is proposed. Instead of taking generic approaches utilizing visually salient features, we take domain-specific approach to exploit the attributes of the soccer video. The proposed scheme consists of two stages: shot classification, and ROI determination. The experimental results show that the proposed scheme offers useful tools for intelligent video display for multimedia mobile devices.

Distributed Video Coding (DVC) is a new paradigm for video compression based on the information theoretical results of Slepian and Wolf (SW), and Wyner and Ziv (WZ). While conventional coding has a rigid complexity allocation as most of the complex tasks are performed at the encoder side, DVC enables a flexible complexity allocation between the encoder and the decoder. The most novel and interesting case is low complexity encoding and complex decoding, which is the opposite of conventional coding. While the latter is suitable for applications where the cost of the decoder is more critical than the encoder's one, DVC opens the door for a new range of applications where low complexity encoding is required and the decoder's complexity is not critical. This is interesting with the deployment of small and battery-powered multimedia mobile devices all around in our daily life. Further, since DVC operates as a reversed-complexity scheme when compared to conventional coding, DVC also enables the interesting scenario of low complexity encoding and decoding between two ends by transcoding between DVC and conventional coding. More specifically, low complexity encoding is possible by DVC at one end. Then, the resulting stream is decoded and conventionally re-encoded to enable low complexity decoding at the other end. Multiview video is attractive for a wide range of applications such as free viewpoint television, which is a system that allows viewing the scene from a viewpoint chosen by the viewer. Moreover, multiview can be beneficial for monitoring purposes in video surveillance. The increased use of multiview video systems is mainly due to the improvements in video technology and the reduced cost of cameras. While a multiview conventional codec will try to exploit the correlation among the different cameras at the encoder side, DVC allows for separate encoding of correlated video sources. Therefore, DVC requires no communication between the cameras in a multiview scenario. This is an advantage since communication is time consuming (i.e. more delay) and requires complex networking. Another appealing feature of DVC is the fact that it is based on a statistical framework. Moreover, DVC behaves as a natural joint source-channel coding solution. This results in an improved error resilience performance when compared to conventional coding. Further, DVC-based scalable codecs do not require a deterministic knowledge of the lower layers. In other words, the enhancement layers are completely independent from the base layer codec. This is called the codec-independent scalability feature, which offers a high flexibility in the way the various layers are distributed in a network. This thesis addresses the following topics: First, the theoretical foundations of DVC as well as the practical DVC scheme used in this research are presented. The potential applications for DVC are also outlined. DVC-based schemes use conventional coding to compress parts of the data, while the rest is compressed in a distributed fashion. Thus, different conventional codecs are studied in this research as they are compared in terms of compression efficiency for a rich set of sequences. This includes fine tuning the compression parameters such that the best performance is achieved for each codec. Further, DVC tools for improved Side Information (SI) and Error Concealment (EC) are introduced for monoview DVC using a partially decoded frame. The improved SI results in a significant gain in reconstruction quality for video with high activity and motion. This is done by re-estimating the erroneous motion vectors using the partially decoded frame to improve the SI quality. The latter is then used to enhance the reconstruction of the finally decoded frame. Further, the introduced spatio-temporal EC improves the quality of decoded video in the case of erroneously received packets, outperforming both spatial and temporal EC. Moreover, it also outperforms error-concealed conventional coding in different modes. Then, multiview DVC is studied in terms of SI generation, which differentiates it from the monoview case. More specifically, different multiview prediction techniques for SI generation are described and compared in terms of prediction quality, complexity and compression efficiency. Further, a technique for iterative multiview SI is introduced, where the final SI is used in an enhanced reconstruction process. The iterative SI outperforms the other SI generation techniques, especially for high motion video content. Finally, fusion techniques of temporal and inter-view side informations are introduced as well, which improves the performance of multiview DVC over monoview coding. DVC is also used to enable scalability for image and video coding. Since DVC is based on a statistical framework, the base and enhancement layers are completely independent, which is an interesting property called codec-independent scalability. Moreover, the introduced DVC scalable schemes show a good robustness to errors as the quality of decoded video steadily decreases with error rate increase. On the other hand, conventional coding exhibits a cliff effect as the performance drops dramatically after a certain error rate value. Further, the issue of privacy protection is addressed for DVC by transform domain scrambling, which is used to alter regions of interest in video such that the scene is still understood and privacy is preserved as well. The proposed scrambling techniques are shown to provide a good level of security without impairing the performance of the DVC scheme when compared to the one without scrambling. This is particularly attractive for video surveillance scenarios, which is one of the most promising applications for DVC. Finally, a practical DVC demonstrator built during this research is described, where the main requirements as well as the observed limitations are presented. Furthermore, it is defined in a setup being as close as possible to a complete real application scenario. This shows that it is actually possible to implement a complete end-to-end practical DVC system relying only on realistic assumptions. Even though DVC is inferior in terms of compression efficiency to the state of the art conventional coding for the moment, strengths of DVC reside in its good error resilience properties and the codec-independent scalability feature. Therefore, DVC offers promising possibilities for video compression with transmission over error-prone environments requirement as it significantly outperforms conventional coding in this case.

Blurring facial images can significantly degrade the performance of face recognition and video surveillance applications. Therefore, an effective image restoration method is essential to address this issue. However, existing methods struggle with varying levels of blur severity, limiting their effectiveness. This study proposes a facial image restoration approach that integrates blur severity classification using a Convolutional Neural Network (CNN) with a U-Net-based deblurring model to overcome these challenges. This method ensures that each blurred image is processed using the most suiTable deblurring model, optimizing the restoration process. The dataset used in this study is Flickr-Faces-HQ (FFHQ), to which a Gaussian blur is applied and categorized into five levels: very low, low, medium, high, and very high. They employ the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index Measure (SSIM) as quantitative metrics to evaluate the model's performance. Experimental results show that the proposed method consistently outperforms fixed kernel size and multi-kernel size deblurring approaches across all blur severities. Specifically, our method achieves a PSNR of 40.001 dB for very low blur severity and an SSIM of 0.990. For low severity, it attains a PSNR of 31.104 dB and an SSIM of 0.946. For medium severity, the results are 27.995 dB (PSNR) and 0.874 (SSIM). At high severity, our model achieves a PSNR of 28.896 dB and an SSIM of 0.855. Finally, for very high severity, the PSNR drops to 26.566 dB, with an SSIM of 0.812. These results demonstrate the effectiveness of the proposed method in enhancing image clarity across different blur levels while preserving facial details. This research contributes to the development of more adaptive and efficient image restoration techniques, particularly for applications that require high-quality facial images, such as face recognition and video surveillance.