Comparative analysis of compression algorithms for four-dimensional light fields

PurposeCone‐beam computerized tomography (CBCT) is routinely performed for verification of patient position in radiotherapy. It produced a large amount of data which require a method to compress them for efficient storage. In this study three video compression algorithms were introduced and their performance was evaluated based on real patient data.Materials and methodsAt first CBCT images in multiple sets of a patient were transferred from reconstruction workstation or exported from treatment planning system. Then CBCT images were sorted according to imaging time (time‐prioritized sequence) or imaging location (location‐prioritized sequence). Next, this sequence was processed by a video compression algorithm and resulted in a movie. Three representative video compression algorithms (Motion JPEG 2000, Motion JPEG AVI, and MPEG‐4) were employed and their compression performance was evaluated based on the CBCT data of 30 patients.ResultsAmong three video compression algorithms, Motion JPEG 2000 has the least compression ratio since it is a lossless compression algorithm. Motion JPEG AVI and MPEG‐4 have higher compression ratios than Motion JPEG 2000 but come with certain image losses. For MPEG‐4, location‐prioritized sequences show higher compression ratio than time‐prioritized sequences. Based on the results achieved on the clinical target verification application, the registration accuracy of CBCT after decompression was comparable to that of the original CBCT.ConclusionsVideo compression algorithms could provide a higher compression ratio comparing to static image compression algorithm. Although the loss of CBCT image due to compression its impact on registration accuracy of patient positioning is almost negligible. Video compression method is an effective way to substantially reduce the size of CBCT images for storage.

In this article we present a novel dictionary learning framework designed for compression and sampling of light fields and light field videos. Unlike previous methods, where a single dictionary with one-dimensional atoms is learned, we propose to train a Multidimensional Dictionary Ensemble (MDE). It is shown that learning an ensemble in the native dimensionality of the data promotes sparsity, hence increasing the compression ratio and sampling efficiency. To make maximum use of correlations within the light field data sets, we also introduce a novel nonlocal pre-clustering approach that constructs an Aggregate MDE (AMDE). The pre-clustering not only improves the image quality but also reduces the training time by an order of magnitude in most cases. The decoding algorithm supports efficient local reconstruction of the compressed data, which enables efficient real-time playback of high-resolution light field videos. Moreover, we discuss the application of AMDE for compressed sensing. A theoretical analysis is presented that indicates the required conditions for exact recovery of point-sampled light fields that are sparse under AMDE. The analysis provides guidelines for designing efficient compressive light field cameras. We use various synthetic and natural light field and light field video data sets to demonstrate the utility of our approach in comparison with the state-of-the-art learning-based dictionaries, as well as established analytical dictionaries.

Light field (LF) imaging has gained significant attention due to its recent success in microscopy, 3-dimensional (3D) displaying and rendering, augmented and virtual reality usage. Postprocessing of LF enables us to extract more information from a scene compared to traditional cameras. However, the use of LF is still a research novelty because of the current limitations in capturing high-resolution LF in all of its four dimensions. While researchers are actively improving methods of capturing high-resolution LF's, using simulation, it is possible to explore a high-quality captured LF's properties. The immediate concerns following the LF capture are its storage and processing time. A rich LF occupies a large chunk of memory ---order of multiple gigabytes per LF---. Also, most feature extraction techniques associated with LF postprocessing involve multi-dimensional integration that requires access to the whole LF and is usually time-consuming. Recent advancements in computer processing units made it possible to simulate realistic images using physical-based rendering software. In this work, at first, a transformation function is proposed for building a camera array (CA) to capture the same portion of LF from a scene that a standard plenoptic camera (SPC) can acquire. Using this transformation, LF simulation with similar properties as a plenoptic camera will become trivial in any rendering software. Artificial intelligence (AI) and machine learning (ML) algorithms ---when deployed on the new generation of GPUs--- are faster than ever. It is possible to generate and train large networks with millions of trainable parameters to learn very complex features. Here, residual convolutional neural network (RCNN) structures are employed to build complex networks for compression and feature extraction from an LF. By combining state-of-the-art image compression and RCNN, I have created a compression pipeline. The proposed pipeline's bit per pixel (bpp) ratio is 0.0047 on average. I show that with a 1% compression time cost and 18x speedup for decompression, our methods reconstructed LFs have better structural similarity index metric (SSIM) and comparable peak signal-to-noise ratio (PSNR) compared to the state-of-the-art video compression techniques used to compress LFs. In the end, using RCNN, I created a network called RefNet, for extracting a group of 16 refocused images from a raw LF. The training parameters of the 16 LFs are set to (\alpha=0.125, 0.250, 0.375, ..., 2.0) for training. I show that RefNet is 134x faster than the state-of-the-art refocusing technique. The RefNet is also superior in color prediction compared to the state-of-the-art

To prolong the life cycle of wireless sensor network, the basic theory of wavelet transform and its application in image compression are described, and several classic image compression algorithms based on wavelet transform are studied in depth. A compression algorithm combining the improved wavelet transform and Set Partitioning in Hierarchical Trees (SPIHT) algorithm of hierarchical wavelet tree set segmentation is proposed to effectively balance the energy consumption of each node in the sensor network and prolong the life of the whole wireless sensor network and an improved distributed image compression and transmission algorithm is proposed based on the distributed multi-node cooperative processing algorithm based on wavelet transformation, and detailed analytical test and energy consumption simulation experiment are carried out to verify the feasibility of the algorithm. The results show that the platform effectively implements and verifies the proposed algorithm, which can effectively realize the compression and transmission of distributed images, equalize the energy consumption of each sensor node in the network, and has strong practicability.

Many image and video compression algorithms work by splitting the image into blocks and producing variable-length code bits for each block data. If variable-length code data are transmitted consecutively over error-prone channel without any error protection technique, the receiving decoder cannot decode the stream properly. So the standard image and video compression algorithms insert some redundant information into the stream to provide some protection against channel errors. One of such redundancy is resynchronization marker, which enables the decoder to restart the decoding process from a known state in the event of transmission errors, but its frequent use should be restricted not to consume bandwidth too much. The Error Resilient Entropy Code (EREC) is well known method which can regain synchronization without any redundant information. It can work with the overall prefix codes, which many image compression methods use. This paper proposes an improvement to FEREC (Fast Error-Resilient Entropy Coding). It first calculates initial searching position according to bit lengths of consecutive blocks. Second, initial offset is decided using statistical distribution of long and short blocks, and initial offset is adjusted to insure all possible offset value can be examined. The proposed algorithm can speed up the construction of EREC slots, and can preserve compressed image quality in the event of transmission errors. The simulation result shows that the quality of transmitted image is enhanced about 0.3--3.5 dB compared with the existing FEREC when random channel error happens.

In machine learning problems, there are two main algorithms: supervised learning and unsupervised learning. Supervised learning algorithms can be used to classify data for tagged data; non-supervised learning algorithms can be used to cluster data for unlabeled data. This paper discusses the basic principles of clustering algorithm and selection of key parameters of clustering algorithm. The application of clustering algorithm in image compression is also analyzed. This paper also emphasizes the problems that should be paid attention to when using clustering. Finally, a practical case of image compression with K-means is given.

Most image and video compression algorithms that have been proposed to improve picture quality relative to compression efficiency have either been designed based on objective criteria such as signal-to-noise-ratio (SNR) or have been evaluated, post-design, against competing methods using an objective sample measure. However, existing quantitative design criteria and numerical measurements of image and video quality both fail to adequately capture those attributes deemed important by the human visual system, except, perhaps, at very low error rates. We present a framework for assessing the quality of and determining the efficiency of foveated and compressed images and video streams. Image foveation is a process of nonuniform sampling that accords with the acquisition of visual information at the human retina. Foveated image/video compression algorithms seek to exploit this reduction of sensed information by nonuniformly reducing the resolution of the visual data. We develop unique algorithms for assessing the quality of foveated image/video data using a model of human visual response. We demonstrate these concepts on foveated, compressed video streams using modified (foveated) versions of H.263 that are standard-compliant. We rind that quality vs. compression is enhanced considerably by the foveation approach.

The subject of study. A new algorithm for real-time data compression in a video surveillance system from an unmanned aerial vehicle is proposed. The algorithm makes it possible to obtain different degrees of video data compression using different codecs for specific video recording conditions (UAV altitude and flight speed, nature of the observed terrain, etc.), and also provides the ability to transmit video data to the flight control point via radio channel. A detailed analysis of the quality of the proposed algorithm for shooting in various video formats that meet international standards (HD and Full HD) was carried out, both in terms of visual perception and objective criteria. The purpose of the study was to conduct a comprehensive research on the problem of improving the quality of the video surveillance system from the UAV and to create a video compression algorithm that allows broadcasting video data to the control center in real time with high quality. Objectives. To develop and programmatically implement a video compression algorithm based on modern codecs for video surveillance sys­tems on board a UAV. To use modern video cameras and a microcomputer platform as hardware. To analyze the quality indicators of the algorithm in different conditions. Methods used. Evaluation of visual perception of video compression results, statistical analysis and processing of the parameters of the analyzed video sequence as a random spatio-temporal process. Results. A new video data compression algorithm for the UAV video surveillance system based on the Raspberry Pi 5 microcomputer using modern video compression codecs H.264 and H.265 was synthesized. The algorithm provides the required compression ratio and data transmission rate over the radio channel between the UAV and the flight control center. Detailed estimates of the quality of the algorithm are obtained. Conclusions. Scientific novelty of the results: A new universal video data compression algorithm for video surveillance systems on board UAVs with the ability to transmit video over a radio channel in real time has been proposed and implemented. The quality of the proposed algorithm is verified on real video surveillance results. The proposed algorithm is implemented in the Python software environment.

Gabor 코사인과 사인 변환은 영상주파수 성분을 국부적으로 표현하므로 영상과 비디오 압축 알고리즘에 사용될 수 있다. 압축과 복원에 사용되는 순방향과 역방향 행렬 변환식의 계산 복잡도는 O(<TEX>$N^3$</TEX>)이다. 이 논문에서는 기저함수들의 길이를 절단하여, 희소기저행렬을 생성하고, 영상압축과 복원에 적용하여 실시간 처리에 용이하게 변환 계산량을 감소시키고자 한다. 기저함수 길이가 감소함에 따라서, 기저함수 에너지에 미치는 절단의 영향을 조사하고 다른 여러 측정량의 변화를 살펴본다. 실험 결과로부터 약 1% 이하의 성능저하로 11배의 곱하기/더하기 수를 감소시킬 수 있음을 보았다. The Gabor cosine and sine transform can be applied to image and video compression algorithm by representing image frequency components locally The computational complexity of forward and inverse matrix transforms used in the compression and decompression requires O(<TEX>$N^3$</TEX>)operations. In this paper, the length of basis functions is truncated to produce a sparse basis matrix, and the computational burden of transforms reduces to deal with image compression and reconstruction in a real-time processing. As the length of basis functions is decreased, the truncation effects to the energy of basis functions are examined and the change in various Qualify measures is evaluated. Experiment results show that 11 times fewer multiplication/addition operations are achieved with less than 1% performance change.

As video technology continues to seamlessly weave itself into the fabric of daily life, there is a growing need for enhanced storage and efficient video transmission. This surge in demand has led to heightened expectations and standards for video compression technology. Machine learning as an up-and-coming technology can play its advantages in the field of video compression. This article reviews the current state of research on combining video compression techniques with machine learning. The article provides an overview of various research avenues for enhancement, spanning from conventional video compression algorithms to the fusion of traditional compression frameworks with machine learning methodologies, and even the development of novel end-to-end compression algorithms. In additional, the article explores the possible various application scenarios of machine learning-based video compression algorithms based on the characteristics of such non-standard and arithmetic demanding algorithms. At the end, the article speculates on the future of video compression algorithms based on the content of the various studies reviewed in the article.

The wavelet transform is more and more widely used in image and video compression. One of the best known algorithms in image compression is the SPIHT algorithm which involves the wavelet transform. As today the parallelization of the wavelet transform is sufficiently investigated this work deals with the parallelization of the compression algorithm itself as a next step.

Focused on the image data storage, compression and transmission of mars rover, system design and key hardware/software architecture design are carried out in this paper based on the analysis of function requirements and key design features. Image compression algorithm and transmission protocol are proposed, and the image storage and compression module in Mars rover is designed and implemented. The multi-camera partition parallel image file system based on FPGA（Field Programmable Gate Array）is realized. The adaptive first 1 run length coding algorithm for image compression based on DSP is realized on this module. Finally the function of multi-camera image data storage and management, flexible control of image compression ratio, quality progressive transmission, coding of region of interest, anti-error code diffusion and image thumbnail downward transmission are realized in the engineering development, meeting the reliable, efficient and flexible application requirements of China’s first Mars exploration mission.

Compression of images process is a step in the image processing. It is concerned with the transmission and storage of digitally created images. Fractal coding is a potential image and video compression approach with good reconstruction fidelity and relatively large compression ratios, because of its simplicity and great performance; fractal image compression (FIC) is a particularly popular approach in image compression applications. However, it has a significant disadvantage in the form of a long encoding time. This is because encoding any small bit necessitates a massive similarity search in the original data As a result; the FIC search time is reduced while the quality of the reconstructed images is maintained acceptable level in many introduced paper and other still a study topic in progress. Fractal images are images that are self-similar in that each individual part is the same as the total. This paper will discusses many attempts for more author that working on image and video compression using fractal compression technique based on various approach and with each discuss focuses on the main parameter of compression such compression ratio (CR), peak signal to noise ratio (PSNR) and encoding time, as well as the details of data set the used for testing also writing with each technique to creating fractal video and image compression. Index Terms— FIC, Video compression, parallel processing, Iteration Function System (IFS), Image processing.

We propose a novel method of 5-D dynamic light field compression using multi-focus images. A light field enables us to observe its scene from various viewpoints. However, it generally consists of 4-D enormous data, that are not suitable for storing or transmitting without effective compression. 4-D light fields are very redundant because they essentially include just 3-D scene information. Actually, a method of reconstructing a light field directly from 3-D information composed of multi-focus images without any scene estimation is successfully derived, though robust 3-D scene estimation such as depth recovery from light fields is not so easy. Previously, based on the method, we proposed novel light field compression via multi-focus images as effective representation of 3-D scenes. In this paper, we extend this method to compression of 5-D light fields composed of multiview videos including the time domain. It achieves significant improvement in compression efficiency by utilizing multiple redundancy of 5-D light fields. We show experimental results using synthetic videos. Quality of reconstructed light fields is evaluated by PSNR and SSIM for analyzing characteristics of its performance. They reveal that our method is much superior to light field compression using HEVC at practical lower bit-rates.

This paper presents implementation aspects of video enhancement and compression algorithms on hardware platforms. The implementation is motivated by the fact that sufficient studies have not been carried out earlier on hardware realization of video algorithms since major focus in the research community has been on analyzing algorithms from a software perspective and many a times ignoring the hardware board level aspects which are very critical from the defence point of view. This paper is a novel attempt to bridge this gap and present the practical considerations of video processing algorithms when run on hardware boards. We utilize both a general purpose board namely Raspberry Pi 3 as well as a dedicated video processing board namely TI's DaVinci DM6437 to show the performance of existing video algorithms so that when prototype development is carried out the designer could tradeoff performance and cost aspects depending on the application scenario. Primary focus in this work is on the utility of video processing algorithms for surveillance applications like UAV's and missiles.