Lossy JPEG compression: easy to compress, hard to compare

In this paper, we evaluate lossless and lossy compression techniques to compress quantitative phase images of red blood cells (RBCs) obtained by an off-axis digital holographic microscopy (DHM). The RBC phase images are numerically reconstructed from their digital holograms and are stored in 16-bit unsigned integer format. In the case of lossless compression, predictive coding of JPEG lossless (JPEG-LS), JPEG2000, and JP3D are evaluated, and compression ratio (CR) and complexity (compression time) are compared against each other. It turns out that JP2k can outperform other methods by having the best CR. In the lossy case, JP2k and JP3D with different CRs are examined. Because some data is lost in a lossy way, the degradation level is measured by comparing different morphological and biochemical parameters of RBC before and after compression. Morphological parameters are volume, surface area, RBC diameter, sphericity index, and the biochemical cell parameter is mean corpuscular hemoglobin (MCH). Experimental results show that JP2k outperforms JP3D not only in terms of mean square error (MSE) when CR increases, but also in compression time in the lossy compression way. In addition, our compression results with both algorithms demonstrate that with high CR values the three-dimensional profile of RBC can be preserved and morphological and biochemical parameters can still be within the range of reported values.

<p>The projection into 2030 of the climate data volume increase brings an important challenge to the climate science community. This is particularly true for the CMIP7 that is projected to need about an Exabyte of storage capacity. Error-bounded lossy compression is explored as a potential solution to the above problem by different climate research teams. Several lossy compression schemes have been proposed leveraging different forms of decorrelation (transforms, prediction, HoSVD, DNN), quantization (linear, non-linear, vector), and encoding (dictionary-based, variable length, etc.) algorithms. Our experience with different applications shows that the compression methods often need to be customized and optimized to fit the specificities of the datasets to compress and the user requirements on the compression quality, ratio, and throughput. However, none of the existing lossy compression software for scientific data has been designed to be customizable. To address this issue, we developed SZ3, an innovative customizable, modular compression framework. SZ3 is a full C++ refactoring of SZ2 enabling the specialization, addition, or removal of each stage of the lossy compression pipeline to fit the specific characteristics of the datasets to compress and the use-case requirements. This extreme flexibility allows adapting SZ3 to many different use-cases, from ultra-high compression for visualization to ultra-high-speed compression between the CPU (or GPU) and the memory. Thanks to its unique set of features: customization, high compression ratio, high compression throughput, and excellent accuracy preservation, SZ3 won a 2021 R&D100 award. In this presentation, we present SZ3 and a new data prediction-based decorrelation method that significantly improves the compression ratios for climate datasets over the state-of-the-art lossy compressors, while preserving the same data accuracy. Experiments based on CESM datasets show that SZ3 can lead to up to 300% higher compression ratios than SZ2 with the same compression error bound and similar compression throughput.</p>

We evaluate the implications of JPEG2000 lossy compression of remote sensing images for spatial analytical purposes. The main issue is to identify which cases and conditions in geostatistical studies are suitable for using lossy compressed images. For these purposes, an extensive test using Landsat, compact airborne spectrographic imager (CASI), and moderate resolution imaging spectroradiometer (MODIS) image series has been analyzed, through applying and comparing two-dimensional and three-dimensional (spectral and time domains) compression methods with a wide range of compression ratios for several dates, different landscape regions, and spectral bands. Due to the massive test bed and consequently to the high time consuming executions, a parallel solution was specifically developed. Variogram analyses showed that all the compression ratios maintain the variogram shapes, but high compression ratios (>20∶1 ) degrade the spatial patterns of the remote sensing images. These alterations are lower for the three-dimensional compression method, which was a considerable improvement (25%) on the two-dimensional method for large three-dimensional series (CASI, MODIS). However, the two methods behave similarly in the Landsat case. Finally, the parallel solution in a distributed environment demonstrates that high performance computing offers a suitable scientific platform for highly demanding time execution applications, such as geostatistical analyses of remote sensing images.

Image compression is the application of data compression techniques to two-dimensional digital images I(x, y) , to reduce the redundancy of the image data for storage or transmission in an efficient form. Image compression can be classified into two categories: lossless or lossy. Lossless compression, which achieves smaller compression ratios than lossy compression, mainly takes advantage of the image contents containing a non uniform probability distribution for a variable-length representation of the image pixels. Such images include technical drawings, icons or comics, and high-value contents such as medical imagery or image scans made for archival purposes. However, lossy compression methods, especially when they achieve a very high compression ratio, can introduce compression artifacts. Nevertheless, lossy compressions are especially suitable for natural images, such as photos, in applications where a minor (sometimes imperceptible) loss of fidelity is acceptable when it is desirable to achieve a substantial reduction in bitrate. Most of the stateof-the-art image compression standards use a combination of lossy and lossless algorithms to achieve the best performance. The Joint Photographic Experts Group (JPEG) [1], a discrete cosine transform(DCT)-based technique, is the most widely used standardized lossy image compression mechanism; it was designed for compressing either full-color or gray-scale images of natural, real-world, scenes. It works well for photographs, naturalistic artwork and similar material if the compression ratio is about 20 : 1, which is much better than the 4 : 1 compression ratio provided by a lossless compression method such as the Graphics Interchange Format (GIF) [2]. However, it does not work so well for lettering, simple cartoons, or line drawings.

Today is the digital era and the human being is surrounded by digital gadgets. Now a days, photography is a part of human being's daily life and digital images are widely used in computer applications. As the megapixels of the digital cameras are increasing, more storage memory is required and at the same time more bandwidth is needed for transmission of digital images. This results in need of image compression. This paper explains the use of Legendre polynomials for lossy and lossless image compression. Different approximations for image transformation have been evaluated as 1-D Legendre polynomials, 1-D adaptive Legendre polynomials, 2-D Legendre polynomials and 2-D adaptive Legendre polynomials. Moreover, the performance of different image scanning methods have been tested. Results have been compared in terms of peak signal-to-noise ratio (PSNR), nominal compression rate (NCR), mean structural similarity (MSSIM) and compression ratio in order to minimize the difference between the approximated polynomial output and the original pixel gray level.

With widespread use of screening digital mammography, efficient storage of the vast amounts of data has become a challenge. While lossless image compression causes no risk to the interpretation of the data, it does not allow for high compression rates. Lossy compression and the associated higher compression ratios are therefore more desirable. The U.S. Food and Drug Administration (FDA) currently interprets the Mammography Quality Standards Act as prohibiting lossy compression of digital mammograms for primary image interpretation, image retention, or transfer to the patient or her designated recipient. Previous work has used reader studies to determine proper usage criteria for evaluating lossy image compression in mammography, and utilized different measures and metrics to characterize medical image quality. The drawback of such studies is that they rely on a threshold on compression ratio as the fundamental criterion for preserving the quality of images. However, compression ratio is not a useful indicator of image quality. On the other hand, many objective image quality metrics (IQMs) have shown excellent performance for natural image content for consumer electronic applications. In this paper, we create a new synthetic mammogram database with several unique features. We compare and characterize the impact of image compression on several clinically relevant image attributes such as perceived contrast and mass appearance for different kinds of masses. We plan to use this database to develop a new objective IQM for measuring the quality of compressed mammographic images to help determine the allowed maximum compression for different kinds of breasts and masses in terms of visual and diagnostic quality.

Convolution Neural Network based lossy compression of hyperspectral images

Lossy compressors are increasingly adopted in scientific research, tackling volumes of data from experiments or parallel numerical simulations and facilitating data storage and movement. In contrast with the notion of entropy in lossless compression, no theoretical or data-based quantification of lossy compressibility exists for scientific data. Users rely on trial and error to assess lossy compression performance. As a strong data-driven effort toward quantifying lossy compressibility of scientific datasets, we provide a statistical framework to predict compression ratios of lossy compressors. Our method is a two-step framework where (i) compressor-agnostic predictors are computed and (ii) statistical prediction models relying on these predictors are trained on observed compression ratios. Proposed predictors exploit spatial correlations and notions of entropy and lossyness via the quantized entropy. We study 8+ compressors on 6 scientific datasets and achieve a median percentage prediction error less than 12%, which is substantially smaller than that of other methods while achieving at least a 8.8× speedup for searching for a specific compression ratio and 7.8× speedup for determining the best compressor out of a collection.

Hyperspectral sensors are able to provide information that is useful for many different applications. However, the huge amounts of data collected by these sensors are not exempt of drawbacks, especially in remote sensing environments where the hyperspectral images are collected on-board satellites and need to be transferred to the earth’s surface. In this situation, an efficient compression of the hyperspectral images is mandatory in order to save bandwidth and storage space. Lossless compression algorithms have been traditionally preferred, in order to preserve all the information present in the hyperspectral cube for scientific purposes, despite their limited compression ratio. Nevertheless, the increment in the data-rate of the new-generation sensors is making more critical the necessity of obtaining higher compression ratios, making it necessary to use lossy compression techniques. A new transform-based lossy compression algorithm, namely Lossy Compression Algorithm for Hyperspectral Image Systems (HyperLCA), is proposed in this manuscript. This compressor has been developed for achieving high compression ratios with a good compression performance at a reasonable computational burden. An extensive amount of experiments have been performed in order to evaluate the goodness of the proposed HyperLCA compressor using different calibrated and uncalibrated hyperspectral images from the AVIRIS and Hyperion sensors. The results provided by the proposed HyperLCA compressor have been evaluated and compared against those produced by the most relevant state-of-the-art compression solutions. The theoretical and experimental evidence indicates that the proposed algorithm represents an excellent option for lossy compressing hyperspectral images, especially for applications where the available computational resources are limited, such as on-board scenarios.

This paper deals with analysis of component-wise and three-dimensional (3D) lossy compression of hyperspectral data. The latter approach is known to be more efficient due to possibility to employ inter-band correlation of images in neighbor sub-bands. However, there are certain peculiarities for this approach which are subject of our study. Methods of lossy compression that employ discrete cosine transform (DCT) in blocks are studied. It is shown that 3D compression is able to provide considerable benefits in compression ratio and/or mean square error (MSE) of introduced distortions but only under certain conditions. Experiments have been done for real-life hyperspectral data with high input signal-to-noise ratio. Their main results are presented.

A common problem in applying lossy compression to a hyperspectral image is predicting its effect on spectral target detection performance. Recent work has shown that light amounts of lossy compression can remove noise in hyperspectral imagery that would otherwise bias a covariance-based spectral target detection algorithm’s background-normalized response to target samples. However, the detection performance of such an algorithm is a function of both the specific target of interest as well as the background, among other factors, and therefore sometimes lossy compression operating at a particular compression ratio (CR) will not negatively affect the detection of one target, while it will negatively affect the detection of another. To account for the variability in this behavior, we have developed a target-centric metric that guides the selection of a lossy compression algorithm’s CR without knowledge of whether or not the targets of interest are present in an image. Further, we show that this metric is correlated with the adaptive coherence estimator’s (ACE’s) signal to clutter ratio when targets are present in an image.

Image compression schemes facilitate the transmission and storage of data in an efficient form by reducing redundant image data in the case of limited transmission bandwidth or storage space. However, due to the irreversible nature of lossy compression, these schemes also introduce various types of color artifacts into the reconstructed image at a high compression ratio. Consequently, a change in color information affects the gamut characteristic of the reconstructed image. Accordingly, this article investigates the relationship between the compression ratio and the gamut variation for a reconstructed image using JPEG and JPEG2000. To analyze the relationship between the compression ratio and the gamut variation, 18 color samples from the Macbeth ColorChecker are initially used as representative colors for all colors due to their uniform distribution in CIELAB color space. Based on the color information shift for representative color samples, 12 natural color images, classified into two groups depending on four color attributes, are used to investigate the relationship between the level of compression and the variation in the gamut area for reconstructed images. After determining the gamut areas for the decompressed images in relation to the compression ratio, gamut fidelity is obtained using the ratio of unique colors relative to the gamut area. Finally, the optimal least square method is applied to obtain fitted curves as an equation minimizing the error between the real data, the gamut area for the decompressed images, and the corresponding approximated values.

Being a fuel of a different origin, the standard design parameters of a diesel engine are not optimum for Karanj methyl ester (KME). This study aims at finding the effect of the engine design parameters, viz. compression ratio (CR) and fuel injection pressure (IP) jointly on the performance and emissions. Brake-specific fuel consumption (BSFC) and brake thermal efficiency (BTE) were considered for performance evaluation, and emission analysis was done for un-burnt hydrocarbon, carbon monoxide (CO), carbon dioxide (CO2), oxides of nitrogen (NO x ), exhaust gas temperature and smoke opacity with KME as the fuel. Comparison of performance and emission was done for different values of CR along with IP to find the optimum combination for operating engine with KME. It is found that the combined increase in CR and IP increases the BTE and reduces BSFC while having lower emissions. For small-sized direct injection constant speed engines used for agricultural applications (3.5 kW), the optimum combination was found to be CR of 18 with IP of 250 bar.

Introduction/purpose: This paper presents the results of the research on visually lossless image compression which is of particular interest because it achieves a high degree of compression, while the visual quality of the image is not impaired, i.e., end users are very satisfied with the image quality. The analysis was carried out using the publicly available large-scale picture-wise KonJND-1k database which contains the results of subjective tests on JPEG and BPG compressed images. Methods: Thanks to the availability of images from the KonJND-1k database, the dependence of objective assessments of image quality on parameters that control the degree of compression of source signals (quality factor for JPEG and quantization parameter for BPG) is analyzed. The results of the visually lossless subjective tests are used for a deep analysis of the boundary and typical values of the parameters that control these two types of compression, as well as for the analysis of the corresponding values of the objective quality scores. Furthermore, reliable features for predicting the boundary between visually lossless and visually lossy compression have been identified. For that purpose, the degree of agreement between the predictions and the ground truth values of the peak signal-to-noise ratio (PSNR) and image representation in bits per pixel (bpp) is used. The visually lossless compression ratio is used to compare JPEG and BPG techniques. Results: It is shown that the boundary between visually lossless and visually lossy image compression is found in a wide range of PSNR values (about 20 dB for JPEG and 15 dB for BPG). The corresponding JPEG image compression quality factor values at this threshold also range widely from 31 to 79, with concentration between 40 and 45. For the BPG encoder, the values of the quantization parameter are grouped around 30, and the boundary values are 25 and 34. Furthermore, it is shown that this boundary can be reliably determined based on simple features derived from the original uncompressed image. Gradient-based features known as spatial frequency and spatial information proved to be the best predictors. The degree of agreement between the predictions obtained from these features with the ground truth values of PSNR and bpp in both types of compression is greater than 85%. A comparative analysis has showed that, using BPG compression, it is possible, on the average, to achieve a twice larger compression ratio of visually lossless compression than for JPEG (80 versus 40). Conclusion: Although a high degree of agreement is achieved between the predictions and the ground truth values of PSNR and bpp of the boundary between visually lossless and visually lossy compression, there is a need for the development of new prediction approaches, especially with the BPG technique, which through the compression ratio proved to be superior to the JPEG technique. The existing databases used for the analysis of visually lossless compression contain color images from the visible part of the electromagnetic spectrum. Considering the increasing use of images from the infrared part of the spectrum, there is a need to conduct similar tests in this spectral range.

We use both lossy and lossless compression methods to reduce large amounts of digital data, making it easier to store and manage. Lossy and lossless compression is frequently used in astronomy, to condense the large amount of data collected by instruments namely, the James Webb Space Telescope and the Hubble Space Telescope. Similarly to diffraction images, astronomical images have large areas of empty space since stellar objects are very bright, yet show very small in images, which after a while, takes up a needless amount of limited digital storage space. Lossy compression achieves higher compression ratios but sacrifices some data quality in the process, while lossless compression preserves the data’s original form, but results in less storage savings. This work, conducted at Brookhaven National Laboratory’s National Synchrotron Light Source-II at the AMX and FMX macromolecular beam lines, aimed to evaluate the effectiveness of the lossy compression method known as Wavelet Compression. Wavelet compression is a method of data compression that uses wavelet transforms to reduce the size of data while preserving its essential features. Crystals are repetitive molecular assemblies that create diffraction patterns when exposed to X-rays, while detectors capture the amplitude of diffraction pattern spots (reflections). Similar to astronomical images, diffraction images contain large areas of empty space. This unnecessary data can quickly consume limited digital storage. By studying and optimizing compression methods, we not only improve data management but also advancements in various scientific disciplines.