Compressed hyperspectral imagery for forestry

Magnetic resonance imaging (MRI) and computer tomography (CT) are very important techniques that are used in disease diagnosis in medicine. In an average sized hospital, many tera-bytes of digital imaging data (MRI and CT) are generated every year, almost all of which has to be kept and archived. The compression of medical images is currently performed by using different algorithms. The most common compression technique is vector quantization. Interframe coding, the discrete Hartley transform, mixed transform, entropy-coded DPCM and JPEG algorithm are also used. Hierarchical finite-state vector quantization (HFSVQ), which is the improved version of vector quantization, has been introduced as a compression technique with high compression ratios for video images. Although the HFSVQ algorithm is the most efficient compression technique according to its compression ratios, we show that it has also a good performance for brain tomography and magnetic resonance images.

With the rapid development of the mobile communication technology, the number of long-term evolution (LTE) users and the volume of data is continually increasing, which greatly increases the amount of data transferred by fronthaul, resulting in a huge investment in optical fiber resource. In order to control the consumption of optical fiber resource, lower the cost of operator and avoid congestion in the premise of increasing data volume, it is necessary to compress the data for fronthaul. According to the characteristics of the LTE baseband signal, this paper proposes a data compression scheme on the basis of discrete cosine transform (DCT) and vector quantization. Firstly, the time domain signal is transformed in DCT. And then, the k-means clustering algorithm is used for vector quantization on the transformed signal. Finally, the Huffman coding is applied to further increase the compression ratio (CR) under the promise of an acceptable error. The simulation results show that the scheme proposed has a good performance in both compression ratio and error vector Magnitude (EVM) for the LTE baseband signal base on orthogonal frequency division multiplexing (OFDM).

A test pattern compression scheme for test data volume and application time reduction is proposed. While compression reduces test data volume, the increased number of internal scan chains due to an on-chip, fixed-rate decompressor reduces test application time proportionately. Through on-chip decompression, both the number of virtual scan chains visible to the ATE and the functionality of the ATE are retained intact. Complete fault coverage is guaranteed by constructing the decompression hardware deterministically through analysis of the test pattern set.

Canada contains 10% of the world's forests, covering an area of 418 million ha. Remote sensing can provide rapid coverage of these forests. Forest information needs are moving from broad forest typing to mapping of major forest species. Hyperspectral sensing can be used to create products for forest inventory, forest health, biomass, and aboveground carbon. The challenge is to develop a process to extract from hyperspectral data the relevant forest information in a timely and reproducible manner with a high degree of confidence in the quality of the resultant products. To answer this challenge, experiments have been conducted on two test sites: the Greater Victoria Watershed District (GVWD) on Vancouver Island, BC, and the Hoquiam test site in Washington State. Three types of data were collected to perform the analysis. First, Hyperion and AVIRIS hyperspectral images were collected for both test sites. Secondly, ground truth reflectance data were acquired from locations within the GVWD test sites simultaneously with these image collections. Finally, foliar samples were taken and chemically analyzed from 30 Douglas fir plots in the GVWD test site and five plots in the Hoquiam test site to provide a sample set of known values to compare against the spectral imagery. This paper reports on the results for chlorophyll and nitrogen mapping for these sites. Difficulties associated with chemistry signature extension are discussed. For nitrogen, an r2 of 0.915 was obtained between ground measured nitrogen and nitrogen predicted from AVIRIS data.

Analysing large-scale data from simulations of turbulent flows is memory intensive, requiring significant resources. This major challenge highlights the need for data compression techniques. In this study, we apply a physics-informed Deep Learning technique based on vector quantisation to generate a discrete, low-dimensional representation of data from simulations of three-dimensional turbulent flows. The deep learning framework is composed of convolutional layers and incorporates physical constraints on the flow, such as preserving incompressibility and global statistical characteristics of the velocity gradients. The accuracy of the model is assessed using statistical, comparison-based similarity and physics-based metrics. The training data set is produced from Direct Numerical Simulation of an incompressible, statistically stationary, isotropic turbulent flow. The performance of this lossy data compression scheme is evaluated not only with unseen data from the stationary, isotropic turbulent flow, but also with data from decaying isotropic turbulence, a Taylor–Green vortex flow, and a turbulent channel flow. Defining the compression ratio (CR) as the ratio of original data size to the compressed one, the results show that our model based on vector quantisation can offer CR with a mean square error (MSE) of , and predictions that faithfully reproduce the statistics of the flow, except at the very smallest scales where there is some loss. Compared to the recent study of Glaws. et al. [Deep learning for in situ data compression of large turbulent flow simulations. Phys Rev Fluids. 2020;5(11):114602], which was based on a conventional autoencoder (where compression is performed in a continuous space), our model improves the CR by more than 30%, and reduces the MSE by an order of magnitude. Our compression model is an attractive solution for situations where fast, high quality and low-overhead encoding and decoding of large data are required.

Hyperspectral remote sensing can provide forest information products for applications in forest inventory, forest chemistry, and the Kyoto Protocol. One of the forest information products is the high accuracy forest species map produced by the classification of hyperspectral data. As part of the Evaluation and Validation of EO-1 for Sustainable Development (EVEOSD) Project, Hyperion data were acquired in 2001 and 2002. Corresponding AVIRIS data were also acquired. All hyperspectral data acquired were calibrated to reflectance and orthorectified. Experiments were conducted to compare the accuracies of the data sets for mapping forest species. Operational accuracies for forest species recognition were achieved with both AVIRIS and Hyperion. Bioindicators were also developed for mapping chlorophyll, nitrogen, and moisture content. These bioindicators were stratified by forest type. In the Greater Victoria Watershed District and the Hoquiam, Washington State test sites, foliar samples were collected, analyzed, and databases were built, which include foliar chemistry and plot parameters. These sites were used to develop the indicators and to validate their success. The forest information products produced under the EVEOSD project demonstrate some of the benefits to be achieved from an operational hyperspectral satellite.

<p>The projection into 2030 of the climate data volume increase brings an important challenge to the climate science community. This is particularly true for the CMIP7 that is projected to need about an Exabyte of storage capacity. Error-bounded lossy compression is explored as a potential solution to the above problem by different climate research teams. Several lossy compression schemes have been proposed leveraging different forms of decorrelation (transforms, prediction, HoSVD, DNN), quantization (linear, non-linear, vector), and encoding (dictionary-based, variable length, etc.) algorithms. Our experience with different applications shows that the compression methods often need to be customized and optimized to fit the specificities of the datasets to compress and the user requirements on the compression quality, ratio, and throughput. However, none of the existing lossy compression software for scientific data has been designed to be customizable. To address this issue, we developed SZ3, an innovative customizable, modular compression framework. SZ3 is a full C++ refactoring of SZ2 enabling the specialization, addition, or removal of each stage of the lossy compression pipeline to fit the specific characteristics of the datasets to compress and the use-case requirements. This extreme flexibility allows adapting SZ3 to many different use-cases, from ultra-high compression for visualization to ultra-high-speed compression between the CPU (or GPU) and the memory. Thanks to its unique set of features: customization, high compression ratio, high compression throughput, and excellent accuracy preservation, SZ3 won a 2021 R&D100 award. In this presentation, we present SZ3 and a new data prediction-based decorrelation method that significantly improves the compression ratios for climate datasets over the state-of-the-art lossy compressors, while preserving the same data accuracy. Experiments based on CESM datasets show that SZ3 can lead to up to 300% higher compression ratios than SZ2 with the same compression error bound and similar compression throughput.</p>

3D single-photon LiDAR imaging has an important role in many applications. However, full deployment of this modality will require the analysis of low signal to noise ratio target returns and very high volume of data. This is particularly evident when imaging through obscurants or in high ambient background light conditions. This paper proposes a multiscale approach for 3D surface detection from the photon timing histogram to permit a significant reduction in data volume. The resulting surfaces are background-free and can be used to infer depth and reflectivity information about the target. We demonstrate this by proposing a hierarchical Bayesian model for 3D reconstruction and spectral classification of multispectral single-photon LiDAR data. The reconstruction method promotes spatial correlation between point-cloud estimates and uses a coordinate gradient descent algorithm for parameter estimation. Results on simulated and real data show the benefits of the proposed target detection and reconstruction approaches when compared to state-of-the-art processing algorithms.

Despite its promise, RFID technology presents numerous challenges, including incomplete data, lack of location and containment information, and very high volumes. In this work, we present a novel data interpretation and compression substrate over RFID streams to address these challenges in enterprise supply-chain environments. Our results show that our inference techniques provide good accuracy while retaining efficiency, and our compression algorithm yields significant reduction in data volume.

We propose a real-time image coding system, which is capable of adapting instantaneously to the available channel bandwidth. The range of operational bandwidths for this system has a finer calibration than ordinary hierarchical vector quantization (HVQ) or wavelet based hierarchical vector quantization (WHVQ) methods suggested in the literature. These properties make it very attractive for networks with fluctuating available bandwidth, like the Internet. All encoder and decoder operations are strictly constant time per pixel, proceeding through table lookups, and are intrinsically suitable for parallel and hardware implementation.

The paper considers the actual problem of improving neural network models for the classification of cardiovascular pathologies by compressing the information contained in electrocardiographic (ECG) signals. Due to the active implementation of artificial intelligence in medical diagnostics, the study focuses on creating a reduced ECG feature vector that allows for a significant reduction in training data volume without losing important diagnostic information. The research is dedicated to the development of an autoencoder model with a specialized architecture that combines convolutional and fully connected layers, attention layers, residual connections, and a symmetric structure with shared weights. This approach enables not only to compress the input multichannel signal but also to form a latent space from which the signal can be restored or used as a feature vector for classification. In the proposed autoencoder architecture, the latent representation is combined with the layer weight vector into a single reduced vector that contains information about both the shape and the structural features of the ECG signal. The model was trained using a composite loss function, which allows for balancing between signal reconstruction quality and classification accuracy. To evaluate the model, several alternative dimensionality reduction methods were used, including downsampling, transformation to Frank's orthogonal leads, calculation of a resultant vector, and principal component analysis (PCA). A number of alternative dimensionality reduction methods were used to test the model: reduction of the sampling frequency; transformation to Frank orthogonal derivations; calculation of the resulting vector; principal component analysis (PCA). All methods were tested on the PTB-XL dataset, which contains 12-lead ECG recordings with a wide range of cardiovascular pathologies. The effectiveness of the reduced feature vector was assessed using two models—a convolutional neural network (EcgNet) and a fully connected network with two hidden layers. The analysis of the results showed that, on average, classification accuracy using the reduced vector decreased by only 2% for EcgNet and increased by 3–6% for the fully connected network, indicating preservation of diagnostic information even with a high compression ratio of 25. In contrast, traditional dimensionality reduction methods such as PCA and orthogonal transformation showed a significant deterioration in classification quality (up to –16%). While downsampling to 75 Hz significantly reduces data volume, it also leads to the loss of high-frequency information critical for detecting ischemia and arrhythmias. Particular attention is given to ventricular and atrial late potentials, which are low-amplitude and high-frequency in nature. The constructed reduced feature vector preserved the informative characteristics of these classes, resulting in improved classification accuracy compared to traditional dimensionality reduction methods. Thus, the developed reduced feature vector demonstrates the ability to retain diagnostic information under significant ECG data compression, achieving a balance between computational efficiency, classification accuracy, and application versatility. The obtained results suggest that the use of the proposed reduced feature vector may serve as a promising solution for compact and efficient systems for automated ECG analysis, particularly in mobile or computing-limited systems, while also enabling faster development and testing of new neural network models for diagnostics.

In this paper, we propose a new low bit rate scheme that combines the Discrete Wavelet transform (IWT) and Vector Quantization (VQ) for image compression with high Compression Ratio (CR) in order to achieve acceptable quality of output images, Image compression using the combination of DWT willy VQ has been considered in many recent works; here in our paper the DWT was performed on several gray scale standard images, each of (256X256) byte. We used 4 images as the training images of the train VO using LBG algorithm and got the final optimal codebook. Then we used another two images to compare our output image with JPEG 2000 standard winch based on Wavelet and Scalar quantization. Our proposed system used a VO codebook with Variable cod hook size and dimensions were used to compress wavelet Transform) coefficients resulting using Danny wavelet types. Our pro system is slowed so that the best Wavelet types that you can use for compression is COIF 2. Our proposed system gave better PSNR than G2000 standard at high (CR), above than (5L), and this achieved a VQ codebook size (1024 -512) with dimension equal to 256. In the paper a typical system for the application of image compression system and new scientific results are present.

Presents two novel algorithms for compression of multilead electrocardiograms (MLECGs). In the first, the MLECG was decorrelated using a discrete cosine transform (DCT) and then a discrete wavelet transform (DWT) was applied on each channel after the removal of the least significant channels in the DCT domain. Significant compression ratios (CR) were achieved by retaining only the consequential DWT coefficients. The full potential of DWT was further exploited by appropriate quantization of these coefficients. Near perfect reconstruction was obtained for a CR of 1:8. This approach is highly suited for real time applications including that of high frequency Holter ECG. In the other algorithm, first level of compression was achieved by truncating the DWT while storing the highly significant coefficients from the truncated portion. Transform domain pole-zero modeling (TDPM) was then applied on the DWT compressed ECG to achieve higher compression ratios. In TDPM the DCT of a cycle of ECG is modeled as the impulse response of a pole-zero filter using Smith's (1983) method. Further compression was achieved by the application of vector quantization (VQ) on the model parameters. This algorithm resulted in a CR of 1:80 without significant distortion. The accuracy of reconstruction was checked using both qualitative and quantitative measures. The proposed algorithms were validated on both standard and local databases.

Reducing data volume in big data: Parallel processing based data filtering techniques Professor Shikharesh Majumdar from Carleton University examines reducing data volume in big data, focusing on parallel processing based data filtering techniques. Volume, velocity, and variety are three well-known characteristics of big data. The large data volumes often introduce formidable challenges to processing such data in a timely and economical manner. Research on data filtering is underway under the leadership of Shikharesh Majumdar at the Real Time and Distributed Systems Research Centre at Carleton University. Users are often interested only in a subset of the raw data.

Images are compressed using lossy and lossless compression schemes to utilize the Bandwidth in an effective way and also to provide enough space for storing voluminous data. Fractal Image Compression (FIC), is one such lossy compression scheme based on contractive mapping theorem. Affine transforms are being employed in Fractal Image compression to map the range blocks and domain blocks incorporating the property of self similarity in the images. As there is a need to enhance the performance measures like compression ratio (CR) and peak signal to noise ratio (PSNR), an attempt is made in this research paper to analyze FIC using Quad-tree Partitioning technique and Embedded Block Coding Optimization Truncation (EBCOT) technique for Medical Images. Here in this paper FIC with EBCOT is being implemented on Medical Images like CT of Bone and MR Images of Brain and the Performance measures like CR, PSNR at different Threshold Values were measured. Mat lab simulated results for these Performance Measures showed that the PSNR and CR is high when FIC is performed along with EBCOT Encoding.