A Practical Assessment of Process Data Compression Techniques

The ordinary kriging interpolation method is adopted to estimate the 2-D strain tensor elements. The results are then compared with those of the linear and spline interpolation methods. The ordinary kriging method is implemented on 12 GPS permanent stations in South California, collected during 2006 to 2012. By employing these time series, the coordinate changes in the UTM coordinate system can be obtained and the strain tensor elements at these 12 stations can be estimated by means of finite difference method. Interpolation of the strain tensor elements over the study area through these three methods (kriging, spline and linear) indicates that the ordinary kriging results, on average are 70% better than those of the spline and linear interpolation methods.

With the rapid advancement in electronics technologies, software-based beamformers for ultrasound array imaging have become feasible, and thus allowing rapid development of a high performance and potentially low cost system. However, to realize a fully software based system, one challenge is data transfer from the analog front end to the software backend, whose data rate can be up to a few GB/sec. It is therefore the purpose of this study to investigate the use of data compression for reducing data transfer requirements and the associated tradeoff with beamforming quality. JPEG (Joint Photographic Experts Group) and JPEG2000 compression were adopted. The acoustic data were acquired with a 128-channel array transducer at a center frequency of 3.5MHz. On each transmit event, the receive channel data are separated into 8×8 blocks, followed by the application of data compression. In one scheme, the compression is done on raw RF data. In another scheme, baseband data are used and only amplitude of the data is compressed. The maximum compression ratio of RF data compression is 15 with JPEG and 20 with JPEG2000 when the average error is less than 5 dB. With baseband data compression, on the other hand, although the maximum compression ratio is less than 8, the image quality is better than RF data compression. The average error in this case is less than 1.5 dB when the compression ratio is less than 7.

Data compression techniques in IoT-enabled wireless body sensor networks: A systematic literature review and research trends for QoS improvement

The rapid growth of digital libraries, e-governance, and internet based applications has caused an exponential escalation in the volume of ‘Big-data’ particularly due to texts, images, audios and videos that are being both archived and transmitted on a daily basis. In order to make their storage and transfer efficient, different data compression techniques are used in the literature. The ultimate motive behind data compression is to transform a big size data into small size data, which eventually implies less space while archiving, and less time in transferring. However, in order to operate/analyze compressed data, it is usually necessary to decompress it, so as to bring back the data to its original form, which unfortunately warrants an additional computing cost. In this backdrop, if operating upon the compressed data itself can be made possible without going through the stage of decompression, then the advantage that could be accomplished due to compression would escalate. Further due to compression, from the data structure and storage perspectives, the original visibility structure of the data also being lost, it turns into a potential challenge to trace the original information in the compressed representation. This challenge is the motivation behind exploring the idea of direct processing on the compressed data itself in the literature. The proposed survey paper specifically focuses on compressed document images and brings out two original contributions. The first contribution is that it presents a critical study on different image analysis and image compression techniques, and highlights the motivational reasons for pursuing document image analysis in the compressed domain. The second contribution is that it summarizes the different compressed domain techniques in the literature so far based on the type of compression and operations performed by them. Overall, the paper aims to provide a perspective for pursuing further research in the area of document image analysis and pattern recognition directly based on the compressed data.

Efficient and accurate imaging spectroscopy data processing asks for perfectly consistent (i.e., ideally uniform) data in both the spectral and spatial dimensions. However, real pushbroom-type imaging spectrometers are affected by various point spread function (PSF) nonuniformity artifacts. First, individual pixels or lines may be missing in the raw data due to bad pixels originating from the detector, readout errors, or even electronic failures. Second, so-called smile and keystone optical aberrations are inherent to imaging spectrometers. Appropriate resampling strategies are required for the preprocessing of such data if emphasis is put on spatial PSF uniformity. So far, nearest neighbor interpolations have been often recommended and used for resampling. This paper shall analyze the radiometric effects if linear interpolation is used to optimize the spatial PSF uniformity. For modeling interpolation effects, an extensive library of measured surface reflectance spectra as well as real imaging spectroscopy data over various land cover types are used. The real measurements are systematically replaced by interpolated values, and the deviation between original and resampled spectra is taken as a quality measure. The effects of nearest neighbor resampling and linear interpolation methods are compared. It is found that linear interpolation methods lead to average radiometric errors below 2% for the correction of spatial PSF nonuniformity in the subpixel domain, whereas the replacement of missing pixels leads to average errors in the range of 10%-20%

Several cybersecurity domains, such as ransomware detection, forensics and data analysis, require methods to reliably identify encrypted data fragments. Typically, current approaches employ statistics derived from byte-level distribution, such as entropy estimation, to identify encrypted fragments. However, modern content types use compression techniques which alter data distribution pushing it closer to the uniform distribution. The result is that current approaches exhibit unreliable encryption detection performance when compressed data appear in the dataset. Furthermore, proposed approaches are typically evaluated over few data types and fragment sizes, making it hard to assess their practical applicability. This paper compares existing statistical tests on a large, standardized dataset and shows that current approaches consistently fail to distinguish encrypted and compressed data on both small and large fragment sizes. We address these shortcomings and design EnCoD, a learning-based classifier which can reliably distinguish compressed and encrypted data. We evaluate EnCoD on a dataset of 16 different file types and fragment sizes ranging from 512B to 8KB. Our results highlight that EnCoD outperforms current approaches by a wide margin, with accuracy ranging from sim 82% for 512B fragments up to sim 92% for 8KB data fragments. Moreover, EnCoD can pinpoint the exact format of a given data fragment, rather than performing only binary classification like previous approaches.

The article is devoted to the issue of definition in situ vertical effective stress in weakly filtering soils (WFS). Shown. that the calculation of the vertical effective stress in soils, taking into account the weighing action of water, is valid only for the special case of the hydrostatic distribution of pore pressure over depth. In this regard, it is proposed to determine the vertical effective stress by the difference between total and pore pressure, in accordance with the principle of effective stresses of K. Terzagi, as more universal. The existing variants of pore pressure calculations in weakly filtering soils are considered. The example shows that zeroing the pore pressure leads to an overestimation of the in situ effective stress in the WFS, and the hydrostatic distribution, as a rule, leads to its underestimation. A computational method for determining the vertical effective stress in the WFS by linear interpolation is presented. Formulas for determining pore pressure and vertical effective stress in the WFS layer are proposed. The features of the application of the interpolation method for different values of groundwater levels in aquifers in depth are considered. A necessary condition for calculating the pore pressure by interpolation is information about the level of groundwater in the underlying aquifer, which must be taken into account when performing engineering-geological surveys. Examples of comparison of the results obtained by various calculation methods with the data of direct measurements of pore pressure by the dissipation method and the method of filled wells are given. The analysis showed that the values of pore pressure in the WFS layers calculated by linear interpolation correspond to direct measurements of pore pressure to the greatest extent. Thus, the linear interpolation method is the most reliable calculation method and can be recommended for use when performing geotechnical calculations.

Sparse-view tomography has many applications such as in low-dose computed tomography (CT). Using under-sampled data, a perfect image is not expected. The goal of this paper is to obtain a tomographic image that is better than the naïve filtered backprojection (FBP) reconstruction that uses linear interpolation to complete the measurements. This paper proposes a method to estimate the un-measured projections by displacement function interpolation. Displacement function estimation is a non-linear procedure and the linear interpolation is performed on the displacement function (instead of, on the sinogram itself). As a result, the estimated measurements are not the linear transformation of the measured data. The proposed method is compared with the linear interpolation methods, and the proposed method shows superior performance.

Processing an image is a process and image analysis which entangling many visual perceptions and in course have some characteristics of input data and output information in binding digital image. Process of changing measure for an image through magnification of resolution or measure often needed to showing the detail, for demonstration at a physic appliance, and for making of document. At the process on enlarging image resolution, method that would be used in this paper is linear interpolation. Linear Interpolation Method is mathematics algorithm which could be applied to appraise the middle price point through a diametrical line, two successive input point. The system in this paper could process image with algorithm of linear interpolation and become a new photo image with image resolution of pixel larger than the original image.Keywords: image, resolution, linear, interpolation, pixel

One critical issue in network real-time kinematic (NRTK) is the interpolation of atmospheric delay for user stations. Some classic interpolation algorithms, such as linear interpolation method (LIM), ignore the strong correlation between tropospheric delay and height factors, and the interpolation accuracy is poor in areas with large height difference. To solve this problem, a troposphere modelling method based on error compensation, namely ECDIM (Error Compensation-Based DIM), is proposed, and this method can be applied to both conventional single Delaunay triangulated network (DTN) and multi-station scenarios. The results of California Real Time Network (CRTN) with large height difference show that compared with LIM, the overall modelling accuracy with ECDIM has been improved by 50.1% to 67.3%, and especially for low elevation satellites (e.g., 10–20 degree), the accuracy is increased from tens of centimetres to a few centimetres. At user end, the positioning error in up direction with LIM has an obvious systematic deviation, and the fix rate of epoch is relatively low. This situation has been improved significantly after using ECDIM. The results of Tianjin Continuously Operating Reference System (TJCORS) show that in areas with small height difference, both methods have achieved high precision interpolation accuracy, and the positioning accuracy with ECDIM in up direction is improved by 21.2% compared with LIM.

Improved linear interpolation method for the estimation of snow-covered area from optical data

In this paper, the stability and smoothness of trajectory planning and attitude control of the manipulator are studied. The DH motion model of Kinova Jaco Gen-2 manipulator was established to solve the forward and inverse kinematics equations. Firstly, the third-order Bezier curve is designed to plan the trajectory of manipulator, which has good continuity and smoothness, and can control the workspace. Secondly, an attitude interpolation method based on quaternion is proposed. And different attitude interpolation methods are compared and analyzed, and spherical linear interpolation method is used to solve the problem of uneven velocity of the linear interpolation method. Finally, MATLAB robot toolbox is used to simulate and verify that the angle of each joint changes smoothly, and the trajectory of manipulator has good smoothness and stability based on Bezier curve and quaternion interpolation. According to the experimental results, the path planning based on the third order Bezier curve and spherical linear interpolation has a continuous smooth joint trajectory curve and a continuous joint angular velocity curve. The manipulator can move smoothly to the target point under the constraint of the end posture. Therefore, the trajectory planning method can be used as an ideal planning method for manipulator.

Analysing large-scale data from simulations of turbulent flows is memory intensive, requiring significant resources. This major challenge highlights the need for data compression techniques. In this study, we apply a physics-informed Deep Learning technique based on vector quantisation to generate a discrete, low-dimensional representation of data from simulations of three-dimensional turbulent flows. The deep learning framework is composed of convolutional layers and incorporates physical constraints on the flow, such as preserving incompressibility and global statistical characteristics of the velocity gradients. The accuracy of the model is assessed using statistical, comparison-based similarity and physics-based metrics. The training data set is produced from Direct Numerical Simulation of an incompressible, statistically stationary, isotropic turbulent flow. The performance of this lossy data compression scheme is evaluated not only with unseen data from the stationary, isotropic turbulent flow, but also with data from decaying isotropic turbulence, a Taylor–Green vortex flow, and a turbulent channel flow. Defining the compression ratio (CR) as the ratio of original data size to the compressed one, the results show that our model based on vector quantisation can offer CR with a mean square error (MSE) of , and predictions that faithfully reproduce the statistics of the flow, except at the very smallest scales where there is some loss. Compared to the recent study of Glaws. et al. [Deep learning for in situ data compression of large turbulent flow simulations. Phys Rev Fluids. 2020;5(11):114602], which was based on a conventional autoencoder (where compression is performed in a continuous space), our model improves the CR by more than 30%, and reduces the MSE by an order of magnitude. Our compression model is an attractive solution for situations where fast, high quality and low-overhead encoding and decoding of large data are required.

Image data compression is one of the most important techniques in digital image processing that used to reduce the size of image data with taking into account getting good quality of the compressed image, before several decades the researchers produced a lot of coding methods to decrease the image size, these methods include loss data compression and lossless data compression. in this research vector quantization techniques have been used to compress color image, firstly vector quantization has been applied using only cosine transform and then using principle component transform together with cosine transform, secondly vector quantization (VQ) has been achieved using wavelet transform only and then using the same transform with the principle component transform, many statistical properties have been shown and many fidelity criteria have been computed. This manuscript also includes the theory concepts of Vector Quantization, cosine transformation, principle component analysis and wavelet transform.

Right now, there is no agreed definition and measurement method for rating concentrator photovoltaic. This paper discusses present situation of the ratings of CPV modules and systems. First, it was analyzed which factors have influence to the performance to CPV modules and systems. Next, the linear interpolation method that is known as a robust estimator especially to multi-junction thin film modules is discussed. The test data was provided from Toyohashi University of Technology where a 550 X CPV module (20 cells connected in series and the total area is 5,445 cm2) was tested independently. The relative standard deviation was 0.89 %, whereas that of the linear interpolation method was 1.2 %. Different from the linear interpolation method, this stratified averaging method is unbiased.