Error detection algorithm for Lempel-Ziv-77 compressed data

Maximum distance separable (MDS) codes are those codes whose minimum distance attains the Singleton bound for a given length and given dimension. It is a well-known fact that the only binary linear block codes that are MDS are the simple parity codes and the repetition codes. Known MDS codes (the Reed-Solomon codes, for instance) are defined over larger alphabets. This approach requires that (i) the encoding and decoding procedures are performed as operations over a finite field and (ii) an update in a single information bit (e.g. in storage applications) requires an update in the redundancy symbols and usually affects a number of bits in each redundancy symbol. Thus, the optimal redundancy is achieved at the expense of additional complexity in the encoding/decoding procedures as well as in the number of redundancy bits affected by an update in the information. We construct MDS codes that have the following two properties: (i) the redundancy bits are computed by simple XOR operations; and (ii) an update in an information bit affects a minimal number of redundancy bits. >

A fast burst error-correction decoder is proposed. It can be used for high-speed decoding of a burst error-correcting Fire code having the generator polynomial G(x) = (1 + x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</sup> )p(x), where p(x) is irreducible and of degree m and the degree of G (x) is r = c + m, r being the number of redundancy bits. The decoder needs at most r -1 cycles to find both the error burst pattern and its location, whereas conventional decoding needs up to n cycles [1] (n is the number of bits in a codeword), or a fraction of n [2]. The decoding scheme is based on new aspects of the structure of the error syndrome and the parity check matrix of Fire code recently observed by the author. These aspects allow partial syndrome processing in the decoding process. This makes hardware costs relatively low and enables very high-speed decoding. The decoding scheme makes use of modern mapping devices such as programmable read only memories (PROM's) and field programmable logic arrays (FPLA's), resulting in relatively compact and cheap hardware. The encoder/decoder could probably be implemented as a single-chip LSI circuit.

Many images you see on the Internet today have undergone compression for various reasons. Image compression can benefit users by having pictures load faster and webpages use up less space on a Web host. Image compression does not reduce the physical size of an image but instead compresses the data that makes up the image into a smaller size. In case of image transmission the noise will decrease the quality of recivide image which obliges us to use channel coding techniques to protect our data against the channel noise. The Reed-Solomon code is one of the most popular channel coding techniques used to correct errors in many systems ((Wireless or mobile communications, Satellite communications, Digital television / DVB,High-speed modems such as ADSL, xDSL, etc.). Since there is lot of possibilities to select the input parameters of RS code this will make us concerned about the optimum input that can protect our data with minimum number of redundant bits. In this paper we are going to use the genetic algorithm to optimize in the selction of input parameters of RS code acording to the channel conditions wich reduce the number of bits needed to protect our data with hight quality of received image.

13 - Error Detection

MPPM based dimming control scheme in visible light communication systems

Error detection is a fundamental need in most computer networks and communication systems in order to combat the effect of noise. Error detection techniques have also been incorporated with lossless data compression algorithms for transmission across communication networks. In this paper, we propose to incorporate a novel error detection scheme into a Shannon optimal lossless data compression algorithm known as Generalized Luröth Series (GLS) coding. GLS-coding is a generalization of the popular Arithmetic Coding which is an integral part of the JPEG2000 standard for still image compression. GLS-coding encodes the input message as a symbolic sequence on an appropriate 1D chaotic map Generalized Luröth Series (GLS) and the compressed file is obtained as the initial value by iterating backwards on the map. However, in the presence of noise, even small errors in the compressed file leads to catastrophic decoding errors owing to sensitive dependence on initial values, the hallmark of deterministic chaos. In this paper, we first show that repetition codes, the oldest and the most basic error correction and detection codes in literature, actually lie on a Cantor set with a fractal dimension of }{} frac{1}{n} , which is also the rate of the code. Inspired by this, we incorporate error detection capability to GLS-coding by ensuring that the compressed file (initial value on the chaotic map) lies on a Cantor set. Even a 1-bit error in the initial value will throw it outside the Cantor set, which can be detected while decoding. The rate of the code can be adjusted by the fractal dimension of the Cantor set, thereby controlling the error detection performance.

We proposed and verified the methods to maintain data qualities as well as to reduce data volume for the Geostationary Ocean Color Imager (GOCI), the world’s first ocean color sensor operated in geostationary orbit. For the GOCI level-2 data, 92.9% of data volume could be saved by only the data compression. For the GOCI level-1 data, however, just 20.7% of data volume could be saved by the data compression therefore another approach was required. First, we found the optimized number of bits per a pixel for the GOCI level-1 data from an idea that the quantization bit for the GOCI (i.e. 12 bit) was less than the number of bits per a pixel for the GOCI level-1 data (i.e. 32 bit). Experiments were conducted using the R2 and the Modulation Transfer Function (MTF). It was quantitatively revealed that the data qualities were maintained although the number of bits per a pixel was reduced to 14. Also, we performed network simulations using the Network Simulator 2 (Ns2). The result showed that 57.7% of the end-toend delay for a GOCI level-1 data was saved when the number of bits per a pixel was reduced to 14 and 92.5% of the end-to-end delay for a GOCI level-2 data was saved when 92.9% of the data size was reduced due to the compression.

In a recent era of modern technology, there are many problems for storage, retrieval and transmission of data. Data compression is necessary due to rapid growth of digital media and the subsequent need for reduce storage size and transmit the data in an effective and efficient manner over the networks. It reduces the transmission traffic on internet also. Data compression try to reduce the number of bits required to store digitally. The various data and image compression algorithms are widely use to reduce the original data bits into lesser number of bits. Lossless data and image compression is a special class of data compression. This algorithm involves in reducing numbers of bits by identifying and eliminating statistical data redundancy in input data. It is very simple and effective method. It provides good lossless compression of input data. This is useful on data that contains many consecutive runs of the same values. This paper presents the implementation of Run Length Encoding for data compression.

In recent years, data hiding techniques have emerged as a prominent area of research due to the increasing demand for information security. Existing data hiding techniques typically do not prioritize the verification of the correctness of the extracted data, nor do they attempt to rectify any erroneous information extracted. This paper presents a simple yet effective method by which a recipient can perform error detection and correction on extracted data. Prior to data embedding, a preprocess is applied to secret data. After converting the secret data to a data stream, a data hider categorizes adjacent and identical bits within the data stream into groups. The number of these bits is then recorded as an indicator to extract information and perform error detection and correction. The number of bits in each group is adjusted to ensure that it is an odd number. Moreover, the collected indicator data stream is encoded using (7,4) Hamming code to compact the recorded information. Finally, the concatenation of the encoded indicator and the encoded secret is embedded into the multiple least significant bits (LSBs) of the original image pixels. As for the receiver, error correction can be applied based on the indicator after the extraction of the encoded data. Experimental results demonstrate that this method allows the data receiver to detect and correct errors while maintaining a certain level of embedding capacity and image quality. In terms of image quality, this method exhibits superior performance compared to existing research when the embedded data volume is relatively small.

The increasing demand for high-density and high-speed digital optical recording systems has made the development of advanced coding and signal processing techniques for fast and reliable data recovery increasingly important. In recent years, the parity-check (PC)-code-based reception technique has been widely studied for magnetic recording systems, and it is projected to be highly promising for high-density optical recording. The PC code is an inner error correction code (ECC), which can detect dominant short error events of the system, using only a few parity bits. This reduces the loss in error correction capability of the outer ECC due to random short errors and results in a simple and efficient solution to improve the overall performance. This thesis is dedicated to the design and analysis of PCcode- based recording systems to achieve higher recording capacity with low implementation complexity, for high-density blue laser disc systems. In particular, most of the key components of the PC-code-based optical recording system have been designed and optimized for different recording densities, and different proportions of white noise and media noise. During the development of advanced coding and detection techniques, it becomes necessary to investigate the system’s performance with different coding schemes and recording densities. In Chapter 2 of the thesis, we propose a generalized Braat-Hopkins model for optical recording channels, which provides a fair basis for the performance comparison of detection schemes over different coding schemes and recording densities, for channels with additive, white Gaussian noise (AWGN) and media noise. Various basic issues associated with the PC-code-based systems are investigated in Chapter 3. These include bounds for bit error rates and error event probabilities, the dominant error events at channel detector output, different PC codes, simple and efficient post-processors, the impact of error events that are split across data block boundaries and the corresponding remedy, as well as the effects of code rates and recording densities. Simulation results show that a 4-bit PC code achieves the best performance. The corresponding bit error rates (BERs) are very close to the performance bounds, at both nominal and high densities. Constrained codes, which serve as a key component in the read-write channel of data storage systems, are desired to have a high code rate with low-complexity encoder/decoder. In Chapter 4, we investigate the design of certain capacity-approaching constrained codes for optical recording systems. In particular, we derive analytically the relationship between the number of encoder states and the maximum size of these codes. We identify the minimum number of encoder states that maximizes the code rate, for any desired codeword length. The design of constrained PC codes is key to the development of PCcode- based systems, and the systematic design of efficient constrained PC codes remains a challenging problem. In Chapter 5, we propose a general and systematic code design technique for constructing capacity-approaching constrained PC codes, which can detect any type of error events of the system, with the minimum code rate loss. Compared to the rate 2/3 code without parity, the newly designed constrained 4-bit PC code achieves a performance gain of 2 dB at nominal density, and 1.5 dB at high density, at BER = 10-5. As the dominant noise for high-density optical recording systems, media noise severely degrades the performance of channel detectors and postprocessors that are designed for AWGN. In Chapter 6, we propose two novel modifications to the bit detector to combat media noise. We further develop a data-dependent post-processing scheme for error correction. Compared to the system designed without considering media noise and without PC codes, the overall performance gain of the developed scheme can be more than 11 dB at high media noise levels. In data storage systems, the ECC failure rate (EFR) serves as the ultimate measure of the data recovery performance. In Chapter 7, we develop semi-analytical approaches for estimating the EFR, and analyze EFRs for the developed PC-code-based systems, for the cases without and with interleaving. Our analysis shows that with the optimum interleaving degrees, compared to the rate 9/13 code without parity, the 2-bit PC code achieves a gain of around 0.5 dB, and the 4-bit PC code gains 0.6 dB, at high record ing density and EFR = 10-16. The thesis concludes in Chapter 8 with some suggestions for further work.

Background Even when a laboratory analyte testing process is in control, routine quality control testing will fail with a frequency that can be predicted by the number of quality control levels used, the run frequency and the control rule employed. We explored whether simply counting the number of assay quality control run failures during a running week, and then objectively determining if there was an excess, could complement daily quality control processes in identifying an out-of-control assay. Methods Binomial statistics were used to determine the threshold number of quality control run failures in any rolling week which would statistically exceed that expected for a particular test. Power function graphs were used to establish error detection (Ped) and false rejection rates compared with popular control rules. Results Identifying quality control failures exceeding the weekly limit (QC FEWL) is a more powerful means of detecting smaller systematic (bias) errors than traditional daily control rules (12s, 13s or 13s/22s/R4s) and markedly superior in detecting smaller random (imprecision) errors while maintaining false identification rates below 2%. Error detection rates also exceeded those using a within- and between-run Westgard multirule (13s/22s/41s/10x). Conclusions Daily review of tests shown to statistically exceed their rolling week limit of expected quality control run failures is more powerful than traditional quality control tools at identifying potential systematic and random test errors and so offers a supplement to daily quality control practices that has no requirement for complex data extraction or manipulation.

Quantum algorithms can be analyzed in a query model to compute Boolean functions. Function input is provided in a black box, and the aim is to compute the function value using as few queries to the black box as possible. A repetition code is an error detection scheme that repeats each bit of the original message r times. After a message with redundant bits is transmitted via a communication channel, it must be verified. If the received message consists of r-size blocks of equal bits, the conclusion is that there were no errors. The verification procedure can be interpreted as an application of a query algorithm, where input is a message to be checked. Classically, for N-bit message, values of all N variables must be queried. We demonstrate an exact quantum algorithm that uses only N/2 queries.

In this paper we propose a new adaptive block based compressed domain data hiding scheme which can embed relatively large number of secret bits without significant perceptual distortion in video domain. Macro blocks are selected for embedding on the basis of low inter frame velocity. From this subset, the blocks with high prediction error are selected for embedding. The embedding is done by modifying the quantized DCT AC coefficients in the compressed domain. The number of coefficients (both zero and non zero) used in embedding is adaptively determined using relative strength of the prediction error block. Experimental results show that this blind scheme can embed a relatively large number of bits without degrading significant video quality with respect to Human Visual System (HVS).

Lossy compression approach to subspace clustering

This paper presents a low-power encoding technique, called chromatic encoding, for the Digital Visual Interface standard (DVI), a digital serial video interface. Chromatic encoding reduces power consumption by minimizing the transition counts on the DVI. This technique relies on the notion of tonal locality, i.e., the observation that the signal differences between adjacent pixels in images follow a Gaussian distribution. Based on this observation, an optimal code assignment is performed to minimize the transition counts. Furthermore, the three color channels of the DVI may be reciprocally encoded to achieve even more power saving. The idea is that given the signal values from the three color channels, one or two of these channels are encoded by reciprocal differences with a number of redundant bits used to indicate the selection. The proposed technique requires only three redundant bits for each 24-bit pixel. Experimental results show up to a 75% transition reduction.