Entropy encoder for low-power low-resources high-quality CFA image compression

This paper proposes an effective color-difference-based (ECDB) interpolation algorithm for color filter array (CFA) Image demosaicking. A CFA image consists of a set of spectrally selective filters which are arranged in an interleaved pattern such that only one color component is sampled at each pixel location. To improve the quality of reconstructed full-color images from CFA images, the ECDB algorithm first analyzes the samples immediately around a missing green pixel to determine suitable samples for interpolating the value of this missing green pixel. After finishing the interpolation operations of all the missing green pixels, a complete green plane (i.e G plane) is obtained. The ECDB algorithm then makes use of the high correlation between the R, G, and B planes to produce the red-green and blue-green color difference planes and further reconstructs the red and blue planes in successive operations. Because the green plane provides twice the information of the red and blue planes, the algorithm uses the information of the green plane more than that of the red/blue planes so that the full color image can be reconstructed more accurately. In essence, the ECDB algorithm uses the red-green and blue-green color difference planes, and develops different conditional operations according to the horizontal, vertical, and diagonal neighboring pixel information with a suitable weighting technique. The experimental results demonstrate that the proposed algorithm performed extremely well.

Lossless compression of HDR color filter array image for the digital camera pipeline

Most digital cameras use a single sensor coupled with a Color Filter Array (CFA) to capture images, and apply demosaicking to interpolate the full color images. In reality, the CFA image is noisy, which causes problems in the demosaicking process. This paper proposes a Joint Denoising and Demo-saicking based on inter-Color correlation (JDDC) scheme. We propose a new framework that linearly combines an extracted luminance image and a low-passed RGB images to get a full color image. Given the noise in the extracted luminance image and the low-passed RGB images are non-stationary and partially correlated, we modify the classical Non-Local Means (NLM) filter to denoise the extracted luminance image and the low-passed RGB images before the combination. Experimental results verify the effectiveness of the proposed scheme both objectively and subjectively.

In general digital camera uses a single-chip image sensor, with spectrally shifted alternating patterns of three or more color filters applied to each pixel location called color filter array. Digital cameras with single sensor use demosaicing to fabricate full color images from the data captured by a color filter array (CFA). The goodness of demosaiced images is subject to influences from noise in the image sensor during the image capturing process. In order to reduce the noise the conventional solution like Joint Demosaicing and denoising may be adopted, that combines both demosaicing and denoising procedures in to a single operation. Another method, Total least square denoising method is also used to improve image quality. But, In PCA based spatially adaptive denoising of CFA, images uses the concept of denoising first and demosaicing later for reducing the noise. It has some advantages such as less noise-caused color artifacts and cost-effective implementation. When compared to Joint Demosaicing and denoising, PCA Based spatially adaptive denoising of CFA Images is more effective in reducing the noise in terms of both objective measurement and visual evaluation. This method effectively suppresses noise while preserving color edges and details. The direct CFA image denoising scheme, followed by a succeeding demosaicing scheme reduces extensively the noise-caused color artifacts in the demosaiced images. Such artifacts often appear in the output full-color images of many demosaicing first and denoising later schemes, as well as joint demosaicing-denoising schemes. The CPSNR ratio is much better, when compared to Joint demosaicing and denoising schemes by preserving the structures in the image which are often smoothed by other denoising schemes.

We present a computationally efficient demosaicing algorithm based on a luminance-chrominance model of the color filter array (CFA) image. We show that the chrominance information can be estimated using simple low-pass filtering. This algorithm allows us to use separable recursive filters, which are particularly adapted for real-time processing. Moreover, while most of demosaicing algorithms are specific to a particular CFA (usually the popular Bayer CFA), our method can be applied to any CFA. We present a linear version of the algorithm and an adaptive extension.

Design and performance analysis of a zonal DCT-based image encoder for Wireless Camera Sensor Networks

Classic McEliece has gathered attention as a candidate in NIST post-quantum cryptography standardization. However, it suffers from high demands on the decryption algorithm, making it unsuitable for resource-constrained devices. In this paper, we propose a novel implementation method, named giant footprint sharing, that reduces memory size during decryption in Classic McEliece. The decryption algorithm processes a large number of intermediate variables computed from a secret key in memory. The giant footprint sharing identifies the largest variable among them and allocates a memory-sharing structure to store it, thereby reducing the overall memory size regardless of the implementation platform. The giant footprint sharing can also be combined with existing acceleration techniques, such as fast Fourier transformation. We evaluate Classic McEliece with the giant footprint sharing on the Arm Cortex-M33 CPU and show that it reduces memory size by up to 46% without significant degradation in computation time compared with the existing fast-implementation by Chen et al. (at TCHES 2021). Extensive experiments with the giant footprint sharing further reveal that it maintains a constant memory size regardless of the compiler optimization, and it also achieves an optimal balance in the trade-off between memory size and computation time. The giant footprint sharing is remarkable for any scheme, that contains a large-scale matrix computation and the life cycle for each variable is limited.

Common consumer RGB cameras use a single image sensor to capture RGB color images with a color filter array (CFA) placed in front of the image sensor. This system captures particularly one color sample ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$R$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$G$</tex-math></inline-formula> , or <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$B$</tex-math></inline-formula> ) at each pixel location. This captured raw image is called a CFA image. The CFA demosaicking is used then to construct the complete color image. A similar approach of working with a single image sensor and extension of CFA to multispectral filter array (MSFA) enables us to develop a low-cost multispectral camera with the help of MSFA demosaicking. However, due to many spectral bands and their very sparse sampling, extending CFA demosaicking to MSFA demosaicking is not easy. This paper proposes an adaptive and progressive MSFA demosaicking method for various MSFA patterns for 5 to 15 bands multispectral images. Binary tree-based MSFA patterns are used to create MSFA images, and these MSFA patterns can be designed for any N-band MSFA image. The spectral bands are arranged specifically so that the middle band has the highest probability of appearance (PoA) in the MSFA pattern. The middle band is interpolated first using PoA based progressive adaptive interpolation, and the modified bilinear spectral difference method is used to estimate all other spectral bands with the interpolated middle band’s help. Experimental results and comparative analysis on two different datasets reveal that our proposed MSFA demosaicking method outperforms the existing generic MSFA demosaicking methods in terms of computational time and various image quality metrics considered.

In positron emission tomography (PET), 3D iterative image reconstruction methods have a huge computational burden. In this paper, we developed a list-mode image reconstruction method using graphics processing units (GPUs). Efficiency of acceleration for GPU implementation largely depends on the method chosen, where a reduced number of conditional statements and a reduced memory size are required. On the other hand, accurate system models are required to improve the quality of reconstructed images. Various accurate system models for conventional CPU implementation have been proposed, but these models basically require many conditional statements and huge memory size. Therefore, we developed a new system model which matches GPU implementation better. In this model, the detector response functions, which vary depending on each line of response (LOR), are pre-computed in CPUs and modeled by sixth-order polynomial functions in order to reduce the memory size occupied in GPUs. Each element of a system matrix is obtained on-the-fly in GPUs by calculating the distance between an LOR and a voxel. Therefore the developed system model enables efficient GPU implementation of the accurate system modeling with a reduced number of conditional statements and a reduced memory size. We applied the developed method to a small OpenPET prototype, in which 4-layered depth-of-interaction (DOI) detectors were used. For image reconstruction, we used the dynamic row-action maximum likelihood algorithm (DRAMA). Compared with a conventional model for GPU implementation, in which DRFs are given as a Gaussian function of fixed width, we saw no remarkable difference for DOI data, but for non-DOI data the proposed model outperformed the conventional at the peripheral region of the field-of-view. The proposed model had almost the same calculation time as the conventional model did. For further acceleration, we tried parallel GPU implementation, and we obtained 3.8-fold acceleration by using 4 GPUs.

A new demosaicking framework for single-sensor imaging devices operating on a Bayer color filter array (CFA) is introduced and analyzed. An efficient data adaptive filtering concept in conjunction with the refined spectral models constitutes the base for the proposed framework. Using a different form of the function mapping the aggregated absolute differences among the CFA inputs to the edge-sensing weighting coefficients, the framework allows to design fully automated demosaicking solutions suitable for common digital imaging apparatus, and alternatively, the proposed solutions can also be used to support PC-based demosaicking of the raw CFA images. Thus, the framework can be seen as a universal tool satisfying the needs of the end-users for i) the instant access and visualization of the captured images, and ii) the interactive processing of the raw sensor data. Moreover, the proposed framework is relatively easy to implement in either software or hardware. Experimental results indicate that the proposed framework exhibits excellent performance in terms of the commonly used objective criteria and at the same time it produces demosaicked images with impressive visual quality.

The paper presents hardware implementation aspects of new efficient image compression algorithm designed for wireless capsule endoscopy with Bayer color filter array (CFA). Since power limitation, small size conditions and specific image data format (CFA) exclude application of traditional image compression techniques dedicated ones are necessary. Discussed algorithm is based on integer version of discrete cosine transform (DCT). Therefore it has low complexity and power consumption. It is demonstrated that the perfor-mance of proposed algorithm is comparable to the performance of JPEG2000 --- very complex, sophisticated wavelet based coder. In the paper a VLSI coder architecture is proposed and power requirements are discussed.

To reconstruct image down-sampled by color filter array on sensor of digital camera, demosaicking, a process of estimating missing color values is required. An improved edge adaptive color demosaicking method is presented. Edges are detected by an improved edge indicator, and missing color values are estimated in chosen direction. In estimation stage, weighted average of Taylor series is adopted. Experimental results show that the proposed scheme with a good balance between memory size and computational complexity not only outperforms other existed methods in subjective visual quality but also in terms of peak signal-to-noise ratio (PSNR) with 1.8 dB improvement on the average over testing images.

This paper proposes a nonuniform-bandwidth onboard channelizing and switching architecture for broadband satellite communications. Detailed simulations are carried out to explicitly verify the flexibility and scalability of the proposed architecture. Significant performance improvement and reduction in memory size have been demonstrated, making the digital channelization technique attractive for future flexible communications satellite payload architectures.

This study addresses the optimization of a camera-based bird's eye view (BEV) segmentation technique that operates in real-time within an embedded system environment while maintaining high accuracy despite limited computational resources. Specifically, it examines three technical approaches for BEV segmentation in autonomous driving: depth-based methods, MLP-based methods, and transformer-based methods, focusing on key techniques such as lift-splat-shoot, HDMapNet, and BEVFormer. A mathematical analysis of these methods is conducted, followed by a comparative performance evaluation using the nuScenes dataset. The optimization process was carried out in three stages: accuracy improvement, latency reduction, and model size optimization. In the first stage of the process, the three modules for BEV segmentation (encoder, view transformation, and decoder) were selected with the goal of maximizing mIoU performance. In the second stage, environmental variable optimization was performed through input resolution adaptation and data augmentation to improve accuracy. Finally, in the third stage, model compression was applied to minimize model size and latency for efficient deployment on embedded systems. Experimental results from the first stage show that the lift-splat-shoot view transformation model, based on the InternImage-B encoder and EfficientNet-B0 decoder, achieved the highest performance with 54.9 mIoU at an input image size of 448×800. Notably, the lift-splat-shoot view transformation model with the InternImage-T encoder and EfficientNet-B0 decoder demonstrated performance of 53.1 mIoU while achieving high efficiency (51.7 ms and 159.5 MB, respectively). The application of the second stage revealed that increasing the input resolution does not always lead to improved accuracy, and there is an optimal resolution size depending on the model. In this study, the best performance was achieved with an input image size of 448×800. During the third stage, FP16 quantization enabled a 50% reduction in memory size and decreased latency while maintaining similar or identical mIoU performance. When deployed on the NVIDIA AGX Orin device, which operates under power constraints, energy efficiency improved, although it resulted in higher latency under certain power supply conditions. As a result, the InternImage encoder-based lift-splat-shoot technique was shown to achieve the highest accuracy performance relative to latency and model size. This approach outperformed the original method by achieving a 29.2% higher mIoU while maintaining similar latency performance and reducing memory size by 32.2%.

In this paper, a new low complexity and lossless image compression system for capsule endoscopy (CE) is presented. The compressor consists of a low-cost YEF color space converter and variable-length predictive with a combination of Golomb-Rice and unary encoding. All these components have been heavily optimized for low-power and low-cost and lossless in nature. As a result, the entire compression system does not incur any loss of image information. Unlike transform based algorithms, the compressor can be interfaced with commercial image sensors which send pixel data in raster-scan fashion that eliminates the need of having large buffer memory. The compression algorithm is capable to work with white light imaging (WLI) and narrow band imaging (NBI) with average compression ratio of 78% and 84% respectively. Finally, a complete capsule endoscopy system is developed on a single, low-power, 65-nm field programmable gate arrays (FPGA) chip. The prototype is developed using circular PCBs having a diameter of 16 mm. Several in-vivo and ex-vivo trials using pig's intestine have been conducted using the prototype to validate the performance of the proposed lossless compression algorithm. The results show that, compared with all other existing works, the proposed algorithm offers a solution to wireless capsule endoscopy with lossless and yet acceptable level of compression.