Методика расчета параметров антиалайсинговых фильтров для коррекции спектральных характеристик в зависимости от используемых структур дискретизации массива светофильтров

Since the birth of charge coupled devices (CCD) and the complementary metal-oxide-semiconductor (CMOS) active pixel sensors, pixel pitch of digital image sensors has been continuously shrinking to meet the resolution and size requirements of the cameras. However, shrinking pixels reduces the maximum number of photons a sensor can hold, a phenomenon broadly known as the full-well capacity limit. The drop in full-well capacity causes drop in signal-to-noise ratio and dynamic range.The Quanta Image Sensor (QIS) is a class of solid-state image sensors proposed by Eric Fossum in 2005 as a potential solution for the limited full-well capacity problem. QIS is envisioned to be the next generation image sensor after CCD and CMOS since it enables sub-diffraction-limit pixels without the inherited problems of pixel shrinking. Equipped with a massive number of detectors that have single-photon sensitivity, the sensor counts the incoming photons and triggers a binary response “1” if the photon count exceeds a threshold, or “0” otherwise. To acquire an image, the sensor oversamples the space and time to generate a sequence of binary bit maps. Because of this binary sensing mechanism, the full-well capacity, signal-to-noise ratio and the dynamic range can all be improved using an appropriate image reconstruction algorithm. The contribution of this thesis is to address three image processing problems in QIS: 1) Image reconstruction, 2) Threshold design and 3) Color filter array design.Part 1 of the thesis focuses on reconstructing the latent grayscale image from the QIS binary measurements. Image reconstruction is a necessary step for QIS because the raw binary measurements are not images. Previous methods in the literature use iterative algorithms which are computationally expensive. By modeling the QIS binary measurements as quantized Poisson random variables, a new non-iterative image reconstruction method based on the Transform-Denoise framework is proposed. Experimental results show that the new method produces better quality images while requiring less computing time.Part 2 of the thesis considers the threshold design problem of a QIS. A spatially-varying threshold can significantly improve the reconstruction quality and the dynamic range. However, no known method of how to achieve this can be found in the literature. The theoretical analysis of this part shows that the optimal threshold should match with the underlying pixel intensity. In addition, the analysis proves the existence of a set of thresholds around the optimal threshold that give asymptotically unbiased reconstructions. The asymptotic unbiasedness has a phase transition behavior. A new threshold update scheme based on this idea is proposed. Experimentally, the new method can provide good estimates of the thresholds with less computing budget compared to existing methods.Part 3 of the thesis extends QIS capabilities to color imaging by studying how a color filter array should be designed. Because of the small pixel pitch of QIS, crosstalk between neighboring pixels is inevitable and should be considered when designing the color filter arrays. However, optimizing the light efficiency while suppressing aliasing and crosstalk in a color filter array are conflicting tasks. A new optimization framework is proposed to solve the problem. The new framework unifies several mainstream design criteria while offering generality and flexibility. Extensive experimental comparisons demonstrate the effectiveness of the framework.

To save the hardware cost, most modern digital video cameras employ the single-sensor technology together with the red-green-blue (RGB) color filter array (CFA) structure to capture real-world scenes. Each pixel in the captured video is only composed of one primary color. This kind of videos is usually referred to be the CFA video (or mosaic video). For the purpose of visual display, it is necessary to convert the captured CFA video into the full-color RGB video. In this paper, we propose a novel quality-efficient universal demosaicing for arbitrary CFA videos. In the proposed method, both the spatial and the temporal correlations among the CFA data are taken into account during the demosaicing. Then an effective least square-based approach is developed to fuse the spatial-based demosaiced result and the temporal-based one. Experimental results on 27 test videos with ten typical RGB CFA structures show that the demosaiced videos generated by the proposed universal demosaicing method yield better quality in terms of average color peak signal-to-noise ratio (CPSNR) and demonstrate less color artifacts in terms of visual perception in comparison with two state-of-the-art universal demosaicing methods.

Non-Bayer color filter array (CFA) sensors have recently drawn attention due to their superior compression of spectral energy, ability to deliver improved signal-to-noise ratio, or ability to provide high dynamic range (HDR) imaging. Demosaicking methods that perform color interpolation of Bayer CFA data have been widely investigated. However, a bottleneck to the adaption of emerging non-Bayer CFA sensors is the unavailability of efficient color-interpolation algorithms that can demosaick the new patterns. Designing a new demosaick algorithm for every proposed CFA pattern is a challenge. In this paper, we propose a hardware-friendly universal demosaick algorithm based on maximum a-posteriori (MAP) estimation that can be configured to demosaick raw images captured using a variety of CFA sensors. The forward process of mosaicking is modeled as a linear operation. We then use quadratic data-fitting and image prior terms in a MAP framework and pre-compute the inverse matrix for recovering the full RGB image from CFA observations for a given pattern. The pre-computed inverse is later used in real-time application to demosaick the given CFA pattern. The inverse matrix is observed to have a Toeplitz-like structure, allowing for hardware-efficient implementation of the algorithm. We use a set of 24 Kodak color images to evaluate the quality of our demosaick algorithm on three different CFA patterns. The PSNR values of the reconstructed full-channel RGB images from CFA samples are reported in the paper.

Common consumer RGB cameras use a single image sensor to capture RGB color images with a color filter array (CFA) placed in front of the image sensor. This system captures particularly one color sample ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$R$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$G$</tex-math></inline-formula> , or <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$B$</tex-math></inline-formula> ) at each pixel location. This captured raw image is called a CFA image. The CFA demosaicking is used then to construct the complete color image. A similar approach of working with a single image sensor and extension of CFA to multispectral filter array (MSFA) enables us to develop a low-cost multispectral camera with the help of MSFA demosaicking. However, due to many spectral bands and their very sparse sampling, extending CFA demosaicking to MSFA demosaicking is not easy. This paper proposes an adaptive and progressive MSFA demosaicking method for various MSFA patterns for 5 to 15 bands multispectral images. Binary tree-based MSFA patterns are used to create MSFA images, and these MSFA patterns can be designed for any N-band MSFA image. The spectral bands are arranged specifically so that the middle band has the highest probability of appearance (PoA) in the MSFA pattern. The middle band is interpolated first using PoA based progressive adaptive interpolation, and the modified bilinear spectral difference method is used to estimate all other spectral bands with the interpolated middle band’s help. Experimental results and comparative analysis on two different datasets reveal that our proposed MSFA demosaicking method outperforms the existing generic MSFA demosaicking methods in terms of computational time and various image quality metrics considered.

An image sensor for an ultra-high-speed video camera was developed. The maximum frame rate, the pixel count and the number of consecutive frames are 1,000,000 fps, 720 x 410 (= 295,200) pixels, and 144 frames. A micro lens array will be attached on the chip, which increases the fill factor to about 50%. In addition to the ultra-high-speed image capturing operation to store image signals in the in-situ storage area adjacent to each pixel, standard parallel readout operation at 1,000 fps for full frame readout is also introduced with sixteen readout taps, for which the image signals are transferred to and stored in a storage device with a large capacity equipped outside the sensor. The aspect ratio of the frame is about 16 : 9, which is equal to that of the HDTV format. Therefore, a video camera with four sensors of the ISIS-V4, which are arranged to form the Bayer’s color filter array, realizes an ultra-high-speed video camera of a semi-HDTV format.

The ordered color filter arrays (CFA) used in single sensor, color digital still cameras introduce distracting color artifacts. These artifacts are due to the phase shifted, aliased signals introduced by the sparse sampling by the CFAs. This work reports the results of an investigation on the possibility of using random patterns as a CFA for single sensor, digital still cameras. From a single blue noise mask pattern, three mutually exclusive, random CFAs are constructed representing the red, green, and blue color filters. An edge adaptive method consisting of missing-pixel edge detection and boundary sensitive interpolation is employed to reconstruct the entire image. Experiments have shown that the random CFA alleviates the problem of the low-frequency color banding associated with ordered arrays. This method also has the advantage of better preserving color free, sharp neutral edges, and results in less deviation from neutral on high frequency, monochrome information.

In general digital camera uses a single-chip image sensor, with spectrally shifted alternating patterns of three or more color filters applied to each pixel location called color filter array. Digital cameras with single sensor use demosaicing to fabricate full color images from the data captured by a color filter array (CFA). The goodness of demosaiced images is subject to influences from noise in the image sensor during the image capturing process. In order to reduce the noise the conventional solution like Joint Demosaicing and denoising may be adopted, that combines both demosaicing and denoising procedures in to a single operation. Another method, Total least square denoising method is also used to improve image quality. But, In PCA based spatially adaptive denoising of CFA, images uses the concept of denoising first and demosaicing later for reducing the noise. It has some advantages such as less noise-caused color artifacts and cost-effective implementation. When compared to Joint Demosaicing and denoising, PCA Based spatially adaptive denoising of CFA Images is more effective in reducing the noise in terms of both objective measurement and visual evaluation. This method effectively suppresses noise while preserving color edges and details. The direct CFA image denoising scheme, followed by a succeeding demosaicing scheme reduces extensively the noise-caused color artifacts in the demosaiced images. Such artifacts often appear in the output full-color images of many demosaicing first and denoising later schemes, as well as joint demosaicing-denoising schemes. The CPSNR ratio is much better, when compared to Joint demosaicing and denoising schemes by preserving the structures in the image which are often smoothed by other denoising schemes.

The advent of efficient short range radio communication coupled with advancement in miniaturization of computing devices has enabled the development of wireless multimedia sensor network (WMSN). Energy is one of the scarcest resources in such networks, especially it is scarce in transmitting multimedia data. This work presents an energy-saving image compression scheme. The nodes in WMSN are equipped with Colour Filter Array (CFA) using Bayer Pattern. Considering alternate column dropping of pixel array of the captured image is one of the viable techniques for low-overhead image comprssion, Bayer CFA lacks in features by means of which image pixel may be reconstructed using its neighbouring pixel information. So primarily we design a new CFA (NCFA) to overcome the said problems of Bayer CFA. Subsequently, the compression scheme is proposed where the alternate columns of the pixel array of each of the macroblocks of an image captured through NCFA, are dropped. At the high-end receiving node (e.g. sink), image is reconstructed from the sequentially transmitted macroblocks. Performance of the scheme is evaluated and compared through qualitative and quantitative analyses. The results show the present scheme's dominance over all the variants of the competing scheme in terms of both energy-efficiency and reconstruction quality.

In CFA 2.0, there are white pixels in a color filter array (CFA) that has proven to help the demosaicing performance for images collected in low light conditions. Here, we evaluate the performance of demosaicing for images collected in low light conditions using an RGBW pattern with 75% white pixels. We term this CFA the CFA 3.0. Using a data set containing 10 images collected in low light conditions, we performed extensive experiments. Both objective and subjective evaluations were used in our experiments.

This paper presents a new weighted interpolation technique to address the Bayer Color Filter Array (CFA) demo-saicking problem. The proposed technique provides two contributions that have not been reported in conventional interpolation methods. Firstly, it exploits the quincuncial nature of the green component of the Bayer CFA. The second contribution treats each of the three planes or lattices of the CFA mosaic as distinct and is weighted uniquely. The proposed technique is compared with other interpolation-based demosaicking algorithms and a significant improvement in performance has been noted through experimental results. In addition, to provide objectivity in analysis, three test measures have been used — CPSNR, CIELAB and CIEDE2000.

This paper describes a demosaicking method for hexagonally-structured color filter array. Demosaicking is essential to acquire color images using color filter array (CFA) in single sensor imaging. Thus, CFA patterns have been discussed in order to improve image quality in single sensor imaging after the Bayer pattern are introduced. Advancements in imaging sensor technology recently introduce a hexagonal CFA pattern. The hexagonal CFA can be considered to be a 45-degree rotational version of the Bayer pattern, thus demosaicking can be implemented by an existing method with backward and forward 45-degree rotations. However, this approach requires heavy computing power and memory in image sensing devices because of the image rotations. To overcome this problem, we proposes a demosaicking method for a hexagonal Bayer CFA without rotations. In addition, we introduce a weighting parameter in our demosaicking method to improve image quality and to unifying exiting method with our method. Experimental results indicate that the proposed method is superior to conventional methods in terms of PSNR. In addition, some optimized values for the weighting parameter are provided experimentally.

Most consumer digital cameras capture color information with a single light sensor and a color filter array (CFA). In these cameras, only one color sample (red, green or blue) is captured at each pixel location. This paper presents a modified H.264 intra prediction scheme for compressing image and video data captured with a color filter array. The H.264 intra prediction modes are modified for the green channel to account for the fact that the green data is not sampled in a rectangular manner in the Bayer pattern, the most popular CFA design. The proposed method increases the compression efficiency of I frames on the green channel by up to 1.2 dB.

This paper proposes two-layer color filter array to obtain a high quality image and demosaic algorithms to interpolate the proposed two-layer color filter array. Although demosaic and color filter array design methods have been improved together, recent research results have shown the performance saturation since the sub-sampling of color channels is inevitable. To overcome this limitation, multilayer color filter arrays have been developed to obtain two or three color data at single pixel position. This paper proposes one of two-layer color filter array, which has a full resolution in G channel and a half resolution in R/B channel. To evaluate the performance, simple demosaic algorithms are presented. Although simple demosaic algorithms are applied, the experimental results show that the proposed color filter array not only increases CPSNR dramatically but also decreases the artifacts as like false color, aliasing.

Metasurface color filters (MCFs) have attracted considerable attention thanks to their compactness and functionality as a candidate of an optical element in a miniaturized image sensor. However, conventional dielectric and plasmonic MCFs that have focused on color purity and efficiency cannot avoid reflection in principle, which degrades image quality by optical flare. Here, we introduce absorptive-type MCFs through truncated-cone hyperbolic metamaterial absorbers. By applying a particle swarm optimization method to design multiple parameters simultaneously, the proposed MCF is theoretically and numerically demonstrated in perceptive color on CIELAB and CIEDE2000 with suppressed-reflection. Then, a color filter array is numerically proven in 255 nm of sub-pixel pitch.

Most digital cameras use a single sensor coupled with a Color Filter Array (CFA) to capture images, and apply demosaicking to interpolate the full color images. In reality, the CFA image is noisy, which causes problems in the demosaicking process. This paper proposes a Joint Denoising and Demo-saicking based on inter-Color correlation (JDDC) scheme. We propose a new framework that linearly combines an extracted luminance image and a low-passed RGB images to get a full color image. Given the noise in the extracted luminance image and the low-passed RGB images are non-stationary and partially correlated, we modify the classical Non-Local Means (NLM) filter to denoise the extracted luminance image and the low-passed RGB images before the combination. Experimental results verify the effectiveness of the proposed scheme both objectively and subjectively.