An automatic wavelet-based nonlinear image enhancement technique for aerial imagery

In real-world environments, the human visual system perceives a wide range of luminance in a scene. However, the full range of tones in a high dynamic range (HDR) scene cannot be displayed on standard display devices due to their low dynamic range (LDR). Therefore, tone mapping is necessary to faithfully display a HDR scene on an LDR display device. Existing tone mapping methods have problems because details and contrast in a scene are not preserved faithfully, and they also distort the colors in a scene. Thus, we propose a tone mapping method for preserving the detail in an HDR scene using a weighted least squares filter, which preserves the global contrast in a scene by using a competitive learning neural network, before applying a tone reproduction operator to preserve the color without shifting the lightness. According to the Helmholtz–Kohlrausch effect, the perception of brightness depends on the lightness, chroma, and hue of a color. For example, the luminance of pixels with specific colors such as red and blue is low in an HDR scene. The proposed method corrects the lightness of pixels according to the color (i.e., lightness, chroma, and hue) of a tone-mapped image. Experimental results with several test sets of images demonstrated that the proposed tone mapping method with contrast preservation and lightness correction is more suitable for dynamic range compression than existing tone mapping methods, while it also preserves the color of a scene in an effective way.

Design of a systolic-pipelined architecture for real-time enhancement of color video stream based on an illuminance–reflectance model

An efficient real-time video stream enhancement algorithm based on illuminance-reflectance model is proposed for improving the visual quality of digital video streams captured under insufficient and/or non-uniform lighting conditions. The paper presents computational methods for estimation of scene illuminance and reflectance, adaptive dynamic range compression of illuminance, and adaptive enhancement for mid-tone frequency components. This algorithm is an effective and efficient technique for image enhancement with relatively simple computational procedures, which makes real-time enhancement of digital videos successfully realized. It also demonstrates strong robustness and high image quality when compared to other techniques.

Magnetic resonance imaging (MRI) plays a significant role in medical detection technologies but is difficult to avoid suffering from un-uniformity, low brightness and noise pollution. These factors might affect the doctor`s diagnosis and can lead to misdiagnosis in practical applications. In this paper, a new MRI images enhancement strategy is proposed which consists of a dynamic range compression (DRC) operation and a details enhancement (DE) operation. The DRC can efficiently compress the high dynamic range to a suitable range and protect the difference between the pixels, and DE adopts an improved bilateral filtering combined with Laplacian pyramid to remove the noise and enhance the details. A series of experiments results of real MRI images demonstrate that the method is more effective than other methods.

Recording video in high dynamic range (HDR) scenes usually suffers from the trouble of getting poor contrast videos. In this paper, an integrated signal processing system, which remaps the image to enhance video visibility, is presented. The system extracts pixel local contrasts and nonlinearly stretches the dynamic range of pixel contrast values through histogram equalization. Undesired artifacts in the contrast map are then smoothed by a similar filter, which averages the pixel luminance values only when colors of neighboring pixels are similar to the central one. The luminance value for a pixel is then replaced by the geometric mean of its original value and smoothed contrast value. Finally, the photographic tone reproduction algorithm normalizes pixel values such that the average of log luminance values in the entire video frame is around 18% gray tone. The proposed approach shows better performance in both tone and color reproduction compared with other popular approaches.

Images from high dynamic range (HDR) scenes must be obtained with minimum loss of information. For this purpose it is necessary to take full advantage of the quantification levels provided by the CCD/CMOS image sensor. LinLog CMOS sensors satisfy the above demand by offering an adjustable response curve that combines linear and logarithmic responses. This paper presents a novel method to quickly adjust the parameters that control the response curve of a LinLog CMOS image sensor. We propose to use an Adaptive Proportional-Integral-Derivative controller to adjust the exposure time of the sensor, together with control algorithms based on the saturation level and the entropy of the images. With this method the sensor’s maximum dynamic range (120 dB) can be used to acquire good quality images from HDR scenes with fast, automatic adaptation to scene conditions. Adaptation to a new scene is rapid, with a sensor response adjustment of less than eight frames when working in real time video mode. At least 67% of the scene entropy can be retained with this method.

Dynamic Range Compression (DRC) is a key component in all modern Hearing Aids. Attack and Release time constants decide the speed with which the DRC should act to the incoming signal amplitude variation. So an accurate estimation of time constants gives a precise control over the DRC behavior. In this paper we examined various errors that occur in the output of the DRC while using conventional methods which affect attack and release time constants adversely. New methods are proposed for a better estimation of time constants in DRC. Since all the modifications are made in the estimation of attack time, there is no need to change the existing hardware for DRC. The proposed algorithm gives perfect output characteristics with zero error for test signals defined in ANSI S3.22 standards for hearing aid specifications.

Medical endoscopic video processing requires real-time execution of color component acquisition, color filter array (CFA) demosaicing, and high dynamic range (HDR) compression under low-light conditions, while adhering to strict thermal constraints within the surgical handpiece. Traditional hardware-aware neural architecture search (NAS) relies on fixed hardware design spaces, making it difficult to balance accuracy, power consumption, and real-time performance. A collaborative “power-accuracy” optimization method is proposed for hardware-aware NAS. Firstly, we proposed a novel hardware modeling framework by abstracting FPGA heterogeneous resources into unified cell units and establishing a power–temperature closed-loop model to ensure that the handpiece surface temperature does not exceed clinical thresholds. In this framework, we constrained the interstage latency balance in pipelines to avoid routing congestion and frequency degradation caused by deep pipelines. Then, we optimized the NAS strategy by using pipeline blocks and combined with a hardware efficiency reward function. Finally, color component acquisition, CFA demosaicing, dynamic range compression, dynamic precision quantization, and streaming architecture are integrated into our framework. Experiments demonstrate that the proposed method achieves 2.8 W power consumption at 47 °C on a Xilinx ZCU102 platform, with a 54% improvement in throughput (vs. hardware-aware NAS), providing an engineer-ready lightweight network for medical edge devices such as endoscopes.

Dynamic range compression (DRC) by hexapeptide libraries increases MS/MS-based identification of lower-abundance proteins in complex mixtures. However, two unanswered questions impede fully realizing DRC's potential in shotgun proteomics. First, does DRC enhance identification of post-translationally modified proteins? Second, can DRC be incorporated into a workflow enabling relative protein abundance profiling? We sought to answer both questions analyzing human whole saliva. Addressing question one, we coupled DRC with covalent glycopeptide enrichment and MS/MS. With DRC we identified ∼2 times more N-linked glycoproteins and their glycosylation sites than without DRC, dramatically increasing the known salivary glycoprotein catalog. Addressing question two, we compared differentially stable isotope-labeled saliva samples pooled from healthy and metastatic breast cancer women using a multidimensional peptide fractionation-based workflow, analyzing in parallel one sample portion with DRC and one portion without. Our workflow categorizes proteins with higher absolute abundance, whose relative abundance ratios are altered by DRC, from proteins of lower absolute abundance detected only after DRC. Within each of these salivary protein categories, we identified novel abundance changes putatively associated with breast cancer, demonstrating feasibility and benefits of DRC for relative abundance profiling. Collectively, our results bring us closer to realizing the full potential of DRC for proteomic studies.

Image enhancement is a crucial pre-processing step for various image processing applications and vision systems. Many enhancement algorithms have been proposed based on different sets of criteria. However, a direct multi-scale image enhancement algorithm capable of independently and/or simultaneously providing adequate contrast enhancement, tonal rendition, dynamic range compression, and accurate edge preservation in a controlled manner has yet to be produced. In this paper, a multi-scale image enhancement algorithm based on a new parametric contrast measure is presented. The parametric contrast measure incorporates not only the luminance masking characteristic, but also the contrast masking characteristic of the human visual system. The formulation of the contrast measure can be adapted for any multi-resolution decomposition scheme in order to yield new human visual system-inspired multi-scale transforms. In this article, it is exemplified using the Laplacian pyramid, discrete wavelet transform, stationary wavelet transform, and dual-tree complex wavelet transform. Consequently, the proposed enhancement procedure is developed. The advantages of the proposed method include: 1) the integration of both the luminance and contrast masking phenomena; 2) the extension of non-linear mapping schemes to human visual system inspired multi-scale contrast coefficients; 3) the extension of human visual system-based image enhancement approaches to the stationary and dual-tree complex wavelet transforms, and a direct means of; 4) adjusting overall brightness; and 5) achieving dynamic range compression for image enhancement within a direct multi-scale enhancement framework. Experimental results demonstrate the ability of the proposed algorithm to achieve simultaneous local and global enhancements.

This paper presents a new approach for effectively compressing the dynamic range of a visual scene while preserving the local contrast in order to reproduce a high quality image on devices with limited dynamic range. In this method, dynamic range compression is performed through a reduction of the illumination effect, by taking into account the characteristics of the human visual system. To precisely extract the illumination component within a reasonable complexity, a multi-scale nonlinear filtering scheme, referred to here as nonlinear multi-scale retinex (NMR), was developed.

A method is proposed to obtain accurate disparity maps for high dynamic range (HDR) scenes using stereo image pairs acquired under different exposure times and viewpoints. In HDR scenes, saturated pixels such as too‐dark or too‐bright pixels are inevitable because of the limited bit‐depth representations of images. These saturated regions include little information for matching images and harm the performance of conventional stereo matching methods. To address this problem, a method is proposed that generates a new cost volume by fusing cost volumes obtained using different stereo image pairs taken under different exposure times as well as different points of view. In addition, the directional semi‐global matching method is proposed to efficiently optimise the cost volume. Experimental results show that the proposed method generates more accurate results for saturated regions than existing methods on various datasets.

Under-Display Camera (UDC) has been widely exploited to help smartphones realize full-screen displays. However, as the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. Particularly, flare and blur in UDC images could severely deteriorate the user experience in high dynamic range (HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with an estimated PSF in HDR scenes. Our network consists of three parts, including a U-shape base network to utilize multi-scale information, a condition branch to perform spatially variant modulation, and a kernel branch to leverage the prior knowledge of the PSF. According to the characteristics of HDR data, we additionally design a tone mapping loss to stabilize network optimization and achieve better visual quality. Experimental results show that the proposed UDC-UNet outperforms the state-of-the-art methods in quantitative and qualitative comparisons. Our approach won second place in the UDC image restoration track of the MIPI challenge. Codes and models are available at https://github.com/J-FHu/UDCUNet.KeywordsUnder display cameraImage restoration

High dynamic range (HDR) scenes are known to be challenging for most cameras. The most common artifacts associated with bad HDR scene rendition are clipped bright areas and noisy dark regions, rendering the images unnatural and unpleasing. This paper introduces a novel methodology for automating the perceptual evaluation of detail rendition in these extreme regions of the histogram for images that portray natural scenes. The key contributions include 1) the construction of a robust database in Just Objectionable Distance (JOD) scores, incorporating annotator outlier detection 2) the introduction of a Multitask Convolutional Neural Network (CNN) model that effectively addresses the diverse context and region-of-interest challenges inherent in natural scenes. Our experimental evaluation demonstrates that our approach strongly aligns with human evaluations. The adaptability of our model positions it as a valuable tool for ensuring consistent camera performance evaluation, contributing to the continuous evolution of smartphone technologies.

Accurate estimation of decay coefficients for dynamic range compressors in hearing aids and a hardware level comparison of different architectures