USNet: underwater image superpixel segmentation via multi-scale water-net

In underwater scenes, the quality of the video and image acquired by the underwater imaging system suffers from severe degradation, influencing target detection and recognition. Thus, restoring real scenes from blurred videos and images is of great significance. Owing to the light absorption and scattering by suspended particles, the images acquired often have poor visibility, including color shift, low contrast, noise, and blurring issues. This paper aims to classify and compare some of the significant technologies in underwater image defogging, presenting a comprehensive picture of the current research landscape for researchers. First we analyze the reasons for degradation of underwater images and the underwater optical imaging model. Then we classify the underwater image defogging technologies into three categories, including image restoration approaches, image enhancement approaches, and deep learning approaches. Afterward, we present the objective evaluation metrics and analyze the state-of-the-art approaches. Finally, we summarize the shortcomings of the defogging approaches for underwater images and propose seven research directions.

Underwater optical imaging is essential for exploring the underwater environment to provide information for planning and regulating underwater activities in various underwater applications, such as aquaculture farm observation, underwater topographical survey, and underwater infrastructure monitoring. Thus, there is a need to investigate the underwater imaging process and propose clear and long-range underwater optical imaging methods to fulfill the demands of academia and industry. In this manuscript, we classify the eighteen most commonly used underwater optical imaging methods into two groups regarding the imaging principle, (1) hardware and (2) software-based methods, each with an explanation of the theory, features, and applications. Furthermore, we also discuss the current challenges and future directions for improving the performance of current methods, such as improving the accuracy of underwater image formation model estimation, enlarging the underwater image dataset, proposing comprehensive underwater imaging evaluation metrics, estimating underwater depth and integrating different methods (e.g., hardware- and software-based methods for computational imaging) to promote the imaging performance not only in the laboratory but also in practical underwater scenarios.

Due to the absorption and the scattering effects of light propagation, underwater images taken by optical devices are often characterized by color casts, low contrast, blurry details, and poor brightness. Current image enhancement methods are difficult to simultaneously deal with multiple degradation problems due to the lack of consideration of the underwater imaging mechanism. To overcome this limitation, this paper proposes an underwater image restoration method via adaptive color balance and multi-purpose oriented contrast enhancement. Concretely, we select a reference channel relying on the highest mean value and simple stretching and use the gray-world assumption to design two gain factors of color compensation, achieving the balanced color distribution of the R, G, and B channels. Then, we fuse multiple priors of backscatter light to build an adaptive standard deviation of a Gaussian filter, separating the pure color veil, which enables us to effectively estimate an accurate backscatter light from different underwater scenes. Finally, we design a multi-purpose-oriented contrast enhancement strategy containing to improve the overall visibility, which takes into account the distribution characteristics of different degradation issues. Extensive experiments on two underwater image datasets show that our method is effective and robust for improving the qualitative and quantitative results of underwater images, i.e.,compared with the second-best comparative method, the average e and r¯ values of our method at least increase by 1.03% and 3.78%, which implies that our method can significantly enhance the contrast, the colors, and the details. Meanwhile, our method produces good generalization ability for restoring complex nighttime images.

A multi-angle image enhancement method based on underwater imaging was proposed to solve the problems of color bias, geometric distortion and low contrast of the observed object in the complex and changeable underwater environment. Firstly, the gray world algorithm is used to preprocess the underwater image. Then an image optimization denoising algorithm based on K-SVD dictionary learning is used to keep the structure and texture information of the original image. Finally, combining with the underwater optical imaging model, the blue-green channel enhancement algorithm is used to realize the underwater imaging defog processing, which makes the restored image closer to the reality. The results show that this method can solve the problems of noise interference, fuzzy underwater image and low contrast.

Superpixel segmentation can be defined as an image partition into connected regions, such that image objects may be represented by the union of their superpixels. In this context, multiple iterations of superpixel segmentation from improved seed sets is a strategy exploited by several algorithms. The Iterative Spanning Forest (ISF) framework divides this strategy into three independent components: a seed sampling method, a superpixel delineation algorithm based on strength of connectedness between seeds and pixels, and a seed recomputation procedure. A recent work shows that object information can be added to each component of ISF such that the user can control the number of seeds inside the objects and so improve superpixel segmentation. However, it is uncertain how the added information impacts each component of the pipeline. Therefore, in this work, a study is conducted to evaluate such inclusion in the seed sampling procedure, partially elucidating its benefits. Additionally, we introduce a novel object-based sampling approach, named Object Saliency Map sampling by Ordered Extraction (OSMOX), and demonstrate the results for supervised and unsupervised object information. The experiments show considerable improvements in under-segmentation error, specially with a low number of superpixels.

Superpixel segmentation targets at grouping pixels in an image into atomic regions whose boundaries align well with the natural object boundaries. This paper first proposes a new feature representation for superpixel segmentation that holistically embraces color, contour, texture, and spatial features. Then, we introduce a clustering-based discriminability measure to iteratively evaluate the importance of different features. Integrating the feature representation and the discriminability measure, we propose a novel content-adaptive superpixel (CAS) segmentation algorithm. CAS is able to automatically and iteratively adjust the weights of different features to fit various properties of image instances. Experiments on several challenging datasets demonstrate that the proposed CAS outperforms the state-of-the-art methods and has a low computational cost.

Captured underwater images usually suffer from severe color cast and low contrast due to wavelength-dependent light absorption and scattering. These degradation issues affect the accuracy of target detection and visual understanding. The underwater image formation model is widely used to improve the visual quality of underwater images. Accurate transmission map and background light estimation are the keys to obtaining clear images. We develop a multi-priors underwater image restoration method with adaptive transmission (MUAT). Concretely, we first propose a calculation method of the dominant channel transmission to cope with pixel interference, which combines two priors of the difference between atmospheric light and pixel values and the difference between the red channel and the blue-green channel. Besides, the attenuation ratio between the superior and inferior channels is adaptively calculated with the background light to solve the color distortion and detail blur caused by the imaging distance. Ultimately, the global white balance method is introduced to solve the color distortion. Experiments on several underwater scene images show that our method obtains accurate transmission and yields better visual results than state-of-the-art methods.

Underwater images suffer from blurring effects, low contrast, and grayed out colors due to the absorption and scattering effects under the water. Many image enhancement algorithms for improving the visual quality of underwater images have been developed. Unfortunately, no well-accepted objective measure exists that can evaluate the quality of underwater images similar to human perception. Predominant underwater image processing algorithms use either a subjective evaluation, which is time consuming and biased, or a generic image quality measure, which fails to consider the properties of underwater images. To address this problem, a new nonreference underwater image quality measure (UIQM) is presented in this paper. The UIQM comprises three underwater image attribute measures: the underwater image colorfulness measure (UICM), the underwater image sharpness measure (UISM), and the underwater image contrast measure (UIConM). Each attribute is selected for evaluating one aspect of the underwater image degradation, and each presented attribute measure is inspired by the properties of human visual systems (HVSs). The experimental results demonstrate that the measures effectively evaluate the underwater image quality in accordance with the human perceptions. These measures are also used on the AirAsia 8501 wreckage images to show their importance in practical applications.

The complexity and challenging underwater environment leading to degradation in underwater image. Measuring the quality of underwater image is a significant step for the subsequent image processing step. Existing Image Quality Assessment (IQA) methods do not fully consider the characteristics of degradation in underwater images, which limits their performance in underwater image assessment. To address this problem, an Underwater IQA (UIQA) method based on color space multi-feature fusion is proposed to focus on underwater image. The proposed method converts underwater images from RGB color space to CIELab color space, which has a higher correlation to human subjective perception of underwater visual quality. The proposed method extract histogram features, morphological features, and moment statistics from luminance and color components and concatenate the features to obtain fusion features to better quantify the degradation in underwater image quality. After features extraction, support vector regression(SVR) is employed to learn the relationship between fusion features and image quality scores, and gain the quality prediction model. Experimental results on the SAUD dataset and UIED dataset show that our proposed method can perform well in underwater image quality assessment. The performance comparisons on LIVE dataset, TID2013 dataset,LIVEMD dataset,LIVEC dataset and SIQAD dataset demonstrate the applicability of the proposed method.

Underwater images often suffer from low contrast, color distortion, and noise due to light absorption and scattering in water. These issues make it challenging for marine applications such as underwater exploration, object detection, and autonomous navigation. Traditional image enhancement techniques struggle to recover natural colors and details effectively. This research proposes a U-Net-based Generative Adversarial Network (U-Net GAN) to enhance underwater images, focusing on preserving structural details while improving visual quality. The U-Net architecture ensures effective feature extraction and segmentation, while the GAN framework enhances image realism by learning high-level transformations. The model is trained and tested on benchmark datasets such as UIEB and EUVP, with performance evaluated using PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), and UIQM (Underwater Image Quality Measure). Experimental results demonstrate that the proposed approach significantly enhances underwater images compared to conventional methods, restoring natural colors and improving visibility. This study contributes to advancing deep learning techniques for real-time underwater image enhancement, benefiting various marine research and surveillance applications.

The paper explores the problems of restoring underwater images exposed to distortions in the form of color and contrast deformations, the presence of haze, etc., arising from the interaction of optical radiation with the aquatic environment. Restoring underwater images is a non-trivial task due to the large variability of the parameters of the aquatic environment and photography conditions. The proposed method, unlike other underwater image recovery algorithms based on an imaging model, is not based on a simplified exponential Beer-Lambert law for estimating optical radiation attenuation in water, but on a more accurate physical approach that simulates the propagation of optical rays in water using the Monte Carlo method, taking into account the main parameters the water environment and the camera. The results of numerical simulation of optical ray propagation in an aquatic environment are used for image processing in the spatial domain by editing the histograms of each image channel in the RGB color space. To test the developed algorithm, 6 real underwater images were selected obtained under various lighting conditions (natural and artificial) and various parameters of the aquatic environment (clear ocean and turbid coastal water). For the purpose of qualitative and quantitative analysis of the obtained results, the following similar underwater image processing methods were used: Fusion, UDCP IATP, Retinex, HE, and UWB VCSE. The Underwater Colour Image Quality Evaluation Metric (UCIQE) and Underwater Image Quality Measure (UIQM) indicators were used to quantify the results obtained. The results of the qualitative assessment demonstrate the high efficiency of the proposed method: regardless of the conditions of the initial image parameters, the application of the developed method improves visual perception and does not lead to excessive contrast enhancement, color distortion, loss of detail, the appearance of artifacts, etc. Quantification of underwater image processing results demonstrates comparable and superior results when comparing the efficiency of the algorithm with similar methods. For the UCIQE parameter, the developed method provided an improvement from 9 % to 51 % relative to the parameter value for the original image, while similar methods demonstrated results from minus 10 % to 82 %. For the UIQM parameter, the developed method provided an improvement from 24 % to 99 % relative to the parameter value for the original image, while similar methods demonstrated results from minus 10 % to 123 %. Unlike analogues, the developed method did not demonstrate the worst values of the UCIQE and UIQM parameters for any processed image, which indicates the stability of the method regardless of the parameters of the aquatic environment and shooting conditions. By dividing the developed method into preliminary and main stages, high image processing speed is ensured: 0.073 seconds for images with a resolution of 400 × 300 pixels and from 8.02 to 8.23 seconds for images with a resolution of 5184 × 3456 pixels. Similar methods demonstrated values from 0.19 to 10.81 seconds for an image with a resolution of 400 × 300 pixels and from 7.65 to 937.83 seconds for an image with a resolution of 5184 × 3456 pixels. The introduction of the proposed method into the geological exploration will increase their efficiency and reliability, and will provide more accurate data for further exploration of solid mineral deposits. Such technique integrated into the machine vision system of underwater vehicles will significantly expand their functionality by enabling automation of operations and improving the efficiency of recognition systems.

The underwater color disparity is an important cue for enhancing an underwater image. Applying the underwater color disparity indiscriminately to the entire underwater image tends to give rise to foreground-background crosstalk with either excessive foreground or insufficient background enhancement. To address the discriminativeness between underwater color disparities in foreground and background regions, we develop a discriminative underwater image enhancement method empowered by large foundation model technology. We first utilize the Segment Anything Model to generate segmentation masks, dividing the underwater image into foreground and background regions. This enables accurate foreground-background separation. Then, we conduct adaptive color compensation and fusion to improve the color histogram similarity for foreground and background regions separately. This corrects color deviations and improves contrasts in a discriminative manner that avoids the foreground-background crosstalk. Finally, we propose high-frequency edge fusion to extract high-frequency components from both the original underwater image and the fused image, and then fuse these components to obtain the final enhanced image. This eliminates blurred details arising from the discriminative processing of foreground and background regions. Our method represents the pioneering application of large foundation model technology to empower underwater image enhancement. Experimental results indicate that our method outperforms nine state-of-the-art underwater image enhancement methods in visual quality, achieves superior results across five underwater image quality evaluation metrics on three underwater image datasets, and is beneficial for practical applications such as underwater feature matching. We release our code at https://gitee.com/wanghaoupc/UIE SAM.

Owing to absorption and scattering of light in water, raw underwater images suffer from low contrast, blurred details, and color casts. These characteristics significantly interfere the visibility of underwater images. In this paper, we propose a new robust adversarial learning framework via physics model based feedback control and domain adaptive mechanism for underwater image enhancement. A new method for simulating underwater-like training dataset by underwater image formation model is proposed. Upon this, a novel framework, which introduces a domain adaptive mechanism and a physics model constraint feedback control, is trained to enhance the underwater scenes. Extensive experiments demonstrate the significant superiority of the proposed framework compared with the state-of-the-art methods.

Underwater image enhancement based on conditional generative adversarial network

To address the issues of color distortion, low contrast and detail loss in underwater images, a network combining Transformer and wavelet transform is proposed for underwater image enhancement, named U-WaveFormer, which includes the convolutional Transformer fusion module (CTFM), the SelfAttention block, the WaveConv feature interaction module (WFIM) and the texture boost module. The CTFM integrates the strengths of Transformer for capturing global information and CNN for extracting local features, effectively balancing both global and local information. The Self Attention block leverages adaptive parameters for dynamic weight adjustment, focusing on severely degraded regions. The WFIM adopts a dual-branch architecture combining discrete wavelet transform and CNN, processing high-frequency and low-frequency components separately to address the limitations of existing methods that often neglect frequency-domain information. Furthermore, the Texture Boost Module utilizes multi-scale feature extraction and fusion to resolve texture detail loss during enhancement. Quantitative and qualitative results on the UIEB and LUSI underwater image datasets demonstrate the superior performance of U-WaveFormer in both subjective perception and evaluation metrics.