Compact underwater single-photon imaging lidar.

For the demand for long-range and high-resolution target reconstruction of slow-moving small underwater targets, research on single-photon lidar target reconstruction technology is being carried out. This paper reports the sequential multimodal underwater single-photon lidar adaptive target reconstruction algorithm based on spatiotemporal sequence fusion, which has strong information extraction and noise filtering ability and can reconstruct the target depth and reflective intensity information from complex echo photon time counts and spatial pixel relationships. The method consists of three steps: data preprocessing, sequence-optimized extreme value inference filtering, and collaborative variation strategy for image optimization to achieve high-quality target reconstruction in complex underwater environments. Simulation and test results show that the target reconstruction method outperforms the current imaging algorithms, and the built single-photon lidar system achieves underwater lateral and distance resolution of 5 mm and 2.5cm@6AL, respectively. This indicates that the method has a great advantage in sparse photon counting imaging and possesses the capability of underwater target imaging under the background of strong light noise. It also provides a good solution for underwater target imaging of small slow-moving targets with long-distance and high-resolution.

Lidar has emerged as an important technology for the high-precision three-dimensional remote sensing of the ocean. While oceanic lidar has been widely deployed on various platforms, its underwater deployment is relatively limited, despite its significance in deep-sea exploration and obstacle avoidance for underwater platforms. Underwater lidar systems must meet stringent requirements for high performance, miniaturization, and high integration. Single-photon lidar, by elevating the detection sensitivity to the single-photon level, enables high-performance detection under the condition of a low-pulse-energy laser and a small-aperture telescope, making it a stronger candidate for underwater lidar applications. However, this imposes demanding requirements for the data acquisition system utilized in single-photon lidar systems. In this work, a self-developed multi-channel acquisition system (MCAS) with a high-resolution and real-time histogram statistics capability was developed. By utilizing field-programmable gate array (FPGA) technology, a method that combines coarse counters with multi-phase clock interpolation achieved an impressive resolution of 0.5 ns and enabled a time of flight duration of 1.5 μs. To address counting instability, a dual-counter structure was adopted in the coarse counter, and real-time histogram statistics were achieved in the data acquisition system through a state machine. Furthermore, the non-uniform phase shift of the clock was analyzed, and a correction algorithm based on code density statistics was proposed to mitigate the periodic modulation of the backscattered signal, with the effectiveness of the algorithm demonstrated through experimental results. The robustness and stability of the MCAS were validated through an underwater experiment. Ultimately, the development of this compact acquisition system enables the implementation of underwater single-photon lidar systems, which will play a crucial role in underwater target imaging, obstacle avoidance in underwater platforms, and deep-sea marine environment monitoring.

Single-photon lidar has emerged as a strong technology for bathymetric measurements. However, its heightened sensitivity additionally makes it susceptible to solar radiation noise, particularly in the green light wavelength where solar radiation is strong, posing challenges for its daytime operation. To address this issue, a single-photon underwater lidar system is proposed and demonstrated. This scheme has these features. 1) Underwater applications not only mitigate the impact of the air-water interface on laser transmission but also significantly attenuate solar radiation reaching the lidar due to the absorption and scattering properties of water. 2) The telescope is designed with a small aperture and narrow field of view to significantly suppress solar radiation. 3) A combination of a narrowband laser and narrowband filter technique is effectively employed to minimize residual solar radiation, thus enabling continuous bathymetric observation capabilities during both day and night. 4) After acquiring the backscattered signal from the bottom, a water depth extraction algorithm utilizing bi-Gaussian fitting is proposed. To demonstrate the robustness of the lidar and the effectiveness of the algorithm, the underwater single-photon lidar system is deployed on a ship to conduct cruise surveys of two bays in the nearshore area, as well as a full-day stationary observation experiment. The lidar measurements are highly consistent with the synchronized sonar observations. The full-day stationary observation experiment showcased its capability to deliver continuous measurements throughout the day and night. These results demonstrate the potential of the system in various applications, including high-precision underwater terrain mapping, obstacle avoidance for underwater platforms, and underwater target imaging.

Underwater imaging plays an important role in sonar detection. Compressive sensing can achieve high-precision airborne sound source localization with a small computational effort, but underwater target localization and ranging have not been fully achieved. This study proposes a metamaterial-based compressive sensing imaging method for underwater sound source localization and target imaging. To overcome the poor wave manipulation performance caused by the impedance difference between water and the structure, a strong coupling interaction between the flexible structure and the medium is introduced to reduce reflection and scattering, dramatically improving the acoustic signal transmission amplitude via the resonance mechanism. Analytical and numerical results show that this device can achieve directional irrelevant modulation of incident acoustic waves in different directions. Besides, the multi-information fusion compressive sensing method and dual-sensor joint compressive sensing method can reconstruct the sound source information from the received mixed signals. Experimental results show that the metamaterial-based compressive sensing imaging device realizes the perception and localization of a single sound source underwater in multiple directions. The metamaterial-based hydroacoustic imaging device eliminates the dependence on large-caliber arrays and complex control systems, thus opening the way to establish a sonar system with low cost and high precision.

We analyze the practical limits of a lidar system based on the use of a laser diode, random binary continuous wave power modulation, and an avalanche photodiode (APD)-based photereceiver, combined with the control and computing power of the digital signal processors (DSP) currently available. The target is to design a compact portable lidar system made all in semiconductor technology, with a low-power demand and an easy configuration of the system, allowing change in some of its features through software. Unlike many prior works, we emphasize the use of APDs instead of photomultiplier tubes to detect the return signal and the application of the system to measure not only hard targets, but also medium-range aerosols and clouds. We have developed an experimental prototype to evaluate the behavior of the system under different environmental conditions. Experimental results provided by the prototype are presented and discussed.

Objective: We propose a deep-learning-based underwater target detection system that can effectively solve the problem of underwater optical image target detection and recognition. Methods: In this paper, based on the depth of the underwater optical image target detection and recognition and using a learning model, we put forward corresponding solutions using the concept of style migration solutions, such as training samples. A lack of variability and poor generalization of practical applications presents a challenge for underwater object identification. The UW_YOLOv3 lightweight model was proposed to solve the problems of calculating energy consumption and storage resource limitations in underwater application scenarios. The detection and recognition module, based on deep learning, can deal with the degradation process of underwater imaging by embedding an image enhancement module into the detection and recognition module for the joint tuning and transferring of knowledge. Results: The detection accuracy of the UW_YOLOv3 model designed in this paper outperformed the lightweight algorithm YOLOV3-TINY by 7.9% at the same image scale input. Compared with other large algorithms, the detection accuracy was lower, but the detection speed was much higher. Compared with the SSD algorithm, the detection accuracy was only 4.7 lower; the speed was 40.9 FPS higher; and the rate was nearly 16 times higher than Faster R-CNN. When the input scale was 224, although part of the accuracy was lost, the detection speed doubled, reaching 156.9 FPS. Conclusion: Based on our framework, the problem of underwater optical image target detection and recognition can be effectively solved. Relevant studies have not only enriched the theory of target detection and glory, but have also provided optical glasses with a clear vision for appropriate underwater application systems.

Traditional laser scanners for 3D distance measurement involve expensive, heavy, (potentially) slow rotating mirrors for light deflection of the scanning TOF (time of flight) distance measurement, not suitable for compact, robust and highly portable LIDAR system. On the other hand MEMS scanners are limited to small apertures not suitable for a precise TOF measurement. To overcome this problem Fraunhofer IPMS presents a large aperture 1D-MEMS scanner array especially designed for LIDAR applications. It is composed of 2 × 7 silicon mirror elements each having an identical design with comparatively large aperture of 2.51 × 9.51mm2 and ±30° optical scan range. All mirrors are driven electrostatically resonant with identical frequency close to design frequency of 250 Hz. By driving control all single scanner elements are synchronized to identical phase and amplitude in respect to a master scanner. This results in a large effective scanner aperture of 334 mm2 for the receiver optics and a filling factor of 80 %. To guarantee the synchronized operation the paper discusses in detail the scanner design to enable a sufficiently large frequency bandwidth of all scanner elements to the compensate frequency tolerances caused by fabrication and packaging. In comparison to LIDAR systems with conventional scanner components, the large aperture 1D-MEMS scanner array enables 3D-LIDAR systems to become significantly smaller, more robust and (potentially) less expensive, also higher scan rates can be realized without additional efforts (e.g. no air bearings are needed).

This paper describes an innovative, compact and eyesafe coherent lidar system developed for use in wind and wake vortex sensing applications. This advanced lidar system is field ruggedized with reduced size, weight, and power consumption (SWaP) configured based on an all-fiber and modular architecture. The all-fiber architecture is developed using a fiber seed laser that is coupled to uniquely configured fiber amplifier modules and associated photonic elements including an integrated 3D scanner. The scanner provides user programmable continuous 360 degree azimuth and 180 degree elevation scan angles. The system architecture eliminates free-space beam alignment issues and allows plug and play operation using graphical user interface software modules. Besides its all fiber architecture, the lidar system also provides pulsewidth agility to aid in improving range resolution. Operating at 1.54 microns and with a PRF of up to 20 KHz, the wind lidar is air cooled with overall dimensions of 30” x 46” x 60” and is designed as a Class 1 system. This lidar is capable of measuring wind velocities greater than 120 +/- 0.2 m/s over ranges greater than 10 km and with a range resolution of less than 15 m. This compact and modular system is anticipated to provide mobility, reliability, and ease of field deployment for wind and wake vortex measurements. The current lidar architecture is amenable for trace gas sensing and as such it is being evolved for airborne and space based platforms. In this paper, the key features of wind lidar instrumentation and its functionality are discussed followed by results of recent wind forecast measurements on a wind farm.

Underwater images suffer from severe color distortion, due to the wavelength-dependent light attenuation and scattering. Various underwater image enhancement methods have been developed to improve the quality of degraded underwater images. However, contemporary approaches often overlook the impact of different scene colors on the overall process, potentially leading to undesired outcomes, such as enhanced images exhibiting excessive redness. In this paper, we observe that the color tones of degraded underwater images exhibit variability under the influence of different underwater targets and scenes. Each degraded color channel can be utilized to guide the color correction of other channels. Given this, a light-weight underwater color correction network, dubbed UCCNet, is presented to alleviate the issue of color corruption. In UCCNet, three parallel branches are designed to excavate the residual information within each color channel, subsequently leveraging these features to improve the quality of underwater images. Moreover, facing the challenge of effectively enhancing underwater images in diverse and complex scenes, the model UCCNet-KT is established based on UCCNet. In UCCNet-KT, the technology of knowledge transfer is designed to improve the generalization ability by enriching the dataset and constructing the loss function. Extensive experiments on various underwater datasets indicate the impressive performance of the UCCNet and UCCNet-KT qualitatively and quantitatively.

Vortex acoustic waves (VAW) with helical phase wavefronts could provide a new dimension for the detection and imaging of targets in the ocean. To explore the propagation characteristics of VAW under the effect of ocean turbulence (OT) and the impact on imaging, the phase screen method is used to simulate the propagation of VAW generated by a phase‐modulated uniform circular array under the effect of OT and the imaging of the target. The phase change and detection probability under the influence of the acoustic wave parameters and OT parameters during the propagation process are studied. Second, the azimuth resolution performance and focussing performance of the target imaging are studied. Under the effect of OT, the VAW of different frequencies and topological modes become scattered, and the degree of scattering is different when the parameters are different; and the resolution performance and focussing performance have also deteriorated. However, the VAW still has a reliable resolution under the effect of medium and low‐intensity OT, which provides theoretical support for the practice of the VAW in underwater target imaging.

The underwater environment is more complex than that on land, resulting in severe static and dynamic blurring in underwater images, reducing the recognition accuracy of underwater targets and failing to meet the needs of underwater environment detection. Firstly, for the static blurring problem, we propose an adaptive color compensation algorithm and an improved MSR algorithm. Secondly, for the problem of dynamic blur, we adopt the Restormer network to eliminate the dynamic blur caused by the combined effects of camera shake, camera out-of-focus and relative motion displacement, etc. then, through qualitative analysis, quantitative analysis and underwater target detection on the enhanced dataset, the feasibility of our underwater enhancement method is verified. Finally, we propose a target recognition network suitable for the complex underwater environment. The local and global information is fused through the CCBC module and the ECLOU loss function to improve the positioning accuracy. The FasterNet module is introduced to reduce redundant computations and parameter counting. The experimental results show that the CFEC-YOLOv7 model and the underwater image enhancement method proposed by us exhibit excellent performance, can better adapt to the underwater target recognition task, and have a good application prospect.

Yuan, H.; Zhang, S.; Chen, G., and Yang, Y., 2020. Underwater image fish recognition technology based on transfer learning and image enhancement. In: Hu, C. and Cai, M. (eds.), Geo-informatics and Oceanography. Journal of Coastal Research, Special Issue No. 105, pp. 124–128. Coconut Creek (Florida), ISSN 0749-0208.To effectively identify fish targets in underwater images, an image processing technology based on secondary migration learning and the Retinex algorithm is proposed to solve the problem of few underwater data sets and unclear underwater images. This method only uses a small-scale underwater image data set to train the network, and the Faster region-based convolutional neural networks (R-CNN) model can quickly detect underwater fish targets. The first transfer learning is applied between the ultra–large-scale ImageNet open-source dataset and the medium-scale Open Images high-definition fish dataset. Then, the Retinex iterative algorithm is used to enhance the underwater image to apply the second transfer learning between the high-resolution medium-scale fish data set and the small-scale underwater data set. Experiments show that this method can train a very effective detection model using small underwater image data sets at a low cost. The effect and performance of the detection model are far superior to traditional machine learning methods. This research can provide a certain reference value for deep-sea exploration, resource protection, and other engineering applications.

The underwater visibility is very important in underwater vision research and target detection. However, most underwater vision systems cannot guarantee to possess the performance under complex water conditions. This is because underwater images are usually degraded by light-water interactions of absorption and scattering. The ambient light is scattered into the cameras line of sight by water molecules and suspended particles in the water medium, which adds a layer of haze to the image and reduces the contrast of the image. This part of scattered light is usually called background light, which is the main reason for underwater image degradation. In this paper, the formations of background light in underwater imaging under two different lighting conditions: natural illumination and artificial lighting, are analyzed by setting up physical models. The models developed include the parameters such as camera parameters, light source parameters, inherent optical properties, and camera-source-object geometry. Based on the models, the relationship between the background light and the above parameters is studied. Computer analysis shows that the global background light under two illumination conditions has a close relationship between the inherent optical properties of water medium and camera parameters. The global background light under natural illumination is proportional to the scattering coefficient and inversely proportional to the attenuation coefficient. The background light under the two illumination conditions both can be described in simple exponential falloff expressions of the global background light. The simple expression greatly reduces the computational complexity of simulations. The intensity of background light mainly depends on the inherent optical properties, camera-scene distance, camera-source distance and cameras imaging angle. The relationship between the global background light and the inherent optical properties can be used to estimate the attenuation coefficient, scattering coefficient and scene depth information. The result of this paper can be very useful for designing and improving the underwater imaging systems.

This paper reports on WaterGAN, a generative adversarial network (GAN) for\ngenerating realistic underwater images from in-air image and depth pairings in\nan unsupervised pipeline used for color correction of monocular underwater\nimages. Cameras onboard autonomous and remotely operated vehicles can capture\nhigh resolution images to map the seafloor, however, underwater image formation\nis subject to the complex process of light propagation through the water\ncolumn. The raw images retrieved are characteristically different than images\ntaken in air due to effects such as absorption and scattering, which cause\nattenuation of light at different rates for different wavelengths. While this\nphysical process is well described theoretically, the model depends on many\nparameters intrinsic to the water column as well as the objects in the scene.\nThese factors make recovery of these parameters difficult without simplifying\nassumptions or field calibration, hence, restoration of underwater images is a\nnon-trivial problem. Deep learning has demonstrated great success in modeling\ncomplex nonlinear systems but requires a large amount of training data, which\nis difficult to compile in deep sea environments. Using WaterGAN, we generate a\nlarge training dataset of paired imagery, both raw underwater and true color\nin-air, as well as depth data. This data serves as input to a novel end-to-end\nnetwork for color correction of monocular underwater images. Due to the\ndepth-dependent water column effects inherent to underwater environments, we\nshow that our end-to-end network implicitly learns a coarse depth estimate of\nthe underwater scene from monocular underwater images. Our proposed pipeline is\nvalidated with testing on real data collected from both a pure water tank and\nfrom underwater surveys in field testing. Source code is made publicly\navailable with sample datasets and pretrained models.\n

Underwater sonar images are crucial in areas like oceanographic research for mapping the seabed and detecting resources, and in marine biology for understanding habitats. They are also important for naval and military uses such as navigation and surveillance. However, due to equipment and environmental limitations, the number of image samples is restricted, impeding further data-driven AI research. Although some works have explored data augmentation of underwater sonar images, they still face the following two problems: 1) inability to generate small-target images; 2) limited diversity of generated images. Toward this end, in this paper we propose a small-target and diversity oriented underwater sonar image augmentation method. Specifically, for small-target images, we propose to first detect and extract the target objects in the seabed sonar images, then perform scale scaling, and fuse them onto the background image using the Poisson fusion algorithm; for diverse images, we ingeniously combine mainstream image generation methods, including GAN, VAE, and Diffusion Models, using the diversity of the generative models to ensure the diversity of the generated images. Meanwhile, we design a Mixture-of-Experts (MoE) enhanced discriminator in GAN to screen the images generated by the three generative models to ensure the quality of the final augmented images. Experimental results prove that our method can effectively increase the proportion of small-target images and ensure the diversity of the augmented images, which further boost related researches based on underwater sonar images.