Foveated ghost imaging based on deep learning

In the ghost imaging system based on the multidimensional vector matrix Walsh transform, the Walsh speckle pattern is used to continuously sample the moving target object, so there is a certain correlation and complementary detail information between multiple frames in the ghost imaging video. In our previous research, we broke through the inherent limitations of the digital micromirror device refresh rate on ghost imaging systems, allowing us to reconstruct more detailed frames. Therefore, by utilizing the correlation between these detailed frames in the time and space domains, we can improve the comfort of ghost imaging videos from the perspective of improving the single-frame quality of ghost imaging videos. To further improve multi-frame quality by utilizing more detailed frames in ghost imaging videos, this paper proposes a ghost imaging video algorithm based on the multidimensional vector matrix Walsh transform of bidirectional N-aligned fusion frames. Combining deep learning with computational ghost imaging, utilizing a bidirectional N-alignment algorithm and a deep learning neural network framework for target frame integration. The ghost imaging videos obtained from previous research can be improved in terms of noise, motion blur, and single-frame detail richness to enhance the imaging quality of the target frame. This paper constructs an encoding module and a corresponding feature fusion module suitable for multidimensional vector Walsh transform ghost imaging from the perspective of network width and feature multi-branch extraction based on GoogleNet Inception V3. A loss function suitable for four-dimensional vector matrix Walsh transform ghost imaging has been defined, which can better eliminate the noise and distortion caused by Walsh speckle sampling. After comparing the experimental results of moving objects, the results show that the algorithm proposed in this paper has significantly improved structural similarity, blur index, noise index, and other aspects compared to existing ghost imaging video optimization methods. The similarity angle of the NRSS structure has increased by 18.58% compared to the original reconstructed image, the blur parameter has increased by 31.9%, and the noise parameter has risen by 9.22%.

The underwater environment is extremely complex and variable, which makes it difficult for underwater robots detecting or recognizing surroundings using images acquired with cameras. Ghost imaging as a new imaging technique has attracted much attention due to its special physical properties and potential for imaging of objects in optically harsh or noisy environments. In this work, we experimentally study three categories of image reconstruction methods of ghost imaging for objects of different transmittance. For high-transmittance objects, the differential ghost imaging is more efficient than traditional ghost imaging. However, for low-transmittance objects, the reconstructed images using traditional ghost imaging and differential ghost imaging algorithms are both exceedingly blurred and cannot be improved by increasing the number of measurements. A compressive sensing method named augmented Lagrangian and alternating direction algorithm (TVAL3) is proposed to reduce the background noise imposed by the low-transmittance. Experimental results show that compressive ghost imaging can dramatically subtract the background noise and enhance the contrast of the image. The relationship between the quality of the reconstructed image and the complexity of object itself is also discussed.

Equipment requirement, quality of reconstructed image, and reconstruction time are important factors in the realization of thermal-light ghost imaging system. In this paper, we propose a new ghost imaging scheme with multiple speckle patterns, named multiple speckle patterns differential compressive ghost imaging scheme. In this scheme, the high temporal resolution requirements for detectors is reduced by continuously detecting multiple independent speckle patterns. We eliminate the background and other noises in ghost imaging system by using differential ghost imaging method. And the reconstruction time is effectively reduced simultaneously to improve the reconstructed image quality by introducing the compressive sensing techniques. Numerical results show that for the two-level grayscale “N” image, the mean square error, in using the proposed scheme with 8000 measurements, is reduced by 96.9%, and the peak signal to noise ratio has improved by 15.1 dB, in comparison with those using the original multiple speckle patterns ghost imaging scheme with 35000 measurements. For the eight-level grayscale “Pepper” image, the peak signal to noise ratio is enhanced by 11.4 dB. The proposed scheme also can decrease the requirements of detection equipment to improve the image quality, and reduce the reconstruction time. Therefore, it may have a broad application prospect.

Ghost imaging (GI) is thought of as a promising imaging method in many areas. However, the main drawback of GI is the huge measurement data and low signal-to-noise ratio. In this paper, we propose a novel mask-based denoising scheme to improve the reconstruction quality of GI. We first design a mask through the maximum between-class variance (OTSU) method and construct the measurement matrix with speckle patterns. Then, the correlated noise in GI can be effectively suppressed by employing the mask. From the simulation and experimental results, we can conclude that our method has the ability to improve the imaging quality compared with traditional GI method.

Influence of detector response speed on the contrast-to-noise ratio of reflective ghost imaging

We study the influence rules of the speckle size of a light source on ghost imaging, and propose a type of speckle pattern to improve the quality of ghost imaging. The results show that image quality will first increase and then decrease with the increase in speckle size, and there is an optimal speckle size for a specific object. At the same time, by using a random distribution of speckle positions, a type of displacement speckle pattern is designed, and the imaging quality is better than that of random speckle patterns. These results are of great significance for finding the best speckle patterns suitable for detecting targets, which further promotes practical applications of ghost imaging.

Ghost imaging is an indirect system that allows the imaging of an object without directly seeing the object. The speckle pattern that contains the information about light and objects has increasingly become a popular topic in pseudothermal light ghost imaging. However, existing research still has encountered problems of poor imaging quality and slow sampling speeds. We propose a ghost imaging method based on N-order speckle patterns to recover the object (NSGI). The N-order speckle patterns combine N independent laser speckles individually produced by passing an expanded and collimated He–Ne laser through a digital micromirror device (DMD). The sampling frequency can be improved by controlling the trigger signals of different DMDs. The results of the simulation and experiment have verified that our method can increase sampling speed and reconstruction accuracy. In addition, NSGI can be applied to more applications by designing multiple independent speckles with different properties.

Ghost imaging (GI) retrieves an image from the correlation between a sequence of illumination patterns on the object and their corresponding bucket detections. Traditionally, GI requires the precise information of the illumination patterns, which raises technology barriers on building a high-speed illumination source, limiting the scope of its application. In this study, we propose a high-speed GI system, which implements a self-correlation with a purely optical operation without determining illumination patterns. The light source is an optical phased array (OPA), built of a set of waveguide-type electro-optic phase modulators. The OPA is driven to randomly change the phases in every 200 ns, generating speckle patterns at a rate of 5 MHz. Although the speckle patterns are not predictable or post-determinable, with the help of the naked-eye GI scheme, the system in real time optically generates the images of the object at a frame rate of more than 1 kHz, which can be directly observed by eyes or using a camera. This method avoids acquiring the information of the illumination, thus providing a simple and easy way to realize high-speed GI. It also inspires a different way of applying OPAs to high-speed imaging.

We present a ghost handwritten digit recognition method for the unknown handwritten digits based on ghost imaging (GI) with deep neural network, where a few detection signals from the bucket detector, generated by the cosine transform speckle, are used as the characteristic information and the input of the designed deep neural network (DNN), and the output of the DNN is the classification. The results show that the proposed scheme has a higher recognition accuracy (as high as 98% for the simulations, and 91% for the experiments) with a smaller sampling ratio (say 12.76%). With the increase of the sampling ratio, the recognition accuracy is enhanced. Compared with the traditional recognition scheme using the same DNN structure, the proposed scheme has slightly better performance with a lower complexity and non-locality property. The proposed scheme provides a promising way for remote sensing.

Ghost imaging uses the high-order correlation of the light field and has the physical properties of entanglement, coherence, and nonlocality. With high performance of anti-interference and stability, ghost imaging has important value in theoretical research and practical application. This paper presents the application of ghost data for object detection and quality assessment. Due to the complexity of the real imaging environment, we generated a large number of synthetic images as original materials through a ghost imaging simulation systems. And then we proposed a detection network for ghost data based on convolutional neural network with several fusion strategies. At the same time, a novel non-reference image quality assessment criteria based on detection performance is designed. Experiments on ghost dataset demonstrated that the early fusion network performed better, and the quality evaluation criteria based on this network is simple, reliable, and has usability.

It is difficult to obtain a clear image in underwater turbulence environment with classical imaging methods due to the absorption, scattering, and underwater turbulence on the propagation beam. However, ghost imaging (GI), a non-locally imaging technique, has shown the turbulence-free ability in atmospheric turbulence by exploiting the second-order correlation between the signal beam and the reference beam. In this paper, we experimentally investigate the imaging quality of GI affected by the underwater environment, where the underwater environment is simulated by a 1 m × 0.4 m × 0.4 m tank with distilled water. The water temperature is controlled by a heater inside the tank, and a temperature gradient is obtained by putting the heater at different positions of the tank. The water vibration is produced by a heavy force, and the turbid medium is obtained by dissolving very small specks of CaCO3 in the water. A set of Hadamard speckle pattern pairs are generated and modulated on the incident beam, and then the beam illuminates on an unknown object after passing through the simulated underwater environment. With the second-order correlations, the image is reconstructed under different temperature gradients, water vibration, and turbid medium ratios. The results show that GI has the turbulence-free ability under lower temperature gradient, water vibration, and turbid media. The structural similarity image measurement (SSIM) values of the reconstructed images only start to decrease when the temperature gradient is greater than 4.0 °C. The same temperature gradient produced at the different positions has a little effect on the quality of the underwater GI.

The attenuation of backscattered light through human tissue makes it difficult to measure the interferometric signal with high accuracy. We present a new method that combines the originally developed Time Domain-OCT (TD-OCT) technique with Ghost Imaging (GI) technique to detect targets at a certain depth in the scattering medium. Our method's first result shows that sample images can be reconstructed through a homogeneous mixture of 0.5% milk solution (1 cm thickness) scattering medium. Theoretical calculations show that it is possible to reconstruct images in higher milk solutions (about 20%) with optical properties close to human tissues.

U-net-based segmentation of foreign bodies and ghost images in panoramic radiographs.

The rapid development and progress in deep machine-learning techniques have become a key factor in solving the future challenges of humanity. Vision-based target detection and object classification have been improved due to the development of deep learning algorithms. Data fusion in autonomous driving is a fact and a prerequisite task of data preprocessing from multi-sensors that provide a precise, well-engineered, and complete detection of objects, scene or events. The target of the current study is to develop an in-vehicle information system to prevent or at least mitigate traffic issues related to parking detection and traffic congestion detection. In this study we examined to solve these problems described by (1) extracting region-of-interest in the images (2) vehicle detection based on instance segmentation, and (3) building deep learning model based on the key features obtained from input parking images. We build a deep machine learning algorithm that enables collecting real video-camera feeds from vision sensors and predicting free parking spaces. Image augmentation techniques were performed using edge detection, cropping, refined by rotating, thresholding, resizing, or color augment to predict the region of bounding boxes. A deep convolutional neural network F-MTCNN model is proposed that simultaneously capable for compiling, training, validating and testing on parking video frames through video-camera. The results of proposed model employing on publicly available PK-Lot parking dataset and the optimized model achieved a relatively higher accuracy 97.6% than previous reported methodologies. Moreover, this article presents mathematical and simulation results using state-of-the-art deep learning technologies for smart parking space detection. The results are verified using Python, TensorFlow, OpenCV computer simulation frameworks.

Point spread function (PSF) of ghost imaging (GI) with pseudo-thermal light source doesn't satisfy the property of space translation invariance and existing GI linear reconstruction algorithms offer images with low quality when the measurement process doesn't reach ergodic. By modifying the intensity value of the speckle patterns recorded by the camera in the reference path, the property of PSF can be optimized and a linear reconstruction method called optimized ghost imaging (OGI) is proposed to stably recover the object's image even in the measurements below Nyquist limit. In comparison with existing GI linear reconstruction algorithms, both the simulated and experimental results demonstrate that the image's SNR can be significantly enhanced by OGI especially when the sampling ratio is larger than 0.68 and the detection SNR is greater than 20 dB.