A novel multi-object detection method in complex scene using synthetic aperture imaging

The occlusion problem is a major challenge in the field of computer vision. Synthetic aperture imaging (SAI) is often used for surface reconstruction of occluded objects. However, SAI usually relies on high-speed information transmission devices. In addition, a large amount of information in the scene is lost, when handling low-resolution input images. This limitation results in unclear reconstructed regions in the synthetic aperture image and thus hinders the application of SAI in downstream tasks. We propose a multi-view super-resolution SAI method. It aims to generate high-resolution synthetic aperture images using images acquired by few of low-resolution acquisition devices. The main contributions of this paper are: (1) a multi-view super-resolution algorithm is proposed. It can generate clear synthetic aperture images in an array with a limited number of cameras. (2) By exploiting the correlation between views, the proposed algorithm can generate super-resolution synthetic aperture images with more accurate image structure and sharper image edges. (3) A feature extraction module is proposed. It can effectively extract the complementary relationship between pictures from different perspectives. The experimental results show that the proposed method can generate a reconstructed image of the occluded object surface with clear edges and accurate structure. Compared to conventional SAI, our method improves 5.7 % / 21.1 % on peak signal-to-noise ratio (PSNR)/structure similarity index measure (SSIM) and 4.4 % / 9.2 % on PSNR/SSIM respectively on two datasets compared to other state-of-the-art super-resolution methods.

Hidden object imaging is challenging problem in the fields of computer vision and image processing, and it’s a key step in many application fields, include intelligent video surveillance, visual tracking and scene understanding. Recently, the camera array synthetic aperture imaging has been proved to be a powerful technology for hidden object detection, and the state-of-art synthetic imaging method can focus on multiple parallel planes so as to achieve seeing hidden through severe occlusion. However, due to the depth variation of hidden object’s surface, it’s difficult for existing method to get a complete clear image. This paper presents a novel synthetic aperture imaging and fusion approach, the main characteristics of proposed method include three parts: (1) A novel imaging framework is presented which integrate synthetic aperture imaging and fusion seamlessly. (2) Several multiple scale image fusion methods have been adopted in our framework to create high resolution and highly detailed images of occluded object. (3)A camera array synthetic aperture imaging and fusion system has been developed based on the proposed approach, and the Stanford light field public dataset and our camera array occlusion object dataset have been used to evaluate the performance of this algorithm. Extensive experiment results demonstrate that compared to the existing state-of-art camera array synthetic aperture imaging approach, our approach can enhance the image detail and quality of hidden objects under occlusion.

GPU-Based Minimum Variance Beamformer for Synthetic Aperture Imaging of the Eye

High‐resolution millimetre wave images of contiguous targets often suffer from the influence of sidelobe artefacts, partial correlation between targets and so on. Owing to the characteristics of near‐field synthetic aperture imaging radiometers [such as the changing point spread function (PSF) and slender sideline], the existing CLEAN algorithms are unsuitable for near‐field synthetic aperture imaging. This study is devoted to establishing a novel CLEAN algorithm (named adaptive CLEAN) to clean the reconstructed millimetre wave images accurately. First, the characteristics of near‐field synthetic aperture imaging are studied. Then, the adaptive CLEAN algorithm is established based on these characteristics. In this study, the authors amend the amplitude of the targets and select the matching PSF for them according to their azimuths. Unlike other CLEAN algorithms, the parameters of this adaptive CLEAN algorithm are calculated by a formula, which is concluded from a lot of simulation experiments. Finally, the effectiveness of the proposed adaptive CLEAN algorithm is tested by several simulation experiments, and the superiority is also demonstrated by comparing it with the existing CLEAN algorithm. The results demonstrate that the proposed method is an efficient, feasible algorithm for cleaning the reconstructed images, irrespective of the point or contiguous targets.

Deep learning has become the main way of the object detection task for autonomous vehicles. Meanwhile, this method typically requires vast amounts of training data to reach their full potential. However, collecting the data from real world and labeling manually is an expensive, time-consuming and error-prone process. Synthetic image has the potential to replace real image for training neural networks, because image creation and labeling annotations are free in this way. For the network trained by synthetic images, the reality gap between real and synthetic images is the main obstacle to use it in the real world. And most previous works are only devoted to generate synthetic images with a good performance on model training, but lack of analysis of the domain gap that affects the performance. This work designs a method of generating the real and synthetic images to analyze and reduce the reality gap between synthetic and real images for car detection. Firstly, this work put one single car with no-background in a random background image to generate real single car and synthetic single car images. In order to further reduce the domain gap in content level, this method keeps the car distribution in synthetic images is similar with the distribution of car in real world. For the purpose of reducing the domain gap in appearance level, the parameters of camera model are same as the camera parameters of image collecting cars and the image is rendered by using the PBRT(Physically Based Ray Tracing) when we generated the synthetic images. Secondly, by training the neural network of instance segmentation with different datasets, the across validation result proves that the reality gap between synthetic and real images is no more than the domain gap between real images. Thirdly, the training results of datasets with different samples diversity show that the diversity of the samples yields better generalization between different datasets for car detection which can effectively reduce the domain gap.

Most multi-object detection and tracking techniques suffer from the well-known "multi-object occlusion" problem. The abundant nodes of wireless video sensor networks (WVSNs) can be utilized to solve the problem, and the video nodes in WVSN have limited calculation capability and energy. In order to achieve effective multi-object tracking using WVSN, the main contributions of our proposed method are that: (1) the limits of field of view (FOV) of every video nodes are utilized to establish the consistent labeling of the objects in different views. (2) Mobile Agent is employed to communicate among network nodes, so the objects are assigned correct labels after multi-object occlusion. The performance of the approach has been demonstrated on real-world and the experimental results show that the proposed method is effective for resolving multi-object occlusions and meets the requirement of WVSN.

Recently, short-lag spatial coherence (SLSC) imaging has drawn much attention from researchers due to benefits such as improved speckle signal-to-noise ratio and contrast-to-noise ratio compared to conventional B-mode imaging based on delay-and-sum beamforming. However, SLSC imaging has good performance just around the transmit focal point, which is quite a drawback. In this paper, we propose a new method that combines SLSC imaging and synthetic aperture imaging to address such a shortcoming. The efficacy of the proposed method is verified through simulations and experiments using Field II echo data and phantom data acquired with an actual ultrasound system.

Synthetic aperture imaging (SAI) is able to achieve the see through effect by blurring out the off-focus foreground occlusions and reconstructing the in-focus occluded targets from multi-view images. However, very dense occlusions and extreme lighting conditions may bring significant disturbances to the SAI based on conventional frame-based cameras, leading to performance degeneration. To address these problems, we propose a novel SAI system based on the event camera which can produce asynchronous events with extremely low latency and high dynamic range. Thus, it can eliminate the interference of dense occlusions by measuring with almost continuous views, and simultaneously tackle the over/under exposure problems. To reconstruct the occluded targets, we propose a hybrid encoder-decoder network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs). In the hybrid network, the spatio-temporal information of the collected events is first encoded by SNN layers, and then transformed to the visual image of the occluded targets by a style-transfer CNN decoder. Through experiments, the proposed method shows remarkable performance in dealing with very dense occlusions and extreme lighting conditions, and high quality visual images can be reconstructed using pure event data.

The synthetic aperture imaging (SAI) and backpropagation imaging (BPI) algorithms are compared for reconstructing a representation of scattering objects from time-domain ground-penetrating radar data. Synthetic data produced by a FDTD forward solver is used as a basis for the comparison. We assume initially a homogeneous, lossless host medium. The BPI algorithm, which includes waveform diffractions, is shown to result in a better focused image than SAI, with fewer artifacts. The superiority of the BPI algorithm is particularly evident at lower frequencies, where the scattering objects are on the order of the wavelength of the incident field.

This paper describes an image compounding technique based on the use of different apodization functions, the evaluation of the signals phases and information from the interaction of different propagation modes of Lamb waves with defects for enhanced damage detection, resolution and contrast. A 16 elements linear array is attached to a 1 mm thickness isotropic aluminum plate with artificial defects. The array can excite the fundamental A0 and S0 modes at the frequencies of 100 kHz and 360 kHz, respectively. For each mode two synthetic aperture (SA) images with uniform and Blackman apodization and one image of Coherence Factor Map (CFM) are obtained. The specific interaction between each propagation mode and the defects and the characteristics of acoustic radiation patterns due to different apodization functions result in images with different resolution and contrast. From the phase information one of the SA images is selected at each pixel to compound the final image. The SA images are multiplied by the CFM image to improve contrast and for the dispersive A0 mode it is used a technique for dispersion compensation. There is a contrast improvement of 47.5 dB, reducing the dead zone and improving resolution and damage detection.

Synthetic aperture imaging and virtual point detection have been exploited to extend the depth of view of photoacoustic microscopy. The approach is commonly based on a constant assumed sound speed, which reduces image quality. We propose a new, to the best of our knowledge, self-adaptive technique to estimate the speed of sound when integrated with this hybrid strategy. It is accomplished through linear regression between the square of time of flight detected at individual virtual detectors and the square of their horizontal distances on the focal plane. The imaging results show our proposed method can significantly improve the lateral resolution, imaging intensity, and spatial precision for inhomogeneous tissue.

During the data processing and interpretation of mine transient electromagnetic method (MTEM), in order to improve the identification of the geo‐electrical interface between low resistivity abnormal body and surrounding rock in advanced detection, transient electromagnetic field was transformed to pseudo‐seismic wave‐field based on the function relationship between spreading electromagnetic field and seismic wave‐field in whole‐space. Meanwhile, the signal of pseudo‐seismic after being transformed was processed with correlative stack to strengthen the amplitude and improve signal‐noise‐ratio (SNR) using the synthetic aperture imaging (SAI). The data of different detection directions on one surveying point in MTEM were imaged as aperture data, which would highlight weak abnormality and improve SNR. In addition, the application effect of SAI was testified with the in‐site advanced detection in roadway using MTEM. The results showed that the SAI of the MTEM could improve SNR of the electrical interface information and highlight the geometrical resolution to increase exploration accuracy. The technique is significant theoretically and practically in accurate advanced detection of aquifer structure by MTEM.

Currently available medical ultrasonic imaging systems are limited in some applications by their resolution. In particular: better resolution is needed to detect small lesions in the female breast. The apertures required to improve resolution can be obtained using large transducers at the expense of depth of field. Alternatively, synthetic aperture techniques offer the potential for achieving the necessary resolution without sacrificing depth of field. Advantages of synthetic aperture imaging are high focus, independent of depth, coupled with the absence of near field aberrations. The goal of this study is to develop a synthetic aperture ultrasonic imaging system specifically designed for breast scanning.

High‐resolution imaging of distant, dynamic targets is fundamentally limited by finite‐aperture diffraction. Unlike radar, optical detectors record only intensity and lack the temporal resolution for direct phase measurement, hindering straightforward extension of synthetic aperture techniques to the optical domain. While Macroscopic Fourier ptychography (FP) offers a promising computational alternative, its reliance on mechanical scanning or engineered illumination restricts its use to cooperative, static scenes. In this paper, we present Inverse Synthetic Aperture Macroscopic Fourier ptychography (ISAMFP), which, to the best of our knowledge, is the first practical realization of inverse synthetic aperture imaging for diffusely reflecting targets within the Fourier ptychography framework. By transforming the natural target motion from a detrimental factor into an effective source of angular diversity, ISAMFP eliminates the need for mechanical scanning or illumination modulation and achieves resolution enhancement far beyond the aperture limit using intensity‐only measurements. Experimental validations on USAF resolution targets, unmanned aerial vehicles, and complex multi‐target scenes highlight both the robustness and versatility of ISAMFP, establishing it as a powerful tool for non‐interferometric synthetic aperture imaging of moving, non‐cooperative targets, with broad implications for surveillance, defense, and remote sensing.

Photonics‐based radar enables a large operation bandwidth that is particularly favourable for high‐resolution synthetic aperture imaging. However, how to implement fast and high‐accuracy imaging with photonics‐based broadband radar is still an open question. In this Letter, the performance of photonics‐based inverse synthetic aperture radar imaging with frequency‐domain and time‐domain algorithms are experimentally investigated and compared. The results show that, frequency‐domain range–Doppler algorithm has fast imaging capability, but the image suffers from defocusing and distortions. Although migration compensation techniques can be applied to improve the imaging accuracy, the imaging speed is significantly reduced. On the other hand, using time‐domain back‐projection (BP) algorithm can achieve well‐focused images, and the imaging speed can be enhanced with fast BP methods. Therefore, the time‐domain imaging method is found to be the best choice for fast and high‐accuracy synthetic aperture imaging with photonics‐based broadband radars.