Research on visual localization technique of UAV based on heterogenous image matching

Heterogeneous images acquired from various platforms and sensors provide complementary information. However, to use that information in applications such as image fusion and change detection, accurate image matching is essential to further process and analyze these heterogeneous images, especially if they have significant differences in radiation and geometric characteristics. Therefore, matching heterogeneous remote sensing images is challenging. To address this issue, we propose a feature point matching method named Cross and Self Attentional Matcher (CSAM) based on Attention mechanisms (algorithms) that have been extensively used in various computer vision-based applications. Specifically, CSAM alternatively uses self-Attention and cross-Attention on the two matching images to exploit feature point location and context information. Then, the feature descriptor is further aggregated to assist CSAM in creating matching point pairs while removing the false matching points. To further improve the training efficiency of CSAM, this paper establishes a new training dataset of heterogeneous images, including 1,000,000 generated image pairs. Extensive experiments indicate that CSAM outperforms the existing feature extraction and matching methods, including SIFT, RIFT, CFOG, NNDR, FSC, GMS, OA-Net, and Superglue, attaining an average precision and processing time of 81.29% and 0.13 s. In addition to higher matching performance and computational efficiency, CSAM has better generalization ability for multimodal image matching and registration tasks.

With the emerging techniques of wireless communication and cloud computing, large volumes of multimedia data are outsourced from resource constrained users to the cloud with abundant resource for both delegated storage and computation. Unfortunately, there is a risk of users’ image privacy leakage in the process of outsourcing to untrusted cloud. Most of the existing work achieved privacy-preserving image feature extraction and matching by using public key (fully) homomorphic encryption (FHE), but the heavy computational overhead and communication overhead cannot adapt to resource-constrained mobile devices. Other works disabled to realize image denoising in the encrypted domain or only focused on the scale-invariant feature transform (SIFT) descriptor that is inappropriate for position-sensitive feature extraction. To address these issues, in this paper, a privacy-preserving shape context based image denoising and matching protocol PPOIM with efficient outsourcing is proposed. Firstly, to improve the accuracy of image matching, a privacy-preserving image denoising scheme PPID is proposed without exploiting public key FHE. Then, based on PPID, a privacy-preserving image matching protocol PPOIM adopting shape context descriptor is devised, where two secure and efficient comparison and counting protocols in the encrypted domain are presented. All the original image privacy, query image privacy and image matching result privacy are well protected. Finally, formal security proof and extensive simulations on real-world data sets demonstrate the efficiency and practicability of our proposed PPOIM.

Synthetic Aperture Radar (SAR) imaging and visible light imaging are the two most commonly used imaging methods for remote sensing satellites. Since the imaging information of the two is highly complementary, many scenarios of data fusion need to use these two heterogeneous data. However, before data fusion, the data of the two modalities need to be aligned, and the performance of heterogeneous data matching algorithm directly affects the performance of obtaining ground control points during alignment. At present, there are many one-stage and two-stage methods for heterogeneous remote sensing image matching. The existing one-stage methods have problems such as large average prediction offset, low matching accuracy, and imbalance of features at different levels when combining features. The stage method cannot meet the actual needs in terms of speed and accuracy. To address these issues, this paper proposes an end-to-end heterogeneous remote sensing image matching framework HB3CF. The framework uses the classic image feature extraction network to construct a pair of pseudo-twin networks, which extract features from two heterogeneous images respectively. Then, the features of each level are uniformly sampled on the channel using the convolution layer to reduce the weight of the high-dimensional features in the joint features, which effectively improves the expression ability of the features of each level of the model. Finally, the matching results are obtained by performing convolution, cross-correlation and up-sampling operations on the high-dimensional features of the SAR image and the optical image. Experiments show that the average offset error of the model is reduced by about 25% compared to state-of-the-art methods. When the average offset error is less than or equal to 0 pixel, 1 pixel, 2 pixel and 3 pixel, the accuracy is increased by 8.53%, 9.54%, 4.16% and 1.12% respectively, reaching 25.90%, 65.03%, 86.65% and 92.15%, which greatly improves the accuracy of the matching method of heterogeneous images, and explores the application of deep learning methods in large-scale heterogeneous remote sensing image matching tasks.

Dementia induced by methyldopa with haloperidol

In the area of computer vision, pattern recognition and image processing, image match is a research hotspot with important theoretical significance and practical value. Recently, the image matching algorithm based on SIFT has drown wide attention for its outstanding local feature matching performance. In view of high computation complexity, poor anti-noise ability, and difficulty for practical use of the original SIFT method, a more brief and efficient SIFT image matching algorithm: BE-SIFT is proposed in this paper for computer vision. Through collaborative improvement and optimization on detection of SIFT scale space feature points, allocation of key point principal direction, and calculation of feature descriptors, the proposed method has achieved more brief dimension expression and more efficient local feature match than other existing methods. Experiments based on the Affine Covariant Features dataset provided by Oxford VGG Group demonstrate that BE-SIFT has stronger robustness for image match under the condition of viewpoint changes, rotation + scale changes, blur changes and noise changes. What's more, the computational overhead is sharply reduced and the real-time performance is greatly improved while ensuring the uniqueness of local feature and satisfactory accuracy.

On 6 September 2008, two optical satellites, HJ-1 A and B (HJ-1 A/B), were successfully launched from China. However, the system geometric correction products of the HJ-1 A/B charge-coupled device (HJ-1 images) have low geometric precision and need to be corrected. The HJ-1 images have a large aspect angle, a wide swath width, and a large image size. Furthermore, the local geometric distortions are too complex in one scene. Given these characteristics of HJ-1 images, geometric correction is still a challenging work. This article proposes an automatic geometric precision correction system (GPCS) based on the automatic registration between HJ-1 images and Landsat Thematic Mapper images. First, the coarse image matching method based on geometric-restricted scale-invariant feature transform (SIFT) is used to determine the coarse global transformation between the HJ-1 image and the reference image. Second, inspired by the hierarchical method of non-rigid registration for medical images, a hierarchical image matching approach is proposed based on the combination of SIFT feature points and template matching. This approach decomposes a matching problem of a whole image into numerous matching problems of image blocks and can overcome the impact of local distortions in HJ-1 images. Hierarchical random sample consensus (RANSAC) based on digital elevation model (H-RANSAC) is used to remove incorrect control points. Third, an HJ-1 image is rectified using a triangulated irregular network. Finally, the automatic evaluation method based on automatic image matching between the corrected HJ-1 image and the reference image is adopted to evaluate the geometric precision. On the one hand, experiments on eight HJ-1 images demonstrate the efficiency and accuracy of the different steps of GPCS. On the other hand, experiments on 1000 HJ-1 images also demonstrated the robustness, accuracy, and suitability for batch processing.

An approach of fast image mosaic is presented, which involves image matching and image intensity smoothing. Image matching includes two procedures, i.e. rough matching and fine matching. In rough matching, the overlapped regions of two adjacent images to be mosaicked are segmented to binary image at first. Then the binary images are filtered by open operation of mathematic morphologic method. In the binary image region of the reference image, feature template is searched and extracted on a given rule. Via XOR operation of the feature template and search region, some possible matching positions in the overlapped region of the other image are got. In the fine matching, the sequential similarity detection algorithm (SSDA) is adopted to perform matching computation in the small regions near the positions got in the rough matching, and then the relative position offsets in X-orientation and Y-orientation between the two adjacent images are got. Based on the result of the image matching, the two images are stitched. An approach of seam-line smoothing is adopted to adjust the intensity of the overlapped area. Simulation experimental results show that the approach greatly improves the operation speed, while the precision remains fine, so it can be applied in real-time mosaicking.

Local features are essential for fundamental visual tasks, e.g., image matching, visual localization, etc. In recent researches, learning-based feature detection and description methods demonstrate remarkable performance. The extracted features are represented as float data and matched in Euclidean space, leading to calculation and memory burden in applications on resource-limited mobile platforms. This paper proposes a single-input multi-output model for feature extraction and a descriptor quantization approach to embedding the features into Hamming space. A self-supervised learning strategy is designed to train the model for high repeatability and matching robustness first. Then we implement the descriptor quantization as an iterative hashing procedure and quantize the extracted features offline to reduce the memory and calculation demands. We also provide a dimensionality reduction process to make a trade-off between performance and resource limitation. The proposed method is suitable for feature-related tasks on hardware-limited platforms and achieved convincing results in the image matching and visual localization benchmarks.

SIFT (Scale Invariant Feature Transform) is wildly used in image matching but suffered from low matching pairs when employed in endoscope image. This paper presented an improved algorithm based on SIFT, the core contribution of which is Zone Matching approach. By collecting all feature points in a neighbor patch around the coordinate of an objective feature in the key image and finding the closest one to the feature in current image, the Zone Matching can obtain more matching pairs in shorter time. The experiment result shows a good improvement on the matching results both in matching number and computing time.

By combining wavelet transform and increment sign correlation with iterative correction, we present a robust image matching method, which effectively reduced mismatch. Firstly, by decomposing the sensed image and the reference image with wavelet, smooth images are generated in coarse scales. By the use of increment and decrement information of adjacent pixels, smooth images are encoded into binary images, and the matching position can be found by use of the cross correlation function. Wavelet decomposition can effectively remove noise and greatly improve the real-time performance of matching algorithm while increment sign correlation method is not sensitive to the gray level of images. The combination of wavelet transform and increment sign correlation can deal with noise, shadow and occlusion as well as improve the matching speed compared with traditional matching methods. Considering the condition of heavy noise and occlusion that the matching method mentioned above can not find the correct position in condition of heavy noise and occlusion, we present an iterative correction image matching method, i.e. use the above matching result as prior knowledge and achieve the correct matching position by correcting matching results iteratively. Theoretical analysis shows the feasibility of the presented method, and our experiments show its validity.

The main contribution of this paper is the reformulation of the simultaneous localization and mapping (SLAM) problem for mobile robots such that the mapping and localization can be treated as two concurrent yet separated processes: D-SLAM (decoupled SLAM). It is shown that SLAM can be decoupled into solving a non-linear static estimation problem for mapping and a low-dimensional dynamic estimation problem for localization. The mapping problem can be solved using an Extended Information Filter where the information matrix is shown to be exactly sparse. A significant saving in the computational effort can be achieved for large scale problems by exploiting the special properties of sparse matrices. An important feature of D-SLAM is that the correlation among landmarks are still kept and it is demonstrated that the uncertainty of the map landmarks monotonically decrease. The algorithm is illustrated through computer simulations and experiments.

Heterogeneous face recognition (HFR) refers to matching face images acquired from different sources (i.e., different sensors or different wavelengths) for identification. HFR plays an important role in both biometrics research and industry. In spite of promising progresses achieved in recent years, HFR is still a challenging problem due to the difficulty to represent two heterogeneous images in a homogeneous manner. Existing HFR methods either represent an image ignoring the spatial information, or rely on a transformation procedure which complicates the recognition task. Considering these problems, we propose a novel graphical representation based HFR method (G-HFR) in this paper. Markov networks are employed to represent heterogeneous image patches separately, which takes the spatial compatibility between neighboring image patches into consideration. A coupled representation similarity metric (CRSM) is designed to measure the similarity between obtained graphical representations. Extensive experiments conducted on multiple HFR scenarios (viewed sketch, forensic sketch, near infrared image, and thermal infrared image) show that the proposed method outperforms state-of-the-art methods.

In order to encourage energy conservation and emission reduction, electric vehicles (EV) have gradually become one of the most important emerging strategic industries in many countries. With the gradual maturity and application of battery technology, policies support of new energy industry and the marketization of EV in many countries, the location problem of charging stations has become one part of the urban development strategies, but the bottleneck that hinders the development of new EV is the factors of vehicle life and the battery life. The focus of problem solving is the reasonable location and deployment of charging stations. Therefore, the charging station location problem has become a research hotspot in recent years. The EV charging station location is essentially an application scenario of facility location problem. As a long-standing problem, many researchers have proposed many classic models and algorithms, this paper reviews the research progress and existing problems related to charging station location in recent years from the aspects of algorithm and model, and finally gives some future research directions.

We propose a phase image simulation method for Electro-Optical Frequency Mapping (EOFM). The proposed method eases the matching of phase images and circuit layout data, which was previously difficult because phase images are very complex. Physical failure analysis based on this matching result is enabled. Further works are required for understandings of simulated phase images in compound logic cells having many states or high-Z states.

Aimed at scene matching problem for taking infrared image as the actual data and the visible image as the referenced data, a multi-resolution matching algorithm which fuses compressive sensing Scale Invariant Feature Transform (SIFT) feature is presented based on Bandelet transform. Two kinds of images are separately transformed into Bandelet domain to compress the feature search space of scene matching based on the best sparse representation of natural images by Bandelet transform. On the basis of adaptive Bayes threshold denoising for infrared image, the concept of sparse feature representation of compressive sensing theory is introduced into SIFT algorithm. For low-frequency image in Bandelet domain, high-dimensional SIFT key point feature description vector is projected on compressive sensing random measurement matrix to achieve dimensionality reduction. Then, the improved Genetic Algorithm (GA) to overcome premature phenomena is used as the search strategy, and the L1 distance measure of SIFT feature vectors of compressive sensing for two kinds of images is applied to the search similarity criterion to match low-frequency image of high scale in Bandelet domain. The matching result is used as the guidance of the matching process for low-frequency image of low scale, and the matching result of full-resolution image is obtained iteratively. Experimental results show that the proposed method has not only high matching accuracy and fast matching speed, but also better robustness in comparison with some classic matching algorithms, which can resist the geometric distortion of rotation for actual image.