Orthographic Video Map Generation Considering 3D GIS View Matching

To improve the navigation safety of inland river ships and enrich the methods of environmental perception, this paper studies the recognition and depth estimation of inland river ships based on binocular stereo vision (BSV). In the stage of ship recognition, considering the computational pressure brought by the huge network parameters of the classic YOLOv4 model, the MobileNetV1 network was proposed as the feature extraction module of the YOLOv4 model. The results indicate that the mAP value of the MobileNetV1-YOLOv4 model reaches 89.25%, the weight size of the backbone network was only 47.6 M, which greatly reduced the amount of computation while ensuring the recognition accuracy. In the stage of depth estimation, this paper proposes a feature point detection and matching algorithm based on the ORB algorithm at sub-pixel level, that is, firstly, the FSRCNN algorithm was used to perform super-resolution reconstruction of the original image, to further increase the density of image feature points and detection accuracy, which was more conducive to the calculation of the image parallax value. The ships’ depth estimation results indicate that when the distance to the target is about 300 m, the depth estimation error is less than 3%, which meets the depth estimation needs of inland ships. The ship target recognition and depth estimation technology based on BSV proposed in this paper makes up for the shortcomings of the existing environmental perception methods, improves the navigation safety of ships to a certain extent, and greatly promotes the development of intelligent ships in the future.

Image stitching refers to the combination of multiple images with a certain over lappingarea rate acquired by the image acquisition device into one image, which has a full picture of each image. Image stitching technology is an important part in the field of digital image processing. With the increasing requirements for image quality and the limited resolution of ordinary cameras, it is impossible to obtain large angle of view and clear image at the same time. In this paper, the key technologies involved in pavement crack image stitching are intensively studied, including image feature point detection and matching, internal and external camera parameter estimation and optimization, and image fusion. The work done in this paper and the main innovations include: (1)Two main feature detection and description algorithms, SIFT algorithm and ORBalgorithm, are studied. In view of the fact that the SIFT algorithm has redundancy when the Gaussian pyramid is established, this paper appropriately reduces the number of groups and maintains the number and stability of feature point detection. Combined with the advantages and disadvantages of the improved SIFT algorithm and ORB algorithm, and the performance of the two algorithms in the detection of pavement cracks, the improvedSIF'T algorithm is used as the feature point detection algorithm, and rBRlEF algorithm for describing the feature points in the ORB algorithm is used as the feature point description algorithm. The description algorithm is called the BSIF'T algorithm. Experiments show that the robustness of BSIFT algorithm is similar to that of SIFT algorithm, which is better than ORB algorithm and faster than SIF'T algorithm. (2)Image fiusion algorithms are studied, including best seam-line search and image fusion. Aiming at the ghost phenomenon that may occur in overlapping regions, this paper proposes a best seam-line search algorithm based on distance transform.

When an earthquake strikes an area, unless the local infrastructure has a structural health monitoring system, it is difficult to obtain its health condition due to a lack of data. A smartphone could be competently used for many monitoring tasks to gather large amounts of data at a low cost. In this study, a smartphone is used as a sensor to obtain the interstory drift of buildings by recording a video of the ceiling with the front camera using a feature point matching algorithm. In addition, static and dynamic tests were carried out. In the static tests, the precision and error were investigated and compared with the reference sensor. In the dynamic tests, the displacement under a seismic wave was acquired by a smartphone and compared with the reference sensor, and the illumination condition was varied to test the robustness of the algorithm. The results indicated that feature point matching was suitable for calculating the displacement, and the smartphone was competent in monitoring the interstory drift during an earthquake.

In order to reduce the proportion of outliers in feature point matching, we propose a feature point matching optimization algorithm based on a wasserstein distance and apply the algorithm to the grid cell model. The wasserstein distance is introduced to calculate the similarity of the distribution of feature points in the grids around each pair of matching points, and the outer points are eliminated by the similarity. We compare the algorithm with the robust algorithm in OpenCV, and the experimental results show that the precision of our algorithm is higher than robust, and the recall is smaller. We open source our implementations at https://github.com/liuzhenboo/Optimization-of-Feature-Point-Matching-Algorithm-Based-on-Wasserstein-Distance.

For the implementation of stereo image-based visual servoing algorithm in the eye-in-hand robotics applications, one of the main concerns is the accurate point feature detection and matching algorithm. Since the visual servoing is carried out in the textureless environment, the feature detection process is even more challenging. To fulfill the requirement of a robust and reliable point feature detection process, in this paper we present the novel deep learning-based algorithm. The approach based on convolutional neural networks and algorithm for detection of manufacturing entities is proposed and detected regions of interest are utilized for the improvement of the point feature detection algorithm. The proposed algorithm is experimentally evaluated in real-world settings by using wheeled nonholonomic mobile robot RAICO equipped with stereo vision system. The experimental results show the improvement of 58% in the accuracy of matched point features in the images obtained during the visual servoing process. Moreover, with the implementation of the proposed deep learning-based approach, the number of successful experimental runs has increased by 80%.

Indoor position technology based on visual becames a hot spot which can provide high position accuracy and more scene information. The system proposed by this paper is operated in real time, robust to severe motion clutter and brightness transformation and low cost which combines several steps include image acquisition and correction, object recognition based on deep learning and stereo matching. And this paper also proposes a new multi-feature fusion feature point extraction and matching algorithm for object accurate location. Finally, we present an evaluation in the dynamic indoor positioning tests and the result of the indoor horizontal positioning error is less than 1m.

ABSTRACTReliable feature point matching is a pivotal step in the integrated 3D reconstruction of aerial and ground images. Nonetheless, the substantial viewpoint and scale differences between aerial and ground images introduce significant challenges in feature point matching. To address this issue, this study presents a robust matching method for feature points from these two types of images, employing partitioned attention aggregation along with a coarse‐to‐fine hierarchical matching approach, enabling efficient matching of large‐scale aerial‐ground images even with hardware limitations. In the coarse matching phase, region segmentation is carried out on the ground image based on its depth map, which is computed using a monocular depth estimation network. Subsequently, a graph neural network serves as an intermediary for information exchange between features, and partitioned attention aggregation is performed while adhering to the constraints imposed by the segmented image regions. During this process, feature points are grouped to form distinct subgraphs, enhancing the structural similarity between corresponding graphs when there are significant depth and viewpoint changes. This also reduces redundant connections within graph neural networks, leading to efficient feature matching. Corresponding local patches are generated from the full‐sized aerial‐ground images using the initial matches, and further feature point extraction and matching from these patches yield accurate point matches. The experimental results indicate the proposed method performs better than state‐of‐the‐art matching methods. Specifically, compared with the best‐performing method among the comparative ones, our method enhances the average number of correct matches and the average matching precision by approximately 6‐fold and 36%, respectively.

Image stitching is an important technology in machine vision inspection whose algorithms focus on the detection and match of feature points. For the solar panel images achieved by machine vision, it's found that the available image stitching algorithms failed to detect enough valid feature points, which led to a large number of incorrect matching points. Here, we improve an algorithm to locate feature points and find dense match points. The proposed algorithm is based on SIFT. The improved algorithm (I-SIFT) can locate the position of feature points in every solar panel images, reducing the effect of invalid feature points on the experiment. Euclidean distance is used to preliminarily ensure the matching points, and then the relative horizontal position of every matching points is used to eliminate the mismatching points caused by the space similarity of the feature points, so that the matching accuracy is improved. Then the improved algorithm is used in the traditional Harris, SIFT and SURF stitching algorithm. The experimental results show that the success rate of I-SIFT algorithm can exceed 95% and the computation time has decreased by nearly 95% than that of the traditional algorithm. In conclusion, the improved algorithm implements accurately and rapidly stitching for solar panel images.

To resolve multi-sensor remote sensing images registration, a new approach which combines feature and regional similarity measure is proposed. This approach firstly adopts SIFT algorithm to match feature points and constructs initial affine transformation function; next uses normalized cross-correlation coefficient (NCC) to define regional similarity measure, and improves NCC with initial affine transformation parameters in order to get further matching points from those unmatched in SIFT algorithm. Then, after gross filtering the effective matching feature points, builds the model for image registration by all the matching feature points. This method combines the advantages of feature matching and regional matching, and solves the problem that the multi-sensor remote sensing images are difficult to match correctly because of geometric and radiometric differences. Experiments show that this method has strong robustness, and are of higher registration accuracy than single registration method.

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

As the front end of visual simultaneous localization and mapping (vSLAM), the feature points matching of the visual odometry (VO) provides important robot pose judgment, while there are mismatches in the feature points matching due to inaccurate information obtained from cameras. In the process of feature points matching, VO initializes the matching of feature points through two adjacent frames of pictures, as SLAM runs, the cumulative error will gradually increase, resulting in mapping and positioning results inaccurate. Aiming to reduce the feature points matching error during initialization, an evidential fusion method for feature points matching is put forward. On the basis of descriptor similarity in Oriented FAST and Rotated BRIEF SLAM (ORB-SLAM) and pixel similarity around feature points, we treat the similarity index as an evidence, and take Belief Jensen-Shannon (BJS) divergence to measure distance between different evidences. The proposed evidential method considers the uncertainties brought by different indexes in accord with the feature points matching, and obtains suitable matching points reducing false matches through the combination of evidences. BJS divergence measure is carried out to measure the discrepancy and conflict degree among the evidences. After that, the reliability of evidence could be obtained to modify the information volume of the evidences. Finally, an example of feature points matching shows that the evidential method is feasible for feature points matching.

With the rapid development of artificial intelligence and computer vision, feature point detection algorithms have emerged as a significant area of research, particularly in applications such as SLAM (Simultaneous Localization and Mapping) and image registration. However, the FAST (Features from Accelerated Segment Test) detection method employed in the ORB (Oriented FAST and Rotated BRIEF) algorithm exhibits limitations in detection accuracy. This paper proposes a feature point detection method based on sequential neural networks and knowledge distillation, aiming to enhance the performance of the FAST algorithm. Firstly, the sequential nature of the data points in the FAST algorithm is analyzed, leading to the development of a binary classification and regression model using a sequential neural network. Considering the significant computational overhead of the model, knowledge distillation techniques are introduced to optimize the efficiency of the neural network. Secondly, the constructed model undergoes performance testing and evaluation, with a detailed comparison against the traditional FAST method. Finally, this paper discusses the future prospects of feature point detection, suggesting potential directions for further optimization.

We address a core problem of computer vision: Detection and description of 2D feature points for image matching. For a long time, hand-crafted designs, like the seminal SIFT algorithm, were unsurpassed in accuracy and efficiency. Recently, learned feature detectors emerged that implement detection and description using neural networks. Training these networks usually resorts to optimizing low-level matching scores, often pre-defining sets of image patches which should or should not match, or which should or should not contain key points. Unfortunately, increased accuracy for these low-level matching scores does not necessarily translate to better performance in high-level vision tasks. We propose a new training methodology which embeds the feature detector in a complete vision pipeline, and where the learnable parameters are trained in an end-to-end fashion. We overcome the discrete nature of key point selection and descriptor matching using principles from reinforcement learning. As an example, we address the task of relative pose estimation between a pair of images. We demonstrate that the accuracy of a state-of-the-art learning-based feature detector can be increased when trained for the task it is supposed to solve at test time. Our training methodology poses little restrictions on the task to learn, and works for any architecture which predicts key point heat maps, and descriptors for key point locations.

Rapid camera rotations (e.g. camera shake) are a significant problem when real-time computer vision algorithms are applied to video from a handheld or head-mounted camera. Such camera motions cause image features to move large distances in the image and cause significant motion blur. Here we propose a very fast method of estimating the camera rotation from a single frame which does not require any detection, matching or extraction of feature points and can be used as a motion estimator to reduce the search range for feature matching algorithms that may be subsequently applied to the image. This method exploits the motion blur in the frame, using features which remain sharp to rapidly compute the axis of rotation of the camera, and using blurred features to estimate the magnitude of the camera's rotation.

With the continuous development and improvement of sensors, sensor technology is also constantly making breakthroughs. It is also widely used in various fields of medicine today. This article summarizes the practice of two types of SLAM (Simultaneous Localization and Mapping) in medicine. The detection and matching of human luminal feature points under SLAM were studied so as to reconstruct the human internal environment so that doctors can obtain three-dimensional reconstruction feedback within the surgical range and improve the safety of surgery. And for different surgical environments, the same SLAM is improved on the basis of innovation, which is more suitable for different scenarios. Three algorithms are described in this article for taking images of the inside of the abdominal cavity and collecting data based on the images using different algorithms. Based on the collected data, the internal abdominal cavity is reconstructed in 3D. Finally, a three-dimensional visualization system for the abdominal cavity was constructed. It has important research significance in the field of medical auxiliary research.