Asymmetric gradient-based image alignment

Tongue image analysis has been an active study in medical imaging. Existing tongue image processing approaches deal with the issue of image alignment in oversimplified ways. These approaches mainly extract patches or simple regions on pre-defined positions, which are severely sensitive to tongue deformations. In this paper, we present a conformal mapping method for tongue image alignment, the principle of which is to determine the interior mapping based on the boundary mapping so that it is robust to the deformations. The conformal alignment consists of two stages: the mapping on the boundary is firstly established via the Fourier descriptor before the mapping is extended onto the interior region via Cauchy's integral and finite-difference method. Average tongues and eigen-tongues are constructed based on the conformal alignment for feature extraction. Experiments show that the proposed alignment is robust against tongue deformations and can be employed to correct existing rigid partition methods. Numerical evaluations on time efficiency and accuracy also show that our method is considerably fast and very accurate, compared with several baseline methods in this field. For the task of disease detection, the features based on the aligned images outperform some state-of-the-art features. The results reveal that the proposed method provides an efficient and accurate tool for deformable medical image alignment and disease diagnosis. A MatLab script of the proposed algorithm is available on https://codeocean.com/capsule/4382908/tree/v1.

Accurate alignment of electron tomographic images without using embedded gold particles as fiducial markers is still a challenge. Here we propose a new markerless alignment method that employs Scale Invariant Feature Transform features (SIFT) as virtual markers. It differs from other types of feature in a way the sufficient and distinctive information it represents. This characteristic makes the following feature matching and tracking steps automatic and more reliable, which allows for estimating alignment parameters accurately. Furthermore, we use Sparse Bundle Adjustment (SPA) with M-estimation to estimate alignment parameters for each image. Experiments show that our method can achieve a reprojection residual less than 0.4 pixel and can approach the same accuracy of marker alignment. Besides, our method can apply to adjusting typical misalignments such as magnitude divergences or in-plane rotation and can detect bad images.

Alignment of medical images is a vital component of a large number of applications throughout the clinical track of events; not only within clinical diagnostic settings, but prominently so in the area of planning, consummation and evaluation of surgical and radiotherapeutical procedures. However, image registration of medical images is challenging because of variations on data appearance, imaging artifacts and complex data deformation problems. Hence, the aim of this study is to develop a robust image alignment method for medical images. An improved image registration method is proposed, and the method is evaluated with two types of medical data, including biological microscopic tissue images and dental X-ray images and compared with five state-of-the-art image registration techniques. The experimental results show that the presented method consistently performs well on both types of medical images, achieving 88.44 and 88.93% averaged registration accuracies for biological tissue images and X-ray images, respectively, and outperforms the benchmark methods. Based on the Tukey's honestly significant difference test and Fisher's least square difference test tests, the presented method performs significantly better than all existing methods (P ≤ 0.001) for tissue image alignment, and for the X-ray image registration, the proposed method performs significantly better than the two benchmark b-spline approaches (P < 0.001). The software implementation of the presented method and the data used in this study are made publicly available for scientific communities to use (http://www-o.ntust.edu.tw/∼cweiwang/ImprovedImageRegistration/). cweiwang@mail.ntust.edu.tw.

Alignment of two-dimensional electrophoresis gels

This paper addresses the problem of image alignment based on random measurements. Image alignment consists of estimating the relative transformation between a query image and a reference image. We consider the specific problem where the query image is provided in compressed form in terms of linear measurements captured by a vision sensor. We cast the alignment problem as a manifold distance minimization problem in the linear subspace defined by the measurements. The transformation manifold that represents synthesis of shift, rotation, and isotropic scaling of the reference image can be given in closed form when the reference pattern is sparsely represented over a parametric dictionary. We show that the objective function can then be decomposed as the difference of two convex functions (DC) in the particular case where the dictionary is built on Gaussian functions. Thus, the optimization problem becomes a DC program, which in turn can be solved globally by a cutting plane method. The quality of the solution is typically affected by the number of random measurements and the condition number of the manifold that describes the transformations of the reference image. We show that the curvature, which is closely related to the condition number, remains bounded in our image alignment problem, which means that the relative transformation between two images can be determined optimally in a reduced subspace.

Congealing is an unsupervised image alignment method for a set of images, and the transformation parameters are obtained by minimizing a sum-of-entropies function. In this paper, we provide a solution to improve the estimation of transformation parameters using a Newton optimization method, under the premise of maintaining the compatibility with feature descriptors. Besides, instead of SIFT descriptor in canonical Congealing, we combine Congealing with POEM (Patterns of Oriented Edge Magnitudes) which catches both edge in- formation and the relation between pixels at a neighboring region. The experiment results show that our alignment method has better ability for the removal of unwanted displacements, and also improves the performance of face recognition.

To provide a robust and convenient method by which radiographic images can be spatially aligned for digital radiographic subtraction without the use of manually selected reference points. An automated method for image alignment is described which begins with the extraction of a large number of edge features (1500 or more pixels) from each of two radiographic images taken of the same anatomical region of a given patient. The features in the first radiograph are paired, pixel by pixel, with those in the second radiograph using a nearest neighbor criterion. The edge features in the first radiograph are aligned with those in the second radiograph by performing an affine transformation that is consistent with the projection geometry for a plantar parallel X-ray beam. Transformation parameters are determined which provide the smallest spatial alignment error between the two sets of equivalent features. These parameters are found by a closed-form analytic solution, thus enabling a computationally efficient implementation. The final transformation is then applied to the entire first image resulting in a close spatial match to the second image. The performance of three dentists using the automatic method was compared with their performance using a manual method of alignment for eight pairs of images. The root mean squared error in image alignment for the automatic method was 14% lower than that with manual alignment. The variability for the automatic method was half that of the maximal method as measured by the residual error. The automatic method was also three times faster than the manual method. This method could make digital subtraction more accessible to researchers and practising dentists. Batch mode implementation could enable the processing of large volumes of data. Restriction to a region of interest and improved feature extraction could further improve performance.

When a purely rotating camera observes a general scene, overlapping views are related by a parallax-free warp which can be estimated by direct image alignment methods that iterate to optimise photo-consistency. However, building globally consistent mosaics from video has usually been tackled as an off-line task, while sequential methods suitable for real-time implementation have often suffered from long-term drift. In this paper we present a high performance real-time video mosaicing algorithm based on parallel image alignment via ESM (Efficient Second-order Minimisation) and global optimisation of a map of keyframes over the whole viewsphere. We present real-time results for drift-free camera rotation tracking and globally consistent spherical mosaicing from a variety of cameras in real scenes, demonstrating high global accuracy and the ability to track very rapid rotation while maintaining solid 30Hz operation. We also show that automatic camera calibration refinement can be straightforwardly built into our framework.

The 3D object detection is becoming indispensable for environmental perception in autonomous driving. Light detection and ranging (LiDAR) point clouds often fail to distinguish objects with similar structures and are quite sparse for distant or small objects, thereby introducing false and missed detections. To address these issues, LiDAR is often fused with cameras due to the rich textural information provided by images. However, current fusion methods suffer the inefficient data representation and inaccurate alignment of heterogeneous features, leading to poor precision and low efficiency. To this end, we propose a plug-and-play module termed range-image fusion (RI-Fusion) to achieve an effective fusion of LiDAR and camera data, designed to be easily accessible by existing mainstream LiDAR-based algorithms. In this process, we design an image and point cloud alignment method by converting a point cloud into a compact range-view representation through a spherical coordinate transformation. The range image is then integrated with a corresponding camera image utilizing an attention mechanism. The original range image is then concatenated with fusion features to retain point cloud information, and the results are projected onto a spatial point cloud. Finally, the feature-enhanced point cloud can be input into a LiDAR-based 3D object detector. The results of validation experiments involving the KITTI 3D object detection benchmark showed that our proposed fusion method significantly enhanced multiple mainstream LiDAR-based 3D object detectors, PointPillars, SECOND, and Part <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}{^{2}}$ </tex-math></inline-formula> , improving the 3D mAP (mean Average Precision) by 3.61%, 2.98%, and 1.27%, respectively, particularly for small objects such as pedestrians and cyclists.

Abstract.In pushbroom-style train imaging systems, the efficiency and accuracy of image alignment are crucial for improving train fault detection accuracy. However, nonlinear scale distortion in linear array images poses significant challenges to alignment precision. To address this, our study introduces an innovative image alignment algorithm for linear arrays, adept at handling nonlinear scale distortions. This algorithm is particularly effective in aligning heterogeneous images, even with substantial differences in texture features. The developed dynamic step-length sliding window strategy, feature point matching using geometric constraints, polynomial-constrained outlier elimination, and interval feature matching fusion significantly enhance both the accuracy and density of feature point matching. Further, the application of weighted radial basis function (WRBF) facilitates precise coordinate transformation in the image remapping process. Comprehensive experimental evaluations demonstrate the algorithm’s superior alignment precision and efficiency in both homogenous and heterogeneous image alignment scenarios, markedly boosting train fault detection accuracy. The algorithm’s versatility extends its utility beyond train fault detection to broader applications in pushbroom-style imaging system alignment tasks.

Markov random field based automatic image alignment for electron tomography

A domain alignment-based hyperspectral image (HSI) classification method was designed to address the heterogeneity in resolution and band between the source domain and target domain datasets of cross-scene hyperspectral images, as well as the resulting reduction in common features. Firstly, after preliminary feature extraction, perform two domain alignment operations: image alignment and distribution alignment. Image alignment aims to align hyperspectral images of different bands or time points, ensuring that they are within the same spatial reference framework. Distribution alignment adjusts the distribution of features of samples of different categories in the feature space to reduce the distribution differences of the same type of features between two domains. Secondly, adjust the consistency of the two alignment methods to ensure that the features obtained through different alignment methods exhibit consistency in the feature space, thereby improving the comparability and reliability of the features. In addition, this method considers multiple losses in the model from different perspectives and makes comprehensive adjustments through a unified optimization process to more comprehensively capture and utilize the correlation information between data. Experimental results on Houston 2013 and Houston 2018 datasets can improve the hyperspectral prediction performance between datasets with different resolutions and bands, effectively solving the problems of high cost and limited training samples in HSI labeling and significantly improving cross-scene HSI classification performance.

Cryo-electron microscopy (cryo-EM) single-particle image analysis is a powerful technique to resolve structures of biomacromolecules, while the challenge is that the cryo-EM image is of a low signal-to-noise ratio. For both two-dimensional image analysis and three-dimensional density map analysis, image alignment is an important step to improve the precision of the image distance calculation. In this paper, we introduce a new algorithm for performing two-dimensional pairwise alignment for cryo-EM particle images, which is based on the Fourier transform and power spectrum analysis. Compared to the existing heuristic iterative alignment methods, our method utilizes the signal distribution and signal feature on images' power spectrum to directly compute the alignment parameters. It does not require iterative computations and is robust against the cryo-EM image noise. Both theoretical analysis and experimental results suggest that our power-spectrum-feature-based alignment method is highly computational-efficient and is capable of offering effective alignment results. This new alignment algorithm is publicly available at: www.csbio.sjtu.edu.cn/bioinf/EMAF/for academic use.

Thermal images produced by the long-wavelength infrared (LWIR) camera are robust and independent from environmental illumination change. They can help the standard visible light camera working under the complicated environmental condition. Breaking through the traditional stereo multi-spectral sensor consisting of a visible-light camera and a LWIR camera, a novel architecture of large field of view (FOV) cooperated infrared and visible spectral sensor for visual odometry is proposed. The novel sensor is equipped with two visible cameras, four infrared cameras covering 120 degrees FOV in horizontal under both bands. Distribution of cameras and related peripheral devices are specifically designed which makes the sensor's volume less than 100 cm (length) × 10 cm (height) × 10 cm (width). The sensor's cameras calibration, distortion correction and measurement principle are elaborated. Feature-based method for visible and multi-windowed optimization-based image alignment for infrared is designed for the visual odometer based on the different imaging mechanism and distribution of cameras in the sensor. The frames and estimated poses management from both bands are proposed. Moreover, all proposed methodologies can be implemented in the sensor's embedded processor. The electrical power consumption is only 12W. Experiments of the sensor's evaluation are performed, experimental results show that large FOV cooperated multi-spectral cameras can efficiently improve the robustness of visual odometry. The real-time performance of the sensor is higher than 10fps with disparity map construction under both bands.

Efficiency of 2D alignment methods