Hierarchical grid-constrained fusion network for image stitching

Analysis on topological features of deterministic hierarchical complex network

Retinopathy of prematurity (ROP) is a retinal vascular disease that primarily affects premature infants with low birth weight. It is a leading cause of childhood blindness worldwide, but it can often be effectively managed with appropriate and timely diagnosis and treatment. To address the impact of image style on model classification performance, this paper proposes a dual-scale Swin Transformer (DS-Swin-T) network for ROP. The network comprises three components: image synthesis (IS), feature alignment, and advanced adversarial learning. The IS module generates synthesis style images as an intermediate latent space between source and target styles, reducing style difference. The DS-Swin-T serves as the primary framework for image feature extraction. Detail and style encoders extract features in the shallow feature space, with detail and style losses aligning these features to ensure consistency across styles. To extract rich style-invariant features and ensure consistent classification within the same category, adversarial learning is applied in the advanced feature space. Finally, feature fusion units process dual-scale classification representations. Our method achieves an average accuracy of 97.91% on the source style dataset. When transferred to other target style datasets, our method effectively mitigates the performance degradation caused by style difference, reaching a maximum average accuracy of 93.66%. Extensive experiments demonstrate the effectiveness of our method.

Many researches have been carried out to explore the ultrastructure of organisms at home and abroad, and different algorithms have been used to reconstruct the images of different samples at the nanoscale. Image “matching” and “fusion” are two significant research fields that directly affect the performance of image Mosaic. As image stitching the first and the last step, if there is no correct image matching and fusion algorithm, it is almost impossible to successful image stitching. Image registration is to map one image to another by looking for a space transformation for two images in a set of image data, so that the points corresponding to the same position in the space in the two images correspond one to one, so as to achieve the purpose of information fusion. In this paper, the feature-based image stitching algorithm was used to transform the analysis of the whole image into the analysis of some features of the image, which greatly reduced the calculation amount. Meanwhile, the feature points were further optimized and selected to reduce the mismatching rate, and the electron microscope images of autistic mice were mosaically splicing. In the registration part, this paper adopts the elastic registration method and makes fine adjustments on this basis to obtain more accurate registration results, which will be helpful for the subsequent segmentation of each organizational structure.

Many applications of remotely sensed digital imagery require images that are corrected for geometric distortions. It is often desirable to rectify different types of satellite imagery to a common data base. A high throughput rectification system is required for commercial application. High geometric and radiometric precision must be maintained. The thesis has accomplished the following tasks: \begin{enumerate} \item The sensors used to obtain remotely sensed imagery have been investigated and the associated geometric distortions inherent with each sensor are identified. \item The transformation between image coordinates and datum coordinates has been determined and the values of the parameters in the transformation are estimated. \item A unified rectification approach has been developed, for all types of remotely sensed digital imagery, which yields a high system throughput. \item Use of digital terrain models in the rectification process to correct for relief displacement has been incorporated. \item An efficient image interpolation algorithm has been developed. This algorithm takes into account the fact that imagery does not always correspond to sampling on a uniform grid. \item The applications of rectified imagery such as mosaicking and multisensor integration have been studied. \item Extension of the rectification algorithm to a future planetary mission has been investigated. \end{enumerate} The sensors studied include TIROS-N Landsat-1, -2 and -3 multispectral scanners, Seasat synthetic aperture radar, Landsat-4 thematic mapper and SPOT linear array detectors. Imagery from the last two sensors is simulated.

In order to reduce the occurrence of wild mushroom poisoning incidents, and at the same time reduce the impact of the complex background of wild mushroom pictures on the recognition accuracy, this paper uses the Squeeze-and-Excitation attention mechanism and feature pyramid to improve the ResNet50 network. First, in order to increase the correlation between channels, the Squeeze-and-Excitation attention mechanism is added to the residual block of the ResNet50 network. Second, the feature pyramid is used to fuse the features between different layers of the network. Next, send the lowest feature map which fused by FPN to the fully connected layer. At last, the final result is normalized by softmax function and classified. The experimental results show that the accuracy of the method can reach 95.97%, which is 2.71% higher than the unimproved ResNet50 network. The comparison results show that it is better than the three network models of VGG19, DenseNet161 and Iception_v3, the accuracy rates are increased by 6.40%, 6.31% and 2.28% respectively.

Image stitching by line-guided local warping with global similarity constraint

Efficient and rapid auxiliary diagnosis of different grades of lung adenocarcinoma is conducive to helping doctors accelerate individualized diagnosis and treatment processes, thus improving patient prognosis. Currently, there is often a problem of large intra-class differences and small inter-class differences between pathological images of lung adenocarcinoma tissues under different grades. If attention mechanisms such as Coordinate Attention (CA) are directly used for lung adenocarcinoma grading tasks, it is prone to excessive compression of feature information and overlooking the issue of information dependency within the same dimension. Therefore, we propose a Dimension Information Embedding Attention Network (DIEANet) for the task of lung adenocarcinoma grading. Specifically, we combine different pooling methods to automatically select local regions of key growth patterns such as lung adenocarcinoma cells, enhancing the model's focus on local information. Additionally, we employ an interactive fusion approach to concentrate feature information within the same dimension and across dimensions, thereby improving model performance. Extensive experiments have shown that under the condition of maintaining equal computational expenses, the accuracy of DIEANet with ResNet34 as the backbone reaches 88.19%, with an AUC of 96.61%, MCC of 81.71%, and Kappa of 81.16%. Compared to seven other attention mechanisms, it achieves state-of-the-art objective metrics. Additionally, it aligns more closely with the visual attention of pathology experts under subjective visual assessment.

Color correction can eliminate the color difference between similar images in image stitching and 3D video reconstruction. The result images generated by the local color correction algorithms usually show structure inconsistency problem with the input images. In order to solve this problem, we propose a structure consistent color correction algorithm for stereoscopic images based on gradient preservation. This method can not only eliminate color difference between reference and target images, but also optimize structure between the input target image and the result image. Firstly, the algorithm extracts the structure information of the target image and style information of the reference image using the SIFT algorithm and generates the structure image and the pixel matching image. Then an initial result image is generated by local pixel mapping. Finally the initial result image is iteratively optimized by the gradient preserving algorithm. The experimental results show that our algorithm can not only optimize the structure inconsistency, but also effectively process image pairs with large color difference.

Bladder endoscopy enables the observation of intravesical lesion characteristics, making it an essential tool in urology. Image stitching techniques are commonly employed to expand the field of view of bladder endoscopy. Traditional image stitching methods rely on feature matching. In recent years, deep-learning techniques have garnered significant attention in the field of computer vision. However, the commonly employed supervised learning approaches often require a substantial amount of labeled data, which can be challenging to acquire, especially in the context of medical data. To address this limitation, this study proposes an unsupervised neural network-based image stitching method for bladder endoscopy, which eliminates the need for labeled datasets. The method comprises two modules: an unsupervised alignment network and an unsupervised fusion network. In the unsupervised alignment network, we employed feature convolution, regression networks, and linear transformations to align images. In the unsupervised fusion network, we achieved image fusion from features to pixel by simultaneously eliminating artifacts and enhancing the resolution. Experiments demonstrated our method's consistent stitching success rate of 98.11% and robust image stitching accuracy at various resolutions. Our method eliminates sutures and flocculent debris from cystoscopy images, presenting good image smoothness while preserving rich textural features. Moreover, our method could successfully stitch challenging scenes such as dim and blurry scenes. Our application of unsupervised deep learning methods in the field of cystoscopy image stitching was successfully validated, laying the foundation for real-time panoramic stitching of bladder endoscopic video images. This advancement provides opportunities for the future development of computer-vision-assisted diagnostic systems for bladder cavities.

In general, the viewing experience of stereoscopic panoramas heavily relies on the accuracy of perceived depth information. However, most of existed stereoscopic panorama stitching methods always focus on the improvement, refinement, and correction during the image alignment or panorama stitching. The impairment to the depth information introduced by later steps is rarely under consideration. Thus, we propose one general post-stitching depth correction technique based on sparsely detected control points. The proposed depth correction can be divided into two steps. The global one aims at minimizing the average depth error by adjusting the relative poses of two panoramas. Then, the local step tries to relieve those regions-based depth issues with the morphing technique. For those regions that still suffer from visible depth error after the global correction, we select control points in those depth anomaly patches and utilize the Thin-Plate-Spline technique to warp those features into their target position. To demonstrate the robustness of the proposed depth adjustment technique, various experiments are conducted under different camera arrangements and scenarios. The result indicates that the stereoscopic panoramas after proposed depth adjustment could deliver more reasonable depth information and bring a better viewing experience.

At present, underwater terrain images are all strip-shaped small fragment images preprocessed by the side-scan sonar imaging system. However, the processed underwater terrain images have inconspicuous and few feature points. In order to better realize the stitching of underwater terrain images and solve the problems of slow traditional image stitching speed, we proposed an improved algorithm for underwater terrain image stitching based on spatial gradient feature block. First, the spatial gradient fuzzy C-Means algorithm is used to divide the underwater terrain image into feature blocks with the fusion of spatial gradient information. The accelerated-KAZE (A-KAZE) algorithm is used to combine the feature block information to match the reference image and the target image. Then, the random sample consensus (RANSAC) is applied to optimize the matching results. Finally, image fusion is performed with the global homography and the optimal seam-line method to improve the accuracy of image overlay fusion. The experimental results show that the proposed method in this paper effectively divides images into feature blocks by combining spatial information and gradient information, which not only solves the problem of stitching failure of underwater terrain images due to unobvious features, and further reduces the sensitivity to noise, but also effectively reduces the iterative calculation in the feature point matching process of the traditional method, and improves the stitching speed. Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image.

An Adaptive Multi-Feature Fusion Network for Predicting Overall Survival of Patients with Head and Neck Cancer

Breast-conserving surgery requires supportive radiotherapy to prevent cancer recurrence. However, the task of localizing the tumor bed to be irradiated is not trivial. The automatic image registration could significantly aid the tumor bed localization and lower the radiation dose delivered to the surrounding healthy tissues. This study proposes a novel image registration method dedicated to breast tumor bed localization addressing the problem of missing data due to tumor resection that may be applied to real-time radiotherapy planning. We propose a deep learning-based nonrigid image registration method based on a modified U-Net architecture. The algorithm works simultaneously on several image resolutions to handle large deformations. Moreover, we propose a dedicated volume penalty that introduces the medical knowledge about tumor resection into the registration process. The proposed method may be useful for improving real-time radiation therapy planning after the tumor resection and, thus, lower the surrounding healthy tissues’ irradiation. The data used in this study consist of 30 computed tomography scans acquired in patients with diagnosed breast cancer, before and after tumor surgery. The method is evaluated using the target registration error between manually annotated landmarks, the ratio of tumor volume, and the subjective visual assessment. We compare the proposed method to several other approaches and show that both the multilevel approach and the volume regularization improve the registration results. The mean target registration error is below 6.5 mm, and the relative volume ratio is close to zero. The registration time below 1 s enables the real-time processing. These results show improvements compared to the classical, iterative methods or other learning-based approaches that do not introduce the knowledge about tumor resection into the registration process. In future research, we plan to propose a method dedicated to automatic localization of missing regions that may be used to automatically segment tumors in the source image and scars in the target image.

A hierarchical attention detector for bearing surface defect detection

To reduce the chance of being infected by the COVID-19, wearing masks correctly when entering and leaving public places has become the most feasible and effective ways to prevent the spread of the virus. It is a concern to how to quickly and accurately detect whether a face is worn a mask correctly while reduce missed detection and false detection in practical applied scenarios. In this paper, an improved algorithm is proposed based on the YOLO-v4 algorithm. The attention mechanism module is added to the appropriate network level to enhance the key feature points of face wearing masks and suppress useless information. Apart from that, three attention mechanism modules are added to different layers of the YOLO-v4 network for ablation experiments, including CBAM (convolutional block attention module), SENet (squeeze-and-excitation networks) and CANet (coordinate attention networks). The path-aggregation network and feature pyramid are used to extract features from images. Two network models were compared and improved in the experiment, and it is found that adding the dual-channel attention mechanism CBAM before the three YOLO heads of YOLOv4 and in the neck network had better detection performance than the single channel attention mechanism SENet and the coordinated attention mechanism CANet. The experimental results show that when the attention module CBAM and the YOLO-v4 model are integrated, the accuracy of the selected MAFA + WIDER Face dataset reaches the highest value of 93.56%, which is 4.66% higher than that of the original YOLO-v4.