GeoAI Dataset for Training Deep Learning-Based Optical Satellite Image Matching Model

The image navigation and registration (INR) performance assessment for the Geostationary Operational Environmental Satellite-R (GOES-R) Advanced Baseline Imager (ABI) is an image registration process at the subpixel level, which involves the registration of images from GOES-R ABI with corresponding images from Landsat at the same scene. We present an optimized image registration algorithm for GOES-R ABI INR performance assessment (GAIPA): a gradient descent algorithm with a Sobel edge-enhancement procedure and a normalized lookup table (NLUT) image transformation (GDSN), which is an area-based image registration approach with Pearson correlation as the similarity metric. Sobel edge enhancements increase the sensitivity of Pearson correlations that lead to much improved registration accuracy. NLUT transformations minimize radiance differences between GOES-R ABI images acquired at different times and Landsat reference images on image registration outcomes, which lead to improvements in correlation values and quality of image registration outputs. Evaluations of registration accuracy and quality metrics in GDSN show considerable improvements in computational efficiency, registration accuracy, and quality over the approach implemented on the image INR performance assessment tool set. Pearson correlations can be effective and robust similarity metrics in image registration at the subpixel level with implementing Sobel edge enhancement and NLUT transformation. The statistical analysis of the image registration outputs for GAIPA shows the new algorithm provides considerable improvements in reducing the uncertainties in the registration outputs.

Portal imaging technology: Past, present, and future

Synthetic Aperture Radar (SAR) image time series play an important role in many applications. To construct pixel-level SAR image time series, we propose a locally adaptive image matching technique for the high-precision geometric registration of SAR images. The basic idea is to adapt the local characteristics of ground objects during the process of image registration. Then, by analyzing the spatial distribution of the error of each matched pair in the previous iteration, local areas are divided based on the local clustering of pairs with large errors. A new polynomial is then used to satisfy the local geometric constraint. Based on this proposed matching technique, we introduce a pixel-level SAR image time series modeling method. The experimental results show that the average geometric error of corresponding pixels in this algorithm is 0.073 pixels, while that of the NEST software is 0.242 pixels. The Pearson correlation coefficients of 20 pixels’ time series are above 0.85, indicating that the series bears high curve similarity and geometric precision, which suggests the proposed technique can provide high-quality SAR image time series.

Recently, the classification of surface textures is carried out using various modelling approaches. To analyse the surface texture, most of the techniques uses large amount of training data which adds up to considerable computational cost. However, the implementation of various neural network models also requires significant amount of training images to classify surface textures. In the proposed paper, a deep learning-based model is presented using convolution neural network (CNN). Further, this model is divided into two sub models knowing model-1 and model-2. The approach is designed with customized parameters configuration to classify surface texture using a smaller number of training samples. The image feature vectors are generated using statistical operations to compute the physical appearance of the surface and a CNN model is used to classify the generated surfaces with appropriate labels into classes. The Kylberg Texture dataset is used to evaluate the proposed models using 16 texture classes. The advantage of proposed models over pre-trained networks is that the entire models is customized according to specific training requirements. Further, to demonstrate the state-of-the-art results, the proposed approach is compared with other existing techniques. Our experimental results are better than the conventional techniques and achieves an accuracy of 92.42% for model-1 and 96.36% for model-2. In addition, the proposed models maintain balance between accuracy and computational cost.

Image registration of satellite imagery is challenging task as the image pair may have been captured by different sensors, view angles or at different times. Conventional satellite image registration tasks are achieved by feature based techniques which are to be identified and matched before estimating transforms and resample the moving or slave image to a fixed or master image. However, automatic image registration is very important requirement for voluminous data sets. Even if feature selection made automatic, even distribution and the density of feature points is always not ensured. A good alternative is to switch over to intensity based image registration techniques which are somewhat image content independent. We bring out details on an intensity based multi temporal satellite image registration task with metric, transform and optimization components based method. An optimization technique employed to find optimal transform parameters by maximizing the chosen similarity measure criteria, provides a robust image registration framework. Adaptive Stochastic Gradient Descent optimization (ASGD) for satellite image registration is a promising new technique, details of which are reported here.

As people's material living standards continue to improve, the types and quantities of household garbage they generate rapidly increase. Therefore, it is urgent to develop a reasonable and effective method for garbage classification. This is important for resource recycling and environmental improvement and contributes to the sustainable development of production and the economy. However, existing deep learning-based garbage image classification models generally suffer from low classification accuracy, insufficient robustness, and slow detection speed due to the large number of model parameters. To this end, a new garbage image classification model is proposed, with the ResNet-50 network as the core architecture. Specifically, first, a redundancy-weighted feature fusion module is proposed, enabling the model to fully leverage valuable feature information, thereby improving its performance. At the same time, the module filters out redundant information from multi-scale features, reducing the number of model parameters. Second, the standard 3×3 convolutions in ResNet-50 are replaced with depth-separable convolutions, significantly improving the model's computational efficiency while preserving the feature extraction capability of the original convolutional structure. Finally, to address the issue of class imbalance, a weighting factor is added to the Focal Loss, aiming to mitigate the negative impact of class imbalance on model performance and enhance the model's robustness. Experimental results on the TrashNet dataset show that the proposed model effectively reduces the number of parameters, improves detection speed, and achieves an accuracy of 94.13%, surpassing the vast majority of existing deep learning-based waste image classification models, demonstrating its solid practical value.

Offline signature verification (OfSV) is essential in preventing the falsification of documents. Deep learning (DL) based OfSVs require a high number of signature images to attain acceptable performance. However, a limited number of signature samples are available to train these models in a real-world scenario. Several researchers have proposed models to augment new signature images by applying various transformations. Others, on the other hand, have used human neuromotor and cognitive-inspired augmentation models to address the demand for more signature samples. Hence, augmenting a sufficient number of signatures with variations is still a challenging task. This study proposed OffSig-SinGAN: a deep learning-based image augmentation model to address the limited number of signatures problem on offline signature verification. The proposed model is capable of augmenting better quality signatures with diversity from a single signature image only. It is empirically evaluated on widely used public datasets; GPDSsyntheticSignature. The quality of augmented signature images is assessed using four metrics like pixel-by-pixel difference, peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and frechet inception distance (FID). Furthermore, various experiments were organised to evaluate the proposed image augmentation model’s performance on selected DL-based OfSV systems and to prove whether it helped to improve the verification accuracy rate. Experiment results showed that the proposed augmentation model performed better on the GPDSsyntheticSignature dataset than other augmentation methods. The improved verification accuracy rate of the selected DL-based OfSV system proved the effectiveness of the proposed augmentation model.

A medical image registration simulation toolkit built using MATLAB graphical user interface (GUI) is presented. The developed Matlab GUI's provide an effective tool for exploring different image registration techniques including a new feature‐based image registration technique proposed by the authors. The toolkit is useful for biomedical engineering students and can be used as a web‐based learning tool. © 2009 Wiley Periodicals, Inc. Comput Appl Eng Educ 18: 225–237, 2010; Published online in Wiley InterScience (www.interscience.wiley.com); DOI 10.1002/cae.20179

To identify the feasibility of deep learning-based diagnostic models for detecting and assessing lower-extremity fatigue fracture severity on plain radiographs. This retrospective study enrolled 1151 X-ray images (tibiofibula/foot: 682/469) of fatigue fractures and 2842 X-ray images (tibiofibula/foot: 2000/842) without abnormal presentations from two clinical centers. After labeling the lesions, images in a center (tibiofibula/foot: 2539/1180) were allocated at 7:1:2 for model construction, and the remaining images from another center (tibiofibula/foot: 143/131) for external validation. A ResNet-50 and a triplet branch network were adopted to construct diagnostic models for detecting and grading. The performances of detection models were evaluated with sensitivity, specificity, and area under the receiver operating characteristic curve (AUC), while grading models were evaluated with accuracy by confusion matrix. Visual estimations by radiologists were performed for comparisons with models. For the detection model on tibiofibula, a sensitivity of 95.4%/85.5%, a specificity of 80.1%/77.0%, and an AUC of 0.965/0.877 were achieved in the internal testing/external validation set. The detection model on foot reached a sensitivity of 96.4%/90.8%, a specificity of 76.0%/66.7%, and an AUC of 0.947/0.911. The detection models showed superior performance to the junior radiologist, comparable to the intermediate or senior radiologist. The overall accuracy of the diagnostic model was 78.5%/62.9% for tibiofibula and 74.7%/61.1% for foot in the internal testing/external validation set. The deep learning-based models could be applied to the radiological diagnosis of plain radiographs for assisting in the detection and grading of fatigue fractures on tibiofibula and foot. • Fatigue fractures on radiographs are relatively difficult to detect, and apt to be misdiagnosed. • Detection and grading models based on deep learning were constructed on a large cohort of radiographs with lower-extremity fatigue fractures. • The detection model with high sensitivity would help to reduce the misdiagnosis of lower-extremity fatigue fractures.

The violence detection is mostly achieved through handcrafted feature descriptors, while some researchers have also employed deep learning-based representation models for violent activity recognition. Deep learning-based models have achieved encouraging results for fight activity recognition on benchmark data sets such as hockey and movies. However, these models have limitations in learning discriminating features for violence activity classification with abrupt camera motion. This research work investigated deep representation models using transfer learning for handling the issue of abrupt camera motion. Consequently, a novel deep multi-net (DMN) architecture based on AlexNet and GoogleNet is proposed for violence detection in videos. AlexNet and GoogleNet are top-ranked pre-trained models for image classification with distinct pre-learnt potential features. The fusion of these models can yield superior performance. The proposed DMN unleashed the integrated potential by concurrently coalescing both networks. The results confirmed that DMN outperformed state-of-the-art methods by learning finest discriminating features and achieved 99.82% and 100% accuracy on hockey and movies data sets, respectively. Moreover, DMN has faster learning capability i.e. 1.33 and 2.28 times faster than AlexNet and GoogleNet, which makes it an effective learning architecture on images and videos.

To increase the productivity, it is important to manage yield and reduce defects in the semiconductor industry. One of the efforts is to identify defect patterns and control the cause factors that affects the defects. Many engineers inspect the quality of each chip and check the defect pattern on the wafer bin maps. To get the accurate and consistent classification results regardless of the level for domain knowledge or experience of engineers, deep learning-based models have recently been studied. Since most previous studies aim to classify the single-type defect patterns, it is needed to consider the mixed-type defect patterns together. Also, they require a lot of labeled data to train the deep learning-based classification model. However, defects occur extremely rarely in actual manufacturing process. Therefore, the method securing the higher accuracy in a situation where enough labeled data are not given is needed. This paper proposes a deep convolutional generative adversarial network for wafer map synthesis (DCGAN-WS) which generates the mixed-type patterns by synthesizing the single-type pattern and adding the pixel-wise summation. To maintain the characteristics of the binary pixel of the wafer bin maps, a thresholding technique is added. MixedWM38 dataset is used for the experiments, and it was verified that the mixed-type patterns were synthesized well. It helps to construct more robust model for single-type pattern classification and to generate the mixed-type patterns that have not occurred before. In the future, it is expected that this model addresses the problem of the lack of labeled data for defect pattern classification models.

Accurate and effective classification of plant diseases is critical for increasing yield and quality in agricultural production, minimizing economic losses through early detection of diseases, and implementing sustainable agriculture approaches. In this study, an approach for the detection and classification of plant leaf diseases is presented. We compare the performance of machine learning and deep learning-based models, and we use GAN-based data synthesis methods on a dataset we created to improve the model performance. For machine learning methods, ResNet-based feature extraction is performed and XGBoost, Random Forest, SVM, and InceptionV3 models are evaluated. In contrast, AlexNet, VGG16, VGG19, DenseNet, and ResNet models are examined within the scope of deep learning. The study was analyzed in three classes Phytophthora Infestans, Potassium Deficiency, and Healthy, and tested on data obtained from 21 different plant species. According to the model performances obtained, the deep learning-based ResNet model showed the highest success in all performance metrics and achieved 99% accuracy, showing superior performance compared to other methods. In the study, a comprehensive evaluation of multiple classification, GAN-based data synthesis, machine learning, and deep learning models was carried out. A valuable contribution was made to the existing studies in the literature.

Fast Deformable Image Registration for Real-Time Target Tracking During Radiation Therapy Using Cine MRI and Deep Learning

Purpose:_To enhance the automatic image registration process for Image Guided Radiation Therapy (IGRT) using biological information acquired by PET imaging. Methods and Materials: We developed an image registration tool which is incorporated with a three‐dimensional semi‐automatic segmentation tool for Positron Emission Tomography (PET) image set. Firstly we registered PET image set to the planning CT image set. Contours for the high uptake volumes were generated by a semi‐automatic segmentation tool. User can choose an intensity profile with a pointing device in Graphical User Interface. Then, region growing algorithm was utilized. Threshold for segmentation can be adjusted in GUI. We exported the segmented volumes and fused into a DICOM‐RT structure set generated in a planning CT. We formulated Spatially Weighted Mutual Information (SWMI) image registration with Structure‐Of‐Interest (SOI) based weight function (SWMI‐SOI) because assigning various importance weight values to geometric locations is not possible with mutual information image registration. SWMI method allows the user can assign the importance weights through the geometric space. A user can choose how much importance weight will be assigned to each SOI. We expanded the organ contours so that each SOI included enough image features such as organ edges. We assigned the higher importance value to Clinical Target Volumes (CTV) and the Organs‐At‐Risks (OAR). The highest importance value was assigned to the volumes which were delineated using PET segmentation tool. Lower weight values were assigned to other structures so that they would not dominate the registration. This insures that PET defined volumes will get a full treatment dose in every fraction. Results and Discussion: We successfully used biological information in IGRT process and SWMI‐SOI registration algorithm. Our method showed consistency between users and was very useful when multiple lesions were considered during image registration process. Further clinical research is necessary to determine optimal weight assignment.

This paper presents a robust image registration technique for creating smooth scene transitions for a Web-based virtual tour system. Easy and effective virtual tour (EEVT). EEVT has advantages over traditional geometry-based 3D rendering approach in that it does not require labor intensive 3D modeling process or high bandwidth for realistic virtual tour yet provides a virtual tour with free navigation and immersive experience through the WWW. EEVT constructs virtual tour from a set of images. It uses several snap shots of conventional photos without special tools, builds a simple 3D space within each photo using the spidery mesh technique, and expands the virtual spaces by connecting each space together. The connection between images is achieved by image registration, which finds correspondences automatically and estimates transformations. The image registration process is crucial for virtual tour applications in order to compose smooth transitional scenes between two views so that virtual tourists perceive continuous scenes during navigation. Our registration method uses a parametric approach and it includes the following key features for robust and accurate computation: 1) coarse-to-fine hierarchical estimation; 2) fast computation based on image feature-correspondences; 3) FFT-based global matching, 4) automatic outlier removal by RANSAC. The expanded virtual space creates a sense of navigational freedom for virtual tourists with less distorted viewing.