Influence of Spectral Bands on Satellite Image Classification Using Vision Transformers

Identifying vegetation from satellite images is very necessary for studies related to agricultural sector. Enhancing the quality and classification of satellite images is a challenge especially when study areas contain complex vegetation. The study aimed to enhance the image classification based on using extraction tool of the ArcGIS. In the present study, the classification of Landsat satellite images before and after extracting specific areas covering vegetation using the ArcGIS was compared. Two regions in Egypt, one with complex vegetation located at Siwa Oasis and the other one with simple vegetation located at Abu Simbel region, were used in this study. Then, several polygons were used to extract specific areas covering vegetation from the satellite images. Images were subsequently classified using Iso cluster unsupervised classification tool. The classification outcomes were compared between the original images before extraction with images after extraction. The results showed that extraction tool greatly enhanced the quality and classification of the image and it was possible to identify different types of vegetation.

Land cover is an important variable of the terrestrial ecosystem that provides information for natural resources management, urban sprawl detection, and environment research. To classify land cover with high-spatial-resolution multispectral remote sensing imagery is a difficult problem due to heterogeneous spectral values of the same object on the ground. Fully convolutional networks (FCNs) are a state-of-the-art method that has been increasingly used in image segmentation and classification. However, a systematic quantitative comparison of FCNs on high-spatial-multispectral remote imagery was not yet performed. In this paper, we adopted the three FCNs (FCN-8s, Segnet, and Unet) for Gaofen-2 (GF2) satellite imagery classification. Two scenes of GF2 with a total of 3329 polygon samples were used in the study area and a systematic quantitative comparison of FCNs was conducted with red, green, blue (RGB) and RGB+near infrared (NIR) inputs for GF2 satellite imagery. The results showed that: (1) The FCN methods perform well in land cover classification with GF2 imagery, and yet, different FCNs architectures exhibited different results in mapping accuracy. The FCN-8s model performed best among the Segnet and Unet architectures due to the multiscale feature channels in the upsampling stage. Averaged across the models, the overall accuracy (OA) and Kappa coefficient (Kappa) were 5% and 0.06 higher, respectively, in FCN-8s when compared with the other two models. (2) High-spatial-resolution remote sensing imagery with RGB+NIR bands performed better than RGB input at mapping land cover, and yet the advantage was limited; the OA and Kappa only increased an average of 0.4% and 0.01 in the RGB+NIR bands. (3) The GF2 imagery provided an encouraging result in estimating land cover based on the FCN-8s method, which can be exploited for large-scale land cover mapping in the future.

Recently, there has been a remarkable growth in Artificial Intelligence (AI) with the development of efficient AI models and high-power computational resources for processing complex datasets. There has been a growing number of applications of machine learning in satellite remote sensing image data processing. In this work, machine learning methods were applied for crop classification of temporal multi- spectral satellite image to achieve better prediction of crop-wise area statistics. In India, agriculture has a huge impact on the national economy and most of the critical decisions are dependent on agricultural statistics. Sentinel-2 satellite image data for the Guntur district region of Andhra Pradesh, India has been used as the study area. The main reason for selecting this region is the diversity of agricultural crops and the availability of ground truth. As a baseline, the performance of machine learning models like Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), and K-Nearest Neighbor (KNN) on crop classification was evaluated on 2016 single time multi-spectral satellite image. SVM performed well with an F1 score of 0.94. Further, the performance of SVM, RF, 1D and 2D Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) and RNN with Gated Recurrent Unit (GRU) on 2018 Kharif season temporal multi-spectral satellite image has been evaluated for estimation of crop areas. The results show that SVM has the best F1 score of 0.99 and achieved a 95.9% agreement with the ground surveyed crop areas.

The attack that occurred recently involved the utilization of malicious software, commonly referred to as malware, along with advanced techniques such as machine learning, specifically deep learning, code transformation, and polymorphism. This makes it harder for cyber experts to detect malware using traditional analysis methods. In view of the low accuracy and high false positive rate of traditional malware detection methods, this research proposes a fine-tuned deep learning model with a novel dataset that is compared with ANN, Support Vector Machine (SVM), random forest (RF), K-Nearest Neighbourhood (KNN) classifiers. Converting a malware code into an image could allow users to effectively identify the presence of malware, even if the original code is modified by the creator. This is due to the fact that the attributes of images remain unchanged, allowing for reliable identification. So, researchers used deep learning technology to detect malware, like detecting malaria from red blood cells. The deep learning model found that a more detailed analysis of malware data sets, focusing on RGB and greyscale images, is needed. These data sets currently rely on publicly available data, but the accuracy of the traditional model could be a lot higher. It also produces many false positive results. The main goal is to create a new data set and model using malware images to identify and categorize malware using deep learning without relying on existing image detection and transfer learning models. The researcher adjusted different hyper parameters, like the number of neurons, filters, stride, hidden units, layers, learning rate, batch size, activation function, optimizer, and epochs. Identifying and correcting issues in models, improving their clarity, stability, and fairness, as well as debugging and monitoring them, is a challenging task. To eliminate this obstacle, assess the model's behaviour and performance by employing various methods such as logging, profiling, testing, visualization, and model clarity. To overcome the challenges posed by the electricity shutdown, we utilized Google's cloud-based GPU and Python 3.7 language to conduct the experiment and train the model. Kali Linux is an operating system that can automatically encrypt file systems and has a lower chance of crashing the system due to malware in a virtual sandbox. The researcher used techniques like early stopping and cross-validation to prevent over fitting and assess generalization in addition to monitoring and evaluating the model's behaviour and performance. The researcher used different methods like L1 and L2 regularization, dropout, and batch normalization to improve the model's performance and avoid over fitting. The binary portable executable (PE) malware dataset is collected from Kaggle, Malimg, Virusshare, Malvis, MS Big2015, and VX-underground and finally converted to greyscale and RGB images to create a novel dataset to fill the lack of image dataset. The raw dataset was then rescaled to a 128x128 greyscale and RGB (red, green, blue) image and flattened to 1024-byte vector images input into convolution neural network (CNN) interpolation to extract features for malware detection. The newly acquired dataset is utilized as an input for the innovative DL algorithms in order to create a tailor-made model capable of accurately predicting malware. The findings in this customized CNN model by fine-tuning hyper-parameters with the three-channel RGB dataset outperformed a greyscale dataset with an accuracy of 98.7% and error rate of 1.3% in current malware compared with other artificial neural network (ANN), ML algorithms such as Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and also Random Forest (RF) models. The proposed approach is compatible with all operating systems (Windows, Linux, and Mac) and can identify different types of malwares, such as packed, polymorphic, obfuscated, metamorphic, or variations of a malware family. There are some limitations to the shortage of malware image datasets and computational costs for fine-tuning hyper parameters in the whole model. Furthermore, the proposed approach detects malware and groups it into families without using common techniques like disassembly, decompilation, de-obfuscation, or running malicious code in a virtual environment.

Classification is a crucial stage in the processing of satellite images that influence considerably the quality of the result. A variety of methods is proposed in the literature for the purposes of image classification. They present many differences in their basic principles, thus in the quality of the results obtained. Therefore, a study of different classification methods seems to be essential. The classification of satellite images with conventional methods can be done in several ways using different algorithms. These algorithms can be divided into two main categories: supervised and non-supervised. Decision tree on the contrary is a machine learning tool. It is a plain model characterized by the simplicity of understanding and interpretation. This work aims firstly, to classify a high resolution Quickbird satellite image of an urban area by the decision tree method and compare it with the conventional classification algorithms in order to evaluate its efficiency. The methodology consists of two main stages: classification and evaluation of results. The second is based on the calculation of a number of statistical indices derived from the confusion matrix: the statistical parameter “kappa’ and the overall coefficient of precision.

A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task

In image classification tasks, commonly used deep learning network architectures include Convolutional Neural Network (CNN) and Vision Transformer (ViT). Still, both are relatively mature, while Graph Neural Network (GNN) network architecture has been attempted less in image classification. In addition, the architecture and design of GNN are still rapidly developing and have enormous potential. This study aims to create an image classification method based on superpixels and graph neural networks. This project attempted three different approaches to preserve superpixel node features with varying numbers of features, designed two training network architectures with different complexities and conducted six sets of tests on image classification tasks on the fashion-mnist dataset. As a result, the use of superpixel and GNN methods achieved good accuracy in image classification tasks, demonstrating the potential of this method in image recognition tasks. It was also found that adding the boundary coordinates of the region to the training in the selection of superpixel node features can improve the accuracy of the final image classification task.

Mackenzie Rive located in Yukon and Northwest Territories is the longest river system in Canada and an important transportation link. During the ice free season it is used for shipping transportation, while during freeze up time it serves as an ice road for trucks. Therefore, knowledge of ice conditions is essential to enable save navigation. Due to the difficulty of in-situ observation, remote sensing offers an effective instrument for river ice measurements. SAR data thereby enables gap free time series as it is an active sensor independent from sun illumination and cloud conditions. However, separating ice and open water in SAR data remains a challenging task due to the principals of the radar signal. On the one hand it is sensitive to surface roughness influencing weather conditions such as wind and rain affecting the appearance of water. On the other hand, ice shows different backscatter characteristics during freeze up and melting period. Therefore threshold or cluster based approaches encounter their limits in separating the features spaces of both classes demanding a more complex approach to cover these diverse patterns in the SAR image.
\nIn this study a Convolutional Neural Network (CNN) and TerraSAR-X data is used to map open water and ice in a time series between January 2014 and December 2015. Convolutional Neural Networks have proven great potential in classification of satellite images. Not focusing on isolated pixel values but regarding larger receptive fields they are also taking into account texture and shape information. Thus they are suitable to define feature spaces for ice and water.
\nThe test site covers the Mackenzie River Delta at the estuary of the Arctic Ocean. A U-Net with 18 convolutional layers and skip connections was used for training and classification. The four Kennaugh elements (K0 total intensity, K3 double/single bounce, K4 polarization, K7 torsion) were calculated from a total of 47 dual polarized HH/VV TerraSAR-X scenes. 42 of the 47 scenes show complete ice free respectively freeze up conditions and were used to train and validate the network with a total of 4000 patches (3000 for training and 1000 for validation). The remaining 5 scenes show transition conditions during the melt or freeze up time in spring or autumn and were used to test the neural network after training phase.
\nThe capability of the network to recognize class specific features in the SAR data is illustrated by visualizing the trained filters of the network showing typical SAR image structures of ice and water. Limitations of the approach occur for hybrid forms of water and ice that show features of both classes.The model was driven to best performance by adjusting hyper parameters including patch size, batch size, number of epochs, number of filters and filter size after each training run. Obtained results for the test images prove that a combination of a CNN and TerraSAR-X data is able to reliably separate open water from ice.

The classification of images has attracted significant attention due to its applications in several computer vision tasks, including satellite imaging, image analysis, surveillance, object recognition, and image retrieval. The primary objective of image classification is to provide class labels to images based on their content. The applications of imagery classification and analysis in remote sensing are significant since they are utilized in many sectors, including military and civilian areas. In satellite imagery, the challenge of image classification is heightened due to the rotation of objects inside a view and the typically varied background. This review paper discusses satellite image classification through various deep learning approaches, including its historical background and current approaches. It finds the most appropriate classification techniques for different types of satellite images and discusses the impact of preprocessing methods on classification accuracy. Moreover, a comparative analysis of several studies is given based on different parameters such as research objectives, used datasets, methods/algorithms, limitations, etc. Specific case studies that have significantly contributed to the understanding of satellite image classification are also discussed; further, an overview of challenges and future research directions in this field is also provided.

Deep Learning, and most notable Deep Neural Networks, have largely driven Artificial Intelligence in the area of remote sensing, mainly image classification tasks. In this paper, we present an approach based on Convolutional Neural Networks to classify Earth Observation satellite images as environmental preserved or non-preserved areas. One interesting feature of our approach is the fact that we used sensors with different spatial resolutions to assess the performance of a traditional network. We relied on images from the Tocantins Cerrado obtained by the Wide-Scan Multispectral and Panchromatic Camera of the CBERS-4A satellite, with a spatial resolution of 8m to create the training dataset. For testing, we set up a set of images of the Sentinel satellite, with a spatial resolution of 10m from Goias Cerrado. Results imply that Convolutional Neural Networks are feasible and are a good alternative for classifying remote sensing areas even when dealing with images from various sensors, and also with different spatial resolutions, where the model used in this study obtained an accuracy of 0.87. This study demonstrates the flexibility of Convolutional Neural Networks concerning the ability to generalize knowledge for classifying remote sensing images.

Land cover and its dynamic information is the basis for characterizing surface conditions, supporting land resource management and optimization, and assessing the impacts of climate change and human activities. In land cover information extraction, the traditional convolutional neural network (CNN) method has several problems, such as the inability to be applied to multispectral and hyperspectral satellite imagery, the weak generalization ability of the model and the difficulty of automating the construction of a training database. To solve these problems, this study proposes a new type of deep convolutional neural network based on Landsat-8 Operational Land Imager (OLI) imagery. The network integrates cascaded cross-channel parametric pooling and average pooling layer, applies a hierarchical sampling strategy to realize the automatic construction of the training dataset, determines the technical scheme of model-related parameters, and finally performs the automatic classification of remote sensing images. This study used the new type of deep convolutional neural network to extract land cover information from Qinhuangdao City, Hebei Province, and compared the experimental results with those obtained by traditional methods. The results show that: (1) The proposed deep convolutional neural network (DCNN) model can automatically construct the training dataset and classify images. This model performs the classification of multispectral and hyperspectral satellite images using deep neural networks, which improves the generalization ability of the model and simplifies the application of the model. (2) The proposed DCNN model provides the best classification results in the Qinhuangdao area. The overall accuracy of the land cover data obtained is 82.0%, and the kappa coefficient is 0.76. The overall accuracy is improved by 5% and 14% compared to the support vector machine method and the maximum likelihood classification method, respectively.

Most popular image classification methods mainly focus on classification ability rather than recognizing new things. However, human lay emphasis on cognition first and then classification, which is closely related to human memory system. Though many memory models have been proposed, they are studied in word list whereas the reports about natural images are still limited. This paper proposes a memory model for image recognition and classification based on convolutional neural network and Bayesian decision. First the image feature is extracted by convolutional neural network and stored in binary form. Then the representation, storage and retrieval processes of visual images are modeled. The test image feature vector is matched in parallel to the studied image vectors, and the likelihood values are calculated. Finally, the odd that the test image belongs to a new class is computed based on all likelihood values. If the odd value is greater than a certain threshold, the test image is regarded as new; otherwise, the Bayesian decision rule for image classification is performed. Experimental results on Caltech-101 and Caltech-256 datasets show that the proposed method can perform well in image recognition and classification tasks. And the hit probability of the method is higher than two typical methods, SRC and ELM, at present while the false alarm rate is far lower than them.

The report presents an algorithm for constructing a convolutional hierarchical neural network classifier, which is a modification of the algorithm for constructing hierarchical neural network classifiers suggested before. The original algorithm was designed to exploit intrinsic class hierarchy to build a class tree with a neural network in each node classifying groups of initial classes (in a non-terminal node) or a subset of original classes (in a terminal node). The convolutional modification utilizes convolutional neural networks instead of regular fully connected networks in order to apply the model to image classification tasks. Use of class hierarchy for image classification should reduce the number of adjusted neural network parameters compared to deep convolutional neural networks, and therefore it should reduce training and inference time. In this context the algorithm may be compared with some pruning techniques. The convolutional hierarchical neural network classifier inherits some hyperparameters of a conventional hierarchical neural network classifier, like the activation threshold and the threshold by the share of voting patterns. The goal of this study was to explore different strategies of choosing these hyperparameters. To test these strategies, we used the CIFAR-10 dataset. Also, for demonstration purposes we apply the convolutional hierarchical neural network classifier to the CIFAR-100 dataset.

Remote sensing techniques have been widely used for identification of land use and land cover features. Land information can be easily collected by classification of satellite images in the context of their use. In this paper study area has been classified into three classes i.e. settlement, trees and agricultural by classification of an image which has been enhanced using fusion of two images. The spatial and spectral resolutions of different satellite images provide better information with the aid of initial processing of image and fusion of both images. The satellite images fused together are multispectral IRS-P6 also called Resourcesat-1 satellite, on board LISS-III sensor provide image with spatial resolution of 23.5 m and an IRS-P5 also called Cartosat-1 satellite provides single band panchromatic image with spatial resolution of 2.5 m. Erdas Imagine 9.1 software has been used for image processing, fusion and supervised classification of the images. The Brovery, Multiplicative and Principal Component Analysis (PCA) method have been used for image fusion. The resultant images have been classified using the supervised classification with maximum likelihood parametric rule for information extraction and comparison between them in terms of their accuracy.

Satellite image classification plays a crucial role in applications such as land cover mapping, urban planning, and environmental monitoring. This project focuses on the development of SpectrumNet, a custom Convolutional Neural Network (CNN) designed to classify satellite images from the EuroSAT dataset into ten distinct land-use categories. The primary goal is to improve the accuracy and efficiency of satellite image classification through the implementation of advanced preprocessing techniques, optimized training strategies, and multiple network configurations. By utilizing deep learning methodologies, SpectrumNet aims to provide reliable insights for real-world applications in remote sensing, geospatial analysis, and environmental studies. The project strives to enhance the ability to analyze and interpret large-scale satellite image data, offering valuable information for decision-making in various fields. Key Words: Deep Learning, Convolutional Neural Networks, EuroSAT, SpectrumNet, Geospatial Analysis, Land Cover Mapping, Environmental Monitoring, Urban Planning.