SatImNet: Structured and Harmonised Training Data for Enhanced Satellite Imagery Classification

Different scenarios can be found in land classification and segmentation of satellite images. First, when prior knowledge is available, the training data is generally selected by randomly picking samples within classes. When no prior knowledge is available the system can pick samples at random among all unlabeled data, which is highly unreliable or it can rely on the expert collaboration to improve progressively the training data applying an active learning function. We suggest a scheme to tackle the lack of prior knowledge without actively involving the expert, whose collaboration may be expensive. The proposed scheme uses a clustering technique to analyze the feature space and find the most representative samples for being labeled. In this case the expert is just involved in labeling once a reliable training data set for being representative of the feature space. Once the training set is labeled by the expert, different classifiers may be built to process the rest of samples. Three different approaches are presented in this paper: the result of the clustering process, a distance based classifier, and support vector machines (SVM).

Identifying vegetation from satellite images is very necessary for studies related to agricultural sector. Enhancing the quality and classification of satellite images is a challenge especially when study areas contain complex vegetation. The study aimed to enhance the image classification based on using extraction tool of the ArcGIS. In the present study, the classification of Landsat satellite images before and after extracting specific areas covering vegetation using the ArcGIS was compared. Two regions in Egypt, one with complex vegetation located at Siwa Oasis and the other one with simple vegetation located at Abu Simbel region, were used in this study. Then, several polygons were used to extract specific areas covering vegetation from the satellite images. Images were subsequently classified using Iso cluster unsupervised classification tool. The classification outcomes were compared between the original images before extraction with images after extraction. The results showed that extraction tool greatly enhanced the quality and classification of the image and it was possible to identify different types of vegetation.

Neural networks play an important role in satellite image classification. We know that the most common neural networks used in image classification tasks are convolutional neural networks (CNNs) (Boesch in Vision transformers (VIT) in image recognition—2022 guide, 2022). In this paper, we explored the influence of the spectral bands in image classification using the Vision Transformer (ViT). Convolution is a local operation, and a convolution layer typically models only the relationships between neighborhood pixels. Transformer is a global operation, and a transformer layer can model the relationships between all pixels. This motivated us to use ViT for satellite image classification. Sentinel-2 EuroSAT image dataset, which consists of 27,000 images in ten classes, is used for the experiment. ViT model is trained with three band dataset, Red-Green-Blue (RGB) and compared with ViT model trained with RGB along with Near InfraRed (NIR) and with multispectral satellite image dataset (13 bands). Experimental results shows that NIR band combined with RGB was able to produce more accurate results comparing to RGB alone, whereas 13 band dataset outperformed both RGB and RGB and NIR datasets.

Abstract. Deep learning models have demonstrated remarkable success in fields such as language processing and computer vision, routinely employed for tasks like language translation, image classification, and anomaly detection. Recent advancements in ocean sciences, particularly in data assimilation (DA), suggest that machine learning can emulate dynamical models, replace traditional DA steps to expedite processes, or serve as hybrid surrogate models to enhance forecasts. However, these studies often rely on ocean models of intermediate complexity, which involve significant simplifications that present challenges when transitioning to full-scale operational ocean models. This work explores the application of convolutional neural networks (CNNs) in data assimilation within the context of the HYbrid Coordinate Ocean Model (HYCOM) in the Gulf of Mexico. The CNNs are trained to correct model errors from a 2-year, high-resolution (1/25°) HYCOM dataset, assimilated using the Tendral Statistical Interpolation System (T-SIS). The CNNs are trained to replicate the increments generated by the T-SIS data assimilation package, aiming to correct model forecasts of sea surface temperature (SST) and sea surface height (SSH). The inputs to the CNNs include real satellite observations of SST from the Group for High Resolution Sea Surface Temperature (GHRSST), along-track altimeter SSH observations (ADT), the model background state (previous forecast), and the innovations (differences between observations and background). We assess the performance of the CNNs across five controlled experiments, designed to provide insights into their application in environments governed by full primitive equations, real observations, and complex topographies. The experiments focus on evaluating (1) the architecture and complexity of the CNNs, (2) the type and quantity of observations, (3) the type and number of assimilated fields, (4) the impact of training window size, and (5) the influence of coastal boundaries. Our findings reveal significant correlations between the chosen training window size – a factor not commonly examined – and the CNNs' ability to assimilate observations effectively. We also establish a clear link between the CNNs' architecture and complexity and their overall performance. This research uses artificial intelligence to enhance ocean forecasting in the Gulf of Mexico. By using convolutional neural networks, the study improves predictions of sea temperatures and heights by integrating real satellite data with existing models. Through five comprehensive experiments, the team found that the amount of training data and the design of the neural networks significantly affect accuracy. These insights pave the way for faster, more reliable ocean models, benefiting environmental monitoring and maritime operations.

Convolutional Neural Networks (CNNs) are highly effective for image reconstruction problems. Typically, CNNs are trained on large amounts of training images. Recently, however, un-trained CNNs such as the Deep Image Prior and Deep Decoder have achieved excellent performance for image reconstruction problems such as denoising and inpainting, without using any training data —except leveraging a few samples for hyper-parameter tuning. Motivated by this development, we address the reconstruction problem arising in accelerated MRI with un-trained neural networks. We propose a highly optimized un-trained recovery approach based on a variation of the Deep Decoder and show that it significantly outperforms other un-trained methods, in particular sparsity-based classical compressed sensing methods and naive applications of un-trained neural networks in terms of reconstruction performance. We also compare performance (both in terms of reconstruction accuracy and computational cost) in an ideal setup for trained methods, specifically on the fastMRI dataset, where the training and test data come from the same distribution. Here, we find that our un-trained algorithm achieves similar performance to a baseline trained neural network, but a state-of-the-art trained network outperforms the un-trained one. Finally, we perform a comparison on a non-ideal setup where the train and test distributions are slightly different, and find that our un-trained method achieves similar performance to a state-of-the-art accelerated MRI reconstruction method.

Mackenzie Rive located in Yukon and Northwest Territories is the longest river system in Canada and an important transportation link. During the ice free season it is used for shipping transportation, while during freeze up time it serves as an ice road for trucks. Therefore, knowledge of ice conditions is essential to enable save navigation. Due to the difficulty of in-situ observation, remote sensing offers an effective instrument for river ice measurements. SAR data thereby enables gap free time series as it is an active sensor independent from sun illumination and cloud conditions. However, separating ice and open water in SAR data remains a challenging task due to the principals of the radar signal. On the one hand it is sensitive to surface roughness influencing weather conditions such as wind and rain affecting the appearance of water. On the other hand, ice shows different backscatter characteristics during freeze up and melting period. Therefore threshold or cluster based approaches encounter their limits in separating the features spaces of both classes demanding a more complex approach to cover these diverse patterns in the SAR image.
\nIn this study a Convolutional Neural Network (CNN) and TerraSAR-X data is used to map open water and ice in a time series between January 2014 and December 2015. Convolutional Neural Networks have proven great potential in classification of satellite images. Not focusing on isolated pixel values but regarding larger receptive fields they are also taking into account texture and shape information. Thus they are suitable to define feature spaces for ice and water.
\nThe test site covers the Mackenzie River Delta at the estuary of the Arctic Ocean. A U-Net with 18 convolutional layers and skip connections was used for training and classification. The four Kennaugh elements (K0 total intensity, K3 double/single bounce, K4 polarization, K7 torsion) were calculated from a total of 47 dual polarized HH/VV TerraSAR-X scenes. 42 of the 47 scenes show complete ice free respectively freeze up conditions and were used to train and validate the network with a total of 4000 patches (3000 for training and 1000 for validation). The remaining 5 scenes show transition conditions during the melt or freeze up time in spring or autumn and were used to test the neural network after training phase.
\nThe capability of the network to recognize class specific features in the SAR data is illustrated by visualizing the trained filters of the network showing typical SAR image structures of ice and water. Limitations of the approach occur for hybrid forms of water and ice that show features of both classes.The model was driven to best performance by adjusting hyper parameters including patch size, batch size, number of epochs, number of filters and filter size after each training run. Obtained results for the test images prove that a combination of a CNN and TerraSAR-X data is able to reliably separate open water from ice.

The report presents an algorithm for constructing a convolutional hierarchical neural network classifier, which is a modification of the algorithm for constructing hierarchical neural network classifiers suggested before. The original algorithm was designed to exploit intrinsic class hierarchy to build a class tree with a neural network in each node classifying groups of initial classes (in a non-terminal node) or a subset of original classes (in a terminal node). The convolutional modification utilizes convolutional neural networks instead of regular fully connected networks in order to apply the model to image classification tasks. Use of class hierarchy for image classification should reduce the number of adjusted neural network parameters compared to deep convolutional neural networks, and therefore it should reduce training and inference time. In this context the algorithm may be compared with some pruning techniques. The convolutional hierarchical neural network classifier inherits some hyperparameters of a conventional hierarchical neural network classifier, like the activation threshold and the threshold by the share of voting patterns. The goal of this study was to explore different strategies of choosing these hyperparameters. To test these strategies, we used the CIFAR-10 dataset. Also, for demonstration purposes we apply the convolutional hierarchical neural network classifier to the CIFAR-100 dataset.

Text sentiment analysis is one of the most important tasks in the field of public opinion monitoring, service evaluation and satisfaction analysis under network environments. Compared with the traditional Natural Language Processing analysis tools, convolution neural networks can automatically learn useful features from sentences and improve the performance of the affective analysis model. However, the original convolution neural network model ignores sentence structure information which is very important for text sentiment analysis. In this paper, we add piece-wise pooling to the convolution neural network, which allows the model to obtain the sentence structure. And the main features of different sentences are extracted to analyze the emotional tendencies of the text. At the same time, the user’s feedback involves many different fields, and there is less labeled data. In order to alleviate the sparsity of the data, this paper also uses the generative adversarial network to make common feature extractions, so that the model can obtain the common features associated with emotions in different fields, and improves the model’s Generalization ability with less training data. Experiments on different datasets demonstrate the effectiveness of this method.

Objectives: The paper presents application of convolution neural network and artificial neural network for image classification problem for clothing dataset along with their performance comparison against different optimizers. The major objective of this paper is to perform image classification on fashion-mnist clothing dataset images. Methods: The methods used here are, the traditional ANN and CNN. Here image classification is performed on Fashion-mnist, clothing dataset using CNN and ANN with different optimizers. The performance of the working of ANN and CNN in classifying images from fashion-mnist dataset is compared against different optimizers namely stochastic gradient Descent, Adagrad, RMS prop and Adam optimizer. Findings: The study found that CNN worked better than ANN yielding training accuracy of 95%, 93% and testing accuracy of 91%, 89% when used with Adam and RmsProp respectively. Novelty: The novelty of this work is to present a comparative study of image classification using CNN, ANN using different optimizers, since not many studies or research articles showed the performance comparison of traditional and convolution neural networks in image classification along with different optimizers. Since the real-world scenarios of today require enormous data to be processed, CNN can fit well to diversify applications since they highly reduce the number of parameters to be trained that speeds up the training process. Moreover, to be specific on image classification problems they require the best and most prominent features to be detected and uncovered; this can be achieved using CNN since it has the concept of convolution using filters at its Core. Hence, CNN is highly recommended for such image classification applications than the traditional artificial-neural-networks because of the aforementioned reasons. Keywords: ANN (Artificial NeuralNetworks); CNN (ConvolutionNeural Networks); Optimization; SGD (Stochastic Gradient Descent); RmsProp (Root mean Square propagation); Adam (Adaptive moment estimation); AdaGrad (Adaptive Gradient)

Objects and processes classification is a common experimental problem. Its solution, first of all, is needed in automatic diagnosis systems, for example, to determine the equipment operation state through the diagnostic signal or to identify abnormalities with medical images. With the development of convolutional neural networks, new prospects for solving such problems have risen up. However, the classification accuracy that can be achieved on these networks is not sufficient enough for all diagnosis issues. It is subject to, for example, timely diagnosis of the onset of transient phenomena. At the same time, another type of neural network, Kohonen self-organizing maps, has a conceptual property for training on unclassified set of classes, that is, giving the opportunity to solve such an issue. Therefore, the accuracy enhancement of the classification on the basis of Kohonen networks implementation in the architecture of convolutional networks is a relevant objective and has practical significance. The article analyzes the means of improving the accuracy of convolutional neural networks and arising problems solution. The ways of increasing the proportion of correct clustering on Kohonen networks due to the ‘growth’ of its grid shape in determining new classes in the learning process are also given. It is this property that makes it possible to recognize transient phenomena. It is determined that the existing solutions of the combination of Kohonen networks and convolutional networks are aimed at improving the efficiency of only self-organizing maps, with the purpose to improve the accuracy of classification by convolutional networks, it became necessary to develop a new architecture. The paper provides a description of this issue. Since the initial information of the Kohonen networks is the weighting matrix values of the grid shape neurons, it was necessary to associate it with the representation of the images in order to process the diagnostic images. The paper proposes the concept of a built-in associative array block based on Kohonen networks. According to the proposed method, a software implementation of a hybrid neural network is developed. The formulation and results of computational experiments are presented. The efficiency of the proposed method is experimentally proved. Keywords: convolutional neural networks, CNN, Kohonen self-organizing maps, SOM, signal classification, image classification.

In recent years Neural Networks and especially Deep Neural Networks (DNN) have seen a rise in popularity. DNNs have increased the overall performance of algorithms in applications such as image recognition and classification, 2D and 3D pose detection, natural language processing, and time series forecasting. Especially in image classification and recognition, so-called Convolutional Neural Network (CNN) have gained high interest as they can reach a high accuracy, which makes them a viable solution in cancer detection or autonomous driving. As CNNs are widely used in image tasks, different visualizations techniques have bee developed to show how their internals are working. Apart from image tasks, CNNs are applicable to other problems, e.g., time series classification, time series forecasting, or natural language processing. CNNs in those contexts behave similarly, allowing for the same visualization techniques to make them more interpretable. In this chapter, we adapt image visualization algorithms to time series problems, allowing us to build granular, intuitively interpretable feature hierarchies to make a time series forecast as understandable as an image recognition task. We do so by using our previous work on power time series forecasting using CNN Auto Encoders (AEs) and applying typical CNN visualization techniques to it. Thus, we guid computer scientists to provide better interpretable figures for a time series forecasting task to application domain experts.KeywordsVisualizationConvolutional neural networksTime series data

In recent years, Convolutional Neural Networks (CNNs) have been widely successful for numerous computer vision tasks using 2D color and 3D point data, such as object detection and classification, and 3D pose estimation. Despite these successes, it comes with a price --- CNNs require a vast amount of labeled training data to be successful. Many training datasets are now available, however, they are mostly limited to general objects in the public domain such as people, cars, buildings, fruit, and more. Thus, not all fields have sufficient training data. Even if an adequate number of data samples can be built into a dataset, manually collecting and labeling the training data remains a laborious task. Several approaches to reduce or eradicate the need for training data have been proposed by researchers. The two most popular approaches are training with synthetic data and training on generalized object features. Training with synthetic data refers to the use of computer graphics to generate training data. While this approach has demonstrated success, it comes with its own challenges. In contrast, training a CNN on generalized object features identifies specific local 3D features or control points on objects. Local 3D features and their descriptors have proven in the past to be universally applicable and effective for general object detection, and thus, this method allows for a generalized CNN. A generalized CNN refers to a network architecture that, after initial training, can be specialized for a new task with a minimum amount or no new training data. This dissertation investigates an encoder-decoder architecture and determines the capabilities of it to generalize the CNN architecture for object detection tasks in point cloud data. A CNN is trained to recognize 3D features and demonstrates, without retraining, its successful applicability for point cloud object detection using disparate data. Additionally, the geometric consistency of the descriptors is evaluated to obtain further insight. The results demonstrate that the proposed 3D feature descriptors, and their increased geometric consistency contribute to the increased encoder-decoder architecture performance. In summary, this dissertation contributes a state-of-the-art, generalized, feature descriptor-based CNN architecture that can be transferred to different objects without retraining. Furthermore, it provides insight explaining the increased performance of the CNN compared to the state-of-the-art.

The application of deep learning, specifically deep convolutional neural networks (DCNNs), to the classification of remotely-sensed imagery of natural landscapes has the potential to greatly assist in the analysis and interpretation of geomorphic processes. However, the general usefulness of deep learning applied to conventional photographic imagery at a landscape scale is, at yet, largely unproven. If DCNN-based image classification is to gain wider application and acceptance within the geoscience community, demonstrable successes need to be coupled with accessible tools to retrain deep neural networks to discriminate landforms and land uses in landscape imagery. Here, we present an efficient approach to train/apply DCNNs with/on sets of photographic images, using a powerful graphical method called a conditional random field (CRF), to generate DCNN training and testing data using minimal manual supervision. We apply the method to several sets of images of natural landscapes, acquired from satellites, aircraft, unmanned aerial vehicles, and fixed camera installations. We synthesize our findings to examine the general effectiveness of transfer learning to landscape-scale image classification. Finally, we show how DCNN predictions on small regions of images might be used in conjunction with a CRF for highly accurate pixel-level classification of images.

Brain tumors, encompassing subtypes with distinct progression and risk profiles, are a serious public health concern. Magnetic resonance imaging (MRI) is the primary imaging modality for non-invasive assessment, providing the contrast and detail necessary for diagnosis, subtype classification, and individualized care planning. In this paper, we evaluate the capability of modern deep learning models to classify gliomas as high-grade (HGG) or low-grade (LGG) using reduced training data from MRI scans. Utilizing the BraTS 2019 best-slice dataset (2185 images in two classes, HGG and LGG) divided in two folders, training and testing, with different images obtained from different patients, we created subsets including 10%, 25%, 50%, 75%, and 100% of the dataset. Six deep learning architectures, DeiT3_base_patch16_224, Inception_v4, Xception41, ConvNextV2_tiny, swin_tiny_patch4_window7_224, and EfficientNet_B0, were evaluated utilizing three-fold cross-validation (k = 3) and increasingly large training datasets. Explainability was assessed using Grad-CAM. With 25% of the training data, DeiT3_base_patch16_224 achieved an accuracy of 99.401% and an F1-Score of 99.403%. Under the same conditions, Inception_v4 achieved an accuracy of 99.212% and a F1-Score of 99.222%. Considering how the models performed across both data subsets and their compute demands, Inception_v4 struck the best balance for MRI-based glioma classification. Both convolutional networks and vision transformers achieved superior discrimination between HGGs and LGGs, even under data-limited conditions. Architectural disparities became increasingly apparent as training data diminished, highlighting unique inductive biases and efficiency characteristics. Even with a relatively limited amount of training data, current deep learning (DL) methods can achieve reliable performance in classifying gliomas from MRI scans. Among the architectures evaluated, Inception_v4 offered the most consistent balance between accuracy, F1-Score, and computational cost, making it a strong candidate for integration into MRI-based clinical workflows.

Animal image classification with CNN (convolutional neural network) is commonly investigated in aera of image recogniation and classification, but major studies focus on species pictures classification with obvious distinctions. For example, CNN is usually employed to distinghish images between dogs and cats. This article puts the effort on similar animal images classification by applying simple 2D CNN via python. It focus on the binary classification for snub-nosed monkeys and normal monkeys. This distinguishment is hard to be done manually in a short time. For constructing complete convolutional neural network, some preparations are done in advance, such as the database construction and preprocess. The database is constructed by python crawler (downloading from google images), with 800 and 200 images for each class respectively as train data and test data. The pre-work includes image resizing, decoding and standardization. After that, the model is trained and then tested for verifying the model reliability. The training accuracy is 96.67% without any abnormality. On the basis of successful training, the test accuracy almost coincides with train accuracy in each 50 generations and plots in a graph. It indicates similar trends and results for them in the whole process. Because of this, CNN model in the study can help people identify rare animals in time and then people can effectively protect them. Therefore, CNN will be helpful in field of animal conservation, especially for rare species.