U-Net-Based Satellite Image Segmentation for Geospatial Applications

Different scenarios can be found in land classification and segmentation of satellite images. First, when prior knowledge is available, the training data is generally selected by randomly picking samples within classes. When no prior knowledge is available the system can pick samples at random among all unlabeled data, which is highly unreliable or it can rely on the expert collaboration to improve progressively the training data applying an active learning function. We suggest a scheme to tackle the lack of prior knowledge without actively involving the expert, whose collaboration may be expensive. The proposed scheme uses a clustering technique to analyze the feature space and find the most representative samples for being labeled. In this case the expert is just involved in labeling once a reliable training data set for being representative of the feature space. Once the training set is labeled by the expert, different classifiers may be built to process the rest of samples. Three different approaches are presented in this paper: the result of the clustering process, a distance based classifier, and support vector machines (SVM).

Possiblistic-Fuzzy C-Means Clustering Approach for the Segmentation of Satellite Images in HSL Color Space

Deep learning (DL) has experienced an exponential development in recent years, with major impact in many medical fields, especially in the field of medical image and, respectively, as a specific task, in the segmentation of the medical image. We aim to create a computer assisted diagnostic method, optimized by the use of deep learning (DL) and validated by a randomized controlled clinical trial, is a highly automated tool for diagnosing and staging precancerous and cervical cancer and thyroid cancers. We aim to design a high-performance deep learning model, combined from convolutional neural network (U-Net)-based architectures, for segmentation of the medical image that is independent of the type of organs/tissues, dimensions or type of image (2D/3D) and to validate the DL model in a randomized, controlled clinical trial. We used as a methodology primarily the analysis of U-Net-based architectures to identify the key elements that we considered important in the design and optimization of the combined DL model, from the U-Net-based architectures, imagined by us. Secondly, we will validate the performance of the DL model through a randomized controlled clinical trial. The DL model designed by us will be a highly automated tool for diagnosing and staging precancers and cervical cancer and thyroid cancers. The combined model we designed takes into account the key features of each of the architectures Overcomplete Convolutional Network Kite-Net (Kite-Net), Attention gate mechanism is an improvement added on convolutional network architecture for fast and precise segmentation of images (Attention U-Net), Harmony Densely Connected Network-Medical image Segmentation (HarDNet-MSEG). In this regard, we will create a comprehensive computer assisted diagnostic methodology validated by a randomized controlled clinical trial. The model will be a highly automated tool for diagnosing and staging precancers and cervical cancer and thyroid cancers. This would help drastically minimize the time and effort that specialists put into analyzing medical images, help to achieve a better therapeutic plan, and can provide a “second opinion” of computer assisted diagnosis.

Need for medical imaging has immensely increased to diagnose functionality of organs and tissues. Application of magnetic resonance imaging (MRI) or computerized tomography scan (CT scan) has automated the process of disease prediction which otherwise would need manual intervention of experienced physicians. Critical transplants and tumor diagnosis of kidney, heart and liver could be very challenging without the application of high-resolution medical imaging. Semantic image segmentation helps to identify features within the image and label them to demarcate the background from organs and tumors formed on them. Current proposed model is built by applying image segmentation using deep learning models by training 3D image information obtained from KITS-19 using CNN U-Net architecture to segregate essential tumor details from the kidney background images. Considering the evaluation of medical condition based on imagery is very challenging due to lack of pre-processing methods to handle unbalanced nature of class distribution which is common in medical imaging. In the current research, we apply state-of-the-art convolution neural network and deep learning models to predict the tumor in MRI imagery. We performed evaluation of a subset of 60 CT scans with slice thickness values obtained from KiTS19 dataset. We also compared the image segmentation model with BRATS’2013 challenging dataset also to validate the prediction of tumor in brain MRI scan images. Performance evaluation of kidney and tumor features obtained through threefold cross validation comparing with ground truth gave us dice coefficient score of 0.96 while tumor segmentation prediction was close to 0.80. We believe the performance can be increased through higher GPU memory requirements, which was a limitation to our existing GPU hardware. Nevertheless, the possibility of applying improved deep learning model based on existing results and can be a promising step forward in medical imaging segmentation.

The Real time monitoring of forest area, coastal regions, sea, river basins, nation borders etc. helps in quick determination of devastations caused by natural or man-made catastrophes, which can lead to emergency situations. Real-time Segmentation of satellite images is essential to detect fire, floods, volcanoes, earthquakes etc. of a specific geographical zone. Effective determination of such calamities and taking proactive measures can save a lot of lives and natural resources. In this paper, an efficient FSM based architecture is proposed for performing segmentation of satellite Images procured from NASA’s Operational Land Imager (OLI) on Landsat 8 and Landsat 5. The K-means clustering algorithm even after five decades of it’s existence is quite effective and mostly used for big data applications, owing to its lower complexity. However, it’s effective implementation has started only in the last couple of years. Here a FSM-based reconfigurable architecture is proposed for K-means algorithm taking into consideration the real-time segmentation of satellite Images. Compared to other architectures in the literature, the proposed one has reduced hardware usage, lower area, power consumption with a satisfactory clock frequency, most importantly it is reconfigurable and can be used with the on-board circuitry within satellites. The testing is carried out on eight latest satellite images of natural and human devastations, taken from NASA’s OLI. The number of possible clusters within the geographical image is studied with the help of histogram analysis based on Otsu’s thresholding method. The performance analysis is carried out by computing the Peak Signal to Noise (PSNR) and Mean Structural Similarity Index (M-SSIM) of the resultant clusters. It is observed that by increasing the number of clusters, the detailed classes were effectively segmented and the quality of clustering is ensured by the improvement in PSNR and M-SSIM values. Comparison of the obtained clusters with their true land features also yielded satisfactory results.

The Image classification is one of the preliminary processes, which humans learn as infants. The fundamentals of image classification lie in identifying basic shapes and geometry of objects around us. It is a process which involves the following tasks of pre-processing the image (normalization), image segmentation, extraction of key features and identification of the class. The current image classification techniques are much faster in run time and more accurate than ever before, they can be used for extensive applications including, security features, face recognition for authentication and authorization, traffic identification, medical diagnosis and other fields. The idea of image classification can be solved by different approaches. But the machine learning algorithms are the best among them. These algorithms are based on the idea proposed years ago, but couldn't be implemented due to lack of computational power. With the idea of deep learning, the models are trained better and are able to identify different levels of image representation. The convolutional neural networks revolutionized this field by learning the basic shapes in the first layers and evolving to learn features of the image in the deeper layers, resulting in more accurate image classification. The idea of Convolutional neural network was inspired by the hierarchical representation of neurons by Hubel and Wiesel in 1962, their work was based on the study of stimuli of the visual cortex in cat. It was a fundamental breakthrough in the field of computer vision in understanding the working of visual cortex in humans and animals. In this paper feature of an images is extracted using convolution neural network using the concept of deep learning. Further classification algorithms are implemented for various applications.

The satellite image segmentation is a key area for current research and numerous work has been done for exploration of this area. The nature inspired optimization algorithms are very promising with image segmentation techniques to provide a platform for processing of satellite images. In this paper a literature review of different nature based optimization algorithms such as modified artificial bee colony (MABC) algorithm, ABC algorithm, particle swarm optimization (PSO), Darwinian PSO, genetic algorithm (GA), Wind driven optimization (WDO) and cuckoo search(CS) using different objective functions has been discussed to find the optimized multilevel thresholds. These nature influenced optimization methods and their performances are compared using different objective functions for optimal multilevel thresholding. The comparative study shows that the MABC algorithm and different variants of CS algorithm are very strong and accurate in results generating using image segmentation. Both methods search multilevel thresholds very efficiently and correctly, and in MABC an improved bee’s search solution are used and in CS the cuckoo search solution are used.

Urban planning applications (energy audits, investment, etc.) require an understanding of built infrastructure and its environment, i.e., both low-level, physical features (amount of vegetation, building area and geometry etc.), as well as higher-level concepts such as land use classes (which encode expert understanding of socio-economic end uses). This kind of data is expensive and labor-intensive to obtain, which limits its availability (particularly in developing countries). We analyze patterns in land use in urban neighborhoods using large-scale satellite imagery data (which is available worldwide from third-party providers) and state-of-the-art computer vision techniques based on deep convolutional neural networks. For supervision, given the limited availability of standard benchmarks for remote-sensing data, we obtain ground truth land use class labels carefully sampled from open-source surveys, in particular the Urban Atlas land classification dataset of $20$ land use classes across $~300$ European cities. We use this data to train and compare deep architectures which have recently shown good performance on standard computer vision tasks (image classification and segmentation), including on geospatial data. Furthermore, we show that the deep representations extracted from satellite imagery of urban environments can be used to compare neighborhoods across several cities. We make our dataset available for other machine learning researchers to use for remote-sensing applications.

Medical image segmentation is a critical task in computer-assisted diagnosis and therapy planning. Accurate and efficient fully automated segmentation methods are essential for accelerating decision-making processes. Compared with natural image segmentation methods, current deep learning–based segmentation models for medical imaging face challenges related to data scarcity, inconsistent annotations, and high computational cost. Consequently, this thesis aims to develop efficiency and generalised medical image segmentation models, which are fundamental to the clinical adoption and industrial deployment of this technology in healthcare settings. This thesis systematically addresses these challenges through a series of progressively advanced deep learning models. The foundational work, CTranS, forms a baseline medical image segmentation model that combines a convolutional neural network (CNN) based encoder with a Transformer-based decoder. This initial contribution leverages the strengths of CNNs for local feature extraction and Transformers for global contextual understanding, achieving state-of-the-art (SOTA) performance. Subsequently, to enhance the generalisability, MO-CTranS is introduced, which is capable of learning from a mixture of multiple partially labelled datasets. MO-CTranS employs a CNN encoder and Transformer decoder with task-specific tokens to handle label inconsistencies and class imbalances, tackling the challenges of training a single unified model from fragmented and heterogeneously annotated medical data. This innovation moves beyond single-task, fully-annotated model training towards a more scalable and clinically realistic approach. Despite the superior performance of CTranS and MO-CTranS, the computational efficiency of the Transformer blocks hinders their clinical application. Thus, CRFTrans is proposed to enhance the model efficiency. As a novel recursive Vision Transformer layer designed to replace the commonly used cascaded Transformer layers, this approach represents a fundamental shift from simply stacking layers to a recursive refinement process, demonstrating that comparable performance can be achieved with significantly reduced computational complexity, thus facilitating deployment in resource-constrained clinical environments. However, these methods still depend on extensive manual annotation, which remains challenging and time-consuming in clinical practice. Thus, to further enhance both efficiency and generality, a novel semi-supervised medical image segmentation framework, referred to as SSL-MedSAM2, has been developed. SSL-MedSAM2 integrates a training-free few-shot segmentation branch for pseudo labelling and a fully supervised learning branch for label refinement. This design allows the model to leverage a small fraction of labelled images together with a large unlabelled set, dramatically decreasing the annotation effort. Together, these contributions result in deep learning models that are significantly more data-efficient and broadly applicable than conventional approaches. The overall impact of this work is to facilitate the reliable and scalable deployment of segmentation models in real-world clinical settings by bridging the gap among accuracy, efficiency, and generalisability.

In recent years, the machine learning field has been inundated with a variety of deep learning methods. Different deep learning model types, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), adversarial neural networks (ANNs), and autoencoders, are successfully tackling challenging computer vision problems including image detection and segmentation in an unconstrained environment. Although image segmentation has received a lot of interest, there have been several new deep learning methods discovered with regard to object detection and recognition. An academic review of deep learning image segmentation methods is presented in this article. In this study, the major goal is to offer a sensible comprehension of the basic approaches that have already made a substantial contribution to the domain of image segmentation throughout the years. The article describes the existing state of image segmentation, and goes on to make the argument that deep learning has revolutionized this field. Afterwards, segmentation algorithms have been scientifically classified and optimized, each with their own special contribution. With a variety of informative narratives, the reader may be able to understand the internal workings of these processes more quickly.

A technique based on the utilization of both textural and contextual information is proposed for the segmentation of satellite images. Markov random fields are taken as a unified framework for the modeling of satellite images, for the representation of textural as well as contextual information, and, finally, for the formulation of the scene labeling problem. The labeling problem is then solved by a new algorithm based on game strategies. In order to improve its effectiveness, information contained in geographic maps is used for the selection of optimal texture feature descriptors of the images. An adaptive estimation technique is also applied to obtain contextual information. Experimental results show that the technique is very efficient and effective. >

Many mapping organisations are unable to revise maps as frequently as is desired by their customers. An important application of automatic interpretation of satellite imagery is map revision. A prototype system for semi-automatic map revision from satellite imagery is presented. The system’s major functions are (1) segmentation of satellite images, (2) image interpretation for identifying map features using information from the satellite images, the result of the segmentation, and the old map, and (3) manipulation of the map data before and after the image interpretation.

Deep Markov models of multidimensional random fields

With the widespread scope of image processing in fields such as ground classification, segmentation of satellite images, biomedical surface inspection and content-based image retrieval (CBIR), texture analysis is an important domain with a wide scope. The process of texture analysis comprises the following important steps: texture classification, segmentation and synthesis. Texture is a significant property of images which is difficult to define even though the human eye may recognize it with ease. Texture classification has remained an intangible pattern recognition task despite numerous studies. The prime issue in any texture classification application is how to select the features and which features to consider for representing texture. Another major issue remains the type of metric to be used for comparing the feature vectors. The traditional way of texture classification is to convert the texture image into a vector representing the features using a set of filters. This is followed by classification with a few smoothening steps involved in between. This paper outlines a picture of the basic features of an image as texture, colour and shape which are to be extracted to form the feature vector. The concept of applying Machine learning for the purpose of classification is explored and implemented. The use of Soft Computing Technique—Artificial Neural Networks is illustrated for the purpose of classification. The texture features are extracted using the GLCM method and used as input and fed to the neural network. The algorithm used in training the neural networks is the traditional backpropagation algorithm. The results show which configuration of the multi-layer feedforward architecture is best suited according to our experimental set-up.

Automatic supervised classification with complex modelling such as deep neural networks requires the availability of representative training data sets. While there exists a plethora of data sets that can be used for this purpose, they are usually very heterogeneous and not interoperable. In this context, the present work has a twofold objective: (i) to describe procedures of open-source training data management, integration, and data retrieval, and (ii) to demonstrate the practical use of varying source training data for remote sensing image classification. For the former, we propose SatImNet, a collection of open training data, structured and harmonized according to specific rules. For the latter, two modelling approaches based on convolutional neural networks have been designed and configured to deal with satellite image classification and segmentation.