Scene terrain classification for autonomous vehicle navigation based on semantic segmentation method

Convolutional Neural Network (CNN) is widely used for semantic segmentation and land-use and land-cover (LULC) mapping of very high-resolution (VHR) remote sensing images. The convolution operation is a powerful method for VHR classification, but the loss of high-frequency detail information caused during its operation decreases the classification accuracy, particularly in the boundary. Thus, it is necessary to supply additional boundary information to the CNN for alleviating this situation. In the classification task (and in LULC mapping), providing more effective information generates a better classification result. Current methods regard the boundary of images as the same category object and process it uniformly, which loses a notable amount of useful information because of the different properties, such as ambiguity and transition, between remote sensing images and their boundaries. Thus, a semantic segmentation method with category boundary for LULC mapping is proposed in this paper. First, a multi-task CNN called the category boundary detection network (CBDN) is designed to extract the boundary information of different category objects. Second, this category boundary and VHR images are used for initial semantic segmentation. Finally, the category boundary and the initial semantic segmentation result (ISSR) are fused to obtain the final LULC map by a two-step strategy, including the explicit fusion and the boundary attention loss function. To verify whether category boundary improved the classification accuracy, a set of comparative experiments were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets. The method in this paper was compared with a semantic segmentation method with no boundary information and a semantic segmentation method with global boundary. The results showed that the proposed method in this paper achieved good performances in the Vaihingen (overall accuracy (OA) = 0.924, Kappa coefficient (K) = 0.898, mean F1 score (mF1) = 0.896 and mean Intersection over Union (mIoU) = 0.817) and Potsdam datasets (OA = 0.890, K = 0.857, mF1 = 0.923, and mIoU = 0.860) based on the eroded labels.

Advancing biomedical waste classification through a hybrid ensemble of deep Learning, reinforcement Learning, and differential evolution algorithms.

Early identification is essential to prevent blindness in glaucoma patients since the disease is an optic neuropathy that, if left untreated, leads to irreversible vision loss. Because it makes it possible to analyze important structural features such as the Optic Cup and Disc (OC and OD), whose morphological alterations are markers of disease progression, retinal fundus imaging is essential for glaucoma diagnosis. Deep learning breakthroughs have led to improvements in segmentation approaches, which help early diagnosis and therapy by examining retinal structures with greater accuracy and efficiency. Numerous methods for segmenting retinal fundus images are reviewed in this paper, including those for Optic Nerve Head (ONH), retinal vasculature, OD, OC, and macula segmentation. In addition to cutting-edge deep learning designs like U-Net, Mask R-CNN, Fully Convolutional Networks (FCN), and DeepLab, conventional techniques like thresholding and region growth are investigated. By improving segmentation precision with machine and deep learning algorithms, less manual interpretation by human experts is required. A comparison of various methods is given, with consideration to datasets, performance measures, advantages, and disadvantages. While Graph Convolutional Networks (GCNs) and Convolutional Neural Networks (CNNs) perform well in tasks like glaucoma detection and classification, more notable approaches include UNet and Mask RCNN, which have proven to have enormous potential in OD/OC segmentation. In addition, the paper discusses the obstacles that glaucoma detection currently faces, such as inconsistent images, limited datasets, and the requirement for more broadly applicable models. To advance automated glaucoma detection and enhance patient care, several challenges must be overcome.

Remote sensing images contain a wealth of geospatial information. To accurately identify different geospatial categories and extract relevant data, image semantic segmentation plays a crucial role. In recent years, deep learning technology has brought significant breakthroughs to semantic segmentation of remote sensing images, significantly enhancing its performance. This paper investigates the application of deep learning technologies in remote sensing image semantic segmentation, based on Convolutional Neural Networks (CNN) and Transformer-based semantic segmentation methods. It conducts an in-depth comparison of their structural characteristics and applicable scenarios, summarizes the achievements and shortcomings of existing research, and provides technical references and theoretical support for future studies, thereby contributing to the further development of deep learning technology in the field of remote sensing. Research results indicate that CNN-based semantic segmentation methods still hold advantages in extracting local features and achieving efficient segmentation, whereas Transformers address CNN's limitations in global context modeling and long-range dependency capture. Therefore, the collaborative integration of CNN and Transformers will become an important research direction for enhancing model performance in the future.

Insect taxonomy lies at the heart of many aspects of ecology, and identification tasks are challenging due to the enormous inter- and intraspecies variation of insects. Conventional methods used to study insect taxonomy are often tedious, time-consuming, labor intensive, and expensive, and recently, computer vision with deep learning algorithms has offered an alternative way to identify and classify insect images into their taxonomic levels. We designed the classification task according to the taxonomic ranks of insects—order, family, and genus—and compared the generalization of four state-of-the-art deep convolutional neural network (DCNN) architectures. The results show that different taxonomic ranks require different deep learning (DL) algorithms to generate high-performance models, which indicates that the design of an automated systematic classification pipeline requires the integration of different algorithms. The InceptionV3 model has advantages over other models due to its high performance in distinguishing insect order and family, which is having F1-score of 0.75 and 0.79, respectively. Referring to the performance per class, Hemiptera (order), Rhiniidae (family), and Lucilia (genus) had the lowest performance, and we discuss the possible rationale and suggest future works to improve the generalization of a DL model for taxonomic rank classification.

In recent years, the convolutional neural network (CNN) has made remarkable achievements in semantic segmentation. The method of semantic segmentation has a desirable application prospect. Nowadays, the methods mostly use an encoder-decoder architecture as a way of generating pixel by pixel segmentation prediction. The encoder is for extracting feature maps and decoder for recovering feature map resolution. An improved semantic segmentation method on the basis of the encoder-decoder architecture is proposed. We can get better segmentation accuracy on several hard classes and reduce the computational complexity significantly. This is possible by modifying the backbone and some refining techniques. Finally, after some processing, the framework has achieved good performance in many datasets. In comparison with the traditional architecture, our architecture does not need additional decoding layer and further reuses the encoder weight, thus reducing the complete quantity of parameters needed for processing. In this paper, a modified focal loss function is also put forward, as a replacement for the cross-entropy function to achieve a better treatment of the imbalance problem of the training data. In addition, more context information is added to the decode module as a way of improving the segmentation results. Experiments prove that the presented method can get better segmentation results. As an integral part of a smart city, multimedia information plays an important role. Semantic segmentation is an important basic technology for building a smart city.

Research on image classification sparked the latest deep-learning boom. Many downstream tasks, including semantic segmentation, benefit from it. The state-of-the-art semantic segmentation models are all based on deep learning, and they sometimes make some semantic mistakes. In a semantic segmentation dataset with a small number of categories, images are often collected from a single scene, and there is a close semantic connection between any two categories. However, in the semantic segmentation dataset collected from multiple scenes, two categories may be irrelevant. The probability of objects in one category appearing next to objects in other categories is different, which is the basis of the paper. Semantic segmentation methods need to solve two problems of positioning and classification. This paper is dedicated to correcting those clearly wrong classifications that are contrary to reality. Specifically, we first calculate the relevancy between different class pairs. Then, based on this knowledge, we infer the category of a connected component according to the relationships of this connected component with its surrounding connected components and correct the obviously wrong classifications made by a deep learning semantic segmentation model. Several well-performing deep learning models are experimented on two challenging public datasets in the field of semantic image segmentation. Our proposed method improves the performance of UPerNet, OCRNet and SETR from 40.7%, 43% and 48.64% to 42.07%, 44.09% and 49.09% mean IoU on the ADE20K validation set, and the performance of PSPNet, DeepLabV3 and OCRNet from 37.26%, 37.3% and 39.5% to 38.93%, 38.95% and 40.63% mean IoU on the COCO-Stuff dataset, which shows the effectiveness of the method.

Semantic segmentation is an object classification such as a perceptual grouping in image analysis tasks. And semantic segmentation methods have been proposed to improve the pixel-wise classification accuracies in the research field of computer vision. Recently, the methods achieve high accuracies by using convolutional neural networks (CNN). These CNN-based methods generally need large number and variety of training images to assure a generalization performance without fine tuning. However, it is difficult to prepare the large-scale image datasets which were given reliable semantic labels. In order to overcome this problem, we propose an efficient method for feature transformation to improve the generalization performance in semantic segmentation. Our method is formulated as a variant of neighborhood preserving embedding (NPE) incorporating the within-class coherency and showed best generalization performances compared with other feature transformation methods.

The current instance segmentation method can achieve satisfactory results in common scenarios. However, under the overlap or partial occlusion between targets caused by the complex scenes, accurate segmentation of pigs remains a challenging task. To address the problem, the authors propose an instance segmentation method based on Mask Scoring region‐based convolutional neural networks (R‐CNN) (MS R‐CNN), which creates the adversarial network called MaskDis in the head branch of MS R‐CNN. The MaskDis is trained as a discriminator using a generative adversarial network, and the MS R‐CNN model is used as a generator during model training. The adversarial training enables the generator to learn context information and features at the pixel level, which effectively improves the segmentation quality under pigs’ overlapping or dense occlusions scenes. Experimental conducted on the pig object segmentation dataset show that the proposed approach achieves a precision of 92.03%, a recall of 92.18%, and an F1 score of 0.9210. Compared with the basic MS R‐CNN model, the approach achieved a 2.25% improvement in precision and 1.18% improvement in F1 score. Furthermore, the improved approach outperformed advanced instance segmentation methods such as YOLACT, Swin Transformer, YOLOv5‐seg, and SOLOv2 on COCO evaluation metrics. These experimental results demonstrate the effectiveness of the proposed approach in instance segmentation of pigs in complex scenes, providing technical support for non‐contact pig automatic management.

We propose an estimation method of water quality of a river based on convolutional neural networks (CNNs), a semantic segmentation method, and a new dataset. Since floods are serious problems for riversides, various technologies have been proposed as a safety system. So far, measurement methods of water level are proposed as one of various systems. However, these methods cannot estimate a flood which is caused by debris flows because the flood does not follow the increasing water level. To estimate floods, water quality is more important information because water become muddy before floods. Hence, we estimate water quality using monitoring images and CNNs which trained new dataset. Furthermore, we combine a pre-processing using a semantic segmentation method. Since regions of no-river water often produces misdetections, we exclude these regions using a semantic segmentation method before CNNs. In simulations, it was observed that the proposed method objectively outperforms state-of-the-art methods in accuracy based on our dataset.

Lentic waters, such as lakes, lagoons, reservoirs, and wetlands are characterized by their absence of current. In recent decades, they have been threatened by pollution and scarcity due to various environmental factors. Therefore, they require frequent monitoring to ensure their health and purity, especially to control the proliferation of harmful cyanobacteria (pollutants). Machine Vision Systems (MVS) on board Autonomous Surface Vehicles (ASVs) is a good option for automatic image processing in this context. ASVs must navigate safely, and obstacle detection is essential. In addition, the segmentation of pollutants in water is crucial. We propose an architecture based on convolutional neural networks that integrates both object detection and semantic segmentation. The goal is to simultaneously extract all available global information to detect objects and amorphous textures (cyanobacterial patches and water bodies), considering their variations in size, pose, and appearance. The architecture includes two branches: object detection and semantic segmentation, sharing the same backbone and neck. We evaluate the model on our dataset and the results show that it can holistically understand lentic water scenes with high accuracy, and the integration of the attention mechanism improves its overall performance.

As an important part of modern marine monitoring systems, ship target identification has important significance in maintaining marine rights and monitoring maritime traffic. With the development of artificial intelligence technology, image detection and recognition based on deep learning methods have become the most popular and practical method. In this paper, two deep learning algorithms, the Mask R-CNN algorithm and the Faster R-CNN algorithm, are used to build ship target feature extraction and recognition models based on deep convolutional neural networks. The established models were compared and analyzed to verify the feasibility of target detection algorithms. In this study, 5748 remote sensing maps were selected as the dataset for experiments, and two algorithms were used to classify and extract warships and civilian ships. Experiments showed that for the accuracy of ship identification, Mask R-CNN and Faster R-CNN reached 95.21% and 92.76%, respectively. These results demonstrated that the Mask R-CNN algorithm achieves pixel-level segmentation. Compared with the Faster R-CNN algorithm, the obtained target detection effect is more accurate, and the performance in target detection and classification is better, which reflects the great advantage of pixel-level recognition.

The unique characteristics of traditional buildings can provide fresh insights for sustainable building development. In this study, a deep learning model and methodology were developed for classifying traditional buildings by using artificial intelligence (AI)-based image analysis technology. The model was constructed based on expert knowledge of East Asian buildings. Videos and images from Korea, Japan, and China were used to determine building types and classify and locate structural members. Two deep learning algorithms were applied to object recognition: a region-based convolutional neural network (R-CNN) to distinguish traditional buildings by country and you only look once (YOLO) to recognise structural members. A cloud environment was used to develop a practical model that can handle various environments in real time.

Semantic segmentation is considered to be one of the basic steps in understanding image content. For semantic segmentation, if multi-spectral images are used together with color images, more successful results are obtained due to complementary information obtained from multi-spectral images. In this paper, a semantic segmentation method was developed in which the images obtained from CCD and thermal sensors were used together. In the proposed method, convolutional neural networks were used in encoder-decoder architecture. The experiments carried out show that the developed method produces better numerical and visual results than the works published in the literature.

Cyanobacterial Harmful Algal Blooms (CyanoHABs) in lakes and reservoirs have increased substantially in recent decades due to different environmental factors. Its early detection is a crucial issue to minimize health effects, particularly in potential drinking and recreational water bodies. The use of Autonomous Surface Vehicles (ASVs) equipped with machine vision systems (cameras) onboard, represents a useful alternative at this time. In this regard, we propose an image Semantic Segmentation approach based on Deep Learning with Convolutional Neural Networks (CNNs) for the early detection of CyanoHABs considering an ASV perspective. The use of these models is justified by the fact that with their convolutional architecture, it is possible to capture both, spectral and textural information considering the context of a pixel and its neighbors. To train these models it is necessary to have data, but the acquisition of real images is a difficult task, due to the capricious appearance of the algae on water surfaces sporadically and intermittently over time and after long periods of time, requiring even years and the permanent installation of the image capture system. This justifies the generation of synthetic data so that sufficiently trained models are required to detect CyanoHABs patches when they emerge on the water surface. The data generation for training and the use of the semantic segmentation models to capture contextual information determine the need for the proposal, as well as its novelty and contribution.Three datasets of images containing CyanoHABs patches are generated: (a) the first contains real patches of CyanoHABs as foreground and images of lakes and reservoirs as background, but with a limited number of examples; (b) the second, contains synthetic patches of CyanoHABs generated with state-of-the-art Style-based Generative Adversarial Network Adaptive Discriminator Augmentation (StyleGAN2-ADA) and Neural Style Transfer as foreground and images of lakes and reservoirs as background, and (c) the third set, is the combination of the previous two. Four model architectures for semantic segmentation (UNet++, FPN, PSPNet, and DeepLabV3+), with two encoders as backbone (ResNet50 and EfficientNet-b6), are evaluated from each dataset on real test images and different distributions. The results show the feasibility of the approach and that the UNet++ model with EfficientNet-b6, trained on the third dataset, achieves good generalization and performance for the real test images.