SRSe-Net: Super-Resolution-Based Semantic Segmentation Network for Green Tide Extraction

Apples are one of the most consumed fruits, and they require efficient harvesting procedures to remains in optimal states for a longer period, especially during transportation. Therefore, automation has been adopted by many orchard operators to help in the harvesting process, which includes apple localization on the trees. The de facto sensor that is currently used for this task is the standard camera, which can capture wide view information of various apple trees from a reasonable distance. Therefore, this paper aims to produce the output mask of the apple locations on the tree automatically by using a deep semantic segmentation network. The network must be robust enough to overcome all challenges of shadow, surrounding illumination, size variations, and occlusion to produce accurate pixel-wise localization of the apples. A high-resolution deep architecture is embedded with an optimized design of group and shuffle operators (GSO) to produce the best apple segmentation network. GSO allows the network to reduce the dependency on a few sets of dominant convolutional filters by forcing each smaller group to contribute effectively to the task of extracting optimal apple features. The experimental results show that the proposed network, GSHR-Net, with two sets of group convolution applied to all layers produced the best mean intersection over union of 0.8045. The performance has been benchmarked with 11 other state-of-the-art deep semantic segmentation networks. For future work, the network performance can be increased by integrating synthetic augmented data to further optimize the training phase. Moreover, spatial and channel-based attention mechanisms can also be explored by emphasizing some strategic locations of the apples, which makes the recognition more accurate.

Sea-sky line detection (SSLD) is pivotal for applications such as unmanned surface vehicles (USVs) navigation and maritime target detection. However, existing algorithms are susceptible to interference from adverse weather, illumination change, and the presence of waves, leading to poor detection accuracy and robustness. To address these challenges, we propose a novel SSLD algorithm based on a deep semantic segmentation network. First, we integrate the strengths of convolutional neural networks (CNNs) and Transformers in a lightweight block named efficient vision transformer (E-ViT). This block enables efficient interaction and aggregation of local and global features with lower computational overhead. Building upon E-ViT, we develop an encoder module that significantly improves the accuracy of semantic segmentation while maintaining the network’s lightweight. Then, we design a robust postprocessing module, which leverages semantic information to effectively remove interferences and filter out candidate points for the sea-sky line, thereby achieving high-precision SSLD. Finally, we construct a well-labeled maritime scene dataset with diverse complex attributes to validate the proposed algorithm. Experimental results demonstrate that our method outperforms several state-of-the-art algorithms in terms of both accuracy and robustness in complex maritime scenarios.

<div class="section abstract"><div class="htmlview paragraph">Lane detection is an important component in automatic pilot system and advanced driving assistance system (ADAS). The stability and precision of lane detection will directly determine precision of control and lane plan of vehicles. Traditional mechanical vision lane detection approaches in complicated environment have the deficiencies of low precision and feature semantic description disabilities. But the lane detection depending on deep learning, e.g. SCNN network, LaneNet network, ENet-SAD network have imbalance problems of splitting precision and storage usage. This paper proposes an approach of high-efficiency deep learning Segnet-LSTM semantic segmentation network. This network structure is composed with encoding network and corresponding decoding networks. First, convolution and maximum pooling. The proposal extracts texture details of five images and stores searching position of maximum pooling. Meanwhile, it will implement interpolate processing to the lost points. Then, by up-sampling and convoluting with decoders and predicts category of a specific pixel with Softmax function. At the same time, it splits with long and short term memories of LSTM network and finally, the output is a complete image that enables detection and tracking of lanes in during the day and at night. Experiments have suggested that this approach provides more fitting precision and feature extraction precision in comparison with Unet-LSTM algorithm. This algorithm provide accuracy that is 3.724% higher than that of Unet-LSTM in the day in perspective of miou value and the precision of lane pixel categorization is 4.126% higher than that of Unet-LSTM. Besides, miou of this algorithm at night is 5.6% higher than that of Unet-LSTM and the precision of lane pixel categorization is 4.1398% higher. Moreover, the algorithm allows balance between storage involvement and precision of lane detection on the basis of high fitting precision, guaranteeing real-time performance and stability in various senses, assisting auto pilots in more efficiently.</div></div>

Driven by the urgent demand for flood monitoring, water resource management and environmental protection, water-body detection in remote sensing imagery has attracted increasing research attention. Deep semantic segmentation networks (DSSNs) have gradually become the mainstream technology used for remote sensing image water-body detection, but two vital problems remain. One problem is that the traditional structure of DSSNs does not consider multiscale and multishape characteristics of water bodies. Another problem is that a large amount of unlabeled data is not fully utilized during the training process, but the unlabeled data often contain meaningful supervision information. In this paper, we propose a novel multiscale residual network (MSResNet) that uses self-supervised learning (SSL) for water-body detection. More specifically, our well-designed MSResNet distinguishes water bodies with different scales and shapes and helps retain the detailed boundaries of water bodies. In addition, the optimization of MSResNet with our SSL strategy can improve the stability and universality of the method, and the presented SSL approach can be flexibly extended to practical applications. Extensive experiments on two publicly open datasets, including the 2020 Gaofen Challenge water-body segmentation dataset and the GID dataset, demonstrate that our MSResNet can obviously outperform state-of-the-art deep learning backbones and that our SSL strategy can further improve the water-body detection performance.

Computer vision-based statistical crack quantification for concrete structures

PeriorbitAI: Artificial Intelligence Automation of Eyelid and Periorbital Measurements

Shipborne sea ice detection aboard icebreakers plays a paramount role in polar navigation. The continuous evolution of deep learning semantic segmentation networks has promoted the advancement of sea ice detection tasks. At this stage, there are relatively few studies on shipboard sea ice detection, and the accuracy of polar sea ice detection will be reduced due to problems such as blurred sea fog and indistinct boundaries. In this study, a shipboard sea ice detection dataset is constructed, and a sea ice detection method that combines multi-branch attention feature alignment and multi-scale feature extraction is proposed. The heterogeneous receptive field enhancement atrous spatial convolution pooling pyramid module is designed, and the feature alignment module based on the attention mechanism is constructed, which strengthens the model’s extraction of sea ice features and elevates representation performance. Experimental results underscore the heightened precision of our approach in sea ice detection, to some extent alleviating the issue of missed detections in new ice. It constitutes a positive contribution towards advancing shipborne sea ice detection in polar environments.

In recent years, semantic segmentation has become one of the most active tasks of the computer vision field. Its goal is to group image pixels into semantically meaningful regions. Deep learning methods, in particular those who use convolutional neural network (CNN), have shown a big success for the semantic segmentation task. In this paper, we will introduce a semantic segmentation system using a reinforced fully convolutional densenet with multiscale kernel prediction method. Our main contribution is to build an encoder-decoder based architecture where we increase the width of dense block in the encoder part by conducting recurrent connections inside the dense block. The resulting network structure is called wider dense block where each dense block takes not only the output of the previous layer but also the initial input of the dense block. These recurrent structure emulates the human brain system and helps to strengthen the extraction of the target features. As a result, our network becomes deeper and wider with no additional parameters used because of weights sharing. Moreover, a multiscale convolutional layer has been conducted after the last dense block of the decoder part to perform model averaging over different spatial scales and to provide a more flexible method. This proposed method has been evaluated on two semantic segmentation benchmarks: CamVid and Cityscapes. Our method outperforms many recent works from the state of the art.

The natural phenomenon of harmful algae bloom (HAB) has a bad impact on the quality of pure and freshwater. It increases the risk to human health, water bodies and overall aquatic ecosystem. It is necessary to continuously monitor and perform proper action against HAB. The inspection of algae blooms by using conventional methods, like algae detection under microscopes, is a difficult, expensive, and time-consuming task, however, computer vision-based deep learning models play a vital role in identifying and detecting harmful algae growth in aquatic ecosystems and water reservoirs. Many studies have been conducted to address harmful algae growth by using a CNN based model, however, the YOLO model is considered more accurate in identifying the algae. This advanced deep learning method is extensively used to detect algae and classify them according to their corresponding category. In this study, we used various versions of the convolution neural network (CNN) based on the You Only Look Once (YOLO) model. Recently YOLOv5 has been getting more attention due to its performance in real-time object detection. We performed a series of experiments on our custom microscopic images dataset by using YOLOv3, YOLOv4, and YOLOv5 to detect and classify the harmful algae bloom (HAB) of four classes. We used pre-processing techniques to enhance the quantity of data. The mean average precision (mAP) of YOLOv3, YOLOv4, and YOLO v5 is 75.3%, 83.0%, and 91.0% respectively. For the monitoring of algae bloom in freshwater, computer-aided based systems are very helpful and effective. To the best of our knowledge, this work is pioneering in the AI community for applying the YOLO models to detect algae and classify from microscopic images.

The European UP-Drive project addresses transportation-related challenges by providing key contributions that enable fully automated vehicle navigation and parking in complex urban areas, which results in a safer, inclusive, affordable and environmentally friendly transportation system. For this purpose, the project consortium developed a prototype electrical vehicle equipped with cameras and LiDARs sensors that is capable to autonomously drive around the city and find available parking spots. In UP-Drive, we created an accurate, robust and redundant multi-modal environment perception system that provides 360° coverage around the vehicle. This paper summarizes the work of the project related to the surround view semantic perception using fisheye and narrow field-of-view semantic virtual cameras. Deep learning-based semantic, instance and panoptic segmentation networks, which satisfy requirements in accuracy and efficiency have been developed and integrated into the final prototype. The UP-Drive automated vehicle has been successfully demonstrated in urban areas after extensive experiments and numerous field tests.

Autonomous Vehicles are required to operate robustly across widely varied scenarios and conditions, however environmental factors such as weather and lighting can impede the capabilities of the perception systems required for safe operation. In this work we investigate the effects lighting changes can have on semantic segmentation of urban road scenes, specifically how segmentation performance is affected by underexposed imagery. Using two publicly available datasets, we simulate incorrectly set camera exposure and compare the performance of a standard pre-trained deep semantic segmentation network on correctly and incorrectly exposed images. We then introduce a novel input optimization network, which aims to modify a given image such that it generates an optimal response from a pre-trained semantic segmentation network. We compare our approach to an adversarially-trained model and demonstrate significantly improved semantic segmentation performance over that of unoptimised images.

Application of computer images in virtual simulation technology-apparel as an example

Research on image classification sparked the latest deep-learning boom. Many downstream tasks, including semantic segmentation, benefit from it. The state-of-the-art semantic segmentation models are all based on deep learning, and they sometimes make some semantic mistakes. In a semantic segmentation dataset with a small number of categories, images are often collected from a single scene, and there is a close semantic connection between any two categories. However, in the semantic segmentation dataset collected from multiple scenes, two categories may be irrelevant. The probability of objects in one category appearing next to objects in other categories is different, which is the basis of the paper. Semantic segmentation methods need to solve two problems of positioning and classification. This paper is dedicated to correcting those clearly wrong classifications that are contrary to reality. Specifically, we first calculate the relevancy between different class pairs. Then, based on this knowledge, we infer the category of a connected component according to the relationships of this connected component with its surrounding connected components and correct the obviously wrong classifications made by a deep learning semantic segmentation model. Several well-performing deep learning models are experimented on two challenging public datasets in the field of semantic image segmentation. Our proposed method improves the performance of UPerNet, OCRNet and SETR from 40.7%, 43% and 48.64% to 42.07%, 44.09% and 49.09% mean IoU on the ADE20K validation set, and the performance of PSPNet, DeepLabV3 and OCRNet from 37.26%, 37.3% and 39.5% to 38.93%, 38.95% and 40.63% mean IoU on the COCO-Stuff dataset, which shows the effectiveness of the method.

Abstract. Semantic instance segmentation from scenes, serving as a crucial role for 3D modelling and scene understanding. Conducting semantic segmentation before grouping instances is adopted by the existing state-of-the-art methods. However, without additional refinement, semantic errors will fully propagate into the grouping stage, resulting in low overlap with the ground truth instance. Furthermore, the proposed methods focused on indoor level scenes, which are limited when directly applied to large-scale outdoor Airborne Laser Scanning (ALS) point clouds. Numerous instances, significant object density and scale variations make ALS point clouds distinct from indoor data. In order to address the problems, we proposed a geometric characterization-aware semantic instance segmentation network, which utilized both semantic and objectness score to select potential points for grouping. And in point cloud feature learning stage, hand-craft geometry features are taken as input for geometric characterization awareness. Moreover, to address errors propagated from previous modules after grouping, we have additionally designed a per-instance refinement module. To assess semantic instance segmentation, we conducted experiments on an open-source dataset. Additionally, we performed semantic segmentation experiments to evaluate the performance of our proposed point cloud feature learning method.

Chapter 12 - Semantic scene segmentation for robotics