Semantic Segmentation Network with Local Multi-Scale Attention for Locating Traffic Sign

Linear feature extraction is crucial for special objects in semantic segmentation networks, such as slot marking and lanes. The objects with linear characteristics have global contextual information dependency. It is very difficult to capture the complete information of these objects in semantic segmentation tasks. To improve the linear feature extraction ability of the semantic segmentation network, we propose introducing the dilated convolution with vertical and horizontal kernels (DVH) into the task of feature extraction in semantic segmentation networks. Meanwhile, we figure out the outcome if we put the different vertical and horizontal kernels on different places in the semantic segmentation networks. Our networks are trained on the basis of the SS dataset, the TuSimple lane dataset and the Massachusetts Roads dataset. These datasets consist of slot marking, lanes, and road images. The research results show that our method improves the accuracy of the slot marking segmentation of the SS dataset by 2%. Compared with other state-of-the-art methods, our UnetDVH-Linear (v1) obtains better accuracy on the TuSimple Benchmark Lane Detection Challenge with a value of 97.53%. To prove the generalization of our models, road segmentation experiments were performed on aerial images. Without data argumentation, the segmentation accuracy of our model on the Massachusetts roads dataset is 95.3%. Moreover, our models perform better than other models when training with the same loss function and experimental settings. The experiment result shows that the dilated convolution with vertical and horizontal kernels will enhance the neural network on linear feature extraction.

Semantic segmentation has been proposed as a tool to accelerate the processing of natural history collection images. However, developing a flexible and resilient segmentation network requires an approach for adaptation which allows processing different datasets with minimal training and validation. This paper presents a cross-validation approach designed to determine whether a semantic segmentation network possesses the flexibility required for application across different collections and institutions. Consequently, the specific objectives of cross-validating the semantic segmentation network are to (a) evaluate the effectiveness of the network for segmenting image sets derived from collections different from the one in which the network was initially trained on; and (b) test the adaptability of the segmentation network for use in other types of collections. The resilience to data variations from different institutions and the portability of the network across different types of collections are required to confirm its general applicability. The proposed validation method is tested on the Natural History Museum semantic segmentation network, designed to process entomological microscope slides. The proposed semantic segmentation network is evaluated through a series of cross-validation experiments designed to test using data from two types of collections: microscope slides (from three institutions) and herbarium sheets (from seven institutions). The main contribution of this work is the method, software and ground truth sets created for this cross-validation as they can be reused in testing similar segmentation proposals in the context of digitization of natural history collections. The cross-validation of segmentation methods should be a required step in the integration of such methods into image processing workflows for natural history collections.

A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation

In remote-sensing image (RSI) semantic segmentation, the dependence on large-scale and pixel-level annotated data has been a critical factor restricting its development. In this letter, we propose an unsupervised semantic segmentation network embedded with geometry consistency (UGCNet) for RSIs, which imports the adversarial-generative learning strategy into a semantic segmentation network. The proposed UGCNet can be trained on a source-domain dataset and achieve accurate segmentation results on a different target-domain dataset. Furthermore, for refining the remote-sensing target geometric representation such as densely distributed buildings, we propose a geometry-consistency (GC) constraint that can be embedded in both image-domain adaptation process and semantic segmentation network. Therefore, our model could achieve cross-domain semantic segmentation with target geometric property preservation. The experimental results on Massachusetts and Inria buildings datasets prove that the proposed unsupervised UGCNet could achieve a very comparable segmentation accuracy with the fully supervised model, which validates the effectiveness of the proposed method.

The objective of this chapter is to detect and recognize the traffic signs using two methods. Traffic signs are silent speakers to the road users. It gives advance information about road conditions ahead. In this chapter, an effective method is proposed to identify the traffic signs in an automated manner. The traffic sign recognition for advanced driver-assistance system of Method I comprises of three stages: (1) detection using Haar-like features with AdaBoost method, (2) feature extraction using speeded up robust features, and (3) classification using support vector machine. Method II comprises of two stages: (1) detection of traffic sign using convolution neural network and (2) classification using convolution neural network. The performance of the system is analyzed. From the experimental results, it is observed that the Method II using convolution neural network gives a better performance compared with Method I

Quantitative identification of sandstone microscopic images is an essential task for sandstone reservoir characterization. The widely-used classical Gazzi-Dickinson point-counting method can be subjective, inconsistent and time-consuming. Furthermore, by directly putting labeled microscopic images of all rock types into image recognition models for training, most previous studies did not address the petrographic principle of artificial identification. In this study, U-Net and U-Net++ semantic segmentation networks that incorporated the sandstone petrographic principle in quantitative identification of sandstone was introduced. Automatic identification of sandstone microscopic images requires prior knowledge learned from the identified sandstones with similar compositions. First, hundreds of thin-sections of typical sandstone reservoirs were selected from several key petroleum basins in China. Second, one-to-one single and orthogonal polarized images were taken for them. Third, the annotation software was used to label the type of each skeleton grain, including quartz, feldspar, lithic fragment and pore. Finally, 480 sets of data, each of which includes single and orthogonal polarized images and their ".json" format annotation results, were obtained for training and testing of the U-Net model to quantitatively analyze sandstone microscopic images. Within the 480 sets of data, 6798 sandstone skeleton grains, including 4542 quartzes, 796 feldspars, 1248 lithic fragments and 212 pores were labeled. The sandstone thin-section quantitative identification model trained by 392 data sets achieved a training accuracy of 96% with the intersection over union at 78% for quartz, and a training accuracy of 88% with the intersection over union at 56% for lithic fragments. The remaining 88 data sets were used for testing, and the accuracy was 87% with its intersection over union at 74% for quartz and a training accuracy of 77% with the intersection over union at 54% for lithic fragments. As a classic fully convolutional network that excels in processing medical images, the U-Net or U-Net++ semantic segmentation network has also performed very well in quantitative identification of sandstone microscopic images. After the proportion of each sandstone skeleton grain has been identified, the simple subdivision descriptive petrographic classification of the sandstone was determined according to the classic Dickinson sandstone taxonomic criteria. In other words, most current deep learning algorithms classify sandstones at the bulk rock level, but this U-Net model has been extended to the mineral level for comprehensive identification. Our vision-based sandstone lithology identification model has not only improved the accuracy of artificial identification but also reduced the instability and subjectivity of the traditional manual processing and expert decision-making approach. In the future, we plan to increase the number and coverage of labeled thin-section images to evaluate the impact on the accuracy and consistency of the U-Net or U-Net++ model, and to expand the approach to identify other terrigenous clastic rock. Furthermore, we hope to improve the capability of the model to identify grains, such as monocrystalline and polycrystalline quartz from "quartz", K-feldspar and plagioclase from "feldspar", and igneous, metamorphic and sedimentary lithic fragments from "lithic fragments".

In traditional 3D reconstruction methods, using a single view to predict the 3D structure of an object is a very difficult task. This research mainly discusses human pose recognition and estimation based on 3D multiview basketball sports dataset. The convolutional neural network framework used in this research is VGG11, and the basketball dataset Image Net is used for pretraining. This research uses some modules of the VGG11 network. For different feature fusion methods, different modules of the VGG11 network are used as the feature extraction network. In order to be efficient in computing and processing, the multilayer perceptron in the network model is implemented by a one‐dimensional convolutional network. The input is a randomly sampled point set, and after a layer of perceptron, it outputs a feature set of n × 16. Then, the feature set is sent to two network branches, one is to continue to use the perceptron method to generate the feature set of n × 1024, and the other network is used to extract the local features of points. After the RGB basketball sports picture passes through the semantic segmentation network, a picture containing the target object is obtained, and the picture is input to the constructed feature fusion network model. After feature extraction is performed on the RGB image and the depth image, respectively, the RGB feature, the local feature of the point cloud, and the global feature are spliced and fused to form a feature vector of N × 1152. There are three branches for this vector network, which, respectively, predict the object position, rotation, and confidence. Among them, the feature dimensionality reduction is realized by one‐dimensional convolution, and the activation function is the ReLU function. After removing the feature mapping module, the accuracy of VC‐CNN_v1 dropped by 0.33% and the accuracy of VC‐CNN_v2 dropped by 0.55%. It can be seen from the research results that the addition of the feature mapping module improves the recognition effect of the network to a certain extent

Traffic signs detection is a hot and important topic in the research field of computer vision and lots of applications, e.g., driver assistance system and autonomous vehicle, etc. Thanks to the development of the convolutional neural network, object detection has achieved promising results in recent years. Nevertheless, traffic sign detection still faces two difficulties due to the characteristics of traffic signs. The first difficulty is the detection of small and blurred traffic signs. The second difficulty is that training samples are sparse and imbalanced. For tackling these two difficulties, Guided Region Enlarging Algorithm (GREA) is proposed in the paper. The GREA consists of two sub-modules, Potential region estimation network (PREN) and NMS-Cropping. The PREN obtains a potential object region of each input image first. Then, the potential object region is enlarged by the NMS-Cropping algorithm for subsequent augmenting training. The GREA makes use of features of a small and blurred traffic sign to enhance the performance of a traffic sign detector. To comprehensively evaluate the proposed approach, two traffic sign datasets include the Swedish traffic sign data set and the Tsinghua-Tencent 100k data set are utilized in the experiment. Experimental results demonstrate the efficiency of the proposed GREA, and our approach is comparable to the state-of-the-art approaches.

In this paper, we propose a traffic sign recognition algorithm based on support vector machine (SVM) and convolutional neural network (CNN) for the traffic sign recognition of intelligent transportation and unmanned vehicle. In the detection of traffic signs, by using color features, firstly we convert the RGB color space to HSV color space, which can get the regions of interest (ROI). Then extract the Histogram of Oriented Gradient (HOG) features, and determine whether it is traffic sign by support vector machine (SVM). This algorithm can successfully detect traffic signs with a high detecting rate. The following is the classification of the detected traffic signs, by using a kind of convolutional neural network, which fuse the RGB information of a picture. In the end, the `German traffic signs' dataset is used to verify the accuracy. Experimental results show that the proposed algorithm can effectively recognize traffic signs, achieving high accuracy and the algorithm complexity is lower.

High-precision land cover maps of remote sensing images based on an intelligent extraction method are an important research field for many scholars. In recent years, deep learning represented by convolutional neural networks has been introduced into the field of land cover remote sensing mapping. In view of the problem that a convolution operation is good at extracting local features but has limitations in modeling long-distance dependence relationships, a semantic segmentation network, DE-UNet, with a dual encoder is proposed in this paper. The Swin Transformer and convolutional neural network are used to design the hybrid architecture. The Swin Transformer pays attention to multi-scale global features and learns local features through the convolutional neural network. Integrated features take into account both global and local context information. In the experiment, remote sensing images from UAVs were used to test three deep learning models including DE-UNet. DE-UNet achieved the highest classification accuracy, and the average overall accuracy was 0.28% and 4.81% higher than UNet and UNet++, respectively. It shows that the introduction of a Transformer enhances the model fitting ability.

In this paper, we propose an automatic myocardial infarction segmentation framework from Delayed Enhancement cardiac MRI (DE-MRI) using a convolutional neural network (CNN) and prior information-based post-treatments. The work was conducted on our DE-MRI dataset, which is collected from daily clinical practice. 195 cases of DE-MRI examinations constitute this dataset, including on average 7 images per case with manually drawn contours by an expert. The objective is to automatically segment myocardial infarctions on both healthy and pathological images in the dataset. In the proposed framework, a downsampling-upsampling segmentation CNN firstly generates high recall segmentations of myocardial infarction from left ventricle DE-MR images, then the proposed prior information-based post-processing method identifies and removes false-positive segmentations from the CNN’s prediction. To obtain a high recall prediction, two U-NET like semantic segmentation networks are investigated: CE-NET and its backbone with Dice loss and Stochastic Gradient Descent (SGD) using a batch size of value 1. The prior information-based post-processing evaluates every single contour in the CNN’s segmentations: region features in each contour are compared to criteria which are firstly estimated based on the training set images and eventually fine-tuned based on the validation set images. All non-conforming contours are removed from the predictions to improve the accuracy of the segmentation. Combining the high recall networks and prior postprocessing information, we achieve segmentation results comparable to those produced by human experts.

Vision-based perception is an import link in driverless technology, and semantic segmentation is one of the mostly uses atrous convolution with a large amount of computation and high memory consumption to extract high-resolution feature maps. As a result, the current mainstream semantic segmentation network lacks the segmentation speed and cannot be effectively applied in driverless technology. To solve this problem, a semantic segmentation network with better real-time performance is proposed. Firstly, a lightweight convolutional neural network is used as the encoder, and stride convolution and regular convolution are used to replace the time-consuming and memory-consuming atrous convolution. Secondly, in order to obtain the feature map similar to Deeplab v3+, a new joint upsampling module, Multi-scale Feature Map Joint Pyramid Upsamping(MJPU), is proposed. By fusing multiple feature maps in the encoder, a high resolution feature map with richer semantic information is generated. Experments on the Cityscapes dataset show that, compared with the popular semantic segmentation network Deeplab V3+, the proposed network can improve the segmentation speed to 31.4 FPS/s without losing a lot of performance(mIoU of 44.03%). Therefore, the network proposed in this paper has better real-time performance and is more suitable for driverless scenes.

The lane line detection methods based on semantic segmentation networks have achieved remarkable results in recent years. However, the semantic segmentation networks are aimed at the pixel level. These methods have a large amount of computation, which can reduce the real-time performance. To reduce the calculation, we propose a Light-weight Lane line detection Network (LLNet). A new sub-layer is established. We also adopt a jumping structure between two sub-layers to enhance the supervisory role of ground truth. Furthermore, we adopt two branches, including instance segmentation and embeddable branch. The combination of two branches can filter out the wrongly detected pixels and further improve the accuracy of detection. Experimental results in the Tusimple dataset show that the detection accuracy of the proposed network is comparable with LaneNet. Meanwhile, it has good real-time performance, and the processing time of a single image is 10.285 ms, which is about one-half of LaneNet.

Traffic sign recognition systems (TSR) are a hot research topic because of their importance in the field of driver safety. Major car companies are developing smart driving systems for this purpose. TSR was introduced in 2022 in some luxury cars, such as the Cadillac XT4 and Mercedes-Benz GLS. TSR is a driver assistance feature that can be used to advise and warn the driver about possible road limitations along the current route. The development of the TSR system aims at two objectives; the first is to develop a fast and inexpensive system that can be added to economic vehicles. The second objective is to apply safe leadership automatically when accessing a fast and reliable TSR system. Following laws can be made compulsory by the vehicle itself, such as not exceeding speed in the event of a speed limit. (except in emergencies).There are many issues and challenges facing the researcher that works on the detection and recognition of traffic signs, such as bad weather, various illumination, occlusion, spped of cars.This research paper aims to present a proposal that facilitates access to a fast, reliable, and low-cost system. The proposed TSR is based on combining the You Only Lock Once (YOLOv5) network with a convolution neural network (CNN) that enhances the accuracy of detection and recognition in addition to speeding up the processing. Two datasets were used for training models. The YOLOv5 was trained on the GTSDB, while CNN was trained on the GTSRB.YOLOv5 is used to detect the traffic signs in the input image. The detected signs are bounded by boxes and labeled according to the group that belongs to them (the dataset is clustered into four groups (prohibitory, mandatory, danger, and others)), and then each bounded box is segmented from the rest of the image background. The segmented image (including the traffic sign) is input to the CNN to be classified into one of the 43 traffic signs.This proposed system achieved an average of 94% detection accuracy by using YOLO and 99.95% classification accuracy in CNN, with a processing time of 0.031 seconds per frame. In real-time applications, this low computation time can improve the system's stability and dependability.

Semantic segmentation plays an increasingly important role in the field of computer vision. The current semantic segmentation algorithms are mainly based on full convolutional network downsampling for feature extraction, however, the reduction of image resolution due to downsampling will inevitably lead to the loss of image information and the process is irreversible. Although features can be reused to compensate for the loss, the problem of lost pixel correlation and inaccurate segmentation results still occur. In order to solve these problems, the attention mechanism and the generative adversarial network are fused into the semantic segmentation network, and a semantic segmentation network is proposed. The attention mechanism is added to the generator and discriminator modules and trained successively, then the generator module with rich information and global information is extracted, and its output is fused with the feature graph sampled above, in order to reduce the loss of local information and complete the task of image segmentation, an interpolation method based on image edge is introduced to sample the image, which improves the bilinear interpolation edge blur. Experiments on Cityscapes dataset show that the proposed network model is effective and reliable.