Oblique Convolution: A Novel Convolution Idea for Redefining Lane Detection

The robustness and stability of lane detection is vital for advanced driver assistance vehicle technology and even autonomous driving technology. To meet the challenges of real-time lane detection in complex traffic scenes, a simple but robust multilane detection method is proposed in this paper. The proposed method breaks down the lane detection task into two stages, that is, lane line detection algorithm based on instance segmentation and lane modeling algorithm based on adaptive perspective transform. Firstly, the lane line detection algorithm based on instance segmentation is decomposed into two tasks, and a multitask network based on MobileNet is designed. This algorithm includes two parts: lane line semantic segmentation branch and lane line Id embedding branch. The lane line semantic segmentation branch is mainly used to obtain the segmentation results of lane pixels and reconstruct the lane line binary image. The lane line Id embedding branch mainly determines which pixels belong to the same lane line, thereby classifying different lane lines into different categories and then clustering these different categories. Secondly, the adaptive perspective transformation model is adopted. In this model, the motion information is used to accurately convert the original image into a bird’s-eye view image, and then the least-squares second-order polynomial fitting is performed on the lane line pixels. Finally, experiments on the CULane dataset show that the proposed method achieved similar or better performance compared with several state-of-the-art methods, the F1 score of the proposed method in the normal test set and most challenge test sets is better than other algorithms, which verifies the effectiveness of the proposed method, and then the field experiments results show that the proposed method has good practical application value in various complex traffic scenes.

As is known to all, there are thousands of people died in traffic accidents in the world every year and 80% of the traffic accidents are due to the drivers' inattention. In order to reduce the occurrence of car accidents, the design of Cognitive Vehicle Safety (CVS) system has attracted wide attention. As an important functional system of the CVS system, Lane Departure Warning system's function is to warn the driver when the vehicle changes the lane without the driver's intention. In this paper, we present a method of lane detection which based on Lane geometric model and image connected region process. The model of the lane is produced by the projection of camera, which makes the parallel lane lines in real world become intersect. The lines on the same side of lane have the same geometric feature that can be used to build lane line geometric model for detection. The lane line can be classified as different lane line models by the geometric feature of it. In this paper, due to the different features of lane line in different images, the operation and the classification of modeling performs at the same time. But before that, we go through some process to pre-process the image, such as the image connected region process, morphology tophat filter, etc. The combination of these modules can overcome the universal lane detection problems, such as the road in different light conditions. Experimental results on real road will be presented to prove the effectiveness of the proposed lane detection algorithm.

<div class="section abstract"><div class="htmlview paragraph">Lane detection plays a critical role in autonomous vehicles for safe and reliable navigation. Lane detection is traditionally accomplished using a camera sensor and computer vision processing. The downside of this traditional technique is that it can be computationally intensive when high quality images at a fast frame rate are used and has reliability issues from occlusion such as, glare, shadows, active road construction, and more. This study addresses these issues by exploring alternative methods for lane detection in specific scenarios caused from road construction-induced lane shift and sun glare. Specifically, a U-Net, a convolutional network used for image segmentation, camera-based lane detection method is compared with a radar-based approach using a new type of sensor previously unused in the autonomous vehicle space: radar retro-reflectors. This evaluation is performed using ground truth data, obtained by measuring the lane positions and transforming them into pixel coordinates. The performance of each method is assessed using the statistical <i>R</i><sup>2</sup> score, indicating the correlation between the detected lane lines and the ground truth. The results show that the U-Net camera-based method exhibits limitations in accurately detecting and aligning the lane lines, particularly in challenging scenarios. However, the radar-based lane detection method demonstrates a strong correlation with the ground truth which implies that the use of this sensor may improve current reliability issues from conventional camera lane detection approach. Furthermore, the study highlights the limitations of the U-Net model for camera lane detection, especially in scenarios with sun glare. This study shows that infrastructure-based radar retro-reflectors can improve autonomous vehicle lane detection reliability. The integration of different sensor modalities and the development of advanced computer vision algorithms are crucial for improving the accuracy, reliability, and energy efficiency of lane detection systems. Addressing these challenges contributes to the advancement of autonomous vehicles and the realization of safer and more efficient transportation systems.</div></div>

CFI-Former: Efficient lane detection by multi-granularity perceptual query attention transformer.

Lane detection is an essential visual capability for the autonomous vehicle navigation used in the intelligent transportation system (ITS). Several approaches for lane detection have been suggested in the past. However, there is still one issue about robustness. This paper presents a robust and real-time multi-lane detection method based on road marking feature points (RMFP) for autonomous vehicle navigation in an urban environment. The key idea in this paper is to apply methods from extracting RMFP and the target tracking domain to identify lane information. Then the RMFP is extracted from the gray-scale image and the IPM image. Additionally, the lane line color and structure features are used to sift through the RMFP that meet lane lines. Lastly, the clustering method is used to generate lane lines, and these lines are tracked by frame association and Kalman Filter. The experiment results based on image sequences taken under various visual conditions showed that the proposed system can identify the lane line markings with 98% accuracy. The method shows much robustness under various complicated scenarios and meets the real-time requirement.

Lane detection plays a pivotal role in the successful implementation of Advanced Driver Assistance Systems (ADASs), which are essential for detecting the road’s lane markings and determining the vehicle’s position, thereby influencing subsequent decision making. However, current deep learning-based lane detection methods encounter challenges. Firstly, the on-board hardware limitations necessitate an exceptionally fast prediction speed for the lane detection method. Secondly, improvements are required for effective lane detection in complex scenarios. This paper addresses these issues by enhancing the row-anchor-based lane detection method. The Transformer encoder–decoder structure is leveraged as the row classification enhances the model’s capability to extract global features and detect lane lines in intricate environments. The Feature-aligned Pyramid Network (FaPN) structure serves as an auxiliary branch, complemented by a novel structural loss with expectation loss, further refining the method’s accuracy. The experimental results demonstrate our method’s commendable accuracy and real-time performance, achieving a rapid prediction speed of 129 FPS (the single prediction time of the model on RTX3080 is 15.72 ms) and a 96.16% accuracy on the Tusimple dataset—a 3.32% improvement compared to the baseline method.

Lane detection plays a vital role in making the idea of the autonomous car a reality. Traditional lane detection methods need extensive hand-crafted features and post-processing techniques, which make the models specific feature-oriented, and susceptible to instability for the variations on road scenes. In recent years, Deep Learning (DL) models, especially Convolutional Neural Network (CNN) models have been proposed and utilized to perform pixel-level lane segmentation. However, most of the methods focus on achieving high accuracy while considering structured roads and good weather conditions and do not put emphasis on testing their models on defected roads, especially ones with blurry lane lines, no lane lines, and cracked pavements, which are predominant in the real world. Moreover, many of these CNN-based models have complex structures and require high-end systems to operate, which makes them quite unsuitable for being implemented in embedded devices. Considering these shortcomings, in this paper, we have introduced a novel CNN model named LLDNet based on an encoder–decoder architecture that is lightweight and has been tested in adverse weather as well as road conditions. A channel attention and spatial attention module are integrated into the designed architecture to refine the feature maps for achieving outstanding results with a lower number of parameters. We have used a hybrid dataset to train our model, which was created by combining two separate datasets, and have compared the model with a few state-of-the-art encoder–decoder architectures. Numerical results on the utilized dataset show that our model surpasses the compared methods in terms of dice coefficient, IoU, and the size of the models. Moreover, we carried out extensive experiments on the videos of different roads in Bangladesh. The visualization results exhibit that our model can detect the lanes accurately in both structured and defected roads and adverse weather conditions. Experimental results elicit that our designed method is capable of detecting lanes accurately and is ready for practical implementation.

Lane detection and tracking technique is the autonomous driving basis for Rubber-Tired Gantries (RTGs), vital to the automation and intelligence updating of man-driven container terminals. However, the existing lane detection methods developed for common road scenarios cannot meet the high-precision and robust all-weather requirements of RTG autonomous driving. In this article, we propose an Adaptive Edge-based Lane Detection and Tracking method considering RTG lanes’ characteristics in this paper. First, the candidate edges of lane lines are detected and paired based on the enhanced gradient features. Next, inverse perspective mapping is employed to search the right edges, followed by an adaptive sliding-window method. Ultimately, we develop an adaptive Kalman filter to track lane lines robustly, detecting confidence weighting by relaxing the constraint of lane line width. The proposed method is tested in an actual container yard, the lane centerline’s average position error is 2.051 pixels, and the detection success rate is close to 100%.

In this paper, we propose a new approach for a robust multirate lane-keeping control scheme with predictive virtual lanes. First, the multirate lane-keeping control scheme is proposed to improve the lane-keeping performance and to reduce the ripple in the yaw rate. To improve the lane-keeping performance on a curved road, the integral of the lateral offset error is added to the state feedback controller. A multirate Kalman filter (KF) has been developed to resolve the problems caused by slow lane detection due to the vision processing system. This multirate KF estimates vehicle states at a fast rate using a microprocessor. Utilizing the estimated states, the linear quadratic state feedback control operates at the same fast update rate of the microprocessor. Thus, a multirate control scheme can reduce the ripple in the yaw rate. Second, we propose a virtual lane prediction method that compensates for the momentary failure of lane detection from unexpected problems. If the camera sensor momentarily fails while obtaining lane information, the predicted virtual lane can be substituted for the lane detection using the camera sensor in the proposed control scheme. Thus, the proposed control scheme can normally operate when the lane information is momentarily unavailable. The performance of the proposed method was evaluated via experiments.

This paper presents a map-matching-based vehicle localization algorithm for application to automated driving on urban road. Vehicle position estimation of centimeter-level with low-priced commercial sensor setup is one of the key issue in urban automated driving. The information fusion method of localization algorithm utilizes vehicle chassis sensor and Around View Monitoring (AVM) module with four fish-eyed cameras. The proposed localization algorithm consists of three sections: a lane detection, a position correction, and a localization filter. A lane information is extracted from AVM image around the vehicle. This lane information is possible to correct vehicle position by the iterative closest point (ICP) algorithm which estimates the rigid transformation between the lane map and lanes obtained by AVM in real-time. The corrected vehicle position by this transformation is fused with the information of vehicle sensors based on an extended Kalman filter (EKF). In order to achieve higher accuracy, the covariance of the ICP algorithm is estimated by using Haralick's method. The performance of proposed localization is verified through vehicle experiments on proving ground and actual urban road.

Lane detection is one of the most fundamental problems in the rapidly developing field of autonomous vehicles. With the dramatic growth of deep learning in recent years, many models have achieved a high accuracy for this task. However, most existing deep-learning methods for lane detection face two main problems. First, most early studies usually follow a segmentation approach, which requires much post-processing to extract the necessary geometric information about the lane lines. Second, many models fail to reach real-time speed due to the high complexity of model architecture. To offer a solution to these problems, this paper proposes a lightweight convolutional neural network that requires only two small arrays for minimum post-processing, instead of segmentation maps for the task of lane detection. This proposed network utilizes a simple lane representation format for its output. The proposed model can achieve 93.53% accuracy on the TuSimple dataset. A hardware accelerator is proposed and implemented on the Virtex-7 VC707 FPGA platform to optimize processing time and power consumption. Several techniques, including data quantization to reduce data width down to 8-bit, exploring various loop-unrolling strategies for different convolution layers, and pipelined computation across layers, are optimized in the proposed hardware accelerator architecture. This implementation can process at 640 FPS while consuming only 10.309 W, equating to a computation throughput of 345.6 GOPS and energy efficiency of 33.52 GOPS/W.

LaneScanNET: A deep-learning approach for simultaneous detection of obstacle-lane states for autonomous driving systems

Lane departure warning system (LDWS) has been regarded as an efficient method to lessen the damages of road traffic accident resulting from driver fatigue or inattention. Lane detection is one of the key techniques for LDWS. To overcome the contradiction between complexity of algorithm and the real‐time requirement for vehicle onboard system, this paper introduces a new lane detection method based on intelligent CCD parameters regulation. In order to improve the real‐time capability of the system, a CCD parameters regulating method is proposed which enhances the contrast between lane line and road surfaces and reduces image noise, so it lays a good foundation for the following lane detection. Hough transform algorithm is improved by selection and classification of seed points. Finally the lane line is extracted through some restrictions. Experimental results verify the effectiveness of the proposed method, which improves not only real‐time capability but also the accuracy of the system.

Lane detection is a crucial task in the field of autonomous driving, as it enables vehicles to safely navigate on the road by interpreting the high-level semantics of traffic signs. Unfortunately, lane detection is a challenging problem due to factors such as low-light conditions, occlusions, and lane line blurring. These factors increase the perplexity and indeterminacy of the lane features, making them hard to distinguish and segment. To tackle these challenges, we propose a method called low-light enhancement fast lane detection (LLFLD) that integrates the automatic low-light scene enhancement network (ALLE) with the lane detection network to improve lane detection performance under low-light conditions. Specifically, we first utilize the ALLE network to enhance the input image’s brightness and contrast while reducing excessive noise and color distortion. Then, we introduce symmetric feature flipping module (SFFM) and channel fusion self-attention mechanism (CFSAT) to the model, which refine the low-level features and utilize more abundant global contextual information, respectively. Moreover, we devise a novel structural loss function that leverages the inherent prior geometric constraints of lanes to optimize the detection results. We evaluate our method on the CULane dataset, a public benchmark for lane detection in various lighting conditions. Our experiments show that our approach surpasses other state of the arts in both daytime and nighttime settings, especially in low-light scenarios.

In order to adapt to various complex lane environments, a general fast lane extraction algorithm was proposed. Firstly, according to the mutability of the local gray value and the characteristics of image saliency, the collected road image was preprocessed, and the region of interest was obtained by the proposed double threshold algorithm. Only in the significant region of the road were the image gray value change and image smoothing carried out to solve the problems of consuming too much time and poor noise resistance in lane detection. The lane line edge was then extracted based on the improved Canny operator. When the Otsu threshold was selected, the Kalman filter algorithm was introduced to quickly predict the optimal threshold of the subsequent image sequence in accordance with the characteristics of optimized autoregressive data processing.. Finally, for the straight line fitted by the Hough transform, an effective multi-layer evaluation function was established to realize the online correction of lane lines. Compared with the traditional lane line extraction algorithm, the experimental results show that the proposed algorithm has better accuracy, real-time performance and robustness.