Automatic Traffic Light Detection Using AI for VLC

Data rate and communication distance are two important criteria for measuring the performance of optical camera communication (OCC) systems. Intensive researches have been made to find possible solutions to improve these two performance indicators. However, most existing literature either provides solutions which closely related to the experimental parameters of their own designed OCC systems or provides models which are suitable for visible light communication (VLC) and cannot be directly used in OCC. Motivated by this, we propose a simplified system model for OCC in this paper. By jointly applying Lambertian model for the transmitter light-emitting diode (LED), channel direct current (DC) gain for free space channel, and mathematical descriptions for the imaging process of the camera receiver, the proposed model provides the relationship between the optical transmission power and the final pixel values on the image sensor. We specifically explore the parameters of the transmission distance and the exposure time based on the proposed model. Simulation results qualitatively and quantitatively explore the influences of these two parameters on the final pixel values. Our proposed model can provide guidance for the design of a generalized OCC system as well as the selection of demodulating threshold, thus it can help to realize a highperformance OCC system.

In this paper, we look at the various approaches to traffic light and street sign detection. In the traditional image processing-based approach, we primarily use a circular Hough transform with color masks to detect traffic lights. For the more complex approach, we deploy convolution neural networks in combination with previously proposed feature extractors to aid with traffic light detection along with a confidence score. We make use of the SSD ResNet V1 trained on the COCO dataset in addition to the Inception V3 architecture trained on the ImageNet dataset. All of these experiments are performed on the Berkeley Deep Drive dataset. We compare these techniques based on parameters like accuracy, computation time, and memory usage, and highlight the advantages and disadvantages of each type.KeywordsComputer visionConvolutional neural networksHough transformImage processingStreet sign detectionTraffic light recognition

As mobile payments become increasingly popular, the security of the transaction process should be taken more seriously. Many mobile phones are equipped with complementary metal-oxide-semiconductor (CMOS) cameras, which can be used as visible light communication (VLC) receivers. An optical camera communication (OCC) system is a special VLC system that uses an optical camera as a receiver. OCC offers acceptable security, but challenges remain. In this paper, we propose an OCC system that uses an optical camera with a rolling shutter as a receiver and a light-emitting diode (LED) panel light as a transmitter. The feature of an LED panel light serving as a surface light source reduces the possibility of overexposure and avoids the blooming effect of the camera. The rolling shutter effect of the CMOS camera is used to increase the data rate, which is numerically much higher than the frame rate. Finally, the results of a symbol error rate (SER) test show that the VLC system exhibits error-free performance. The data rate is approximately 4.2kb/s, while the net data rate is approximately 2.46kb/s due to the redundancy of the 3-9 code bar code and the redundant cycle. This data rate is sufficient for mobile payments, which require only 60bits of data at a time.

To enable a higher transmission rate in optical camera communication (OCC) systems, severe inter-symbol interference (ISI) occurs owing to the reduction of the number of pixel-row-per-bit (PPB). Therefore, those pixels representing data bit 0 or 1 may have same gray scale values, significantly deteriorating the bit decision when using the conventional thresholding scheme. In this Letter, a simple yet efficient scheme, referred to as nearest neighbor bit assisted decision (NNBAD) scheme, is proposed and experimentally demonstrated for signal decision in OCC systems. NNBAD leverages the nearest neighbor bit to jointly assist bit decision for pixels with severe ISI. Experimental results show that, for OCC systems with on-off keying (OOK) modulation, those pixels with severe ISI cannot be distinguished by the conventional thresholding scheme. Yet, the NNBAD scheme exhibits strong robustness against ISI, remarkably improving the bit error rate (BER) performance. The proposed scheme can achieve a throughput of 8.2 kbps with OOK modulation under an illuminance of 600 lx.

An automatic traffic light recognition system is proposed in this paper so that car drivers have sufficient information to make a correct decision. This in turn facilitates the construction of an ITS (Intelligent Transportation System). The proposed method can be applied to movable cameras, is adaptable to environmental variations and is proper for real‐time implementation. It consists of two phases: traffic light detection and classification. The two phases are based on color, region and border information. At the detection stage, the RGB color space is first converted into the HSI color space so as to find those regions with specific colors of traffic lights. The morphology technology is employed to remove holes and noise. Region labeling is then involved to detect candidate regions of traffic lights. At the classification stage, border detection is employed to obtain region borders which together with region colors are matching features. In this study, circle and arrow traffic lights can both be coped with. Moreover, our method is adaptable to environmental variations in numbers of lights, types of lights and time slots. Various experiments have been conducted to demonstrate the effectiveness and practicability of the proposed method.

Nowadays, underwater activities are becoming more and more important. As the number of underwater sensing devices grows rapidly, the amount of bandwidth needed also increases very quickly. Apart from underwater communication, direct communication across the water–air interface is also highly desirable. Air-to-water wireless transmission is crucial for sending control information or instructions from unmanned aerial vehicles (UAVs) or ground stations above the sea surface to autonomous underwater vehicles (AUVs). On the other hand, water-to-air wireless transmission is also required to transmit real-time information from AUVs or underwater sensor nodes to UAVs above the water surface. Previously, we successfully demonstrated a water-to-air optical camera-based OWC system, which is also known as optical camera communication (OCC). However, the reverse transmission (i.e., air-to-water) using OCC has not been analyzed. It is worth noting that in the water-to-air OCC system, since the camera is located in the air, the image of the light source is magnified due to diffraction. Hence, the pixel-per-symbol (PPS) decoding of the OCC pattern is easier. In the proposed air-to-water OCC system reported here, since the camera is located in the water, the image of the light source in the air will be diminished in size due to diffraction. Hence, the PPS decoding of the OCC pattern becomes more difficult. In this work, we propose and experimentally demonstrate a wide field-of-view (FOV) air-to-water OCC system using CUDA Deep-Neural-Network Long-Short-Term-Memory (CuDNNLSTM). Due to water turbulence and air turbulence affecting the AUV and UAV, a precise line-of-sight (LOS) between the AUV and the UAV is difficult to achieve. OCC can provide wide FOV without the need for precise optical alignment. Results revealed that the proposed air-to-water OCC system can support a transmission rate of 7.2 kbit/s through a still water surface, and 6.6 kbit/s through a wavy water surface; this satisfies the hard-decision forward error correction (HD-FEC) bit-error-rate (BER).

Recent improvements in object detection are driven by the success of convolutional neural networks (CNN). They are able to learn rich features outperforming hand-crafted features. So far, research in traffic light detection mainly focused on hand-crafted features, such as color, shape or brightness of the traffic light bulb. This paper presents a deep learning approach for accurate traffic light detection in adapting a single shot detection (SSD) approach. SSD performs object proposals creation and classification using a single CNN. The original SSD struggles in detecting very small objects, which is essential for traffic light detection. By our adaptations it is possible to detect objects much smaller than ten pixels without increasing the input image size. We present an extensive evaluation on the DriveU Traffic Light Dataset (DTLD). We reach both, high accuracy and low false positive rates. The trained model is real-time capable with ten frames per second on a Nvidia Titan Xp. Code has been made available at https://github.com/julimueller/tl_ssd.

Traditional traffic light detection systems rely on handcrafted features and classical machine learning algorithms. Accurately detecting traffic lights amidst the complexities of real-world traffic scenarios presents formidable challenges. This paper enhances YOLOv8 for accurate and efficient traffic light detection. A Dual Attention Network (DUANet) integrates into the backbone for improved feature representation. A Spatial Pyramid Pooling Cross-Stage Partial Connection (SPPCSPC) layer in the neck handles varying input sizes. The integration of DUANet facilitates the capture of both channel-wise and spatial dependencies within feature maps, enhancing feature representation and enabling more accurate traffic light localization. DUA-YOLOv8 outperforms the baseline YOLOv8 model, achieving an impressive mean Average Precision (mAP) of 0.995 % while maintaining real-time inference speeds of 55.248 frames per second (FPS). Comparative analysis against existing methods further validates the superiority of our approach, particularly in complex traffic scenarios. Our study contributes to the advancement of autonomous driving technology by addressing the critical task of traffic light detection.

Traffic light detection is an important system because it can alert driver on upcoming traffic light so that he/she can anticipate a head of time. In this paper we described our work on detecting traffic light color using machine learning approach. Using HSV color representation, our approach is to extract features based on an area of X×X pixels. Traffic light color model is then created by applying a learning algorithm on a set of examples of features representing pixels of traffic and non-traffic light colors. The learned model is then used to classify whether an area of pixels contains traffic light color or not. Evaluation of this approach reveals that it significantly improves the detection performance over the one based on value-range color segmentation technique.

Currently, wireless communication systems that use radio frequency are commonly deployed, for example, mobile communication systems, satellite systems, and the Internet of Things (IoT) systems. Based on their easy installation, wireless communication systems have benefits over other wired communication systems. However, using high frequencies to transfer data via wireless communication can hold significant risks for human health. Several researchers have studied this topic using visible light instead of Radio Frequency (RF) waveforms in communication systems. Many potential approaches are relevant in this regard, i.e., visible light communication, light fidelity, free-space optical, and optical camera communication. Artificial intelligence is also influencing the future of industry and people and is used to solve complex problems, create intelligent solutions, and replace human intelligence as the driving force behind emerging technologies such as big data, smart factories, and the IoT. In this paper, we proposed the architecture of the MIMO C-OOK (Multiple-Input Multiple-Output Camera On–Off Keying) scheme, which uses a convolutional neural network for light-emitting diode detection and a deep learning neural network for threshold predictions considering long-distance communication and mobility support. Our suggested method aimed to improve the performance of the traditional camera on–off keying scheme by increasing data rate, communication distance, and low bit error rate. Our suggested technique may achieve a communication distance of up to 22 m with a low error rate when considering the mobility impact (2 m/s, i.e., walking velocity) by controlling the exposure time, focal length, and employing Forward Error Correction code.

In order to increase the data rate of the optical camera communication (OCC) system, the 8-composite-amplitude-shift-keying modulation (8CASK) OCC system is used in this work. However, if the static decision thresholds are employed to demodulate multi-level ASK signal, uneven illumination of LED lamps and LED light-emitting instability lead to the fluctuation of the gray range in the picture and degrade the bit-error-rate (BER) performance. In this work, we propose and demonstrate a demodulation scheme, using the uneven illumination compensation algorithm, the pixel matrix threshold overall update algorithm and the secondary decision algorithm, to mitigate the impact of illumination unevenness and LED light-emitting instability. The BER performance is evaluated and compared with other demodulation schemes. The experimental results demonstrate that the communication rate of our proposed scheme can reach 9kbit/s at a distance of 250 cm where the illumination is 135lux, and the BER is 8.01 × 10−5.

Emerging intelligent transportation systems (ITS) are based on several technologies. A new concept of integrating visible light communications (VLC) in ITS is introduced in this paper. VLC in ITS is a cost effective method of implementation. This paper presents a VLC broadcast system considering LED-based traffic lights. It discusses the integration of traffic light road side unit (RSUs) within the existing ITS architecture. A prototype demonstrator of the designed VLC systems is also presented. A robust modulation technique based on direct sequence spread spectrum (DSSS) sequence inverse keying (SIK) has been implemented to minimize the effect of noise sources. Experimental results show data communication range of over 40m with 200mm custom designed traffic light, even during bright sun light.

A new traffic light detection and recognition algorithm for electronic travel aid was realized. Traffic lights were detected by rectangular and circular shapes extraction. Color distributions of the circular regions in HSI and RGB color space, combined with the position of the circular shape in the rectangle were used to recognize the traffic light. Gray level normalization, top hat transformation and image interception were used to reduce the disturbances of the environment. The average recognition rate is 97% and the speed is 0.086s per frame. Video analysis was used to improve the performance of the system.

There is great concern over growing road accidents and associated fatalities. In order to reduce accidents, improve congestion and offer smooth flow of traffic, several measures, such as providing intelligence to transport, providing communication infrastructure along the road, and vehicular communication, are being undertaken. Traffic safety information broadcast from traffic lights using Visible Light Communication (VLC) is a new cost effective technology which assists drivers in taking necessary safety measures. This chapter presents the VLC broadcast system considering LED-based traffic lights. It discusses the integration of traffic light Roadside Units (RSUs) with upcoming Intelligent Transportation Systems (ITS) architecture. Some of the offered services using this technology in vehicular environment together with future directions and challenges are discussed. A prototype demonstrator of the designed VLC systems is also presented.

Traffic lights detection and their state recognition is a crucial task that autonomous vehicles must reliably fulfill. Despite scientific endeavors, it still is an open problem due to the variations of traffic lights and their perception in image form. Unlike previous studies, this paper investigates the use of inaccurate and publicly available GIS databases such as OpenStreetMap. In addition, we are the first to exploit conic section geometry to improve the shape cue of the traffic lights in images. Conic section also enables us to estimate the pose of the traffic lights with respect to the camera. Our approach can detect multiple traffic lights in the scene, it also is able to detect the traffic lights in the absence of prior knowledge, and detect the traffics lights as far as 70 meters. The proposed approach has been evaluated for different scenarios and the results show that the use of stereo cameras significantly improves the accuracy of the traffic lights detection and pose estimation.