Pervasive Artificial Intelligence Research Labs

This paper proposes a deep learning-based mmWave radar and RGB camera sensor early fusion method for object detection and tracking and its embedded system realization for ADAS applications. The proposed system can be used not only in ADAS systems but also to be applied to smart Road Side Units (RSU) in transportation systems to monitor real-time traffic flow and warn road users of probable dangerous situations. As the signals of mmWave radar are less affected by bad weather and lighting such as cloudy, sunny, snowy, night-light, and rainy days, it can work efficiently in both normal and adverse conditions. Compared to using an RGB camera alone for object detection and tracking, the early fusion of the mmWave radar and RGB camera technology can make up for the poor performance of the RGB camera when it fails due to bad weather and/or lighting conditions. The proposed method combines the features of radar and RGB cameras and directly outputs the results from an end-to-end trained deep neural network. Additionally, the complexity of the overall system is also reduced such that the proposed method can be implemented on PCs as well as on embedded systems like NVIDIA Jetson Xavier at 17.39 fps.

To overcome the limitations of standard datasets with data at a wide-variety of scales and captured in the various conditions necessary to train neural networks to yield efficient results in ADAS applications, this paper presents a self-built open-to-free-use ‘iVS dataset’ and a data annotation tool entitled ‘ezLabel’. The iVS dataset is comprised of various objects at different scales as seen in and around real driving environments. The data in the iVS dataset are collected by employing a camcorder in vehicles driving under different conditions, e.g., light, weather and traffic, and driving scenarios ranging from city traffic during peak and normal hours to freeway traffics during busy and normal conditions. Thus, the collected data are wide-ranging and captured all possible objects at various scales appearing in real-time driving situations. The data collected in order to build the dataset has to be annotated before use in training the CNNs and so this paper presents an open-to-free-use data annotation tool, ezLabel, for data annotation purposes as well.

The 2021 embedded deep learning object detection model compression competition for traffic in Asian countries held in IEEE ICMR2021 Grand Challenges focuses on the object detection technologies in autonomous driving scenarios. The competition aims to detect objects in traffic with low complexity and small model size in the Asia countries (e.g., Taiwan), which contains several harsh driving environments. The target detected objects include vehicles, pedestrians, bicycles and crowded scooters. There are 89,002 annotated images provided for model training and 1,000 images for validation. Additional 5,400 testing images are used in the contest evaluation process, in which 2,700 of them are used in the qualification stage competition, and the rest are used in the final stage competition. There are in total 308 registered teams joining this competition this year, and the top 15 teams with the highest detection accuracy entering the final stage competition, from which 9 teams submitted the final results. The overall best model belongs to team "as798792", followed by team "Deep Learner" and team "UCBH." Two special awards of best accuracy award best and bicycle detections go to the same team "as798792," and the other special award of scooter detection goes to team "abcda."

Deep reinforcement learning (RL), where the agent learns from mistakes, has been successfully applied to a variety of tasks. With the aim of learning collision-free policies for unmanned vehicles, deep RL has been used for training with various types of data, such as colored images, depth images, and LiDAR point clouds, without the use of classic map--localize--plan approaches. However, existing methods are limited by their reliance on cameras and LiDAR devices, which have degraded sensing under adverse environmental conditions (e.g., smoky environments). In response, we propose the use of single-chip millimeter-wave (mmWave) radar, which is lightweight and inexpensive, for learning-based autonomous navigation. However, because mmWave radar signals are often noisy and sparse, we propose a cross-modal contrastive learning for representation (CM-CLR) method that maximizes the agreement between mmWave radar data and LiDAR data in the training stage. We evaluated our method in real-world robot compared with 1) a method with two separate networks using cross-modal generative reconstruction and an RL policy and 2) a baseline RL policy without cross-modal representation. Our proposed end-to-end deep RL policy with contrastive learning successfully navigated the robot through smoke-filled maze environments and achieved better performance compared with generative reconstruction methods, in which noisy artifact walls or obstacles were produced. All pretrained models and hardware settings are open access for reproducing this study and can be obtained at https://arg-nctu.github.io/projects/deeprl-mmWave.html

A deep ensemble convolutional neural network (CNN) model to inspect printed circuit board (PCB) board dual in-line package (DIP) soldering defects with Hybrid-YOLOv2 (YOLOv2 as a foreground detector and ResNet-101 as a classifier) and Faster RCNN with ResNet-101 and Feature Pyramid Network (FPN) (FRRF) achieved a detection rate of 97.45% and a false alarm rate (FAR) of 20%–30% in the previous study [34] . However, applying the method to other production lines, environmental variations, such as lighting, orientations of the sample feeds, and mechanical deviations, led to the degradation in detection performance. This article proposes an effective self-adaption method that collects “exception data” like the samples with which the Artificial Intelligent (AI) model made mistakes from the automated optical inspection inference edge to the training server, retraining with exceptions on the server and deploying back to the edge. The proposed defect detection system has been verified with real tests that achieved a detection rate of 99.99% with an FAR 20%–30% and less than 15 s of inspection time on a resolution $7296 \times 6000$ PCB image. The proposed system has proven capable of shortening inspection and repair time for online operators, where a 33% efficiency boost from the three production lines of the collaborated factory has been reported [6] . The contribution of the proposed retraining mechanism is threefold: 1) because the retraining process directly learns from the exceptions, the model can quickly adapt to the characteristic of each production line, leading to a fast and reliable mass deployment; 2) the proposed retraining mechanism is a necessary self-service for conventional users as it incrementally improves the detection performance without professional guidance or fine-tuning; and 3) the semiautomatic exception data collection method helps to reduce the time-consuming manual labeling during the retraining process.

ABSTRACT When genetically modified (GM) maize is planted in an open field, it may cross-pollinate with the nearby non-GM maize under certain airflow conditions. Suitable sampling methods are crucial for tracing adventitious GM content. By using field data and bootstrap simulation, we evaluated the performance of common sampling schemes to determine the adventitious GM content in small maize fields in Taiwan. A pollen dispersal model that considered the effect of field borders, which are common in Asian agricultural landscapes, was used to predict the cross-pollination (CP) rate. For the 2009–1 field data, the six-transect (Tsix), JM method for low expected flow (JM[L]), JM method for high expected flow (JM[H]), and V-shaped transect (TV) methods performed comparably to simple random sampling (SRS). Tsix, TV, JM(L), and JM(H) required only 13% or less of the sample size required by SRS. After the simulation and verification of the 2009–2 and 2010–1 field data, we concluded that Tsix, TV, JM(L), and systematic random sampling methods performed equally as well as SRS in CP rate predictions. Our findings can serve as a reference for monitoring the pollen dispersal tendencies of maize in countries with smallholder farming systems.

In this paper, we developed and integrated a realtime emotion recognition system using an AI system-on-chip design. The emotion recognition platform combined three different physiological signals, Electroencephalogram (EEG), electrocardiogram (ECG), and photoplethysmogram (PPG) as the classification resources. A 3-to-1 Bluetooth piconet was deployed to transmit all physiological signals on a single platform access point and to make use of low power wireless technologies. The system then integrated an AI computing chip with a convolution neural network (CNN) structure to classify three emotions, happiness, anger, and sadness. The average accuracy for a subject-independent classification reached 72.66%. The proposed system was integrated with the RISC-V processor and AI SOC to implement real-time monitoring and classification on edge.

It is a challenge to predict the long-term future data from time series data. This paper proposes to use a Transformer with soft dynamic time wrapping for early stopping criteria, called a soft-DTW Transformer. Our experiment in an open-source dataset HouseTwenty shows that the average prediction error rate with soft-DTW Transformer is 27.79%, greatly reduced from 45.70% for using SVR, a common time series method.

The conventional PCB (Printed Circuit Board) DIP (Dual Inline Package) process solder defect detection was done by labor inspection, which is not only time-intensive but also labor-intensive. This paper proposes a deep ensemble method to inspect the PCB solder defects to replace the labor inspection. To achieve a high detection rate and a low false alarm rate, two distinct detection models, a hybrid YOLOv2 (YOLOv2 as a foreground detector and ResNet-101 as a classifier) and Faster RCNN with ResNet-101 and FPN are separately trained to obtain a high detection rate result. The final ensemble model aggregates the result from the two detection models. That achieves a 96.73% detection rate and a 19.73% false alarm rate in real tests. The detection time is less than 15 seconds for inferencing a PCB image with a resolution of 7296*6000. The proposed method has been proven efficient in terms of guiding operators to identify and fix PCB solder defects [1] and thus is able to reduce 33% of labor demand for each PCB production line at our real test site. [1].

Many of the strongest game playing programs use a combination of Monte Carlo tree search (MCTS) and deep neural networks (DNN), where the DNNs are used as policy or value evaluators. Given a limited budget, such as online playing or during the self-play phase of AlphaZero (AZ) training, a balance needs to be reached between accurate state estimation and more MCTS simulations, both of which are critical for a strong game playing agent. Typically, larger DNNs are better at generalization and accurate evaluation, while smaller DNNs are less costly, and therefore can lead to more MCTS simulations and bigger search trees with the same budget. This paper introduces a new method called the multiple policy value MCTS (MPV-MCTS), which combines multiple policy value neural networks (PV-NNs) of various sizes to retain advantages of each network, where two PV-NNs f_S and f_L are used in this paper. We show through experiments on the game NoGo that a combined f_S and f_L MPV-MCTS outperforms single PV-NN with policy value MCTS, called PV-MCTS. Additionally, MPV-MCTS also outperforms PV-MCTS for AZ training.