Efficient Image Restoration for Autonomous Vehicles and Traffic Systems: A Knowledge Distillation Approach to Enhancing Environmental Perception

Pedestrian detection is another significant special application of object detection in autonomous vehicles. In contrast to universal object detection, it has similarities and special traits. Nevertheless, there are some difficulties that influence pedestrian detection performance, namely (i) occlusion and deformation (ii) low-quality and multispectral images consisting primarily of lighting conditions, small-scale detection, and target detection extensively, and (iii) true-false pedestrians. Deep learning (DL) methods are a class of artificial intelligence method that can solve the issues mentioned above of pedestrian detection. This paper initially gives an elaborate description of pedestrian detection, difficulties in pedestrian detection, and latest advancements in solving them using the assistance of DL methods with informative discussions, aiming to provide insights to the readers. (2) A new pedestrian detection algorithm (PDA) of true/false pedestrian is suggested here, in which a new YOLO-3D CNN model is applied to reject true/false pedestrian. The primary purpose is to evaluate the performance of the existing 3D CNN taking into consideration the problem of rejecting true false pedestrians based on images captured using the car's onboard cameras and light detection and ranging (LiDAR) sensors. PDA initially utilizes YOLOv3 to capture the entire image for training detector model capable of real-time forecasting. Next, as a feature extractor, it utilizes the MobileNet-SSD that provides great accuracy as well as good trading uptime. PDA then implements the Faster R-CNN method to detect different parts of the object, over the convolutional layer. Lastly, data augmentation techniques are applied in PDA to augment the data coverage by fully exploiting available training data. Simulation results indicate that the proposed pedestrian detection model and PDA improve the accuracy of real and false pedestrians while maintaining real-time requirements.

Deploying high-performance image restoration models on drones is critical for applications like autonomous navigation, surveillance, and environmental monitoring. However, the computational and memory limitations of drones pose significant challenges to utilizing complex image restoration models in real-world scenarios. To address this issue, we propose the Simultaneous Learning Knowledge Distillation (SLKD) framework, specifically designed to compress image restoration models for resource-constrained drones. SLKD introduces a dual-teacher, single-student architecture that integrates two complementary learning strategies: Degradation Removal Learning (DRL) and Image Reconstruction Learning (IRL). In DRL, the student encoder learns to eliminate degradation factors by mimicking Teacher A, which processes degraded images utilizing a BRISQUE-based extractor to capture degradation-sensitive natural scene statistics. Concurrently, in IRL, the student decoder reconstructs clean images by learning from Teacher B, which processes clean images, guided by a PIQE-based extractor that emphasizes the preservation of edge and texture features essential for high-quality reconstruction. This dual-teacher approach enables the student model to learn from both degraded and clean images simultaneously, achieving robust image restoration while significantly reducing computational complexity. Experimental evaluations across five benchmark datasets and three restoration tasks—deraining, deblurring, and dehazing—demonstrate that, compared to the teacher models, the SLKD student models achieve an average reduction of 85.4% in FLOPs and 85.8% in model parameters, with only a slight average decrease of 2.6% in PSNR and 0.9% in SSIM. These results highlight the practicality of integrating SLKD-compressed models into autonomous systems, offering efficient and real-time image restoration for aerial platforms operating in challenging environments.

In recent years, autonomous vehicles have become more and more popular due to their broad influence over society, as they increase passenger safety and convenience, lower fuel consumption, reduce traffic blockage and accidents, save costs, and enhance reliability. However, autonomous vehicles suffer from some functionality errors which need to be minimized before they are completely deployed onto main roads. Pedestrian detection is one of the most considerable tasks (functionality errors) in autonomous vehicles to prevent accidents. However, accurate pedestrian detection is a very challenging task due to the following issues: (i) occlusion and deformation and (ii) low-quality and multi-spectral images. Recently, deep learning (DL) technologies have exhibited great potential for addressing the aforementioned pedestrian detection issues in autonomous vehicles. This survey paper provides an overview of pedestrian detection issues and the recent advances made in addressing them with the help of DL techniques. Informative discussions and future research works are also presented, with the aim of offering insights to the readers and motivating new research directions.

Focusing on features with higher importance degree: A significance-wise mechanism for image restoration

AMAKD: Adversarial multi-modal attention-based knowledge distillation for robust behaviour anomaly detection in real-world environments

Image restoration (IR) methods based on neural network algorithms have shown great success. However, the hardware circuits that can perform real-time IR task with high-effective analog computation are few in the literature. To address such problem, we propose a memristor-based continuous Hopfield neural network (HNN) circuit for processing the IR task in this work. In our circuit, a single memristor crossbar array is used to represent synaptic weights and perform matrix operations. Current feedback operation amplifiers are utilized to achieve integral operation and output function. Given these designs, the proposed circuit can perform continuous recursive operations in parallel and process different optimization problems with the programmability of the memristor array. On the basis of the proposed circuit, binary and greyscale image restorations are conducted through self-organizing network operations, providing a hardware implementation platform for IR tasks. Comparative simulations show the designed HNN circuit provides effective improvements in terms of speed and accuracy compared with software simulation. Moreover, the hardware circuit shows good robustness to memristive variation and input noise.

As a long-standing task, image restoration aims to recover the latent sharp image from its degraded counterpart. In recent years, owing to the strong ability of self-attention in capturing long-range dependencies, Transformer based methods have achieved promising performance on multifarious image restoration tasks. However, the canonical self-attention leads to quadratic complexity with respect to input size, hindering its further applications in image restoration. In this paper, we propose a Strip Attention Network (SANet) for image restoration to integrate information in a more efficient and effective manner. Specifically, a strip attention unit is proposed to harvest the contextual information for each pixel from its adjacent pixels in the same row or column. By employing this operation in different directions, each location can perceive information from an expanded region. Furthermore, we apply various receptive fields in different feature groups to enhance representation learning. Incorporating these designs into a U-shaped backbone, our SANet performs favorably against state-of-the-art algorithms on several image restoration tasks. The code is available at https://github.com/c-yn/SANet.

Local to non-local: Multi-scale progressive attention network for image restoration

The application of multimodal image fusion has become increasingly widespread across various fields in the era of deep learning. Existing fusion methods integrate infrared and visible images to provide complementary content and enhance the robustness of complex real-world scenes for high-level visual tasks, such as semantic segmentation and object detection. In return, high-level visual tasks facilitate the fusion of infrared and visible by providing mid-level semantic information. However, such frameworks rely heavily on multimodal data and require strict registration of images from different modalities before fusion, seriously limiting their practical applications due to the common realistic situations of missing modalities or misregistration. To move beyond this limitation, we propose a novel hierarchical knowledge distillation (HKD) framework tailored for unimodal image segmentation with the guidance of multi-modality. This framework aims to retain as much diverse information from multimodal image fusion as possible, thereby enhancing downstream high-level visual tasks when only the unimodal images are available during the inference phase. Our proposed method is two-stage, and we construct a robust multimodal fusion and segmentation interaction network in the first stage as a powerful teacher model. In the second stage, we design a hierarchical distillation method to transfer the fused and segmented multi-layer knowledge from the multimodal teacher model to the unimodal student model. Extensive experimental results on two public datasets, i.e., MFNet and FMB, demonstrate that the proposed hierarchical knowledge distillation framework can effectively transfuse multimodal knowledge into the unimodal student model for image enhancement and segmentation under incomplete multimodal conditions, and achieves considerably competitive results compared to multimodal image fusion and segmentation models.

Recent works that utilized deep models have achieved superior results in various image restoration (IR) applications. Such approach is typically supervised, which requires a corpus of training images with distributions similar to the images to be recovered. On the other hand, the shallow methods, which are usually unsupervised remain promising performance in many inverse problems, e.g., image deblurring and image compressive sensing (CS), as they can effectively leverage nonlocal self-similarity priors of natural images. However, most of such methods are patch-based leading to the restored images with various artifacts due to naive patch aggregation in addition to the slow speed. Using either approach alone usually limits performance and generalizability in IR tasks. In this paper, we propose a joint low-rank and deep (LRD) image model, which contains a pair of triply complementary priors, namely, internal and external, shallow and deep, and non-local and local priors. We then propose a novel hybrid plug-and-play (H-PnP) framework based on the LRD model for IR. Following this, a simple yet effective algorithm is developed to solve the proposed H-PnP based IR problems. Extensive experimental results on several representative IR tasks, including image deblurring, image CS and image deblocking, demonstrate that the proposed H-PnP algorithm achieves favorable performance compared to many popular or state-of-the-art IR methods in terms of both objective and visual perception.

Fast and efficient object detection such as pedestrian detection and vehicle detection is an important task for autonomous driving. In camera-based sensing systems, object detection offers the fundamental ability for autonomous vehicles to perceive the dynamic environment in a real-time fashion. In this talk, I will be providing an overview of object detection based on computer vision and its applications for autonomous driving and the primary challenges as well as technical improvements in recent years. A number of topics will be covered in this talk, including the milestone detection approaches in history, detection datasets, performance metrics, speed-up techniques, and the recent state-of-the-art detection methods.

Advances in deep learning models have significantly improved performance in medical tasks, but their complex structures and high computational requirements pose challenges for clinical implementation. Additionally, data privacy concerns limit the availability of comprehensive datasets needed to train accurate models. To address these issues, we propose a novel adaptive knowledge distillation (KD) framework for medical imaging segmentation that integrates intermediate and high-level feature pairwise relationships between teacher and student models. Our framework features adaptive multi-teacher distillation, where multiple teacher models, each trained on limited data from different sites and hospitals with various scanning protocols, distill their knowledge to a student model using adaptive weighting. This method allows each teacher to convey deep feature representations to the student’s intermediate layers, enhancing performance without increasing complexity. To validate the efficacy of our framework, we conducted extensive experiments on two publicly available medical datasets, focusing on prostate and spleen tumor segmentation tasks. Our adaptive KD approach significantly improved dice scores by up to 9%, surpassing all tested baseline models. These results highlight the potential of our KD framework to enhance medical imaging segmentation while ensuring data privacy and security. • Implement both single and multi-teacher distillation scenarios. • Apply adaptive multi-teacher distillation using limited data. • Distill intermediate and high-level features. • Conduct CT and MRI medical imaging segmentation experiments using public datasets.

Recent advances in deep learning bring impressive performance for multimedia applications. Hence, compressing and deploying these applications on resource-limited edge devices via model compression becomes attractive. Knowledge distillation (KD) is one of the most popular model compression techniques. However, most well-behaved KD approaches require the original dataset, which is usually unavailable due to privacy issues, while existing data-free KD methods perform much worse than data-required counterparts. In this paper, we analyze previous data-free KD methods from the data perspective and point out that using a single pre-trained model limits the performance of these approaches. We then propose a Data-Free Ensemble knowledge Distillation (DFED) framework, which contains a student network, a generator network, and multiple pre-trained teacher networks. During training, the student mimics behaviors of the ensemble of teachers using samples synthesized by a generator, which aims to enlarge the prediction discrepancy between the student and teachers. A moment matching loss term assists the generator training by minimizing the distance between activations of synthesized samples and real samples. We evaluate DFED on three popular image classification datasets. Results demonstrate that our method achieves significant performance improvements compared with previous works. We also design an ablation study to verify the effectiveness of each component of the proposed framework.

At the current stage, the rapid Development of autonomous driving has made object detection in traffic scenarios a vital research task. Object detection is the most critical and challenging task in computer vision. Deep learning, with its powerful feature extraction capabilities, has found widespread applications in safety, military, and medical fields, and in recent years has expanded into the field of transportation, achieving significant breakthroughs. This survey is based on the theory of deep learning. It systematically summarizes the Development and current research status of object detection algorithms, and compare the characteristics, advantages and disadvantages of the two types of algorithms. With a focus on traffic signs, vehicle detection, and pedestrian detection, it summarizes the applications and research status of object detection in traffic scenarios, highlighting the strengths, limitations, and applicable scenarios of various methods. It introduces techniques for optimizing object detection algorithms, summarizes commonly used object detection datasets and traffic scene datasets, along with evaluation criteria, and performs comparative analysis of the performance of deep learning algorithms. Finally, it concludes the development trends of object detection algorithms in traffic scenarios, providing research directions for intelligent transportation and autonomous driving.

In this paper, we propose the dense disparity map-based pedestrian detection method for intelligent vehicle. The dense disparity map is utilized to improve the pedestrian detection performance. Our method consists of several steps namely, obstacle area detection using road feature information and column detection, pedestrian area detection using dense disparity map-based segmentation, and pedestrian detection using optimal feature. The first step is to detect all obstacle areas using column detection and pedestrian height information. However, there are many objects in single obstacle area. Thus each obstacle area needs to be separated into single object for improving pedestrian detection performance. Thus, the second step is performed to segment the detected obstacle area. The last step is to detect only pedestrian using classifier trained by optimal feature. The optimal feature is extracted by positive and negative training images. ETH database is utilized to evaluate our proposed pedestrian detection method.