LighterBEV: LiDAR Global Localization Meets Online Learning

The concept of Industry 4.0 uses cyber-physical systems and the Internet of Things to create "smart factories" that enable automated and connected production. However, the complex and diverse nature of cyber-physical systems makes them vulnerable to physical threats, where any fault in their physical components can lead to serious consequences, from financial losses to human safety. To reduce the risk of such failures, it is essential to have techniques in place to monitor the condition of these physical components closely. In recent years, data-driven predictive maintenance methods have become common due to the growth in data and advancements in machine learning. This thesis focuses on fault diagnosis, a key part of predictive maintenance, which aims to detect and identify faults in physical components. Incorporating advanced techniques, including machine learning algorithms and sensor data analysis, enhanced fault diagnosis systems' accuracy and efficiency. However, several real-world barriers challenge the effectiveness of these techniques. This research is structured around three primary objectives. The first objective aims to establish an efficient fault diagnosis system that is highly sensitive to fault occurrences while being resilient to uncertainties such as variability in fault patterns and data imbalance. To address variability in fault patterns, we developed a multi-step approach integrating clustering techniques and an adaptive neuro-fuzzy inference system, which significantly improved diagnosis robustness even under noisy conditions. For imbalanced datasets, a two-step method was introduced, combining permutation entropy for fault detection and envelope analysis for fault isolation. This approach was enhanced by an automatic frequency band selection method leveraging wavelet transform and statistical criteria, resulting in high fault diagnosis accuracy and reliability, even with limited labeled faulty data. The second objective focuses on developing fault diagnosis systems that can maintain high performance despite domain shifts and limited training data in both source and target domains. To address the challenge of domain shifts, a transfer learning technique was introduced, and a systematic literature review was conducted. This study bridges a technical gap by providing a comprehensive taxonomy, state-of-the-art methods, and insights into the challenges of applying transfer learning to predictive maintenance. Additionally, a conceptual framework was proposed to explore scenarios for transferring diagnostic knowledge from different source domains to target domains with insufficient training datasets. In a subsequent study, a new diagnostic framework was developed using digital twins and partial transfer learning to address the lack of real training datasets in both source and target domains as well as label space inconsistencies between them. The framework includes a general approach for developing digital twins of real machines to generate synthetic diagnostic data as source domain data. Furthermore, by incorporating a deep learning network and a double-layer attention mechanism, a partial transfer learning diagnostic model was developed to transfer diagnostic knowledge from the digital twin to the real target domain despite label space inconsistencies. The proposed framework was validated through an industrial case study involving rolling bearings in rotating equipment. The third objective focuses on developing an efficient fault diagnosis system capable of maintaining reliability under unseen and evolving working conditions. To achieve this, a novel self-adaptive fault diagnosis system was proposed, incorporating the MAPE-K loop, a domain generalization network model, and digital twin technologies. This self-adaptive fault diagnosis system leverages the MAPE-K loop to adapt in real time to evolving working conditions. Digital twins were employed to continuously generate data aligned with the real system’s operational context, enabling accurate fault diagnosis in dynamic environments. The domain generalization network model enhances generalization to unseen target working conditions by addressing overfitting and feature loss through multi-domain data augmentation, adversarial learning, and domain discrepancy modules. Validation using multiple rotating machinery datasets demonstrated that the developed fault diagnosis system outperformed state-of-the-art methods across various scenarios. In summary, this dissertation proposes a comprehensive and practical approach to addressing variability in fault patterns, imbalance datasets, data scarcity, domain shifts, and unseen and evolving working conditions challenges in existing fault diagnosis systems.

Global localization in a prior map is an important field in virtual and augmented reality systems, but it is always a challenge to conduct point cloud based localization in the large-scale scene prior map. A large-scale prior map usually means huge amount of calculation for point cloud processing, which leads to the long time required for global localization. To deal with this problem, we propose a fast point cloud global localization method based on point clouds segmentation and learning-based descriptor. On the one hand, cylindrical filtering, ground-point removal and point cloud segmentation are adopted to eliminate a large number of useless points and retain points with rich structures, which improves the efficiency of point cloud registration. On the other hand, reliable 3D point cloud descriptor, two-phase search strategy for place recognition and geometric consistency verification are used to ensure the localization accuracy. Experiments prove that the proposed method achieves good localization effect on both KITTI and MVSEC datasets. Under the condition of ensuring the high localization accuracy, the time for point clouds to complete the global localization is greatly reduced.

Purpose – One of the biggest problems in the e-learning distribution process is the lack of quality content and learners' discredit in e-learning content. In order to respond to the various demands of the corporate education field appropriately, it is necessary to search for directions of new e-learning models that are out of traditional e-learning contents. The purpose of this study is to identify recent trend issues related to corporate e-learning and to suggest directions for development. Research design, data, and methodology – Based on the literature review, trend issues that should be considered important in corporate e-learning were derived. Online survey was conducted to evaluate the importance-feasibility of each issue to 13 experts on e-learning and corporate education. The contents of the questionnaire are as follows: 1) recognition of importance and feasibility of trend issues to be considered important in the future corporate education field; 2) factors to be considered in developing future e-learning contents. Results – Six trends derived from a comprehensive literature review. The most important e-learning trends for corporate education field were 'mobile learning', 'micro learning', 'blended learning', 'social learning', 'adaptive learning', ‘engaged learning'. As a result of evaluating the importance and feasibility of each issue, experts point out that 'mobile learning' and 'micro learning' should be actively considered for introduction and utilization at present. In addition, 'social learning' and 'blended learning' need to be actively considered in the near future. On the other hand, experts recognized that 'adaptive learning' and 'engaged learning' need to be prepared from a long-term perspective. Conclusions – There are two main reasons for this result. First, in corporate e-learning, it is important to 1) be able to update on time, 2) the connection with the workplace is important. Second, it requires realistic verification of the expected performance of the learning model. To be considered part of the future are as follows: First, the value and effectiveness of the new e-learning type should be studied. Seconds, e-learning contents should be developed through adopting SAM or Agile methodology. Through this process, we would be able to enhance the quality in e-learning content.

Abstract. The article discusses the current state and main directions of development of human re-identification (re-ID) technologies in intelligent video surveillance systems. The formal statement of the re-ID problem is defined and its role in ensuring continuous automatic monitoring of objects between non-overlapping cameras is characterized. The main challenges that complicate the solution of this problem in real conditions are analyzed, including variability of appearance, changes in angle and lighting, partial overlap, domain shift, limited training data and hardware resources. The paper summarizes the approaches to feature extraction, image matching, and deep learning methods used to build effective re-ID models. Particular attention is paid to current trends in the use of generative adversarial networks (GANs) and attention mechanisms that improve identification accuracy. Promising areas for further research, in particular in the field of multimodal learning, adaptation to new domains, and ethical implementation of technologies in public environments, are presented. Keywords: person re-identification, intelligent video systems, computer vision, deep learning, feature space, metric learning, generative adversarial networks, attention mechanism.

The Segment Anything Model (SAM) is widely used for segmenting a diverse range of objects in natural images from simple user prompts like points or bounding boxes. However, SAM's performance decreases substantially when applied to non-natural domains like microscopic imaging. Furthermore, due to SAM's interactive design, it requires a precise prompt for each image and object, which is un-feasible in many automated biomedical applications. Previous solutions adapt SAM by training millions of parameters via fine-tuning large parts of the model or of adapter layers. In contrast, we show that as little as 2,048 additional parameters are sufficient for turning SAM into a usecase specialist for a certain downstream task. Our novel PTSAM (prompt-tuned SAM) method uses prompt-tuning, a parameter-efficient fine-tuning technique, to adapt SAM for a specific task. We validate the performance of our approach on multiple microscopic and one medical dataset. Our results show that prompt-tuning only SAM's mask decoder already leads to a performance on-par with state-of-the-art techniques while requiring roughly 2,000× less trainable parameters. For addressing domain gaps, we find that additionally prompt-tuning SAM's image encoder is beneficial, further improving segmentation accuracy by up to 18% over state-of-the-art results. Since PTSAM can be reliably trained with as little as 16 annotated images, we find it particularly helpful for applications with limited training data and domain shifts.

Currently, the research on recommend system has drawn intensive attention from many researchers. However, it is still a concerned focus to make accurate recommendations in a short time period. Recently, some researchers have proposed a formal model based on learning to rank for Top-N recommendation. Aiming at improving the effect of Top-N recommendation and solving the problem of timeliness, we analyze and summarize out some factors that affect the timeliness of recommendation, and propose a ranking model of recent feedback which combined with online learning. Experimental results show that after the second ranking of users' recent feedback and basic recommendation output, the recommendation accuracy is improved effectively. Furthermore, we also explores the application of online learning algorithm based on the ranking model of recent feedback. The simulation experiments show that the model can still perform an accurate recommendation after being updated online in real time.

The landscape of higher education is continuously evolving, with online learning emerging as a prominent educational modality. This cross-sectional study, conducted at the Holy Spirit University of Kaslik (USEK) in Lebanon from January to October 2023, aimed to assess Lebanese students’ perceptions regarding the effectiveness of online learning as a mode of education and to develop strategies and recommendations for enhancing online learning in Lebanon. Using a convenience-sampling method, we collected 309 responses, focusing on educators’ preparedness, student satisfaction, adaptability in online learning, and preferences for future learning modalities. The results highlighted a pressing demand for innovative course design strategies that promote engagement and interactivity, with 41.4% of respondents expressing this need. Additionally, 34.6% of learners emphasized the need for accessible technical support. Furthermore, 38.8% of participants underscored the necessity of comprehensive training programs for educators and learners in navigating online learning environments effectively. Moreover, the preference for online or hybrid learning models (55%) emphasized the importance of flexibility while preserving the value of in-person interactions. The study also highlighted the potential of educational advances to further enhance online learning environments. These technologies could provide opportunities for personalized learning experiences, tailoring course content, delivery methods, and assessments to meet the diverse needs of students, thus fostering a more adaptable and inclusive learning ecosystem. These insights are crucial for refining online education strategies and addressing the evolving needs of students in Lebanon.

Many deep learning (DL)-based detectors have been developed for optical remote sensing object detection in recent years. However, most of the recent detectors are developed toward the pursuit of a higher accuracy, but little toward a balance between accuracy, deployability and inference time, which hinders the practical application for these detectors, especially in embedded devices. In order to achieve a higher detection accuracy and reduce the computational consumption and inference time simultaneously, a novel convolutional network named YOLO-DSD was developed based on YOLOv4. Firstly, a new feature extraction module, a dense residual (DenseRes) block, was proposed in a backbone network by utilizing a series-connected residual structure with the same topology for improving feature extraction while reducing the computational consumption and inference time. Secondly, convolution layer–batch normalization layer–leaky ReLu (CBL) ×5 modules in the neck, named S-CBL×5, were improved with a short-cut connection in order to mitigate feature loss. Finally, a low-cost novel attention mechanism called a dual channel attention (DCA) block was introduced to each S-CBL×5 for a better representation of features. The experimental results in the DIOR dataset indicate that YOLO-DSD outperforms YOLOv4 by increasing mAP0.5 from 71.3% to 73.0%, with a 23.9% and 29.7% reduction in Params and Flops, respectively, but a 50.2% improvement in FPS. In the RSOD dataset, the mAP0.5 of YOLO-DSD is increased from 90.0~94.0% to 92.6~95.5% under different input sizes. Compared with the SOTA detectors, YOLO-DSD achieves a better balance between the accuracy, deployability and inference time.

A key challenge with supervised learning (e.g., image classification) is the shift of data distribution and domain from training to testing datasets, so-called “domain shift” (or “distribution shift”), which usually leads to a reduction of model accuracy. Various meta-learning approaches have been proposed to prevent the accuracy loss by learning an adaptable model with training data, and adapting it to test time data from a new data domain. However, when the domain shift occurs in multiple domain dimensions (e.g., images may be transformed by rotations, transitions, and expansions), the average predictive power of the adapted model will deteriorate. To tackle this problem, we propose a domain disentangled meta-learning (DDML) framework. DDML disentangles the data domain by dimensions, learns the representations of domain dimensions independently, and adapts to the domain of test time data. We evaluate our DDML on image classification problems using three datasets with distribution shifts over multiple domain dimensions. Comparing to various baselines in meta-learning and empirical risk minimization, our DDML approach achieves consistently higher classification accuracy with the test time data. These results demonstrate that domain disentanglement reduces the complexity of the model adaptation, thus increases the model generalizability, and prevents it from overfitting.

Unsupervised domain adaptation (UDA) techniques, extensively studied in hyperspectral image (HSI) classification, aim to use labeled source domain data and unlabeled target domain data to learn domain invariant features for cross-scene classification. Compared to natural images, numerous spectral bands of HSIs provide abundant semantic information, but they also increase the domain shift significantly. In most existing methods, both explicit alignment and implicit alignment simply align feature distribution, ignoring domain information in the spectrum. We noted that when the spectral channel between source and target domains is distinguished obviously, the transfer performance of these methods tends to deteriorate. Additionally, their performance fluctuates greatly owing to the varying domain shifts across various datasets. To address these problems, a novel shift-sensitive spatial-spectral disentangling learning (S4DL) approach is proposed. In S4DL, gradient-guided spatial-spectral decomposition (GSSD) is designed to separate domain-specific and domain-invariant representations by generating tailored masks under the guidance of the gradient from domain classification. A shift-sensitive adaptive monitor is defined to adjust the intensity of disentangling according to the magnitude of domain shift. Furthermore, a reversible neural network is constructed to retain domain information that lies not only in semantic but also the shallow-level detailed information. Extensive experimental results on several cross-scene HSI datasets consistently verified that S4DL is better than the state-of-the-art UDA methods. Our source code will be available at https://github.com/xdu-jjgs/IEEE_TNNLS_S4DL.

The classification of remote sensing images is inherently challenging due to the complexity, diversity, and sparsity of the data across different image samples. Existing advanced methods often require substantial modifications to model architectures to achieve optimal performance, resulting in complex frameworks that are difficult to adapt. To overcome these limitations, we propose a lightweight ensemble method, enhanced by pure data correction, called the Exceptionally Straightforward Ensemble. This approach eliminates the need for extensive structural modifications to models. A key innovation in our method is the introduction of a novel strategy, quantitative augmentation, implemented through a plug-and-play module. This strategy effectively corrects feature distributions across remote sensing data, significantly improving the performance of Convolutional Neural Networks and Vision Transformers beyond traditional data augmentation techniques. Furthermore, we propose a straightforward algorithm to generate an ensemble network composed of two components, serving as the proposed lightweight classifier. We evaluate our method on three well-known datasets, with results demonstrating that our ensemble models outperform 48 state-of-the-art methods published since 2020, excelling in accuracy, inference speed, and model compactness. Specifically, our models achieve an overall accuracy of up to 96.8%, representing a 1.1% improvement on the challenging NWPU45 dataset. Moreover, the smallest model in our ensemble reduces parameters by up to 90% and inference time by 74%. Notably, our approach significantly enhances the performance of Convolutional Neural Networks and Vision Transformers, even with limited training data, thus alleviating the performance dependence on large-scale datasets. In summary, our data-driven approach offers an efficient, accessible solution for remote sensing image classification, providing an elegant alternative for researchers in geoscience fields who may have limited time or resources for model optimization.

Filter pruning is a technique that reduces computational complexity, inference time, and memory footprint by removing unnecessary filters in convolutional neural networks (CNNs) with an acceptable drop in accuracy, consequently accelerating the network. Unlike traditional filter pruning methods utilizing zeroing-out filters, we propose two techniques to achieve the effect of pruning more filters with less performance degradation, inspired by the existing research on centripetal stochastic gradient descent (C-SGD), wherein the filters are removed only when the ones that need to be pruned have the same value. First, to minimize the negative effect of centripetal vectors that gradually make filters come closer to each other, we redesign the vectors by considering the effect of each vector on the loss-function using the Taylor-based method. Second, we propose an adaptive gradient learning (AGL) technique that updates weights while adaptively changing the gradients. Through AGL, performance degradation can be mitigated because some gradients maintain their original direction, and AGL also minimizes the accuracy loss by perfectly converging the filters, which require pruning, to a single point. Finally, we demonstrate the superiority of the proposed method on various datasets and networks. In particular, on the ILSVRC-2012 dataset, our method removed 52.09% FLOPs with a negligible 0.15% top-1 accuracy drop on ResNet-50. As a result, we achieve the most outstanding performance compared to those reported in previous studies in terms of the trade-off between accuracy and computational complexity.

Semi-supervised video object segmentation is to segment the target objects given the ground truth annotation of the first frame. Previous successful methods mostly rely on online learning or static image pre-train to improve accuracy. However, online learning methods require huge time costs at inference time, thus restrict their practical use. Methods with static image pre-train require heavy data augmentation that is complicated and time-consuming. This paper presents a fast Hierarchical Embedding Guided Network (HEGNet) which is only trained on Video Object Segmentation (VOS) datasets and does not utilize online learning. Our HEGNet integrates propagation-based and matching-based methods. It propagates the predicted mask of the previous frame as a soft cue and extracts hierarchical embedding at both deep and shallow layers to do feature matching. The produced label map of the deep layer is also used to guide the matching of the shallow layer. We evaluated our method on the DAVIS-2016 and DAVIS-2017 validation sets and achieved overall scores of 84.9% and 71.9% respectively. Our method surpasses the methods without online learning and static image pre-train and runs at 0.08 seconds per frame.

This chapter describes a pilot study aiming at testing the integration of online and active learning features into the computer-assisted translation workbench developed within the CASMACAT project. These features can be used to take advantage of the new knowledge implicitly provided by human experts when they generate new translations. Online learning (OL) allows the system to learn from user feedback in real time by incrementally adapting the parameters of the statistical models involved in the translation process. On the other hand, active learning (AL) determines those sentences that need to be supervised by the user so as to maximize the final translation quality minimizing user effort and, at the same time, improving the statistical model parameters. We investigate the effect of these features on translation productivity, using interactive translation prediction (ITP) as a baseline. ITP is a computer assisted translation approach where the user interactively collaborates with a statistical machine translation system to generate high quality translations. User activity data was collected from ten translators using key-logging and eye-tracking. We found that ITP with OL performs better than standard ITP, especially in terms of typing effort required from the user to generate correct translations. Additionally, ITP with AL provides better translation quality than standard ITP for the same levels of user effort.

Medical image segmentation tasks hitherto have achieved excellent progresses with large-scale datasets, which empowers us to train potent deep convolutional neural networks (DCNNs). However, labeling such large-scale datasets is laborious and error-prone, which leads the noisy (or incorrect) labels to be an ubiquitous problem in the real-world scenarios. In addition, data collected from different sites usually exhibit significant data distribution shift (or domain shift). As a result, noisy label and domain shift become two common problems in medical imaging application scenarios, especially in medical image segmentation, which degrade the performance of deep learning models significantly. In this paper, we identify a novel problem hidden in medical image segmentation, which is unsupervised domain adaptation on noisy labeled data, and propose a novel algorithm named "Self-Cleansing Unsupervised Domain Adaptation" (S-CDUA) to address such issue. S-CUDA sets up a realistic scenario to solve the above problems simultaneously where training data (i.e., source domain) not only shows domain shift w.r.t. unsupervised test data (i.e., target domain) but also contains noisy labels. The key idea of S-CUDA is to learn noise-excluding and domain invariant knowledge from noisy supervised data, which will be applied on the highly corrupted data for label cleansing and further data-recycling, as well as on the test data with domain shift for supervised propagation. To this end, we propose a novel framework leveraging noisy-label learning and domain adaptation techniques to cleanse the noisy labels and learn from trustable clean samples, thus enabling robust adaptation and prediction on the target domain. Specifically, we train two peer adversarial networks to identify high-confidence clean data and exchange them in companions to eliminate the error accumulation problem and narrow the domain gap simultaneously. In the meantime, the high-confidence noisy data are detected and cleansed in order to reuse the contaminated training data. Therefore, our proposed method can not only cleanse the noisy labels in the training set but also take full advantage of the existing noisy data to update the parameters of the network. For evaluation, we conduct experiments on two popular datasets (REFUGE and Drishti-GS) for optic disc (OD) and optic cup (OC) segmentation, and on another public multi-vendor dataset for spinal cord gray matter (SCGM) segmentation. Experimental results show that our proposed method can cleanse noisy labels efficiently and obtain a model with better generalization performance at the same time, which outperforms previous state-of-the-art methods by large margin. Our code can be found at https://github.com/zzdxjtu/S-cuda.