CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning

To further improve the accuracy of recurrent neural network in predicting the gas concentration in the upper corner of the mine tunnel, this paper proposes a method to construct a gas concentration prediction model based on multiple sequence long and short memory network, considering the spatial correlation between the gas concentration in the return airway and upper corner. The reliability of the model construction is improved by using the white noise test and smoothness test to verify the interpretability of the data in this paper and constructing supervised learning type data for gas concentration prediction model training and testing by means of data set division and data windowing. Through experimental comparison, grid search, and time series decomposition, the model algorithm, training parameters, and experimental results were combined to make an in-depth analysis of the influence of each parameter on the model training and the prediction. A training model of the spatially fused gas concentration prediction model with a network layer of 1 and a number of neurons of 32 as the model structure, Adam as the optimization algorithm, and a learning rate of 0.001 and a batch size of 32 as the training parameters was finally determined. The gas concentration prediction model trained in this paper performed well in the test set with a mean square error (MSE) of 0.0013, and its superiority was verified by comparing it with other models to provide some experience and basis for subsequent studies on gas concentration prediction in the upper corner.

To translate artificial intelligence (AI) algorithms into clinical practice requires generalizability of models to real-world data. One of the main obstacles to generalizability is data shift, a data distribution mismatch between model training and real environments. Explainable AI techniques offer tools to detect and mitigate the data shift problem and develop reliable AI for clinical practice. Most medical AI is trained with datasets gathered from limited environments, such as restricted disease populations and center-dependent acquisition conditions. The data shift that commonly exists in the limited training set often causes a significant performance decrease in the deployment environment. To develop a medical application, it is important to detect potential data shift and its impact on clinical translation. During AI training stages, from premodel analysis to in-model and post hoc explanations, explainability can play a key role in detecting model susceptibility to data shift, which is otherwise hidden because the test data have the same biased distribution as the training data. Performance-based model assessments cannot effectively distinguish the model overfitting to training data bias without enriched test sets from external environments. In the absence of such external data, explainability techniques can aid in translating AI to clinical practice as a tool to detect and mitigate potential failures due to data shift. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

Traditional machine learning methods rely on the training data and target data having the same feature space and data distribution. The performance may be unacceptable if there is a difference in data distribution between the training and target data, which is called cross-domain learning problem. In recent years, many domain adaptation methods have been proposed to solve this kind of problems and make much progress. However, existing domain adaptation approaches have a common assumption that the number of the data in source domain (labeled data) and target domain (unlabeled data) is matched. In this paper, the scenarios in real manufacturing site are considered, that the target domain data is much less than source domain data at the beginning, but the number of target domain data will increase as time goes by. A novel method is proposed for fault diagnosis of rolling bearing with online imbalanced cross-domain data. Finally, the proposed method which is tested on bearing dataset (CWRU) has achieved prediction accuracy of 95.89% with only 40 target samples. The results have been compared with other traditional methods. The comparisons show that the proposed online domain adaptation fault diagnosis method has achieved significant improvements. In addition, the deep transfer learning model by adaptive- network-based fuzzy inference system (ANFIS) is introduced to interpretation the results.

Domain adaptation techniques such as importance weighting modify the training data to better represent a different test data distribution, a process that may be particularly vulnerable to a malicious attack in an adversarial machine learning scenario. In this work, we explore the level of such vulnerability of importance weighting to poisoning attacks. Importance weighting, like other domain adaptation approaches, assumes that the distributions of training and test data are different but related. An intelligent adversary, having full or partial access to the training data, can take advantage of the expected difference between the distributions, and can inject well crafted malicious samples into the training data, resulting in an incorrect estimation of the importance ratio. In this work, we demonstrate the vulnerability of one of the simplest yet most effective approaches for directly estimating the importance ratio, namely, modifying the training distribution using a discriminative classifier such as the logistic regression. We test the robustness of the importance weighting process using well-controlled synthetic datasets, with an increasing number of attack points in the training data. Under the worst case perfect knowledge scenario, where the attacker has full access to the training data, we demonstrate that importance weighting can be dramatically compromised with the insertion of even a single attack point. We then show that even under limited knowledge scenario, where the attacker has limited access to the training data, the estimation process can still be significantly compromised.

Over the years, multimodal mobile sensing has been used extensively for inferences regarding health and well-being, behavior, and context. However, a significant challenge hindering the widespread deployment of such models in real-world scenarios is the issue of distribution shift. This is the phenomenon where the distribution of data in the training set differs from the distribution of data in the real world---the deployment environment. While extensively explored in computer vision and natural language processing, and while prior research in mobile sensing briefly addresses this concern, current work primarily focuses on models dealing with a single modality of data, such as audio or accelerometer readings, and consequently, there is little research on unsupervised domain adaptation when dealing with multimodal sensor data. To address this gap, we did extensive experiments with domain adversarial neural networks (DANN) showing that they can effectively handle distribution shifts in multimodal sensor data. Moreover, we proposed a novel improvement over DANN, called M3BAT, unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training, to account for the multimodality of sensor data during domain adaptation with multiple branches. Through extensive experiments conducted on two multimodal mobile sensing datasets, three inference tasks, and 14 source-target domain pairs, including both regression and classification, we demonstrate that our approach performs effectively on unseen domains. Compared to directly deploying a model trained in the source domain to the target domain, the model shows performance increases up to 12% AUC (area under the receiver operating characteristics curves) on classification tasks, and up to 0.13 MAE (mean absolute error) on regression tasks.

Dimensionality reduction has emerged as one of the prominent fields of research since it provides a solution to a wide class of problems, such as compression, classification, regression, feature analysis, and visual recognition. Generally, subspace learning algorithms find a low-dimensional subspace from a given high-dimensional data, wherein samples from different classes can be well separated. In the past decades, several types of dimensionality reduction and subspace selection algorithms have been widely used for visual data analysis. Conventional algorithms such as principal component analysis (PCA) and linear discriminant analysis (LDA), perform well under the assumption that training and test data follow similar distributions. However, the main drawback still remains in the sense that the distribution of the training data and that of the test data are mismatched when samples are drawn from different but related sources. They also fail to incorporate temporal statistical variations in data distribution when applied to time-series data(videos). Consequently they result in unsatisfactory recognition performance when they deal with real world visual analysis problems. To overcome the above issues, in this research, our aim is to consider invariance and stationarity in subspace analysis pertaining to many computer vision applications such as video retrieval, human behaviour analysis, event analysis and activity recognition. More specifically, we will address the challenging and fundamental tasks of 1. Video Classification (Scene, dynamic texture and action recognition) 2. Visual Domain Adaptation. To address the first task, we propose a subspace learning approach that focuses on extracting the stationary parts of all videos in the same class. The notion of stationarity is intuitively well-adapted to model the temporal nature of the video signal and lets us make use of many image features. Instead of modeling temporal information in the features, our method explicitly accounts for it when learning the latent space. As a consequence, the resulting video representation is particularly well-suited for classification purpose. Our experimental evaluation shows that our approach outperforms baselines from the different groups of methods in several video classification tasks such as dynamic texture recognition, scene classification and action recognition.The second task addressed by this work is domain adaptation for visual recognition. In our proposed approach, we follow the intuitive idea of learning the domain invariant subspace by matching the distributions of the transformed training and test data using Maximum Mean Discrepancy (MMD). This, we believe, makes better use of the expressive power of the kernel in MMD compared to the other approaches like Transfer Component Analysis (TCA). Although motivated by MMD, in TCA, the distance between the sample means is measured in a lower-dimensional space rather than in Reproducing Kernel Hilbert Space (RKHS), which somewhat contradicts the intuition behind the use of kernels. Furthermore, we extend the framework for the semi-supervised scenario. Experiments on benchmark domain adaptation dataset for visual recognition, show that in comparison to well-known methods, the proposed approach obtains a significant improvement in classification accuracy. To address the second task, we propose another domain adaptation method that exploits the Riemannian structure of the statistical manifold when learning the invariant subspaces or samples. To this end, we introduce the use of the Hellinger distance which is related to the geodesic distance on the space of probability distributions. While the Hellinger distance has been employed for dimensionality reduction, to the best of our knowledge, our approach is the first attempt at exploiting the Riemannian geometry of the statistical manifold for domain adaptation. We showed that our sample-selection and subspace-based approaches in conjunction with Hellinger distance for distribution matching consistently outperform the similar approaches on the tasks of visual domain adaptation and WiFi localization.

Cohesive human–robot collaboration can be achieved through seamless communication between human and robot partners. We posit that the design aspects of human–robot communication (HRCom) can take inspiration from human communication to create more intuitive systems. A key component of HRCom systems is perception models developed using machine learning. Being data-driven, these models suffer from the dearth of comprehensive, labelled datasets while models trained on standard, publicly available datasets do not generalize well to application-specific scenarios. Complex interactions and real-world variability lead to shifts in data that require domain adaptation by the models. Existing domain adaptation techniques do not account for incommensurable modes of communication between humans and robot perception systems. Taking into account these challenges, a novel framework is presented that leverages existing domain adaptation techniques off-the-shelf and uses statistical measures to start and stop the training of models when they encounter domain-shifted data. Statistically informed multimodal (domain adaptation by transfer) learning (SIMLea) takes inspiration from human communication to use human feedback to auto-label for iterative domain adaptation. The framework can handle incommensurable multimodal inputs, is mode and model agnostic, and allows statistically informed extension of datasets, leading to more intuitive and naturalistic HRCom systems.

The machine learning method, now widely used for predicting well performance from unconventional reservoirs in the industry, generally needs large data sets for model development and training. The large data sets, however, are not always available, especially for newly developed unconventional plays. The objective of this work is to develop an innovative machine learning method for predicting well performance in unconventional reservoirs with a relatively small data set. For a small training data set, the corresponding machine learning model can significantly suffer from so-called overfitting meaning that the model can match the training data but has poor predictivity. To overcome this, our new method averages predictions from multiple models that are developed with the same model input, but different initial guesses of model parameters that are unknowns in a machine learning algorithm and determined in the model training. The averaged results are used for the final model prediction. Unlike traditional ensemble learning methods, each prediction in the new method uses all the input data rather than its subset. We mathematically prove that the averaged prediction provides less model uncertainty and under certain conditions the optimum prediction. It is also demonstrated that the method practically minimizes the overfitting and gives relatively unique prediction. The usefulness of the method is further confirmed by its successful application to the data set collected from less than 100 wells in an unconventional reservoir. Sensitivity results with the trained machine learning model show that the model results are consistent with the domain knowledge regarding the production from the reservoir.

Froth flotation is a popular ground ore separation method with a wide range of applications in the mining industry due to its versatility and simplicity. In this method, a separation with a certain grade and recovery is achieved thanks to the difference in the surface hydrophobicity of the different insoluble compounds that are present in an industrial ground ore. Under a process intensification scope, the study of froth flotation columns is of remarkable interest due to their higher gas phase surface-to-volume ratios and longer bubble residence times that result in higher grades and recoveries while reducing the space requirements in a plant layout and the associated capital and operating costs. However, despite the progress made in flotation physics throughout the years, its complex nature has forced many controloriented modelling efforts to rely on empirical or mixed approaches with limited generalizability for a broad range of operating conditions. In contrast, while the use of high spatial resolution methods has improved our comprehension of flotation physics at both microscopic and macroscopic levels, the computational cost constitutes a drawback when developing tools for monitoring and control purposes. In view of this, a rework of a fundamental model originally developed in our group to describe a single batch flotation cell into a spatially-finite framework is proposed. Such rework features the use of discrete bubble size classes and population balance equations to track the evolution of gas holdup in the pulp and froth zones, and combines this discretized scheme with first-order attachment and detachment mechanisms to describe bubble-particle interactions as a function of time and height. Training data gathered in-situ from experiments using a conductivity probe inserted in the pulp zone, a machine vision-based measurement system (VisioFroth™) mounted on top of the froth zone, and experimental protocols to measure the cumulative cumulative solids recovery were used to train the model parameters in the pulp, interface and froth zones. Once trained, the predictive capability of the model was qualitatively assessed by carrying out parametric studies using two independent operating variables: the air inlet flow rate and the stirring rate. Subsequently, quantitative assessments were conducted by contrasting model predictions with testing data, both under operating conditions different to those of the training data. These procedures were conducted using experimental data gathered under a hybrid operation mode, where agitation was provided by a mechanical impeller inside the JKTech cell located at the base of the column flotation device. The parametric fitting revealed that the bubble-particle attachment phenomena were considered the driving force in the modelling of solids recovery, with a reasonable match between model trends and training experimental data (SSE for cumulative solids recovery of 690.4, 289.6 and 117.9 in the experimental datasets HY 900RPM - 5LMIN, HY 900RPM 11LMIN 200PPM and HY 1500RPM 11LMIN, respectively). The parametric studies and the Normalized Forward Sensitivity Index analysis revealed that the solids recovery predictions made by the trained model had a higher sensitivity to variations in the air inlet flow rate, which was evidenced in the predictive studies with opposing performances in the prediction of the testing experimental data (SSE for cumulative solids recovery of 102.8 and 3909.5 in the experimental datasets HY 900RPM 11LMIN and HY 1500RPM 5LMIN, respectively) despite the acceptable performance of the trained model in the prediction of the bubble size distribution dynamics in both zones. The proposed strategies to improve the predictive capability of the model included the retraining of the model under higher training and testing data volumes, the use of alternative data split ratios and the use of estimation methods to refine the accuracy of the predictions in real time.

Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. However, the training data in source domain required by most of the existing methods is usually unavailable in real-world applications due to privacy preserving policies. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation. Firstly, we produce robust pseudo-labels for target data with spherical k-means clustering, whose initial class centers are the weight vectors (anchors) learned by the classifier of pretrained model. Furthermore, we propose to estimate the class-conditioned feature distribution of source domain by exploiting target data and corresponding anchors. Finally, we sample surrogate features from the estimated distribution, which are then utilized to align two domains by minimizing a contrastive adaptation loss function. Extensive experiments show that the proposed method achieves state-of-the-art performance on multiple DA benchmarks, and even outperforms traditional DA methods which require plenty of source data.

Predicting accurate remaining useful life (RUL) is quite necessary in many industrial cases. Recently, data-driven method has been widely used for RUL prediction because of its promising performance and ease of use. However, one practical challenging issue of the data-driven method is lack of labeled data. One way for dealing with this issue is domain adaptation. With domain adaptation technique, a RUL prediction model is built using labeled data of source domain and then the trained model is used to predict RUL of target domain where labeled data is unavailable. In this paper, we propose our method for RUL prediction through domain adaptation. Another practical issue of RUL prediction is that a single RUL prediction value may not be enough for successful maintenance of target systems. In other words, we need to know how uncertain RUL prediction is. In this paper, we study Monte Carlo (MC) dropout-based uncertainty metric for realizing better RUL prediction result. Through experiment, we first show that our domain adaptation technique shows reasonable performance. We also show that MC dropout-based approach generates the uncertainty metric only meaningful for source domain.

Deep learning has been applied to tackle the seismic inversion problem, bringing more efficiency and accuracy. However, bad spatial continuity and poor generalizability limit the practical application. To solve these problems, we propose a 2D end-to-end seismic inversion method based on domain adaption. Firstly, the proposed 2D network learns the inversion mapping of seismic data under the constraint of domain adaption layer, which can reduce the difference between the features of real seismic data and synthetic seismic data, improving the generalization ability on real seismic data. Then, the trained model is finetuned with well logging data. In the first process, the spatial continuity of the inversion result is guaranteed by the 2D training scheme. Meanwhile, due to the constraint of the domain adaption layer, our model not only performs well on the synthetic data but also has good generalization ability on the real seismic data. And we carefully discuss the mechanism of domain adaption layer. In the second process, finetuning introduces well logging information, which can further improve the ability to invert details. Moreover, in order to improve the inversion accuracy on real seismic data, we develop a new training data generation method that can generate the synthetic samples close to the real samples, and a 2.5D training strategy is adopted to improve the continuity of the 3D data. The experiments on both synthetic and real seismic data show that our method performs better than both the recursive inversion method and the 1D closed-loop CNN methods.

Recent progress in machine learning has been mainly due to the availability of large amounts of annotated data used for training complex models with deep architectures. Annotating this training data becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. Moreover, these trained models fail to generalize to new categories or new varieties of the same categories. This is because new categories or new varieties have data distribution different from the training data distribution. To tackle these problems, this thesis proposes to develop a family of transfer-learning techniques that can deal with different training (source) and testing (target) distributions with the assumption that the availability of annotated data is limited in the testing domain. This is done by using the auxiliary data-abundant source domain from which useful knowledge is transferred that can be applied to data-scarce target domain. This transferable knowledge serves as a prior that biases target-domain predictions and prevents the target-domain model from overfitting. Specifically, we explore structural priors that encode relational knowledge between different data entities, which provides more informative bias than traditional priors. The choice of the structural prior depends on the information availability and the similarity between the two domains. Depending on the domain similarity and the information availability, we divide the transfer learning problem into four major categories and propose different structural priors to solve each of these sub-problems. This thesis first focuses on the unsupervised-domain-adaptation problem, where we propose to minimize domain discrepancy by transforming labeled source-domain data to be close to unlabeled target-domain data. For this problem, the categories remain the same across the two domains and hence we assume that the structural relationship between the source-domain samples is carried over to the target domain. Thus, graph or hyper-graph is constructed as the structural prior from both domains and a graph/hyper-graph matching formulation is used to transform samples in the source domain to be closer to samples in the target domain. An efficient optimization scheme is then proposed to tackle the time and memory inefficiencies associated with the matching problem. The few-shot learning problem is studied next, where we propose to transfer knowledge from source-domain categories containing abundantly labeled data to novel categories in the target domain that contains only few labeled data. The knowledge transfer biases the novel category predictions and prevents the model from overfitting. The knowledge is encoded using a neural-network-based prior that transforms a data sample to its corresponding class prototype. This neural network is trained from the source-domain data and applied to the target-domain data, where it transforms the few-shot samples to the novel-class prototypes for better recognition performance. The few-shot learning problem is then extended to the situation, where we do not have access to the source-domain data but only have access to the source-domain class prototypes. In this limited information setting, parametric neural-network-based priors would overfit to the source-class prototypes and hence we seek a non-parametric-based prior using manifolds. A piecewise linear manifold is used as a structural prior to fit the source-domain-class prototypes. This structure is extended to the target domain, where the novel-class prototypes are found by projecting the few-shot samples onto the manifold. Finally, the zero-shot learning problem is addressed, which is an extreme case of the few-shot learning problem where we do not have any labeled data in the target domain. However, we have high-level information for both the source and target domain categories in the form of semantic descriptors. We learn the relation between the sample space and the semantic space, using a regularized neural network so that classification of the novel categories can be carried out in a common representation space. This same neural network is then used in the target domain to relate the two spaces. In case we want to generate data for the novel categories in the target domain, we can use a constrained generative adversarial network instead of a traditional neural network. Thus, we use structural priors like graphs, neural networks and manifolds to relate various data entities like samples, prototypes and semantics for these different transfer learning sub-problems. We explore additional post-processing steps like pseudo-labeling, domain adaptation and calibration and enforce algorithmic and architectural constraints to further improve recognition performance. Experimental results on standard transfer learning image recognition datasets produced competitive results with respect to previous work. Further experimentation and analyses of these methods provided better understanding of machine learning as well.

Driver drowsiness is considered as a significant risk factor that impacted to large number of road accidents. This paper presents a vehicle safety system developed using computer vision technique to detect driver drowsiness while driving. Deep learning method, YOLOv2 has been chosen as the artificial intelligence model to detect the percentage of eyes closure of the driver. A threshold value for percentage eye closure for the driver, where is when the set threshold value of 20% has been reached, response from a connected buzzer will alert the driver. This system development involves a few stages of dataset preparation and training. An accuracy of 86.58% has been reached to detect eyes closure in this training model. The trained model had been simulated using NVIDIA Jetson Nano platform to simulate real world application and find out a few areas of improvement such as the frequency of yawning additional feature parameter to detect drowsiness and self-training mode for face recognition and eyes dataset of the driver was recommended to personalize the development application and minimizing inaccurate output.KeywordsComputer visionAutonomous vehicle technologyDrowsiness detectionJetson nanoPERCLOS and YOLO

In this work, the right training criterion for optimal prediction of the desired output by the neural network model was our focus. The application of the early stopping technique in the neural network training of radial basis function model in comparison to the same model neural network training without application of early stopping technique was the basis for the work and the work validates the importance of early stopping technique for good network generalization through prevention of overfitting. An over-fitted radial basis function neural network model possesses the tendency for dataset memorization including noise and thus, poorly learns the dataset pattern. From the training results, the application of an early stopping technique inclusive of other vital hyper-parameters during the training of the radial basis function model demonstrates a measure that is very imperative in the prevention of over-fitting during network training. The training results show that the radial basis function model training without the application of an early stopping technique displayed a “learning and slow” phenomenon where the model stopped learning the dataset at a stage without optimum prediction of the dataset pattern. This phenomenon was evaded by the application of an early stopping technique at the ratio of 70%:15%:15% for training, testing, and validation of the standardized dataset. The radial basis function model neural network training regression results show optimum prediction results of R=0.99420 with a training set of R=0.9985 and a test set of R=0.9899 showing very close prediction of the training set by the test set on training with early stopping technique in comparison to the model training without the application of early stopping technique where R= 0.97200 with a training set of R= 0.9840 and a test set of R=0.9612 showing very wide prediction of the training set by the test set. Overfitting often results in good model training but poor test error as the network model learns the wrong structure in the training data or stops learning at a point as seen from the work. This was optimally evaded on the application of the early stopping technique in the work for robust prediction of signal power loss using a long-term evolution dataset.