Soft sensor modeling method for Pichia pastoris fermentation process based on substructure domain transfer learning

This paper introduces a novel soft sensor modeling method based on BDA-IPSO-LSSVM designed to address the issue of model failure caused by varying fermentation data distributions resulting from different operating conditions during the fermentation of different batches of Pichia pastoris. First, the problem of significant differences in data distribution among different batches of the fermentation process is addressed by adopting the balanced distribution adaptation (BDA) method from transfer learning. This method reduces the data distribution differences among batches of the fermentation process, while the fuzzy set concept is employed to improve the BDA method by transforming the classification problem into a regression prediction problem for the fermentation process. Second, the soft sensor model for the fermentation process is developed using the least squares support vector machine (LSSVM). The model parameters are optimized by an improved particle swarm optimization (IPSO) algorithm based on individual differences. Finally, the data obtained from the Pichia pastoris fermentation experiment are used for simulation, and the developed soft sensor model is applied to predict the cell concentration and product concentration during the fermentation process of Pichia pastoris. Simulation results demonstrate that the IPSO algorithm has good convergence performance and optimization performance compared with other algorithms. The improved BDA algorithm can make the soft sensor model adapt to different operating conditions, and the proposed soft sensor method outperforms existing methods, exhibiting higher prediction accuracy and the ability to accurately predict the fermentation process of Pichia pastoris under different operating conditions.

Recent progress in machine learning has been mainly due to the availability of large amounts of annotated data used for training complex models with deep architectures. Annotating this training data becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. Moreover, these trained models fail to generalize to new categories or new varieties of the same categories. This is because new categories or new varieties have data distribution different from the training data distribution. To tackle these problems, this thesis proposes to develop a family of transfer-learning techniques that can deal with different training (source) and testing (target) distributions with the assumption that the availability of annotated data is limited in the testing domain. This is done by using the auxiliary data-abundant source domain from which useful knowledge is transferred that can be applied to data-scarce target domain. This transferable knowledge serves as a prior that biases target-domain predictions and prevents the target-domain model from overfitting. Specifically, we explore structural priors that encode relational knowledge between different data entities, which provides more informative bias than traditional priors. The choice of the structural prior depends on the information availability and the similarity between the two domains. Depending on the domain similarity and the information availability, we divide the transfer learning problem into four major categories and propose different structural priors to solve each of these sub-problems. This thesis first focuses on the unsupervised-domain-adaptation problem, where we propose to minimize domain discrepancy by transforming labeled source-domain data to be close to unlabeled target-domain data. For this problem, the categories remain the same across the two domains and hence we assume that the structural relationship between the source-domain samples is carried over to the target domain. Thus, graph or hyper-graph is constructed as the structural prior from both domains and a graph/hyper-graph matching formulation is used to transform samples in the source domain to be closer to samples in the target domain. An efficient optimization scheme is then proposed to tackle the time and memory inefficiencies associated with the matching problem. The few-shot learning problem is studied next, where we propose to transfer knowledge from source-domain categories containing abundantly labeled data to novel categories in the target domain that contains only few labeled data. The knowledge transfer biases the novel category predictions and prevents the model from overfitting. The knowledge is encoded using a neural-network-based prior that transforms a data sample to its corresponding class prototype. This neural network is trained from the source-domain data and applied to the target-domain data, where it transforms the few-shot samples to the novel-class prototypes for better recognition performance. The few-shot learning problem is then extended to the situation, where we do not have access to the source-domain data but only have access to the source-domain class prototypes. In this limited information setting, parametric neural-network-based priors would overfit to the source-class prototypes and hence we seek a non-parametric-based prior using manifolds. A piecewise linear manifold is used as a structural prior to fit the source-domain-class prototypes. This structure is extended to the target domain, where the novel-class prototypes are found by projecting the few-shot samples onto the manifold. Finally, the zero-shot learning problem is addressed, which is an extreme case of the few-shot learning problem where we do not have any labeled data in the target domain. However, we have high-level information for both the source and target domain categories in the form of semantic descriptors. We learn the relation between the sample space and the semantic space, using a regularized neural network so that classification of the novel categories can be carried out in a common representation space. This same neural network is then used in the target domain to relate the two spaces. In case we want to generate data for the novel categories in the target domain, we can use a constrained generative adversarial network instead of a traditional neural network. Thus, we use structural priors like graphs, neural networks and manifolds to relate various data entities like samples, prototypes and semantics for these different transfer learning sub-problems. We explore additional post-processing steps like pseudo-labeling, domain adaptation and calibration and enforce algorithmic and architectural constraints to further improve recognition performance. Experimental results on standard transfer learning image recognition datasets produced competitive results with respect to previous work. Further experimentation and analyses of these methods provided better understanding of machine learning as well.

This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.

Unsupervised domain adaptation aims to transfer knowledge from labeled source domain to unlabeled target domain. Recently, multisource domain adaptation (MDA) has begun to attract attention. Its performance should go beyond simply mixing all source domains together for knowledge transfer. In this article, we propose a novel prototype-based method for MDA. Specifically, for solving the problem that the target domain has no label, we use the prototype to transfer the semantic category information from source domains to target domain. First, a feature extraction network is applied to both source and target domains to obtain the extracted features from which the domain-invariant features and domain-specific features will be disentangled. Then, based on these two kinds of features, the named inherent class prototypes and domain prototypes are estimated, respectively. Then a prototype mapping to the extracted feature space is learned in the feature reconstruction process. Thus, the class prototypes for all source and target domains can be constructed in the extracted feature space based on the previous domain prototypes and inherent class prototypes. By forcing the extracted features are close to the corresponding class prototypes for all domains, the feature extraction network is progressively adjusted. In the end, the inherent class prototypes are used as a classifier in the target domain. Our contribution is that through the inherent class prototypes and domain prototypes, the semantic category information from source domains is transformed into the target domain by constructing the corresponding class prototypes. In our method, all source and target domains are aligned twice at the feature level for better domain-invariant features and more closer features to the class prototypes, respectively. Several experiments on public data sets also prove the effectiveness of our method.

As an important and widely studied research topic, the detection of fresh milk plays a crucial role in protecting consumer health. However, due to the diversity of milk brands and environmental conditions in real-world application scenarios, the accuracy of existing detection models often drops significantly when applied to new domains. A significant amount of new data needs to be gathered in order to retrain the model for new domain, which greatly increases both time and labor costs. To tackle this problem, this paper introduces a transfer learning-based cross-domain milk freshness detection method. The method uses transfer learning to generate data for unknown categories within the target domain. Specifically, the transfer generation model trains on data from both the target and source domain categories. With the proposed model, data from other categories in the source domain can be utilized to generate corresponding target domain data. This method comprehensively considers data generation from multiple perspectives of time, frequency, and spatial, to enhance the authenticity of the generated data. It uses a transfer generation architecture, consisting of a transfer network and a decoder, to learn the mapping relationship between the source and target domain, helping to narrow the gap between domains. Additionally, a feature subspace decomposition method is introduced into the decoder, and a cross-domain consistency loss function is formulated to strengthen the model’s learning capability. The proposed method is shown to generate high-quality data for unknown target-domain categories in multiple cross-domain settings, leading to a performance improvement ranging from 17.64% to 32.44% in the detection of cross-domain milk freshness.

The concept of Industry 4.0 uses cyber-physical systems and the Internet of Things to create "smart factories" that enable automated and connected production. However, the complex and diverse nature of cyber-physical systems makes them vulnerable to physical threats, where any fault in their physical components can lead to serious consequences, from financial losses to human safety. To reduce the risk of such failures, it is essential to have techniques in place to monitor the condition of these physical components closely. In recent years, data-driven predictive maintenance methods have become common due to the growth in data and advancements in machine learning. This thesis focuses on fault diagnosis, a key part of predictive maintenance, which aims to detect and identify faults in physical components. Incorporating advanced techniques, including machine learning algorithms and sensor data analysis, enhanced fault diagnosis systems' accuracy and efficiency. However, several real-world barriers challenge the effectiveness of these techniques. This research is structured around three primary objectives. The first objective aims to establish an efficient fault diagnosis system that is highly sensitive to fault occurrences while being resilient to uncertainties such as variability in fault patterns and data imbalance. To address variability in fault patterns, we developed a multi-step approach integrating clustering techniques and an adaptive neuro-fuzzy inference system, which significantly improved diagnosis robustness even under noisy conditions. For imbalanced datasets, a two-step method was introduced, combining permutation entropy for fault detection and envelope analysis for fault isolation. This approach was enhanced by an automatic frequency band selection method leveraging wavelet transform and statistical criteria, resulting in high fault diagnosis accuracy and reliability, even with limited labeled faulty data. The second objective focuses on developing fault diagnosis systems that can maintain high performance despite domain shifts and limited training data in both source and target domains. To address the challenge of domain shifts, a transfer learning technique was introduced, and a systematic literature review was conducted. This study bridges a technical gap by providing a comprehensive taxonomy, state-of-the-art methods, and insights into the challenges of applying transfer learning to predictive maintenance. Additionally, a conceptual framework was proposed to explore scenarios for transferring diagnostic knowledge from different source domains to target domains with insufficient training datasets. In a subsequent study, a new diagnostic framework was developed using digital twins and partial transfer learning to address the lack of real training datasets in both source and target domains as well as label space inconsistencies between them. The framework includes a general approach for developing digital twins of real machines to generate synthetic diagnostic data as source domain data. Furthermore, by incorporating a deep learning network and a double-layer attention mechanism, a partial transfer learning diagnostic model was developed to transfer diagnostic knowledge from the digital twin to the real target domain despite label space inconsistencies. The proposed framework was validated through an industrial case study involving rolling bearings in rotating equipment. The third objective focuses on developing an efficient fault diagnosis system capable of maintaining reliability under unseen and evolving working conditions. To achieve this, a novel self-adaptive fault diagnosis system was proposed, incorporating the MAPE-K loop, a domain generalization network model, and digital twin technologies. This self-adaptive fault diagnosis system leverages the MAPE-K loop to adapt in real time to evolving working conditions. Digital twins were employed to continuously generate data aligned with the real system’s operational context, enabling accurate fault diagnosis in dynamic environments. The domain generalization network model enhances generalization to unseen target working conditions by addressing overfitting and feature loss through multi-domain data augmentation, adversarial learning, and domain discrepancy modules. Validation using multiple rotating machinery datasets demonstrated that the developed fault diagnosis system outperformed state-of-the-art methods across various scenarios. In summary, this dissertation proposes a comprehensive and practical approach to addressing variability in fault patterns, imbalance datasets, data scarcity, domain shifts, and unseen and evolving working conditions challenges in existing fault diagnosis systems.

Multisource unsupervised domain adaptation (MUDA) is an important and challenging topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is difficult to map all source domains and target domain into a common feature space for classifying the targets well. In this article, a new progressive multisource domain adaptation network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. First, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. By doing this, we can generally reduce the distribution discrepancy between each source and target domains, as well as the discrepancy between different source domains. It can efficiently explore useful knowledge from the integrated source domain. Second, to mine assistance knowledge from each source domain as much as possible, the distribution of the target domain is separately aligned with that of each source domain. A weighted fusion method is employed to combine the multiple classification results for making the final decision. In the optimization of domain adaption, weighted hybrid maximum mean discrepancy (WHMMD) is proposed, and it considers both the interclass and intraclass discrepancies. The effectiveness of the proposed PMSDAN is demonstrated in the experiments comparing with some state-of-the-art methods.

Domain-transfer learning is a machine learning task to explore a source domain data set to help the learning problem in a target domain. Usually, the source domain has sufficient labeled data, while the target domain does not. In this paper, we propose a novel domain-transfer convolutional model by mapping a target domain data sample to a proxy in the source domain and applying a source domain model to the proxy for the purpose of prediction. In our framework, we firstly represent both source and target domains to feature vectors by two convolutional neural networks and then construct a proxy for each target domain sample in the source domain space. The proxy is supposed to be matched to the corresponding target domain sample convolutional representation vector well. To measure the matching quality, we proposed to maximize their squared-loss mutual information (SMI) between the proxy and target domain samples. We further develop a novel neural SMI estimator based on a parametric density ratio estimation function. Moreover, we also propose to minimize the classification error of both source domain samples and target domain proxies. The classification responses are also smoothened by manifolds of both the source domain and proxy space. By minimizing an objective function of SMI, classification error, and manifold regularization, we learn the convolutional networks of both source and target domains. In this way, the proxy of a target domain sample can be matched to the source domain data and thus benefits from the rich supervision information of the source domain. We design an iterative algorithm to update the parameters alternately and test it over benchmark data sets of abnormal behavior detection in video, Amazon product reviews sentiment analysis, etc.

In the absence of the labeled samples in a domain referred to as target domain, Domain Adaptation (DA) techniques come in handy. Generally, DA techniques assume there are available source domains that share similar predictive function with the target domain. Two core challenges of DA typically arise, variance that exists between source and target domains, and the inherent source hypothesis bias. In this paper, we first propose a Stability Transfer criterion for selecting relevant source domains with low variance. With this criterion, we introduce a TARget learning Assisted by Source Classifier Adaptation (TARASCA) method to address the two core challenges that have impeded the performances of DA techniques. To verify the robustness of TARASCA, extensive experimental studies are carried out with comparison to several state-of-the-art DA methods on the real-world Sentiment and Newsgroups datasets, where various settings for the class ratios of the source and target domains are considered.

In many real-world applications, the domain of model learning (referred as source domain) is usually inconsistent with or even different from the domain of testing (referred as target domain), which makes the learnt model degenerate in target domain, i.e., the test domain. To alleviate the discrepancy between source and target domains, we propose a domain adaptation method, named as Bi-shifting Auto-Encoder network (BAE). The proposed BAE attempts to shift source domain samples to target domain, and also shift the target domain samples to source domain. The non-linear transformation of BAE ensures the feasibility of shifting between domains, and the distribution consistency between the shifted domain and the desirable domain is constrained by sparse reconstruction between them. As a result, the shifted source domain is supervised and follows similar distribution as target domain. Therefore, any supervised method can be applied on the shifted source domain to train a classifier for classification in target domain. The proposed method is evaluated on three domain adaptation scenarios of face recognition, i.e., domain adaptation across view angle, ethnicity, and imaging sensor, and the promising results demonstrate that our proposed BAE can shift samples between domains and thus effectively deal with the domain discrepancy.

Domain adaptation has achieved promising results in many areas, such as image classification and object recognition. Although a lot of algorithms have been proposed to solve the task with different domain distributions, it remains a challenge for multi-source unsupervised domain adaptation. In addition, most of the existing algorithms learn a classifier on the source domain and predict the labels for the target data, which indicates that only the knowledge derived from the hyperplane is transferred to the target domain and the structure information is ignored. In light of this, we propose a novel algorithm for multi-source unsupervised domain adaptation. Generally speaking, we aim to preserve the whole structure from source domains and transfer it to serve the task on the target domain. The source and target data are put together for clustering, which simultaneously explores the structures of the source and target domains. The structure-preserved information from source domain further guides the clustering process on the target domain. Extensive experiments on two widely used databases on object recognition and face identification show the substantial improvement of our proposed approach over several state-of-the-art methods. Especially, our algorithm can take use of multi-source domains and achieve robust and better performance compared with the single source domain adaptation methods.

When deep neural networks are deployed in a highly automated vehicle for environment perception tasks in an unseen (target) domain that differs from the training (source) domain, the mismatch will result in decreased performance. Domain adaptation methods aim at overcoming this mismatch. Many recently investigated methods for unsupervised domain adaptation train a model using labeled source data and unlabeled target data at the same time. These methods assume that data from the target domain is available during the source domain training, which is not always the case in real applications. In this paper we present a way to perform an online style transfer for continual domain adaptation which improves performance on (multiple) unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given source-trained model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain representation of the source domain style, which is particularly important considering the hardware limitations in an autonomous vehicle.

Domain adaptation (DA) is an efficient approach that transfers the knowledge gained from the source domain to learn an accurate classifier for the target domain. But the majority of the current domain adaptation algorithms assume that both the training and testing data have similar feature space with unaligned data distribution. Such algorithms fail when the feature representations vary. Heterogeneous Domain Adaptation (HDA) effectively addresses multi-modal data analysis where the features and distribution of source and target domain data are different. But, some of the existing heterogeneous domain adaptation approaches utilize either statistical or geometrical data alignment. Hence, we propose Joint Cross-Domain Preserving and Distribution Adaptation (JCDDA) for semi-supervised HDA, which learns a new latent subspace where the data samples from the source and target domain are projected. JCDDA preserves the cross-domain structure of the data and minimizes the discrimination differences between the domains. As our approach is semi-supervised HDA, we exploit the selective pseudo-labeling method to utilize unlabelled target data. We did extensive experiments using two benchmark object recognition datasets, and from our results, it is evident that our algorithm outperforms all of them.

Easy Domain Adaptation for cross-subject multi-view emotion recognition

Unsupervised domain adaptation (UDA) has been widely used to transfer knowledge from a labeled source domain to an unlabeled target domain to counter the difficulty of labeling in a new domain. The training of conventional solutions usually relies on the existence of both source and target domain data. However, privacy of the large-scale and well-labeled data in the source domain and trained model parameters can become the major concern of cross center/domain collaborations. In this work, to address this, we propose a practical solution to UDA for segmentation with a black-box segmentation model trained in the source domain only, rather than original source data or a white-box source model. Specifically, we resort to a knowledge distillation scheme with exponential mixup decay (EMD) to gradually learn target-specific representations. In addition, unsupervised entropy minimization is further applied to regularization of the target domain confidence. We evaluated our framework on the BraTS 2018 database, achieving performance on par with white-box source model adaptation approaches.