A Honeynet-Driven Privacy Preserving Incremental Learning Framework for Cyberattack Classification in Industry 4.0 Environments

Based on the current digital industry background, the production processes become centralized and closely related to each other. If industrial production fails, it will bring great damage to it, which makes it more and more important to accurately identify the status of industrial equipment and repair it in real time by collecting time series data with industrial sensors. However, due to the large number, variety and high-frequency sampling of sensors, industrial big data has certain complexity, i.e. high spatial dimension, complex logical relationship, changeable rules, large amount of data and so on. At present, end-to-end algorithms of deep learning are widely used in many fields. From the perspective of application of deep learning, this paper studies its application in industrial big data time series classification, and analyzes the characteristics of industrial data and the challenges of industrial time series classification from three aspects: accurate classification, efficient classification and incremental learning.

In practical industrial inspection scenarios, the challenges of limited samples and severe domain shift render achieving reliable few-shot image classification a pressing challenge. In response to this, we propose a lightweight small-sample industrial quality inspection image classification and confidence calibration framework based on Fast AutoAugment. The framework employs EfficientNetV2 as the basic feature extractor, using an advanced policy search mechanism enabled by Fast AutoAugment, which successfully enhances the model’s generalization ability to defect features. In relation to feature fusion, we design a multi-branch feature fusion module that combines spatial attention, channel recalibration and contrast alignment mechanisms to further promote discriminant expression in small sample conditions. Additionally, we design an uncertainty calibration module based on Dirichlet distribution, which integrates task-adaptive priors and temperature scaling strategies to effectively calibrate the model’s confidence and reduce the risk of "overconfidence errors" in industrial settings. Experiments on the NEU-CLS industrial quality inspection dataset show that the proposed method outperforms the existing Few-shot baseline, demonstrating a classification accuracy that is 4.6%-7.1% higher than that of the existing few-shot baselines.

Detection and classification are two significant tasks for industrial fault diagnosis. However, conventional methods typically treat these tasks as separate and independent problems, and necessitate a retraining process when new fault samples or classes are collected. Therefore, an incremental support vector data description scheme using Gaussian kernel function is pro-posed for industrial process fault diagnosis in a unified frame-work. In this framework, the decision boundary is updated incrementally only based on the specific original support vectors and newly collected samples. An adaptive threshold and a restructured radius are proposed to promote accuracy in the fault detection. In the classification procedure, the hyperspheres for all known classes are constructed by decision tree. The new sample that does not belong to any known class is identified as an unknown class. Without a time-consuming retraining process, the proposed diagnosis method with the incremental learning capability can synchronously achieve the fault detection and classification task. Experimental results demonstrate the effectiveness and superiority in terms of diagnosis performance. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In practical industrial processes, new fault samples are collected and new fault classes emerge continually. Under such scenarios, fault diagnosis with incremental learning capability is becoming increasingly important. This work proposes a unified incremental framework for fault diagnosis based on support vector data description, which is able to achieve fault detection and classification synchronously. For fault detection, an adaptive threshold and a restructured radius are developed. For fault classification, hypersphere-shaped boundaries of all fault classes are given via the decision tree-based strategy. The proposed method is updated to include new fault samples and fault classes without dimensionality reduction or any distributional assumptions.

Contrastive Learning (CL) has been proved to be a powerful self-supervised approach for a wide range of domains, including computer vision and graph representation learning. However, the incremental learning issue of CL has rarely been studied, which brings the limitation in applying it to real-world applications. Contrastive learning identifies the samples with the negative ones from the noise distribution that changes in the incremental scenarios. Therefore, only fitting the change of data without noise distribution causes bias, and directly retraining results in low efficiency. To bridge this research gap, we propose a self-supervised Incremental Contrastive Learning (ICL) framework consisting of (i) a novel Incremental InfoNCE (NCE-II) loss function by estimating the change of noise distribution for old data to guarantee no bias with respect to the retraining, (ii) a meta-optimization with deep reinforced Learning Rate Learning (LRL) mechanism which can adaptively learn the learning rate according to the status of the training processes and achieve fast convergence which is critical for incremental learning. Theoretically, the proposed ICL is equivalent to retraining, which is based on solid mathematical derivation. In practice, extensive experiments in different domains demonstrate that, without retraining a new model, ICL achieves up to 16.7x training speedup and 16.8x faster convergence with competitive results.

To develop a comprehensive deep learning and radiomics-based multi-task pipeline for the detection and classification of key fetal anatomical structures in first-trimester ultrasound images, using a diverse multi-center dataset to ensure high variability, reproducibility, and generalizability. A total of 4,532 fetal ultrasound scans (gestational age 11–14 weeks), retrospectively collected from nine medical centers, were included in this study. Two detection models, You Only Look Once version 11 (YOLOv11) and shifted window transformer (Swin Transformer), were trained to localize nine fetal brain and craniofacial structures. From each detected region, 215 radiomic features and 1,792 deep features were extracted. Feature stability was ensured through intra-class correlation coefficient (ICC) filtering (threshold ≥ 0.75), Pearson correlation analysis, and least absolute shrinkage and selection operator (LASSO) regression. The selected features were then used to train a Transformer-based model for tabular data (TabTransformer) to classify fetal anatomical structures into clinically defined categories based on their sonographic appearance. Model performance was evaluated using Accuracy, area under the receiver operating characteristic curve (AUC), and Sensitivity across training, internal validation, and external test datasets. Fusion models integrating radiomic and deep features consistently outperformed single-modality models in both detection and classification. On the external test set, classification accuracy reached 96.1%, with AUCs up to 96.89%, and sensitivity exceeding 95% for key anatomical structures. Swin Transformer achieved superior localization performance compared to YOLOv11, with Intersection over Union (IoU) values up to 0.97 and F1-scores ≥ 0.94. Feature reproducibility remained above 75% across centers. The TabTransformer classifier demonstrated strong generalization and robustness, effectively leveraging the fused feature space for high-precision classification. This study presents the fully integrated, multi-task framework for fetal anatomical structure detection and classification using multi-center ultrasound data. The proposed approach demonstrates high reproducibility and diagnostic performance, offering strong clinical potential for early and objective fetal anomaly screening in the first trimester.

With the advent of parameter-efficient fine-tuning techniques for pre-trained vision-language models, interest in adapting them for various incremental learning scenarios has grown, i.e., sequential increments on task, class, and domain. However, no high-performance incremental learning framework has integrated these three incremental scenarios to achieve Unified Incremental Learning (Uni-IL) in complex settings. In this work, we propose an incremental learning framework called Mixture of Attribute-Guided Experts (MAGE) to alleviate the long-term forgetting in vision-language model incremental learning. Our approach involves acquiring image attribute knowledge via LLMs to form an attribute pool. We match the most relevant attributes as inputs to the Mixture of Experts (MoE) to fine-tune the pre-trained CLIP. Then the expert routers learn to select specific expert combinations based on the data and attribute features, alleviating catastrophic forgetting. The attribute pool incorporates both domain and class knowledge, enabling our approach to adapt to the three types of incremental learning scenarios and thus facilitating unified incremental learning. Through extensive experiments on our newly proposed benchmark and existing incremental learning scenarios, the results demonstrate that our proposed method not only performs well on the new Uni-IL tasks but also consistently outperforms previous state-of-the-art methods. Source code is available at https://github.com/ElectricField/Uni-IL.

The ability to learn more concepts from incrementally arriving data over time is essential for the development of a lifelong learning system. However, deep neural networks often suffer from forgetting previously learned concepts when continually learning new concepts, which is known as the catastrophic forgetting problem. The main reason for catastrophic forgetting is that past concept data are not available, and neural weights are changed during incrementally learning new concepts. In this article, we propose an incremental concept learning framework that includes two components, namely, ICLNet and RecallNet. ICLNet, which consists of a trainable feature extractor and a dynamic concept memory matrix, aims to learn new concepts incrementally. We propose a concept-contrastive loss to alleviate the magnitude of neural weight changes and mitigate the catastrophic forgetting problems. RecallNet aims to consolidate old concepts memory and recall pseudo samples, whereas ICLNet learns new concepts. We propose a balanced online memory recall strategy to reduce the information loss of old concept memory. We evaluate the proposed approach on the MNIST, Fashion-MNIST, and SVHN data sets and compare it with other pseudorehearsal-based approaches. Extensive experiments demonstrate the effectiveness of our approach.

Intent classification (IC) and slot filling (SF) are critical building blocks in task-oriented dialogue systems. These two tasks are closely-related and can flourish each other. Since only a few utterances can be utilized for identifying fast-emerging new intents and slots, data scarcity issue often occurs when implementing IC and SF. However, few IC/SF models perform well when the number of training samples per class is quite small. In this paper, we propose a novel explicit-joint and supervised-contrastive learning framework for few-shot intent classification and slot filling. Its highlights are as follows. (i) The model extracts intent and slot representations via bidirectional interactions, and extends prototypical network to achieve explicit-joint learning, which guarantees that IC and SF tasks can mutually reinforce each other. (ii) The model integrates with supervised contrastive learning, which ensures that samples from same class are pulled together and samples from different classes are pushed apart. In addition, the model follows a not common but practical way to construct the episode, which gets rid of the traditional setting with fixed way and shot, and allows for unbalanced datasets. Extensive experiments on three public datasets show that our model can achieve promising performance.

A deep learning framework for ECG denoising and classification

A deep learning framework for road marking extraction, classification and completion from mobile laser scanning point clouds

Short text classification has gained significant attention in the information age due to its prevalence and real-world applications. Recent advancements in graph learning combined with contrastive learning have shown promising results in addressing the challenges of semantic sparsity and limited labeled data in short text classification. However, existing models have certain limitations. They rely on explicit data augmentation techniques to generate contrastive views, resulting in semantic corruption and noise. Additionally, these models only focus on learning the intrinsic consistency between the generated views, neglecting valuable discriminative information from other potential views. To address these issues, we propose a Simple graph contrastive learning framework for Short Text Classification (SimSTC). Our approach involves performing graph learning on multiple text-related component graphs to obtain multi-view text embeddings. Subsequently, we directly apply contrastive learning on these embeddings. Notably, our method eliminates the need for data augmentation operations to generate contrastive views while still leveraging the benefits of multi-view contrastive learning. Despite its simplicity, our model achieves outstanding performance, surpassing large language models on various datasets.

It is a big problem that a model of deep learning for a picking robot needs many labeled images. Operating costs of retraining a model becomes very expensive because the object shape of a product or a part often is changed in a factory. It is important to reduce the amount of labeled images required to train a model for a picking robot. In this study, we propose a multi-task learning framework for few-shot classification using feature vectors from an intermediate layer of a model that detects grasping positions. In the field of manufacturing, multitask for shape classification and grasping-position detection is often required for picking robots. Prior multi-task learning studies include methods to learn one task with feature vectors from a deep neural network (DNN) learned for another task. However, the DNN that was used to detect grasping positions has two problems with respect to extracting feature vectors from a layer for shape classification: (1) Because each layer of the grasping position detection DNN is activated by all objects in the input image, it is necessary to refine the features for each grasping position. (2) It is necessary to select a layer to extract the features suitable for shape classification. To tackle these issues, we propose a method to refine the features for each grasping position and to select features from the optimal layer of the DNN. We then evaluated the shape classification accuracy using these features from the grasping positions. Our results confirm that our proposed framework can classify object shapes even when the input image includes multiple objects and the number of images available for training is small.

Few-shot relation classification seeks to classify incoming query instances\nafter meeting only few support instances. This ability is gained by training\nwith large amount of in-domain annotated data. In this paper, we tackle an even\nharder problem by further limiting the amount of data available at training\ntime. We propose a few-shot learning framework for relation classification,\nwhich is particularly powerful when the training data is very small. In this\nframework, models not only strive to classify query instances, but also seek\nunderlying knowledge about the support instances to obtain better instance\nrepresentations. The framework also includes a method for aggregating\ncross-domain knowledge into models by open-source task enrichment.\nAdditionally, we construct a brand new dataset: the TinyRel-CM dataset, a\nfew-shot relation classification dataset in health domain with purposely small\ntraining data and challenging relation classes. Experimental results\ndemonstrate that our framework brings performance gains for most underlying\nclassification models, outperforms the state-of-the-art results given small\ntraining data, and achieves competitive results with sufficiently large\ntraining data.\n

In this study, a deep learning framework for multiclass ECG classification based on a 2D-to-1D data transformation is presented. The proposed methodology integrates a deep autoencoder and a long short-term memory network to enable sequential modeling of compressed 2D signal representations. This strategy effectively addresses the challenge of data format compatibility, enabling the training of models on data initially unsuitable for such architectures. ECG signals were transformed into 2D representations using techniques such as recurrence plots, visibility matrices, Gram matrices, and Markov transition fields, followed by compression into low-dimensional vectors suitable for sequence processing. The proposed method was rigorously evaluated using the MIT-BIH arrhythmia database and demonstrated superior performance compared to contemporary state-of-the-art models. The method achieved high metric values, with MTFs yielding the best results: 99.8% accuracy, 99.8% precision, 99.8% recall, 99.8% specificity, 99.8% F1-score, 99.7% Matthews correlation coefficient, and 99.3% Cohen’s kappa, confirming its effectiveness for real-time ECG classification in clinical scenarios. Furthermore, the model’s versatility was validated through its successful application to image classification tasks, highlighting its adaptability to a wide range of data types beyond ECG signals.

Cancer remains a major global health burden, and conventional treatments such as surgery, radiotherapy, and chemotherapy are often limited by systemic toxicity, drug resistance, and high cost. Anticancer peptides (ACPs) have emerged as promising therapeutic candidates because of their selective activity against tumor cells; however, experimental identification of ACPs is labor-intensive and time-consuming, and peptide activity may vary across tissue contexts. Although many computational models have been developed for general ACP prediction, tissue-specific ACP classification remains insufficiently explored. In this study, we developed ACP-Boost, a tissue-aware machine learning framework for tissue-specific ACP classification across nine cancer-related tissues: blood, brain, breast, cervix, colon, liver, lung, prostate, and skin. Experimentally validated peptide records were integrated from CancerPPD2 and DCTPep, followed by preprocessing, redundancy control, and tissue annotation. Peptide sequences were encoded into 473-dimensional feature vectors comprising amino acid composition (AAC), dipeptide composition (DPC), physicochemical property composition (PCP), and pseudo-amino acid composition (PseAAC). To address class imbalance while preserving the original sequence distribution, we formulated the task as a one-versus-rest classification problem, applied group-aware train-test splitting based on peptide sequences, and emphasized evaluation metrics suitable for imbalanced data. We systematically compared five machine learning algorithms, including support vector machine, random forest, logistic regression, k-nearest neighbors, and XGBoost. Among them, XGBoost showed the most stable overall performance across tissues. The results indicate that peptide sequence-derived descriptors contain measurable tissue-associated signals, although predictive separability remains moderate for several cancer types. Feature importance analysis further suggested that both shared charge-related properties and tissue-dependent sequence descriptors contribute to model discrimination. Overall, this study provides a comparative computational framework for tissue-specific ACP classification and highlights the importance of incorporating biological heterogeneity into peptide prediction tasks.