JOSAL: Joint Learning Framework for Open-Set Active Learning

Federated learning enables multiple devices to collaboratively train a high-performance model on the central server while keeping their data on the devices themselves. However, due to the significant variability in data distribution across devices, the aggregated global model's optimization direction may differ from that of the local models, making the clients lose their personality. To address this challenge, we propose a Bidirectional Decoupled Distillation For Heterogeneous Federated Learning (BDD-HFL) approach, which incorporates an additional private model within each local client. This design enables mutual knowledge exchange between the private and local models in a bidirectional manner. Specifically, previous one-way federated distillation methods mainly focused on learning features from the target class, which limits their ability to distill features from non-target classes and hinders the convergence of local models. To solve this limitation, we decompose the network output into target and non-target class logits and distill them separately using a joint optimization of cross-entropy and decoupled relative-entropy loss. We evaluate the effectiveness of BDD-HFL through extensive experiments on three benchmarks under IID, Non-IID, and unbalanced data distribution scenarios. Our results show that BDD-HFL outperforms state-of-the-art federated distillation methods across five baselines, achieving at most 3% improvement in average classification accuracy on the CIFAR-10, CIFAR-100, and MNIST datasets. The experiments demonstrate the superiority and generalization capability of BDD-HFL in addressing personalization challenges in federated learning.

Unlabeled training examples are readily available in many applications, but labeled examples are fairly expensive to obtain. For instance, in our previous works on classification of peer-to-peer (P2P) Internet traffics, we observed that only about 25% of examples can be labeled as “P2P”or “NonP2P” using a port-based heuristic rule. We also expect that even fewer examples can be labeled in the future as more and more P2P applications use dynamic ports. This fact motivates us to investigate the techniques which enhance the accuracy of P2P traffic classification by exploiting the unlabeled examples. In addition, the Internet data flows dynamically in large volumes (streaming data). In P2P applications, new communities of peers often join and old communities of peers often leave, requiring the classifiers to be capable of updating the model incrementally, and dealing with concept drift. Based on these requirements, this paper proposes an incremental Tri-Training (iTT) algorithm. We tested our approach on a real data stream with 7.2 Mega labeled examples and 20.4 Mega unlabeled examples. The results show that iTT algorithm can enhance accuracy of P2P traffic classification by exploiting unlabeled examples. In addition, it can effectively deal with dynamic nature of streaming data to detect the changes in communities of peers. We extracted attributes only from the IP layer, eliminating the privacy concern associated with the techniques that use deep packet inspection.

Feature selection is typically employed before or in conjunction with classification algorithms to reduce the feature dimensionality and improve the classification performance, as well as reduce processing time. While particular approaches have been developed for feature selection, such as filter and wrapper approaches, some algorithms perform feature selection through their learning strategy. In this paper, we are investigating the effect of the implicit feature selection of the PRISM algorithm, which is rule-based, when compared with the wrapper feature selection approach employing four popular algorithms: decision trees, na'ive bayes, k-nearest neighbors and support vector machine. Moreover, we investigate the performance of the algorithms on target classes, i.e. where the aim is to identify one or more phenomena and distinguish them from their absence (i.e. non-target classes), such as when identifying benign and malign cancer (two target classes) vs. non-cancer (the non-target class).

Joint learning with diverse knowledge for re-identification

In drug development, protocols define how clinical trials are conducted, and are therefore of paramount importance. They contain key patient-, investigator-, medication-, and study-related information, often elaborated in different sections in the protocol texts. Granular-level parsing on large quantity of existing protocols can accelerate clinical trial design and provide actionable insights into trial optimization. Here, we report our progresses in using deep learning NLP algorithms to enable automated protocol analytics. In particular, we combined a pre-trained BERT transformer model with joint-learning strategies to simultaneously identify clinically relevant entities (i.e. Named Entity Recognition) and extract the syntactic relations between these entities (i.e. Relation Extraction) from the eligibility criteria section in protocol texts. When comparing to standalone NER and RE models, our joint-learning strategy can effectively improve the performance of RE task while retaining similarly high NER performance, likely due to the synergy of optimizing toward both tasks’ objectives via shared parameters. The derived NLP model provides an end-to-end solution to convert unstructured protocol texts into structured data source, which will be embedded into a comprehensive clinical analytics workflow for downstream trial design missions such like patient population extraction, patient enrollment rate estimation, and protocol amendment prediction.

The efficiency of a binary support vector machine- (SVM-) based classifier depends on the combination and the number of input features extracted from raw signals. Sometimes, a combination of individual good features does not perform well in discriminating a class due to a high level of relevance to a second class also. Moreover, an increase in the dimensions of an input vector also degrades the performance of a classifier in most cases. To get efficient results, it is needed to input a combination of the lowest possible number of discriminating features to a classifier. In this paper, we propose a framework to improve the performance of an SVM-based classifier for sensor fault classification in two ways: firstly, by selecting the best combination of features for a target class from a feature pool and, secondly, by minimizing the dimensionality of input vectors. To obtain the best combination of features, we propose a novel feature selection algorithm that selects m out of M features having the maximum mutual information (or relevance) with a target class and the minimum mutual information with nontarget classes. This technique ensures to select the features sensitive to the target class exclusively. Furthermore, we propose a diversified-input SVM (DI-SVM) model for multiclass classification problems to achieve our second objective which is to reduce the dimensions of the input vector. In this model, the number of SVM-based classifiers is the same as the number of classes in the dataset. However, each classifier is fed with a unique combination of features selected by a feature selection scheme for a target class. The efficiency of the proposed feature selection algorithm is shown by comparing the results obtained from experiments performed with and without feature selection. Furthermore, the experimental results in terms of accuracy, receiver operating characteristics (ROC), and the area under the ROC curve (AUC-ROC) show that the proposed DI-SVM model outperforms the conventional model of SVM, the neural network, and the k-nearest neighbor algorithm for sensor fault detection and classification.

Joint learning on diverse ultrasonic source scenarios presents a challenge in preserving stable generalization due to the combination of heterogeneity of different sources and the inconsistency of joint learning features. Previous joint learning studies, which are not source-resilient frameworks, may not preserve stable generalization when trained on diverse source scenarios. Furthermore, the limited variations in single-source data and the interference from ultrasound imaging, which are common in ultrasonic source scenarios, further decrease generalization. To address these problems, we proposed a source-resilient joint learning framework consisting of three stages: 1) Source transforming, where our 1-to-N transformation unifies diverse source scenarios for source-resiliency. 2) Our feature enhancement modules model the source-resilient joint learning network, including a manifold-constraint normalization module (MCNM) for addressing heterogeneity by minimizing manifold-based loss, a task-consistent attention module (TCAM) shares the multi-scale features with self-attention to address inconsistency, and an adaptive feature-shifting module (AFSM) for feature-level augmentation to overcome single-source data. 3) Our ultrasound-hybrid linear mapping (USmapping) cascades speckle randomization and mask-guiding Monge-Kantorovitch linear mapping to achieve ultrasonic style randomization for addressing the interference of ultrasonic data. Our framework was evaluated on eight ultrasound datasets from various scanners at multiple centers and surpassed previous comparable studies in both segmentation (DSC$WAvg$ of 75.7%) and classification (AUROC$WAvg$ of 68.8%) tasks. Our framework has the potential to serve as a general framework for enhancing the performance of joint learning under diverse ultrasonic source scenarios.

Despite the fact that nonlinear subspace learning techniques (e.g. manifold learning) have successfully applied to data representation, there is still room for improvement in explainability (explicit mapping), generalization (out-of-samples), and cost-effectiveness (linearization). To this end, a novel linearized subspace learning technique is developed in a joint and progressive way, called joint and progressive learning strategy (J-Play), with its application to multi-label classification. The J-Play learns high-level and semantically meaningful feature representation from high-dimensional data by (1) jointly performing multiple subspace learning and classification to find a latent subspace where samples are expected to be better classified; (2) progressively learning multi-coupled projections to linearly approach the optimal mapping bridging the original space with the most discriminative subspace; (3) locally embedding manifold structure in each learnable latent subspace. Extensive experiments are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with previous state-of-the-art methods.

Semantic change detection (SCD) in high-resolution (HR) remote sensing images faces two issues: (1) isolated network branch for binary change detection (BCD) within multi-task architecture result in suboptimal SCD performance; (2) false alarms or missed detections caused by illumination differences or seasonal transform. To address these issues, this study proposes a bi-temporal binary change enhancement network (Bi-BCENet). Specifically, we introduce a binary change enhancement (BCE) strategy based on multi-network joint learning to achieve superior SCD via improving change areas prediction. Within the network’s reasoning process, we develop a cross-attention fusion module (CAFM) to enhance the global similarity modeling via cross-network prompt fusion, and we employ a cosine similarity-based auxiliary loss to optimize non-change’s semantic consistency. The experiments on SECOND and CINA-FX datasets demonstrate that Bi-BCENet outperforms representative SCD networks, achieving 62.08%, 84.95% in FSCD and 66.88%, 83.10% in mIoUsCD, respectively. And the ablation analysis of network validates Bi-BCENet’s effectiveness in reducing false alarms and missed detections in SCD results. Moreover, for specific SCD of cropland, Bi-BCENet shows its strong potential in single-to-multi SCD.

The micro-Doppler effect refers to the modulation of radar signals caused by the motion of objects with internal moving parts, such as rotating blades, oscillating components, or vibrating structures. This effect can be utilized in various applications such as target recognition and classification of moving objects. Automotive radar target classification, using micro-Doppler signatures, has become a crucial technique for enhancing advanced driver assistance systems (ADAS) and autonomous driving capabilities. Micro-Doppler signatures arise from small-scale movements such as the rotation of wheels, pedestrian’s arms swinging, pedals and wheels rotation of bicycle or vibration of objects superimposed on the primary Doppler shift caused by the object’s overall motion. Effective classification improves object detection accuracy, enhances safety and helps in predictive decision-making for autonomous driving systems. This paper presents a lightweight and efficient framework for six-class classification by combining shallow learning and ensemble methods. A single-layer CNN with only four filters is used for feature extraction, followed by wavelet transform for dimension reduction instead of traditional pooling. Further reduction and class separability are achieved using 2D Linear Discriminant Analysis. Finally, an ensemble of 200 decision trees is employed for classification. This hybrid approach offers a novel alternative to deep models, balancing performance with low computational cost. Simulation results show that the proposed method has 97.33% classification accuracy with 30dB signal to noise ratio (SNR), which outperforms existing deep learning models in terms of classification accuracy and computational efficiency, demonstrating its effectiveness for automotive radar target classification.

Active learning for ranking, which is to selectively label the most informative examples, has been widely studied in recent years. In this paper, we propose a general active learning for ranking strategy called Variance Maximization (VM). The algorithm relies on noise injection to perturb the original unlabeled examples and generate the rank distribution of each example. Using a DCG-like gain function to measure each ranked list sampled from the rank distribution, Variance Maximization selects the unlabeled example with the largest variance in the gain. The VM strategy is applied at both the query level and the document level, and a two-stage active learning algorithm is further derived. Experimental results on both the LETOR 4.0 dataset and a real-world Web search ranking dataset have demonstrated the effectiveness of the proposed active learning approach.

SDSV: Angle Measurement for Supervised Classification

Evaluation of adulteration in soy-based beverages by water addition using chemometrics applied to ATR-FTIR spectroscopy

Clustering-based proxy measure for optimizing one-class classifiers

Educational literature contains extensive descriptions and analysis of active and collaborative learning techniques. For example, experiments in mathematics date back to at least the 1980s. Physicists have been particularly active in developing courses that promote active learning and integrate laboratory activities with lectures. In the mid 1990s, Hake collected data on 6542 students in 62 introductory physics courses to conclude that lecture-based courses yield only meager learning of physics concepts, while courses utilizing interactive engagement are substantially better in promoting student learning of concepts. Further, extensive studies have shown that active and collaborative learning techniques improve skill development, decrease failure rates, encourage success and interest among women and under-represented groups, and promote higher-order thinking skills.In computer science, much discussion at SIGCSE conferences and other forums has focused upon active and collaborative learning. For the most part, however, faculty have focused on small-scale activities within classes, such as small-group exercises, kinesthetic learning activities, hands-on activities, and student-faculty dialog. Further, while many courses contain a component involving active learning, these courses often continue with extensive use of lecture. In many cases, it seems that active and collaborative activities are not trusted to promote learning, so lectures continue.The pedagogy described in this paper utilizes a lab-based style emphasizing collaboration. In its pure form, developed and refined by the author in several courses since Fall 1992, students work in pairs within introductory-level computing courses to complete about 45 laboratory exercises in a course that meets for 56 sessions in a semester. In this format, the instructor lectures about 4 hours per month --- often in 5--15 minute segments. In an alternative format, students complete about 35 laboratory exercises, some of which extend over two class periods.In comparing this approach with a traditional format that utilizes lectures and separate lab sections, the lab-based approach covers more material, student performance on tests is improved, students are highly motivated and consistently prepared for class, students are engaged in the material, and retention rates are very high.