MetaNER: Named Entity Recognition with Meta-Learning

Medical image segmentation is vital for clinical diagnosis, yet deep learning models face domain shift challenges when test data distributions differ from training data. Single-source domain generalization (DG) has emerged as a solution to these limitations of deep learning models by training models on a single source domain to generalize to unseen target domains. Current single-source DG methods employ domain randomization (e.g., input/feature-space perturbations) to simulate unseen target domains, but face two key limitations: (1) Texture bias in CNN-based architectures due to their local processing nature, leading to overfitting and poor generalization; (2) Restricted augmented style diversity caused by solely source-dependent feature perturbations and input- or feature-only augmentation, resulting in insufficient coverage of target domain distributions and degraded model's generalizationability. Motivated by the Fourier transform's capability to model global dependency and decouple domain invariant features, we propose Random Domain Generalization (RandDG), a frequency-aware domain randomization method for single-source DG in medical image segmentation. This method enhances generalization ability through coordinated input and feature spaces perturbations while maintaining parameter efficiency via lightweight architectural design in the frequencydomain. The proposed RandDG incorporates a novel Global U-Shape Network (GUNet) segmentation architecture for efficient long-range dependency modeling and a Uniform Low Frequency spectrum Transform (ULoFT) filter for feature-space perturbation to enhance domain generalization ability, facilitating the learning of global domain invariant structural information through consistency constraints within the dual-space randomization framework. More specifically, first, the GUNet architecture combines a lightweight encoder with frequency-domain global filtering and forms a U-shape structure through a skip-connected decoder, utilizing Fourier transforms to capture long-range dependencies while minimizing parameters. Second, the ULoFT filter perturbs feature-space styles by Bernoulli mixing statistics of the source low-frequency amplitude spectrum with uniformly sampled values, thereby reducing sole reliance on source statistics and broadening the coverage of potential target domains. Third, a dual-space randomization framework simultaneously applies input-space style augmentation via global intensity nonlinear augmentation (GIN) filters and feature-space amplitude spectrum perturbation via ULoFT filter, regularized by a consistency loss to enforce domain invariant representationlearning. Experiments were conducted on an abdominal CT-MRI dataset for abdominal multi-organ segmentation from CT to T2 spectral presaturation with inversion recovery (T2-SPIR) MRI and a cross-center prostate dataset for prostate segmentation from one center to others. The proposed RandDG method demonstrated superior generalization ability, significantly improving robustness against domain shifts compared to competitive methods. When tested on unseen domains, the proposed method achieved an average DSC of 87.96% and an average HD of 4.82 mm on the abdominal CT-MRI dataset, as well as an average DSC of 75.95% and an average HD of 8.36 mm on the cross-center prostate dataset. Compared to the UNet baseline, the proposed method achieved an improvement of 10.65% in average DSC and a reduction of 5.23 mm in average HD on the abdominal CT-MRI dataset, as well as 19.36% higher average DSC and 3.39 mm lower average HD on the cross-center prostate dataset. Ablation studies confirmed the effectiveness of each component of the proposed RandDGmethod. The proposed RandDG method effectively addresses the critical challenges of texture bias and limited style diversity in single-source DG for medical image segmentation through frequency-aware dual-space randomization framework. This method shows promise for practical deployment in clinical settings, where target domain data may not always beavailable.

Over the years, multimodal mobile sensing has been used extensively for inferences regarding health and well-being, behavior, and context. However, a significant challenge hindering the widespread deployment of such models in real-world scenarios is the issue of distribution shift. This is the phenomenon where the distribution of data in the training set differs from the distribution of data in the real world---the deployment environment. While extensively explored in computer vision and natural language processing, and while prior research in mobile sensing briefly addresses this concern, current work primarily focuses on models dealing with a single modality of data, such as audio or accelerometer readings, and consequently, there is little research on unsupervised domain adaptation when dealing with multimodal sensor data. To address this gap, we did extensive experiments with domain adversarial neural networks (DANN) showing that they can effectively handle distribution shifts in multimodal sensor data. Moreover, we proposed a novel improvement over DANN, called M3BAT, unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training, to account for the multimodality of sensor data during domain adaptation with multiple branches. Through extensive experiments conducted on two multimodal mobile sensing datasets, three inference tasks, and 14 source-target domain pairs, including both regression and classification, we demonstrate that our approach performs effectively on unseen domains. Compared to directly deploying a model trained in the source domain to the target domain, the model shows performance increases up to 12% AUC (area under the receiver operating characteristics curves) on classification tasks, and up to 0.13 MAE (mean absolute error) on regression tasks.

Center-aligned domain adaptation network for image classification

A key challenge with supervised learning (e.g., image classification) is the shift of data distribution and domain from training to testing datasets, so-called “domain shift” (or “distribution shift”), which usually leads to a reduction of model accuracy. Various meta-learning approaches have been proposed to prevent the accuracy loss by learning an adaptable model with training data, and adapting it to test time data from a new data domain. However, when the domain shift occurs in multiple domain dimensions (e.g., images may be transformed by rotations, transitions, and expansions), the average predictive power of the adapted model will deteriorate. To tackle this problem, we propose a domain disentangled meta-learning (DDML) framework. DDML disentangles the data domain by dimensions, learns the representations of domain dimensions independently, and adapts to the domain of test time data. We evaluate our DDML on image classification problems using three datasets with distribution shifts over multiple domain dimensions. Comparing to various baselines in meta-learning and empirical risk minimization, our DDML approach achieves consistently higher classification accuracy with the test time data. These results demonstrate that domain disentanglement reduces the complexity of the model adaptation, thus increases the model generalizability, and prevents it from overfitting.

Named Entity Recognition (NER) is a fundamental task in information extraction from unstructured text. Most previous machine-learning-based NER systems are domain-specific, which implies that they may only perform well on some specific domains (e.g., Newswire ) but tend to adapt poorly to other related but different domains (e.g., Weblog ). Recently, transfer learning techniques have been proposed to NER. However, most transfer learning approaches to NER are developed for binary classification, while NER is a multiclass classification problem in nature. Therefore, one has to first reduce the NER task to multiple binary classification tasks and solve them independently. In this article, we propose a new transfer learning method, named Transfer Joint Embedding (TJE), for cross-domain multiclass classification, which can fully exploit the relationships between classes (labels), and reduce domain difference in data distributions for transfer learning. More specifically, we aim to embed both labels (outputs) and high-dimensional features (inputs) from different domains (e.g., a source domain and a target domain) into a unified low-dimensional latent space, where 1) each label is represented by a prototype and the intrinsic relationships between labels can be measured by Euclidean distance; 2) the distance in data distributions between the source and target domains can be reduced; 3) the source domain labeled data are closer to their corresponding label-prototypes than others. After the latent space is learned, classification on the target domain data can be done with the simple nearest neighbor rule in the latent space. Furthermore, in order to scale up TJE, we propose an efficient algorithm based on stochastic gradient descent (SGD). Finally, we apply the proposed TJE method for NER across different domains on the ACE 2005 dataset, which is a benchmark in Natural Language Processing (NLP). Experimental results demonstrate the effectiveness of TJE and show that TJE can outperform state-of-the-art transfer learning approaches to NER.

In recent years, as cybersecurity threats have become increasingly severe and cyberattacks have occurred frequently, higher requirements have been put forward for cybersecurity protection. Therefore, the Named Entity Recognition (NER) technique, which is the cornerstone of Cyber Threat Intelligence (CTI) analysis, is particularly important. However, most existing NER studies are limited to recognizing single-layer flat entities, ignoring the possible nested entities in CTI. On the other hand, most of the existing studies focus on English CTIs, and the existing models performed poorly in a limited number of Chinese CTI studies. Given the above challenges, we propose in this paper a novel unified model, RBTG, which aims to identify flat and nested entities in Chinese CTI effectively. To overcome the difficult boundary recognition problem and the direction-dependent and distance-dependent properties in Chinese CTI NER, we use Global Pointer as the decoder and TENER as the encoder layer, respectively. Specifically, the Global Pointer layer solves the problem of the insensitivity of general NER methods to entity boundaries by utilizing the relative position information and the multiplicative attention mechanism. The TENER layer adapts to the Chinese CTI NER task by introducing an attention mechanism with direction awareness and distance awareness. Meanwhile, to cope with the complex feature capture of hierarchical structure and dependencies among Chinese CTI nested entities, the TENER layer solves the problem by following the structure of multiple self-attention layers and feed-forward network layers superimposed on each other in the Transformer. In addition, to fill the gap in the Chinese CTI nested entity dataset, we further apply the Large Language Modeling (LLM) technique and domain knowledge to construct a high-quality Chinese CTI nested entity dataset, CDTinee, which consists of six entity types selected from STIX, including nearly 4000 entity types extracted from more than 3000 threatening sentences. In the experimental session, we conduct extensive experiments on multiple datasets, and the results show that the proposed model RBTG outperforms the baseline model in both flat NER and nested NER.

Current work in named entity recognition (NER) uses either cross entropy (CE) or conditional random fields (CRF) as the objective/loss functions to optimize the underlying NER model. Both of these traditional objective functions for the NER problem generally produce adequate performance when the data distribution is balanced and there are sufficient annotated training examples. But since NER is inherently an imbalanced tagging problem, the model performance under the low-resource settings could suffer using these standard objective functions. Based on recent advances in area under the ROC curve (AUC) maximization, we propose to optimize the NER model by maximizing the AUC score. We give evidence that by simply combining two binary-classifiers that maximize the AUC score, significant performance improvement over traditional loss functions is achieved under low-resource NER settings. We also conduct extensive experiments to demonstrate the advantages of our method under the low-resource and highly-imbalanced data distribution settings. To the best of our knowledge, this is the first work that brings AUC maximization to the NER setting. Furthermore, we show that our method is agnostic to different types of NER embeddings, models and domains. The code of this work is available at https://github.com/dngu0061/NER-AUC-2T.

Generalising deep models to new data from new centres (termed here domains) remains a challenge. This is largely attributed to shifts in data statistics (domain shifts) between source and unseen domains. Recently, gradient-based meta-learning approaches where the training data are split into meta-train and meta-test sets to simulate and handle the domain shifts during training have shown improved generalisation performance. However, the current fully supervised meta-learning approaches are not scalable for medical image segmentation, where large effort is required to create pixel-wise annotations. Meanwhile, in a low data regime, the simulated domain shifts may not approximate the true domain shifts well across source and unseen domains. To address this problem, we propose a novel semi-supervised meta-learning framework with disentanglement. We explicitly model the representations related to domain shifts. Disentangling the representations and combining them to reconstruct the input image allows unlabeled data to be used to better approximate the true domain shifts for meta-learning. Hence, the model can achieve better generalisation performance, especially when there is a limited amount of labeled data. Experiments show that the proposed method is robust on different segmentation tasks and achieves state-of-the-art generalisation performance on two public benchmarks.

The use of Convolutional Neural Networks (CNNs) to solve Domain Adaptation (DA) image classification problems in the context of remote sensing has proven to provide good results but at high computational cost. To avoid this problem, a deep learning network for DA in remote sensing hyperspectral images called TCANet is proposed. As a standard CNN, TCANet consists of several stages built based on convolutional filters that operate on patches of the hyperspectral image. Unlike the former, the coefficients of the filter are obtained through Transfer Component Analysis (TCA). This approach has two advantages: firstly, TCANet does not require training based on backpropagation, since TCA is itself a learning method that obtains the filter coefficients directly from the input data. Second, DA is performed on the fly since TCA, in addition to performing dimensional reduction, obtains components that minimize the difference in distributions of data in the different domains corresponding to the source and target images. To build an operating scheme, TCANet includes an initial stage that exploits the spatial information by providing patches around each sample as input data to the network. An output stage performing feature extraction that introduces sufficient invariance and robustness in the final features is also included. Since TCA is sensitive to normalization, to reduce the difference between source and target domains, a previous unsupervised domain shift minimization algorithm consisting of applying conditional correlation alignment (CCA) is conditionally applied. The results of a classification scheme based on CCA and TCANet show that the DA technique proposed outperforms other more complex DA techniques.

Remote Photoplethysmography (rPPG) is a non-contact method that uses facial video to predict changes in blood volume, enabling physiological metrics measurement. Traditional rPPG models often struggle with poor generalization capacity in unseen domains. Current solutions to this problem is to improve its generalization in the target domain through Domain Generalization (DG) or Domain Adaptation (DA) techniques. However, traditional DA methods usually require access to both source domain data and target domain data, which cannot be implemented in scenarios with limited access to source data due to the privacy issue. In this paper, we propose the first Source-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which overcomes these limitations by enabling effective domain adaptation without access to source domain data. Our framework incorporates a Three-Branch Spatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency across domains. Furthermore, we propose a new rPPG distribution alignment loss based on the Frequency-domain Wasserstein Distance (FWD), which leverages optimal transport to align power spectrum distributions across domains effectively and further enforces the alignment of the three branches. Extensive cross-domain experiments and ablation studies demonstrate the effectiveness of our proposed method in source-free domain adaptation settings. Our findings highlight the significant contribution of the proposed FWD loss for distributional alignment, providing a valuable reference for future research and applications. The codes are available at https://github.com/XieYiping66/SFDA-rPPG.

Ensuring that deep learning models generalize across distinct regimes remains a fundamental challenge in Earth system modeling. Due to the inherent violation of the independent and identically distributed (i.i.d.) assumption, models optimized for local conditions rarely exhibit robust performance on unseen domains. While Unsupervised Domain Adaptation (UDA) is a well-established technique for mitigating such distribution shifts in computer vision, its application to Earth system modeling remains underexplored. In this study we investigate the efficacy of UDA for the super-resolution of atmospheric fields, utilizing kilometer-scale COSMO simulations [1] and the RainShift benchmark dataset [2] to quantify model robustness across different regions. We apply residual learning to jointly super-resolve precipitation and surface pressure, incorporating static predictors such as topography. To quantify transferability, we propose a systematic framework that trains on source domains and evaluates on unseen target domains, treating spatial transfer as a proxy for model robustness under distribution shifts. We introduce a consistency metric to evaluate model adaptation by comparing mean performance on seen versus unseen domains. We assess a hierarchy of adaptation methods, ranging from simple regularization to physics-informed approaches. These include domain-specific regularization and distribution alignment methods, domain adversarial training, and geometry-robust training via group-equivariant convolutions. Preliminary results on the COSMO simulations demonstrate that even elementary adaptation strategies, such as dropout and data augmentation, improve cross-domain consistency. This work establishes a controlled setup for benchmarking generalization, suggesting that UDA offers a viable pathway to bridge the gap between locally trained models and global applicability.[1]: Cui, R., Thurnherr, I., Velasquez, P., Brennan, K. P., Leclair, M., Mazzoleni, A., et al. (2025). A European hail and lightning climatology from an 11-year kilometer-scale regional climate simulation. Journal of Geophysical Research: Atmospheres, 130, e2024JD042828. https://doi.org/10.1029/2024JD042828[2]: Paula Harder et al. RainShift: A Benchmark for Precipitation Downscaling Across Geographies. 2025. arXiv: 2507.04930 [cs.CV]. url: https://arxiv.org/abs/2507.04930.

Abstract. Geographic variability of the classes of interest, differences in sensor characteristics and changes in atmospheric conditions during image acquisition, among other factors, make it challenging to use a pre-trained deep learning classifier on new remote sensing data without a substantial drop in classification accuracy. This phenomenon occurs due to the so-called domain shift problem. Deep domain adaptation techniques have been used to mitigate the problem and thus avoid the time-consuming and costly collection of new labeled samples. Most recent domain adaptation approaches rely on single-source and single-target domains, refraining from exploiting other data distributions that are usually available. This work introduces a new unsupervised multi-target domain adaptation in the context of a change detection application, namely deforestation detection. The proposed approach addresses the substantial class imbalance typical of such application by applying unsupervised algorithms for selecting pseudo-labels in the target domain that will later serve as additional training references. We report results of experiments to evaluate the proposed method in four distinct sites of two Brazilian biomes using Sentinel-2 images. The results indicate that the proposed unsupervised domain adaptation method is a promising solution to reduce the effects of domain shift and to deal with the scarcity of labeled training data.

There is a large amount of out-of-distribution data (OOD) in remote sensing, which hinders high-accuracy segmentation models under the assumption of independent identical distribution (i.i.d.) from stable and reliable performance in real-world remote sensing applications. And Domain Adaptation (DA) is presented to seamlessly extend classifiers to the label-scarce target domain in the presence of the label-sufficient source domain with different data distributions. However, given that the domain shift, i.e. the distribution difference between the two domains, is more serious in remote sensing images, the current DA methods for image segmentation in Computer Vision (CV) typically perform unsatisfactorily in remote sensing, even suffering from the negative domain alignment. To this end, this paper proposes the Self-Training Adversarial Domain Adaptation (STADA) method for remote sensing image segmentation, which not only performs adversarial learning to extract domain-invariant features, but also implements Self-Training using pseudo-labels in the target domain denoised by the conditional adversarial loss for classifier adaptation. The ISPRS and WHU datasets are employed to conduct extensive experiments to investigate the effectiveness of STADA and the specific effect of its each DA component. And the experimental results demonstrate that STADA outperforms other state-of-the-art DA methods in the remote sensing image segmentation task.

While pre-trained models in the general domain have proliferated, existing methods for transferring these models to specific domains often depend on source domain data for distribution alignment and are typically tailored for single tasks. We propose a source data-free approach, Fourier Self-Adaptation (FSA), which effectively adapts general models to a wide range of specific domains. Our method leverages the distinct properties of Fourier phase and amplitude: phase contains high-level structural and positional information, which is less affected by domain shifts, while amplitude contains details and brightness information, which is more affected by domain shifts. FSA adjusts the image distribution by initializing a trainable adaptive image from a normal distribution. It then interpolates the amplitude of the target domain image with that of the adaptive image, where the interpolation ratio is dynamically controlled by learnable weight and bias. During training, the model captures advanced phase information of the target image and refines the data distribution through amplitude interpolation. Additionally, a dual regularization loss constrains the model representation, encouraging it to focus on the intrinsic relationships of the target domain data while discarding irrelevant knowledge. We evaluate FSA using general pre-trained models on 11 unimodal image classification datasets and 6 multimodal visual question answering datasets, covering specific domains such as radiology, pathology, remote sensing, and art. Our method consistently achieves state-of-the-art performance across multiple datasets, with performance improvements ranging from 1% to 8% compared to basic pre-trained models. Source code are available at https://github.com/Alivelei/FSA.

Named Entity Recognition (NER) is a supervised machine learning task that finds various applications in automated content analysis, as the identification of entities is vital for understanding public discourse. However, sometimes the standard NER labels are not specific enough for a given domain. We introduce Public Entity Recognition (PER). PER is a domain-specific version of NER, that is trained for five entity types that are common to public discourse: politicians, parties, authorities, media, and journalists. PER can be used for pre-processing documents, in a pipeline with other classifiers or directly for analyzing information in texts. The taxonomy for PER is taken from the database of (German) public speakers and aims at low-threshold integration into computational social science research. We experiment with different training settings, involving weakly supervised training and training on manually annotated data. We evaluate multilingual transformer models of different sizes against rule-based entity matching and find that the models do not only outperform the baseline but also reach competitive absolute scores of around .8 and higher in F1. We further test for generalization and domain adaptation. We show that with only around 100–150 additional sentences, the model can be adapted to new languages.