Non-parallel text style transfer with domain adaptation and an attention model

We study the problem of extracting cross-lingual topics from non-parallel multilingual text datasets with partially overlapping thematic content (e.g., aligned Wikipedia articles in two different languages). To this end, we develop a new bilingual probabilistic topic model called comparable bilingual latent Dirichlet allocation (C-BiLDA), which is able to deal with such comparable data, and, unlike the standard bilingual LDA model (BiLDA), does not assume the availability of document pairs with identical topic distributions. We present a full overview of C-BiLDA, and show its utility in the task of cross-lingual knowledge transfer for multi-class document classification on two benchmarking datasets for three language pairs. The proposed model outperforms the baseline LDA model, as well as the standard BiLDA model and two standard low-rank approximation methods (CL-LSI and CL-KCCA) used in previous work on this task.

Domain adaptation techniques such as importance weighting modify the training data to better represent a different test data distribution, a process that may be particularly vulnerable to a malicious attack in an adversarial machine learning scenario. In this work, we explore the level of such vulnerability of importance weighting to poisoning attacks. Importance weighting, like other domain adaptation approaches, assumes that the distributions of training and test data are different but related. An intelligent adversary, having full or partial access to the training data, can take advantage of the expected difference between the distributions, and can inject well crafted malicious samples into the training data, resulting in an incorrect estimation of the importance ratio. In this work, we demonstrate the vulnerability of one of the simplest yet most effective approaches for directly estimating the importance ratio, namely, modifying the training distribution using a discriminative classifier such as the logistic regression. We test the robustness of the importance weighting process using well-controlled synthetic datasets, with an increasing number of attack points in the training data. Under the worst case perfect knowledge scenario, where the attacker has full access to the training data, we demonstrate that importance weighting can be dramatically compromised with the insertion of even a single attack point. We then show that even under limited knowledge scenario, where the attacker has limited access to the training data, the estimation process can still be significantly compromised.

Text style transfer is an important task to render artistic texts from a reference image or style, and is widely desired in many visual creations. Previous works have brought some efficient methods for text style transfer, which facilitate users to design various artistic texts automatically. However, these works mainly focus on relatively simple text effects, and do not perform well on complex reference styles. In this paper, we propose a coarse-to-fine framework to generate exquisite texts with complex texture and structure in an unsupervised way, achieving real-time control of style scales (i.e., text stylistic degree or deformation degree). The key idea is to decouple the overall task into two steps, prototype generation and detail refinement, and explore delicate networks for each step to imitate the features at different levels. Based on this idea, in the first step, we present a novel pro-gen GAN to generate prototypes of artistic texts using the reference style, and develop a deformable module to empower the pro-gen GAN to continuously characterize the multi-scale shape features without network retraining. Furthermore, we propose a mix-attention training scheme for text style transfer, which can avoid artifacts and retain a clear text background. In the second step, we introduce two optimized networks for detail refinements. Experimental results show that the proposed method can synthesize exquisite stylized texts with complex reference styles, and surpass the state of the arts in texture reconstruction, contour imitation, and text image quality drastically. The code is available on the page: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/WendongMao/Intelligent_Typography</uri> .

Text style transfer (TST) modifies a source sentence to match a target style while preserving its semantics.While existing models perform well on simple styles like sentiment and formality, they struggle with complex, entangled styles such as poetry and brand-specific tones, which require advanced operations to disentangle content and style.We propose a multi-agent self-check framework that contains a large language model (LLM) as a planner for disentangling subtasks and expert agents for executing the subtasks.This training-free multi-agent framework decomposes TST into manageable components, enabling iterative refinement through a self-check module that balances style adherence and content preservation.Experiments on both simple and complex style datasets show our framework significantly improves style strength and content preservation, with strong adaptability in few-shot settings.Yelp-

We introduce a simple method for text style transfer that frames style transfer as denoising: we synthesize a noisy corpus and treat the source style as a noisy version of the target style. To control for aspects such as preserving meaning while modifying style, we propose a reranking approach in the data synthesis phase. We evaluate our method on three novel style transfer tasks: transferring between British and American varieties, text genres (formal vs. casual), and lyrics from different musical genres. By measuring style transfer quality, meaning preservation, and the fluency of generated outputs, we demonstrate that our method is able both to produce high-quality output while maintaining the flexibility to suggest syntactically rich stylistic edits.

Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of non-parallel data to guide transferring different text styles. (ii) colossal performance degradation when fine-tuning the model in new domains. In this work, we propose DAML-ATM (Domain Adaptive Meta-Learning with Adversarial Transfer Model), which consists of two parts: DAML and ATM. DAML is a domain adaptive meta-learning approach to learning general knowledge in multiple heterogeneous source domains, capable of adapting to new unseen domains with a small amount of data. Moreover, we propose a new unsupervised TST approach Adversarial Transfer Model (ATM), composed of a sequence-to-sequence pre-trained language model and uses adversarial style training for better content preservation and style transfer. Results on multi-domain datasets demonstrate that our approach generalizes well on unseen low-resource domains, achieving state-of-the-art results against ten strong baselines.

BackgroundCognitive impairment and dementia pose a significant challenge to the aging population, impacting the well-being, quality of life, and autonomy of affected individuals. As the population ages, this will place enormous strain on health care and economic systems. While computerized cognitive training programs have demonstrated some promise in addressing cognitive decline, adherence to these interventions can be challenging.ObjectiveThe objective of this study is to improve the accuracy of predicting adherence lapses to ultimately develop tailored adherence support systems to promote engagement with cognitive training among older adults.MethodsData from 2 previously conducted cognitive training intervention studies were used to forecast adherence levels among older participants. Deep convolutional neural networks were used to leverage their feature learning capabilities and predict adherence patterns based on past behavior. Domain adaptation (DA) was used to address the challenge of limited training data for each participant, by using data from other participants with similar playing patterns. Time series data were converted into image format using Gramian angular fields, to facilitate clustering of participants during DA. To the best of our knowledge, this is the first effort to use DA techniques to predict older adults’ daily adherence to cognitive training programs.ResultsOur results demonstrated the promise and potential of deep neural networks and DA for predicting adherence lapses. In all 3 studies, using 2 independent datasets, DA consistently produced the best accuracy values.ConclusionsOur findings highlight that deep learning and DA techniques can aid in the development of adherence support systems for computerized cognitive training, as well as for other interventions aimed at improving health, cognition, and well-being. These techniques can improve engagement and maximize the benefits of such interventions, ultimately enhancing the quality of life of individuals at risk for cognitive impairments. This research informs the development of more effective interventions, benefiting individuals and society by improving conditions associated with aging.

Wearable sensors provide enormous opportunities to identify activities and events of interest for various applications. However, a major limitation of the current systems is the fact that machine learning algorithms trained on particular sensors need to be retrained upon any changes in configuration of the system, such as adding a new sensor. In this paper, we aim to seamlessly train machine learning algorithms for the new sensors to identify activities and observations that are detectable by the pre-existing sensors. We create a domain adaptation method to expand training algorithms from known wearable sensors to new sensors, eliminating the need for manual training of machine learning algorithms. Specifically, our proposed approach eliminates the need for capturing substantial amount of data on new sensors. We propose the concept of stochastic features for human activity recognition, and design a new architecture of deep neural network to approximate the posterior distribution of the features. This approximation aligns the feature space of the new and old sensors by using limited, unlabeled data from the new sensor so that the previously defined classifier can be used with the new sensor. The experimental results show that (i) stochastic features are more robust against additive noise compared to typical convolutional neural networks based on deterministic features (ii) our framework outperforms the state-of-the-art domain adaptation algorithms. It can also achieve 10% improvement when training new sensors with limited unlabeled training data compared to training a model from scratch for the new sensor.

Recent advances on road damage detection relies on a large amount of labeled data, whilst collecting pavement image is labor-intensive and time-consuming. Unsupervised Domain Adaptation (UDA) provides a promising solution to adapt a source domain to the target domain, however, cross-domain crack detection is still an open problem. In this paper, we propose domain adaptive road damage detection termed as DA-RDD, by incorporating image-level with instance-level feature alignment for domain-invariant representation learning in an adversarial manner. Specifically, importance weighting is introduced to evaluate the intermediate samples for image-level alignment between domains, and we aggregate RoI-wise feature with multi-scale contextual information to recover the crack details for progressive domain alignment at instance level. Additionally, a large-scale road damage dataset (based on Road Damage Dataset 2020 (RDD2020)) named as RDD2021 is constructed with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$100k$ </tex-math></inline-formula> synthetic labeled distress images. Extensive experimental results on damage detection across different countries demonstrate the universality and superiority of DA-RDD, and empirical studies on RDD2021 further claim its effectiveness and advancement. To our best knowledge, it is the first time to investigate domain adaptative pavement crack detection, and we expect the contributions in this work would facilitate the development of generalized road damage detection in the future.

Currently, most text style transfer methods encode the text into a style-independent latent representation and decode it into new sentences with the target style. Due to the limitation of the latent representation, previous works can hardly get satisfactory target style sentence especially in terms of semantic remaining of the original sentence. We propose a “Mask and Generation” structure, which can obtain an explicit representation of the content of original sentence and generate the target sentence with a transformer. This explicit representation is a masked text that masks the words with the strong style attribute in the sentence. Therefore, it can preserve most of the semantic meaning of the original sentence. In addition, as it is the input of the generator, it also simplified this process compared to the current work who generate the target sentence from scratch. As the explicit representation is readable and the model has better interpretability, we can clearly know which words changed and why the words changed. We evaluate our model on two review datasets with quantitative, qualitative, and human evaluations. The experimental results show that our model generally outperform other methods in terms of transfer accuracy and content preservation.

Natural language generation (NLG) spans a broad range of tasks, each of which serves for specific objectives and desires different properties of generated text. The complexity makes automatic evaluation of NLG particularly challenging. Previous work has typically focused on a single task and developed individual evaluation metrics based on specific intuitions. In this paper, we propose a unifying perspective based on the nature of information change in NLG tasks, including compression (e.g., summarization), transduction (e.g., text rewriting), and creation (e.g., dialog). Information alignment between input, context, and output text plays a common central role in characterizing the generation. With automatic alignment prediction models, we develop a family of interpretable metrics that are suitable for evaluating key aspects of different NLG tasks, often without need of gold reference data. Experiments show the uniformly designed metrics achieve stronger or comparable correlations with human judgement compared to state-of-the-art metrics in each of diverse tasks, including text summarization, style transfer, and knowledge-grounded dialog.

This study utilizes domain adaptation to enhance the integration of diverse geoscience datasets, aiming to improve the identification of ore bodies. Traditional mineral exploration methods often face challenges in merging different geoscience data types, which leads to models that do not perform well across varying domains. Domain adaptation is a deep learning strategy aimed at adapting a model developed in one domain (source) to perform well in a different domain (target). To adapt models trained on detailed, labeled drilling data (source) to interpret broader, unlabeled geophysical data (target), Domain-Adversarial Neural Networks (DANNs) were applied, chosen for their robust performance in scenarios where the target domain does not provide labels. This approach was indirectly validated through the minimal overlap between regions identified as candidate ore and borehole locations marked as host rocks, with qualitative validation provided by t-Distributed Stochastic Neighbor Embedding (t-SNE) visualizations showing improved data integration across domains.

Advancing instance segmentation and WBC classification in peripheral blood smear through domain adaptation: A study on PBC and the novel RV-PBS datasets

This paper addresses the issue of 2D human upper-body pose estimation under cluttered environments using a discriminative structured framework. Most previous approaches focus on solving such a problem using generative models. However, a generative model has two drawbacks: a) not suitable for real-time application due to its slow inference algorithm and b) prone to over fitting given limited training data. In this work, we propose to use structured discriminative regression models for 2D human upper-body pose estimation in a model-free manner to overcome the aforementioned drawbacks. In contrast to a standard discriminative regression model, a structured regression model for human pose estimation can not only learn the relevance between image features and the presentation of human pose but also catch the inner relationship between each output. Our experimental results demonstrate the benefits brought by using structured discriminative models to articulated human pose estimation problem on cluttered images from the benchmarking Buffy the Vampire Slayer dataset and the highly challenging images from PASCAL VOC 2007 and 2008 Challenge datasets.

We propose a novel model that apply an extension of the Long Short-Term Memory neural network for sentence compression task. In our model, only the most relevant context of each word is concentrated to avoid the redundant information. Our model is based on two new models that have been successfully used recently in neural machine translation. The first is Bidirectional model that can be trained using all the available input information in the past and future. The second is Attention model that focus not only the whole sentence information but also the particular context of each word in this sentence. Experimental results show that our model significantly outperforms all the recently state-of-the-art method, the Bidirectional and the Attention model on the Google sentence compression dataset.