A two-step abstractive summarization model with asynchronous and enriched-information decoding

Endowing dialogue systems with personas is essential to deliver more human-like conversations. However, this problem is still far from well explored due to the difficulties of both embodying personalities in natural languages and the persona sparsity issue observed in most dialogue corpora. This paper proposes a pre-training based personalized dialogue model that can generate coherent responses using persona-sparse dialogue data. In this method, a pre-trained language model is used to initialize an encoder and decoder, and personal attribute embeddings are devised to model richer dialogue contexts by encoding speakers' personas together with dialogue histories. Further, to incorporate the target persona in the decoding process and to balance its contribution, an attention routing structure is devised in the decoder to merge features extracted from the target persona and dialogue contexts using dynamically predicted weights. Our model can utilize persona-sparse dialogues in a unified manner during the training process, and can also control the amount of persona-related features to exhibit during the inference process. Both automatic and manual evaluation demonstrates that the proposed model outperforms state-of-the-art methods for generating more coherent and persona consistent responses with persona-sparse data.

A hierarchical and parallel framework for End-to-End Aspect-based Sentiment Analysis

Abstractive summarization is useful in providing a summary or a digest of news or other web texts and enhancing users reading experience, especially when they are reading on small displays such as mobile phones. However, existing encoder-decoder summarization models have difficulty learning the latent alignment between source documents and summaries because of their vast disparity in length. In this paper, we propose a extractor-abstractor framework in which the keyword-based extractor selects a few sets of salient sentences from the input document and then the abstractor paraphrases these sets of sentences in parallel, which are more aligned to the summary, to generate the final summary. The new extractor and abstractor are pretrained from a set of “pseudo summaries” extracted by specially designed heuristics, and then further trained together in a reinforcement learning framework. The results show that the proposed model generates high-quality summaries with faster training speed and less training memory footprint, and outperforms the state-of-the-art models on CNN/Daily Mail, Webis-TLDR-17, Webis-Snippet-20, WikiHow and DUC-2002 datasets.

LMcast: A pretrained language model guided long-term memory transformer for precipitation nowcasting.

Purpose: This study addresses the limitations of current short abstracts of DBPEDIA entities, which often lack a comprehensive overview due to their creating method (i.e., selecting the first two-three sentences from the full DBPEDIA abstracts). Methodology: We leverage pre-trained language models to generate abstractive summaries of DBPEDIA abstracts in six languages (English, French, German, Italian, Spanish, and Dutch). We performed several experiments to assess the quality of generated summaries by language models. In particular, we evaluated the generated summaries using human judgments and automated metrics (Self-ROUGE and BERTScore). Additionally, we studied the correlation between human judgments and automated metrics in evaluating the generated summaries under different aspects: informativeness, coherence, conciseness, and fluency. Findings: Pre-trained language models generate summaries more concise and informative than existing short abstracts. Specifically, BART-based models effectively overcome the limitations of DBPEDIA short abstracts, especially for longer ones. Moreover, we show that BERTScore and ROUGE-1 are reliable metrics for assessing the informativeness and coherence of the generated summaries with respect to the full DBPEDIA abstracts. We also find a negative correlation between conciseness and human ratings. Furthermore, fluency evaluation remains challenging without human judgment. Value: This study has significant implications for various applications in machine learning and natural language processing that rely on DBPEDIA resources. By providing succinct and comprehensive summaries, our approach enhances the quality of DBPEDIA abstracts and contributes to the semantic web community.

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain-adaptive pre-trained large language model

The aim of this survey was to test the applicability of the Nursing Interventions Classification (NIC) system for use in a future nursing information system for documenting nursing in an electronic patient record in Iceland. Also, the aim was to test the translation of NIC into Icelandic. In order to be applicable to nursing NIC needs to be sensitive enough to describe the work nurses do, differentiate between specialities in nursing, and be understandable to nurses. A sample of 198 nurses was asked to identify how often they used each of 433 NIC nursing interventions. Of the 36 most frequently used interventions half are within the physiological domain. Core nursing interventions were different between specialities, e.g. Analgesic Administration had a high mean score in surgical nursing, and Health Education in primary health care. anova for the 27 classes in NIC showed significant differences (p < 0.01) by all nursing specialities except one, Crisis Management. A Tukey post hoc test showed how nursing specialities were reflected differently in the NIC domains, e.g. medical/surgical nursing in the Physiological: Basic Domain, but psychiatric nursing in the Behavioural Domain. Factor analysis of classes in NIC show good resemblance with the domains in NIC and the structure of the classification is strongly supported, except the Safety Domain. The results from this study indicate that nurses in the sample consider NIC to be applicable to describe nursing. The language is a powerful tool and is central in reflecting nursing practice as well as supporting the construct of knowledge. The translation of NIC to Icelandic is one step in many in preparing nurses to use a standardized language which can also be used in an electronic patient record.

Leveraging natural language processing to identify anxiety states from social media has been widely studied. However, existing research lacks deep user-level semantic modeling and effective anxiety feature extraction. Additionally, the absence of clinical domain knowledge in current models limits their interpretability and medical relevance. Therefore, in this paper we propose a framework for anxiety recognition that combines pre-trained language models and clinical anxiety scales. The framework first leverages pre-trained language models to extract semantic representations of social media posts and anxiety scales, respectively. Second, the anxiety score of each post is calculated by the semantic similarity between the post and the anxiety scale, and the posts with high anxiety scores are selected. Finally, a joint training strategy consisting of sequence classification (corresponding to anxiety scores) and text classification (corresponding to high anxiety score posts) is developed for anxiety recognition. To evaluate the effectiveness of the proposed framework, we construct a Dataset for Anxiety User Recognition (DAUR) which contains 3,170 users with 936,400 posts on Sina Weibo platform. In addition, to explore the capability of our method in the scenario of early anxiety recognition, we remove the posts with obvious anxiety keywords and all posts following them from the anxiety users to obtain a new dataset DAUR_PRE. Comparative experiments demonstrate that the proposed anxiety recognition framework outperforms existing methods on both datasets and achieves high classification accuracy on DAUR_PRE.

Interpretable deep learning model for building energy consumption prediction based on attention mechanism

Face recognition has become a fascinating field for researchers. The motivation behind the enormous interest in the topic is the need to improve the accuracy of many real-time applications. The complexity of the human face and the changes due to different effects make it more challenging to design as well as implement a powerful computational system for human face recognition. In this work, we presented an enhanced approach to improve human face recognition using a Back-Propagation Neural Network (BPNN) and features extraction based on the correlation between the training images. A key contribution of this work is the generation of a new set called the T-Dataset from the original training dataset, which is used to train the BPNN. We generated the T-Dataset utilizing the correlation between the training images without using a conventional technique of image density. The correlated T-Dataset provides a high distinction layer between the training images, which helps the BPNN to converge faster and achieve better accuracy. Data and features reduction is essential in the face recognition process, and researchers have recently focused on the modern Neural Network (NN). Therefore, we used a Principal Component Analysis (PCA) descriptor to prove that there is a potential improvement even using traditional methods. We applied five distance measurement algorithms and then combined them to obtain the T-Dataset, which we fed into the BPNN. We achieved higher face recognition accuracy with less computational cost compared to the current approach by using reduced image features. We test the proposed framework on two small datasets, the YALE and AT&T datasets, as the ground truth. We achieved tremendous accuracy. Furthermore, we evaluate our method on one of the state-of-the-art benchmark datasets, Labeled Faces in the Wild (LFW), where we produce a competitive face recognition performance. Also, we proposed an enhanced framework to improve the face registration using deep transfer learning. We used pre-trained models such as ResNet50 and SeNet50 which trained on the VGGFace2 dataset. We used the LFW dataset to evaluate our method, and we achieve high accuracy.

This research introduces a multi-view consistent 3D modeling method CoGSD Cosistent Gaussian Splatting Dreamer , this is construct 3D models rapidly with multi-view consistency through 3D Gaussian Splatting. 3D Gaussian Splatting generation fails to construct effective 3D assets due to the lack of stable ground truth. In addition, the expansion characteristics of 3D Gaussian Splatting itself lead to abnormal expansion of saturated Gaussian points and multi-view inconsistency problems. At the same time, the lack of credible ground truth will also lead to multi-view inconsistency problems. To solve this problem, we use a pre-trained consistent diffusion model to generate consistent viewpoints. In our framework, instead of generating diffusion with a single a priori perspective, the 2D image generation method of SDS uses a controlnet-tuned pre-trained model to generate 2D images with coherent viewpoints, resulting in high-quality 3D model generation. The method in this paper provides an effective solution for 3D modeling and is expected to be widely used in the field of 3D modeling and visual effects.

Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects in videos. Most methods obtain identities by associating detection boxes whose scores are higher than a threshold. The objects with low detection scores, e.g. occluded objects, are simply thrown away, which brings non-negligible true object missing and fragmented trajectories. To solve this problem, we present a simple, effective and generic association method, tracking by associating almost every detection box instead of only the high score ones. For the low score detection boxes, we utilize their similarities with tracklets to recover true objects and filter out the background detections. When applied to 9 different state-of-the-art trackers, our method achieves consistent improvement on IDF1 score ranging from 1 to 10 points. To put forwards the state-of-the-art performance of MOT, we design a simple and strong tracker, named ByteTrack. For the first time, we achieve 80.3 MOTA, 77.3 IDF1 and 63.1 HOTA on the test set of MOT17 with 30 FPS running speed on a single V100 GPU. ByteTrack also achieves state-of-the-art performance on MOT20, HiEve and BDD100K tracking benchmarks. The source code, pre-trained models with deploy versions and tutorials of applying to other trackers are released at https://github.com/ifzhang/ByteTrack.KeywordsMulti-object trackingData associationDetection boxes

Aiming at the situation that the existing Seq2Seq model is prone to deviate from the original meaning, lose contextual semantic information, and fabricate facts when generating news Summarization. This paper, based on the improvement of pointer generation network model, proposes a hybrid pointer generation network model based on BART pre-training language model. Firstly, the BART pre-trained language model based on denoising encoder is used to extract deep semantic features, and these semantic features are fused with the original news text and input into the pointer generation network model with coverage mechanism. The Bi-LSTM model is used to encode, and then the LSTM model with attention mechanism is used to decode the semantic vector to get the summarization content. In the decoding process, pointer mechanism solves the problem of out-of-vocabulary (OOV), coverage mechanism alleviates the problem of generating repeated words, and Beam search is used to improve the accuracy of summarization generation. The experimental results on NLPCC2017 data set show that the BART-PGN summarization model proposed in this paper obtains a higher Rouge score, which confirming its effectiveness in the field of Chinese news summarization.

Strategies for efficiently and accurately extracting relationships from power system administrative shift texts are explored using an optimized pre-trained language model, such as BERT (Bidirectional Encoder Representations from Transformers). Tailored improvements to the model address the nuanced and variable nature of power texts, including the integration of domain-specific pre-training corpora and architectural enhancements. The model, enhanced with the R-BERT (Relation-Bidirectional Encoder Representations from Transformers) algorithm, demonstrates advanced proficiency in identifying entity positions and yields exceptional results in experimental evaluations, particularly achieving a high F1 score for the identification of cooperative relationships. The findings indicate that this innovation can significantly enhance the level of intelligence in power system administrative shift management. Subsequent developments may involve the incorporation of additional expert knowledge to further refine the model.