• Home
  • Search
  • Prompt Learning with Structured Semantic Knowledge Makes Pre-Trained Language Models Better
  • Cite Icon1
  • https://doi.org/10.3390/electronics12153281Copy DOI Icon

Prompt Learning with Structured Semantic Knowledge Makes Pre-Trained Language Models Better

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Pre-trained language models with structured semantic knowledge have demonstrated remarkable performance in a variety of downstream natural language processing tasks. The typical methods of integrating knowledge are designing different pre-training tasks and training from scratch, which requires high-end hardware, massive storage resources, and long computing times. Prompt learning is an effective approach to tuning language models for specific tasks, and it can also be used to infuse knowledge. However, most prompt learning methods accept one token as the answer, instead of multiple tokens. To tackle this problem, we propose the long-answer prompt learning method (KLAPrompt), with three different long-answer strategies, to incorporate semantic knowledge into pre-trained language models, and we compare the performance of these three strategies through experiments. We also explore the effectiveness of the KLAPrompt method in the medical field. Additionally, we generate a word sense prediction dataset (WSP) based on the Xinhua Dictionary and a disease and category prediction dataset (DCP) based on MedicalKG. Experimental results show that discrete answers with the answer space partitioning strategy achieve the best results, and introducing structured semantic information can consistently improve language modeling and downstream tasks.

Loading PDF

Similar Papers
  • Conference Article

GTSP: A Unified Framework for Graph Pretraining and Task-Specific Prompting

  • Sep 19, 2025
  • Fanghua Lu
  • Research Article
  • Citations2

Adaptive Boosting LLMs for Text Classification.

  • Jan 01, 2026
  • IEEE transactions on neural networks and learning systems
  • Mengyao Wang +6
  • Video Transcripts

Robust Transfer Learning with Pretrained Language Models through Adapters

  • Aug 01, 2021
  • Underline Science Inc.
  • Wenjuan Han +2
  • PDF
  • Research Article
  • Citations10

AGI-P: A Gender Identification Framework for Authorship Analysis Using Customized Fine-Tuning of Multilingual Language Model

  • Jan 01, 2024
  • IEEE Access
  • Raheem Sarwar +6
  • Conference Article
  • Citations9

On the Usage of Continual Learning for Out-of-Distribution Generalization in Pre-trained Language Models of Code

  • Nov 30, 2023
  • Martin Weyssow +4
  • PDF
  • Conference Article
  • Citations8

ANNA”:" Enhanced Language Representation for Question Answering

  • Jan 01, 2022
  • Changwook Jun +6
  • Conference Article
  • Citations26

ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction

  • May 13, 2024
  • Jianghao Lin +9
  • Book Chapter
  • Citations2

Enhance Performance of Ad-hoc Search via Prompt Learning

  • Jan 01, 2023
  • Shenghao Yang +4
  • Research Article
  • Citations78

Pre-train, Prompt, and Recommendation: A Comprehensive Survey of Language Modeling Paradigm Adaptations in Recommender Systems

  • Dec 14, 2023
  • Transactions of the Association for Computational Linguistics
  • Peng Liu +2
  • PDF
  • Conference Article
  • Citations22

Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning

  • Jan 01, 2023
  • Fan Zhou +4
  • Conference Article
  • Citations3

Chinese-Korean Weibo Sentiment Classification Based on Pre-trained Language Model and Transfer Learning

  • May 06, 2022
  • Hengxuan Wang +3
  • PDF
  • Conference Article
  • Citations4

CoCoLM: Complex Commonsense Enhanced Language Model with Discourse Relations

  • Jan 01, 2022
  • Changlong Yu +3
  • Research Article
  • Citations16

CSS-LM: A Contrastive Framework for Semi-Supervised Fine-Tuning of Pre-Trained Language Models

  • Jan 01, 2021
  • IEEE/ACM Transactions on Audio, Speech, and Language Processing
  • Yusheng Su +7
  • PDF
  • Research Article
  • Citations3

CLIP-Llama: A New Approach for Scene Text Recognition with a Pre-Trained Vision-Language Model and a Pre-Trained Language Model.

  • Nov 19, 2024
  • Sensors (Basel, Switzerland)
  • Xiaoqing Zhao +3
  • Conference Article
  • Citations1

Subword Tokenization of Noisy Housing Defect Complaints for Named Entity Recognition

  • Jul 07, 2023
  • Proceedings of the ... ISARC
  • Kahyun Jeon +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.