• Home
  • Search
  • Large Language Models and Empathy: Systematic Review
  • Cite Icon166
  • https://doi.org/10.2196/52597Copy DOI Icon

Large Language Models and Empathy: Systematic Review

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

BackgroundEmpathy, a fundamental aspect of human interaction, is characterized as the ability to experience another being’s emotions within oneself. In health care, empathy is a fundamental for health care professionals and patients’ interaction. It is a unique quality to humans that large language models (LLMs) are believed to lack.ObjectiveWe aimed to review the literature on the capacity of LLMs in demonstrating empathy.MethodsWe conducted a literature search on MEDLINE, Google Scholar, PsyArXiv, medRxiv, and arXiv between December 2022 and February 2024. We included English-language full-length publications that evaluated empathy in LLMs’ outputs. We excluded papers evaluating other topics related to emotional intelligence that were not specifically empathy. The included studies’ results, including the LLMs used, performance in empathy tasks, and limitations of the models, along with studies’ metadata were summarized.ResultsA total of 12 studies published in 2023 met the inclusion criteria. ChatGPT-3.5 (OpenAI) was evaluated in all studies, with 6 studies comparing it with other LLMs such GPT-4, LLaMA (Meta), and fine-tuned chatbots. Seven studies focused on empathy within a medical context. The studies reported LLMs to exhibit elements of empathy, including emotions recognition and emotional support in diverse contexts. Evaluation metric included automatic metrics such as Recall-Oriented Understudy for Gisting Evaluation and Bilingual Evaluation Understudy, and human subjective evaluation. Some studies compared performance on empathy with humans, while others compared between different models. In some cases, LLMs were observed to outperform humans in empathy-related tasks. For example, ChatGPT-3.5 was evaluated for its responses to patients’ questions from social media, where ChatGPT’s responses were preferred over those of humans in 78.6% of cases. Other studies used subjective readers’ assigned scores. One study reported a mean empathy score of 1.84-1.9 (scale 0-2) for their fine-tuned LLM, while a different study evaluating ChatGPT-based chatbots reported a mean human rating of 3.43 out of 4 for empathetic responses. Other evaluations were based on the level of the emotional awareness scale, which was reported to be higher for ChatGPT-3.5 than for humans. Another study evaluated ChatGPT and GPT-4 on soft-skills questions in the United States Medical Licensing Examination, where GPT-4 answered 90% of questions correctly. Limitations were noted, including repetitive use of empathic phrases, difficulty following initial instructions, overly lengthy responses, sensitivity to prompts, and overall subjective evaluation metrics influenced by the evaluator’s background.ConclusionsLLMs exhibit elements of cognitive empathy, recognizing emotions and providing emotionally supportive responses in various contexts. Since social skills are an integral part of intelligence, these advancements bring LLMs closer to human-like interactions and expand their potential use in applications requiring emotional intelligence. However, there remains room for improvement in both the performance of these models and the evaluation strategies used for assessing soft skills.

Similar Papers
  • Research Article
  • Citations1

Comparing Large Language Models and Traditional Machine Translation Tools for Translating Medical Consultation Summaries: Quantitative Pilot Feasibility Study.

  • Apr 13, 2026
  • JMIR formative research
  • Andy Li +4
  • Research Article
  • Citations58

Quality of Answers of Generative Large Language Models Versus Peer Users for Interpreting Laboratory Test Results for Lay Patients: Evaluation Study

  • Apr 17, 2024
  • Journal of Medical Internet Research
  • Zhe He +8
  • PDF
  • Research Article
  • Citations11

Semantic Clinical Artificial Intelligence vs Native Large Language Model Performance on the USMLE

  • Apr 22, 2025
  • JAMA Network Open
  • Peter L Elkin +9
  • PDF
  • Research Article
  • Citations6

Reasoning-based LLMs surpass average human performance on medical social skills

  • Oct 17, 2025
  • Scientific Reports
  • Khalid Ibraheem Alohali +4
  • Research Article
  • Citations3

Evaluating Medical Text Summaries Using Automatic Evaluation Metrics and LLM-as-a-Judge Approach: A Pilot Study

  • Dec 19, 2025
  • Diagnostics
  • Yuriy Vasilev +8
  • Research Article
  • Citations31

A dataset and benchmark for hospital course summarization with adapted large language models.

  • Dec 30, 2024
  • Journal of the American Medical Informatics Association : JAMIA
  • Asad Aali +11
  • Research Article
  • Citations6

Large Language Models in Randomized Controlled Trials Design: Observational Study

  • Sep 03, 2025
  • Journal of Medical Internet Research
  • Liyuan Jin +6
  • Research Article
  • Citations2

An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessment

  • Jun 19, 2025
  • ACM Transactions on Intelligent Systems and Technology
  • Xuanxin Wu +1
  • Research Article
  • Citations9

Comparative Evaluation of Large Language Models for Translating Radiology Reports into Hindi

  • Sep 04, 2024
  • Indian Journal of Radiology and Imaging
  • Amit Gupta +3
  • Research Article
  • Citations8

Benchmarking Vision Capabilities of Large Language Models in Surgical Examination Questions.

  • Apr 01, 2025
  • Journal of surgical education
  • Jean-Paul Bereuter +8
  • Supplementary Content
  • Citations21

Knowledge-Practice Performance Gap in Clinical Large Language Models: Systematic Review of 39 Benchmarks

  • Dec 01, 2025
  • Journal of Medical Internet Research
  • Eun Jeong Gong +3
  • Research Article
  • Citations16

The performance of large language models in intercollegiate Membership of the Royal College of Surgeons examination.

  • Mar 06, 2024
  • Annals of the Royal College of Surgeons of England
  • J Chan +2
  • Research Article
  • Citations12

A statistical framework for evaluating the repeatability and reproducibility of large language models

  • Nov 04, 2025
  • medRxiv
  • Cathy Shyr +10
  • Research Article
  • Citations35

Large Language Model Ability to Translate CT and MRI Free-Text Radiology Reports Into Multiple Languages.

  • Dec 01, 2024
  • Radiology
  • Aymen Meddeb +23
  • Research Article

#2924 Comparison of large language models and traditional natural language processing techniques in predicting arteriovenous fistula failure

  • May 23, 2024
  • Nephrology Dialysis Transplantation
  • Suman Lama +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.