• Home
  • Search
  • Automated Inline Comment Smell Detection and Repair with Large Language Models
  • https://doi.org/10.1109/ase63991.2025.00088Copy DOI Icon

Automated Inline Comment Smell Detection and Repair with Large Language Models

  • Nov 16, 2025
  • Hatice Kübra Çağlar +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Context: Code comments play a critical role in improving code readability, maintainability, and collaborative development. However, comments may deviate from best practices due to software evolution, where code changes are not reflected in comments, as well as practitioner-related issues such as vague descriptions, redundancy, or misaligned intent. These issues lead to various comment smells that degrade software quality. While prior studies have explored comment inconsistencies, most are limited in scope, either addressing a narrow subset of smells or focusing solely on detection without considering repair.Objective: This study evaluates the effectiveness of large language models (LLMs) in both detecting and repairing inline code comment smells, using a comprehensive taxonomy of code comment smell types.Method: We extended a prior data set by incorporating repaired versions of smelly comments, resulting in 2,211 unique instances. Four LLMs—GPT-4o-mini, o3-mini, DeepSeek-V3, and Codestral-2501—are evaluated under zero-shot and few-shot prompting strategies. To account for non-deterministic behavior in LLM outputs and ensure robustness, each configuration is executed five times. Detection performance is measured using accuracy, macro F1 score, and Matthews correlation coefficient (MCC); repair is evaluated using SBERT similarity, METEOR, and ROUGE-L. Our multi-stage pipeline feeds detection outputs into the repair phase, where the detection result with the highest macro F1 score is used to simulate the best possible repair scenario. Median scores across runs are reported for comparison.Results: o3-mini with few-shot prompting achieves the highest median detection performance: macro F1 of 0.41, MCC of 0.50, and accuracy of 0.72, exceeding the baseline of GPT-4. For repair, Codestral-2501 in the zero-shot setting yields the best results with a median SBERT score of 0.61, followed by DeepSeek-V3 and GPT-4o-mini at 0.53, and o3-mini at 0.46. Few-shot prompts improve detection, while zero-shot prompts are more effective for repair.Conclusion: Lightweight LLMs such as o3-mini can achieve strong detection performance when guided by effective few-shot prompts. For example, o3-mini with few-shot prompting attains the highest median detection results: macro F1 of 0.41, MCC of 0.50, and accuracy of 0.72, surpassing the GPT-4 baseline. In contrast, repair tasks benefit more from zero-shot prompting, though they introduce challenges such as overfitting and the risk of generating new smells. Our findings support the development of practical tools, including a GitHub-integrated comment repair assistant, and motivate future work on dynamic prompt selection and multilingual benchmark construction.

Similar Papers
  • Research Article
  • Citations2

Large Language Models in Crisis Informatics for Zero and Few-Shot Classification

  • May 22, 2025
  • ACM Transactions on the Web
  • Cinthia Sánchez +2
  • Research Article

BeigeSage: sentiment classification in economic texts utilizing LLMs and fine-tuned BERT models

  • Mar 25, 2026
  • Applied Economics
  • Charlie Smith +1
  • Research Article

P0386 Automated Montreal Classification Extraction from Real-World Colonoscopy Narratives Using a Hybrid NLP–LLM Framework

  • Jan 01, 2026
  • Journal of Crohn’s and Colitis
  • B F Agargun +13
  • Research Article

Detecting \u201clarge language models fingerprint\u201d for Japanese texts generated by six LLMs

  • Jan 01, 2026
  • Frontiers in Artificial Intelligence
  • Wataru Zaitsu +4
  • Research Article

Exploring the potential of large language models for assessing medication adherence to the ESC heart failure guidelines

  • Nov 03, 2025
  • JAMIA Open
  • Noman Dormosh +4
  • Research Article
  • Citations19

Ophthalmological Question Answering and Reasoning Using OpenAI o1 vs Other Large Language Models

  • Jul 31, 2025
  • JAMA Ophthalmology
  • Sahana Srinivasan +18
  • Research Article

Evaluation of GPT-5 for Esophageal Cancer Staging Using Fluorodeoxyglucose Positron Emission Tomography Maximum-Intensity Projection Images: Comparative Pilot Study.

  • Feb 23, 2026
  • JMIR cancer
  • Hiroki Maruyama +7
  • Research Article

Detecting Stigmatizing Language in Clinical Notes with Large Language Models for Addiction Care

  • Aug 12, 2025
  • medRxiv
  • R Sethi +8
  • PDF
  • Research Article
  • Citations32

Automatic structuring of radiology reports with on-premise open-source large language models

  • Oct 10, 2024
  • European Radiology
  • Piotr Woźnicki +11
  • Research Article
  • Citations25

Exploring the Effectiveness of LLMs in Automated Logging Statement Generation: An Empirical Study

  • Dec 01, 2024
  • IEEE Transactions on Software Engineering
  • Yichen Li +7
  • Research Article

Context-Aware Sentence Classification of Radiology Reports Using Synthetic Data: Development and Validation Study.

  • Apr 10, 2026
  • Journal of medical Internet research
  • Tomohiro Kikuchi +6
  • Research Article

Factors Associated With an LLM Arriving at the Same Diagnosis as a Musculoskeletal Specialist.

  • Jun 17, 2026
  • The Journal of the American Academy of Orthopaedic Surgeons
  • Alexander Drost +3
  • Research Article
  • Citations8

How to leverage large language models for automatic ICD coding.

  • May 01, 2025
  • Computers in biology and medicine
  • Youngju Yoo +1
  • Research Article
  • Citations5

Can general purpose large language models assist pediatricians in predicting infants with serious bacterial infection?

  • Nov 14, 2025
  • BMC Medical Informatics and Decision Making
  • Ivan Šimunović +9
  • Research Article

Large Language Models to Detect Stigmatizing Language in Critically Ill Patients With Substance Use Disorders

  • May 01, 2025
  • American Journal of Respiratory and Critical Care Medicine
  • M Afshar +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.