• Cite Icon4
  • https://doi.org/10.1145/3524842.3528475Copy DOI Icon

Comments on comments

  • May 23, 2022
  • Nikitha Rao +3 more
Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

A central function of code review is to increase understanding; helping reviewers understand a code change aids in knowledge transfer and finding bugs. Comments in code largely serve a similar purpose, helping future readers understand the program. It is thus natural to study what happens when these two forms of understanding collide. We ask: what documentation-related comments do reviewers make and how do they affect understanding of the contribution? We analyze ca.700K review comments on 2,000 (Java and Python) GitHub projects, and propose several filters to identify which comments are likely to be either in response to a change in documentation and/or call for such a change. We identify 65K such cases. We next develop a taxonomy of the reviewer intents behind such "comments on comments". We find that achieving a shared understanding of the code is key: reviewer comments most often focused on clarification, followed by pointing out issues to fix, such as typos and outdated comments. Curiously, clarifying comments were frequently suggested (often verbatim) by the reviewer, indicating a desire to persist their understanding acquired during code review. We conclude with a discussion of implications of our comments-on-comments dataset for research on improving code review, including the potential benefits for automating code review.

Loading PDF

Similar Papers
  • Conference Article

LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM

  • Nov 16, 2025
  • Yuxin Zhang +4
  • Conference Article
  • Citations7

Understanding code snippets in code reviews

  • May 16, 2022
  • Liming Fu +2
  • Conference Article
  • Citations53

CORE: Automating Review Recommendation for Code Changes

  • Feb 01, 2020
  • Jing Kai Siow +4
  • Book Chapter
  • Citations5

An Empirical Study of Link Sharing in Review Comments

  • Jan 01, 2019
  • Jing Jiang +2
  • Conference Article
  • Citations3

Recommending Code Reviews Leveraging Code Changes with Structured Information Retrieval

  • Oct 01, 2023
  • Ohiduzzaman Shuvo +2
  • Conference Article
  • Citations5

Potential Technical Debt and Its Resolution in Code Reviews: An Exploratory Study of the OpenStack and Qt Communities

  • Sep 19, 2022
  • Liming Fu +5
  • PDF
  • Research Article
  • Citations48

An exploratory study on confusion in code reviews

  • Jan 01, 2021
  • Empirical Software Engineering
  • Felipe Ebert +3
  • Book Chapter
  • Citations9

Automated Code Review Comment Classification to Improve Modern Code Reviews

  • Jan 01, 2022
  • Miroslaw Ochodek +3
  • Conference Article

How Does Code Reviewing Feedback Evolve?: A Longitudinal Study at Dell EMC

  • May 01, 2022
  • Ruiyin Wen +2
  • Conference Article
  • Citations18

Resolving Code Review Comments with Machine Learning

  • Apr 14, 2024
  • Alexander Froemmgen +15
  • PDF
  • Conference Article
  • Citations16

Primers or reminders?

  • Jun 27, 2020
  • Davide Spadini +2
  • Conference Article
  • Citations88

Characteristics of useful code reviews: an empirical study at Microsoft

  • May 16, 2015
  • Amiangshu Bosu +2
  • Conference Article
  • Citations56

An empirical study of design discussions in code review

  • Oct 11, 2018
  • Farida El Zanaty +4
  • Book Chapter
  • Citations5

Measuring the Effectiveness of Software Code Review Comments

  • Jan 01, 2020
  • Syeda Sumbul Hossain +4
  • Conference Article
  • Citations161

Characteristics of Useful Code Reviews: An Empirical Study at Microsoft

  • May 01, 2015
  • Amiangshu Bosu +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.