Demystifying LLM-Based Software Engineering Agents

Large Language Models (LLMs) have demonstrated powerful capabilities in the field of code generation, with a deep understanding of the semantics and functionality of code. Building websites is one of the most important tasks in software development, as it utilizes rich frontend displays and backend processing to achieve various service functions. It is one of the most widely used interactive software models. Although there have been some efforts in Web website generation, these efforts have been limited to the automation of generating Web pages. The advent of LLMs provides a new approach to Web site generation tasks. However, there is currently a lack of comprehensive evaluation of the generation performance of LLMs in this context, making it difficult to optimize and improve the generated results in a targeted manner. To address this issue, this paper conducts a multi-angle investigation and analysis of the performance of LLMs in Web site generation tasks. Firstly, Web generation requirements are collected, and effective prompt engineering is designed. These prompts are then input into different LLMs to initiate the self-iteration process. Next, the generated code is fed back into the LLM for security self-iteration, where the model performs vulnerability detection and repair on the code it has generated. The security-enhanced code is subsequently subjected to manual review, where it is evaluated using predefined quantitative metrics to generate indicator values. Finally, through testing, the quality, security, and code defects of the generated front-end and back-end Web code across different LLMs are analyzed, providing a comprehensive evaluation of the generation results. The experiments demonstrate that LLM systems perform well in completing and implementing the functions and layouts of pages in prompts for Web generation tasks, but there remains room for improvement in the security of the Web code.

Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair benchmark of ULT with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research 1 . ULT is available at https://github.com/huangd1999/UnLeakedTestBench .

As autonomous agents and assistants, large language models (LLMs) often struggle with "hallucinations."Fundamentally, the problem is one of prioritization and balance: the LLM needs to understand or infer when it needs to be creative and balance that with its need to be accurate.Most efforts focus on either updating intrinsic knowledge via targeted post-training or by adding external knowledge sources which the LLM can reference neurosymbolically (e.g., via retrieval-augmented generation).However, these all eventually rely on the LLM's implicit reasoning ability during generation, still allowing for these random hallucinations despite high-quality training examples and references.Using aspect-oriented summarization as a case study, we propose LOgit REwriting (LORE), a new controlled generation paradigm which can simultaneously be faithful to external knowledge and to the LLM's intentions.LORE works by adding a rewriting module at left-to-right inference time, continuously reflecting on the newest prediction and trying to find a replacement that is more faithful to the source document.Then, it merges the logits of the replacement with those of the original prediction to generate the next token.We proposed a new long-context aspect-oriented summarization dataset, SLPAspect, and find that LORE generates 5.8% better summaries compared to the LLM without LORE-rewriting. 1

Unit testing is crucial for software development and maintenance. Effective unit testing ensures and improves software quality, but writing unit tests is time-consuming and labor-intensive. Recent studies have proposed deep learning (DL) techniques or large language models (LLMs) to automate unit test generation. These models are usually trained or fine-tuned on large-scale datasets. Despite growing awareness of the importance of data quality, there has been limited research on the quality of datasets used for test generation. To bridge this gap, we systematically examine the impact of noise on the performance of learning-based test generation models. We first apply the open card sorting method to analyze the most popular and largest test generation dataset, Methods2Test, to categorize eight distinct types of noise. Further, we conduct detailed interviews with 17 domain experts to validate and assess the importance, reasonableness, and correctness of the noise taxonomy. Then, we propose CleanTest, an automated noise-cleaning framework designed to improve the quality of test generation datasets. CleanTest comprises three filters: a rule-based syntax filter, a rule-based relevance filter, and a model-based coverage filter. To evaluate its effectiveness, we apply CleanTest on two widely-used test generation datasets, i.e., Methods2Test and Atlas. Our findings indicate that 43.52% and 29.65% of datasets contain noise, highlighting its prevalence. Finally, we conduct comparative experiments using four LLMs (i.e., CodeBERT, AthenaTest, StarCoder, and CodeLlama7B) to assess the impact of noise on test generation performance. The results show that filtering noise positively influences the test generation ability of the models. Fine-tuning the four LLMs with the filtered Methods2Test dataset, on average, improves its performance by 67% in branch coverage, using the Defects4J benchmark. For the Atlas dataset, the four LLMs improve branch coverage by 39%. Additionally, filtering noise improves bug detection performance, resulting in a 21.42% increase in bugs detected by the generated tests.

The rapid evolution of Artificial Intelligence (AI) has brought about significant advancements in multiple domains, including software development. One of the most promising innovations is AI-powered code generation through Large Language Models (LLMs), such as OpenAI’s GPT-3 and GPT-4. These models, having been trained on large amounts of programming data, have the ability to produce human-readable code from natural language inputs, which is a big potential for simplifying and optimizing software development processes. The aim of this paper is to analyze the performance of LLMs in automated software development by testing their performance on a variety of tasks such as code generation, debugging, and optimization of software. The research explores both the strengths and weaknesses that these models have to offer, in terms of some of the most important indicators like code quality, generation time, and maintainability of the code. According to our observation, although LLMs hold immense potential to automate mundane programming tasks and enhance developer productivity, they still struggle to cope with more intricate, domain-specific programming tasks involving a higher level of understanding, for example, designing architectures and top-level decision-making. In spite of such shortcomings, LLMs can tremendously enhance software development processes, particularly for small-scale projects or act as helpers for more senior developers. The paper summarizes by reflecting on the potential for LLMs to transform software development processes in the future, while also the importance of the model's reliability, coding quality, and security to be improved if it is to be made applicable to larger, more crucial uses.

Large language models (LLMs) offer significant promise as a knowledge source for task learning. Prompt engineering has been shown to be effective for eliciting knowledge from an LLM, but alone it is insufficient for acquiring relevant, situationally grounded knowledge for an embodied agent learning novel tasks. We describe a cognitive-agent approach, STARS, that extends and complements prompt engineering, mitigating its limitations and thus enabling an agent to acquire new task knowledge matched to its native language capabilities, embodiment, environment, and user preferences. The STARS approach is to increase the response space of LLMs and deploy general strategies, embedded within the autonomous agent, to evaluate, repair, and select among candidate responses produced by the LLM. We describe the approach and experiments that show how an agent, by retrieving and evaluating a breadth of responses from the LLM, can achieve 77-94% task completion in one-shot learning without user oversight. The approach achieves 100% task completion when human oversight (such as an indication of preference) is provided. Further, the type of oversight largely shifts from explicit, natural language instruction to simple confirmation/discomfirmation of high-quality responses that have been vetted by the agent before presentation to a user.

Purpose: Large language models (LLMs) improve medical writing efficiency but introduce methodological, ethical, and legal risks. This review examines current evidence on the limitations of LLM-assisted medical writing and proposes principles for its responsible integration into biomedical research.Current concepts: LLMs are commonly used for draft generation, language editing, literature summarization, reference handling, statistical code generation, and manuscript structuring. Studies consistently report improved readability and reduced writing time, particularly among non-native English-speaking authors. However, recurrent challenges include factual hallucinations; fabricated or inaccurate citations; incomplete retrieval of recent literature due to training cutoffs; prompt-sensitive statistical errors; ambiguity regarding authorship and accountability; risks of unintended plagiarism; and concerns related to patient data privacy. These limitations arise from the probabilistic nature of LLMs and their lack of intrinsic fact-verification mechanisms or ethical reasoning.Discussion and conclusion: Risks associated with LLM use vary by manuscript stage and therefore require differentiated oversight. LLMs should be confined primarily to language refinement rather than fact generation, and literature-related outputs must be verified against primary sources, preferably using retrieval-augmented tools. Statistical analyses should remain under human control, with independent validation of all outputs. Ethical governance requires transparent disclosure of LLM use, clear assignment of human responsibility, and strict safeguards for sensitive data. A dual framework combining human-in-the-loop and human-on-the-loop oversight offers a pragmatic model for balancing efficiency with scientific rigor. When positioned as augmentative tools rather than autonomous agents, LLMs can be responsibly integrated into medical research without compromising integrity or reproducibility.

Automated debugging is an emerging research field that aims to automatically find and repair bugs. In this field, Fault Localization (FL) and Automated Program Repair (APR) gain the most research efforts. Most recently, researchers have adopted pre-trained Large Language Models (LLMs) to facilitate FL and APR and their results are promising. However, the LLMs they used either vanished (such as Codex) or outdated (such as early versions of GPT). In this paper, we evaluate the performance of recent commercial closed-source general-purpose LLMs on FL and APR, i.e., ChatGPT 3.5, ERNIE Bot 3.5, and IFlytek Spark 2.0. We select three popular LLMs and evaluate them on 120 real-world Java bugs from the benchmark Defects4J. For FL and APR, we designed three kinds of prompts for each, considering different kinds of information. The results show that these LLMs could successfully locate 53.3% and correctly fix 12.5% of these bugs.

Automated Program Repair (APR) aims to automatically generate patches for rectifying software bugs. Recent strides in Large Language Models (LLM), such as ChatGPT, have yielded encouraging outcomes in APR, especially within the conversation-driven APR framework. Nevertheless, the efficacy of conversation-driven APR is contingent on the quality of the feedback information. In this article, we propose ContrastRepair , a novel conversation-based APR approach that augments conversation-driven APR by providing LLMs with contrastive test pairs. A test pair consists of a failing test and a passing test, which offer contrastive feedback to the LLM. Our key insight is to minimize the difference between the generated passing test and the given failing test, which can better isolate the root causes of bugs. By providing such informative feedback, ContrastRepair enables the LLM to produce effective bug fixes. The implementation of ContrastRepair is based on the state-of-the-art LLM, ChatGPT, and it iteratively interacts with ChatGPT until plausible patches are generated. We evaluate ContrastRepair on multiple benchmark datasets, including Defects4J, QuixBugs, and HumanEval-Java. The results demonstrate that ContrastRepair significantly outperforms existing methods, achieving a new state-of-the-art in program repair. For instance, among Defects4J 1.2 and 2.0, ContrastRepair correctly repairs 143 out of all 337 bug cases, while the best-performing baseline fixes 124 bugs.

Large language models illuminate a progressive pathway to artificial intelligent healthcare assistant

This scientific article presents the results of a study focused on the current practices and future prospects of AI-tools usage, specifically large language models (LLMs), in software development (SD) processes within European IT companies. The Pan-European study covers 35 SD teams from all regions of Europe and consists of three sections: the first section explores the current adoption of AI-tools in software production, the second section addresses common challenges in LLMs implementation, and the third section provides a forecast of the tech future in AI-tools development for SD. The study reveals that AI-tools, particularly LLMs, have gained popularity and approbation in European IT companies for tasks related to software design and construction, coding, and software documentation. However, their usage for business and system analysis remains limited. Nevertheless, challenges such as resource constraints and organizational resistance are evident. The article also highlights the potential of AI-tools in the software development process, such as automating routine operations, speeding up work processes, and enhancing software product excellence. Moreover, the research examines the transformation of IT paradigms driven by AI-tools, leading to changes in the skill sets of software developers. Although the impact of LLMs on the software development industry is perceived as modest, experts anticipate significant changes in the next 10 years, including AI-tools integration into advanced IDEs, software project management systems, and product management tools. Ethical concerns about data ownership, information security and legal aspects of AI-tools usage are also discussed, with experts emphasizing the need for legal formalization and regulation in the AI domain. Overall, the study highlights the growing importance and potential of AI-tools in software development, as well as the need for careful consideration of challenges and ethical implications to fully leverage their benefits.

Large Language Models (LLMs) have become integral to a wide spectrum of applications, ranging from traditional computing tasks to advanced artificial intelligence (AI) applications. This widespread adoption has spurred extensive research into LLMs across various disciplines, including the social sciences. Notably, studies have revealed that LLMs possess emotional intelligence, which can be further developed through positive emotional stimuli. This discovery raises an intriguing question: can negative emotions similarly influence LLMs, potentially enhancing their performance? In response to this question, we introduce NegativePrompt, a novel approach underpinned by psychological principles, involving ten specifically designed negative emotional stimuli. We embark on rigorous experimental evaluations of five LLMs including Flan-T5-Large, Vicuna, Llama 2, ChatGPT, and GPT-4, across a set of 45 tasks. The results are revealing: NegativePrompt markedly enhances the performance of LLMs, evidenced by relative improvements of 12.89% in Instruction Induction tasks and 46.25% in BIG-Bench tasks. Moreover, we conduct attention visualization experiments to decipher the underlying mechanisms of NegativePrompt's influence. Our research contributes significantly to the understanding of LLMs and emotion interaction, demonstrating the practical efficacy of NegativePrompt as an emotion-driven method and offering novel insights for the enhancement of LLMs in real-world applications. The code is available at https://github.com/wangxu0820/NegativePrompt.

Self-Admitted Technical Debt (SATD), cases where developers intentionally acknowledge suboptimal solutions in code through comments, poses a significant challenge to software maintainability. Left unresolved, SATD can degrade code quality, increase maintenance costs, and hinder long-term software sustainability. Manually identifying and resolving SATD is time-consuming and error-prone, often competing with feature development and release deadlines. Automating SATD repayment is therefore essential to alleviate developer burden. While Large Language Models (LLMs) have shown promise in tasks like code generation and program repair, their potential in automated SATD repayment remains underexplored. In this paper, we identify three key technical challenges in training and evaluating LLMs for SATD repayment: (1) dataset representativeness and scalability, (2) removal of irrelevant SATD repayment samples, and (3) limitations of existing evaluation metrics, i.e., BLEU and CrystalBLEU. To address the first two dataset-related challenges, we adopt a language-independent SATD tracing tool and design a 10-step filtering pipeline to extract SATD repayments from repository commit histories, leveraging LLM-as-judge for relevance filtering. This results in two new large-scale SATD repayment datasets: 58,722 items for Python and 97,347 items for Java. To improve evaluation, we introduce two diff-based metrics, BLEU-diff and CrystalBLEU-diff, which measure code changes rather than whole code, thereby mitigating the impact of SATD-irrelevant source code. Additionally, we propose a new metric, Line-Level Exact Match on Diff (LEMOD), which is both interpretable and informative, providing fine-grained insights into SATD repayment quality. Using our new benchmarks and evaluation metrics, we evaluate two types of automated SATD repayment methods: fine-tuning smaller models (a few hundred million parameters) and prompt engineering with large-scale models, including four SOTA open-source LLMs and GPT-4o-mini. Our results reveal that while fine-tuned smaller models achieve Exact Match (EM) scores comparable to prompt-based approaches, they underperform on BLEU-based metrics and LEMOD. The best EM performance is achieved by Gemma-2-9B, correctly addressing 10.1% of Python SATDs and 8.1% of Java SATDs with a simple prompt. The only previous study in this domain focused solely on Java, achieving an EM score of 2.3%. When using BLEU-diff, CrystalBLEU-diff, and LEMOD metrics, Llama-3.1-70B-Instruct and GPT-4o-mini deliver the highest performance. While our three proposed metrics demonstrate strong correlations with EM (0.65~0.84), both BLEU and CrystalBLEU show almost no correlation with this intuitive metric (0.01~0.08), highlighting their largely unpredictable behavior when applied directly to the entire code rather than the diff. Our work contributes a robust benchmark, improved evaluation metrics, and a comprehensive evaluation of LLMs, advancing research on automated SATD repayment.

AI technology’s rapid advancements have led to its widespread adoption in diverse industries, such as support agents, healthcare, education, software development and finance because such AI technologies has the ability to generated human-like text to solve a complex user’s problem. Nevertheless, the effectiveness and success of AI models greatly depends on large language model that contain extensive, varied, and top-notch datasets and social network platforms for both training and evaluation purposes that is sourced from web and directly form user interaction that will also contain sensitive information. However, this power capabilities and progress have raised significant concerns related data security and privacy, particularly regarding protection of user data including social network during training and deployment phase. Large language models process and generate huge amount of data which have the potential to unintentionally memorise and repeat personal sensitive information like location data, social security number, and phone numbers which are shared by users that can lead to pose a risk of data leaking and degrade the accuracy of model training and information. This paper will perform a comprehensive and systematic review to examine the fundamentals principle of large language model with particularly concentration on data privacy concern associated with large language model during training data and potential biases. Furthermore, we analysis and evaluated the current level of vulnerabilities, privacy challenges, and examine new security and privacy attacks targeting LLMs that cause of misuse and information leakage. Additionally, we proposed a set of best practices aimed to protect user data. Recognising the significance of conducting such research on protecting privacy and improving security is crucial because it ensures the continuous progress and public trust in LLM technology.

The use of large language models (LLMs) for generating code in software development is on the rise. While LLMs demonstrate impressive capabilities, there remains a need to evaluate the quality of generated code beyond functional correctness. This paper addresses a gap in current research by evaluating Python code generated by nine state-of-the-art LLMs according to the four main code quality categories (maintainability, reliability, performance efficiency, and security) defined by the ISO/IEC 5055:2021 standard. The evaluation spans three popular application domains (high-performance computing, machine learning, and data processing). It employs a stratified prompting approach with varying levels (short, medium, and long) of detail. Using widely adopted static code analysis tools, the study identifies which LLMs perform best across these categories. The analysis involved selecting nine algorithms across three application domains, and the generated code was compared against a human developer using four static code analysis tools. Metrics were organized into the four ISO 5055 categories, and composite scores were calculated for each category following preprocessing to ensure accurate evaluation. The results showed that GPT-4-Turbo produced the most reliable, performance-efficient, and secure code, while Gemini excelled in generating maintainable Python code among the evaluated models. The study concludes that properly prompted and configured LLMs can produce code that meets or even exceeds human-developed code in terms of the four ISO categories. Future work will continue to refine and expand upon these methodologies on other programming languages.