With the increasing adoption of smart speakers, Virtual Personal Assistant (VPA) applications have become integral to daily life, enabling users to access news, entertainment, and smart device control through Voice User Interfaces (VUI). However, many VPA apps suffer from quality issues, such as unexpected terminations and failures to process common user commands, highlighting the urgent need for systematic and efficient VUI testing. Existing chatbot-style and model-based testing approaches lack global and semantic awareness, resulting in ineffective test case generation and inefficient state exploration. To address these challenges, we introduce Elevate, a model-enhanced, LLM-driven VPA testing framework that employs a multi-agent architecture to enhance VUI behavior testing. Elevate comprises three specialized LLM agents—Observer, Generator, and Planner—that collaboratively perform state extraction, test case generation, and guided state exploration. Additionally, a deterministic finite automaton (DFA)-based behavior model is designed to abstract app behavior and provide structured guidance to LLM agents, enhancing testing performance. Elevate also incorporates a feedback mechanism that refines testing strategies based on observed behaviors, ensuring continuous improvement. Implemented using GPT-4-Turbo and DeepSeek-R1, Elevate has been evaluated on problem detection, sentence/semantic coverage, and large-scale testing. Experimental results show that Elevate outperforms state-of-the-art methods (Vitas and LLM-based chatbots), detecting at least 18 and 37 more problems, respectively, and achieving over 10% and 30% higher state coverage. In a large-scale evaluation on 4,000 Alexa skills, Elevate further demonstrated 15% higher coverage than Vitas, confirming its effectiveness, scalability, and potential for widespread application in VUI testing.

Most log-based anomaly detectors assume logs are stable, though in reality they are often unstable due to software or environmental changes. Anomaly detection on unstable logs (ULAD) is therefore a more realistic, yet under-investigated challenge. Current approaches predominantly employ machine learning (ML) models, which often require extensive labeled data for training. To mitigate data insufficiency, we propose FlexLog , a novel hybrid approach for ULAD that combines ML models — decision tree, k-nearest neighbors, and a feedforward neural network — with a Large Language Model (Mistral) through ensemble learning. FlexLog also incorporates a cache and retrieval-augmented generation (RAG) to further enhance efficiency and effectiveness. To evaluate FlexLog , we configured four datasets for ULAD, namely ADFA-U, LOGEVOL-U, SynHDFS-U, and SYNEVOL-U. FlexLog outperforms all baselines by at least 1.2 percentage points (pp) in F1 score while using much less labeled data (62.87 pp reduction). When trained on the same amount of data as the baselines, FlexLog achieves up to a 13 pp increase in F1 score on ADFA-U across varying training dataset sizes. Additionally, FlexLog maintains inference time under one second per log sequence, making it suitable for most applications, except latency-sensitive systems. Further analysis reveals the positive impact of FlexLog ’s key components: cache, RAG, and ensemble learning.

Error logs like Java exceptions play a crucial role in diagnosing and resolving errors within the industry. Nonetheless, the extensive logging of Java exceptions may result in exception fatigue in large-scale Java systems at an industrial level, where the frequency of Java exceptions being generated surpasses developers’ ability to manage them effectively. Regrettably, there is a lack of research on the seriousness, prevalence, and solutions to this problem. To close this gap, we first make a comprehensive investigation into the exception fatigue problem within a prominent Internet corporation in China, namely Alibaba, confirming its importance in the industry. Consequently, we introduce a novel solution called ABEL , designed to automatically pinpoint the most relevant exceptions associated with software failures. The key challenge lies in the randomness of exceptions, which prevents existing sequence-based techniques from being effective. To address this challenge, ABEL establishes correlations between Java exceptions and the Key Performance Indicator (KPI) of applications, enabling the identification of exceptions leading to irregularities in KPI. Our evaluation of ABEL across four Java applications and five business KPIs within Alibaba illustrates its capability to pinpoint the primary cause of exception logs with an AC@5 (top-5 accuracy) exceeding 90%, effectively mitigating the exception fatigue problem within Alibaba. Furthermore, it can identify the root-cause exceptions in a real software failure within just 4 minutes, outperforming the manual investigation process by over an hour.

Automated Program Repair (APR) aims to automatically generate patches for rectifying software bugs. Recent strides in Large Language Models (LLM), such as ChatGPT, have yielded encouraging outcomes in APR, especially within the conversation-driven APR framework. Nevertheless, the efficacy of conversation-driven APR is contingent on the quality of the feedback information. In this article, we propose ContrastRepair , a novel conversation-based APR approach that augments conversation-driven APR by providing LLMs with contrastive test pairs. A test pair consists of a failing test and a passing test, which offer contrastive feedback to the LLM. Our key insight is to minimize the difference between the generated passing test and the given failing test, which can better isolate the root causes of bugs. By providing such informative feedback, ContrastRepair enables the LLM to produce effective bug fixes. The implementation of ContrastRepair is based on the state-of-the-art LLM, ChatGPT, and it iteratively interacts with ChatGPT until plausible patches are generated. We evaluate ContrastRepair on multiple benchmark datasets, including Defects4J, QuixBugs, and HumanEval-Java. The results demonstrate that ContrastRepair significantly outperforms existing methods, achieving a new state-of-the-art in program repair. For instance, among Defects4J 1.2 and 2.0, ContrastRepair correctly repairs 143 out of all 337 bug cases, while the best-performing baseline fixes 124 bugs.

With the rapid advancement of quantum computing, research on quantum machine learning (QML) algorithms has grown significantly. Among these, the Quantum Neural Network (QNN) stands out as one of the promising algorithms that integrates the principles of quantum computing with artificial neural networks to process data. Inspired by applications of QNN across fields, we investigate their use in software testing for the Cancer Registry of Norway (CRN), part of the Norwegian Institute of Public Health (NIPH), responsible for cancer statistics among the Norwegian population. CRN develops a complex socio-technical software system, Cancer Registration Support System ( \(\mathtt{CaReSS}\) ), interacting with many entities (e.g., hospitals, medical laboratories, and other patient registries) to achieve its task. For cost-effective testing of \(\mathtt{CaReSS}\) , CRN has employed \(\mathtt{EvoMaster}\) , an AI-based REST API testing tool combined with an integrated classical machine learning model \(\mathtt{EvoClass}\) . Within this context, we propose \(\mathtt{EvoQlass}\) to investigate the feasibility of using, inside \(\mathtt{EvoMaster}\) , a QNN classifier, instead of the existing classical machine learning model. Results indicate that \(\mathtt{EvoQlass}\) can achieve performance comparable to that of \(\mathtt{EvoClass}\) . We further explore the effects of various QNN configurations on performance and offer recommendations for optimal QNN settings for future QNN developers.

The rapid advancement of Large Language Models (LLMs) has opened up new opportunities for leveraging artificial intelligence in a variety of application domains, including cybersecurity. As the volume and sophistication of cyber threats continue to grow, there is an increasing need for intelligent systems that can automatically detect vulnerabilities, analyze malware, and respond to attacks. In this survey, we conduct a comprehensive review of the literature on the application of LLMs in cybersecurity (LLM4Security). By comprehensively collecting over 40K relevant papers and systematically analyzing 185 papers from top security and software engineering venues, we aim to provide a holistic view of how LLMs are being used to solve diverse problems across the cybersecurity domain. Through our analysis, we identify several key findings. First, we observe that LLMs are being applied to an expanding range of cybersecurity tasks, including vulnerability detection, malware analysis, and network intrusion detection. Second, we analyze application trends of different LLM architectures (such as encoder-only, encoder-decoder, and decoder-only) across security domains. Third, we identify increasingly sophisticated techniques for adapting LLMs to cybersecurity, such as advanced fine-tuning, prompt engineering, and external augmentation strategies. A significant emerging trend is the use of LLM-based autonomous agents, which represent a paradigm shift from single-task execution to orchestrating complex, multi-step security workflows. Furthermore, we find that the datasets used for training and evaluating LLMs are often limited, highlighting the need for more comprehensive datasets and the use of LLMs for data augmentation. Finally, we discuss the main challenges and opportunities for future research, including the need for more interpretable models, addressing the inherent security risks of LLMs, and their potential for proactive defense. Overall, our survey provides a comprehensive overview of the current state-of-the-art in LLM4Security and identifies several promising directions for future research. We believe that the insights and findings presented in this survey will contribute to the growing body of knowledge on the application of LLMs in cybersecurity and provide valuable guidance for researchers and practitioners working in this field.

Automated vulnerability detection is a critical issue in software security. The advent of deep learning (DL) has led to numerous studies employing DL to detect vulnerabilities in software source code. However, existing approaches still perform poorly, particularly with real-world vulnerabilities, due to the difficulty in accurately capturing their properties. To this end, we introduce PVDetector, a DL-based approach that utilizes rich code semantics, incorporates vulnerability knowledge, and leverages pretrained code representations for precise vulnerability detection. At its core, PVDetector employs a new model called Vulnerability-enriched Code Semantic Graph (VCSG), which accurately characterizes functions by distinguishing the semantics of identical variables and more finely capturing control dependencies, data dependencies, and vulnerability relationships. Additionally, we introduce four pretraining tasks specifically designed to learn the semantics of control, data, vulnerability, and variables from the VCSG model. These pretraining tasks significantly enhance PVDetector's capability to detect vulnerabilities in downstream tasks. Experimental results indicate that PVDetector outperforms SOTAs by 5.0%-12.5% in precision, 0.2%-9.7% in recall, and 3.0%-15.1% in F1-score. Additionally, it supports six programming languages and demonstrates high efficiency (e.g., 10.6 \(\times\) faster than DeepDFA). When applied to seven software products, PVDetector discovered 55 vulnerabilities, including 10 silently patched flaws that had not been previously reported.

Empathy, defined as the ability to understand and share others’ perspectives and emotions, is essential in software engineering (SE), where developers often collaborate with diverse stakeholders. It is also considered as a vital competency in many professional fields such as medicine, healthcare, nursing, animal science, education, marketing, and project management. Despite its importance, empathy remains under-researched in SE. To further explore this, we conducted a socio-technical grounded theory (STGT) study through in-depth semi-structured interviews with 22 software developers and stakeholders. Our study explored the role of empathy in SE and how SE activities and processes can be improved by considering empathy. Through applying the systematic steps of STGT data analysis and theory development, we developed a theory that explains the role of empathy in SE. Our theory details the contexts in which empathy arises, the conditions that shape it, the causes and consequences of its presence and absence. We also identified contingencies for enhancing empathy or overcoming barriers to its expression. Our findings provide practical implications for SE practitioners and researchers, offering a deeper understanding of how to effectively integrate empathy into SE processes.

Parallel fuzzing introduces two key steps: task division and task merging to improve efficiency and effectiveness. However, existing works lack effective analysis of task results during task merging; they employ simple differential seed distribution strategies during task division, which cannot prevent task conflicts. Moreover, there is a lack of coordination between task division and task merging phases, and the task division process does not fully utilize feedback information from task merging, reducing the collaboration efficiency between parallel sub-nodes. In this paper, we first introduce a dual-loop multi-phase parallel framework. Under this framework, we combine the “exploration” capability of Structure-Unaware (SU) task division strategy with the “exploitation” capability of Structure-Aware (SA) task division strategy to form a new hybrid task division strategy, aiming to improve the efficiency and effectiveness of parallel fuzzing. Meanwhile, in the task merging phase, we design a comprehensive information collection mechanism that can effectively analyze task results, and in the task division phase, we can further iterate the hybrid task division strategy based on real-time testing feedback. We implement PFuzzer according to the proposed scheme. In our evaluation experiments, PFuzzer successfully discovered vulnerabilities in carefully fuzzed programs that baseline fuzzers failed to detect. Furthermore, on the Fuzzbench test suite, PFuzzer discovered more code branches in most programs. Most importantly, the PFuzzer framework demonstrates good scalability, effectively parallelizing different base fuzzing engines.

Reactive synthesis is an automated procedure to obtain a correct-by-construction reactive system from its temporal logic specification. Despite significant research progress in the past decades, reactive synthesis is still in early-stage of use. Previous studies found that the lack of development methods for reactive synthesis specifications is one barrier to its wider adoption. In this paper, we adapt two development methods, an incremental method and a modular method, to the context of reactive synthesis specifications. The methods are based on existing software development methods on the one hand and studies about reactive synthesis on the other hand. Then, we report on an exploratory case study in which participants developed specifications using the two methods. We evaluated the methods using a mixed-method analysis that combines grounded theory analysis of Slack communication with participants, quantitative exploratory data analysis of the synthesis IDE usage logs, and qualitative independent expert review of the final specifications. Our findings show clear benefits of modular specification development in terms of ease of planning, synthesis time, fewer unrealizability issues, and faster debugging. However, the incremental development method was more natural and easy to use, and specifications developed incrementally were also easier to validate during the development process.