Despite the rapid progress in Generative AI, building applications that fulfill complex demands still requires the manual orchestration of multiple AI models into workflows, which is time-consuming and error prone. Existing large language model (LLM)-based automation methods suffer from planning brittleness: they often produce infeasible plans due to an unstructured understanding of model capabilities, and their tightly-coupled architectures are vulnerable to rapid changes in the underlying model ecosystem. To address this, we present AgencyTree , a novel multi-agent framework that introduces a formal Abstract Capability Tree (ACT) as a shared ontological knowledge base. This tree hierarchically abstracts concrete AI models into canonical functional nodes, decoupling high-level task planning from low-level model execution. Built upon this abstraction, five specialized agents - Knowledge Engineer, System Analyst, System Architect, Project Engineer, and Intent Consistency Verifier-collaborate to automate the workflow lifecycle. Evaluated on a comprehensive benchmark, AgencyTree achieves significantly higher task success and output quality than state-of-the-art baselines. Crucially, the framework exhibits exceptional robustness to model registry changes, with over 90% of abstract workflows being fully reusable without modification, thereby reducing adaptation overhead by orders of magnitude compared to replanning from scratch. This work establishes a new paradigm for building reliable and dynamically-adaptive AI assembly systems.

Mobile application accessibility is crucial for digital inclusion. This paper presents a systematic literature review (SLR) following PRISMA guidelines, synthesizing advancements in accessibility issue detection and repair techniques. We analyze representative mobile accessibility tools and industrial practices to complement SLR evidence. Nine major databases (ACM DL, IEEE Xplore, ScienceDirect, SpringerLink, Wiley, Web of Science, Scopus, CNKI, arXiv) were searched, with 76 high-quality studies published up to July 2025 analyzed. Common accessibility issues are categorized into four types—perceptibility, operability, understandability, and robustness—aligning with user capability modeling. In accessibility issue detection, evolution from static analysis to deep learning and large language models (LLMs) has shifted research from rule-based matching to semantic reasoning, enhancing automation and generalization. However, detection and repair remain loosely coupled and fragmented, with high false-positive rates and weak inter-module feedback loops. For repair, we propose a taxonomy of rule-driven, learning-based, and LLM-assisted approaches, revealing a gap between detection breadth and repair research depth. LLMs show potential for semantic understanding, issue reasoning, and repair generation, paving the way toward intelligent agents for accessibility. Looking ahead, future research should focus on semantic-enhanced detection, multimodal repair, refined user capability modeling, and unified evaluation standards—collectively advancing accessibility engineering toward an accessibility-by-default design paradigm.

With the rapid advancement of large language models (LLMs), code translation has become a critical yet challenging task in software engineering. Existing evaluations of this task remain unreliable due to two key limitations: the absence of specialized function-level code translation benchmarks and flawed evaluation pipelines. Specifically, current pipelines lack post-processing mechanisms to extract target code from LLMs’ inherently unstable output formats, directly resulting in non-executable translations and false negatives in evaluation. To address these issues, we introduce FuncTransEval, an enhanced multilingual benchmark. It comprises 400×4 functions across four programming languages (Python, C++, Java, and JavaScript) and 12 bidirectional translation pairs, each paired with executable unit tests for fine-grained functional validation. Using FuncTransEval, we systematically assess LLMs’ true capabilities in code translation and identify their core limitations: inadequate understanding of the implicit information such as function logic, type specifications, and cross-language differences. To mitigate these shortcomings, we propose ICAM-Trans (Implicit-Context-Aware Multi-Agent Framework for function-level code translation). ICAM-Trans is a hierarchical, autonomous multi-agents architecture that explicitly uncovers and leverages these implicit contextual information during translation. It employs a Translation Orchestrate Agent (TOA) to autonomously coordinate four specialized Context Analysis Agents (CAAs), which conduct pre-translation analyses on different aspects. Insights from these analyses guide a Context-Aware Translation Agent (CTA) to generate semantically faithful target code. Experiments on FuncTransEval demonstrate that ICAM-Trans consistently outperforms strong baselines, validating its effectiveness in achieving high-fidelity and interpretable function-level code translation.

The conformance testing of programming language implementations is crucial to support correct and consistent execution environments. Researchers often use coverage information in the mechanized language specification to generate or check the quality of conformance tests. Since specifications use inductive definitions to describe the semantics of language features, traditional graph coverage criteria for software can still be applied. However, they may not produce high-quality conformance tests because language implementations often have specialized execution paths for different features, even when their semantics descriptions use the same functions. Traditional graph coverage may not distinguish test requirements of such language features, degrading conformance testing quality. Similarly, it may not distinguish test requirements of different parts of the same language feature if their semantics use the same functions. We introduce feature-sensitive (FS) coverage as a novel coverage criterion for generating high-quality conformance tests for language implementations. The core idea is to enhance traditional graph coverage by incorporating the innermost enclosing language features. To further improve the quality of conformance tests, we extend this approach to feature-call-path-sensitive (FCPS) coverage and its \(k\) -limiting variant. To assess the effectiveness of the new coverage criteria, we apply them to the mechanized JavaScript specification for ES13 and extend JEST, a state-of-the-art JavaScript conformance test synthesizer. Using five coverage criteria, our tool synthesizes 237,981 conformance tests for ES13 in 50 hours. The tool detected 157 conformance bugs (45 in engines and 112 in transpilers), 139 confirmed by the developers, and 136 newly discovered bugs. However, a higher \(k\) value may lead to an excessive number of tests. To resolve this issue, we present a selective FS/FCPS that utilizes the key feature stacks for the target language implementation. The selective approach for transpiler conformance testing with ECMA-262 for ES15 successfully reduces the number of synthesized tests by 70.7% on average. Still, it detects 53 bugs compared to 57 bugs detected by the non-selective approach.

Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair benchmark of ULT with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research 1 . ULT is available at https://github.com/huangd1999/UnLeakedTestBench .

In software quality assurance, Just-In-Time Defect Prediction (JIT-DP) and Just-In-Time Defect Localization (JIT-DL) automate the identification of potential defective code and localization of defect lines by analyzing program change characteristics and code commit contexts. These techniques enables just-in-time quality monitoring and defect prevention during development, while also reducing manual costs and improving reliability. Due to the high dynamism and unstructured noise in code change data within agile development scenarios, as well as the difficulty in capturing complex nonlinear synergistic effects of multidimensional expert features, existing work faces dual challenges: lack of data robustness and lack of fine-grained modeling of expert features. To address these challenges, we propose a multi-task interaction technology JIT-DCK , which contains three core components: defect feature preprocessing component, defect prediction component, and defect localization component. The defect feature preprocessing component creates a subword vocabulary, controls extreme values and removes interference to generate data with three aspects, sending the line-level structure to the defect localization component and the semantic aggregation and expert features to the defect prediction component. The defect prediction component is structured with a three-layer cooperative architecture. It works on fusing semantic and line-level features and also models the complex nonlinear relationships of 14-dimensional expert features to achieve effective defect prediction. The defect localization component builds a closed-loop decision chain. It integrates commit-level global features and line-level local semantics through multi-level encoding and dynamic attention mechanisms for accurately localizing defects. Experiments show that JIT-DCK significantly outperforms SOTA models in defect prediction and localization metrics such as F1-Score +2.76%, AUC +1.79%, CodeChurn@20%Recall +2.88%, Top5-Acc +3.10%, Top10-Acc +2.55%, Recall@20%CodeChurn l +6.46%, CodeChurn@20%Recall l +18.58%, and IFA +3.57%, demonstrating its competence in code defect prediction and localization tasks.

Call graph construction serves as a foundational component in Python static analysis and software engineering. However, it faces significant challenges due to the language's unique characteristics. That is, the dynamic typing system, extensive use of third-party libraries, and runtime-dependent behaviors in Python create substantial obstacles for traditional static analysis methods. Although these methods typically achieve high precision, they suffer from low recall, making it difficult to strike a balance between soundness and precision. In this paper, we present InferCG , a hybrid framework that addresses these limitations by combining coarse-grained static analysis with large language model (LLM)–based semantic reasoning. InferCG first constructs a high-recall candidate call set using permissive name- and arity-based matching, and then employs powerful LLMs such as DeepSeek and Qwen to semantically filter implausible edges. This two-stage design balances recall and precision without requiring dynamic execution or extensive type annotations. Experiments on curated open-source Python projects and the DyPyBench benchmark demonstrate that InferCG outperforms state-of-the-art static tools, which achieves up to 13.9% improvement in average recall and a 5.0% gain in average F1 score over PyCG. Moreover, in a real-world vulnerable-function detection scenario, InferCG enables more accurate reachability analysis than dependency-based tools.

Recent advances in automated programming have the potential to reduce human involvement in the software engineering process, but this can lead to less trustworthy software. We envision a three-pronged approach to automating the engineering of trustworthy software that involves (1) eliciting requirements from users and automatically generating formal specifications encoding users’ intent, (2) automatically synthesizing source code conforming to those specifications, and (3) automatically synthesizing formal proofs to verify the correctness of the produced software. We describe this vision and the state of the art in each of these three areas, and the research challenges that must be overcome in each area and in their integration.

TEE-shielded secure inference offers an efficient solution to protect valuable edge-deployed models from potential thefts. Nevertheless, existing methods are lack of theoretical security analysis, failing to achieve the optimal security. Furthermore, while feasible for small models, existing methods are excessively heavyweight for Large Language Models (LLM). For LLaMA-7B, they introduce GB-level secure memory requirement and hundredfold inference latency, severely compromising real-time utility. To solve these problems, we first present a Bayesian theory framework of Model Stealing (MS) attacks, which decomposes MS into prior and posterior knowledge leakage. Based on this framework, LLMGuard is proposed, which presents two components: First, Intrinsic Parameters Shielding is designed to shield all private parameters, preventing prior knowledge leakage. This approach significantly decreases the secure memory usage and achieves inference speedup. Second, since OTP is not applicable to LLMs, Random Slices Composition is developed to obfuscate intermediate distributions with no computational overhead, minimizing posterior knowledge leakage efficiently. Experimental results demonstrate that LLMGuard downgrades model to black-box inference with negligible accuracy loss, while delivering \(43\times\) inference speedup on LLaMA compared to fully-shielded methods. The proposed LLMGuard effectively addresses concerns related to intellectual property theft on edge, boosting the secure deployment of LLMs on untrusted devices.

The Model Context Protocol (MCP) is an emerging open standard that defines a unified, bi-directional communication and dynamic discovery protocol between AI models and external tools or resources, aiming to enhance interoperability and reduce fragmentation across diverse systems. This paper conducts a systematic study of MCP from both architectural and security perspectives. We first define the full lifecycle of an MCP server, comprising four phases (creation, deployment, operation, and maintenance), further decomposed into 16 key activities that capture its functional evolution. Building on this lifecycle analysis, we construct a comprehensive threat taxonomy that categorizes security and privacy risks across four major attacker types: malicious developers, external attackers, malicious users, and security flaws, encompassing 16 distinct threat scenarios. To validate these risks, we develop and analyze real-world case studies that demonstrate concrete attack surfaces and vulnerability manifestations within MCP implementations. Based on these findings, the paper proposes a set of fine-grained, actionable security safeguards tailored to each lifecycle phase and threat category, offering practical guidance for secure MCP adoption. We also analyze the current MCP landscape, covering industry adoption, integration patterns, and supporting tools, to identify its technological strengths as well as existing limitations that constrain broader deployment. Finally, we outline future research and development directions aimed at strengthening MCP’s standardization, trust boundaries, and sustainable growth within the evolving ecosystem of tool-augmented AI systems. All collected data and implementation examples are publicly available at https://github.com/security-pride/MCP_Landscape .