ICAM-Trans: Implicit-Context-Aware Multi-Agent Framework for Function-Level Code Translation

Generative AI systems based on Large Language Models (LLMs) have demonstrated remarkable capabilities in software engineering tasks, from code generation to natural language processing. However, a significant gap persists between curated demonstrations and production-grade deployment. This technical report presents a practitioner-driven analysis of LLM limitations encountered during sustained, real-world use across software development workflows. Drawing from multiple case studies—including natural-language-driven development (“vibe coding”), API integration, multi-file refactoring, and general-purpose question answering—we identify and taxonomize four critical failure modes: (1) the Complexity Cliff, where LLM performance degrades non-linearly as task interdependency grows; (2) Context Window Blindness, where finite attention spans cause silent contract violations across distributed codebases; (3) the Memory Illusion, where session discontinuity erases accumulated architectural knowledge; and (4) Confident Hallucination, where models generate plausible but fabricated outputs indistinguishable in tone from correct ones. We formalize the Verification Paradox—an inverse relationship between a user’s need for AI assistance and their capacity to validate its outputs—and propose a practical five-strategy framework for effective human–AI collaboration in software engineering contexts. We further introduce the concept of Contextual Reasoning Failure, evidenced by cases where LLMs optimize for literal query patterns while ignoring situational logic obvious to any human observer. Our findings suggest that current LLMs, while powerful pattern-matching engines, lack the contextual reasoning, persistent memory, and epistemic self-awareness necessary for reliable autonomous operation, and that practitioner expertise remains the critical safeguard against AI-induced defects in production systems.

With the rise in using Large Language Models (LLMs), research proved the feasibility of using LLMs in various software engineering tasks including code generation. However, it remains a major challenge to ensure the correctness and the efficiency of the LLM-generated code. This paper applies the "Chain-of-thought" prompting approach to improve GPT performance in code generation and its ability to convert coding requirements from HackerRank and LeetCode online coding platforms into correct Python, C++, and JavaScript code. We evaluated the performance of GPT-3.5 and GPT-4 across 558 programming challenges on HackerRank and LeetCode platforms. Our evaluation framework employed a comprehensive test case validation system, where each generated solution was tested against predefined input-output pairs, with success determined by exact matches between expected and actual outputs. The evaluation revealed significant variations in success rates across platforms (HackerRank: 46.83% vs. LeetCode: 7.35%) and programming languages (Python leading with 28.6% overall success rate). These results demonstrate that while the chain of thought approach enhances code generation accuracy, particularly for structured programming tasks, significant challenges remain in handling complex algorithmic problems.

Software development involves collaborative interactions where stakeholders express opinions across various platforms. Recognizing the sentiments conveyed in these interactions is crucial for the effective development and ongoing maintenance of software systems. For software products, analyzing the sentiment of user feedback, e.g., reviews, comments, and forum posts can provide valuable insights into user satisfaction and areas for improvement. This can guide the development of future updates and features. However, accurately identifying sentiments in software engineering datasets remains challenging. This study investigates bigger large language models (bLLMs) in addressing the labeled data shortage that hampers fine-tuned smaller large language models (sLLMs) in software engineering tasks. We conduct a comprehensive empirical study using five established datasets to assess three open source bLLMs in zero-shot and few-shot scenarios. Additionally, we compare them with fine-tuned sLLMs, using sLLMs to learn contextual embeddings of text from software platforms. Our experimental findings demonstrate that bLLMs exhibit state-of-the-art performance on datasets marked by limited training data and imbalanced distributions. bLLMs can also achieve excellent performance under a zero-shot setting. However, when ample training data are available or the dataset exhibits a more balanced distribution, fine-tuned sLLMs can still achieve superior results.

The paper studies the problem of software code migration during the transition from the old version of the SAP ERP 6.0 system to modern ERP platforms, in particular SAP S/4HANA. The necessity of automation of the migration process using artificial intelligence methods and tools is substantiated. A comprehensive analysis of existing academic and industrial case studies confirmed the feasibility of using large language models for code migration. A number of advantages of the approach based on large language models are revealed, in particular, the ability to handle various migration scenarios. The key risks and limitations common in the enterprise environment are highlighted, including data security, privacy, high cost of API calls and limited knowledge of models regarding the subject area. A method is developed for selecting a large open source language model that can be safely used in the enterprise landscape, thereby mitigating the risks associated with cloud solutions. The method is based on the developed system of criteria, including: model size, context window size, license type, support for fine-tuning training, focus on code quality (measured using a reference dataset), support for model quantization and community support. The hybrid AHP-TOPSIS method is proposed for systematic ranking of candidate models. The AHP method was used to check the weights of the criteria, while TOPSIS was used to rank the models based on their proximity to the ideal solution. Based on the developed method, three large language models were selected: Qwen 2.5 Coder 14B, DeepSeek-Coder-V2 16B and Llama 3.1 8B. A comprehensive testing plan for the selected models was developed for further experimental studies. The scientific novelty of the work lies in the development of a method for selecting large language models for custom code migration tasks, which differs from the existing system of model selection criteria and the use of a hybrid AHP-TOPSIS approach for ranking candidate models. The practical value of the work is that the developed method allows you to select a model taking into account the constraints of the corporate environment and information security requirements, which will increase the level of automation in the migration of AVAP when switching to new versions of ERP systems from SAP SE. The developed method can be applied to the selection of large language models when upgrading any large computer systems developed in both open source and proprietary programming languages.

Code summarization is an important task in software engineering that helps developers understand and maintain code by generating natural language summaries. Existing approaches predominantly rely on single models, facing a dilemma: directly deploying large language models (LLMs) incurs high training costs, while lightweight models specialized for summarization are constrained by the quality of training data and their ability to capture the complex structural semantics of code. This highlights the urgent need for synergistic collaboration between large and small models in cloud computing environments. To address these issues, this paper proposes a cloud-assisted code summarization framework. First, we achieve code enhancement by invoking cloud-deployed LLM services. The specific workflow involves using preset prompt templates to guide the model in evaluating code quality and automatically repairing defects based on its feedback, thereby constructing high-quality datasets Java-QE and Python-QE. Second, for efficient edge deployment, we introduce HiSum: AST Hierarchy-Aware Code Summarization model, a lightweight model. HiSum transforms code AST into Directed Syntax Graphs (DSG) to preserve structural semantics, encodes them via a directed graph convolutional network and decode to improve summary quality. Experimental results show that our framework significantly enhances code summarization performance. On the constructed Java-QE and Python-QE datasets, the HiSum model achieves notable improvements over state-of-the-art baselines in BLEU, METEOR, and ROUGE-L metrics (increases of 1.06%, 1.98%, 3.12% for Java-QE, and 1.46%, 3.24%, 2.20% for Python-QE, respectively). This research provides a solution that utilizes cloud LLM-assisted data enhancement to empower a lightweight hierarchical-aware model.

Generative AI in cybersecurity: A comprehensive review of LLM applications and vulnerabilities

Automated code translation is vital for software modernisation.While large language models (LLMs) show promise, existing methods often use uniform strategies for all code complexities and lack specialised error handling.We present AdaptTrans, a method featuring complexity-aware strategy selection and iterative error refinement.It first assesses code translation task complexity: simple tasks receive direct translation, while complex tasks employ specific enhancement strategies.An error-driven iterative refinement system then identifies error types and applies targeted optimisations.Evaluated on CodeNet and TransCoder benchmarks across multiple programming languages, AdaptTrans demonstrates significant improvements in translation accuracy over state-of-the-art baselines.The complexity-aware mechanism reduced average iteration count for simple translations by 8.5% while improving pass rate by 4.1%, confirming enhanced efficiency and effectiveness.

Programming language translation is essential in modern software development, facilitating cross-platform compatibility and the adaptation of legacy systems. This study examines the performance of large language models (LLMs), such as ChatGPT, in Python-to-Java code translation. Using a dataset of ten diverse algorithmic problems and advanced prompt engineering techniques, we evaluate the models’ effectiveness in maintaining computational accuracy (CA) and preserving method correctness (PMC). Results indicate that LLMs perform well on standard tasks but encounter challenges in complex scenarios involving advanced data structures and recursion. These findings uncover the potential of LLMs in code translation while highlighting the need for improved prompt strategies and domain-specific fine-tuning for complex tasks.

Failure localization in microservice environments is increasingly challenging. While large language models (LLMs) have shown promise in software engineering tasks, existing approaches struggle to effectively integrate multi-modal telemetry data (e.g., log, metric and trace) and provide interpretable results. This paper presents LocaleXpert, a novel failure localization system that combines specialized LLM-based agents with traditional AIOps methods to diagnose issues in microservice environments. LocaleXpert introduces three key innovations: (1) a modular pipeline that transforms metrics, logs, and traces into natural language descriptions that LLMs can effectively process, (2) specialized expert agents that analyze each data type and collaborate to identify root causes, and (3) an interpretation mechanism that produces clear, actionable explanations of its reasoning process. Evaluation results show that LocaleXpert significantly outperforming baseline approaches in both accuracy and interpretability. The system has been successfully deployed in Microsoft's AIOpsLab benchmark, demonstrating its effectiveness.

Artificial Intelligence (AI) driven code generation tools are increasingly used throughout the software development lifecycle to accelerate coding tasks. However, the security of AI-generated code using large language models (LLMs) remains underexplored, and recent studies have revealed various risks and weaknesses. This paper presents a measurement study of LLM-generated code across four programming languages (Python, Java, C++, and C) and five widely used LLM families. We construct a manually curated dataset of 200 programming tasks, grouped into seven functional and security-relevant categories, each with language-neutral specifications. For every combination of task, language, and model, we generate code and evaluate it along three axes: syntactic validity and compilation success, semantic correctness using 4,000 per program unit test files, and software quality and security using SonarQube and CodeQL, complemented by manual review of key static analysis findings. Our results show clear language effects: Python and Java achieve higher compilation and semantic correctness rates and produce fewer security findings than C and C++, where we observe more memory safety issues, hard-coded secrets, and cryptographic misuses. We also find that many models fail to make use of modern security features available in recent compiler and toolkit updates (i.e., in Java 17), and that outdated methods remain common, particularly in C++. These findings highlight the need to advance LLMs so that they better align with emerging secure coding practices and language-specific best practices. All code and data are available at GitHub.

Self-bootstrapping automated program repair: using LLMs to generate and evaluate synthetic training data for bug repair

Automating hardware design could obviate a signif-icant amount of human error from the engineering process and lead to fewer errors. Verilog is a popular hardware description language to model and design digital systems, thus generating Verilog code is a critical first step. Emerging large language models (LLMs) are able to write high-quality code in other programming languages. In this paper, we characterize the ability of LLMs to generate useful Verilog. For this, we fine-tune pre-trained LLMs on Verilog datasets collected from GitHub and Verilog textbooks. We construct an evaluation framework comprising test-benches for functional analysis and a flow to test the syntax of Verilog code generated in response to problems of varying difficulty. Our findings show that across our problem scenarios, the fine-tuning results in LLMs more capable of producing syntactically correct code (25.9% overall). Further, when analyzing functional correctness, a fine-tuned open-source CodeGen LLM can outperform the state-of-the-art commercial Codex LLM (6.5% overall). We release our training/evaluation scripts and LLM checkpoints as open source contributions.

Generative artificial intelligence (AI) has greatly advanced the code translation process, particularly through large language models (LLMs), which translate source code from one programming language into another. This translation has historically been error-prone, labor-intensive, and highly dependent on manual intervention. Although traditional tools such as compilers and transpilers have restrictions in managing complex programming paradigms, recent generative AI models most importantly those based on transformer architectures, have shown promise. This systematic mapping study intends to evaluate and compile studies on Generative AI applications in code translation released between 2020 and 2025. Using five main criteria, the publication year and channel, research type, publication type, empirical study type, and AI models. A total of 53 relevant articles were chosen and examined. The results show that conferences and journals are the most often used publishing venues. Although historical-based evaluations and case studies were the empirical methodologies most often used, researchers have primarily focused on implementing transformer-based artificial intelligence models.

The advent of large language models (LLMs) has revolutionized the field of code translation, enabling automated translation between programming languages. Despite these advancements, the accuracy and reliability of these models often falter in complex translation tasks due to a lack of contextual understanding. This paper introduces a novel approach to enhance code translation through Few-Shot Learning augmented with retrievalbased techniques. By leveraging a repository of existing code translations, we dynamically retrieve the most relevant examples to guide the model in translating new code segments. Our method, based on Retrieval-Augmented Generation (RAG), significantly improves translation quality by providing contextual examples that the model can learn from in real-time. We chose RAG over traditional fine-tuning methods due to its ability to leverage existing codebases or a locally stored corpus of code, allowing it to dynamically adapt to diverse translation tasks without the need for extensive retraining. Extensive experiments on diverse datasets, using open LLM models such as Starcoder, Llama3-70B Instruct, CodeLlama-34B Instruct, Granite-34B Code Instruct, and Mixtral-8x22B, and commercial LLM models such as GPT-3.5 turbo, and GPT-4o demonstrate the superiority of our approach over traditional zero-shot, particularly in translating between Fortran and C++.We also explored different numbers of shots (examples provided to the model during inference)specifically 1, 2, and 3 shots -and various embedding models for RAG, including Nomic-Embed, Starencoder, and CodeBERT, to evaluate the robustness and effectiveness of our approach.

Large Language Models (LLMs) became wide-spread during only the last couple of years and are used in almost every scientific and non-scientific domain. Understanding opportunities, applications and limitations of LLMs is crucial for a risk-free, effective and useful implementation of LLMs into scientific workflows. We demonstrate that their effectiveness is maximized not through autonomous operation but through careful integration with specialized tools and contextual knowledge bases. Using local deployments of modern LLMs (QWen-2.5-coder, LLaMA, Mistral) comes with a number of benefits in a scientific context. Our approach employs vector embeddings for enhanced context retention and metadata databases for structured data access, enabling guided, context-aware interactions with the LLM. Local deployments allow for improved data handling and privacy, improved cost management and a higher degree of customization. Energy consumption can more easily be observed and managed, which can be a crucial property of such a system, especially compared to the newest generation of LLMs, which have extensive power  (and cost) requirements.Opportunities and limitations are explored through two case studies: (1) an LLM-driven system that queries metadata databases to retrieve data from common open data sources and harmonizes patio-temporal subsets into data-cubes, and (2) a VBA-to-Python code translation project to preserve a legacy selection-system forest management software, which was developed in ACCESS / VBA over more than two decades. The LLM's translation process and reasoning are preserved in a vector database for consistent context maintenance and the original as well as the ‚new‘ code is searchable using the LLM to aid rebuilding a modernized software. Results suggest that this tool-augmented approach leads to a more reliable and maintainable solution compared to purely LLM-driven implementations, suggesting a new paradigm for integrating AI in scientific workflows where LLMs rather facilitate than replace domain-specific tools and human expertise.