Residual investigation: predictive and precise bug detection

Soundness and completeness are two primary concerns of a static analysis tool for finding defects in software. Exhaustive static analysis of the program through all paths is not always possible, especially for a large software causing incompleteness in the analysis. Also, exhaustive testing of the program to detect all bugs is not possible. In this work, we describe a technique which uses coverage data from testing to remove the tested paths and then statically analyzes the remaining code. This pruning of tested paths allows a static analyzer to perform a more thorough analysis of the reduced code, thereby improving its effectiveness. This work is a step towards integration of static analysis and testing frameworks. The proposed technique is applied with a few static analyzers publicly available. Our experience shows that the approach results in lesser false positives as well as detection of more serious errors which might have gone unnoticed otherwise.

A typestate property describes which operations are available on an object or a group of inter-related objects, depending on this object’s or group’s internal state, the typestate. Researchers in the field of static analysis have devised static program analyses to prove the absence of typestate-property violations on all possible executions of a given program under test. Researchers in runtime verification, on the other hand, have developed powerful monitoring approaches that guarantee to capture property violations on actual executions. Although static analysis can greatly benefit runtime monitoring, up until now, most static analyses are incompatible with most monitoring tools. We present Clara, a novel framework that makes these approaches compatible. With Clara, researchers in static analysis can easily implement powerful typestate analyses. Runtime-verification researchers, on the other hand, can use Clara to specialize AspectJ-based runtime monitors to a particular target program. To make aspects compatible to Clara, the monitoring tool annotates them with so-called dependency state machines. Clara uses the static analyses to automatically convert an annotated monitoring aspect into a residual runtime monitor that is triggered by fewer program locations. If the static analysis succeeds on all locations, this proves that the program fulfills the stated typestate properties, making runtime monitoring entirely obsolete. If not, the residual runtime monitor is at least optimized. We instantiated Clara with three static typestate analyses and applied these analyses to monitoring aspects generated from tracematches. In two-thirds of the cases in our experiments, the static analysis succeeds on all locations, proving that the program fulfills the stated properties, and completely obviating the need for runtime monitoring. In the remaining cases, the runtime monitor is often significantly optimized.

This paper presents an approach to enforce information flow policies using a multi-valued type-based analysis followed by an instrumentation when needed. The target is a core imperative language. Our approach aims at reducing false positives generated by static analysis, and at reducing execution overhead by instrumenting only when needed. False positives arise in the analysis of real computing systems when some information is missing at compile time, for example the name of a file, and consequently, its security level. The key idea of our approach is to distinguish between negative and may responses. Instead of rejecting the possibly faulty commands, they are identified and annotated for the second step of the analysis; the positive and negative responses are treated as is usually done. This work is a hybrid security enforcement mechanism: the maybe-secure points of the program detected by our type based analysis are instrumented with dynamic tests. The basic type based analysis has been reported in [6], this paper presents the modification of the type system and the newly presented instrumentation step. The novelty of our approach is the handling of four security types, but we also treat variables and channels in a special way. Programs interact via communication channels. Secrecy levels are associated to channels rather than to variables whose security levels change according to the information they store. Thus the analysis is flow-sensitive.KeywordsSecurity LevelInference AlgorithmDownward FlowChannel VariableTyping RuleThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Bulk foam analysis (static test) is simple and fast, which makes it a cost-effective method for screening and ranking hundreds of surfactants being considered for foam applications. Coreflood tests (dynamic test) can also be used, but it is quite laborious and costly. However, previous reports show that ranking based on static tests sometimes differs from ranking based on dynamic tests. To date, the reason for such a discrepancy is not well understood. Some believe that it may be due to faulty experimental design while some others believe that there is no discrepancy if the right foam performance indices are used to describe and compare the results from both methods. For the first time, this study reports a systematic series of static tests conducted on different foaming solutions (with surfactant concentration ranging from 0.025 to 5 wt%) and duplicated in dynamic tests using the same core sample for all the surfactant solutions. The dynamic test was also repeated on three different rocks of a wide permeability range (26–5000 mD) for each of the surfactant solutions. Unlike previous studies, here multiple dynamic foam indices (limiting capillary pressure, apparent viscosity, trapped foam, and trapped to mobile foam ratio) were measured and compared with the performance indices measured from the static tests (foam texture and foam half-life). Dynamic tests were in total agreement with static tests for all the foam formulations. However, it was observed that the pore size of the base filter disk used in the static foam analyzer can be a potential source of conflicting results when comparing with dynamic test. This is because a threshold pore size exists above which some foam properties (apparent viscosity and trapped foam) significantly decreased compared to the properties before that threshold. Foam limiting capillary pressure is the only foam property that does not show such a trend. It also appears that such threshold occurs above a certain surfactant concentration (0.025 wt%). Apparently, it becomes imperative that the pore size of the filter disk used in the static test and the porous medium used in dynamic tests must be on the same side of the threshold point, otherwise there may be disparity in their results. The threshold surfactant concentration should also be determined. The role of these two factors (pore size and surfactant concentration) requires further investigation.

Assertions are helpful in program analysis, such as software testing and verification. The oracles encoded in the assertions help detect potential flaws and release engineers from the manually check of the reported weaknesses, which is error-prone, burdensome and time-consuming. While in practice, few engineers would write assertions during programming, and it is challenging to generate assert statements, and insert them into proper locations automatically. In this paper, we propose a statically directed assertion recommendation approach for C programs. It combines static analysis, dynamic testing, and program verification to automatically recommend and validate weakness-oriented assertions, which is defined as an assert statement used to detect program weaknesses. Firstly, we integrate a static analysis tool such as FlawFinder and some learned patterns about CWE (Common Weakness Enumeration) to report potential program flaws. Secondly, we insert the corresponding assertions into the suspicious locations of those flaws. Then, we validate the program inserted with the assertions through two methods, the first is to execute the code with some test cases generated by automatic test-case generators such as Klee and Dart, and the second is to verify the program with some automatic verifier such as CPAchecker and Smack. Finally, we report on whether those flaws could be a real weakness. Experimental results show that our approach helps to find 125 real weaknesses in open source software from Github. Furthermore, our performance in detecting static analysis' true positives can reach 81.42%.

Automated Program Repair (APR) tools have advanced in recent years, yet their effectiveness can improve when integrated into Continuous Integration/Continuous Deployment (CI/CD) pipelines. Motivated by this, we designed the Continuous Automatic Repair Framework (CARF), which detects build failures, routes each bug to the most suitable repair tool, and automatically commits the generated fix. During development, we identified a critical bottleneck: the need for instant bug classification within CI/CD. Accurate classification is essential to determine whether a fault lies in program code or the test suite, ensuring the defect is routed to the appropriate repair tool. Objectives: This study aims to design and evaluate a just-in-time bug classifier capable of distinguishing between program and test bugs during CI/CD execution, thereby enabling CARF to maintain workflow efficiency by directing defects to appropriate repair tools. We implemented a heuristic analysis tool that extracts discriminative structural features by comparing Abstract Syntax Trees (ASTs) between buggy and pre-buggy commits. These features are input to machine learning models for bug classification. Empirical validation demonstrated the accuracy and operational feasibility of the approach within CI/CD environments. Our approach achieved up to 73% accuracy in identifying regression bugs across 67 real-world projects, effectively distinguishing between program bugs and test bugs while requiring only 10% of the dataset for training. In contrast, prior studies reported an accuracy of 69% on artificially injected bugs derived from successive versions of only two projects, with 90% of the data used for training. CARF facilitates the integration of automated program repair into CI/CD pipelines, enabling faster, more accurate, and more flexible software maintenance. The just-in-time bug classifier demonstrates that defect classification can scale efficiently without symbolic execution or manual bug reports, providing a practical foundation for continuous automated repair. • Designed an orchestration layer to unify heterogeneous automated repair tools within CI/CD pipelines. • Identified the need for a bug classifier to distinguish between program and test bugs at build failure. • Developed a lightweight, heuristic-based bug classifier for fast and reliable bug differentiation.

Leveraging and acting on the results of static analysis is a well-known best practice. Static analysis also is an integral part of software quality assurance which is highlighted by the inclusion of static analysis results in software quality models like ColumbusQM and Quamoco. Although there are studies that explore if tools for static analysis are used and how they are configured, few publications explore the longitudinal effects of acting on static analysis results on software and on the evolution of software. Especially effects on quality criteria, e.g., software quality metrics, defects, or readability are missing. With our research, we will bridge this gap and measure the effects of static analysis on software quality evolution. We will measure the effect the removal of code that generates static analysis warnings has on software quality metrics. Furthermore, we will measure long term effects on external quality attributes, e.g., reported issues and defects. Finally, we want to predict false positives of static analysis warnings by training predictive models on our collected data.

Recent research proposed efficient methods for software verification combining static and dynamic analysis, where static analysis reports possible runtime errors (some of which may be false alarms) and test generation confirms or rejects them. However, test generation may time out on real-sized programs before confirming some alarms as real bugs or rejecting some others as unreachable.To overcome this problem, we propose to reduce the source code by program slicing before test generation. This paper presents new optimized and adaptive usages of program slicing, provides underlying theoretical results and the algorithm these usages rely on. The method is implemented in a tool prototype called sante (Static ANalysis and TEsting). Our experiments show that our method with program slicing outperforms previous combinations of static and dynamic analysis. Moreover, simplifying the program makes it easier to analyze detected errors and remaining alarms.

Embedded software often involves intensive numerical computations and suffers from a number of runtime errors. The technique of numerical static analysis is of practical importance for checking the correctness of embedded software. However, most of the existing approaches of numerical static analysis consider sequential programs, while interrupts are a commonly used facility that introduces concurrency in embedded systems. Therefore, a numerical static analysis approach is highly desired for embedded software with interrupts. In this article, we propose a static analysis approach specifically for interrupt-driven programs based on sequentialization techniques. We present a method to sequentialize interrupt-driven programs into nondeterministic sequential programs according to the semantics of interrupts. The key benefit of using sequentialization is the ability to leverage the power of state-of-the-art analysis and verification techniques for sequential programs to analyze interrupt-driven programs, for example, the power of numerical abstract interpretation to analyze numerical properties of the sequentialized programs. Furthermore, to improve the analysis precision and scalability, we design specific abstract domains to analyze sequentialized interrupt-driven programs by considering their specific features. Finally, we present encouraging experimental results obtained by our prototype implementation.

Programming errors in traversal programs over structured data

The biggest problem in computer security is that most systems aren't constructed with security in mind. Being aware of common security weaknesses in programming might sound like a good way to avoid them, but awareness by itself often proves to be insufficient. Understanding security is one thing and applying your understanding in a complete and consistent fashion to meet your security goals is quite another. For this reason, static analysis is advocated as a technique for finding common security errors in source code. Manual security static analysis is a tedious work, so automatic tools which can guide programmers to detect security concerns is suggested. Good static analysis tools provide a fast way to get a detailed security related evaluation of program code. In this paper a new architecture (CBRFD) for software flaw detector, based on the concept of clone detection and case base reasoning, is proposed and various issues which concern detection of security weakness of codes through code clone detector is investigated.

Programming tools have expanded both in scope of the problems they solve, and in the kinds of techniques they use. Traditionally, programming tools have focused on detecting errors in programs. Recently, this scope has broadened to help with other programming tasks, including inferring specifications, helping diagnose root cause of errors during debugging, and managing knowledge in large projects. Also, traditionally programming tools have been based on either static or dynamic program analysis. Modern programming tools combine static and dynamic program analysis together with techniques from other disciplines such as statistical and probabilistic inference, and information retrieval. This paper reports on some such tools built by the Rigorous Software Engineering group at Microsoft Research India.

Software fault injection techniques have been largely used as means for evaluating the dependability of systems in presence of certain types of faults. Despite the large diversity of tools offering the possibility of emulating the presence of software faults, there is little practical support for emulating the presence of software faults in Python applications, which are increasingly being used to support business critical cloud services. In this paper, we present FIT4Python, a tool for injecting software faults in Python code and then use it, in a mutation testing campaign, to analyse the effectiveness of OpenStack’s test suite against new probable software faults. We begin by analysing the types of faults affecting Nova Compute, the core component of OpenStack. We use our tool to emulate the presence of new faults in Nova Compute API to understand how well OpenStack’s battery of unit, functional, and integration tests cover these new, but probable, situations. Results show clear limitations in the effectiveness of OpenStack developers’ test suites, with many cases of injected faults passing undetected through all three types of tests and that nearly half of the analysed problems could be detected with trivial changes or additions to the unit tests.

In today’s era of Generative AI, the Large Language Model (LLMs)applications are largely used to generate functioning source code. These applications simply does not require any programming skill except a prompt. However, the LLM generated codes are prone to safety and efficiency concerns. A common safety concerns in popular programming languages source code like Java, C++ and C are SQL injection, pointer dereferencing, improper input validation, etc. There are several academic studies and industry approaches to enable a strong and safe AI code generation such as prompt inversion, formal verification, and many more. In this paper, we analyze both the industry and academic approach to reduce and fix the threats over AI generated code and discussed key findings. The platform like Veracode that uses static analysis, dynamic testing and the platform CrowdStrike working to develop self-learning, multi-agent AI systems that employ Red Teaming capabilities. The academic benchmark like CodeLMSec that proposed black-box vulnerability probing framework, FormAI-v2 applies a large-scale formal verification for C language vulnerabilities, and the CodeSecEval benchmark that is based on prompt inversion to evaluate secure code generation and repair. Our studies on the mentioned benchmark highlights that the currently LLM generated source code persists vulnerabilities due to lack of benchmark dataset. The unsafe code fixes are approached in one direction only, either based on static analysis or dynamic analysis.

Fuzzers are often considered one of the most effective tools to uncover bugs in software, including security vulnerabilities. Current tools are designed to test the full programs by employing strategies that select and generate testcases that increase code coverage. In this paper, we propose to evolve these ideas to make fuzzers focus on specific portions (i.e., lines) of the code, which we call targets, avoiding wasting resources on the other regions. The aim is to support efficient testing of evolving software, and other practical scenarios such as confirming the results from static analysis.