Concurrency analysis in the presence of procedures using a data-flow framework

Multi-frontal based approach for concurrent finite element analysis

Data races are a prevalent class of concurrency bugs in shared-memory parallel programs, posing significant challenges to software reliability and reproducibility. While there is an extensive body of research on detecting data races and a wealth of practical detection tools across various programming languages, considerably less effort has been directed toward automatically fixing data races at an industrial scale. In large codebases, data races are continuously introduced and exhibit myriad patterns, making automated fixing particularly challenging. In this paper, we tackle the problem of automatically fixing data races at an industrial scale. We present Dr.Fix, a tool that combines large language models (LLMs) with program analysis to generate fixes for data races in real-world settings, effectively addressing a broad spectrum of racy patterns in complex code contexts. Implemented for Go—the programming language widely used in modern microservice architectures where concurrency is pervasive and data races are common—Dr.Fix seamlessly integrates into existing development workflows. We detail the design of Dr.Fix and examine how individual design choices influence the quality of the fixes produced. Over the past 18 months, Dr.Fix has been integrated into developer workflows at Uber demonstrating its practical utility. During this period, Dr.Fix produced patches for 224 (55%) from a corpus of 404 data races spanning various categories; 193 of these patches (86%) were accepted by more than a hundred developers via code reviews and integrated into the codebase.

Programming with tasks is a promising approach to write performance portable parallel code. In this model, the programmer explicitly specifies tasks and the task parallel runtime employs work stealing to distribute tasks among threads. Similar to multithreaded programs, task parallel programs can also exhibit data races. Unfortunately, prior data race detectors for task parallel programs either run the program serially or do not handle locks, and/or detect races only in the schedule observed by the analysis. This paper proposes PTRacer, a parallel on-the-fly data race detector for task parallel programs that use locks. PTRacer detects data races not only in the observed schedule but also those that can happen in other schedules (which are permutations of the memory operations in the observed schedule) for a given input. It accomplishes the above goal by leveraging the dynamic execution graph of a task parallel execution to determine whether two accesses can happen in parallel and by maintaining constant amount of access history metadata with each distinct set of locks held for each shared memory location. To detect data races (beyond the observed schedule) in programs with branches sensitive to scheduling decisions, we propose static compiler instrumentation that records memory accesses that will be executed in the other path with simple branches. PTRacer has performance overheads similar to the state-of-the-art race detector for task parallel programs, SPD3, while detecting more races in programs with locks.

In parallel programs concurrency bugs are often caused by unsynchronized accesses to shared memory locations, which are called data races. In order to support programmers in writing correct parallel programs, it is therefore highly desired to have tools on hand that automatically detect such data races. Today, most of these tools only consider unsynchronized read and write operations on a single memory location. Concurrency bugs that involve multiple accesses on a set of correlated variables may be completely missed. Tools may overwhelm programmers with data races on various memory locations, without noticing that the locations are correlated. In this paper, we propose a novel approach to data race detection that automatically infers sets of correlated variables and logical operations by analyzing data and control dependencies. For data race detection itself, we combine a modified version of the lockset algorithm with happens-before analysis providing the first hybrid, dynamic race detector for correlated variables. We implemented our approach on top of the Valgrind, a framework for dynamic binary instrumentation. Our evaluation confirmed that we can catch data races missed by existing detectors and provide additional information for correct bug fixing.

Programming environments for smartphones expose a concurrency model that combines multi-threading and asynchronous event-based dispatch. While this enables the development of efficient and feature-rich applications, unforeseen thread interleavings coupled with non-deterministic reorderings of asynchronous tasks can lead to subtle concurrency errors in the applications. In this paper, we formalize the concurrency semantics of the Android programming model. We further define the happens-before relation for Android applications, and develop a dynamic race detection technique based on this relation. Our relation generalizes the so far independently studied happens-before relations for multi-threaded programs and single-threaded event-driven programs. Additionally, our race detection technique uses a model of the Android runtime environment to reduce false positives. We have implemented a tool called DroidRacer. It generates execution traces by systematically testing Android applications and detects data races by computing the happens-before relation on the traces. We analyzed 15 Android applications including popular applications such as Facebook, Twitter and K-9 Mail. Our results indicate that data races are prevalent in Android applications, and that DroidRacer is an effective tool to identify data races.

<div class="section abstract"><div class="htmlview paragraph">Preventing systematic software failures is of paramount importance for any highly automatic vehicle control system, in particular for safety-critical AUTOSAR software. Among the most critical software defects are runtime errors like buffer overflows or data races. They may cause erroneous or erratic behavior, induce system failures, and constitute security vulnerabilities. Sound static analysis can be used to report all such defects in the code, or to prove their absence. It can also determine dependencies between software components and show freedom of interference without missing any data and control flow through data or function pointers. In the past, AUTOSAR projects often had to be decomposed or simplified to achieve satisfactory analysis time or memory consumption. Creating the analysis model, i.e., determining the tasks and ISRs to analyze, their priorities, synchronization, etc., required significant manual effort. In this article we present novel analysis concepts, developed in the Astrée analyzer, to support fully automatic integration analysis of AUTOSAR projects. The execution model is automatically derived from the ARXML specification. No changes to the software under analysis are required. All specified tasks and ISRs are analyzed assuming fully concurrent execution; runtime errors, data races and deadlocks are reported. Task priorities, core assignment, resources, spinlocks, and critical sections enclosed by primitives to disable/enable interrupts are automatically handled. To minimize false alarms, Astrée supports tuning the analyzer to the software under analysis by locally boosting analysis precision. We report on practical experience with real-life industry projects, giving an overview of alarm rates, analysis time and memory consumption, as well as reduction of false alarms by fine-tuning the analysis precision to the software under analysis.</div></div>

Concurrency analysis is a static analysis technique that determines whether two statements or operations in a shared memory program may be executed by different threads concurrently. Concurrency relationships can be derived from the partial ordering among statements imposed by synchronization constructs. Thus, analyzing barrier synchronization is at the core of concurrency analyses for many parallel programming models. Previous concurrency analyses for programs with barriers commonly assumed that barriers are named or textually aligned. This assumption may not hold for popular parallel programming models, such as OpenMP, where barriers are unnamed and can be placed anywhere in a parallel region, i.e., they may be textually unaligned. We present in this paper the first interprocedural concurrency analysis that can handle OpenMP, and, in general, programs with unnamed and textually unaligned barriers.We have implemented our analysis for OpenMP programs written in C and have evaluated the analysis on programs from the NPB and SpecOMP2001 benchmark suites.

Interrupt mechanisms are widely used to process multiple concurrent tasks in the software without OS abstraction layer in various cyber physical systems (CPSs), such as space flight control systems. Data races caused by interrupt preemption frequently occur in those systems, leading to unexpected results or even severe system failures. In recent Chinese space projects, many software defects related to data races have been reported. How to detect interrupt based data races is an important issue in the quality assurance for aerospace software. In this paper, we propose a tool named Race Checker that can statically detect data races for interrupt-driven software. Given the source code or binary code of interrupt-driven software, the tool aggressively infers information such as interrupts priority states, interrupt enable states and memory accesses at each program point using our extended interprocedural data flow analysis. With the information above, it identifies the suspicious program points that may lead to data races. Race Checker is explicitly designed to find data race bugs in real-life aerospace software. Up to now, the tool has been applied in aerospace software V&V and found several severe data race bugs that may lead to system failures.

With a growing demand of concurrent software to exploit multi-core hardware capability, concurrency vulnerabilities have become an inevitable threat to the security of today's IT industry. Existing concurrent program detection schemes focus mainly on detecting concurrency errors such as data races, atomicity violation, etc., with little attention paid to detect concurrency vulnerabilities that may be exploited to infringe security. In this paper, we propose a heuristic framework that combines both static analysis and fuzz testing to detect targeted concurrency vulnerabilities such as concurrency buffer overflow, double free, and use-after-free. The static analysis locates sensitive concurrent operations in a concurrent program, categorizes each finding into a potential type of concurrency vulnerability, and determines the execution order of the sensitive operations in each finding that would trigger the suspected concurrency vulnerability. The results are then plugged into the fuzzer with the execution order fixed by the static analysis in order to trigger the suspected concurrency vulnerabilities.

Shared-memory based parallel programming with OpenMP and Posix-thread APIs becomes more common to fully take advantage of multiprocessor computing environments. One of the critical risks in multithreaded programming is data races that are hard to debug and greatly damaging to parallel applications if they are uncaught. Although ample effort has been made in building specialized data race detection techniques, the state of art tools such as the Intel Thread Checker still have various functionality and performance problems. In this paper, we present a Versatile On-the-fly Race Detection (VORD) tool that provides an agile, efficient, and scalable race detection environment for various parallel programming models. VORD can automatically construct an empirically optimal set of race engines by utilizing classification and adaptation mechanisms. A Race-Detection Classification (RDC) table is created to categorize adequate engines in the aspect of labeling, detecting, and filtering. An Engine Code Property Selector (ECPS) uses the RDC table to adapt optimal engines for the given target programming models. In addition to RDC and ECPS, we have also implemented an OpenMP parser and a source instrumentor. The functionality and efficiency of VORD were compared with those of the Intel Thread Checker by using a set of OpenMP based kernel programs. The experimental results show that VORD can detect data races in more challenging programming models such as nested thread and synchronization models, and can achieve a couple of orders of a magnitude faster processing time than the Intel Thread Checker in the large parallel programs.

This article is devoted to the problem of verifying parallel programs that may contain special types of errors associated with the synchronization of parallel executed threads and access to shared memory. Such errors include deadlocks and data races. There is a division of parallel program verification methods into static and dynamic. The second ones require running the code and allow to check only the current implementation of the program for races, which, if there are a large number of branches, can lead to missing races. Among static methods, analytical methods (for example, based on deductive analysis) and model checking methods are most widely used. However, they are difficult to implement, and model checking still require a significant amount of manual work from the programmer to build such a model. In this regard, it is necessary to use models that can be built automatically. Previously, the authors developed a model based on an extension of Petri nets, which allows automatic creation based on sequential code and its conversion into parallel code. Automatic creation of a model of a parallel program introduces new, previously unused requirements related to the interaction of parallel threads. Thus, this article discusses the features of modeling using extended Petri nets with semantic relations of the main synchronization primitives implemented in most languages and parallel programming technologies for shared memory systems. In the future, these models will be used to search for data races and deadlocks in parallel programs.

A challenge for programming language research is to design and implement\nmulti-threaded low-level languages providing static guarantees for memory\nsafety and freedom from data races. Towards this goal, we present a concurrent\nlanguage employing safe region-based memory management and hierarchical locking\nof regions. Both regions and locks are treated uniformly, and the language\nsupports ownership transfer, early deallocation of regions and early release of\nlocks in a safe manner.\n

Data races are a class of concurrency errors where two threads access a shared memory location without proper synchronization. Data races are hard to reveal and debug. This paper presents RaceHunter - a dynamic data race detection technique which monitors executions of shared memory concurrent programs, discovers pairs of conflicting memory accesses and systematically verifies them for data races. RaceHunter does not report false data races when the target software exploits non-standard synchronization primitives or unknown synchronization protocols and can find data races missed by other techniques. Dynamic data race detectors can monitor continuous, e.g. real-life, program executions or they can verify relatively short program executions, e.g. organized by system tests. The latter is the primary use case scenario for RaceHunter.

Detecting data races in multithreaded programs is a crucial part of debugging such programs, but traditional data race detectors are too slow to use routinely. This paper shows how to speed up race detection by spreading the work across multiple cores. Our strategy relies on uniparallelism , which executes time intervals of a program (called epochs ) in parallel to provide scalability, but executes all threads from a single epoch on a single core to eliminate locking overhead. We use several techniques to make parallelization effective: dividing race detection into three phases, predicting a subset of the analysis state, eliminating sequential work via transitive reduction, and reducing the work needed to maintain multiple versions of analysis via factorization. We demonstrate our strategy by parallelizing a happens-before detector and a lockset-based detector. We find that uniparallelism can significantly speed up data race detection. With 4x the number of cores as the original application, our strategy speeds up the median execution time by 4.4x for a happens-before detector and 3.3x for a lockset race detector. Even on the same number of cores as the conventional detectors, the ability for uniparallelism to elide analysis locks allows it to reduce the median overhead by 13% for a happens-before detector and 8% for a lockset detector.

Data races in parallel programs are notoriously difficult to detect and resolve. Existing research has mostly focused on data race detection at the user level and significant progress has been made in this regard. However, a very few efforts have attempted to address OS kernel level races. In contrast to user-level applications, the code in kernels uses vastly more complicated synchronization mechanisms, including different kinds of locks, hardware and software interrupts, widely used signal/wait primitives, and various types of low-level shared resources. It is therefore difficult to apply detection methods designed for user-level applications to identify OS kernel level races. In this paper, we present a new detection tool that is able to effectively detect race conditions in the Linux kernel environment. We use a dynamic detection approach, employing hardware debug registers available on commodity processors, to catch races on the fly during runtime. Preliminary experimental results show that our tool can effectively identify real data race instances.