Effective static deadlock detection

We present a novel technique for static race detection in Java programs, comprised of a series of stages that employ a combination of static analyses to successively reduce the pairs of memory accesses potentially involved in a race. We have implemented our technique and applied it to a suite of multi-threaded Java programs. Our experiments show that it is precise, scalable, and useful, reporting tens to hundreds of serious and previously unknown concurrency bugs in large, widely-used programs with few false alarms.

Multithreaded programs are hard to develop and test. In order for programs to avoid unexpected concurrent behaviors at runtime, for example data-races, synchronization mechanisms are typically used to enforce a safe subset of thread interleavings. Also, to test multithreaded programs, devel- opers need to enforce the precise thread schedules that they want to test. These tasks are nontrivial and error prone.

Atomicity is a fundamental correctness property in multithreaded programs. A method is atomic if, for every execution, there is an equivalent serial execution in which the actions of the method are not interleaved with actions of other threads. Atomic methods are amenable to sequential reasoning, which significantly facilitates subsequent analysis and verification. This article presents a type system for specifying and verifying the atomicity of methods in multithreaded Java programs using a synthesis of Lipton's theory of reduction and type systems for race detection. The type system supports guarded, write-guarded, and unguarded fields, as well as thread-local data, parameterized classes and methods, and protected locks. We also present an algorithm for verifying atomicity via type inference. We have applied our type checker and type inference tools to a number of commonly used Java library classes and programs. These tools were able to verify the vast majority of methods in these benchmarks as atomic, indicating that atomicity is a widespread methodology for multithreaded programming. In addition, reported atomicity violations revealed some subtle errors in the synchronization disciplines of these programs.

The development of tools for automated analysis of data collected for mine countermeasures (MCM) search, classify and map (SCM) has predominantly focused on automated target recognition for individual mines. As MCM target detection becomes more automated, detecting spatial patterns in the positioning of mines should follow. Natural restrictions on mine deployment such as depth and bottom contour as well as mine deployment methodology and protocol effectively create mine line patterns that are typical to coastal areas. We have developed a mine line detection algorithm that uses nonparametric spatial analysis to identify mine-like contacts (MLCs) that fit into mine line patterns dictated by standard mine laying protocol in coastal regions. In general, a set of MLCs reported from a mine detection sensor will consist of both actual mines and false alarms; the algorithm classifies every MLC in this set into one of three classes: fixed — the MLCψfits into a mine line pattern with fixed mine spacing; random -the MLCψfits into a mine line pattern with random mine spacing; or false alarm-ψ-the MLCψdoes not fit into a mine line pattern and is subsequently not considered a mine. The latest version of the algorithm utilizes nonparametric techniques and requires no knowledge of either the detection sensor's probability of detection (P d ) or probability of false alarm (P fa ). A nonparametric anomaly detector first identifies regions with unusually high MLC density for more detailed analysis. Suspected regions are subsequently analyzed for unusually regular patterns of MLCs. The probabilities that the observed target densities and target patterns could occur simply by chance are estimated by comparison with results from nearby areas using basic information about mine deployment. To ensure that a mine line's full extent is captured, the algorithm attempts to grow a line after it is detected. And a more detailed probability analysis distinguishes random mine lines from fixed-spaced mine lines. Extensive Matlab simulations characterize the algorithm's performance across a wide range of operating environments and simulated minefield configurations. In total, ten replications each of 1,367 different configurations have been evaluated. These configurations include three different water depths (10 ft., 6 ft. and 200 ft.), six false notional alarm rates (1, 2,5,10,15 and 20 false alarms per sq. NM), three false alarm distributions (uniform, sloped, and striped), three sensor probabilities of detections (0.4, 0.6 and 0.9), three target confidence scores (0.6,0.8 and 1.0) and four false alarm confidence scores (0.5, 0.6, 0.8 and 1.0). The range of simulation parameters was selected in part as an attempt to push the algorithm past its acceptable performance limits. However, the algorithm performed remarkably well even under the worst conditions. For MLC sets with false alarm rates as high as 20 per square nautical mile, the algorithm correctly classifies over 98% of the MLCs as mines and detects 97% of the mine lines. Furthermore, it reduces the false alarm rate from 20 to 0.2 per square nautical mile with a false mine line rate of 0.14 false lines per square nautical mile. The algorithm achieves even lower false alarm rates when the initial false alarm rate is less than 20. The automated mine line detection discovers many obscure lines that are virtually undetectable to the naked eye.

We present RDIT, a novel dynamic technique to detect data races in multithreaded programs with incomplete trace information, i.e., in the presence of missing events. RDIT is both precise and maximal: it does not report any false alarms and it detects a maximal set of true traces from the observed incomplete trace. RDIT is underpinned by a sound BarrierPair model that abstracts away the missing events by capturing the invocation data of their enclosing methods. By making the least conservative abstraction that a missing method introduces synchronization only when it has a memory address in scope that overlaps with other events or other missing methods, and by formulating maximal thread causality as logical constraints, RDIT guarantees to precisely detect races with maximal capability. RDIT has been applied in seven real-world large concurrent systems and has detected dozens of true races with zero false alarms. Comparatively, existing algorithms such as Happens-Before, Causal- Precedes, and Maximal-Causality which are known to be precise all report many false alarms when missing synchronizations.

Purpose – This paper aims to find an objects representation scheme with high precision and to compute the objects’ separation distance effectively in final analysis. Proximity queries have been used widely in robot trajectory planning, automatic assembly planning, virtual surgery and many other applications. The core of proximity query is the precise computation of (minimum) separation distance in narrow phase, and specific object representation scheme corresponds to different methods of separation distance computation. Design/methodology/approach – In this paper, a second-order cone programming (SOCP)-based (minimum) separation distance computation algorithm was proposed. It treats convex superquadrics, descriptive primitives of complex object as the study objects. The separation distance between two convex superquadrics was written as a general nonlinear programming (NLP) problem with superquadric constraints and then transformed into an SOCP problem with the method of conic formulation of superquadric constraints. Finally, a primal-dual interior point method embedded in MOSEK was used for solving the SOCP problem. Findings – The proposed algorithm achieved exact separation distance computation between convex superquadrics, with a relative error of 10-6. It is particularly suitable for proximity queries in narrow phase of static collision detection algorithms. Further, the proposed algorithm achieved continuous collision detection between rectilinear translation superquadrics. Originality/value – The proposed algorithm in narrow phase of static collision detection algorithms makes objects’ separation distance effectively computed. Proximity queries are easy and more accurate to perform in this way.

Static analysis is a tremendous help when trying to find faults in complex software. Writing multi-threaded programs is difficult, because the thread scheduling increases the program state space exponentially, and an incorrect thread synchronization produces faults that are hard to find. Program checkers have become sophisticated enough to find faults in real, large-scale software. In particular, Jlint, a very fast Java program checker; can check packages in a highly automated manner. The original version, Jlint1, still lacked full support for synchronization statements in Java. We extended Jlint1's model to include synchronizations on arbitrary objects, and named our version Jlint2. Our statistical analysis proves that these extensions are relevant and useful. Applying Jlint2 to various large software packages, including commercial packages from Trilogy, found 12 faults, two of which related to multi-threading.

Currently, developing a multithreading Application Programming Interface (API) for special use is extensive. Deadlock in the multithreading API is the most concerned problem because we cannot find the deadlock in the application and the cost for defect fixing later on the development phase is even higher and more complex. Detecting deadlock in early phase of the Software Development Life Cycle or using static analysis is the way more effective. Since we still do not know how developer uses the API, detecting deadlock in the source code using static analysis is our selection for this research. We have successfully developed the Deadlock Detection Algorithm that brings the concept of five deadlock conditions by Mayur Naik published in 2009 and two code patterns by Frank Otto published in 2008. We present seven deadlock conditions that are the Aliasing Condition, the Reverse Order Locking Condition or the Cyclic Lock Dependency Condition, the Escaping Condition, the Parallel Condition, the Non-Guarded Lock Condition and the Superfluous Lock Condition. In addition, we develop the prototype of the Deadlock Detection Tool to demonstrate the use of the Deadlock Detection Algorithm. The result of the implementation of the Deadlock Detection Algorithm and the Deadlock Detection Tool are satisfied and provide correct result as expected.

Dynamic analysis is a promising technique for finding concurrency bugs in multithreaded programs. However, testing a dynamic analysis tool can be difficult. Researchers end up writing large amounts of small benchmark programs. Since the benchmarks themselves are concurrent programs, they may execute nondeterministically, complicating testing of the analysis tool.We propose testing dynamic analyses by writing traces in a simple trace description language, Tiddle. Our implementation, written in Haskell, generates deterministic multithreaded Java programs for testing dynamic analyses. We report that it is substantially easier to write programs with incriminating bugs such as race conditions in Tiddle than the corresponding Java source code version, reducing the amount of source code to maintain and understand. Although our implementation is targeted towards Java, the ideas extend to any other languages which support mutable fields and multiple threads.

Previous work has shown that program slicing can be a useful step in model-checking software systems. We are interested in applying these techniques to construct models of multi-threaded Java programs. Past work does not address the concurrency primitives found in Java, nor does it provide the rigorous notions of slice correctness that are necessary for reasoning about programs with non-deterministic behaviour and potentially infinite computation traces. In this paper, we define the semantics of a simple multi-threaded language with concurrency primitives matching those found in the Java Virtual Machine, we propose a bisimulation-based notion of correctness for slicing in this setting, we identify notions of dependency that are relevant for slicing multi-threaded Java programs, and we use these dependencies to specify a program slicer for the language presented in the paper. Finally, we discuss how these dependencies can be refined to take into account common programming idioms of concurrent Java software.

We present our experience of combining, in a realistic setting, a static analyzer with a statistical analysis. This combination is in order to reduce the inevitable false alarms from a domain-unaware static analyzer. Our analyzer named Airac(Array Index Range Analyzer for C) collects all the true buffer-overrun points in ANSI C programs. The soundness is maintained, and the analysis’ cost-accuracy improvement is achieved by techniques that static analysis community has long accumulated. For still inevitable false alarms (e.g. Airac raised 970 buffer-overrun alarms in commercial C programs of 5.3 million lines and 737 among the 970 alarms were false), which are always apt for particular C programs, we use a statistical post analysis. The statistical analysis, given the analysis results (alarms), sifts out probable false alarms and prioritizes true alarms. It estimates the probability of each alarm being true. The probabilities are used in two ways: 1) only the alarms that have true-alarm probabilities higher than a threshold are reported to the user; 2) the alarms are sorted by the probability before reporting, so that the user can check highly probable errors first. In our experiments with Linux kernel sources, if we set the risk of missing true error is about 3 times greater than false alarming, 74.83% of false alarms could be filtered; only 15.17% of false alarms were mixed up until the user observes 50% of the true alarms.KeywordsFalse AlarmProbable False AlarmTrue ErrorUser AnnotationProgram PointThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Byte-code scheduling of Java programs with branches for desktop grid

This paper presents a novel approach for false alarm suppression using machine learning tools. It proposes a multi-modal detection algorithm to find the true beats using the information from all the available waveforms. This method uses a variety of beat detection algorithms, some of which are developed by the authors. The outputs of the beat detection algorithms are combined using a machine learning approach. For the ventricular tachycardia and ventricular fibrillation alarms, separate classification models are trained to distinguish between the normal and abnormal beats. This information, along with alarm-specific criteria, is used to decide if the alarm is false. The results indicate that the presented method was effective in suppressing false alarms when it was tested on a hidden validation dataset.

Writing correct multithreaded programs is difficult. Existing tools for finding bugs in multithreaded programs primarily focus on finding generic concurrency problems such as data races, atomicity violations, and deadlocks. However, these generic bugs may sometimes be benign and may not help to catch other functional errors in multithreaded programs. In this paper, we focus on a high-level programming error, called typestate error, which happens when a program does not follow the correct usage protocol of an object. We present a novel technique that finds typestate errors in multithreaded programs by looking at a successful execution. An appealing aspect of our technique is that it not only finds typestate errors that occur during a program execution, but also many other typestate errors that could have occurred in a different execution. We have implemented this technique in a prototype tool for Java and have experimented it with a number of real-world Java programs.

This paper presents a context-sensitive and path-sensitive, intra-thread and inter-thread solution to combined pointer analysis, escape analysis and data dependence analysis of multithreaded Java programs which uses a sparse representation. We build and maintain a complete Static Single Assignment (SSA) form even for fields variables. We show how to compute inter-thread dependencies for multithreaded programs with structured fork-join constructs, open-ended threads, recursively generated threads, monitors, and wait-notify synchronization. We have implemented our algorithm in a slicer for Java programs. Our experimental results show that a sparse representation improves the analysis time and strong updates on field variables improves the precision.