Supporting asynchronization in OpenMP for event-driven programming

The World model is an existing functional input-output mechanism for event-driven programming. It is used in numerous popular textbooks and curricular settings. The World model conflates two different tasks -- the definition of an event processor and its execution -- into one. This conflation imposes a significant (even unacceptable) burden on student users in several educational settings where we have tried to use it, e.g., for teaching physics. While it was tempting to pile on features to address these issues, we instead used the Scheme language design dictum of removing weaknesses that made them seem necessary. By separating the two tasks above, we arrived at a slightly different primitive, the reactor, as our basis. This only defines the event processor, and a variety of execution operators dictate how it runs. The new design enables programmatic control over event-driven programs. This simplifies reflecting on program behavior, and eliminates many unnecessary curricular dependencies imposed by the old design. This work has been implemented in the Pyret programming language. The separation of concerns has enabled new curricula, such as the Bootstrap:Physics curriculum, to take flight. Thousands of students use this new mechanism every year. We believe that reducing impedance mismatches improves their educational experience.

The following topics are dealt with: parallel processing; learning (artificial intelligence); graphics processing units; graph theory; parallel algorithms; scheduling; application program interfaces; parallel architectures; storage management; parallel machines.

Asynchronous programming is generally one of the hardest programming concepts to grasp. It is even harder to understand what it means when it comes to dealing with server applications, notably web applications. Besides the general concept, the asynchronous programming model in the .NET Framework was also hard to grasp in the early days of the framework. With the introduction of Task Parallel Library (TPL) in .NET version 4.0, it became easier, but the programming model was still not ideal. With the new asynchronous programming language features of C# in .NET 4.5, it got easier still. It now may be said to offer the ideal programming model as well.

Programming systems incorporating aspects of functional programming, e.g., higher-order functions, are becoming increasingly popular for large-scale distributed programming. New frameworks such as Apache Spark leverage functional techniques to provide high-level, declarative APIs for in-memory data analytics, often outperforming traditional "big data" frameworks like Hadoop MapReduce. However, widely-used programming models remain rather ad-hoc; aspects such as implementation trade-offs, static typing, and semantics are not yet well-understood. We present a new asynchronous programming model that has at its core several principles facilitating functional processing of distributed data. The emphasis of our model is on simplicity, performance, and expressiveness. The primary means of communication is by passing functions (closures) to distributed, immutable data. To ensure safe and efficient distribution of closures, our model leverages both syntactic and type-based restrictions. We report on a prototype implementation in Scala. Finally, we present preliminary experimental results evaluating the performance impact of a static, type-based optimization of serialization.

In this paper, TOPAS, a new parallel programming environment for distributed systems is presented. TOPAS automatically analyzes data dependence among tasks and synchronizes data, which reduces the time needed for parallel program developments. TOPAS also provides supports for scheduling, dynamic load balancing and fault tolerance. Experiments show simplicity and efficiency of parallel programming in TOPAS environment.KeywordsFault ToleranceParallel ProgramTask GraphFaulty NodeDynamic Load BalanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Parallel programming can be extremely challenging. Programming models have been proposed to simplify this task, but wide acceptance of these remains elusive for many reasons, including the demand for greater accessibility and productivity. In this paper, we introduce a parallel programming model and framework called CharmPy, based on the Python language. CharmPy builds on Charm++, and runs on top of its C++ runtime. It presents several unique features in the form of a simplified model and API, increased flexibility, and the ability to write everything in Python. CharmPy is a high-level model based on the paradigm of distributed migratable objects. It retains the benefits of the Charm++ runtime, including dynamic load balancing, asynchronous execution model with automatic overlap of communication and computation, high performance, and scalability from laptops to supercomputers. By being Python-based, CharmPy also benefits from modern language features, access to popular scientific computing and data science software, and interoperability with existing technologies like C, Fortran and OpenMP. To illustrate the simplicity of the model, we will show how to implement a distributed parallel map function based on the Master-Worker pattern using CharmPy, with support for asynchronous concurrent jobs. We also present performance results running stencil code and molecular dynamics mini-apps fully written in Python, on Blue Waters and Cori supercomputers. For stencil3d, we show performance similar to an equivalent MPI-based program, and significantly improved performance for imbalanced computations. Using Numba to JIT-compile the critical parts of the code, we show performance for both mini-apps similar to the equivalent C++ code.

Prevailing trend in design of chip multiprocessors (CMP) has been that single-core processors are replicated. Therefore, they typically define asynchronous computational model, require heavily locality-aware memory allocation, and present high overheads in intercommunication. This kind of properties make parallel programming very challenging and prone to errors. We introduce our new dual-mode MultiBunched/Threaded Architecture with Chaining (MBTAC) processor core, the main building block of the REPLICA CMP. It provides a modern, sophisticated way for writing general purpose parallel programs backed up by native execution capabilities/realization of key concepts. These include support for cost-efficient machine instruction-level synchronization and uniform shared global memory for enabling easy-to-program memory allocation of data structures and data movement. MBTAC makes use of low-overhead thread-context switching solution; it has parallel computing savvy functional unit organization to exploit inter-thread instruction-level parallelism and highly efficient multioperations. To evaluate the goodness of our proposal, we implemented three MBTAC constellations featuring up to 2048 parallel threads on FPGA, compared it with respect to DLX and Intel’s Core i7 processors. The results point toward high performance in communication-intensive problems, simplified parallel programmability, and regular, implementation-friendly structure.

The NAS Parallel Benchmarks for evaluating C++ parallel programming frameworks on shared-memory architectures

Genericity of parallel programming environments, enabling development of portable parallel programs, is expected to result in performance penalties. Furthermore, programmability and tool support of programming environments are important issues if a choice between programming environments has to be made. In this paper we propose a methodology to compare native and generic parallel programming environments, taking into account such competing issues as portability and performance. As a case study, this paper compares the Iserver-Occam, Parix, Express and PVM parallel programming environments on a 512-node Parasytec GCel. Furthermore, we apply our methodology to compare Parix and PVM on a new architecture, a 32-node Parsytec PowerXplorer, which is based on the PowerPC chip. In our approach we start with a representative application and isolate the basic (environment)-dependent building blocks. These basic building blocks, which depend on floating-point performance and communication capabilities of the environments, are analysed independently. We have measured point-to-point communication times, global communication times and floating-point performance. All information is combined into a time complexity analysis, allowing comparison of the environments on different degrees of functionality. Together with demands for portability of the code and development time (i.e. programmability), an overall judgement of the environments is given.

We present a design methodology for real-time vision applications aiming at significantly reducing the design-implement-validate cycle time on dedicated parallel platforms. This methodology is based upon the concept of algorithmic skeletons, i.e., higher order program constructs encapsulating recurring forms of parallel computations and hiding their low-level implementation details. Parallel programs are built by simply selecting and composing instances of skeletons chosen in a predefined basis. A complete parallel programming environment was built to support the presented methodology. It comprises a library of vision-specific skeletons and a chain of tools capable of turning an architecture-independent skeletal specification of an application into an optimized, deadlock-free distributive executive for a wide range of parallel platforms. This skeleton basis was defined after a careful analysis of a large corpus of existing parallel vision applications. The source program is a purely functional specification of the algorithm in which the structure of a parallel application is expressed only as combination of a limited number of skeletons. This specification is compiled down to a parametric process graph, which is subsequently mapped onto the actual physical topology using a third-party CAD software. It can also be executed on any sequential platform to check the correctness of the parallel algorithm. The applicability of the proposed methodology and associated tools has been demonstrated by parallelizing several realistic real-time vision applications both on a multi-processor platform and a network of workstations. It is here illustrated with a complete road-tracking algorithm based upon white-line detection. This experiment showed a dramatic reduction in development times (hence the term fast prototyping), while keeping performances on par with those obtained with the handcrafted parallel version.

Node.js is one of the hottest open source web platforms currently available. It's built on Google Chrome's V8 JavaScript runtime engine and it allows you to write all kinds of network applications and servers in just a few lines of code. Node.js uses an asynchronous programming model built on non-blocking I/O and a single-threaded event loop. What this means, basically, is that you don't need to be concerned with awful race conditions or synchronization issues that arise when programming for a concurrent multi-user environment. This workshop is aimed mainly at web development instructors that would like to consider teaching a single programming language, JavaScript, for both client-side and server-side coding. Participants will learn how to employ Node.js on Windows, Mac OS or Linux in order to write scalable web servers and applications. Additionally, the Express web framework will be introduced in order to demonstrate how to quickly program traditional webapps and single-page applications (SPA) with the aid of jQuery, AJAX and RESTful web services. The resulting programs will be usable from any modern web browser, including those found in desktop and laptop computers, and mobile devices such as tablets and smart-phones. Participants should have prior working knowledge of client-side (running on a browser) JavaScript and HTML. More information: http://node.arielortiz.info/ Laptop Required.

Traditional MVC model is a monopoly-like patterns in response to the request, although there are a lot of advantages, but the performance of the layer is weak, and AJAX just make up for shortcomings in this regard.Ajax is the popular Web client technology, providing a asynchronous programming model by the adoption of Java Script and DHTML, so as to enhance the interactive web pages and the ability to respond so that the Web application offer the human-computer users a better experience. In this paper, it illustrates the development of Web applications by combination of both, and puts forward the reconstruction of the MVC pattern.

The root system is critical for the survival of nearly all land plants and a key target for improving abiotic stress tolerance, nutrient accumulation, and yield in crop species. Although many methods of root phenotyping exist, within field studies, one of the most popular methods is the extraction and measurement of the upper portion of the root system, known as the root crown, followed by trait quantification based on manual measurements or 2D imaging. However, 2D techniques are inherently limited by the information available from single points of view. Here, we used X-ray computed tomography to generate highly accurate 3D models of maize root crowns and created computational pipelines capable of measuring 71 features from each sample. This approach improves estimates of the genetic contribution to root system architecture and is refined enough to detect various changes in global root system architecture over developmental time as well as more subtle changes in root distributions as a result of environmental differences. We demonstrate that root pulling force, a high-throughput method of root extraction that provides an estimate of root mass, is associated with multiple 3D traits from our pipeline. Our combined methodology can therefore be used to calibrate and interpret root pulling force measurements across a range of experimental contexts or scaled up as a stand-alone approach in large genetic studies of root system architecture.

The need to parallelise desktop applications is becoming increasingly essential with the mainstream adoption of multi-cores. In object-oriented languages, sequential iterators handle iterative computations of a sequential program; similarly, the parallel iterator was developed to handle the iterative computations of a parallel program. This paper presents the progress of the parallel iterator concept. New features, such as support for reductions and global break semantics, allow the parallel iterator to undertake more situations. With a slight contract modification, the parallel iterator interface now imitates that of the sequential iterator. All these features combine together to promote minimal, if any, code restructuring. The reduction frequently outperforms related work and the importance of providing simple and flexible fine-tuning capability is affirmed.

This research examines code restructuring and data analytics to improve distributed system energy efficiency. The main goal is to optimize software design and use data-driven insights to decrease energy usage without compromising performance. The secondary data-based assessment examines code refactoring methods like algorithm optimization and memory management and data analytics tools like predictive models and real-time monitoring. Key findings show that code refactoring streamlines algorithms, reduces redundant processes, and improves task distribution. At the same time, data analytics enables adaptive energy management through predictive forecasting, anomaly detection, and dynamic resource allocation. Combining these methods yields a scalable distributed energy efficiency solution. However, ongoing data processing energy costs and integration complexity persist. The report emphasizes the need for incentives for technology investments, training, and established best practices to promote energy-efficient distributed systems. These results indicate that a balanced strategy combining code optimization and powerful data analytics may maintain and improve energy efficiency in the continually changing distributed computing ecosystem.