SAVE: Software Architecture Visualization and Evaluation

Metric-based Evaluation of Implemented Software Architectures

Introduced in the early stages of software development, the Charmy framework assists the software architect in making and evaluating architectural choices. Rarely, the software architecture of a system can be established once and forever. Most likely poorly defined and understood architectural constraints and requirements force the software architect to accept ambiguities and move forward to the construction of a suboptimal software architecture. Charmy aims to provide an easy and practical tool for supporting the iterative modeling and evaluation of software architectures. From an UML-based architectural design, an executable prototype is automatically created. Charmy simulation and model checking features help in understanding the functioning of the system and discovering potential inconsistencies of the design. When a satisfactory and stable software architecture is reached, Java code conforming to structural software architecture constraints is automatically generated through suitable transformations. The overall approach is tool supported.

This research recognizes the limitation and challenges of adapting and applying Process Mining as a powerful tool and technique in the Hypothetical Software Architecture (SA) Evaluation Framework with the features and factors of lightweightness. Process mining deals with the large-scale complexity of security and performance analysis, which are the goals of SA evaluation frameworks. As a result of these conjectures, all Process Mining researches in the realm of SA are thoroughly reviewed, and nine challenges for Process Mining Adaption are recognized. Process mining is embedded in the framework and to boost the quality of the SA model for further analysis, the framework nominates architectural discovery algorithms Flower, Alpha, Integer Linear Programming (ILP), Heuristic, and Inductive and compares them vs. twelve quality criteria. Finally, the framework’s testing on three case studies approves the feasibility of applying process mining to architectural evaluation. The extraction of the SA model is also done by the best model discovery algorithm, which is selected by intensive benchmarking in this research. This research presents case studies of SA in service-oriented, Pipe and Filter, and component-based styles, modeled and simulated by Hierarchical Colored Petri Net techniques based on the cases’ documentation. Process mining within this framework deals with the system’s log files obtained from SA simulation. Applying process mining is challenging, especially for a SA evaluation framework, as it has not been done yet. The research recognizes the problems of process mining adaption to a hypothetical lightweight SA evaluation framework and addresses these problems during the solution development.

: Software architecture is critical to the quality of a software-intensive system. For an acquisition organization, such as the Department of Defense (DoD), the ability to evaluate software architectures early in an acquisition can have a favorable impact on the delivered system. This technical note discusses the role of software architecture evaluations in a system acquisition and describes the contractual elements that are needed to accommodate architecture evaluations in an acquisition. The note then provides an example of contractual language that incorporates the Architecture Tradeoff Analysis Method (ATAM) as a software architecture evaluation method in a system acquisition.

The need for software architecture evaluation is based on the realization that software development, like all engineering disciplines, is a process of continuous modeling and refinement. Detecting architectural problems before the bulk of development work is done allows re‐architecting activities to take place in due time, without having to rework what has already been done. At the same time, tuning activities allow software performance to be enhanced and maintained during the software lifetime. When dealing with product families, architectural evaluations have an even more crucial role: the evaluations are targeted to a set of common products. We have tried different approaches to software assessments with our mobile phone software, an embedded real‐time software platform, which must support an increasingly large number of different product variants. In this paper, we present a case study and discuss the experiences gained with three different assessment techniques that we have worked on during the past five years. The assessment techniques presented include scenario‐based software architecture assessment, software performance assessment and experience‐based assessment. The various evaluation techniques are complementary and, when used together, constitute a tool which a software architect must be aware of in order to maintain and evolve a large software intensive system. Copyright © 2006 John Wiley & Sons, Ltd.

Quantitative properties of modern software systems are often defined as a part of a service-level agreement (SLA) that fixes the maximal load to be submitted to a system and guarantees bounds for the response time or delay. The evaluation of software architectures in order to validate SLAs is a challenging task since the systems tend to be complex, highly dynamic and to some extent unpredictable. Thus, there is a need for fast and abstract techniques to evaluate the performance of modern software architectures based on the information available in the SLAs. The paper presents an efficient approach to compute bounds on the delay of composed systems based on available bounds for the load and the response times of components. The technique can be used by a user of a software architecture to validate SLAs of composed services based on SLAs of the components. It can also be used by a provider of a software architecture to validate whether additional users can be accepted or to compute required service capacities to fulfill an SLA.

Bioinformaticists use the Basic Local Alignment Search Tool (BLAST) to characterize an unknown sequence by comparing it against a database of known sequences, thus detecting evolutionary relationships and biological properties. mpiBLAST is a widely-used, high-performance, open-source parallelization of BLAST that runs on a computer cluster delivering super-linear speedups. However, the Achilles heel of mpiBLAST is its lack of modularity, thus adversely affecting maintainability and extensibility. Alleviating this shortcoming requires an architectural refactoring to improve maintenance and extensibility while preserving high performance. Toward that end, this paper evaluates five different software architectures and details how each satisfies our design objectives. In addition, we introduce a novel approach to using mixin layers to enable mixing-and-matching of modules in constructing sequence-search applications for a variety of high-performance computing systems. Our design, which we call "mixin layers with refined roles", utilizes mixin layers to separate functionality into complementary modules and the refined roles in each layer improve the inherently modular design by precipitating flexible and structured parallel development, a necessity for an open-source application. We believe that this new software architecture for mpiBLAST-2.0 will benefit both the users and developers of the package and that our evaluation of different software architectures will be of value to other software engineers faced with the challenges of creating maintainable and extensible, high-performance, bioinformatics software.

There is a plethora of models, techniques and tools that support model-and component-based software development of vehicular distributed embedded systems. However, a large majority of them have a limited or no support to model and specify end-to-end resource reservations on the software architectures of these systems. Resource reservations allow flexibility during the development and execution of such complex systems without jeopardizing their predictable behavior. As a result, several applications in the system that share the same hardware platform can be developed independently. In this paper we identify several requirements that any existing component model should fulfill in order to support the modeling of end-to-end resource reservations on the software architectures of such systems. As a proof of concept, we extend the Rubus Component Model (RCM) by fulfilling these requirements. RCM is used for the development of control functionality in vehicular embedded systems by several international companies. We also provide a technique to extract execution models from the software architectures of these systems with resource reservations. In order to show the usability of our technique, we model a vehicular distributed embedded system with the extended component model and extract the execution model from the software architecture augmented with end-to-end resource reservations.

Agile practices have recently gained popularity among large number of companies as a mechanism for reducing cost and increasing ability to handle change in dynamic market conditions. Based on the principles of the Agile manifesto [1, 2], researchers and practitioners have proposed several software development approaches such as Extreme Programming, Scrum and Feature-Driven Development. These and other agile approaches have had significant impact on industrial software development practices. However, there is also a significant concern and perplexity about the role and importance of the issues related to a system’s software architecture, which is considered one of the most important initial design artefacts. It is argued that software architecture is an effective tool to cut development cost and time and to increase the quality of a system. Many practitioners of Agile approaches appear to view software architecture in the context of the plan-driven development paradigm [3]. For them, upfront design and evaluation of software architecture requires too much work, which may have very little value to the customers of a system. Hence, they perceive architectural work as part of high ceremony processes, which usually require large amount of documentation. We maintain that these two seemingly opposing views to software engineering can be integrated but it requires that experts from both fields work together to overcome evident challenges in bridging these two paradigms together. Indeed, software architecture researchers and practitioners appear to believe that sound architectural practices cannot be followed using Agile approaches. However, these two extreme views of Agile and architecture appear to neglect that many agile experts emphasises the importance of paying attention to good design and architecture early in the development process [4, 5]. Recently, there is growing recognition of the importance of paying more attention to architectural aspects in agile approaches [3, 6, 7]. We argue that there is a vital need for devising a research agenda for identifying and dealing with architecture-centric challenges in agile software development. Such research agenda should make it possible to guide the future research on integrating architecture-centric methods in agile approaches and give advice to the software industry on dealing with architecture related challenges. Some of the questions to stimulate discussion in the workshop are: What is the role of software architecture in Agile software development? What are the key architecture-centric challenges and potential solutions in Agile software development projects? What is the strength of the evidence that attention to architectural issues can be counterproductive in Agile development or vice versa? What are the prerequisites for integrating Architecture-Centric methods in agile development and potential implications of such integration?

The function and significance of mission-critical software-intensive systems have got substantial recognition. Software architecture has become a new field since system software is all the time more intricate. Agile software development counters the advancement in requirement, besides to attend to the fixed plan. In this paper, the effort has been made to find parameters for software architecture evaluation and then evaluate software architecture under agile environment based on the determined parameters.

The analysis of multi-unit extracellular recordings of brain activity has led to the development of numerous tools, ranging from signal processing algorithms to electronic devices and applications. Currently, the evaluation and optimisation of these tools are hampered by the lack of ground-truth databases of neural signals. These databases must be parameterisable, easy to generate and bio-inspired, i.e. containing features encountered in real electrophysiological recording sessions. Towards that end, this article introduces an original computational approach to create fully annotated and parameterised benchmark datasets, generated from the summation of three components: neural signals from compartmental models and recorded extracellular spikes, non-stationary slow oscillations, and a variety of different types of artefacts. We present three application examples. (1) We reproduced in-vivo extracellular hippocampal multi-unit recordings from either tetrode or polytrode designs. (2) We simulated recordings in two different experimental conditions: anaesthetised and awake subjects. (3) Last, we also conducted a series of simulations to study the impact of different level of artefacts on extracellular recordings and their influence in the frequency domain. Beyond the results presented here, such a benchmark dataset generator has many applications such as calibration, evaluation and development of both hardware and software architectures.

Software performance based on performance models can be applied at early phases of the software development cycle to characterize the quantitative behavior of software systems. We propose an approach based on queueing networks models for performance prediction of software systems at the software architecture level, specified by UML. Starting from annotated UML Use Case, Activity and Deployment diagrams we derive a performance models based on multichain and multiclass Queueing Networks (QN). The UML model is annotated according to the UML Profile for Schedulability, Performance and Time Specification. The proposed algorithm translates the annotated UML specification into QN performance models, which can then be analyzed using standard solution techniques. Performance results are reported back at the software architecture level in the UML diagrams. As our approach can be fully automated and uses standard UML annotations, it can be integrated with other performance modeling approaches. Specifically, we discuss how this QN-based approach can be integrated with an existing simulation-based performance modeling tool.

Thorough and continuous architecting is the key to overall success in software engineering, and architecture evaluation is a crucial part of it. This tutorial presents a pragmatic architecture evaluation approach and insights gained from its application in more than 75 projects with industrial customers in the past decade. It presents context factors, empirical data, and example cases, as well as lessons learned on mitigating the risk of change through architecture evaluation. By providing comprehensive answers to many typical questions and discussing more frequent mistakes and lessons learned, the tutorial allows the audience to not only learn how to conduct architecture evaluations and interpret its results, but also to become aware of risks such as false conclusions, manipulating data, and unsound lines of argument. It equips the audience to become confident in assessing quantitative measurement results and recognize when it is better to rely on qualitative expertise. The target audience includes both practitioners and researchers. By demonstrating its impact and providing clear guidelines, data, and examples, it encourages practitioners to conduct architecture evaluations. At the same time, it offers researchers insights into industrial architecture evaluations, which serve as the basis for guiding research in this area and will inspire future research directions. Both groups will get an overview of the foundations and history of architecture evaluation. The tutorial covers the following important aspects of architecture evaluation • Architecture drivers: types of drivers, importance of drivers, elicitation of drivers, documentation of drivers as architecture scenarios • Solution Adequacy Check: check whether an architecture is adequate for its drivers, what are risks, assumptions, tradeoffs (based on ATAM (architecture tradeoff analysis method) and enhanced with other techniques for increasing confidence of results) • Documentation Quality Check: How adequate is the documentation of an architecture to be understandable and to serve its purposes? • Architecture Compliance Check: How to check whether an intended architecture is consistently reflected in the source code? For all aspects, pragmatic methodical support is provided in the tutorial and all checks are well integrated in the overall architecture evaluation method.

Architectural evaluation to determine a software architecture's fitness with respect to its desired quality attributes is one of the most important issues in architecture-based software development. However, existing techniques have too many limitations for widespread application, such as the inappropriate representation of architecture and ambiguities in the evaluation process. Therefore, this paper presents an approach to evaluate software architecture with a 4+1 view model in UML. Our approach is divided into three main areas of activity: the work involved in preparation, execution, and completion of the evaluation. By performing these activities, architecture evaluation can be explicitly described and its result can be systematically organized based on the 4+1 view model. In addition, we show the practical applicability and features of our approach via an ATM example.

The software architecture is typically defined as the fundamental organization of the system embodied in its components, their relationships to one another and to the system's environment. It also encompases principles governing the system's design and evolution. In order to manage the architecture of a large software system the architect needs a holistic model that supports continuous integration and verification for all system artifacts. In earlier papers we proposed a unified graph-based approach to the problem of managing knowledge about the architecture of a software system. In this paper we demonstrate that this approach facilitates convenient and efficient project measurement. First, we show how existing software metrics can be translated into our model in a way that is independent of the programming language. Second, we introduce new metrics that cross the programming language boundaries and are easily implementable using our approach. We conclude by demonstrating how the new model can be implemented using existing tools. In particular, graph databases are a convenient implementation of an architectural repository. Graph query languages and graph algorithms are an effective way to define metrics and specialized graph views.