Towards Model Based Approach to Hadoop Deployment and Configuration

Software architecture (SA) is the blueprint of the software system and considered as one of the most important artifacts in component based development. The design and analysis of SA can be very complex. Under the inspiration of Model-Driven Development, the design of SA has been no more constrained in one stage. It is a trend to construct multiple SA models in multiple stages during the software life cycle. Thus, the traceability between these SA models becomes a new challenge. The information between these SA models in deferent stages is usually not recorded well and easy to be lost lately, which makes the maintenance and evolution difficult and error-prone. In this paper, we present an approach to recording the information between SA models via a traceability model for reducing the loss of design decisions and helping developers understand the software system well.

This paper presents the design, implementation, and evaluation of a new system for provisioning of Hadoop clusters across multiple cloud domains. The Hadoop clusters are created on-demand and are composed of virtual machines from multiple cloud sites linked with bandwidth-provisioned network pipes. The prototype uses an existing federated cloud control framework called Open Resource Control Architecture (ORCA), which orchestrates the leasing and configuration of virtual infrastructure from multiple autonomous cloud sites and network providers. ORCA enables computational and network resources from multiple clouds and network substrates to be aggregated into a single virtual slice of resources, built to order for the needs of the application. The experiments examine various provisioning alternatives by evaluating the performance of representative Hadoop benchmarks and applications on resource topologies with varying bandwidths. The evaluations examine conditions in which multi-cloud Hadoop deployments pose significant advantages or disadvantages during Map/Reduce/Shuffle operations. Further, the experiments compare multi-cloud Hadoop deployments with single-cloud deployments and investigate Hadoop Distributed File System (HDFS) performance under varying network configurations. The results show that networked clouds make cross-cloud Hadoop deployment feasible with high bandwidth network links between clouds. As expected, performance for some benchmarks degrades rapidly with constrained inter-cloud bandwidth. MapReduce shuffle patterns and certain Hadoop Distributed File System (HDFS) operations that span the constrained links are particularly sensitive to network performance. Hadoop's topology-awareness feature can mitigate these penalties to a modest degree in these hybrid bandwidth scenarios. Additional observations show that contention among co-located virtual machines is a source of irregular performance for Hadoop applications on virtual cloud infrastructure.

The project entitled as Big Data, Internet of Things, and Mobile Devices, in Portuguese Banco de Dados, Internet das Coisas e Dispositivos Moveis (BDIC-DM) was implemented at the Brazilian Aeronautics Institute of Technology (ITA) on the 1st Semester of 2015. It involved 60 graduate students within just 17 academic weeks. As a starting point for some features of real time Online Transactional Processing (OLTP) system, the Relational Database Management System (RDBMS) MySQL was used along with the NoSQL Cassandra to store transaction data generated from web portal and mobile applications. Considering batch data analysis, the Apache Hadoop Ecosystem was used for Online Analytical Processing (OLAP). The infrastructure based on the Apache Sqoop tool has allowed exporting data from the relational database MySQL to the Hadoop File System (HDFS), while Python scripts were used to export transaction data from the NoSQL database to the HDFS. The main objective of the BDIC-DM project was to implement an e-Commerce prototype system to manage credit card transactions, involving large volumes of data, by using different technologies. The used tools involved generation, storage, and consumption of Big Data. This paper describes the process of integrating NoSQL and relational database with Hadoop Cluster, during an academic project using the Scrum Agile Method. At the end, processing time significantly decreased, by using appropriate tools and available data. For future work, it is suggested the investigation of other tools and datasets.

Software architecture research focuses on models of software architectures as specified in architecture description languages (ADLs). As research progresses in specific areas of software architectures, more and more architectural information is created. Ideally, this information can be stored in the model. An extensible modeling language is crucial to experimenting with and building tools for novel modeling constructs that arise from evolving research. Traditional ADLs typically support a small set of modeling constructs very well, but adapt to others poorly. XML provides an ideal platform upon which to develop an extensible modeling language for software architectures. Previous XML-based ADLs successfully leveraged XML's large base of off-the-shelf tool support, but did not take advantage of its extensibility. To give software architecture researchers more freedom to explore new possibilities and modeling techniques, while maximizing reuse of tools and modeling constructs, we have developed xADL 2.0, a highly extensible XML-based ADL. xADL 2.0 supports run-time and design time modeling, architecture configuration management and model-based system instantiation. Additionally, xADL 2.0 has a set of extensible infrastructure tools that support the creation, manipulation, and sharing of xADL 2.0 documents.

The aspect-oriented software architectural weaving mechanism, to integrate aspectual components that encapsulate crosscutting behaviors and features into components and connectors constituting software architecture (SA), contributes to analyze and verify overall behaviors and quality attributes of SA. This paper proposes a new kind of weaving process and rules, which offer the measure to transform the unwoven SA model into the woven SA model. The unwoven SA model is described in a special aspect-oriented architecture description language, called AC2-ADL, and specifies the locations, times and constrains of injection. Then a woven SA model only containing components and connectors is acquired after weaving, and easier to analyze and verify.

The approach aims at solving the above problems by including the analysis and verification of two different levels of software development process–design level and implementation level-and bridging the gap between software architecture analysis and verification and the software product. In the architecture design level, to make sure the design correctness and attack the large scale of complex systems, the compositional verification is used by dividing and verifying each component individually and synthesizing them based on the driving theory. Then for those properties that cannot be verified on the design level, the design model is translated to implementation and runtime verification technique is adapted to the program. This approach can highly reduce the work on the design verification and avoid the state-explosion problem using model checking. Moreover, this approach can ensure both design and implementation correctness, and can further provide a high confident final software product. This approach is based on Software Architecture Model (SAM) that was proposed by Florida International University in 1999. SAM is a formal specification and built on the pair of component-connector with two formalisms – Petri nets and temporal logic. The ACV approach places strong demands on an organization to articulate those quality attributes of primary importance. It also requires a selection of benchmark combination points with which to verify integrated properties. The purpose of the ACV is not to commend particular architectures, but to provide a method for verification and analysis of large scale software systems in architecture level. The future research works fall in two directions. In the compositional verification of SAM model, it is possible that there is circular waiting of certain data among different component and connectors. This problem was not discussed in the current work. The translation of SAM to implementation is based on the restricted Petri nets due to the undecidable issue of high level Petri nets. In the runtime analysis of implementation, extraction of the execution trace of the program is still needed to get a white box view, and further analysis of execution can provide more information of the product correctness.

Introduced in the early stages of software development, the Charmy framework assists the software architect in making and evaluating architectural choices. Rarely, the software architecture of a system can be established once and forever. Most likely poorly defined and understood architectural constraints and requirements force the software architect to accept ambiguities and move forward to the construction of a suboptimal software architecture. Charmy aims to provide an easy and practical tool for supporting the iterative modeling and evaluation of software architectures. From an UML-based architectural design, an executable prototype is automatically created. Charmy simulation and model checking features help in understanding the functioning of the system and discovering potential inconsistencies of the design. When a satisfactory and stable software architecture is reached, Java code conforming to structural software architecture constraints is automatically generated through suitable transformations. The overall approach is tool supported.

The Internet of Things-based systems and software allow computations anywhere at any time by interconnecting individuals, networks, services, computers and artefacts that allow autonomous systems to form digitized communities. As the blueprint for software-intensive applications, and software architecture that precise the complexity of a network’s planning, development, and changing phases to effectively and efficiently build complex IoT-driven applications. In any case, there exists no comprehensive analysis in the state of the research on the adoption of MSA for IoT systems. This study effort is needed to explore architectural concepts and practices for designing and developing IoT software to excel state-of-the-art for IoTs along with suggestions and recommendations for IoT software to the adoption of MSA to fulfil the identified gaps. A systematic analysis was coordinated, covering up the literature on existing IoT solutions by studying 140 qualitatively selected articles performed between 2005 and Jan 2020. One hundred forty articles were comprised in this SLR. The findings of this study demonstrated different research topics including software architectural styles, patterns, and models to build IoT software. This research presents cloud-based computing environments, autonomous, software-defined networking, and responsive applications, and IoT-driven agent-based systems, (1) thirteen MSA architectural and design patterns for IoTs and classification of patterns, (2) classification of software architectures for IoTs into nine main categories and their sub-categories, (3) twenty-three most investigated IoT challenges, and (4) mapping of IoT challenges with software architectural solutions. The study revealed the innovative work on IoT software architecture and trends that help in the creation and dynamic adaptation of IoT software for reusability, automation and human decision-making. The outputs of this SLR are useful in revealing many recommendations to the software industry, software engineering community, and computer sciences community with over the past 15 years of research into the adoption of MSA. This study reflects a distilled awareness of architectural practices and principles to assist researchers and practitioners in promoting information sharing software architectural roles and responsibilities for the Internet of Things software.

An increasingly important requirement for software system is the capability to evolve dynamically at runtime according to the users' requirements and computing environments. From the angle of software architecture, in this paper, a framework of supporting dynamic evolution based on reflective aspect-oriented software architecture-RAOSA is proposed and software architecture is reified as an explicit entity. Notably, the concept of aspect of AOP is introduced into the traditional software architecture and a core model of aspect-oriented software architecture is established. Evolution rules and strategies are formalized to support deduction of the system evolutionary behaviors; by using reflection mechanism, RAOSA, which is causal-connected to software specification and implementation, monitors the contextual information of runtime system to guide the evolution and update the software specification, ensuring system consistency and integrity. Furthermore, a simple application case illustrates the feasibility of proposed framework.

While software architecture performance analysis is a well-studied field, it is less understood how the analysis results (i.e., mean values, variances, and/or probability distributions) trace back to the architectural model elements (i.e., software components, interactions among components, deployment nodes). Yet, understanding this traceability is critical for understanding the analysis result in context of the architecture. The goal of this paper is to automate the traceability between software architectural models and performance analysis results by investigating the uncertainty while bridging these two domains. Our approach makes use of performance antipatterns to deduce the logical consequences between the architectural elements and analysis results and automatically build a graph of traces to identify the most critical causes of performance flaws. We developed a tool that jointly considers SOftware and PErformance concepts (SoPeTraceAnalyzer), and it automatically builds model-to-results traceability links. The benefit of the tool is illustrated by means of a case study in the e-health domain.

Software reliability is one of the most important quality indices in software quality evaluation, with the development of software engineering and component reusing, reliability analysis based on software architecture has become a researching hotspot. This paper makes a review on existed architecture-based reliability evaluation models at first, proposes existed architecture based reliability evaluation types and typical models, putting forward the shortage of these models as well. As petri net has a good ability of description for concurrency and asynchrony. This paper focuses on the research of petri net model, divides transitions into time transitions and instantaneous transitions, and adds time-related reliability-time-delay reliability and temporal reliability, by these means we establish the software architecture modeling. Then by the study of the relationship between component state and system reachable state, we set up the reachable state transition matrix and proceeded system reliability evaluation. At last, we use the method above into a avionics embedded software, accomplish the architecture modeling, reliability evaluation and error analysis, proving the effectiveness and accuracy of this method.

Large and complex software systems cannot be created in one piece. They need to be separated into manageable units, i.e., modules, which can be developed independently from each other. Precise definition and documentation of the modules and other software artifacts is essential for a successful software project. The module documentation is part of the overall software architecture documentation. However, nowadays most of the utilized tools for documenting software architectures (e.g., word processors) are not fully sufficient for this task. This thesis provides a concept for an extensible tool, which is specialized for documenting software architectures and modules. The concept supports management and visualization of explicitly definable relations, e.g., dependencies among modules or references to external implementation artifacts. Moreover, this thesis suggests a template for module documentation based on its concept. Finally, a proof of concept prototype is implemented and evaluated with promising results.

This article is a strong pleading for the importance of architecture modeling, i.e. for a careful development and maintenance of software systems in order to solve the actual software problems, namely maintenance, quality improvement and reuse. This is due to the fact that a software architecture is the “essential” structure of a software system and that most of the software documents are directly or indirectly dependent on this structure.

The growth of various software architectural frameworks and models provides a standard governing structure for different types of organizations. Selection of a suitable framework for a particular environment needs much more detailed information in various aspects and a reference guide of features should be provided. This paper brings out the history of software architecture with a new evolution tree. It also technically analyses well known frameworks used in industries and other governmental organizations and lists out the supportive tools for them. This paper presents the comparative chart that can be used as a reference guide to understand top level frameworks and to further research to enable and promote the utilization of these frameworks in various environments. The basic principles of 'software architecture' have been applied since the mid 1980's and it crossed various stages from algorithm's era by borrowing the concepts from others to get a shaped form. In 1928, An Algorithm was formulated to solve the problem by the finite sequence of instructions. Von Neumann developed 'Flow Chart' that has a visual representation of the instruction flow, to plan computer programs in 1947 by inheriting the idea from the flow process chart(1921) and multi flow chart(1944)which were used mostly in the area of electrical engineering. But, there is a gap to point out the flow of control. So, 'Control Flow Diagram' (CFD) was developed in the late 1950's to describe the control flow of a business process and program. This was not enough to view the complex systems. The high level view of the work and immediate access of particular points can't be represented using this diagram. So, to reveal the entire system by dividing into blocks, 'Block Diagram' was developed in late 1950's. A specific function for each block and the connection between blocks will be shown in a diagram. The introduction of abstraction concept became a booster in the field of software architecture. It made a revolution and tremendous growth to that area. By that way, data structures that have similar behaviour, data structures that have similar behaviour, certain data types and modules of one or more programming languages that have similar semantics are grouped in the late 1960's.This was happened by the introduction of Abstract data types. It again leads to 'Modular Programming' that introduces the concept of separate parts called modules in software in 1968. Separation of concerns with the logical boundaries between components is called as modules. In 1977, 'Three Schema Approach' that adopts layered architecture based on the modular programming was developed. It is used to build information systems using three different views in systems development. Here an application will be broken into tiers and developers have to modify a

Software architecture has become an important area of research and practice since the late 1980s (Shaw and Clements 2006). The term “software architecture” started gaining acceptance by the software engineering community in early 1990s but the foundations of this field were laid by the seminal work of Edsger Dijkstra, David Parnas, and others between 1960s and 1980s (Clements 2000). The increasing size and complexity of software systems and demand for high quality are some of the most important factors that have driven the increased interest in this sub-discipline of software engineering. It has generally been realized that a high-level design description can play an important role in successfully understanding and managing large and complex software systems (Clements et al. 2002; Lung and Kalaichelvan 2000). The high-level design decisions regarding the software architecture of a system are not only the hardest and most expensive to change but also play a fundamental role in setting the boundaries for the required quality attributes such as maintainability, reliability, usability, performance, and flexibility of a system (Bass et al. 2003; Clement and Northorp 1996). As a result of increasing realization of the important role of software architecture in large-scale software development and evolution projects, the software architecture community has developed several methods, techniques, and tools to support the software Empir Software Eng (2011) 16:539–543 DOI 10.1007/s10664-011-9168-6