Supporting Multiplicity and Hierarchy in Model-Based Configuration: Experiences and Lessons Learned

Feature models are frequently used to capture the knowledge about configurable software systems and product lines. However, feature modeling of large-scale systems is challenging as many models are needed for diverse purposes. For instance, feature models can be used to reflect the perspectives of product management, technical solution architecture, or product configuration. Furthermore, models are required at different levels of granularity. Although numerous approaches and tools are available, it remains hard to define the purpose, scope, and granularity of feature models. In this paper we thus present experiences of developing feature models for two large-scale industrial automation software systems. Specifically, we extended an existing feature modeling tool to support models for different purposes and at multiple levels. We report results on the characteristics and modularity of the feature models, including metrics about model dependencies. We further discuss lessons learned during the modeling process.

With the increasing importance of complex software systems in the software industry, the need for using agent technologies to develop large-scale commercial and industrial software systems is growing rapidly. Such systems are complex, and there is a pressing need for system modeling techniques to support reliable, maintainable and extensible design. G-nets are a type of Petri net defined to support the modeling of a system as a set of independent and loosely-coupled modules. In this paper, we first introduce an extension of G-nets - the agent-based G-net - as a generic model for agent design. Then, to progress from an agent-based design model to an agent-oriented model, new mechanisms to support inheritance modeling are introduced. To illustrate our formal modeling technique for multi-agent systems, an example of an agent family in electronic commerce is provided.

Adopting an accurate anomaly detection mechanism is crucial for industrial software systems in order to prevent system outages that can deteriorate system availability. However, employing a supervised machine learning technique to detect anomalies in large production scale industrial software systems is highly impractical due to the requirement of annotated data. This raises the need for comprehensive semi-supervised and unsupervised anomaly detection mechanisms. This paper presents the application of Generative Adversarial Network (GAN) based models to detect system anomalies using semi-supervised one-class learning. We show that the use of a variant of GAN known as bidirectional GAN (BiGAN) gives augmented results when compared to the traditional GAN based anomaly detection, for the selected industrial system. Moreover, the experiments clearly show that the performance of the BiGAN has a direct correlation with the dimensions of the dataset used for training. The BiGAN even tends to outperform the well-established semi-supervised One-class SVM classifier and a prominent generative network for semi-supervised anomaly detection, Variational Autoencoders (VAEs) when the size of the feature space increases.

Agents are becoming one of the most important topics in distributed and autonomous decentralized systems, and there are increasing attempts to use agent technologies to develop large-scale commercial and industrial software systems. The complexity of such systems suggests a pressing need for system modeling techniques to support reliable, maintainable, and extensible design. G-nets are a type of Petri net defined to support system modeling in terms of a set of independent and loosely-coupled modules. In this paper, we customize the basic G-net model to define a so-called "agent-based G-net" that can serve as a generic model for agent design. Then, to progress from an agent-based design model to an agent-oriented model, new mechanisms to support inheritance modeling are introduced. To illustrate our formal modeling technique for multiagent systems, an example of an agent family in electronic commerce is provided. Finally, we demonstrate how we can use model checking to verify some key behavioral properties of our agent model. This is facilitated by the use of an existing Petri net tool.

Anomaly detection plays a critical role in large-scale industrial systems, where real-time streaming data is generated at high volumes. This research journal presents an in-depth study on developing an efficient anomaly detection algorithm specifically designed for such industrial systems. The goal is to identify abnormal patterns and deviations from normal behavior, enabling proactive maintenance, improved operational efficiency, and reduced downtime. The proposed algorithm leverages machine learning and stream processing techniques to handle the challenges associated with real- time streaming data analysis. The research covers algorithm design, implementation, evaluation, and performance analysis using large-scale datasets from industrial domains. Keywords: anomaly detection, real-time streaming data, large-scale industrial systems, machine learning, stream processing

We evaluate the benefits of applying aspect-oriented software development techniques in the context of a large-scale industrial embedded software system implementing a number of crosscutting concerns. Additionally, we assess the feasibility of automatically extracting these crosscutting concerns from the source code. In order to achieve this, we present an approach for reverse engineering aspects from an ordinary application automatically. This approach incorporates both a concern verification and an aspect construction phase. Our results show that such automated support is feasible, and can lead to significant improvements in source code quality.

Dealing with clones is a common problem in large-scale software projects. While most of the research in this area is focused on detecting, and investigating the reasons of clones, little research has been done on the use of clone technology in large-scale industrial software. In this paper we present an in-depth case study of cloning in Command and Control Software Framework (CCSF) developed by HAVELSAN, a large software company in the Turkish defense industry. We proposed a practical classification schema for clones based on their removal strategies. A detailed analysis on categorization of clones, and lessons learned in clone management of large-scale industrial software systems are presented.

In this article, a correlation-aided neural network (CANN) framework is proposed for distributed process monitoring of large-scale industrial automation systems. First, to model the nonlinear data relationship of multiple subsystems, a feature space is learned for each subsystem by means of neural network-based nonlinear projection. Second, different from conventional distributed monitoring methods, a distributed learning method and a novel objective function are proposed, where the influence of uncertainties is decreased by considering the correlation of variables among multiple subsystems. Third, a process monitoring method is proposed based on the CANN framework. The impact of the fault is analyzed, and the optimal monitoring performance is achieved by considering the correlation of all subsystems. To further reduce communication overhead among multiple subsystems, a tradeoff between monitoring performance and communication overhead is made based on gradients of the CANN model. Theoretical analysis demonstrates the superiority of the proposed method. The case studies on a multizone heating, ventilation, and air-conditioning system and the Tennessee Eastman benchmark process are given to evaluate and compare the effectiveness of the proposed method.

To assure high software quality for large-scale industrial software systems, traditional approaches of software quality assurance, such as software testing and performance engineering, have been widely used within Alibaba, the world's largest retailer, and one of the largest Internet companies in the world. However, there still exists a high demand for software quality assessment to achieve high sustainability of business growth and engineering culture in Alibaba. To address this issue, we develop an industrial solution for software quality assessment by following the GQM paradigm in an industrial setting. Moreover, we integrate multiple assessment methods into our solution, ranging from metric selection to rating aggregation. Our solution has been implemented, deployed, and adopted at Alibaba: (1) used by Alibaba's Business Platform Unit to continually monitor the quality for 60+ core software systems; (2) used by Alibaba's R&D Efficiency Unit to support group-wide quality-aware code search and automatic code inspection. This paper presents our proposed industrial solution, including its techniques and industrial adoption, along with the lessons learned during the development and deployment of our solution.

Architecture-based software reliability analysis methods shall help software architects to identify critical software components and to quantify their influence on the system reliability. Although researchers have proposed more than 20 methods in this area, empirical case studies applying these methods on large-scale industrial systems are rare. The costs and benefits of these methods remain unknown. On this behalf, we have applied the Cheung method on the software architecture of an industrial control system from ABB consisting of more than 100 components organized in nine subsystems with more than three million lines of code. We used the Littlewood/Verrall model to estimate subsystems failure rates and logging data to derive subsystem transition probabilities. We constructed a discrete time Markov chain as an architectural model and conducted a sensitivity analysis. This paper summarizes our experiences and lessons learned. We found that architecture-based software reliability analysis is still difficult to apply and that more effective data collection techniques are required.

Object-Relational Mapping (ORM) provides developers a conceptual abstraction for mapping the application code to the underlying databases. ORM is widely used in industry due to its convenience; permitting developers to focus on developing the business logic without worrying too much about the database access details. However, developers often write ORM code without considering the impact of such code on database performance, leading to cause transactions with timeouts or hangs in large-scale systems. Unfortunately, there is little support to help developers automatically detect suboptimal database accesses. In this paper, we propose an automated framework to detect ORM performance anti-patterns. Our framework automatically flags performance anti-patterns in the source code. Furthermore, as there could be hundreds or even thousands of instances of anti-patterns, our framework provides sup- port to prioritize performance bug fixes based on a statistically rigorous performance assessment. We have successfully evaluated our framework on two open source and one large-scale industrial systems. Our case studies show that our framework can detect new and known real-world performance bugs and that fixing the detected performance anti- patterns can improve the system response time by up to 98%.

An effective system regression testing for consecutive releases of very large software systems, such as modern telecommunications systems, depends considerably on the selection of test cases for execution. Classification models can classify, early in the test planning phase, those test cases that are likely to detect faults in the upcoming regression test. Due to the high uncertainties in regression test, classification models based on fuzzy logic are very useful. Recently, methods have been proposed for automatically generating fuzzy if-then rules by applying complicated rule generation procedures to numerical data. In this research, we introduce and demonstrate a new rule-based fuzzy classification (RBFC) modeling approach as a method for identifying high effective test cases. The modeling approach, based on test case metrics and the proposed rule generation technique, is applied to extracting fuzzy rules from numerical data. In addition, it also provides a convenient way to modify rules according to the costs of different misclassification errors. We illustrate our modeling technique with a case study of large-scale industrial software systems and the results showed that test effectiveness and efficiency was significantly improved.

In large-scale industrial recommendation systems, model checkpoints are instrumental in maintaining training goodput and numerical correctness during system failures and job preemptions. The increasing prevalence of multi-terabyte models has rendered frequent regular model checkpoints impractical, resulting in substantial lost progress when recovering from failures. As model sizes continue to grow, researchers and practitioners are compelled to investigate more efficient and scalable solutions. This paper presents DECK, a novel approach to delta model checkpointing designed for real-world industrial systems. Specifically, DECK focuses on extracting delta states with near-zero overhead, staging and streaming delta checkpoints without interrupting the training process, and merging delta checkpoints in an optimal and decoupled manner. Experimental results demonstrate that DECK achieves a 12-fold increase in checkpoint frequency while maintaining negligible impact on training throughput, thereby attaining state-of-the-art (SOTA) production performance.

Non-deterministically behaving test cases cause developers to lose trust in their regression test suites and to eventually ignore failures. Detecting flaky tests is therefore a crucial task in maintaining code quality, as it builds the necessary foundation for any form of systematic response to flakiness, such as test quarantining or automated debugging. Previous research has proposed various methods to detect flakiness, but when trying to deploy these in an industrial context, their reliance on instrumentation, test reruns, or language-specific artifacts was inhibitive. In this paper, we therefore investigate the prediction of flaky tests without such requirements on the underlying programming language, CI, build or test execution framework. Instead, we rely only on the most commonly available artifacts, namely the tests' outcomes and durations, as well as basic information about the code evolution to build predictive models capable of detecting flakiness. Furthermore, our approach does not require additional reruns, since it gathers this data from existing test executions. We trained several established classifiers on the suggested features and evaluated their performance on a large-scale industrial software system, from which we collected a data set of 100 flaky and 100 non-flaky test- and code-histories. The best model was able to achieve an F1-score of 95.5% using only 3 features: the tests' flip rates, the number of changes to source files in the last 54 days, as well as the number of changed files in the most recent pull request.

After investigating the conception and features of PLC (Programmable Logic Controller) and embedded system, in this paper the conception of a Quasi-PLC is proposed, which combine the advantages of PLC and embedded system. The architecture of Quasi-PLC and a Top-Down design method with collaboration design of software and hardware for Quasi-PLC embedded control system are researched then. Finally, setting the large-scale industrial valve control system as an example, the conception of Quasi-PLC and related technologies are used in industrial valve control system, and a satisfied application effect is obtained, which proves the rationality of the conception of Quasi-PLC and related technologies.