Assessment of access control systems using mutation testing

As the need for more pervasive and more complex access controls grows, the challenge of ensuring the performance of access control systems is becoming apparent. Researchers have proposed several solutions to mitigate performance problems, including: adjusting the policy set; re-engineering the policy decision point (PDP); and decomposing the policies and distributing their evaluation. However, since the benefits and tradeoffs depend heavily upon the actual scenario, security administrators typically do not have objective justification for adopting particular mitigation actions. In response, we present the ATLAS framework, comprising: 1) DomainManager , which facilitates modelling the domain as closely as possible and automatically generates large numbers of representative policies and associated requests; 2) STACS , which enables controlled experiments to be performed using the generated policies/requests, to collect comprehensive measurements of PDP performance; and 3) PARPACS , which aids the understanding and worth of the measurement data and, by using rigorous validation techniques, reduces the risk of spurious insights or incorrect recommendations. We present a discussion of ATLAS as applied to an enterprise communication scenario, where access control is realised via XACML PDPs. Notable insights include that the SunXacml 2.0 PDP performs relatively poorly in terms of policy evaluation performance and that adding additional memory and/or processor cores to a XACML PDP server is not guaranteed to improve performance significantly.

Traditional access control systems often use the extensible access control markup language (XACML), a widely adopted standard for defining and enforcing access control policies. XACML is flexible and compatible with various access control models. It comprises key components such as the Policy Decision Point (PDP), Policy Enforcement Point (PEP), Policy Administration Point (PAP), and Policy Information Point (PIP), which are centrally managed in traditional setups. However, this central management introduces vulnerabilities like Single Point of Failure (SPOF) and other security risks. This research addresses these challenges by proposing a decentralized access control architecture built on Hyperledger Fabric. The design distributes the four core components of XACML— PEP, PDP, PAP, and PIP- across multiple blockchain nodes, eliminating SPOF while enhancing scalability and data integrity. A proof of concept (PoC) implementation is developed to evaluate the system’s performance. The architecture and PoC are analyzed against predefined threats and requirements, demonstrating how the system delivers a secure, scalable, and resilient access control solution.

Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

In conventional centralized authorization models, the evaluation performance of policy decision point (PDP) decreases obviously with the growing numbers of rules embodied in a policy. Aiming to improve the evaluation performance of PDP, a distributed policy evaluation engine called XDPEE is presented. In this engine, the unicity of PDP in the centralized authorization model is changed by increasing the number of PDPs. A policy should be decomposed into multiple subpolicies each with fewer rules by using a decomposition method, which can have the advantage of balancing the cost of subpolicies deployed to each PDP. Policy decomposition is the key problem of the evaluation performance improvement of PDPs. A greedy algorithm with O(nlgn) time complexity for policy decomposition is constructed. In experiments, the policy of the LMS, VMS, and ASMS in real applications is decomposed separately into multiple subpolicies based on the greedy algorithm. Policy decomposition guarantees that the cost of subpolicies deployed to each PDP is equal or approximately equal. Experimental results show that (1) the method of policy decomposition improves the evaluation performance of PDPs effectively and that (2) the evaluation time of PDPs reduces with the growing numbers of PDPs.

Mutation testing is a well known technique for estimating and improving the quality of test suites. Given a test suite T for a system S, mutation testing systematically creates mutants of S and executes T to measure how many mutants T detects. If T does not detect some (non-equivalent) mutants, T can be improved by adding test cases that detect those mutants. Mutants are created by applying mutation operators. Mutation operators are important because they define the characteristics of the system that are tested as well as the characteristics that are improved in the test suite. While mutation operators are well defined for a number of programming paradigms such as sequential or multi-threaded, to the best of our knowledge, mutation operators have not been defined for the actor programming model. In this paper, we define and classify mutation operators that can be used for mutation testing of actor programs.

A probabilistic framework for mutation testing in deep neural networks

An efficient policy evaluation engine for XACML policy management

SummaryFor mutation testing, the huge cost of running test suites on a large number of mutants has been a serious obstacle. To resolve this problem, we propose a learning‐based mutant reduction techniqueMuTrain.MuTrainuses cost‐considerate linear regression (i.e., CLARS) to learn amutation model, which predicts the mutation score of a test suite based on the mutation testing results of a previous version of a target program. Then,MuTrainapplies the mutation model for subsequent versions to predict mutation scores with significantly fewer mutants. For effective mutant reduction and accurate mutation score prediction,MuTrainuses fine‐grained mutation operators refined from the existing coarse‐grained mutation operators. The experiment results show thatMuTrainreduces the number of mutants effectively (i.e., selecting only 1.6% of mutants). Moreover,MuTrainpredicts mutation score far more accurately than the existing mutant reduction techniques and random mutant selection. We also found thatMuTrainachieves much greater mutant reduction when it uses the fine‐grained mutation operators than the traditional coarse‐grained mutation operators (i.e., 1.6% vs. 14.6%).

Mutation analysis is extensively used for the comparison of state-based testing methods that work from a finite state machine (FSM); In this thesis, we report on results from an experiment during which we compared different mutation operators used to generate FSM mutants. We randomly generated multiple synthetic FSMs and selected real-world FSMs; We created all the possible mutants from a complete set of FSM mutation operators; We generated multiple test suites using a pool of test paths; We executed all those test suites on FSM mutants while using different types of oracles, and compared mutation operators using a metric we define. Results help us identify mutation operators that lead to easy-to-reveal faults, possibly too easy to help discriminate between test suite construction techniques, as well as study the impact of various oracle strategies. We have developed a tool infrastructure called μFSM that automates the whole process of this experiment.

Test suites play an important role in developing reliable software applications. Generally, the behaviour of software applications is verified by executing test suites to find defects. The quality of a test suite needs to be evaluated and enriched (if needed) especially for testing critical systems, such as plane-navigation system. This paper presents a novel method for comparing concrete and executable test suites using equivalence classes. This comparison identifies gaps in test suites with respect to each other. These gaps indicate potential weaknesses in the test suites. Furthermore, this method provides a mechanism to enrich the test suites using these gaps. In this method, we devise equivalence classes, and associate each test case to an equivalence class. We, then, simulate the comparison of test suites by comparing sets of equivalence classes. The method compares test suites in a platform independent manner. The test suites, which are compared, are smaller than the original test suites because the redundant test cases are removed from the test suites, which makes it efficient. We exercise our method over three case studies to demonstrate its viability and effectiveness. The first case study illustrates the application of the method and evaluates its effectiveness using a mutation analysis. The second case study evaluates its effectiveness using mutation and coverage analyses. The final case study evaluates it on a real case study, which is Lucene search engine.

Traditionally, mutation testing is used as test set generation and/or test evaluation criteria once it is considered a good fault model. This paper uses mutation testing for evaluating an automated static analyzer. Since static analyzers, in general, report a substantial number of false positive warnings, the intention of this study is to define a prioritization approach of static warnings based on their correspondence with mutations. On the other hand, knowing that Mutation Test has a high application cost, another possibility is to try to identify mutations of some specific mutation operators, which an automatic static analyzer is not adequate to detect. Therefore, this information can be used to prioritize the order of incrementally applying mutation operators considering, firstly, those with no correspondence with static warnings. In both cases, contributing to the establishment of incremental strategies on using automatic static analysis or mutation testing or even a combination of them. We used mutation operators as a fault model to evaluate the direct correspondence between mutations and static warnings. The main advantage of using mutation operators is that they generate a large number of programs containing faults of different types, which can be used to decide the ones most probable to be detected by static analyzers. We provide evidences on the correspondence between mutations and some types of static warnings. The results obtained for a set of 19 open-source programs indicate that: 1) static warnings may be prioritized based on their correspondence level with mutations; 2) specific set of mutation operators and their mutations may be prioritized based on their correspondence level with warnings. It is possible to provide an incremental testing strategy aiming at reducing the cost of both static analysis and mutation testing using the correspondence information between these activities/artifacts.

Mutation testing is the fault-based software testing approach that is widely applicable for assessing the effectiveness of a test suite. The test suite effectiveness is measured through artificial seeding of faults into the programs under test. Mutation testing tools (MTTs) are significant enablers of the conversion of mutation testing from the research perspective into the real life and mostly applicable testing process. Without using the automatic MTT, mutation testing can’t be really connected in this present reality and is unrealistic to be acknowledged by the industry. Authors analyze six open-source JAVA-based MTT (Jester, JavaMut, MuJava, Jumble, Judy, and Javalanche). The results are based on the performance of various JAVA programs and two real-life applications. Benchmark comparison among the MTT is presented in terms of mutants, mutation operator, mutation score, and quality output. On the basis of comparative analysis, the performance of each tool is explained with the protocol for finding the appropriate tool among the six MTTs. The results show that the MuJava performs best compared to the others.KeywordsMutation testingFault-based testingAdequacy criterionMutation testing tools

Android applications have experienced an outstanding growth in recent years with a projected figure of 378 billion in the number of application downloads by 2020. As a result, verification and validation of Android apps and their graphical user interface through testing is crucial for quality assurance. One way to assess quality assurance is through the use of mutation testing. Mutation testing modifies the code of the System Under Test in small ways in order to produce faulty versions of the program called mutants and compare them to the original version. However, the cost of mutation testing can be expensive and time consuming. We propose a graphical user-friendly Android mutation tool called Edroid whose primary purpose is to mutate Android's main components such as activities, services, content providers and broadcast receivers using the source code of XML files. We introduce fourteen Android mutation operators, where ten of them are new, that are derived from operators used in different fields such as mutation testing for Graphical User Interface and XML schema. We evaluate Edroid on five real world Android applications using testing methodologies such as random testing and black box testing. Results from these experiments demonstrate that Edroid facilitates the generation of mutants. In addition and equally important, the set of proposed mutation operators are effective in revealing errors that improve the quality of a given test suite.

Traditional access control mechanisms prevent illegal access by controlling access right before executing an action; they belong to a class of a priori security solutions and, from this point of view, they have some limitations, like inflexibility in unanticipated circumstances. By contrast, a posteriori mechanisms enforce policies not by preventing unauthorized access, but rather by deterring it. Such access control needs evidence to prove violations. Evidence is derived from one or several log records, which trace each user's actions. Efficiency of violation detection mostly depends on the compliance of log records with the access control policy. In order to develop an efficient method for finding these violations, we propose restructuring log records according to a security policy model. We illustrate our methodology by applying it to the healthcare domain, taking care of the IHE (Integrating the healthcare enterprise) framework, particularly its basic security profile, ATNA (Audit Trail and Node Authentication). This profile defines log records established on the analysis of common health practice scenarios. We analyze and establish how ATNA log records can be refined in order to be integrated into an a posteriori access and usage control process, based on an expressive and contextual security policy like the OrBAC policy.

Mutation testing is a technique in which faults (mutants) are injected into a program or application to assess its test suite effectiveness. It works by inserting mutants and running the application’s test suite to identify if the mutants are detected (killed) or not (survived) by the tests. Although computationally expensive, it has proven to be an effective method to assess application test suites. Several mutation testing frameworks and tools have been built for the various programing languages, however, very few tools have been built for the JavaScript language, more specifically, there is a lack of mutation testing tools for the Node.js runtime and npm based applications. The npm Registry is a public collection of modules of open-source code for Node.js, front-end web applications, mobile applications, robots, routers, and countless other needs of the JavaScript community. The over 700,000 packages hosted in npm are downloaded more than 5 billion times per week. More and more software is published in npm every day, representing a huge opportunity to share code and solutions, but also to share bugs and faulty software. In this paper, we briefly describe prior work for mutation operators in JavaScript and Node.js, and propose Mutode, an open source tool which leverages the npm package ecosystem to perform mutation testing for JavaScript and Node.js applications. We empirically evaluated Mutode effectiveness by running it on 12 of the top 20 npm modules that have automated test suites.