Using artificial intelligence to automatically test GUI

Software testing in general and graphical user interface (GUI) testing in particular is one of the major challenges in the lifecycle of any software system. GUI testing is inherently more difficult than the traditional and command-line interface testing. Some of the factors that make GUI testing different from the traditional software testing and significantly more difficult are: a large number of objects, different look and feel of objects, many parameters associated with each object, progressive disclosure, complex inputs from multiple sources, and graphical outputs. The existing testing techniques for the creation and management of test suites need to be adapted/enhanced for GUIs, and new testing techniques are desired to make the creation and management of test suites more efficient and effective. In this article, a methodology is proposed to create test suites for a GUI. The proposed methodology organizes the testing activity into various levels. The tests created at a particular level can be reused at higher levels. This methodology extends the notion of modularity and reusability to the testing phase. The organization and management of the created test suites resembles closely to the structure of the GUI under test.

Mark Watson Morgan Kaufmann Publishers, San Francisco, 1997, 377 pp. ISBN 1-55860-420-0, $44.95 This book is well written and organized to cover how to apply AI (Artificial Intelligence) techniques for practical Internet and Intranet tools by using the Java programming language. In the book, the author uses neural network and genetic algorithms to develop distributed systems that are applied for the Internet and Intranets, such as client/server model and information retrieval tool and so on. Part 1 begins with a good general introduction to AI. The history and survey of Artificial Intelligence research are given briefly. In addition, symbolic processing versus neural networks approaches are discussed. This author opts to use neural network to develop example Internet/Intranet systems in the book. Part 2 consists of Java programs developed for AI, neural networks, genetic algorithms, natural language processing, and expert systems. Chapter 2 illustrates Graphical User Interface (GUI) class that is derived from Applet and Canvas classes in Java, which is used in most example programs in the book. Chapter 3 introduces a frame data structure that is used as a message in distributed systems in the later chapters. Chapter 4 shows neural network classes developed in Java and its theory and implementation. Chapter 5 explains the genetic algorithms and the Java genetic algorithm classes. Chapter 6 describes how to develop and implement natural language processing classes using Java. Chapter 7 shows how to share data objects in distributed systems. For example, distributed AI agent classes are developed to share data objects, WWW data collection agent classes are shown to retrieve documents in the Internet, and small POP (Post Office Protocol) mail classes are developed to send and receive mails. Chapter 8 illustrates how to develop expert systems by using Java Jess, which is based on OPS5/CLIPS expert systems. Part 3 explains how to generate and implement the projects using the Java classes that are developed in Part 2. Chapter 9 shows the project that demonstrates the use of genetic algorithms in chapter 5 to control enemy ships in a scrolling arcade game. Chapter 10 describes the real-time handwriting recognition system developed by using the neural network classes in chapter 4. Chapter 11 illustrates a greedy algorithm to optimize neural networks that is based on the author’s hand-tuning neural training data. Chapter 12 improves neural network training data by applying genetic algorithms, to discard bad training data and to avoid discarding good training data. Chapter 13 describes the distributed natural language processing system that uses natural language processing classes in chapter 6. The data structures in chapter 2 are used to store historical data in a separate server application on any other computer in the Internet or Intranets. The final chapter provides the design and implementation of a complete agent-based information retrieval tool for the Internet. The retrieved documents are stored in ASCII or HTML format. This book is a very good reference book for anyone interested in developing systems using Java in distributed environments, especially applying AI techniques. Also, the author provides the CD-ROM that includes the source code for Java classes developed in the book. Haklin Kimm, University of Tennessee

تناولت الدراسة الحالية واحدة من أهم المشاكل التي تعاني منها المنظمات الرياضية عالية الاداء وهي عدم الاستفادة القصوى من تطبيقات تقنيات الذكاء الاصطناعي، فلقد تمثلت مشكلة البحث الأساسية في معرفة أثر تطبيق تقنيات الذكاء الاصطناعي في ادارة المنظمات الرياضية عالية الأداء وما مدى مساهمة تطبيقات تقنيات الذكاء الاصطناعي في تشخيص ومعالجة المعوقات الادارية والتقنية في اقسام النشاط الرياضي والمدرسي في المديريات العامة للتربية في محافظة بغداد للتوجه نحو الاداء العالي وفي هذا السياق، فلقد تمثل هدف الدراسة الرئيس في بناء مقياس للاطار المفاهيمي لتطبيقات تقنيات الذكاء الاصطناعي في المنظمات الرياضية عالية الاداء -اقسام النشاط الرياضي والمدرسي في المديريات العامة للتربية في محافظة بغداد أنموذجاً. ولغرض تحقيق أهداف الدراسة اتبعت الباحثتان المنهج الوصفي بالأسلوب التحليلي لملامته لطبيعة المشكلة كذلك القيام بدراسة استطلاعيه غير مقننه في اقسام النشاط الرياضي والمدرسي وكانت الاستبانة هي الأداة المستخدمة في ذلك، وتوصلت الدراسة إلى بناء مقياس رصين لتطبيقات تقنيات الذكاء الاصطناعي في المنظمات الرياضية عالية الاداء ومكون من خمس محاور وهي النظم الخبيرة والواقع الافتراضي والوكلاء الاذكياء وتقنيات المعلومات واتمتة العمليات وقد جاء اختيار حجم العينة مساويا للمجتمع لتلام طبيعة البحث، وقد تم اثبات صدق وموثوقية هذا المقياس من خلال الوسائل الإحصائية كمعامل الفا كرو نباخ بمقدار 89,7، لقد احتوت محاور المقياس الخمسة على خمسة وعشرون عبارة، ولقد تراوحت الأهمية النسبية لهذه العبارات ما بين (63-84)%، وكانت اعلى قيمه للأهمية النسبية تبلغ 84% لعباره الثانية من محور النظم الخبيرة وهي (تعمل النظم الخبيرة كخبير استشاري للإدارة العليا في اقسام النشاط الرياضي والمدرسي لتسهم في اتخاذ القرارات الصحيحة)، بينما كانت اقل قيمة للأهمية النسبية لعبارة الأولى من محور الواقع الافتراضي وهي (استخدام الواقع الافتراضي في اقسام النشاط الرياضي والمدرسي يسهم في اكتساب مهارات متنوعة للموظفين ويعمل على تطوير الاداء الاداري والتقني)

The practitioner interested in reducing software verification effort may found herself lost in the many alternative definitions of Graphical User Interface (GUI) testing that exist and their relation to the notion of system testing. One result of these many definitions is that one may end up testing the same parts of the Software Under Test (SUT), specifically the application logic, twice. To clarify two important testing activities and avoid duplicate testing effort, this paper empirically evaluates to what extent GUI tests exercise the application logic of the software under test (and not only the GUI code). Experimental results show that GUI tests do not necessarily entirely exercise application logic functionality, at least not as much as system tests directly interacting with application logic code.

The study comprehensively reviews artificial intelligence (AI) techniques for addressing algorithmic bias in job hiring. More businesses are using AI in curriculum vitae (CV) screening. While the move improves efficiency in the recruitment process, it is vulnerable to biases, which have adverse effects on organizations and the broader society. This research aims to analyze case studies on AI hiring to demonstrate both successful implementations and instances of bias. It also seeks to evaluate the impact of algorithmic bias and the strategies to mitigate it. The basic design of the study entails undertaking a systematic review of existing literature and research studies that focus on artificial intelligence techniques employed to mitigate bias in hiring. The results demonstrate that the correction of the vector space and data augmentation are effective natural language processing (NLP) and deep learning techniques for mitigating algorithmic bias in hiring. The findings underscore the potential of artificial intelligence techniques in promoting fairness and diversity in the hiring process with the application of artificial intelligence techniques. The study contributes to human resource practice by enhancing hiring algorithms’ fairness. It recommends the need for collaboration between machines and humans to enhance the fairness of the hiring process. The results can help AI developers make algorithmic changes needed to enhance fairness in AI-driven tools. This will enable the development of ethical hiring tools, contributing to fairness in society.

Graphical User Interface (GUI) has become one of the most significant parts of mobile applications (apps). It is a direct bridge between mobile apps and end users, which directly affects the end user’s experience. Neglecting GUI quality can undermine the value and effectiveness of the entire mobile app solution. Significant research efforts have been devoted to GUI testing, one effective method to ensure mobile app quality. By conducting rigorous GUI testing, developers can ensure that the visual and interactive elements of the mobile apps not only meet functional requirements but also provide a seamless and user-friendly experience. However, traditional solutions, relying on the source code or layout files, have met challenges in both effectiveness and efficiency due to the gap between what is obtained and what app GUI actually presents. Vision-based mobile app GUI testing approaches emerged with the development of computer vision technologies and have achieved promising progress. In this survey article, we provide a comprehensive investigation of the state-of-the-art techniques on 271 articles, among which 92 are vision-based studies. This survey covers different topics of GUI testing, like GUI test generation, GUI test record & replay, GUI testing framework, and so on. In particular, we highlight the emerging role of vision-based techniques and analyze how they reshape traditional approaches to mobile app GUI testing. Based on the investigation of existing studies, we outline the challenges and opportunities of (vision-based) mobile app GUI testing and propose promising research directions with the combination of emerging techniques.

Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Software testing has become an important phase in software applications’ lifecycle. Graphical User Interface (GUI) components can be found in a large number of desktops and web applications and also in a wide variety of mobile devices. In the last years GUIs have become more and more complex and interactive, their testing process requiring an interaction with the GUI components, mainly by generating mouse, keyboard and touch events. Given their increased importance, GUIs verification for correctness contributes to the establishment of the correct functionality of the corresponding software application. The current research on GUI testing methodologies primarily focuses on automated testing. This paper presents EvoGUITest, a novel automated GUI testing framework based on evolutionary algorithms which tests the GUI independently from the application code itself. The framework is designed for testing GUIs of web applications. Results have been compared, based on specific metrics, with others existing frameworks.

Mobility for graphical user interfaces (GUIs) is a challenging problem, as different GUIs need to be constructed for different device capabilities and changing context, preferences and users’ locations. GUI developers frequently create multiple user interface versions for different devices. The solution lies in using a single, abstract, user interface description that is used later to automatically generate user interfaces for different devices. Various techniques are proposed to adapt GUIs from an abstract specification to a concrete interface. Design-time techniques have the possibility of creating better performing GUIs but, in contrast to run-time techniques, lack flexibility and mobility. Run-time techniques’ mobility and autonomy can be significantly improved by using mobile agent technology and an indirect GUI generation paradigm. Using indirect generation enables analysis of computer-human interaction and application of artificial intelligence techniques to be made at run-time, increasing GUIs’ performance and usability.

In recent times, software industry has seen the immense expansion in its popularity. Graphical user interface (GUI) is of the most important and ground breaking factor behind the exponential growth of acceptance of software systems. Beside this popularity, software organizations are trying best to optimize the quality of delivered products. In software development life cycle, software testing is one major phase having focus towards improved software quality. Increased attention towards GUI in development helped in materialization of GUI testing as a vital branch of software testing. Manual efforts to test GUI have revealed a lot of problems, which, ultimately led towards automation of GUI testing. This paper presents a novel approach towards automation of GUI test coverage analysis based on an evolutionary algorithm, Particle Swarm Optimization (PSO). Another objective that has been tried to achieve is keeping number of test cases at minimum side.

Software testing is one of the most important Software Engineering processes, being the primary activity to check the conformance between the software requirements and its actual behavior. The automation of software testing activities is essential to certify productivity and effectiveness in such activities. Test automation leads testing activities to be conducted under systematic and accurate criteria, raising the chance of testers to reveal faults or inconsistencies. Test oracles are elementary members in software testing automation, being the mechanism responsible for indicating the correctness of software outputs. In testing environments, test oracles can be effectively implemented based on several sources of information about the Software Under Testing (SUT): software specifications, assertions, formal methods (Finite State Machines (FSM), formal specifications, etc, machine-learning methods, and metamorphic relations. Regardless of the implementation strategy, test oracles are vulnerable to false positive/negative verdicts, configuring what the literature describes as "the oracle problem". Therefore, test oracles are a non-trivial and challenging object of studies of the software engineering research area. SUT's outputs in unusual formats make it harder the oracle problem. Audio, images, three-dimensional objects, virtual reality environments, complex statistical compositions, etc, are examples of non-trivial output formats. In the software testing context, SUTs with unusual outputs can be called "complex-output systems". In this doctorate dissertation, we propose and evaluate a novel test oracle approach for complex-output systems called feature-based test oracles. The purpose of feature-based test oracles is the appropriation of a processing image technique called Content-Based Image Retrieval (CBIR) to collect information from features extracted from the SUT's outputs to compose test oracles. Given a query image, CBIR combines feature extraction and similarity functions to alleviate the problem of searching for digital images in large databases. In previous research, we have integrated CBIR concepts in a testing framework to support the automation of testing activities in processing image systems and systems with Graphical User Interfaces (GUI). In this doctorate dissertation, we extended that framework and its concepts to general complex-output systems, addressing the feature-based test oracle approach. We use Text-To-Speech (TTS) systems to validate empirically our test oracle technique. Through the results of five empirical analyses, three of them conducted in line with problems of a real-world industry TTS system, show the proposed technique is a valuable instrument to automate testing activities and alleviate practitioners' efforts on testing complex output systems. We conclude the proposed test oracles are effective because they systematically evaluate the SUT's sensorial output rather than produce verdicts based on subjective specifications. As future work, we plan to conduct investigations towards the reduction of false positives/negatives and the association of the test oracles with machine learning techniques and metamorphic relations.

Artificial intelligence technology is an important transformative force for teaching innovation in the intelligent era. It is being widely used in American school teaching, including the design of intelligent tutoring systems to achieve precise problem solving, the machine learning technology to ensure personalized activity design, the creation of intelligent virtual reality to promote classroom teaching contextualization, and the development of intelligent evaluation systems to ensure the scientific evaluation of capabilities. In the process of advancing the teaching and application of artificial intelligence technology, the United States has built a linkage mechanism of federal leadership, university follow‐up, and social collaboration and implemented the smart technology in school teaching and professors’ academic governance. This paper is aimed at studying the professors’ academic governance of American research universities by Internet data mining, historical analysis method, documentary method, survey method, and other methods. Professors’ academic governance is a vital part of the modern university system that causes the institutional reform of the internal governance structure of modern universities. The United States is a powerful country in higher education, and professors in American research universities have always participated in university academic governance for centuries. By studying the definition, history, and development and mode of operation of professors’ academic governance in American research universities, the results indicate a clear division of power and responsibility between the professors and administrators based on an artificial intelligence decision system in American research universities. Also, there is a good communication platform based on artificial intelligence environment for professors to discuss their opinions on academic affairs. Third, professors exercise academic power under the guarantee of diversified guaranteed systems based on the artificial intelligence evaluation system and the ideology of mutual respect based on the artificial intelligence management and service system. Studying the application of artificial intelligence techniques in operating mode and enlightenment of professors’ academic governance in an American research university is of great significance to promote the construction of other modern universities’ professors’ academic governance system.

The purpose of the study. The techniques of artificial intelligence in educational hardware and software are able to deduce knowledge and skills, and the importance of the study is due to the use of artificial intelligence techniques in analyzing the situation of skill performance, the use of artificial intelligence techniques in the system of skill analysis of the skills under study, the link of artificial intelligence with kinetic variables For the skills under study and the study aimed to apply artificial intelligence techniques in skill analysis, building an artificial intelligence system from motor variables and motor performance. Materials and methods. The study samples The researcher determined the study population of karate players by choosing the study sample in a random way from karate players in Kafr El-Sheikh Sports Club. Results. Performance through data and recommendations came to use artificial intelligence techniques in various games and areas of physical education. Conclusions. Artificial intelligence techniques work on skill analysis through programming neurons. Deep learning gives indications of improving performance through data. The use of artificial intelligence techniques in various games and areas of physical education

The use of artificial intelligence (AI) techniques is significantly changing the analysis of medical images, accelerating and standardizing the diagnosis process. To train an AI model, however, a large dataset is typically required, especially when using the most powerful techniques. Therefore, not all specialties are taking advantage of AI techniques in the same way. For instance, they are seldomly used in areas such as the diagnosis of rare diseases since, due to their low prevalence, not enough data are typically available to train an AI model. In this paper, we address the use of AI techniques to diagnose a particular rare disease: Collagen VI-related Congenital Muscular Dystrophy from confocal microscopy images. We apply both classical machine learning and modern deep learning techniques and we show that, when using the appropriate data management and training procedures, one can successfully derive a highly-accurate classifier even with a limited amount of training data. Due to the generality of the explored techniques, this conclusion is likely to hold also for most of the rare diseases whose diagnosis relies on the examination of histological images.

The development of ever intelligent systems in various application fields is nowadays a hot research topic. Artificial Intelligence (AI) techniques become a key enabler of the transition between classical static, hard-coded algorithms and innovative, flexible ones. Actually, the automotive sector can undoubtedly benefit from the usage of the aforementioned techniques, aiming at building a novel smart automotive industry. Indeed, the application of AI spreads all round the automotive sector, ranging from on–board measuring systems to customer satisfaction analysis and demand prediction. This paper aims to review the possible applications of Artificial Intelligence techniques to the automotive sector, with a special focus on innovative measurement systems and metrology. Indeed the focus will be, between others, on Advanced Driver Assistance Systems (ADAS), in-vehicle IoT systems and intelligent industrial measuring systems, thus allowing to both increase road safety and design accurate predictive maintenance, additive manufacturing systems and, in substance, to build the smart automotive factory of the future.