A comparison of usability evaluation methods for evaluating e-commerce websites

The Ganesha University of Education Library uses the INLISLite 3.1 application to facilitate library services and management. The application helps the library monitor the availability of books, new book lists, book borrowing, and book returning. This research was conducted in order to evaluate the usability of the application to determine the level of effectiveness, errors, and user satisfaction by evaluating responses to the system using the User Experience Questionnaire (UEQ). The response test results are evaluated by the Heuristic Evaluation method and the Think Aloud method. Then data analysis is carried out so that it will produce qualitative and quantitative data. Qualitative data based on evaluation with the Heuristic Evaluation Method and Think Aloud Method on end users, and quantitative data based on user response testing. The UEQ results for 25 users state that the clarity and novelty aspects get above average results, which means 25% of dataset products are considered better while the other 50% of products are considered worse. Then for the aspects of Attractiveness, Efficiency, Accuracy, and Stimulation got good results, which means 10% of dataset products are considered better while 75% of other products are considered worse. The results of the Heuristic Evaluation conducted by three expert evaluators showed that there were no problems that had the highest serverity rating that needed to be fixed immediately. There are only two minor problems and eight cosmetic problems. The results of Think Aloud on 10 users revealed problems that on average experience the same problems in their use. The data that has been obtained from the three methods will later be used as a reference for application improvement recommendations in the form of wireframes. Keywords: Usability INLISLite 3.1, User experience Questionnaire (UEQ), Heuristic Evaluation, Think Aloud, end user, saverty rating, wireframe.

The administrative and final project service submission system is an information system that will be used at the Information Technology Department of Tadulako University in terms of carrying out the administrative process and final assignments for both lecturers, education staff, and students. SIPENDEKAR is designed for lecturers, education personnel, and students SIPENDEKAR is designed to integrate all administrative processes and final assignments. This research uses the heuristic evaluation method. The heuristic evaluation method is a method used to identify problems in the use of interface design, so that usability problems can be corrected through the redesign process. Through the application of Heuristic Evaluation, we can find obstacles and receive suggestions for improvement from experts based on usability principles that apply to websites that have 10 principles, namely Visibility of system status, Match between system and the real world, User control and freedom, Consistency and standards, Error Prevention, Recongnition rather than recall, Flexibility and efficiency of use, Aesthetic and minimalist design, Help users recognize, Dialogue, and recovers from errors, and Help and documentation. The questionnaire testing involved 37 respondents, the results of the validity and reliability test were declared valid and reliable. This research uses 6 research stages Identification of problems and literature studies, determination of evaluation methods, preparation for evaluation, and preparation for evaluation. The implications of this research show that the application of heuristic evaluation principles can identify and correct usability problems effectively, which in turn can increase productivity and user satisfaction in managing academic administration

Nowadays, e-commerce websites have become part of people’s daily lives; therefore, it has become necessary to seek help in assessing and improving the usability of the services of e-commerce websites. Essentially, usability studies offer significant information about users’ assessment and perceptions of satisfaction, effectiveness, and efficiency of online services. This research investigated the usability of two e-commerce web-sites in Saudi Arabia and compared the effectiveness of different behavioral measurement techniques, such as heuristic evaluation, usability testing, and eye-tracking. In particular, this research selected the Extra and Jarir e-commerce websites in Saudi Arabia based on a combined approach of criteria and ranking. This research followed an experimental approach in which both qualitative and quantitative approaches were employed to collect and analyze the data. Each of the behavioral measurement techniques identified usability issues ranging from cosmetic to catastrophic issues. It is worth mentioning that the heuristic evaluation by experts provided both the majority of the issues and identified the most severe usability issues compared to the number of issues identified by both usability testing and eye-tracking combined. Usability testing provided fewer problems, most of which had already been identified by the experts. Eye-tracking provided critical information regarding the page design and element placements and revealed certain user behavior patterns that indicated certain usability problems. Overall, the research findings appeared useful to user experience (UX) and user interface (UI) designers to consider the provided recommendations to enhance the usability of e-commerce websites.

Comparison of two heuristic evaluation methods for evaluating the usability of health information systems

One new type of a security system is a decentralized authorization system that allows an owner of a resource to decide who can use this resource - even without trust towards external third parties. The idea itself is fascinating, but how about its usability? Security systems are generally considered to be difficult to understand and use. This paper compares the usability features and problems of three decentralized authorization systems: PolicyMaker, KeyNote and SPKI/SDSI. The analysis is based on the Heuristic Evaluation and Cognitive Walkthrough usability test methods. The analysis suggests that approaches of public key infrastructure type, such as the SPKI/SDSI authorization system, have the best usability features but even then the current state of the art is far from the end-user requirements. Designing and measuring usability of these systems is hard, complicated by the fact that security is only a means to achieve the user goal, not the goal as such.

Keywords Cite This Article as User interface (UI) evaluation is an important aspect of software development, ensuring that software or systems are efficient and user satisfactory. Among the various usability testing methods available, Cognitive Walkthrough Method, Heuristic Evaluation Method and Concurrent Usability Testing Method stand out as most popular approaches. This research paper presents a comparative analysis of these testing methods, emphasizing their strengths, flaws, and applicability in different situations. Through a detailed analysis of each testing method's procedural steps, and practical concerns, we aim to provide insights into their effectiveness and suitability for evaluating User Interface (UI) designs. Additionally, we discuss potential synergies and inadequacies when combining these testing methods in practice. Our results and findings contribute to a deeper understanding of UI evaluation techniques and provide guidance for selecting the most appropriate approach for testing team’s specific needs.

The Indonesian government initiated the implementation of E-government as a public service in 2003 with Presidential Instruction No.3 of 2003 issuance of National Policy and Strategy for E-Government Development. This rule was strengthened in 2007 by Regulation of the Minister of Communication and Information Technology No.41/PER/M.KOMINFO/2007 concerning General Guidelines for National Information and Communication Technology Governance. However, many provincial government sites still have not implemented a usability test. This problem is reinforced by the website interface not meeting the standards set by the government. So, usability testing is needed to find the problems found on the provincial government website. The method used in this usability test is the heuristic evaluation method. The evaluation was carried out by involving three evaluators to evaluate 28 provincial government sites. The evaluator detects problems based on ten heuristic principles proposed by Jakob Nielsen and is assessed with a severity rating. analysis results show the province of Aceh and the Special Region of Yogyakarta website is ranked first with the lowest severity rating value with a value of 0.43. In contrast, North Kalimantan is ranked last with the highest severity rating value with a value of 1.08. Around 71.4% of provincial government websites have no problems with user control & freedom, and consistency & standards. Then half of the provincial government's websites have problems with recognition rather than recall. Last, more than 50% of websites have problems with other heuristic principles.

Heuristic evaluation is an informal method used to analyze the presentation of an interface design to several evaluators. Heuristic evaluation is done by conducting several interface tests and analyzing the interpretation of the interface. English Kids Talk application is an English learning application for elementary school children. Based on the test results on the initial version of the English Kids Talk Application, user interface improvements are essential. English Kids Talk (Ekids) application development of the Ekids application still does not pay attention to UX. However, User Experience improvements are also necessary, minimizing disruption to the comfort and loyalty of users English Kids Talk application. In this paper, we propose prototype recommendations with a low evaluation value that improving User Experience. A prototype recommendation will be generated that suits the user's needs. The result is usability testing and an interaction cost between the initial version of the English Kids Talk Application and the new version. The result shows higher interaction costs from the new English Kids Talk application than the old one. Usability testing of the new English Kids Talk application is better than the old one.

The growth of the Internet and related technologies has enabled the development of a new breed of dynamic websites that is growing rapidly in use and that has had a huge impact on many businesses. One type of websites that have been widely spread and are being widely adopted is the educational websites. There are many forms of educational websites, such as free online websites and Web-based server software. This creates challenges regarding their continuing evaluation and monitoring in order to measure their efficiency and effectiveness, to assess user satisfaction and, ultimately, to improve their quality. The lack of an adaptive usability checklist for improvement of the usability assessment process for educational systems represents a missing piece in ‘usability testing’. This paper presents an adaptive Domain-Specific Inspection (DSI) checklist as a tool for evaluating the usability of educational systems. The results show that the adaptive educational usability checklist helped evaluators to facilitate the evaluation process. It also provides an opportunity for website owners to choose the usability area(s) that they think need to be evaluated. Moreover, this method was more efficient and effective than user testing (UT) and heuristics evaluation (HE) methods.

Considering the importance associated with e-commerce website accessibility and usability, a study on one of the most relevant Portuguese e-commerce websites has been performed using both automatic and manual assessment procedures. In an initial stage, we evaluated the chosen website with a Web accessibility and usability automatic tool called SortSite; after that, we performed a manual evaluation to verify each previously detected error and present possible solutions to overcome those faults. In a third phase, three usability specialists have been used to perform a heuristic evaluation of the chosen website. Finally, user tests with blind people were carried out in order to fully assess the compliance with accessibility and usability guidelines and standards. The results showed that the platform had a good score regarding the automatic evaluation; however, when the heuristic and manual evaluations were performed, some accessibility and usability problems were discovered. Moreover, the user test results showed bad marks regarding efficiency, effectiveness, and satisfaction by the group of participants. As a conclusion, we highlight user interaction problems and propose seven recommendations focused on enhancing accessibility and usability of not only the evaluated e-commerce website, but also of other similar ones.

The complexity of an electronic medication administration record (eMAR) has been underestimated by most designers in the past. Usability issues, such as poorly designed user application flow in eMAR, are therefore of vital importance, since they can have a negative impact on nursing activities and result in poor outcomes. The purpose of this study was to evaluate the usability of an eMAR application during its development. A usability evaluation was conducted during the development of the eMAR application. Two usability methods were used: a heuristic evaluation complemented by usability testing. Each eMAR application version provided by the vendor was evaluated by 2 hospital pharmacists, who applied the heuristic method. They reviewed the eMAR tasks, detected usability problems and their heuristic violations, and rated the severity of the usability problems. Usability testing was used to assess the final application version by observing how 3 nurses interacted with the application. Thirty-four versions were assessed before the eMAR application was considered usable. During the heuristic evaluation, the usability problems decreased from 46 unique usability problems in version 1 (V1) to 9 in version 34 (V34). In V1, usability problems were categorized into 154 heuristic violations, which decreased to 27 in V34. The average severity rating also decreased from major usability problem (2.96) to no problem (0.23). During usability testing, the 3 nurses did not encounter new usability problems. A thorough heuristic evaluation is a good method for obtaining a usable eMAR application. This evaluation points key areas for improvement and decreases usability problems and their severity.

A number of conventional methods are available for usability testing of smartphones, but their effectiveness is yet to be established. The objective of this study was to compare the effectiveness of the usability testing methods in identifying the problems of a smartphone design. Three usability methods, Think-Aloud (TA), Cognitive Walkthrough (CW), and Heuristic Evaluation (HE), were selected for comparison, and 15 smartphone users and 16 usability experts were chosen as participants in the comparative study. The results of this study indicate that HE method is the most effective one and has the highest severity rating. Additionally, this study finds that eight evaluators are sufficient for using HE method to find most of the usability problems. There has been substantial increase recently in smartphone sales that trigger a tough competition among the smartphone manufacturers. The usability of the smartphone has become among most smartphone users consider before buying their phones. This study proposed a guideline for usability testing of smartphones.

Introduction and purpose:Radiology information system (RIS) in order to reduce workload and improve the quality of services must be well-designed. Heuristic evaluation is one of the methods that understand usability problems with the least time, cost and resources. The aim of present study is to evaluate the usability of RISs in hospitals.Research Method:This is a cross-sectional descriptive study (2015) that uses heuristic evaluation method to evaluate the usability of RIS used in 3 hospitals of Tabriz city. The data are collected using a standard checklist based on 13 principles of Nielsen Heuristic evaluation method. Usability of RISs was investigated based on the number of components observed from Nielsen principles and problems of usability based on the number of non-observed components as well as non-existent or unrecognizable components.Results:by evaluation of RISs in each of the hospitals 1, 2 and 3, total numbers of observed components were obtained as 173, 202 and 196, respectively. It was concluded that the usability of RISs in the studied population, on average and with observing 190 components of the 291 components related to the 13 principles of Nielsen is 65.41 %. Furthermore, problems of usability were obtained as 26.35%.Discussion and Conclusion:The established and visible nature of some components such as response time of application, visual feedbacks, colors, view and design and arrangement of software objects cause more attention to these components as principal components in designing UI software. Also, incorrect analysis before system design leads to a lack of attention to secondary needs like Help software and security issues.

Laboratorium mandiri (iLab) Univeristas Gunadarma merupakan salah satu lab yang memberikan kemudahan dalam memberikan informasi melalui website. Kemudahan dalam mengakses website tidak lepas dari usabilitas website tersebut. Usabilitas yang baik sangat berpengaruh bagi pengguna, semakin baik usabilitasnya semakin mudah pengguna menggunakan website tersebut. Agar suatu website dapat dilihat oleh pengguna dengan waktu yang cukup lama, maka usabilitas adalah syarat penting yang harus dipenuhi. Dalam penelitian ini metode yang digunakan adalah heuristic evaluation. Heuristic adalah metode evaluasi yang digunakan untuk menemukan kesalahan pada desain antar muka. Berdasarkan pengukuran website iLab menggunakan metode evaluasi heuristic oleh Nielsen didapatkan bahwa 10 aspek yang diteliti mendapatkan nilai satu dan dua, yang berarti website memiliki beberapa kekurangan yang tidak menjadikan sebuah masalah atau dengan kata lain tidak dipermasalahkan dan tidak mengganggu pengguna saat mengakses website. Nilai severity rating tertinggi yaitu sebesar 2.00 terdapat pada aspek membantu pengguna untuk mengenali, mendiagnosa, dan memulihkan dari kesalahan.

Pubg Mobile game is a game developer from "Tencent Games", Pubg Mobile was released on February 9, 2018. The genre of this game is Battle royale, where players can play with 100 people at once online. There are several negative reviews from Pubg Mobile game users, namely, character bugs often occur during the game, often force close on 3 GB RAM smartphones and loading into the game is very long so that users are very uncomfortable when using it. This study uses the Heuristic Evaluation method as a measurement of user experience. This study aims to evaluate the user experience in the Pubg Mobile game application using the Heuristic Evaluation method. The results of this study indicate that the Visibility Of System Status variable gets a total value of p1 75.6%, total p2 72.9%, total p3 75.6% with the status "Good", Match Between System and Real World gets a total value of p1 64.8%, total p2 75.6%, total p3 72.9% with the status "Good". User Control And Freedom gets a total value of p1 72.9%, total p2 75.6%, with the status of "Good", Consistency And Standards gets a total value of p1 51.3%, total p2 70.2% with the status of "Not Good", Error Prevention gets a total value of p1 70.2%, total p2 70.2%, with the status of "Good", Recognition Rather Than Recall gets a total value of p1 72.9%, total p2 72.9%, with the status of "Good", Flexibility And Efficiency Of Use gets a total value of p1 56.7%, with the status of "Quite Good", Aesthetic And Minimalist Design gets a total value of p1 78.3%, with the status of "Good", Help User Recognize, Diagnose And Recover From Errors gets a total value of p1 78.3%, with the status of "Good", and the last variable Help And Documentation gets a total p1 78.4%, with the status of "Good". It is necessary to make improvements or enhancements to the Consistency And Standards variables to get better values ​​than before.