A Method and Tool for Analyzing Usability Testing Data in the Business Sector

Scientific data volumes have been growing exponentially. This has resulted in the need for new tools that enable users to operate on and analyze data. Cyber infrastructure tools, including workflow tools, that have been developed in the last few years has often fallen short if user needs and suffered from lack of wider adoption. User-centered Design (UCD) process has been used as an effective approach to develop usable software with high adoption rates. However, UCD has largely been applied for user-interfaces and there has been limited work in applying UCD to application program interfaces and cyber infrastructure tools. We use an adapted version of UCD that we refer to as Scientist-Centered Design (SCD) to engage with users in the design and development of Tigres, a workflow application programming interface. Tigres provides a simple set of programming templates (e.g., sequence, parallel, split, merge) that can be can used to compose and execute computational and data transformation pipelines. In this paper, we describe Tigres and discuss our experiences with the use of UCD for the initial development of Tigres. Our experience-to-date is that the UCD process not only resulted in better requirements gathering but also heavily influenced the architecture design and implementation details. User engagement during the development of tools such as Tigres is critical to ensure usability and increase adoption.

—Thailand, as well as many other countries worldwide, is facing a shortage of medical staff. We propose a solution to improve medical services in rural areas: a robot teleoperation system that allow patients to consult with doctors from public hospitals, and for doctors to examine and make decisions about their required care. To develop such a system, we are following a user-centred design (UCD) process. Two research questions have been addressed: how should an effective robot teleoperation system be designed to support medical diagnosis and patient communication? And does the use of a social robot improve the social interaction and examination quality in a remote consultation? This paper discusses the overall UCD process, starting from requirement analysis with Thai end-users (finished), system design and development on Pepper robot (in-progress), and user-centred evaluation (future plan).

BackgroundInternet-based treatments for child anxiety may help to increase access to evidence-based therapies; however, user engagement, uptake, and adherence within routine clinical practice remain as challenges. Involving the intended end users in the development process through user-centered design and usability testing is crucial for maximizing user engagement and adoption of internet-based treatments, but so far this has been lacking for internet-based treatments for child anxiety.ObjectiveThe aim of this study is to develop an internet-based treatment for child anxiety through a process of user-centered design (phase 1) and usability testing (phase 2), based on an existing evidence-based, face-to-face, therapist-supported, parent-led cognitive behavioral therapy intervention. It is intended that the internet-based version of this treatment would consist of a parent website, case management system for clinicians, and mobile game app for children.MethodsParents, children, and clinicians who were familiar with the face-to-face version of the treatment were recruited from 2 National Health Service clinics. In phase 1, participants participated in 3 workshops to gain feedback on the overall concept, explore their wants and needs for the websites and game, generate ideas on how the treatment may look, and gain feedback on initial mock-ups of the websites and game. In phase 2, participants attended 3 individual usability testing sessions where they were presented with working prototypes of the website or game and asked to perform a series of tasks on the website (parents and clinicians) or play the game (children). The frequency and details on usability errors were recorded. Participants were asked for their feedback on the website and game using a standardized usability questionnaire and semistructured interviews. The websites and game were iterated after each round of usability testing in response to this feedback.ResultsIn phase 1, participants approved the general concept and rated the initial mock-ups of the website and game positively. In phase 2, working prototypes were rated positively and usability errors declined across the iterations and were mainly cosmetic or minor issues relating to esthetic preference, with few issues regarding ability to navigate the website or technical issues affecting functionality. Feedback from the semistructured interviews further supported the positive response of participants to the website and game, and helped identify areas for improvement during the iteration process. The final iteration of the website and game are presented.ConclusionsTaking an iterative approach to development through user-centered design and usability testing has resulted in an internet-based treatment for child anxiety (Online Support and Intervention for child anxiety) that appears to meet the needs and expectations of the intended users (parents, children, and clinicians) and is easy and enjoyable to use.

Even though User-Centred Design (UCD) is widely accepted and employed in the design and development of interactive systems, there are limited guiding procedures on how such approaches can be translated to the development of products, systems, and services that are focusing primarily on accessibility. This paper reports on the application of UCD for such a system within an EU project CONTENT4ALL under the domain accessibility for the Deaf. Each step in the UCD process and the respective activities within each step are described in detail with a focus on the methods and techniques adopted for assisting in the development of the novel technology. The insights gained during the entirety of the user-centred design and evaluation process have led to a compilation of important factors for creating sign-translated media content. This possible guideline comprises a list of useful and necessary components pertaining to sign language delivery in media, particularly sign television. The objective of this paper is to highlight lessons learned presented in the form of recommendations for human factors researchers on key UCD procedures for the development of accessibility products, systems and services based on the performed user activities within the project. An attempt has been made to reduce the gap in literature and add to a possible UCD guiding process exclusively for accessibility.

Human-Computer Interaction (HCI) research has been focusing on properties such as usability, accessibility, inclusive design, user experience. As new security risks are emerging with the continuous increase of Internet interconnections and the development of the Internet of Things, more recently security, trust and resilience have also become important for the development of interactive systems. Since users have been identified as one of the major security weaknesses in today’s technologies, HCI becomes a fundamental pillar for designing more secure (but still usable) systems. However, interactive system properties might overlap and sometimes create conflicts in user-centered development processes. For example, security could reduce system usability by placing a burden on users when they have to deal with passwords. The HCI research has provided several tools and techniques that can support designers in making decisions, but there are no “cookbooks”. This workshop promotes sharing of experiences in managing and resolving conflicts of multiple interactive system properties within the context of a user-centered design process. We are concerned by theories, methods and approaches for dealing with interactive system properties, managing potential conflicts and trade-offs. This workshop is organized by the IFIP WG 13.2 on Methodology for User-Centered System Design and the IFIP WG 13.5 on Human Error, Resilience, Reliability and Safety in System Development.

The volume of data in the world is growing very fast and generated from verity of sources like social media, sensors airline industry or scientific data in different formats. Biggest challenge is how to infer meaningful insights from such a varietyful and big data along with concern of data storage and management of fast growing data. The size of the databases used in today’s enterprises has been growing at exponential rates day by day. Hence, industries requirement to quickly process and analyze the big data volumes for business decision making and customer insights has also grown exponentially. Data pouring from various sources may be can be structured or unstructured in nature. Structured data refers to a relatively well-organized information, which can be further inserted into traditional RDBMS. As Traditional RDBMS are efficient and easy queries by simple, straightforward search algorithms or SQL queries. In contrast to structured data, unstructured data can be considered as information, which does not, comes in a pre-defined data format, well organized data storage model, or cannot be stored well into relational tables. It is assumed to be fastest growing type of data, e.g. image, sensors data, web chats, social networking messaging data, video, documents, log files, and email data. There are many techniques and software available, which can process and provide efficient storage of unstructured data and help organization to perform analytics on unstructured data. Unstructured data does not well-organized and not stored in predefined manner e.g. logs, web chats. The variety and on ordered nature of data makes storage methods and structure makes execution a time and resource-consuming affair. Advancement into technology has open floodgates to push huge volume of unstructured type of data. Multimedia data is one of the example of unstructured big data, which spans all over the Internet. This needs high execution capability to extract useful information. Rapid processing of multimedia data such as video is important for e.g. criminal investigations, surveillance monitoring, news analysis, sports analytics domain, emotion extraction, etc. Hence, analysis of multimedia data in minimum timeframe is one of the latest research areas. Therefore, we have researched techniques for analyzing unstructured data to extract meaningful information hidden in the big data. In addition, we will describe about various techniques and software used to Manage, process unstructured big data in efficient manner, and increases the performance of complexity analysis.

ATM interfaces nowadays present serious issues in usability and accessibility, frustrating users’ interaction and leading them to make operational mistakes. For this reason, we present the development of graphical interfaces for ATMs of BBVA Continental Bank, which follows a process of user-centered design. The priority was to identify the real needs of users based on the context, and to understand how they interact with the interfaces. In order to validate our proposal, we established a model and a functional prototype. Later, we evaluated this new design with a usability test. Based on the defects that were found, we applied changes to improve the interfaces. Finally, after an in-depth analysis, we concluded that the designed interfaces were suitable for implementation in the ATMs of BBVA Continental Bank, due to the process followed and the acceptance by users.

This article describes a co-design process in the context of user experience (UX) and usability testing and analysis of a first proof of concept of e-collaboration features based on unified communications, co-designed within an organization aiming to optimize users' communication cognitive load. An initial digital prototype with a detailed graphical interface, and simulated user narratives was established and the qualitative validation process is described and discussed. The implemented R&D process is mainly supported on user-centred design (UCD) methodology, namely action research with service design thinking method and co-design techniques. Qualitative data was gathered with concurrent think-aloud activities (CTA) stimulated by user experience expectation questions, observation notes, with integration in an eye tracking technology system. The UCD process and results are discussed, substantiating the added value due to the individual contributions and consequent usefulness of a final unified communication service for the organization.

In the era of Covid-19, people are more sensitive to their health and hygiene. SAC is present in the community as a service to clean each customer’s favourite shoes. Even though SAC has been around since 2013, until now they have not touched the digital realm. For this reason, innovation for SAC needs to be implemented so they are not left behind by competitors who are starting to become aware of digitalization. So that the implementation of digitalization can save costs and time, the approach to making digital products is through product development. To innovate through the use of product development in digital, a User-Centered Design (UCD) method approach with a Design Thinking (DT) mindset is used. The UCD process includes understanding context, specify user requirements, design solution, and evaluate. With the DT mindset, the process of emphatizing, defining, and ideation is applied to the understand context and specify user requirements so that this research will understand the usability side for users. With the addition of 5 UCD elements from Jesse James Garret, the digitalization process will be more in-depth and detailed in terms of creating interface design solutions. Primary data will be collected through In-depth Interview resulting abundant data that create persona and Minimum Viable Product(MVP). This research aims to analyze and identify the user experience flow for the prototype mobile web app product design SAC with the identifying the MVP and design prototype mobile website application and later do some usability testing to getting result that can meet the needs and uses for SAC customers. The result of this design approach method is a high-fidelity mobile web app prototype which was evaluated using Usability Testing (UT). The persona of each customer and the desire are the first key in development. Strong framework is formed with information architecture, low-fidelity, and high-fidelity prototypes. The final stage of the UCD iteration is to evaluate Usability Testing with a binary task success rate of 90%. Another UT results of quantitative data analysis with the Single Ease Question (SEQ) showed the number 6.42 and the System Usability Scale (SUS) showed the number 87.5 with quality A.

BackgroundStressful situations during intraoperative emergencies have negative impact on human cognitive functions. Consequently, task performance may decrease and patient safety may be compromised. Cognitive aids can counteract these effects and support anesthesiologists in their crisis management. The Professional Association of German Anesthesiologists set up a project to develop a comprehensive set of digital cognitive aids for intraoperative emergencies. A parallel development for several software platforms and stationary and mobile devices will accommodate the inhomogeneity of the information technology infrastructure within German anesthesia departments.ObjectiveThis paper aimed to provide a detailed overview of how the task of developing a digital cognitive aid for intraoperative crisis management in anesthesia was addressed that meets user requirements and is highly user-friendly.MethodsA user-centered design (UCD) process was conducted to identify, specify, and supplement the requirements for a digital cognitive aid. The study covered 4 aspects: analysis of the context of use, specification of user requirements, development of design solutions, and evaluation of design solutions. Three prototypes were developed and evaluated by end users of the application. Following each evaluation, the new requirements were prioritized and used for redesign. For the first and third prototype, the System Usability Scale (SUS) score was determined. The second prototype was evaluated with an extensive Web-based questionnaire. The evaluation of the third prototype included a think-aloud protocol.ResultsThe chosen methods enabled a comprehensive collection of requirements and helped to improve the design of the application. The first prototype achieved an average SUS score of 74 (SD 12), indicating good usability. The second prototype included the following main revisions: 2-column layout, initial selection of patient type (infant, adult, or parturient), 4 offered search options, and the option to check off completed action steps. Its evaluation identified the following major revision points: add quick selection for resuscitation checklists, design the top bar and tabs slightly larger, and add more pictograms to the text. The third prototype achieved an average SUS score of 77 (SD 15). The evaluation of the think-aloud protocol revealed a good intuitiveness of the application and identified a missing home button as the main issue.ConclusionsAnesthesiology—as an acute medical field—is particularly characterized by its high demands on decision making and action in dynamic, or time-critical situations. The integration of usability aspects is essential for everyday and emergency suitability. The UCD process allowed us to develop a prototypical digital cognitive aid, exhibiting high usability and user satisfaction in the demanding environment of anesthesiological emergencies. Both aspects are essential to increase the acceptance of the application in later stages. The study approach, combining different methods for determining user requirements, may be useful for other implementation projects in a highly demanding environment.

The rapid advancement of AI technology has led to increasingly complex and opaque systems, creating a critical need for explainable AI (XAI) that enhances transparency and user trust. Despite extensive research on XAI methods and applications, the user-centered design (UCD) process for XAI remains fragmented and unclear. This systematic review analyzes 27 studies from 2020 to 2024 to develop a comprehensive framework for user-centered XAI system design (UCXAISD). We identify five key stages: contextual inquiry, explanation needs identification, XAI method selection, user interface design, and evaluation and refinement. Our framework aligns with traditional UCD processes while incorporating specialized elements for XAI, including user-centricity, transparency, and actionability. For each stage, we provide evidence-based guidelines derived from the literature. The framework serves as a blueprint for developing XAI systems that balance technical sophistication with user needs.

Usability testing is a widely used evaluation method for product design during and after the development. Conventional usability testing applies short and discrete test tasks and task scenarios that are based on the tasks the product is designed to support. Thus, conventional test task design relies heavily on the representations of the specified context of use and the specified user requirements of the proposed design solution. However, a premature commitment to the specified context, requirements and proposed solutions may limit the scope of usability testing in a manner that hinders its capability to elicit and validate new user requirements, which is one of the objectives of the evaluation phase in the iterative user-centered design process. In this paper, we introduce a user-centered task design approach, which allows test participants to follow their natural work flow and freely express their needs during a test session. The main idea of this open-ended task approach is to break the tight link between the produced design solutions and the tasks used in the usability test and in this way increase the probability that novel user needs can emerge during a test session. Empirical results from a case study are used to depict the approach and its prerequisites, strengths, and limitations are discussed.

This study presents the user-centred design and evaluation process of a Collaborative Information System (CIS), developed for a new telehealth service for remote monitoring of chronic obstructive pulmonary disease patients after hospital discharge. The CIS was designed based on the information gathered in a workshop, where target end-users described the context of use, a telehealth workflow and their preferred ways of interaction with the solution. Evaluation of the iterative refinements were made through user tests, semi-structured interviews and a questionnaire. A field trial reported results on the ease of use and user satisfaction during the interaction with the fully developed system. The implemented CIS was successfully deployed within the secured Norwegian Health Network. The research was a result of cooperation between international partners within the EU FP7 project United4Health.

Recent research comparing usability assessment methods has been interpreted by some to imply that usability heuristic evaluation is no longer necessary because other techniques such as usability testing can find some usability problems more easily and cost-effective than heuristic evaluation. Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. Student portal service provides users with channels of information and resources needed for their studies this includes access to library service, course information, and materials. However, there are challenges that come with this development such as lack of effectiveness, efficiency, accuracy, and performance of the student portal service. This article provides information on usability assessment and testing on the student portal of University Utara Malaysia. The study was conducted using a l7 item questionnaire based on efficiency, effectiveness, performance, and accuracy of the student portal service. The result shows that more than 70% of the users are satisfied with student portal service and also agree to its efficiency and performance. This article will impact the designers and system developers by informing them about the views of a user on a performance of the portal and also give them the window for improvement.

Forest and land fires have become a national issue every year, especially in West Kalimantan. From 2015 to 2020, around 331,268.35 hectares of forest and land were burned in West Kalimantan. As a result of forest and land fires, the haze disrupts public health, the economy, and river, land, sea, and air transportation. As anticipation and prevention, the community and government monitor forest and land fires using the Forest Fire Monitoring System Application. The purpose of this study was to the User Experience (UX) evaluation for design improvement in the Forest Fire Monitoring System Application (SIPONGI) inWest Kalimantan. The method used was User Centered Design (UCD) and Website Usability Evaluation Tool (WEBUSE) to provide new design solutions in the form of a website prototype. The research methodology included a literature study of the SIPONGI application. The study used a sample of 25 respondents with different work backgrounds to represent the population using the SIPONGI application. The results of this study showed that usability points per attribute and category are superior after making UI/UX improvements using the UCD process in prototype form. In conclusion, using the UCD method is better if it is accompanied by the WEBUSE method in improving the design of an application.