User involvement in the design process

Usability in a medical technology context assessment of methods for usability evaluation of medical equipment

As a consequence of the growing importance of services in contemporary economies, and the shift to a service-dominant logic in marketing, new service development (NSD) is unarguably important for a large majority of firms. However, developing new services remains a difficult task and the majority of NSD projects eventually fail. Therefore an urgent need exists to further improve the NSD process. In this article, we focus on improving the NSD process by better understanding the role of user involvement in the innovation framing stage. In this article, our aim is to develop a comprehensive conceptual framework investigating user involvement in the framing stage of service innovation projects by synthetizing key literature. Based on the literature, we observe that (1) the intensity of user involvement in innovation framing is positively correlated with the perceived technological uncertainty; (2) collaboration between user and firm in innovation framing is more intensive in case of continuous involvement than in discrete involvement; (3) collaboration between user and firm in innovation framing is stronger in business to business setting than in business to user setting; (4) user involvement appears to positively affect the degree of acceptance of the proposal, the degree of implementation of the associated NSD project and innovation speed, and finally (5) the degree of user involvement has an inverted-U shaped relationship with the effectiveness of innovation framing.

Where Is the User in All of This?

Introduction Usability evaluation is essential to make sure that software products newly released are easy to use, efficient, and effective to reach goals, and satisfactory to users. For example, when a software company wants to develop and sell a new product, the company needs to evaluate usability of the new product before launching it at a market to avoid the possibility that the new product may contain usability problems, which span from cosmetic problems to severe functional problems. Three widely used methods for usability evaluation are Think Aloud (TA), Heuristic Evaluation (HE) and Cognitive Walkthrough (CW). TA method is commonly employed with a lab-based user testing, while there are variants of TA methods, including thinking out aloud at user's workplace instead of at labs. What we discuss here is the TA method that is combined with a lab-based user testing, in which test users use products while simultaneously and continuously thinking out aloud, and experimenters record users' behaviors and verbal protocols in the laboratory. HE is a usability inspection method, in which a small number of evaluators find usability problems in a user interface design by examining an interface and judging its compliance with well-known usability principles, called heuristics. CW is a theory-based method, in which evaluators evaluate every step necessary to perform a scenario-based task, and look for usability problems that would interfere with learning by exploration. These three methods have their own advantages and disadvantages. For instance, TA method provides good qualitative data from a small number of test users, but laboratory environment may influence test user's behaviors. HE is a cheap, fast and easy-to-use method, while it often finds too specific and low-priority usability problems, including even not real problems. CW helps find mismatches between users' and designers' conceptualization of a task, but it needs extensive knowledge of cognitive psychology and technical details to apply. However, even though these advantages and disadvantages show overall characteristics of three major usability evaluation methods, we cannot compare them quantitatively and see their efficiency clearly. Because one of reasons why so-called discounted methods, such as HE and CW, were developed is to save costs of usability evaluation, cost-related criteria for comparing usability evaluation are meaningful to usability practitioners as well as usability researchers. One of the most disputable issues related to cost of usability evaluation is sample size. That is, how many users or evaluators are needed to achieve a targeted usability evaluation performance, for example, 80% of overall discovery rate? The sample size of usability evaluation is known to depend on an estimate of problem discovery rate across participants. The overall discovery rate is a common quantitative measure that is used to show the effectiveness of a specific usability evaluation method in most of usability evaluation studies. It is also called overall detection rate or thoroughness measure, which is the ratio of 'the sum of unique usability problems detected by all experiment participants' against 'the number of usability problems that exist in the evaluated systems', ranging between 0 and 1. The overall discovery rates were reported more than any other criterion measure in the usability evaluation experiments and also a key component for projecting required sample size for usability evaluation study. Thus, how many test users or evaluators participate in the usability evaluation is a critical issue, considering its cost-effectiveness.

This workshop aims to engage scholars and practitioners interested in user involvement in sharing experiences and reflections upon evaluation, sustainability and long-term effects of Participatory Design (PD) projects. Despite having 'people' as core interest, PD projects rarely venture into systematic evaluation of what the involved parties gained, how to ensure sustainability of initiatives beyond a project, or assessing long-term effects. Many PD projects aim to design a technology, provide a proof-of-concept, or assess a method or technique for involving people in design processes. While these are all laudable aims, we believe attention to evaluation, sustainability and long-term effects could provide fruitful feedback on these issues as well as further ensure that PD initiatives last beyond the singular project through acquired skills of participants and in the organization. Through this workshop, we aim to spur discussions and reflections upon gains, sustainability and long-term effects that can strengthen the lasting outcomes of PD initiatives.

BackgroundAcademic literature and international standards bodies suggest that user involvement, via the incorporation of human factors engineering methods within the medical device design and development (MDDD) process, offer many benefits that enable the development of safer and more usable medical devices that are better suited to users' needs. However, little research has been carried out to explore medical device manufacturers' beliefs and attitudes towards user involvement within this process, or indeed what value they believe can be added by doing so.MethodsIn-depth interviews with representatives from 11 medical device manufacturers are carried out. We ask them to specify who they believe the intended users of the device to be, who they consult to inform the MDDD process, what role they believe the user plays within this process, and what value (if any) they believe users add. Thematic analysis is used to analyse the fully transcribed interview data, to gain insight into medical device manufacturers' beliefs and attitudes towards user involvement within the MDDD process.ResultsA number of high-level themes emerged, relating who the user is perceived to be, the methods used, the perceived value and barriers to user involvement, and the nature of user contributions. The findings reveal that despite standards agencies and academic literature offering strong support for the employment formal methods, manufacturers are still hesitant due to a range of factors including: perceived barriers to obtaining ethical approval; the speed at which such activity may be carried out; the belief that there is no need given the 'all-knowing' nature of senior health care staff and clinical champions; a belief that effective results are achievable by consulting a minimal number of champions. Furthermore, less senior health care practitioners and patients were rarely seen as being able to provide valuable input into the process.ConclusionsMedical device manufacturers often do not see the benefit of employing formal human factors engineering methods within the MDDD process. Research is required to better understand the day-to-day requirements of manufacturers within this sector. The development of new or adapted methods may be required if user involvement is to be fully realised.

End user involvement and input into the design and evaluation of information systems has been recognized as being a critical success factor in the adoption of information systems. Nowhere is this need more critical than in the design of health information systems. Consistent with evidence from the general software engineering literature, the degree of user input into design of complex systems has been identified as one of the most important factors in the success or failure of complex information systems. The participatory approach goes beyond user-centered design and co-operative design approaches to include end users as more active participants in design ideas and decision making. Proponents of participatory approaches argue for greater end user participation in both design and evaluative processes. Evidence regarding the effectiveness of increased user involvement in design is explored in this contribution in the context of health IT. The contribution will discuss several approaches to including users in design and evaluation. Challenges in IT evaluation during participatory design will be described and explored along with several case studies.

Despite the increased focus on usability and on the processes and methods used to increase usability, a substantial amount of software is unusable and poorly designed. Much of this is attributable to the lack of cost-effective usability evaluation tools that provide an interaction-based framework for identifying problems. We developed the user action framework and a corresponding evaluation tool, the usability problem inspector (UPI), to help organize usability concepts and issues into a knowledge base. We conducted a comprehensive comparison study to determine if our theory-based framework and tool could be effectively used to find important usability problems in an interface design, relative to two other established inspection methods (heuristic evaluation and cognitive walkthrough). Results showed that the UPI scored higher than heuristic evaluation in terms of thoroughness, validity, and effectiveness and was consistent with cognitive walkthrough for these same measures. We also discuss other potential advantages of the UPI over heuristic evaluation and cognitive walkthrough when applied in practice. Potential applications of this work include a cost-effective alternative or supplement to lab-based formative usability evaluation during any stage of development.

3 - Big U and little u usability

Development of an off-the-shelf' product typically starts with a product idea and only limited knowledge of the eventual users. Since the functionality is partially predefined, the most natural focus for participatory design in product development companies is the human-computer interface. However, these organizations contain inherent obstacles to involving existing or potential users even in interface design and development. Today's large product development companies were formed before the human-computer interface attained its present prominence: their organizational structures and development processes have evolved with minimal consideration of the particular needs of interface development. This paper outlines the problems in achieving and benefiting from user involvement in design that stem from typical divisions of responsibility and development processes. Overcoming such organizational constraints may ultimately require organizational change; in the meantime, those working within the organization must be aware of the problems and constantly seek constructive paths around them.

There are four cognitive aspects revealed aim to develop learning application (apps) for children with mentally retardation (MR), namely perception, association, memory, and motoric. However, motoric aspects is less noticed in apps which had being developed. In this study, we aim to design the interaction of the apps which accommodate motoric aspect. Apps may be used as companion tool aside with formal learning. Apps is focused on motoric aspect, while another four aspects are adopted from existing apps. In order to reach high level of usability, interaction design (IxD) is measured iteratively using double measurements, cognitive walkthrough (CW) and heuristic evaluation (HE). Both method is covering each other. This way is suitable to verify previous measurement and strengthen usability. Result of the study stated that apps can properly used by students and teacher and relevant with user research. There is no major usability problems found either in CW or HE measurement. Moreover, HE method ensures end-user’s satisfaction is good design principles also.

BackgroundDesign processes such as human-centered design, which involve the end user throughout the product development and testing process, can be crucial in ensuring that the product meets the needs and capabilities of the user, particularly in terms of safety and user experience. The structured and iterative nature of human-centered design can often present a challenge when design teams are faced with the necessary, rapid, product development life cycles associated with the competitive connected health industry.ObjectiveWe wanted to derive a structured methodology that followed the principles of human-centered design that would allow designers and developers to ensure that the needs of the user are taken into account throughout the design process, while maintaining a rapid pace of development. In this paper, we present the methodology and its rationale before outlining how it was applied to assess and enhance the usability, human factors, and user experience of a connected health system known as the Wireless Insole for Independent and Safe Elderly Living (WIISEL) system, a system designed to continuously assess fall risk by measuring gait and balance parameters associated with fall risk.MethodsWe derived a three-phase methodology. In Phase 1 we emphasized the construction of a use case document. This document can be used to detail the context of use of the system by utilizing storyboarding, paper prototypes, and mock-ups in conjunction with user interviews to gather insightful user feedback on different proposed concepts. In Phase 2 we emphasized the use of expert usability inspections such as heuristic evaluations and cognitive walkthroughs with small multidisciplinary groups to review the prototypes born out of the Phase 1 feedback. Finally, in Phase 3 we emphasized classical user testing with target end users, using various metrics to measure the user experience and improve the final prototypes.ResultsWe report a successful implementation of the methodology for the design and development of a system for detecting and predicting falls in older adults. We describe in detail what testing and evaluation activities we carried out to effectively test the system and overcome usability and human factors problems.ConclusionsWe feel this methodology can be applied to a wide variety of connected health devices and systems. We consider this a methodology that can be scaled to different-sized projects accordingly.

The design and development of educational technologies is a complex, interdisciplinary endeavor. Learning science research reveals principles of learning and instruction, and advances in computer science implement these principles in innovative technologies. This chapter promotes a complementary discipline—human systems engineering or “user science”—that emphasizes designing with human users' goals, needs, capabilities, and limitations in mind. Systematic and iterative human systems engineering should contribute to educational technologies that are more functional, usable, desirable, and ultimately more effective. The authors overview key human systems engineering principles (e.g., usability and user experience) and methods (e.g., cognitive task analysis, contextual inquiry, heuristic evaluation, and participatory design), and then consider example applications from research on automated writing evaluation technologies. The chapter concludes with broad research questions posed to researchers, developers, and educators in the field of educational technology.

There are two main kinds of approach to considering usability of any system: empirical and analytical. Empirical techniques involve testing systems with users, whereas analytical techniques involve usability personnel assessing systems using established theories and methods. We report here on a set of studies in which four different techniques were applied to various digital libraries, focusing on the strengths, limitations and scope of each approach. Two of the techniques, Heuristic Evaluation and Cognitive Walkthrough, were applied in text-book fashion, because there was no obvious way to contextualize them to the Digital Libraries (DL) domain. For the third, Claims Analysis, it was possible to develop a set of re-usable scenarios and personas that relate the approach specifically to DL development. The fourth technique, CASSM, relates explicitly to the DL domain by combining empirical data with an analytical approach. We have found that Heuristic Evaluation and Cognitive Walkthrough only address superficial aspects of interface design (but are good for that), whereas Claims Analysis and CASSM can help identify deeper conceptual difficulties (but demand greater skill of the analyst). However, none fit seamlessly with existing digital library development practices, highlighting an important area for further work to support improved usability.

A Multi-step Usability Evaluation of a Self-Management App to Support Medication Adherence in Persons Living with HIV