Cloud-Based Intelligent User Interface for Cloud Manufacturing: Model, Technology, and Application

The idea of cloud manufacturing is gradually transforming the way enterprises do business in this age of globalization. The development of cloud manufacturing is closely linked to domain specific users in manufacturing industry. The huge amount of users with diverse roles and various requirements in manufacturing industry will face great challenges of cloud system usability problems. Thus user interface issues play important roles in this new area. Current research on cloud manufacturing pays more attention to internal cloud while makes light of external end-users’ usability, i.e. user interface issues. In this paper we discuss the key characteristics of Intelligent User Interface (IUI) for cloud manufacturing, i.e., naturality, smart mobility, self-configuration, and flexible customization. Further, a cloud-plus-IUI model for cloud end-users is presented. Then we discuss the enabling technologies, i.e., automatic configuration based on virtualization, context-aware adaption and recommendation, and multimodal interaction.

The human-computer interaction (HCI) is a research field that supports the user interface development. Furthermore, HCI influences the intelligent user interface (IUI) area. IUI is a user interface that uses intelligent technology to reach the communication between humans and machines. The construction of IUI is not easy and requires the knowledge of different methods, techniques and technologies from many domains (such as, HCI, artificial intelligence, ergonomics, software engineering, etc.). In this way, the existence of a process is an important tool for IUI developers. The literature presents design processes and frameworks for user interface, but most of them do not offer any knowledge regarding the methods, techniques, and technologies that can be used to support the activities involved. Thus, this work proposes a development process for IUIs which offers recommendations about methods, techniques, and technologies. We hope that this process will help analysts and designers who work in IUI area.

A critical problem with existing computer-based geometric modeling is the labor intensive task involved in specifying data input for the description of three-dimensional (3-D) objects. This paper describes a new, 3-D input system aimed at alleviating this problem. It is based on the use of a three-dimensional digitizer for the direct input of spatial coordinates, and an intelligent interactive user interface. The intent of this system is to create a high level, intelligent interface between the designer and a geometric solid modeler which would lighten the designer's burden in performing arduous 3-D geometric description tasks. The user interface is developed around the Knowledge Craft expert system building tool, using the rule-based Carnegie Representation Language OPS5 (CRL-OPS). The system uses schematic networks or frames and production rules to encode knowledge about geometric primitive digitization methods, object feature operators, solid modeler requirements, and input command functions. It also employs a forward-chaining inference strategy to direct the knowledge. This ensures that only a minimal amount of valid data entry is required by the user. However, if excessive data is entered the intelligent interface has the capability to extract the required information. As a result, the solid modeler can automatically create the appropriate object “primitive” or the specific object “feature” upon recognition by the expert system. It will be demonstrated that these capabilities can simplify the 3-D model description process.

Manufacturing enterprise management mode from the traditional manufacturing production gradually transformed into service-oriented manufacturing in cloud manufacturing. As a carrier for manufacturing resources and manufacturing capabilities, the manufacturer directly affects the advantages and disadvantages of cloud manufacturing services. Therefore, manufacturer optimization has received increasing attention. This paper studies the problem of manufacturer optimization in cloud manufacturing environment, analyzes the 11 kinds of factors which affect the service quality of the manufacturer, meanwhile, it establishes the evaluation indexes system and objective function, optimization algorithm for solving the objective function, and then, it proposes optimization algorithm for solving the objective function, realizes the optimal selection of manufacturers under different demands. The simulation results show that the established evaluation indexes system can display the manufacturing resources and capabilities more fully. Based on the evaluation indexes system model, all the factors that influence the optimization of the manufacturer can be combined together, and the complex optimization problem can be transformed into the mathematical problem. The algorithm can be used to select the best manufacturer under different requirements, and to provide the best service for users.

The physical and virtual access of manufacturing equipment to the cloud platform is an important direction of cloud manufacturing research, but the current research results in this direction are not enough to support the "landing" and wide application of cloud manufacturing model. In the traditional cloud manufacturing model, the direct access of manufacturing equipment to the cloud brings huge network pressure and unbearable latency, and also poses a privacy and security risk. Aiming at the cloudification of manufacturing equipment in the cloud manufacturing environment, the 3D printer is selected as the research object, and a cloud-edge collaboration architecture for 3D printers based on digital twin is proposed. To solve the lack of network reliability brought about by traditional cloudification, time-sensitive services are deployed at the edge, realizing the local extension of cloud services. The digital twin information model of FDM additive manufacturing is defined. Real time controlling and monitoring based on digital twin is realized. By cloud-edge collaboration, an application case is implemented, which verifies the effectiveness of the system and provides a reference solution for the cloudification of other manufacturing equipment in the environment of cloud manufacturing.

In current multi-agent systems, the user is typically interacting with a single agent at a time through relatively inflexible and modestly intelligent interfaces. As a consequence, these systems force the users to submit simplistic requests only and suffer from problems such as the low-level nature of the system services offered to users, the weak reusability of agents, and the weak extensibility of the systems. In this paper, a framework for multi-agent systems called the open agent architecture (OAA) which reduces such problems, is discussed. The OAA is designed to handle complex requests that involve multiple agents. In some cases of complex requests from users, the components of the requests do not directly correspond to the capabilities of various application agents, and therefore, the system is required to translate the user's model of the task into the system's model before apportioning subtasks to the agents. To maximize users' efficiency in generating this type of complex requests, the OAA offers an intelligent multi-modal user interface agent which supports a natural language interface with a mix of spoken language, handwriting, and gesture. The effectiveness of the OAA environment including the intelligent distributed multi-modal interface has been observed in our development of several practical multi-agent systems.

Portable devices come with different limitations in user interaction like limited display size, small keyboard, and different sorts of input and output capabilities. With the advance of speech recognition and speech synthesis technologies, their complementary use becomes attractive for mobile devices in order to implement real multimodal user interaction. However, current systems and formats do not sufficiently integrate advanced multimodal interactions. We introduce an advanced generic multimodal interaction and rendering system (MIRS) dedicated for mobile devices. MIRS incorporates efficient processing of XML specification languages for limited, mobile devices and comes with the XML-based dialog and interface specification language (DISL). DISL can be considered as an UIML subset, which is enhanced by the means of state-oriented dialog specifications. The dialog specification is based on ODSN (object oriented dialog specification notation), which has been introduced to define user interface control by means of interaction states with transition rules.

This paper discusses the development of Virtual Control Interface (VCI), a Virtual Reality (VR) system for simulating and exploring multimodal context-sensitive interactions between Augmented Reality (AR) glasses and Internet of Things (IoT) devices.Current trends in controlling IoT devices focus on using existing control interfaces such as smartphones. Parallel technological trends indicate that AR glasses are likely to become the next consumer terminal. Therefore, there is a need to explore a new paradigm for IoT control interfaces that leverages the benefits of AR.Towards developing future-proof AR-IoT interfaces, we observed the HCI community’s challenges in using existing hardware to explore the full potential and the limitations of future AR-IoT interfaces. That became the foundation for developing VCI.This paper provides an overview of the advantages and shortcomings of developing such interfaces on smartphones, state-of-the-art AR/Mixed Reality (MR) glasses, and our proposed VR simulated AR environment. This overview outlines how our proposed system could be a superior choice for researchers to investigate, challenge assumptions, prototype, develop, and experimentally test AR-IoT interactions and interfaces with currently available technologies.Furthermore, we present the three pillars we followed to guide the design and development of VCI to make it resemble an actual AR-IoT framework as much as possible while abstracting parts that have little to no effect on AR-IoT HCI research. These pillars are: design transferable architecture, use viable multimodal interactions and explore user interfaces in real-life scenarios.These three principles informed the VCI’s system architecture built upon three main subsystems: handling multi-modal user input, IoT device emulation and connectivity, and managing the AR user interface (UI).Finally, we provide a heuristic evaluation and future work for VCI.KeywordsSmart homeInternet of ThingsAugmented realityVirtual realityAR-IoTMultimodal interactionVirtual Control Interface

Event Abstract Back to Event Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS Adrian Curtin1, 2* and Hasan Ayaz1, 3, 4 1 School of Biomedical Engineering, Science and Health Systems, Drexel University, United States 2 Shanghai Jiao Tong University, School of Biomedical Engineering, China 3 University of Pennsylvania, Department of Family and Community Health, United States 4 Children's Hospital of Philadelphia, The Division of General Pediatrics, United States The way in which we effortlessly understand speech makes it relatively easy to overlook the complexity of the neural mechanisms which support its operation. This perceived ease of speech comprehension has inspired researchers and interface designers seek to incorporate additional auditory channels of information to consumer and industrial devices, especially in circumstances where the user of the device is otherwise engaged in manual or visual tasks such as handsfree communication while driving. However, previous research has demonstrated that there are unconscious cognitive costs associated with simple speech comprehension which can be exacerbated by task performance(Treffner & Barrett, 2004) and audio quality(Delogu, Conte, & Sementina, 1998; Francis & Nusbaum, 2009). In a previous exploratory work, we have observed performance deficits and neural activation differences due to low-quality synthetic speech during speech comprehension tasks(Curtin & Ayaz, 2017), and here we extend this exploration by examining how the quality of synthetic speech affects auditory working memory (AWM) performance and neural load using functional Near Infrared Spectroscopy (fNIRS), a continuous and noninvasive neuroimaging technique capable of investigating brain dynamics under naturalistic and applied settings (Ayaz et al., 2013). Working memory is an important executive function in which a limited storage system is updated and maintained during the execution of other processing tasks. As a limited resource, tasks which require significant working memory resources are especially sensitive to distortions and other factors which may competitively demand additional attention(Banich, 2009), hence the development of systems and user interfaces must carefully consider added cognitive burdens imposed as part of the design process. Beyond design considerations, the wide variety of situations which require auditory working memory (AWD) and the way in which degraded audio quality adversely can affect these has important implications in day-to-day life whether the degraded audio signal is due to environmental noise, hearing loss, distortion produced by hearing aid devices, or low-quality synthetic speech. Here we present a neuroergonomic evaluation on the effects of synthetic speech quality during an auditory working memory task while functional Near-Infrared Spectroscopy (fNIRS) was used to noninvasively monitor both the dorsolateral-Prefrontal Cortex (dlPFC) and the left-Temporal-Parietal Junction region (TPJ) which includes the primary auditory cortex, Broca’s area and Wernicke’s area. Sixteen right-handed healthy controls with no history of neurological or mental illness were recruited to participate in this study. Participants completed an auditory version of the N-Back task using vocalized letters as stimuli while the audio quality was manipulated by using low-quality and high-quality synthetic voices. Results show that reaction time was significantly increased by the presentation of low-quality synthetic speech under both the 0-Back and 2-Back conditions. Additionally, increased working memory load was associated with behavioral performance decreases in terms of accuracy (Figure 1, F(1,225.12)=8.34,p<0.005) and reaction time (Figure 2, F(1,22.46)=30.32,p<0.001) and neural differences as measured by changes in oxygenated hemoglobin [HbO] as measured by fNIRS in both the left-dlPFC (F(1_200.43)=33.2,p<0.001) and the left-TPJ (Figure 3, F(1,160)=8.32,p<0.005), near Wernicke’s Area. Our results suggest both that the interpretation of low-quality voices results in significant behavioral deficits in terms of reaction time (F(1,22.33)=29.91, p<0.001) and that increased AWM load is reflected by changes in activation in not only areas of executive function, but also associative auditory areas. These preliminary results highlight the potential significant impact of neuroimaging for usability and user interface research. Various characteristics of audio user interface can negatively impact cognitive workload in ways that should be accounted for during the design of complex systems or interfaces. Wearable and mobile brain function monitoring can be used to detect these otherwise unobserved characteristics. Figure 1: Auditory N-Back accuracy vs Workload and Synthetic Audio Quality. Figure 2: Mean reaction time during Auditory N-Back vs Workload and Synthetic Audio Quality. Figure 3: Mean [HbO] changes in Left-Temporal region vs Working Memory Load and Synthetic Audio Quality Figure 1 Figure 2 Figure 3 Acknowledgements The authors would like to thank Dr. Paul Crawford for his help with this study. This study is made possible by a research award from the Intel Corporation. The content of the information herein does not necessarily reflect the position or the policy of the sponsor and no official endorsement should be inferred. Keywords: auditory working memory, fNIRS, n-back, synthetic speech, Prefrontal Cortex, Auditory Cortex Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Poster Presentation Topic: Neuroergonomics Citation: Curtin A and Ayaz H (2019). Quality of synthetic speech and auditory working memory performance: neuroergonomic perspectives from fNIRS. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00039 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 03 Apr 2018; Published Online: 27 Sep 2019. * Correspondence: Mr. Adrian Curtin, School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States, abc48@drexel.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Adrian Curtin Hasan Ayaz Google Adrian Curtin Hasan Ayaz Google Scholar Adrian Curtin Hasan Ayaz PubMed Adrian Curtin Hasan Ayaz Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Automated user modeling for intelligent interface

Extensive decision support system composed of several modules providing different functionalities for the user can be a challenge in designing of a usable, plain and consistent user interface. A characteristic of advisory systems in the domain of metallurgy is their complex architecture designed to deal with multiple types of tasks in many areas. The objective of this paper is to present the possibility of creating an intelligent interface providing the user with complete freedom to act without precise knowledge of the multiple functionality of the system. The paper presents the architecture of developed advisory system in the domain of metallurgy enriched with the description of mechanisms, which enable obtaining an intelligent user interface. The intelligent user interface should guide the user at navigation through all system modules in order to meet all needs that are the reason for this system usage. In designing of such interface, the Case-Based Reasoning methodology is proposed to be used. Case-Based Reasoning ensures a pattern for building systems that are able to learn on the basis of human actions associated with human decisions and use such knowledge to resolve the currently appearing problems. Each user query directed to the system, and each of his decisions about the choice of functionalities of the system modules supplies the system with the crucial information on proper use of the whole decision support unit related with the needs of a user. Skillful use of this knowledge can provide convenient and efficient performance of intelligent interface.

In this paper we present a solution to compose synchronized multimodal computer human interaction with a pattern-oriented approach. The focus of this paper is how to synchronize the different modalities of interaction.In our application scenario the system consists of mobile agents and the user interface is generated from a dialog description language, which is XML-encoded. The generated User Interface is executed by one or more UI-engines, which can use one or more modalities like a text chat system, a speech-based system or a graphical, window-oriented user interface. The user interface is separated and also structured by so-called dialog moves which are suitable to be adapted to different UI-engines. The system supports more than one UI-engine at the same time which results in a possibly multimodal interaction. The interaction is synchronized by observing the events produced by the dialog moves of each UI-engine.The system is evaluated in a project called MAP1 which deals with new human computer interaction methods in future mobile work environments.

Welcome to the 24th ACM International Conference on User Modeling, Adaptation, and Personalization (UMAP 2016) in Halifax, Canada, July 13-16, 2016. UMAP is the premier international conference for researchers and practitioners working on systems that adapt to individual users or to groups of users. UMAP is the successor of the biennial User Modeling (UM) and Adaptive Hypermedia and Adaptive Web-based Systems (AH) conferences that were merged in 2009. It has traditionally been organized under the auspices of User Modeling Inc. This year (2016) UMAP became an ACM conference, sponsored by ACM SIG CHI and SIG WEB. The conference spans a wide scope of topics related to user modeling, adaptation, and personalization. UMAP 2016 is focused on bringing together cutting-edge research from user interaction and modeling, adaptive technologies, and delivery platforms. It includes high-quality peer-reviewed papers featuring substantive new research in one of five research areas, each chaired by leaders in the field: User Modeling for Recommender Systems (chairs: Alexander Felfernig & Pasquale Lops)Adaptive & Personalized Educational Systems (chairs: Antonija Mitrovic & Kalina Yacef)Personalization in the Social Web & Crowdsourcing Era (chairs: Alessandro Bozzon & Harith Alani)Adaptive, Intelligent, & Multimodal User Interfaces (chairs: Julien Epps & Hatice Gunes)Architectures, Techniques, & Methodologies for UMAP (chairs: Stephan Weibelzahl & Mihaela Cocea) This year we received 123 submissions. In keeping with UMAPs rigorous standards, each paper was carefully reviewed by members of the Program Committee (PC) while the Area Chairs (ACs) coordinated the reviews and provided recommendations to the Program Chairs. The international Program Committee (PC) consisted of 132 members who were assisted by 49 subreviewers. These were leading researchers as well as highly promising young researchers. Papers were assigned to at least 4 members of the PC and to 1 AC member based on their expertise, interests, and other factors. Each paper received at least 3 reviews, and 95% received 4 reviews. After the initial reviews were submitted, the designated AC facilitated discussion amongst reviewers in order to resolve differences and correct misunderstandings. The AC then provided a summative meta-review and a recommendation to the Program Chairs. The final decisions were based on these recommendations, the meta-reviews, and reviewer scores. We accepted 21 long papers (23.9% acceptance rate) and 13 short papers (27.6% acceptance rate) for oral presentation and an additional 17 extended abstracts for poster presentation and inclusion in the proceedings. The program also features posters, demos, and late breaking results, which collectively showcase the wide spectrum of novel ideas and latest results in user modeling, adaptation and personalization. We also invited three distinguished keynote speakers, each illustrating significant issues and prospective directions for the field. Hossein Derakhshan (shared keynote speaker with Hypertext 2016) is an Iranian-Canadian blogger who was imprisoned in Tehran from November 2008 to November 2014. He has been called the "father of Persian blogging" and has helped promote podcasting in Iran. His talk, "Killing the Hyperlink, Killing the Web: the Shift from Library-Internet to Television-Internet," reflects his views on the Internet today.Lada Adamic leads the Product Science group within Facebook's Data Science Team. She is also an adjunct associate professor at the University of Michigan's School of Information and Center for the Study of Complex Systems. Her talk "The Life and Times of Information in Networks" focuses on cascades of information-sharing and resharing within social media.Sandra Carberry was one of the founders of the User Modeling research area at the first workshop in Maria Laach in 1986. As appropriate for the 30th anniversary, her talk "User Modeling: The Past, The Present and The Future" discusses how the field evolved, insights into where the field is headed, and the hottest topics for exploration. The conference includes a doctoral consortium that provides an opportunity for doctoral students to explore and develop their research interests under the guidance of distinguished scholars. This track received 18 submissions of which nine were accepted as full papers and eight as posters. A set of seven workshops and two tutorials round out the program. (Workshop) IFUP: Workshop on Multi-dimension Information Fusion for Modeling and Personalisation (half-day) organized by Robin Burke (DePaul University, USA), Feida Zhu (Singapore Management University, Singapore), Neil Yorke-Smith (American University of Beirut, Lebanon), and Guibing Guo (Northeastern University, China)(Workshop) INRA: News Recommendation and Analytics (half-day) organized by Jon Atle Gulla (Norwegian University of Science and Technology Trondheim, Norway), Luc Martens (Minds-UGent-WiCa Ghent, Belgium), Özlem Özgöbek (Norwegian University of Science and Technology Trondheim, Norway), and Nafiseh Shabib (TNS Gallup, Oslo, Norway)(Workshop) SOAP: Workshop on Surprise, Opposition, and Obstruction in Adaptive and Personalized Systems (half-day) organized by Peter Knees (Johannes Kepler University Linz, Austria), Kristina Andersen (Studio for Electro Instrumental Music, Amsterdam, the Netherlands), Alan Said (Recorded Future, Gothenburg, Sweden), and Marko Tkalcic (Free University of Bozen-Bolzano, Italy)(Workshop) HAAPIE: Human Aspects in Adaptive and Personalised Interactive Environments (half-day) organized by Panagiotis Germanakos (SAP SE, Germany), Marios Belk (Department of Computer Science, University of Cyprus), George Samaras (Department of Computer Science, University of Cyprus), and Vania Dimitrova (University of Leeds, UK)(Workshop) EvalUMAP: Towards comparative evaluation in the user modelling, adaptation and personalization space (full-day) organized by Owen Conlan (Trinity College Dublin, Ireland), Liadh Kelly (Trinity College Dublin, Ireland), Kevin Koidl (Trinity College Dublin, Ireland), Séamus Lawless (Trinity College Dublin, Ireland), Killian Levacher (Trinity College Dublin, Ireland), and Athanasios Staikopoulos (Trinity College Dublin, Ireland)(Workshop) FuturePD: The future of personal data: envisioning new personalized services enabled by Quantified Self technologies (half-day) organized by Amon Rapp (University of Torino, Italy), Federica Cena (University of Torino, Italy), Judy Kay (University of Sidney, Australia), Bob Kummerfeld (University of Sydney, Australia), Frank Hopfgartner (University Gardens Glasgow, UK), Jakob Eg Larsen (Technical University of Denmark, Denmark), and Elise van den Hoven (University of Technology Sydney, Australia). (Workshop) PALE: Personalization Approaches in Learning Environments (full-day) organized by Milos Kravcik (RWTH Aachen University, Germany), Olga C. Santos (UNED, Spain), Jesus G. Boticario (UNED, Spain), and Maria Bielikova (FIIT STUBA, Slovakia)(Tutorial) Semantics-Aware Techniques for Social Media Analysis, User Modeling, and Recommender Systems (half-day) by Pasquale Lops and Cataldo Musto (University of Bari Aldo Moro, Italy)(Tutorial) Games, Gamification and Personalization (half-day) by Amon Rapp (University of Torino, Italy).

: Generic expert systems are reasoning systems that can be used in many application domains, thus requiring application domain independence. The user interface for a generic expert system must contain an intelligence in order to maintain this domain independence and manage the complex interactions between the user and generic expert system. This paper explores the uncertainty-based reasoning contained in an intelligent user interface called GESIA. GESIA's interface architecture and dynamically constructed Bayesian network are examined in detail to show how uncertainty-based reasoning enhances the capabilities of this generic expert system user interface.

Generic expert systems are reasoning systems that can be used in many application domains, thus requiring domain independence. The user interface for a generic expert system must contain intelligence in order to maintain this domain independence and manage the complex interactions between the user and the expert system. This paper explores the uncertainty-based reasoning contained in an intelligent user interface called GESIA. GESIA's interface architecture and dynamically constructed Bayesian network are examined in detail to show how uncertainty-based reasoning enhances the capabilities of this user interface.