Perception of dynamic audiotactile feedback to gesture input

The ability of technology to transmit multi-media is very dependent on compression techniques. In particular lossy compression has been used in image compression (jpeg) audio compression (mp3) and video compression (mpg) to allow the transmission of audio and video over broadband network connections. Recently the sense of touch or haptics is becoming more important with its addition in computer games or in cruder applications such as vibrations in a cell phone. As haptic technology improves the ability to transmit compressed force sensations becomes more critical. Most lossy audio and visual compression techniques rely on the lack of sensitivity in humans to pick up detailed information in certain scenarios. Similarly limitations in the sensitivity of human touch could be exploited to create haptic models with much less detail and thus requiring smaller bandwidth. The focus of this paper is on the force thresholds of the human haptic system that can be used in a psychophysically motivated lossy haptic (force) compression technique. Most of the research in this field has measured the just noticeable difference (JND) of the human haptic system with a human user in static interaction with a stationary rigid object. In this paper our focus involves cases where the human user or the object are in relative motion. An example of such an application would be the haptic rendering of the user's hand in contact with of a high-viscous material or interacting with a highly deformable object. Thus an approach is presented to measure the force threshold based on the velocity of the user's hand motion. Two experiments are conducted to detect the absolute force threshold (AFT) of the human haptic system using methodologies from the field of psychophysics. The AFTs are detected for three different ranges of velocity of the user's hand motion. This study implies that when a user's hand is in motion fewer haptic details are required to be stored calculated or transmitted. Finally the implications of this study on a more complete future study will be discussed.

Dialogic feedback demands an active role by lecturers and students to become effective. However, sometimes students do not engage with the feedback received. The use of technology and different channels to provide feedback (using audio and video feedback) in online learning environments could contribute to make students more active with the feedback and improve its effectiveness. The aim of this article is to investigate the use of different feedback channels (text, audio or video) and contrast their impact on academic achievement, as well as to analyse whether the feedback channel influences students’ perception of feedback in terms of their preferences. A quasi-experimental study was designed, whereby students received feedback both after they had drawn up the first draft of a written assignment and upon its completion. The results suggest that the channel through which feedback is provided does not have a bearing on performance. However, the study does identify significant differences between the quality of the first draft and that of their final submission. With regard to preferences, students preferred the video channel over the audio or written channels. In addition, they perceived video as the channel that is most conducive to greater interaction and dialogue between lecturers and students, and that also produces the greatest sense of closeness. The results obtained are discussed in light of their importance in an online environment.

Many studies have discussed the importance of technology in delivering feedback to students’ assignments since its deployment provides convenience, timeliness, rich information, asynchronous discussion, and social presence. However, the effectiveness of audio feedback has received very scant investigation in the context of higher education in Indonesia. To fill such an empirical void, this exploratory sequential mixed method probes the students’ perceptions towards the use of audio feedback. Thirty-two undergraduate students from one of the public universities in Bengkulu, Indonesia, majoring in Early Childhood and Studies and taking English as a compulsory subject, participated in this study. Three data collection techniques were deployed to triangulate the data. A questionnaire was distributed to examine students’ perceptions of audio feedback, followed by a freeform writing and semi-structured interview to draw more pictures of the students’ experiences of this feedback mode. The results show that even though all participants in this study had never received this type of feedback, most participants had positive perceptions of using audio feedback. They mentioned they got thorough, detailed, and personal feedback, so they felt engaged in the learning processes. This finding is expected to enrich the knowledge of the effectiveness of audio feedback and to encourage eager teaching practitioners to use it in their teaching processes, as it could increase students’ engagement in the learning process.

Users often make errors in touch or swipe while interacting with mobile phones. One of the common areas of concern is accidental swiping off important notifications. We show that these unintentional notification swipes can be accurately detected by using simple touch and swipe features recorded while performing the gesture. The pre-installed touch and grip sensors were used to record data of 20 different participants asked to perform intentional and unintentional touch gestures. The various features taken into account are extracted from user's hand movement on the screen and by identifying single-handed or two-handed grip. In addition to three previously published features- Touch Time, Swipe Velocity and Average Touch Size, we introduce three novel features in our system namely Swipe Stretch, Nearest Edge Gap based on grip and Notification Expansion Action. We trained our model using Random Forest (RF) classifier and Neural Networks (NN) and achieved the accuracy of 98.8% and 100% respectively. The results prove that the model can successfully detect unintentional notification swipe and touch gestures in real time. The novelty of our research lies in considerable improvement of accuracy over previous published works attributed to a larger feature set inclusive of proposed features.

In this paper we describe how coverbal iconic gestures can be used to express shape-related references to objects in a Virtual Construction Environment. Shape information is represented using Imagistic Description Trees (IDTs), an extended semantic representation which includes relational information (as well as numerical data) about the objects' spatial features. The IDTs are generated online according to the trajectory of the user's hand movements when the system is instructed to select an existing or to create a new object. A tight integration of the semantic information into the objects' data structures allows to access this information via so-called semantic entities as interfaces during the multimodal analysis and integration process.

One of the limitation of current prosthetics is the ability to provide sensory feedback to the human user. Due to this constraint, approximately 60–80 percent of amputees experience a phenomenon known as phantom limb pain, an ongoing painful sensations that to the individual, seems to be coming from the part of the limb that is no longer there. The lack of sensory feedback also limits the intuitive control of the user's hand movement, i.e. sense of grip or position. To address these limitations, we created am implantable system that could provide peripheral nerve stimulation, recording and motor control. The architecture of our Sensory-Stimulation Lead (SSL) system consist of multiple satellites connected to Draper's custom designed nerve electrodes. In this phase of the design, the implanted system is connected to a controller via percutaneous connections. The active electronics of the satellite is enclosed in a hermetic package approximately 14mm in diameter and less than 5mm thick. A custom ceramic feedthrough substrate provides the electrical connections of the internal electronics board to both the nerve electrodes and percutaneous leads. In this paper, we will describe the various packaging components of the system and the design, fabrication, and assembly considerations that drove our technology choices.

In recent years, virtual reality (VR) has been applied to many medical fields. Rehabilitation with VR technologies is one of the emerging research directions. In this paper, a rehabilitation exercise - English character handwriting training exercise is described in the haptic virtual environment. The approach can be used to guide user's hand movement on the right track of the predefined trajectory according to the real-time guidance force. A haptic device, PHANTOM Premium 1.0, is used for writing English characters as a virtual pen, and also for giving the user force feedbacks as a guidance tool. The user with hand dysfunctions can be trained for precise hand movements through this writing rehabilitation interface. The experimental results and performance analysis are also given in the paper

This research focuses on the use of hand tracking technology in a drawing program based on the MediaPipe framework. The aim of this study is to develop a digital drawing system that can track hand movements in real-time without additional input devices like a mouse or stylus. This technology utilizes computer vision algorithms to detect and track the user's hand movements, which are then translated into strokes on the screen. The study employs a descriptive-qualitative method with a software experimentation approach. The results show that the system has a high level of accuracy and is responsive to hand movements, providing a more natural and intuitive user experience. The implications of this research are significant in supporting technology-based educational and creative applications.

Numerous devices that either track hand gestures or provide haptic feedback have been developed with the aim of manipulating objects within Virtual Reality(VR) and Augmented Reality(AR) environments. However, these devices implement lengthy calibration processes to ease out individual differences. In this research, a wearable device that simultaneously recognizes hand gestures and outputs haptic feedback: UnlimitedHand is suggested. Photo-reflectors are placed over specific muscle groups on the forearm to read in hand gestures. For output, electrodes are placed over the same muscles to control the user's hand movements. Both sensors and electrodes target main muscle groups responsible for moving the hand. Since the positions of these muscle groups are common between humans, UnlimitedHand is able to reduce the time spent on performing calibration.

A wearable electrical impedance tomography (EIT) system for hand prosthesis motion control is demonstrated. The system captures the user's hand motion by measuring the impedance alterations caused by muscle and bone movement inside the forearm. These impedance data are sent to an artificial neural network for motion classification which is then used to manipulate a hand prosthesis. During the live demonstration, a sensor band is put on a volunteers' forearm for data acquisition. After signal processing, hand gestures learnt by the neural network can be recognized and the same hand motion can be recreated through the hand prosthesis in real-time.

This article consists of a collection of slides from the author's conference presentation. Present the gesture recognition products developed by eyeSight, a companies specializing in touch-free technology. The company's touch free technology provides an enhanced user experience, allowing to easily and intuitively control a variety of devices using simple hand gestures. It's Natural Use Interface solution uses a standard two dimensional camera along with advanced real-time image processing and machine vision algorithms, to track the user's hand motions. Explains the major components of the eysSight gesture recognition products and examines its prospects for use in the marketplace.

A system which employs a stylus as an input device is suitable for creative activities like writing and painting. However, such a system does not always provide the user with a GUI that is easy to operate using the stylus. In addition, system usability is diminished because the stylus is not always integrated into the system in a way that takes into consideration the features of a pen. The purpose of our research is to improve the usability of a system which uses a stylus as an input device. We propose shortcut actions, which are interaction techniques for operation with a stylus that are controlled through a user's hand motions made in the air. We developed the Context Sensitive Stylus as a device to implement the shortcut actions. The Context Sensitive Stylus consists of an accelerometer and a conventional stylus. We also developed application programs to which we applied the shortcut actions; e.g., a drawing tool, a scroll supporting tool, and so on. Results from our evaluation of the shortcut actions indicate that users can concentrate better on their work when using the shortcut actions than when using conventional menu operations.

Consumer electronic devices are becoming more complex and diverse. In order to provide users with an easy and convenient way of controlling various consumer electronics, it is necessary to use a universal way to control various devices. This requires using a user-friendly and intuitive interface. We introduce a wristwatch-type of remote that offers a unified way to control various devices. Total of seven gestures that are based on hand motions are defined. These seven gestures are suitable to the wearable wristwatch-type device. For a variety of systems, these gestures can be used in the same way. Gestures are not designed and analyzed in a discrete manner; instead, they are designed in continuous hand motions and analyzed in gesture commands. Also, a method of modeling a virtual menu from the menu on a consumer electronic device is introduced. With the virtual menu, the user is able to control various devices by using gestures. The virtual menu has to reflect hand motion characteristics and represents functions of electronic appliances. The virtual menu is implemented in XML to represent the basic menu construction and its properties. In order to use virtual menu, we introduce how a user's hand motions can be used in a fast and effective way. Finally we compare our wearable remote control to the conventional remote control through user tests in terms of user convenience and efficiency.

In this paper we propose a remote touch screen (RTS) as an application for flexible tactile array units. RTS is a combination of a flexible tactile display, a camera, a conventional touch screen and a monitor. While the user may press a button shown on the monitor under the touch screen in a conventional touch monitor configuration, in RTS, the user may press a button that is felt and shown. For visual guidance the user's hand motion is captured by an overhead camera, and is merged with the screen menu image and shown on the monitor. For tactual guidance, a tactile pattern is provided at the center of the button locations shown on the monitor. Hence the user's button pressing motion is visually as well as tactually guided in RTS configuration. To confirm the effectiveness of tactile display units in RTS, experiments were performed with and without the tactile display units.

In this report, we share our experience and observations on the challenges blind people face with text entry on touch-based mobile phones, particularly from the perspective of one of the authors, who is blind. To better understand these issues we developed and tested Multimodal Text Input Touchscreen Keypad (MTITK), an audio-tactile text entry prototype based on multitap, which relies on a telephone keypad layout organized into five key groups with distinct audio-tactile feedback. Users explore the screen to identify the current selected key, tap to enter text, and gesture to edit it, while receiving the corresponding voice, audio, and tactile feedback; no additional equipment is necessary in our software-only approach. We implemented a prototype on Android and tested its usability with visually impaired participants; they welcomed its multimodality and the familiar layout, but also expressed the need to increase vibration pattern differentiation and refine the character selection mechanism.