MOBAJES: Multi-user Gesture Interaction System with Wearable Mobile Device

The big data in education faces one methodology problem, i.e., how to acquire, store and apply the big data, especially to find one mediator to associate and solve the problem overall. In this paper, we create our big data model for one school firstly, which includes the behaviour subject information model and its related activity model. The behaviour subjects include the school, class, teacher, students, parents, etc. The information model relates with the subjects' basic and the activity information. Secondly, the collection means of big data is presented, the detailed big data about the activities are acquired from the existing and extended school information systems, e.g., as the administration systems for student attendance, quiz, shopping, etc. Thirdly, the wearable mobile internet devices act as one new means to provide more information and one key to associate different data from heteronomous sources. Finally, we verify our big data model through new applications, such as school-bus route optimisation, timecost impact to learning, and parents' affection for students' development, etc.

In the current age of ubiquitous computing age that uses high bandwidth network, wearable and hand-held mobile devices with small cameras and wireless communication will be widespread in the near future. Thus, computer vision and image processing for mobile devices have recently attracted a lot of attention. Especially, many approaches to detect image texts containing useful information for automatic annotation, indexing, and structuring of image are important for a prerequisite stage of recognition in dictionary application using mobile devices equipped with a camera. To detect image texts on the mobile devices that have limited computational resources, recent works are based on two methodologies; the image texts are detected not by automatically but by manually using stylus pen to reduce the computational resources, and the server is used to detect image texts requiring many floating-computations. The main disadvantage of the manual method is that users directly select tentative text regions, and recall and precision rates are determined by the selected regions. The second method to automatically detect the image texts is difficult to perform it in real-time, due to transmission time between the mobile device and the server. Accordingly, this paper proposes a real-time automatic word detection system without support of the server. To minimize the computational time, one word in the central region of the image is considered as a target of the system. The word region is tentatively extracted by using edge density and window transition, and the tentatively extracted region is then verified by measuring uniform distribution among sub-windows of the extracted region. In the experiments, the proposed method showed high precision rates for one word in the central region of the image, and showed fast computational time on the mobile devices.

To develop a new method of evaluating swallowing behaviour. Sixty-nine healthy participants were divided into a younger (16 males and 16 females, mean age 39.09 ± 12.16 years) and older (18 males and 19 females, mean age 71.43 ± 5.50 years) group. The participants ingested water and yoghurt twice (directed and free swallowing) at rest and after performing simple daily life tasks (calculation and exercise). To measure swallowing frequency, we employed a smartphone-based, portable and neck-worn swallowing-sound-monitoring device. This device monitors swallowing behaviour continuously by collecting biological sounds from the neck without imposing behavioural restrictions. A neural network model of swallowing sound identification by deep learning was used for the subsequent evaluation. This device was used to obtain two types of saliva-swallowing sounds associated with different ingestants, at rest and after performing a stimulating task. Furthermore, we assessed the associated subjective psychological states. The younger group showed a higher directed swallowing frequency (for both water and yoghurt) than the older group did. Regarding the type of ingestant, the swallowing frequency for yoghurt was higher during free swallowing in both the young and the older groups. 'Feeling calm' was reported significantly more often in the older group after swallowing yoghurt following exercise. Swallowing status in daily life was measured non-invasively using a wearable mobile device. It is important to consider the type of ingestant, daily living activities, and age when assessing swallowing.

While multi-touch design allows natural and flexible figure gestures in interacting with mobile devices, the palm-size screen often limits its potential in application and arouses issues such as inefficient mode switch and incomplete gesture. To enrich the vocabulary of finger gestures in user interaction, we present Thumb Widgets, a mechanism that adopts thumb as assisted input channel to enhance the capabilities of multi-touch on touch-based mobile devices. Thumb Widgets expand the design space in gesture-based UIs and promote efficiency for tasks such as mode switching, parameter setting, etc. Also, Thumb Widgets enable incomplete gestures to be performed around the periphery of the device.

<p>Human activity recognition (HAR) based on wearable devices is an emerging field of great interest. HAR can provide additional information on a human subject’s physical status. Utilising new technologies for HAR will become very meaningful with the development of deep learning. This study aims to mine deep learning models for HAR prediction with the highest accuracy on the basis of time-series data collected by mobile wearable devices. To this end, convolutional neural networks (CNN) and long short-term memory neural networks (LSTM) are combined in a deep network model to extract behavioural facts. The proposed CNN model contains two convolutional layers and a maximum pooling layer, and batch normalisation is added after each convolutional layer to improve convergence speed and avoid overfitting. This structure yields significant results in terms of performance. The model is evaluated on the MHEALTH dataset with a test set accuracy of 99.61% and can be used for the intelligent recognition of human activity. The results of this study show that the proposed model has better robustness and motion pattern detection capability compared to other models.</p> <p> </p>

In this paper, a real-time system that utilizes hand gestures to interactively control the presentation is proposed. The system employs a thermal camera for robust human body segmentation to handle the complex background and varying illumination posed by the projector. A fast and robust hand localization algorithm is proposed, with which the head, torso, and arm are sequentially localized. Hand trajectories are segmented and recognized as gestures for interactions. A dual-step calibration algorithm is utilized to map the interaction regions between the thermal camera and the projected contents by integrating a Web camera. Experiments show that the system has a high recognition rate for hand gestures, and corresponding interactions can be performed correctly.

Device-free hand gesture is one of the most natural ways to interact with everyday objects. However, existing WiFi-based gesture recognition solutions are typically restricted to indoor environments due to limited outdoor coverage. Furthermore, to achieve high sampling rates, they may interfere with normal data transmissions. In this paper, we aim to develop a robust dynamic gesture interaction system that can be ubiquitously deployed using Long-term Evolution (LTE) mobile terminals. Through both empirical studies and in-depth analysis using the Fresnel zone model, we reveal the key factors that contribute to the repeatability and discernibility of gestures. We show that the optimal location and orientation to perform gestures indeed exist and can be identified without prior knowledge of the position of LTE base stations (BSs) relative to a terminal. Guided by the design principles derived from Fresnel zone characteristics around a 4G terminal, we design highly repeatable and discernible gestures with salient received signal profiles. A gesture interaction system has been developed and implemented to achieve robust recognition with this careful design. Extensive experiments have been conducted in both indoor and outdoor environments, for different relative placements of mobile terminal and BS, and with different users. The proposed system can automatically identify the direction of BSs with a median error of less than 15 degrees and achieve gesture recognition accuracy as high as 98% in all scenarios without the need to acquire any training data.

With the rapid advancement of wireless network technologies and the constant increase in the number of mobile devices, enormous amount of data is generated from a set of user equipments (UEs), such as mobile phones and wearable devices. How to power these devices wirelessly becomes a challenging issue. Meanwhile, data and energy integrated communication networks (DEINs) are emerging as a new type of wireless networks emphasizing on the cooperation between wireless information and energy transfer. This means that the DEINs can be a solution to tackle with powering UEs wirelessly and further achieving a higher data rate. However, there are no researches that have been studied about how to obtain the optimal scheme in DEINs under the consideration of data traffic. Hence, we propose a fair energy-efficient resource allocation algorithm for DEINs in this paper. Our objective is to optimize the energy efficiency with the consideration of providing fair queuing for data traffic by minimizing the queue length difference among the UEs' buffers. Besides, when designing the downlink energy beamforming and the downlink-uplink power-and-time allocation scheme, power sensitivity of conversion circuits and system stability are taken into account. Simulation results illustrate that the proposed resource allocation algorithm does not only improve the fairness among UEs by queue balancing, but also achieve close-to-optimal system energy efficiency by the beamforming design and resource allocation.

Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.

One of the main problems that may put people's safety in danger is the lack of real-time detection, evaluation, and recognition of predictable safety risks. Current real-time risk identification solutions are limited to proximity sensing, which lack providing the exposed person with risk-specific information in real-time. Combined values of concurrently presented risks are either unrecognized or underestimated. This study goes beyond the proximity sensing state-of-the-art by envisioning, planning, designing, developing, assembling, and examining an automated intelligent real-time risk (AIR) assessment system. A holistic safety assessment approach is followed to include identification, prioritization, detection, evaluation, and control at risk exposure time. Multi-sensor technologies based on RFID are integrated with a risk assessment intelligent system. System prototype is developed and examined to prove the concept for on-foot building construction workers. The evaluation of AIR assessment system's performance proved its validity, significance, simplicity, representation, accuracy, precision, and timeliness. The reliability of providing quantitative proximity values of risk can be limited due to the signal attenuation; however, it can be reliable in providing risk proximity in a subjective linguistic fashion (Near/Far). The main contributions of the AIR assessment system are that the mobile wearable device can provide a linguistic meaningful risk assessment resultant value, the value represents the combined evaluation of concurrently presented risks, and can be sound delivered to the exposed person in real-time of exposure. Therefore, AIR system can be used as an effective prognostic risk assessment tool that can empower workers with real-time recognition and measurability of risk exposure.

Contact lenses were originally designed for cosmetic and vision correction purposes. Recently, they have become mobile wearable devices that integrate drug delivery, diagnosis, and treatment. The portability of contact lenses, the advancement of modern technology, and the public's growing demand for personalized diagnosis and treatment have accelerated the research progress of smart contact lenses. This article summarizes the development trends of smart contact lenses in the diagnosis and treatment of eye diseases, along with future prospects, opportunities, and challenges.

It is important to consider the physiological and behavioral mechanisms that allow users to physically interact with virtual environments. Inspired by a neuroanatomical model of perception and action known as the two visual systems hypothesis, we conducted a study with two controlled experiments to compare four different kinds of spatial interaction: (1) voice-based input, (2) pointing with a visual cursor, (3) pointing without a visual cursor, and (4) pointing with a time-lagged visual cursor. Consistent with the two visual systems hypothesis, we found that voice-based input and pointing with a cursor were less robust to a display illusion known as the induced Roelofs effect than pointing without a cursor or even pointing with a lagged cursor. The implications of these findings are discussed, with an emphasis on how the two visual systems model can be used to understand the basis for voice and gestural interactions that support spatial target selection in large screen and immersive environments.

Diabetes represents an ever-increasing risk for the world population, and the rise of its prevalence is not expected to cease. To combat this alarming problem is the concomitant rise of new and improving mobile phones and wearable devices that can be exploited to fight diabetes. This paper presents a prototype of a monitoring system that will serve as a tool for improved patient monitoring by health care professionals (HCPs). Several physiological measurements were selected to perform the monitoring so that important data about a patient’s status can be collected and processed. Mobile technologies, cloud services and wireless protocols were used to develop a mobile application that can handle several devices and the data they gather. The resulting implementation provides a powerful infrastructure that could supply feedback to HCPs and patients as meaningful data, which can then be processed and potentially extend the assessment capabilities of an entire health care system.

Human–computer communication using hand gestures has always been difficult. More than half a century ago, people used differentways of interactionwith computers from the early mediums such as perforated game cards. Nowadays, if a richer lexicon of gestures is given, people can communicate more effectively with computers. Machine learning is now used to recognize and classify hand gestures in amore preciseway. In order to increase the communication between computers and humans, we proposed a technique, which uses a wearable low-cost device to generate the electrical impedance tomography (EIT) images to recover the inner impedance structure of a user’s wrist. This is done by measuring the transverse impedance between all the 16 pairs of electrodesofwrist band that lie on the skin of the user hand. The proposedtechnique is enough to integrate the technology into the prototypewrist band tomonitor and classify gestures in real time. We have conducted a study of 16 gestures with a focus on gross hand and pinch finger gestures. The results evaluation shows that the gross hand gestures achieved 90% accuracy inwrist position, while pinch gestures achieved 93% accuracy.

In recent years, many computational techniques have been proposed to help specialists in the fracture reduction process. This field of research is open and faces important challenges due to its intrinsic high complexity. The reduction of a complex bone fracture requires identifying the bone fragments, to estimate their proper position and to select and place adequate fixation devices in order to stabilise the fracture. The development of computer-assisted techniques aids the planning and the execution of those tasks. In this paper, the possibilities of mobile and wearable devices in a fracture reduction process are introduced and discussed from a global perspective. Specifically, we have reviewed opportunities and challenges of mobile and wearable devices in the following stages of the treatment of a bone fracture: diagnosis, planning, training, aiding and rehabilitation. Regarding mobile devices, the focus is placed on the visualisation of medical data, the human–computer interaction-specific issues, collaborative aspects and augmented reality possibilities.