CNN-Based Camera-less User Attention Detection for Smartphone Power Management

User context recognition using smartphone sensors and classification models

BackgroundMobile electronic devices, such as mobile phones and PDAs, have emerged as potentially useful tools in the facilitation and maintenance of weight loss. While RCTs have demonstrated a positive impact of mobile interventions, the extent to which mobile electronic devices are more effective than usual care methods is still being debated.ResultsElectronic databases were systematically searched for RCTs evaluating the effectiveness of mobile electronic device interventions among overweight and obese adults. Weighted mean difference for change in body weight was the primary outcome. The search strategy yielded 559 citations and of the 108 potentially relevant studies, six met the criteria. A total of 632 participants were included in the six studies reporting a mean change in body weight. Using a random-effects model, the WMD for the effect of using mobile electronic devices on reduction in body weight was −1.09 kg (95% CI −2.12, −0.05). When stratified by the type of mobile electronic device used, it suggests that interventions using mobile phones were effective at achieving weight loss, WMD = −1.78 kg (95% CI −2.92, −0.63).ConclusionsThis systematic review and meta-analysis suggests that mobile electronic devices have the potential to facilitate weight loss in overweight and obese populations, but further work is needed to understand if these interventions have sustained benefit and how we can make these mHealth tools most effective on a large scale. As the field of healthcare increasingly utilizes novel mobile technologies, the focus must not be on any one specific device but on the best possible use of these tools to measure and understand behavior. As mobile electronic devices continue to increase in popularity and the associated technology continues to advance, the potential for the use of mobile devices in global healthcare is enormous. More RCTs with larger sample sizes need to be conducted to look at the cost-effectiveness, technical and financial feasibility of adapting such mHealth interventions in a real clinical setting.

The use of metaphor is essential in user interface design, particularly for the mobile landscape, as the visual environment continues to be populated with more and more mobile electronic devices. A metaphor allows us to understand one concept in terms of another. Although considerable research has gone into the mobile technology, little attention has been paid to mobile interface metaphor, which is the key to user interaction. This paper explores the role of metaphor in interfaces in facilitating user interaction with mobile devices. It presents a classification of metaphors. It also proposes a framework with salient factors in relation to visual communication with metaphors. It also offers some thoughts on the use of new metaphors.

In this study, the purpose was to evaluate the effect of mobile electronic devices (MEDs) use on the sleep states of university students. The study was conducted in a cross-sectional fashion between April 21, 2019 and May 31, 2019 with the participation of the students registered in a public university at medical faculties and faculty of health sciences. Television viewing, MED and desktop computer use form, Morningness-Eveningness Questionnaire, Pittsburgh Sleep Quality Index (PSQI), Epworth Sleepiness Scale (ESS), and Sleepiness Severity Index (SSI) were used as data collection tools. The data of 752 students (56.5% girls, mean age: 20.57 [SD = 1.54] years) were analyzed. In this study, overall MED use times and PSQI, ESS, and SSI total scores were higher in students who had an evening-oriented chronotype. There was a negative relationship between overall and evening MED use times and the chronotype scores of the students. As a result of the regression analyses applied to the PSQI, ESS, and SSI dependent variables, a positive and significant relationship was detected with smartphone use times (P < 0.05, P = 0.05, and P < 0.001, respectively). In addition, there was a positive relationship between PSQI and ESS variables and iPod touch use times (P < 0.001 and P < 0.05, respectively). Overall MED use times were determined to be higher in students who had Type E chronotype. It was also determined that their sleep quality was poorer and their insomnia severity was higher, and there was more sleepiness in the students with chronotype E. Comprehensive interventions must be planned by professional healthcare staff for students on the topics of sleep hygiene and conscious media use.

BackgroundPatient activation has an impact on the management of patients’ health, clinical outcomes, and treatment costs. Mobile electronic devices (MEDs) have shown the potential to engage patients in wellness behavior. Furthermore, the potentially positive role of MEDs is evident in supporting health professionals in their practice.ObjectiveThis study aims to explore the impact of MEDs on patient activation to search for information on chronic conditions and medications and the impact of MEDs on the empowerment of health professionals or future health professionals.MethodsWe conducted 6 focus groups—2 with health sciences students, 2 with health professionals, and 2 with hospitalized patients with chronic conditions. A protocol comprising eight questions was used to guide discussions. Audio-recorded data were transcribed verbatim and analyzed thematically; a ranking system was used to analyze the relevance of identified themes and subthemes, using a coding system depicted by the + symbol, to indicate different relevance levels.ResultsOur results suggest that MEDs can positively affect patient activation to search for chronic conditions and medication information by facilitating patients’ information-seeking behavior. Key drivers leading to patients’ activation to seek information related to chronic conditions and medications through MEDs were the accessibility and abundance of available and detailed information, reduced search time, information updates, and convenience in finding information at any time and place. The lack of accurate information in one’s native language, access to incorrect information, and limited access to the internet were key obstacles to seeking information related to chronic conditions and medications via MEDs. In addition, findings of this study suggest that MEDs in general and mobile apps, in particular, may have a positive impact on the work routine of health care professionals as they enable them to make quicker decisions by accessing the required information faster, thus improving practice efficiency. Furthermore, the appropriate usage of MEDs by patients for seeking information about their chronic conditions and medications may positively impact the physician-patient relationship. All focus groups recognized the questionable reliability of health information on the internet and its potential negative effects on patients. Therefore, our findings suggest the need for an additional role of health professionals in assisting patients in using MEDs to search for health and medication information, such as providing reliable websites and mobile apps where patients can safely search for health-related information on the web.ConclusionsThe use of MEDs may help activate patients to seek chronic conditions and medication-related information, potentially leading to better management of their chronic conditions and medications. Our findings also highlight the positive impact MEDs may have on empowering health professionals in their practice and the need for health professionals to help patients through specific education that addresses MEDs utilization for chronic conditions and medication information seeking.

Fuel load is a crucial parameter for evaluating design fires for buildings. However, the availability of fuel load data is currently hindered by the lack of an efficient on-site data collection method. This research develops a new methodology for fuel load surveys that can facilitate the collection, storage, and analysis of fuel load data for a variety of building occupancies. The new survey method harnesses recent developments in mobile electronic devices, cloud storage, and machine vision to efficiently complete fuel load surveys in buildings. A four-step method is developed comprising digital inventory, data organization, item matching through computer vision, and fuel load estimation. The method is completed through an interactive electronic surveying form accessible on any mobile device with an internet connection, such as a tablet. The application allows taking digital measurements and pictures of the room and content, which are stored, and later searched through retail search engines using image recognition. Automatic matching of the picture with an online catalogue item gives access to further information about this item which are then used to evaluate the item fuel load, using a table of materials calorific values coded in the application. A wide adoption of this method could provide benefits by progressively populating such digital fuel load database. The results can then be used to provide design guidelines for fuel load density in codes and standards, for application in performance-based design of structures in fire.

This study investigated the relation between preschoolers’ mobile electronic device (MED) use and sleep disturbances. A national sample of 402 predominantly college-educated and Caucasian mothers of 3–5-year-olds completed a survey assessing their preschoolers’ MED use, bedtime resistance, sleep duration, and daytime sleepiness. Heavier evening and daily tablet use (and to some extent, smartphone use) were related to sleep disturbances. Other forms of MED use were not consistently related to sleep disturbances. In addition, playing games on MEDs at bedtime was related to compromised sleep duration, although other forms of MED use at bedtime were not related to sleep outcomes. Although the relations between MED use and sleep disturbances were small in size, they were larger than the relations between sleep and other predictors in the models. Continued work should investigate how MED exposure is related to children’s cognitive, psychological, emotional, and physiological development, particularly given the popularity and widespread use of these devices.

Convolutional neural networks (CNNs) are widely employed in many image recognition applications. With the proliferation of embedded and mobile devices, such applications are becoming commonplace on mobile devices. Network pruning is a commonly used strategy to reduce the memory and storage footprints of CNNs on mobile devices. In this article, we propose customized versions of the sparse matrix multiplication algorithm to speed up inference on mobile devices and make it more energy efficient. Specifically, we propose a Block Compressed Sparse Column algorithm and a bit-representation-based algorithm (BitsGEMM) that exploit sparsity to accelerate the fully connected layers of a network on the NVIDIA Jetson TK1 platform. We evaluate the proposed algorithms using real-world object classification and object detection applications. Experiments show that performance speedups can be achieved over the original baseline implementation using cuBLAS. On object detection CNNs, an average speedup of 1.82× is obtained over baseline cuBLAS in the fully connected layer of the VGG model, whereas on classification CNNs, an average speedup of 1.51× is achieved for the fully connected layer of the pruned-VGG model. Energy consumption reduction of 43--46% is also observed due to decreased computational and memory bandwidth demands.

Automatic feature selection for context recognition in mobile devices

FitCNN: A cloud-assisted and low-cost framework for updating CNNs on IoT devices

Among the many deep learning methods, the convolutional neural network (CNN) model has an excellent performance in image recognition. Research on identifying and classifying image datasets using CNN is ongoing. Animal species recognition and classification with CNN is expected to be helpful for various applications. However, sophisticated feature recognition is essential to classify quasi-species with similar features, such as the quasi-species of parrots that have a high color similarity. The purpose of this study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN model based on transfer learning (some parrots have quite similar colors and shapes). We acquired the images in two ways: collecting them directly from the Seoul Grand Park Zoo and crawling them using the Google search. Subsequently, we have built advanced CNN models with transfer learning and trained them using the data. Next, we converted one of the fully trained models into a file for execution on mobile devices and created the Android package files. The accuracy was measured for each of the eight CNN models. The overall accuracy for the camera of the mobile device was 94.125%. For certain species, the accuracy of recognition was 100%, with the required time of only 455 ms. Our approach helps to recognize the species in real time using the camera of the mobile device. Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area.

There is tremendous growth in the amount of digital video creation and communication with the proliferation of mobile consumer electronic devices such as cellular phones with video capture and display capabilities. This trend will continue with the demand for higher spatial and temporal video resolutions accompanied by 3D and multiview video. Since it is not practical to store or transmit the raw digital video data, it is essential to reduce it with digital video compression. To cope with the vast amount of raw digital video data, digital video compression standards use increasingly more complex algorithms. These algorithms achieve higher compression at the same quality at the expense of significant computational complexity. Therefore, it is essential to design low power digital video compression hardware especially for mobile devices. In this talk, I will present two sets of power reduction techniques for digital video compression hardware. The ones in the first set reduce the power consumption with no impact on the video quality. The ones in the second set trade off video quality for power consumption which also enables adjusting video quality based on available energy in the battery of a mobile device for graceful degradation. I will show the impact of these techniques on the power consumption of digital video compression hardware implementing H.264/AVC and H.265/HEVC standards.

Low operating temperature micro solid oxide fuel cell (micro-SOFC) systems which utilize liquid fuels are attractive for high energy efficiency and density. Long time operation is possible for micro-SOFCs by refueling and it is not restricted by battery charge. Hence, it is expected to alternative power source for compact mobile electronic devices which consume a large amount of power. In previous, diverse types of micro-SOFCs focused on its performance was reported and a maximum power density of 1 W/cm2 at 500oC was achieved. However, they have many problems of durability and repeatability for practical use. Additionally, it is necessary to consider how to heat up micro-SOFC cells to operating temperature, compatibility with other electronic circuits and thermally self-sustained condition. In this study, micro-SOFC system was proposed in order to adopt the micro-SOFCs into a power source of compact mobile electric devices. We designed the micro-SOFC package with thermal insulation and heat exchanger. Its insulation performance was experimentally confirmed and power generation test with micro heater in the micro-SOFC package was performed. Micro-SOFC packages require thermal insulation to keep the surface temperature below about 85oC, which is the maximum operating temperature of other electronic circuits. We applied vacuum insulation widely used for thermal insulation in MEMS devices to the package. The heat of the exhaust gas was recovered by the liquid fuel through heat exchanger. From the calculation of temperature distribution by the finite element method, surface temperature of the package was kept from 57 to 88oC when operating temperature is 500oC. As long as the liquid is evaporated, temperature of the heat exchanger is kept constant. Therefore, heat conduction from high temperature glass flow channels to vacuum insulator was prevented and low surface temperature was achieved. Applying micro-SOFC system as independent power sources, thermally self-sustained condition is required, i.e. micro-SOFC packages should keep operating temperature more than 300oC only by the generated heat from reaction of fuel cells during power generation. For thermally self-sustained condition, the amount of heat generated during power generation needs to exceed that of heat loss from surface of the package and transferred heat by exhaust gas. We calculated thermally self-sustained condition of a micro-SOFC system with 4 W output fueled by methanol and water that has reformer and heat exchanger from exhaust gas. From the result, generated heat from reaction of fuel cells was 4.8W, heat loss from surface was 1.5 W and transferred heat by exhaust gas was 4.6 W at 500oC. The result confirmed that transferred heat by exhaust gas was dominated in this system. Hence, recovering the heat of exhaust gas with the gas-liquid heat exchanger was essential to achieve thermally self-sustained condition. Micro-SOFC cells and micro heater were fabricated in general MEMS technology. Power generation test and electrochemical impedance spectroscopy of micro-SOFC (Pt-Pd | 15 mol% Y-doped BaZrO3 | Pt-Pd) by micro heater were measured in the temperature range of 200-330oC. A maximum power density of 1.12 mW/cm2 and open circuit voltage of 1.09 V at 330oC were obtained using wet H2 as fuel and air as the oxidant. As a result of impedance measurement, electrode reaction resistance was very high in cell total resistance. Therefore, the improvement of electrode will reduce cell resistance to a level for actual utility. We actually fabricated the micro-SOFC package installed vacuum structure and heat exchanger to evaluate cooling performance of the insulation package and performed power generation test in the package. In the case of heat exchanger filled with 50 vol% ethanol solution, surface temperature was 89.6oC when micro heater temperature was 500oC. In accordance with the result, it was possible to keep surface temperature of the package sufficiently low with heat exchanger. As a result of power generation test of micro-SOFC inside the package, a maximum power density of 1.27 mW/cm2 and open circuit voltage of 0.94 V at 300oC were obtained using wet 25% or 50% H2 as fuel and air as the oxidant. Additionally, surface temperature of the package was kept under 80oC. Therefore I achieved power generation in compact micro-SOFC system with thermal insulation. The micro-SOFC package was designed for applying micro-SOFCs to power source of mobile electronic devices in this study. As a result of calculation, the package installed vacuum insulation and heat exchanger could achieve thermally self-sustained condition and low surface temperature. Power generation test of micro-SOFC inside the prototype package was achieved and the package kept low surface temperature. Hence, this study presented that applicability of micro-SOFC system to power source for compact mobile electronic devices.

Background/Objectives: Running out of battery on phones, laptops and iPads while in public places is a problem that we are trying to avoid. Power consumption of smartphone and other mobile electronics applications is one of the major concerns due to the limited battery capacity. In public places, the grid power supply socket is difficult to find. This paper describes the development of the electric source vending machine for public used. Methods/Statistical Analysis: Development of this machine involves the structural mechanical design and electronic system. The photo sensor is used to detect the Light-Emitting Diode (LED) blinking from the digital electric meter to count the total power use by the different loads. Standard coin acceptor used to determine whether coins are genuine or counterfeit. Adruino microcontroller serves as the main system that controls and process the entire machine system. Findings: The systems are able produce stable 240 V AC and 5 V DC output voltage. Mobile electronic devices can use the electric source from the machine by connecting the adapter or Universal Serial Bus (USB) wire into the ports in the vending machine. Applications/Improvements: An electric source vending machine was developed. It can it can provide an electric source system in public places. This system provides a stable source of electrical 240 V AC and 5 V DC for charging various mobile electronic devices.

Artificial Intelligence (AI) has had a boom in recent years thanks to technological development, it is increasingly used in various fields of research and many of these use mobile devices. Emotion recognition is one of them, as it helps in neuroscience, computer science, and medical applications. There is information where applications that are developed on mobile devices for emotion recognition, but none considers the performance of the algorithm used. The purpose of this article was to evaluate how it affects the performance of a mobile application based on Artificial Intelligence (AI) for emotion recognition according to the properties of mobile devices of different ranges. For this purpose, two different Convolutional Neural Network (CNN) architectures are evaluated, which will be analyzed according to the following metrics: response time, RAM, CPU usage, and accuracy. The results show that a deep layered CNN has better performance and lower computational cost compared to a conventional CNN on mobile devices.