Smartphone‐based fatigue detection system using progressive locating method

As the number of road accidents increases, it is critical to avoid making driving mistakes. Driver fatigue detection is a concern that has prompted researchers to develop numerous algorithms to address this issue. The challenge is to identify the sleepy drivers with accurate and speedy alerts. Several datasets were used to develop fatigue detection algorithms such as electroencephalogram (EEG), electrooculogram (EOG), electrocardiogram (ECG), and electromyogram (EMG) recordings of the driver’s activities e.g., DROZY dataset. This study proposes a fatigue detection system based on Fast Fourier Transform (FFT) and Discrete Wavelet Transform (DWT) with machine learning and deep learning classifiers. The FFT and DWT are used for feature extraction and noise removal tasks. In addition, the classification task is carried out on the combined EEG, EOG, ECG, and EMG signals using machine learning and deep learning algorithms including 1D Convolutional Neural Networks (1D CNNs), Concatenated CNNs (C-CNNs), Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), k-Nearest Neighbor (KNN), Quadrature Data Analysis (QDA), Multi-layer Perceptron (MLP), and Logistic Regression (LR). The proposed methods are validated on two scenarios, multi-class and binary-class classification. The simulation results reveal that the proposed models achieved a high performance for fatigue detection from medical signals, with a detection accuracy of 90% and 96% for multiclass and binary-class scenarios, respectively. The works in the literature achieved a maximum accuracy of 95%. Therefore, the proposed methods outperform similar efforts in terms of detection accuracy.

With the development of technology and society, cars have become an indispensable part of people's daily life and business. As a result, the problem of traffic accidents has also become a considerable cause of casualties in society today. Driver fatigue detection is considered a promising technology to reduce traffic accidents. Cars can remind drivers or make corresponding actions when noticing that drivers are tired through the data transmitted by the detection system. Machine learning is the essential part of driver fatigue detection, which has the advantages of high performance and accuracy. Most driver fatigue detection Systems will use machine learning to get a better result. This article will analyze the Machine Learning currently used in driver fatigue detection, including Hidden Markov Model, Support Vector Machine, Gradient Boosting Decision Tree, Convolutional Neural Network, Deep Belief Networks, Recurrent neural network, and Radial Basic Function Network.

Eye detection can be used in intelligent human-computer interfaces, driver drowsiness detection, security, and biology systems. In this paper a new method for eye detection based on rectangle features and pixel-pattern-based texture feature (PPBTF) is proposed. First, Adaboost cascade classifier by rectangle features is constructed to do rough eye detection in a front facial image. Second, the result image patches are cropped and scaled to 24×12 to compute the features of PPBTF, then, put these features into an Adaboost and SVM classifier for an accurate detection. Some eye open front facial images from FERET Database are chosen to do the experiments, and the results show that the approach is real time and is effective for eye detection.

In recent years, fatigue driving has become the main cause of traffic accidents, leading to increased attention towards fatigue detection systems. However, the pooling and strided convolutional operations in fatigue detection algorithm based on traditional deep learning methods may led to the loss of some useful information. This paper proposed a novel -shaped convolutional network ( ) aiming to address this issue. Unlike traditional network structures, incorporates the operations of upsampling features and concatenating high- and low-level features, enabling full utilization of useful information. Moreover, considering that the fatigue state is a mental state involving temporal evolution, we proposed the novel long short-term memory (LSTM)- -shaped convolutional network (LSTM- ), a parallel structure composed of LSTM and for fatigue detection, where extracts time-invariant features with location information, and LSTM extracts long temporal dependencies. We compared LSTM- with six competing methods based on two datasets. Results showed that the proposed algorithm achieved higher classification accuracy than the other methods, with 94.25% on EEG data (binary classification) and 82.19% on EOG data (triple classification). Additionally, the proposed algorithm exhibits low computational cost, good training stability, and robustness against insufficient training. Therefore, it is promising for further implementation of fatigue online detection systems.

Driving fatigue is a physiological phenomenon that often occurs during driving. When the driver enters a fatigue state, they will become distracted and unresponsive, which can easily lead to traffic accidents. The driving fatigue detection method based on a single information source has poor stability in a specific driving environment and has great limitations. This work helps with being able to judge the fatigue state of the driver more comprehensively and achieving a higher accuracy rate of driving fatigue detection. This work mainly introduces research into different signal fusion methods to detect fatigue drive. This work will take the normal driver’s breathing signal, eye signals, and steering wheel signal as research objects and collect and isolate the characteristics of the fatigue detection signal. Research was then conducted on different signal fusion methods for the detected depth of breath. Change of steering angle, eyelid closure, and blinking marks and the fatigue driving experiment was designed to evaluate the results of different data fusion methods. Experimental results show that the detection accuracy of the heterogeneous signal fusion method in fatigue detection is as high as 80%.

Fatigue detection has many applications, one of them is intelligent transportation for accident prevention. It is challenging to detect driver fatigue adopting photometric stereo and 3D fatigue-related facial action unit identification. During 3D reconstruction shadows are encountered, causing reconstruction error, therefore, for accurate fatigue detection shadow removal is crucial. This study presents a novel snake based approach for shadow handling in photometric stereo. Seven light sources were employed and region visibility of the light source for each region of interest (ROI) was deduced. Surface normal estimation enhancement was achieved via snake model based shadow labeling. Discrepancies measured among shadowed regions and ROIs were recorded. Light sources having high intersection with the shadow map were excluded from the reconstruction process at pixel level. This results in an automatic driver fatigue related action unit detection employing uncalibrated Lambertian surface under shadow conditions. The developed method was tested and compared with other established driver fatigue detection methods and higher performance in terms of accuracy ( i.e. 97.85%) was achieved. The developed method proves to be well suited for driver fatigue detection under uncalibrated conditions.

This paper proposes a mobile system for real-time fetal heart rate (FHR) monitoring and fetal distress detection. In the proposed system, FHR values are collected using machine vision techniques and analyzed using a pattern matching approach. When a fetal distress is detected, a medical alarm will automatically notify medical experts through a mobile GSM network. Through the proposed system, not only a pregnant woman can track the healthy status of her baby, but also medical experts can allocate medical resources in time. The implementation result shows that the developed mobile system can be a helpful application for medical management.

Muscle fatigue is a problem that is often encountered in daily life. To facilitate muscle fatigue detection, a pulse width modulation (PWM) and ESP8266-based fatigue detection and recovery system is introduced in this paper to help alleviate muscle fatigue. The ESP8266 is employed as the main controller and communicator, and PWM technology was employed to achieve adaptive muscle recovery. Muscle fatigue can be detected by surface electromyography (sEMG) signals and monitored in real time via a wireless network. With the help of the proposed system, human muscle fatigue status can be monitored in real-time, and the recovery vibration motor status can be optimized according to muscle activity state. Experimental results showed that the proposed system is sensitive and stable, requires minimal power, and can be of benefit to muscle fatigue monitoring and recovery.

Driver's fatigue detection has been realized based on driver's mouth geometrical features. It can give some information when driver is fatigue. For better speed and reliability, a new method which was based on combined Adaboost algorithm and particle filter was applied. Then the mouth verification was applied according to prior knowledge. Driver's state was judged by the geometrical features in a period of time. As a result, the detection accuracy was improved. This method can meet the requirement of driver's fatigue detection.

Fatigue driving detection helps reduce the occurrence of traffic accidents. Currently, the fatigue detection methods still have following challenges: 1) In the visual fatigue detection system, face detection is the foundation of fatigue detection and it is always susceptible to changes of illumination and facial expression, which lowers the accuracy of face detection. 2) It is so difficult to distinguish some behaviors of a driver such as singing and yawning in a single frame that making false fatigue detection in the visual fatigue detection system. To overcome the problems mentioned above, a fatigue detection method based on sequential mouth feature with long short-term memory network is proposed. The proposed method mainly consists of two parts: locating of facial landmark based on face and mouth detection, and yawning detection based on residual network (Resnet) and long short-term memory (LSTM). The face detection method is based on Histogram of Oriented Gradient (HOG) feature and Support Vector Machine (SVM) classifier, which is robust to the changes of illumination and slightly facial expression. The mouth detection is based on cascaded regression tree, which extracts the mouth region from the detected face. The Resnet and LSTM based yawning detection extracts spatial-temporal features from the mouth regions between frames and distinguishes the behaviors about yawning to identify fatigue of a driver with sequential mouth feature. The proposed method achieves accuracy of 94.9% on the YawDD dataset, which outperforms the compared methods. Experimental results demonstrate that the proposed method satisfies the performance of fatigue detection.

Android operating system has gained a wide popularity in the recent years due to the open environmental nature of the Android framework. This has given a tremendous flux to the Malware developers to target the Android users. Thus, personal privacy theft has become a major issue. Various mobile malware detection systems are proposed in the recent years to address this issue. Operation level, Analysis technique and Accuracy are the core design aspects for any detection method. This paper focuses on surveying the existing mobile malware detection systems. Also, it provides a detailed comparison to summarize and analyze the detection methods under concern.

Drowsy driving is one of the main reasons of road accidents. Different techniques have been reported in literature to detect driver's drowsiness, but almost all the prevailing systems only alert the driver if drowsiness is detected. Consequently, the drowsy driver continues driving, with a high risk of devastating accident. In this paper, we proposed and verified an EEG based system which not only alerts the driver by alarm, but also puts the vehicle in semiautomatic parking mode by controlling fuel supply if drowsiness is detected. At the same time, it reports nearby police station by SMS which contains necessary information to take essential steps locating the vehicle. Stored EEG signals, obtained with wireless wearable headsets from numerous subjects in different conditions by different research groups, were used in this work. Power spectrum analyses were carried out in MATLAB to determine the dominant frequency components in the brain signals. The slow wave to fast wave ratios of EEG activities were assessed for a number of epochs to determine driver's drowsiness. GPS and GSM modules were used with Arduino MEGA for tracking, remote notification and servomotor control. Performance of the proposed system was evaluated by stored data which confirmed its feasibility and reliability.

Vehicular fatigue and distraction detection systems are important tools to avoid traffic accidents. Furthermore, there are many related papers proposing compositions using these techniques. Nevertheless, most of the validation tests performed with these devices happen in simulated conditions or environments, without a test with actual users on a real situation. Also, few works analyze behavioral features using these systems. In this work, we analyze the behavioral aspects of users from a fatigue and distraction detection system with and without audio feedback. Our results indicate that this feature has a positive effect on the drivers’ behavior.

With the rapid development of digital image processing and recognition, the intelligent grid wins a great opportunity. We could be able to get the information from digital images quickly based on computer analysis, algorithms of deep learning and machine learning, grid data extraction, model training and image recognition. The article firstly reviewed the current status of traditional grid, including a broad overview of the specific strengths and weaknesses, secondly analyzed the intelligence algorithms of deep learning and machine learning based on neural network, then achieved the recognition and reconstruction of wiring diagram through image recognition based on neural network, lastly showed the prospects of image recognition in power system.

Single-modal data has a limitation on fatigue detection, while the shortage of labeled data is pervasive in multimodal sensing data. Besides, it is a time-consuming task for board-certified experts to manually annotate the physiological signals, especially hard for EEG sensor data. To solve this problem, we propose FedUSL (Federated Unified Space Learning), a federated annotation method for multimodal sensing data in the driving fatigue detection scenario, which has the innate ability to exploit more than four multimodal data simultaneously for correlations and complementary with low complexity. To validate the efficiency of the proposed method, we first collect the multimodal data (aka, camera, physiological sensor) through simulated fatigue driving. The data is then preprocessed and features are extracted to form a usable multimodal dataset. Based on the dataset, we analyze the performance of the proposed method. The experimental results demonstrate that FedUSL outperforms other approaches for driver fatigue detection with carefully selected modal combinations, especially when a modality contains only \(10\% \) labeled data.