Key frame extraction for falling detection

Falls are one of the greatest risks for older adults living alone at home. This paper presents a novel visual-based fall detection approach to support independent living for older adults through analysing the motion and shape of the human body. The proposed approach employs a new set of features to detect a fall. Motion information of a segmented silhouette when extracted can provide a useful cue for classifying different behaviours, while variation in shape and the projection histogram can be used to describe human body postures and subsequent fall events. The proposed approach presented here extracts motion information using best-fit approximated ellipse and bounding box around the human body, produces projection histograms and determines the head position over time, to generate 10 features to identify falls. These features are fed into a multilayer perceptron neural network for fall classification. Experimental results show the reliability of the proposed approach with a high fall detection rate of 99.60% and a low false alarm rate of 2.62% when tested with the UR Fall Detection dataset. Comparisons with state of the art fall detection techniques show the robustness of the proposed approach.

The world’s elderly population is growing every year. It is easy to say that the fall is one of the major dangers that threaten them. This paper offers a Trained Model for fall detection to help the older people live comfortably and alone at home. The purpose of this paper is to investigate appropriate methods for diagnosing falls by analyzing the motion and shape characteristics of the human body. Several machine learning technologies have been proposed for automatic fall detection. The proposed research reported in this paper detects a moving object by using a background subtraction algorithm with a single camera. The next step is to extract the features that are very important and generally describe the human shape and show the difference between the human falls from the daily activities. These features are based on motion, changes in human shape, and oval diameters around the human and temporal head position. The features extracted from the human mask are eventually fed in to various machine learning classifiers for fall detection. Experimental results showed the efficiency and reliability of the proposed method with a fall detection rate of 81% that have been tested with UR Fall Detection dataset.

Key frame selection, which aims to choose a small number of frames that well represent the entire video contents, is one of the important research issues in video content analysis. Many existing methods developed so far use some prior information about the contents of video frames based on high-level semantic features, and thus they are applicable only in particular problems. Without any prior information, approaches based on low-level index features such as clusters, correlation, and consecutive frame difference are useful. Key frames found by these approaches are expected to be as much different from each other as possible. Therefore, accurately evaluating the difference among candidate key frames is a key challenge. However, a standard similarity metric such as the squared error and the cross correlation tends to provide redundant key frames. To cope with this problem, we propose to use quadratic mutual information (QMI) as an alternative similarity measure, which allows us to capture higher-order correlation for more than two variables simultaneously. Through experiments, we demonstrate the usefulness of the proposed QMI-based key frame selection method that uses an accurate and computationally efficient least-squares QMI estimator.

This paper presents U-LanD, a framework for automatic detection of landmarks on key frames of the video by leveraging the uncertainty of landmark prediction. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in R2 score, with almost no overhead imposed on the model size.

This paper presents a semi-automatic algorithm for video object segmentation. Our algorithm assumes the use of multiple key video frames in which a semantic object of interest is defined in advance with human assistance. For video frames between every two key frames, the specified video object is tracked and segmented automatically using Learning Vector Quantization (LVQ). Each pixel of a video frame is represented by a 5-dimensional feature vector integrating spatial and color information. We introduce a parameter K to adjust the balance of spatial and color information. Experimental results demonstrate that the algorithm can segment the video object consistently with less than 2% average error when the object is moving at a moderate speed.

Wireless capsule endoscopy (WCE) is a non-invasive imaging technique to examine the entire small intestine. This paper presents a novel hierarchical key frame extraction algorithm based on the saliency map to automatically select a small number of key informative frames in the WCE video to assist the diagnostic process. We start from calculating the saliency map and extracting the inside colour, texture and shape features. We then compute the image information entropies with adapted weights on three features; calculate the differences of entropies in the successive frames and extract the first level key frames to divide the video into multi-segments. Finally, the mean shift clustering method is applied on each segment to obtain the final key frames. The proposed technique achieves the fidelity of 95.07% and the compression ratio of 87.9% on average, validating that the proposed scheme is highly promising for the key frame extraction in the WCE video.

Recognition of human movements is very useful for several applications, such as smart rooms, interactive virtual reality systems, human detection and environment modeling. The objective of this work focuses on the detection and classification of falls based on variations in human silhouette shape, a key challenge in computer vision. Falls are a major health concern, specifically for the elderly. In this study, the detection is achieved with a multivariate exponentially weighted moving average (MEWMA) monitoring scheme, which is effective in detecting falls because it is sensitive to small changes. Unfortunately, an MEWMA statistic fails to differentiate real falls from some fall-like gestures. To remedy this limitation, a classification stage based on a support vector machine (SVM) is applied on detected sequences. To validate this methodology, two fall detection datasets have been tested: the University of Rzeszow fall detection dataset (URFD) and the fall detection dataset (FDD). The results of the MEWMA-based SVM are compared with three other classifiers: neural network (NN), naive Bayes and K-nearest neighbor (KNN). These results show the capability of the developed strategy to distinguish fall events, suggesting that it can raise an early alert in the fall incidents.

We present an automatic and efficient method to register and stitch thousands of video frames into a large panoramic mosaic. Our method preserves the robustness and accuracy of image stitchers that match all pairs of images while utilizing the ordering information provided by video. We reduce the cost of searching for matches between video frames by adaptively identifying key frames based on the amount of image-to-image overlap. Key frames are matched to all other key frames, but intermediate video frames are only matched to temporally neighboring key frames and intermediate frames. Image orientations can be estimated from this sparse set of matches in time quadratic to cubic in the number of key frames but only linear in the number of intermediate frames. Additionally, the matches between pairs of images are compressed by replacing measurements within small windows in the image with a single representative measurement. We show that this approach substantially reduces the time required to estimate the image orientations with minimal loss of accuracy. Finally, we demonstrate both the efficiency and quality of our results by registering several long video sequences

Elderly falls are a growing phenomenon observed within the world. According to World Health Organization (WHO), it is the second leading cause of unintentional or accidental deaths among the elderly. Thus, the need for research regarding the development of fall detection systems is imperative. Researchers have utilized various approaches to develop fall detection systems, significant number of which have employed Machine Leaning (ML) algorithms for fall detection. In this study, we evaluated the efficiency of six ML algorithms on a public fall detection dataset. A robust deep neural network for fall detection (FD-DNN) is identified to be the current state-of-the-art, it detects falls by using a self-built sensor that consumes low power. By evaluating the efficiency of six machine learning algorithms on a publicly available joint fall detection dataset, the accuracy of the fall detection was increased from 99.17% to 99.88% by using the K-nearest Neighbor indicating that common machine learning algorithms can achieve identical or higher accuracy rendering the complex and expensive deep neural network-based fall detection systems inefficient.KeywordsFall detectionElderly fall detectionMachine learning algorithmsWearable fall detection systemSisFall datasetMobiFall datasetK-nearest neighborsKNNSupport vector machinesSVMDecision treeRandom forestLogistic regressionNaïve Bayes

Recognizing Human action in video is an very active research topic. There are a growing variety of human action datasets with different video length, different practitioners. Make human action recognition becomes a very difficult topic. A majority researchers solve the problem by extracting key frames from the videos. Most paper use feature Clustering methods to extract key frames in videos. On one hand, the large variety of visual content in videos make handcraft feature isn't effective enough, since there are no fixed descriptors can describe all video cases. On the other hand, traditional clustering algorithms are easily influenced by the choice of initial clustering centers. An Unsupervised feature learning and clustering method for key frame extraction is proposed in this paper, which can be used for human action recognition. Stacked auto-encoder(SAE) is trained using videos from 10 different human actions, SAE is used as a feature extractor to learn features representing human actions. Affinity Propagation Clustering algorithm is used to select key frames from video sequences. We use a variety of videos to do the experiments. Experiments demonstrate that our method can be effectively summarizing video shots considering different human actions.

The marine researchers analyze the behaviors of fish in the sea by manually viewing the full video for their research activity.Searching events of interest from a video database is a time consuming and tedious process.Video summary refers to representing the whole video using few frames.The objective of this work is to design and develop a statistical video summarization to perform the automatic detection of events of interest in underwater video.In this proposed work, a video is partitioned into adjacent and non-overlapping datacubes.Then, the video frames are transformed into wavelet subbands and the standard deviation between two consecutive frames is computed.Pixels of interest in frames are identified using threshold values.Key frames are identified using Local Maxima and Local Minima.The proposed work effectively detects even the movement of small water bodies such as crabs which is not detected using the existing methods.Finally, this paper presents the experimental results of proposed method and existing methods in terms of metrics that measure the valid of the work.

Automatic fall detection approaches are essential for elderly people, particularly for those who live alone, because of the pressing need for immediate medical assistance. In this paper, we proposed a highly effective fall detection method based on a joint motion map using two parallel convolutional neural networks. Compared with the most commonly used joint trajectory method (JTM), our proposed method provided three major improvements. First, the three channels (R, G and B) of a pixel were creatively used to store relative motion information of a certain joint in 3D coordinates. Thus, the human action recognition problem was simplified as a multi-class problem after actions were encoded as images. Moreover, the input parameters were dramatically reduced because of concentration only on 25 joints of the human skeleton. Second, human motion information in each frame was encoded as an independent slice of a motion image, which avoids the information loss problem caused by action trajectory overlap. Third, under the guidance of a medical experiment, the limit of stability test (LOST), the start key frame and end key frame of a possible fall can be exactly estimated. Therefore, motion images can be generated with a fixed size. Our method was evaluated on two publicly available datasets: the Telecommunication Systems Team fall detection dataset V2 (TST v2) and UTKinect-Action3D Dataset (UT-A3D). The experimental results show that our method achieved an accuracy of 97.35% on TST v2 and performed excellently in the fall discrimination capability test on UT-A3D.

Key frame extraction is an important manner of video summarization. It can be used to interpret video content quickly. Existing approaches first partition the entire video into video clips by shot boundary detection, and then, extract key frames by frame clustering. However, in most team-sport videos, a video clip usually includes many events, and it is difficult to extract the key frames related to all of these events accurately, because different events of a game shot can have features of similar appearance. As is well known, most events in team-sport videos are attack and defense conversions, which are related to global translation. Therefore, by using fine-grained partition based on the global motion, a shot could be further partitioned into more video clips, from which more key frames could be extracted and they are related to the events. In this study, global horizontal motion is introduced to further partition video clips into fine-grained video clips. Furthermore, global motion statistics are utilized to extract candidate key frames. Finally, the representative key frames are extracted based on the spatial–temporal consistence and hierarchical clustering, and the redundant frames are removed. A dataset called SportKF is built, which includes 25 videos of 197,878 frames in 112 min and 764 key frames from four types of sports (basketball, football, American football and field hockey). The experimental results demonstrate that the proposed scheme achieves state-of-the-art performance by introducing global motion statistics.

With rapid increasing research in emotion detection, facial expression recognition becomes more popular as a vital emotion measurement instrument. Happy is one of the most common facial expressions that indicates a positive human emotional status. Although many facial expression recognition methods have been well established for recognizing multiple emotions, studies on happy detection are very limited, especially for processing the videos. In this paper, we propose a new single emotion recognition method to recognize happy emotion from key frames of facial expression videos. In our method, we extract key frames that demonstrate the highest intensity level of expressions among all frames in two steps. In the first step, an Inception ResNet framework is utilized to segment an facial expression process in a video into three parts: onset, apex and offset. In the second phase, we choose first three frames in the apex segment as key frames. A fine-tuned convolutional neural network (CNN) then classifies the key frames into happy and non-happy classes. Our experimental results demonstrate that the proposed approach achieves higher accuracy than four counterpart methods in recognizing happy emotions, with the accuracy of 98.4% and 94.2% on two benchmark facial expression datasets, i.e., the extended Cohn-Kanade (CK+) and MMI.

This paper proposes a statistical approach to detect and classify human falls based on both visual data from camera and accelerometric data captured by accelerometer. Specifically, we first use a Shewhart control chart to detect the presence of potential falls by using accelerometric data. Unfortunately, this chart cannot distinguish real falls from fall-like actions, such as lying down. To bypass this difficulty, a neural network classifier is then applied only on the detected cases through visual data. To assess the performance of the proposed method, experiments are conducted on the publicly available fall detection databases: the University of Rzeszow's fall detection (URFD) dataset. Results demonstrate that the detection phase play a key role in reducing the number of sequences used as input into the neural network classifier for classification, significantly reducing computational burden and achieving better accuracy.