Video shot segmentation using fusion of SVD and mutual information features

In this paper, we propose a new method for detecting shot boundaries in video sequences by performing Hilbert transform and extracting feature vectors from Gray Level Co-occurrence Matrix (GLCM). The proposed method is capable of detecting both abrupt and gradual transitions such as dissolves, fades and wipes in the video sequences. The derived features on processing through Kernel k-means clustering procedure results in efficient detection of abrupt and gradual transitions, as tested on TRECVid video test set containing various types of shot transition with illumination effects, object and camera movement in the shots. The results show that the proposed method yields better result compared to that of the existing tran- sition detection methods.

We propose a technique for video shot segmentation and classification based on singular value decomposition (SVD). For the input video sequence, we create a feature-frame matrix A, and perform the SVD on it. From this SVD, we are able to not only derive the refined feature space to better segment the video sequence along the time axis, but also define metrics to enable classifications of the detected video shots. Using these SVD properties, we achieve the two goals of accurate video shot segmentation, and visual content-based shot classification at the same time.

Video segmentation is fundamental to a number of applications related to video retrieval and analysis. Shot change detection is the initial step of video segmentation and indexing. There are two basic types of shot changes. One is the abrupt change or cut, and the other is the gradual shot transition. The smooth variations of the video feature values in a gradual transition produced by the editing effects are often confused with those caused by camera or object motions. To overcome this difficulty, it is reasonable to estimate the motions and suppress the disturbance caused by them. In this thesis, we explore the possibility to exploit motion and illumination estimation in a video sequence to detect both abrupt and gradual shot changes. A generalized optical flow constraint that includes an illumination parameter to model local illumination changes is employed in the motion and illumination estimation. An iterative process is used to refine the generalized optical flow constraints step by step. A robust measure that is the likelihood ratio of the corresponding motion-compensated blocks in the consecutive frames is used for detecting abrupt changes. For the detection of gradual shot transitions, we compute the average monotony of intensity variations on the stable pixels in the images in a twin-comparison framework. We test the proposed algorithm on a number of video sequences in TREC 2001 and compare the detection results with the best results reported in the TREC 2001 benchmark. The comparisons indicate that the proposed shot change detection algorithm is competitive against the best existing algorithms.

This article proposes a new framework using singular value decomposition based on existing methods for detecting shot boundaries in video sequences. The experimental results show that this algorithm and framework are efficient because sharp cuts are detected precisely and transition cuts are detected efficiently.

We present a novel technique for detecting the presence of a gradual transition in video sequences and automatically identifying its type. Our scheme focuses on analyzing the characteristics of the underlying special edit effects and estimates actual transitions by polynomial data interpolation. In particular, a B-spline interpolation curve fitting technique is used. We make use of goodness of fitting to determine the presence of gradual transitions. Our approach is able to recover the original transition behavior of an edit effect even if it is distorted by various post-processing stages. Our gradual transition detectors have been extensively tested on various genres of real video sequences to evaluate the performance of the proposed algorithms.

In sports programs, there are many special cases making shot boundary detection difficult. Targeted for these special cases, not be considered by existing work, this paper presents a shot boundary detection scheme to detect both cuts and gradual transition efficiently. For shot detection, the algorithm is proposed to resist continuous flashes, camera occlusion or image blur that have not been considered before. For gradual transition detection, a unified method is presented to detect various transitions or special effects, together with an algorithm to reduce the false positives caused by fast camera or object motions. The cut detection and gradual transition detection are implemented serially to avoid repeated detection operations. Compared with existing typical works, the proposed scheme obtains higher correct detection rate and fast detection speed, and is more suitable for sports program analysis.

Video shot boundary detection (SBD) is the first step of video analysis, summarization, indexing, and retrieval. In SBD process, videos are segmented into basic units called shots. In this paper, a new SBD method is proposed using color, edge, texture, and motion strength as vector of features (feature vector). Features are extracted by projecting the frames on selected basis vectors of Walsh-Hadamard transform (WHT) kernel and WHT matrix. After extracting the features, based on the significance of the features, weights are calculated. The weighted features are combined to form a single continuity signal, used as input for Procedure Based shot transition Identification process (PBI). Using the procedure, shot transitions are classified into abrupt and gradual transitions. Experimental results are examined using large-scale test sets provided by the TRECVID 2007, which has evaluated hard cut and gradual transition detection. To evaluate the robustness of the proposed method, the system evaluation is performed. The proposed method yields F1-Score of 97.4% for cut, 78% for gradual, and 96.1% for overall transitions. We have also evaluated the proposed feature vector with support vector machine classifier. The results show that WHT-based features can perform well than the other existing methods. In addition to this, few more video sequences are taken from the Openvideo project and the performance of the proposed method is compared with the recent existing SBD method.

Recently, multilabel classification algorithms play an increasingly significant role in data mining and machine learning. However, some existing mutual information-based algorithms ignore the influence of the proportions of labels on the correlation degree between features and label sets. Besides, the correlation degree of label sets cannot be accurately measured in most traditional ReliefF algorithms, and the repeated calculation arises from the division of heterogeneous neighbors. To overcome these shortcomings, this paper proposes a multilabel feature selection method using mutual information and improved multilabel ReliefF (ML-ReliefF). First, the proportion of each label is calculated in label space and combined with the mutual information of features and labels to construct a novel correlation degree between features and label sets to preprocess multilabel datasets, which is used to reduce runtime of ML-ReliefF. Second, the mutual information of label sets is introduced into improving accuracy of the correlation degree among label sets. Furthermore, two types of correlation degree for label sets based on ML-ReliefF are developed to divide similar and heterogeneous samples more clearly. Third, a divided method of heterogeneous neighbors is presented to effectively avoid the repeated calculation in ML-ReliefF, and a novel method of feature weighting based on ML-ReliefF is constructed to evaluate the importance of features. Finally, a multilabel feature selection algorithm based on mutual information and ML-ReliefF for multilabel classification is designed to improve the performance of multilabel classification. Experiments under fourteen multilabel datasets show the effectiveness of our algorithm and improve the classification performance for multilabel datasets.

In recent years multimodal sentiment analysis (MSA) has been devoted to developing effective fusion mechanisms and has made advances, however, there are several challenges that have not been addressed adequately: the models make insufficient use of important information (inter-modal relevance and independence information) resulting in additional noise, and the traditional ternary symmetric architecture cannot well solve the problem of uneven distribution of task-related information among modalities. Thus, we propose Mutual Information Maximization and Feature Space Separation and Bi-Bimodal Modality Fusion (MFSBF)framework which effectively alleviates these problems. To alleviate the problem of underutilization of important information among modalities, a mutual information maximization module and a feature space separation module have been designed. The mutual information module maximizes the mutual information between two modalities to retain more relevance (modality-invariant) information, while the feature separation module separates fusion features to prevent the loss of independence(modality-specific) information during the fusion process. As different modalities contribute differently to the model, a bimodal fusion architecture is used, which involves the fusion of two bimodal pairs. The architecture focuses more on the modality that contains more task-ralated information and alleviates the problem of uneven distribution of useful information among modalities. The experiment results of our model on two publicly available datasets (CUM-MOSI and CUM-MOSEI) achieved better or comparable results than previous models, which demonstrate the efficacy of our method.

In this study, due to the redundant and irrelevant features contained in the multi-dimensional feature parameter set, the information fusion performance of the subspace learning algorithm was reduced. To solve the above problem, a mutual information (MI) and fractal dimension-based unsupervised feature parameters selection method was proposed. The key to this method was the importance ordering algorithm based on the comprehensive consideration of the relevance and redundancy of features, and then the method of fractal dimension-based feature parameter subset evaluation criterion was adopted to obtain the optimal feature parameter subset. To verify the validity of the proposed method, a brushless direct current (DC) motor performance degradation test was designed. Vibrational sample data during motor performance degradation was used as the data source, and motor health-fault diagnosis capacity and motor state prediction effect ware evaluation indexes to compare the information fusion performance of the subspace learning algorithm before and after the use of the proposed method. According to the comparison result, the proposed method is able to eliminate highly-redundant parameters that are less correlated to feature parameters, thereby enhancing the information fusion performance of the subspace learning algorithm.

Approximation-free running SVD and its application to motion detection

Retinopathy based techniques are preferred for diabetes detection due to their advantage of bloodless diagnosis. In concert to similar works currently going on, we propose, such a diabetes detection method using Artificial Neural Network (ANN) and a feature set formed by adopting Singular Value Decomposition (SVD) and Principal Component Analysis (PCA). A robust and computationally efficient approach for the localization of the different features in a fundus retinal image is presented in this work. The work proposes certain approaches for extraction of diabetes retinopathy features using SVD and PCA and applying the composite form to ANN for training. The detection of hemorrhages and exudates are important in order to diagnose diabetes retinopathy for preventing loss of eye sight. Experimental results show that the ANN-SVD+PNN composition is a reliable means of diabetes detection with less computational complexity and high accuracy.

An efficient data fusion model based on Bayesian model averaging for robust water quality prediction using deep learning strategies

Applications using face biometric are ubiquitous in various domains. We propose an efficient method using Discrete Wavelet Transform (DWT), Extended Directional Binary codes (EDBC), three matrix decompositions and Singular Value Decomposition (SVD) for face recognition. The combined effect of Schur, Hessenberg and QR matrix decompositions are utilized with existing algorithm. The discrimination power between two different persons is justified using Average Overall Deviation (AOD) parameter. Fused EDBC and SVD features are considered for performance calculation. City-block and Euclidean Distance (ED) measure is used for matching. Performance is improved on YALE, GTAV and ORL face databases compared with existing methods.

The use of micro‐Doppler signatures experimentally collected by a multistatic radar system to recognise and classify different people walking is discussed. A suitable feature based on singular value decomposition of the spectrograms is proposed and tested with different types of classifiers. It is shown that high accuracy of between 97 and 99% can be achieved when multistatic data are used to perform the classification.