Research on intelligent interactive music information based on visualization technology

This paper develops a wearable sport activity classification system and its associated deep learning-based sport activity classification algorithm for accurately recognizing sport activities. The proposed wearable system used two wearable inertial sensing modules worn on athletes’ wrist and ankle to collect sport motion signals and utilized a deep convolutional neural network (CNN) to extract the inherent features from the spectrograms of the short-term Fourier transform (STFT) of the sport motion signals. The wearable inertial sensing module is composed of a microcontroller, a triaxial accelerometer, a triaxial gyroscope, an RF wireless transmission module, and a power supply circuit. All ten participants wore the two wearable inertial sensing modules on their wrist and ankle to collect motion signals generated by sport activities. Subsequently, we developed a deep learning-based sport activity classification algorithm composed of sport motion signal collection, signal preprocessing, sport motion segmentation, signal normalization, spectrogram generation, image mergence/resizing, and CNN-based classification to recognize ten types of sport activities. The CNN classifier consisting of two convolutional layers, two pooling layers, a fully-connected layer, and a softmax layer can be used to divide the sport activities into table tennis, tennis, badminton, golf, batting baseball, shooting basketball, volleyball, dribbling basketball, running, and bicycling, respectively. Finally, the experimental results show that the proposed wearable sport activity classification system and its deep learning-based sport activity classification algorithm can recognize 10 sport activities with the classification rate of 99.30%.

Sparsity-based and deep learning-based image reconstruction algorithms are two promising approaches to accelerate the image acquisition process for localization-based super-resolution microscopy, by allowing a higher density of fluorescing emitters to be imaged in a single frame. Despite the surging popularity, a comprehensive parametric study guiding the practical applications of sparsity-based and deep learning-based image reconstruction algorithms is yet to be conducted. In this study, we examined the performance of sparsity- and deep learning-based algorithms in reconstructing super-resolution images using simulated fluorescent microscopy images. The simulated images were synthesized with varying levels of sparsity and connectivity. We found the deep learning-based VDSR recovers image faster, with a higher recall rate and localization accuracy. The sparsity-based SPIDER recovers more zero pixels truthfully. We also compared the two algorithms using images acquired from a real super-resolution experiment, yielding results agreeing with the results from the evaluation using simulated images. We concluded that VDSR is preferable when accurate emitter localization is needed while SPIDER is more suitable when evaluation of the number of emitters is critical.

IntroductionWe investigated the impact of ascites and peritoneal metastatic (PM) lesion volumes, measured by deep learning-based algorithm, on survival outcomes in advanced epithelial ovarian cancer.MethodsApplying our newly developed deep learning-based...

Convolutional neural networks (CNNs) have shown great advantages in computer vision fields, and loss functions are of great significance to their gradient descent algorithms. Softmax loss, a combination of cross-entropy loss and Softmax function, is the most commonly used one for CNNs. Hence, it can continuously increase the discernibility of sample features in classification tasks. Intuitively, to promote the discrimination of CNNs, the learned features are desirable when the inter-class separability and intra-class compactness are maximized simultaneously. Since Softmax loss hardly motivates this inter-class separability and intra-class compactness simultaneously and explicitly, we propose a new method to achieve this simultaneous maximization. This method minimizes the distance between features of homogeneous samples along with Softmax loss and thus improves CNNs' performance on vision-related tasks. Experiments on both visual classification and face verification datasets validate the effectiveness and advantages of our method.

As one of the indispensable components in convolutional neural network (CNN), loss function assists in updating parameters of CNN models during the training phase. Generally, different loss functions can assist convolutional neural network (CNN) to learn different feature representations, and different feature representations can be treated as different knowledge learned from objects. In this paper we introduce a novel training framework, namely Loss Transferring (LT), to improve the generalization ability of CNN. LT contains multiple training phases, and each training phase uses a different loss function. Under this framework, CNN models can combine different knowledge of objects by transferring the knowledge learned via one loss function to another. LT contains two components, i.e., loss function set and training strategy. In order to build appropriate loss function set, we establish two basic guides. And according to these basic guides, we design a new loss function in the last layer of CNN models (layer before softmax operation), namely Near Classifier Hyper-Plane (N-CHP) loss, which makes the learned object features belonging to the same category have the minimum intra-class distance and be near the classifier hyper-plane. Based on the two loss function set $\{\text {MSE, softmax}\}$ and $\{\text {N-CHP, softmax}\}$ , we setup two specific training methods, $LT_{\text {MSE, softmax}}$ and $LT_{\text {N-CHP, softmax}}$ , which can be universally applied to different CNN models with low additional computation cost. Meanwhile, two training strategies, multi-phase strategy 1 and multi-phase strategy 2, are further proposed to improve the training efficiency of LT. Extensive experimental results on shallow, moderate and deep models with four benchmark datasets, including MNIST, SVHN, CIFAR-10 and CIFAR-100, demonstrate that CNN models can bring obvious performance improvements when working with $LT_{\text {MSE, softmax}}$ and $LT_{\text {N-CHP, softmax}}$ , which verifies the effectiveness of LT and the proposed two basic guides.

In this paper, we develop a system that provides a new way to browse streaming music on mobile device. Our system visualizes music information and social interaction history for users. The main design goals of our system are twofold. First, we provide a timeline-based way to capture music information at a glance. Second, we keep social interaction history and show by visualization to release exploration. Finally, we implement our prototype and perform evaluation to show the feasibility and effectiveness.Keywordsvisualizationstreaming musicmobile device

SummaryWith the improvement of informatization and the development of computer technology, identity recognition with fingerprint has been the growing trend, but the stained fingerprint will make recognition difficult. To solve the above problem, this paper briefly introduced the traditional point matching fingerprint recognition algorithm and the damaged fingerprint recognition method based on the convolution neural network (CNN). Then in MATLAB software, the damaged fingerprint recognition method based on CNN was simulated and compared with the traditional point matching recognition method and traditional CNN recognition method. The results showed that the improved CNN recognition method iterated fewer times and achieved smaller training error during the training; the improved CNN method had lower false recognition rate and rejection rate in identifying the unknown fingerprint; the traditional point matching method spent the most time in identifying the unknown fingerprint, followed by the traditional CNN method and the improved CNN recognition method. This study used CNN for fingerprint recognition and improved CNN to improve its recognition accuracy for the damaged fingerprint. The improved CNN can effectively enhance the recognition accuracy of the damaged fingerprint, which provides a useful reference for the improvement of the accuracy and applicability of the fingerprint recognition system.

Understanding the spatial distribution of dead pine trees (DPTs) in mountainous areas is vital for diseased wood cleanup and the prediction of pine wilt disease. The study induced a deep learning (DL) model of convolutional neural networks (CNNs) for the DPT detection using unmanned aerial vehicle images. Two thousand manually labelled DPT and Non-DPT samples were collected to train (80%) and optimize (20%) the CNNs of AlexNet and GoogLeNet, and 768 samples from other three areas were used to predict the categories in Jinjiang, Fujian, southeastern China. The same dataset was used to compare the classification accuracy between CNNs and the traditional template matching (TM) method. In addition, the potential factors influencing the overall accuracy of the DL model for DPT detection were evaluated. Training results of AlexNet and GoogLeNet showed an accuracy of over 93.30% and 97.38% respectively for the top-1 test after the final training iteration. For the DPT classification in the prediction areas of pure forests, the TM method showed the 56–65% of overall accuracy while CNNs showed 65–80%. CNNs-based DPT detection was more accurate than the traditional TM method. Terrains and red broadleaf trees influence the CNNs-based DPT detection from aerial visible-images in mixed artificial forest and peaks areas.

In recent years, convolutional neural networks have been widely used in image processing and have shown good results. Particularly useful was their ability to automatically extract image features (textures and shapes of objects). The article proposes a method that improves the accuracy and speed of recognition of an ultra-precise neural network based on image recognition of people’s faces. At first, a recurrent neural network is introduced into the convolutional neural network, thereby studying the characteristics of the image more deeply. Deep image characteristics are studied in parallel using a convolutional and recurrent neural network. In line with the idea of skipping the ResNet convolution layer, a new ShortCut3- ResNet residual module is built. A double optimization model is created to fully optimize the convolution process. A study of the influence of various parameters of a convolutional neural network on network performance is demonstrated, also analyzed using simulation experiments. As a result, the optimal parameters of the convolutional neural network are established. Ex- periments show that the method presented in this paper can study various images of people’s faces regardless of age, gender, and also improves the accuracy of feature extraction and image recognition ability.

The music visualization algorithm described in this study allows users to construct piano audio files using imported image files. This paper contributes to previous studies and designs of sonification by highlighting the effectiveness of utilizing onset detection in creating intuitive sonic changes. The audio-visual correspondences employed in this study could be expanded to many other syntheses and sample manipulation techniques. Translating visual information into sonic changes could yield many creative applications in music production, as it offers musicians a simultaneously optical and auditory production experience. This approach to audio manipulation also increases the unpredictability of the sound output, which could be appealing to experimental musicians seeking to control sounds with the visual structure of artworks that they enjoy, as opposed to precise parameters. It is looking forward to seeing creative implementations of the techniques in audio-visual artworks, music production tools, and interactive multimedia systems. These results shed light on guiding further exploration of AI composing.

PurposeTo perform and evaluate water–fat signal separation of whole‐body gradient echo scans using convolutional neural networks.MethodsWhole‐body gradient echo scans of 240 subjects, each consisting of 5 bipolar echoes, were used. Reference fat fraction maps were created using a conventional method. Convolutional neural networks, more specifically 2D U‐nets, were trained using 5‐fold cross‐validation with 1 or several echoes as input, using the squared difference between the output and the reference fat fraction maps as the loss function. The outputs of the networks were assessed by the loss function, measured liver fat fractions, and visually. Training was performed using a graphics processing unit (GPU). Inference was performed using the GPU as well as a central processing unit (CPU).ResultsThe loss curves indicated convergence, and the final loss of the validation data decreased when using more echoes as input. The liver fat fractions could be estimated using only 1 echo, but results were improved by use of more echoes. Visual assessment found the quality of the outputs of the networks to be similar to the reference even when using only 1 echo, with slight improvements when using more echoes. Training a network took at most 28.6 h. Inference time of a whole‐body scan took at most 3.7 s using the GPU and 5.8 min using the CPU.ConclusionIt is possible to perform water–fat signal separation of whole‐body gradient echo scans using convolutional neural networks. Separation was possible using only 1 echo, although using more echoes improved the results.

Rapid and accurate assessment of endotracheal tube (ETT) location is essential in the intensive care unit (ICU) setting, where timely identification of a mispositioned support device may prevent significant patient morbidity and mortality. This study proposes a series of deep learning-based algorithms which together iteratively identify and localize the position of an ETT relative to the carina on chest radiographs. Using the open-source MIMIC Chest X-Ray (MIMIC-CXR) dataset, a total of 16,000 patients were identified (8000 patients with an ETT and 8000 patients without an ETT). Three different convolutional neural network (CNN) algorithms were created. First, a regression loss function CNN was trained to estimate the coordinate location of the carina, which was then used to crop the original radiograph to the distal trachea and proximal bronchi. Second, a classifier CNN was trained using the cropped inputs to determine the presence or absence of an ETT. Finally, for radiographs containing an ETT, a third regression CNN was trained to both refine the coordinate location of the carina and identify the location of the distal ETT tip. Model accuracy was assessed by comparing the absolute distance of prediction and ground-truth coordinates as well as CNN predictions relative to measurements documented in original radiologic reports. Upon five-fold cross validation, binary classification for the presence or absence of ETT demonstrated an accuracy, sensitivity, specificity, PPV, NPV, and AUC of 97.14%, 97.37%, 96.89%, 97.12%, 97.15%, and 99.58% respectively. CNN predicted coordinate location of the carina, and distal ETT tip was estimated within a median error of 0.46 cm and 0.60 cm from ground-truth annotations respectively. Overall final CNN assessment of distance between the carina and distal ETT tip was predicted within a median error of 0.60 cm from manual ground-truth annotations, and a median error of 0.66 cm from measurements documented in the original radiology reports. A serial cascaded CNN approach demonstrates high accuracy for both identification and localization of ETT tip and carina on chest radiographs. High performance of the proposed multi-step strategy is in part related to iterative refinement of coordinate localization as well as explicit image cropping which focuses algorithm attention to key anatomic regions of interest.

In order to overcome the problem of low throughput and time-consuming of traditional fingerprint minutiae matching methods, a new convolutional-deep neural network is proposed for parallel fingerprint minutiae matching. This method realises the preprocessing of the initial image through four steps: normalisation, image enhancement, parallel thinning and image segmentation. The convolutional-deep neural network is constructed from convolution kernel, convolution layer, pooling layer and full connection layer to extract minutiae of fingerprint image. Through feature minutiae matching, local matching and global matching, the matching results of fingerprint parallel nodes are obtained. The experimental results show that compared with the traditional matching method, the fingerprint matching throughput of convolutional-deep neural network is increased by 25%, and the matching time is saved by about 8 seconds.

The impact of partner interaction on brief social buffering in adolescent female rats as analyzed by deep learning-based object detection algorithms.

People who are unable to type on a computer or a mobile phone due to inadequacies produced by their hands, such as osteoarthritis, carpal tunnel syndrome, trigger finger, Ganglion cysts, and other disorders, can benefit from eye vision technology. There are currently a number of commercial and non-commercial eye-tracking solutions available, including model-based and appearance-based methods; however, some of these solutions are expensive or unreliable in real-world situations, and others require explicit user calibration, which can be time-consuming. As a result, research into deep learning-based eye-tracking systems have switched to improving these systems. Recent eye-tracking research has focused on the development of deep learning-based eye-tracking algorithms that don't require explicit user calibration. Because of the recent emergence of deep learning, gaze estimation models based on convolutional neural networks (CNNs) are becoming more significant and common. In our research, the proposed system will provide a new user -friendly keyboard interface for the user. User can see the keyboard layout and can type the text by the movement of his or her eyes. The user can enhance his typing rate by making use of a word prediction engine. Word prediction is an assistive technology tool that suggests words while typing.