A Light Weight Model for Video Shot Occlusion Detection

The objective of the study was to enhance quality education in the traditional pre-school piano education. Deep Learning (DL) technology is applied to piano education of children to improve their interest in learning music. Firstly, the problems of the traditional piano education of children were analyzed with the teaching patterns discussed under educational psychology, and a targeted music education plan was established. Secondly, musical instrument recognition technology was introduced, and the musical instrument recognition model was implemented based on DL. Thirdly, the proposed model was applied to the piano education of children to guide the music learning of students and improve their interest in piano learning. The feature recognition and acquisition of the proposed model were improved. Finally, the different teaching patterns were comparatively analyzed through the Questionnaire Survey (QS). The experimental results showed that the instrument recognition accuracy of Hybrid Neural Network (HNN) is 97.2%, and with the increase of iterations, the recognition error rate of the model decreases and stabilizes. Therefore, the proposed HNN based on DL for musical instrument recognition can accurately identify musical features. The QS results showed that the introduction of musical instrument recognition technology in the piano education of children can improve their interest in piano learning. Therefore, the establishment of the piano education patterns based on the piano education model can improve the effectiveness of teaching piano to students. This research provides a reference for the intelligentization of children's piano education.

This paper proposes a novel occlusion detection method for urban true orthophoto generation. In this new method, occlusion detection is performed using a ghost image; this method is therefore considerably different from the traditional Z-buffer method, in which occlusion detection is performed during the generation of a true orthophoto (to avoid ghost image occurrence). In the proposed method, a model is first established that describes the relationship between each ghost image and the boundary of the corresponding building occlusion, and then an algorithm is applied to identify the occluded areas in the ghost images using the building displacements. This theory has not previously been applied in true orthophoto generation. The experimental results demonstrate that the method proposed in this paper is capable of effectively avoiding pseudo-occlusion detection, with a success rate of 99.2%, and offers improved occlusion detection accuracy compared with the traditional Z-buffer detection method. The advantage of this method is that it avoids the shortcoming of performing occlusion detection and true orthophoto generation simultaneously, which results in false visibility and false occlusions; instead, the proposed method detects occlusions from ghost images and therefore provides simple and effective true orthophoto generation.

Unmanned aerial vehicles (UAV) and related technologies have played an active role in the prevention and control of novel coronaviruses at home and abroad, especially in epidemic prevention, surveillance, and elimination. However, the existing UAVs have a single function, limited processing capacity, and poor interaction. To overcome these shortcomings, we designed an intelligent anti-epidemic patrol detection and warning flight system, which integrates UAV autonomous navigation, deep learning, intelligent voice, and other technologies. Based on the convolution neural network and deep learning technology, the system possesses a crowd density detection method and a face mask detection method, which can detect the position of dense crowds. Intelligent voice alarm technology was used to achieve an intelligent alarm system for abnormal situations, such as crowd-gathering areas and people without masks, and to carry out intelligent dissemination of epidemic prevention policies, which provides a powerful technical means for epidemic prevention and delaying their spread. To verify the superiority and feasibility of the system, high-precision online analysis was carried out for the crowd in the inspection area, and pedestrians’ faces were detected on the ground to identify whether they were wearing a mask. The experimental results show that the mean absolute error (MAE) of the crowd density detection was less than 8.4, and the mean average precision (mAP) of face mask detection was 61.42%. The system can provide convenient and accurate evaluation information for decision-makers and meets the requirements of real-time and accurate detection.

Basketball shooting angle calculation and analysis by deeply-learned vision model

In view of the rapid growth of municipal solid waste (MSW), a large number of MSW are transported to the outside of the city for landfill or incineration, only part of the MSW is treated innocuously, the speed of MSW treatment is slow, and the level of garbage classification intelligence is low, this paper proposes an algorithm of MSW classification and recycling based on deep learning technology, and uses convolution neural network to build garbage intelligence simultaneous interpreting and classification algorithm, which improves the accuracy and speed of garbage image recognition. The algorithm is compared with the traditional BP neural network algorithm. The simulation results show that the algorithm is 30% faster than the traditional algorithm. The classification algorithm has faster response speed, higher accuracy and stronger robustness.

Videos have become the new preference comparing with images in recent years. However, during the recording of videos, the cameras are inevitably occluded by some objects or persons that pass through the cameras, which would highly increase the workload of video editors for searching out such occlusions. In this paper, for releasing the burden of video editors, a frame-level video occlusion detection method is proposed, which is a fundamental component of automatic video editing. The proposed method enhances the extraction of spatial-temporal information based on C3D yet only using around half amount of parameters, with an occlusion correction algorithm for correcting the prediction results. In addition, a novel loss function is proposed to better extract the characterization of occlusion and improve the detection performance. For performance evaluation, this paper builds a new large scale dataset, containing 1,000 video segments from seven different real-world scenarios, which could be available at: https://junhua-liao.github.io/Occlusion-Detection/. All occlusions in video segments are annotated frame by frame with bounding-boxes so that the dataset could be utilized in both frame-level occlusion detection and precise occlusion location. The experimental results illustrate that the proposed method could achieve good performance on video occlusion detection compared with the state-of-the-art approaches. To the best of our knowledge, this is the first study which focuses on occlusion detection for automatic video editing.

Pavement distress detection is of great significance to pavement maintenance and management. Currently, image-processing methods have been developed, especially with the widespread use of the convolutional neural networks (CNN). However, most CNN applications are still limited to the general object detection model (GODM), which is difficult to adapt to the detection of complex road scenes. In this study, a dedicated object detection model, RoadID, was designed based on the CNN to detect multiple pavement distresses in a complex environment. In terms of training data, a natural pavement distress data set was established, in which eight pavement distress types—distress, crack, patch-crack, net-crack, patch-net, pothole, patch-pothole, manhole, and hinged-joint—were annotated. More than 60,000 images and 140,000 distresses are described in the data set. Various data augmentation methods, such as motion blur, frequency noise, and random rotation, were used to expand the original data set about 2.5 times. Resnet152 was chosen as the basic feature extraction network and pretrained with manhole images, which promoted the expression ability of the model and the convergence efficiency of training. To improve the imbalance of positive and negative samples, focal loss and mean squared error were combined as loss functions for training. In the prediction stage, modular design was adapted for each distress category for convenience of the expansion of other detection categories, enhancing the scalability of RoadID. Road markings and shadows were added to the data training as disturbances to improve the generalization performance of the model in complex scenes. The mean accuracy of the model reached 83.5%. Compared with the GODM, RoadID has advantages in both recognition accuracy and prediction speed. The proposed method can be used for rapid investigation of pavement conditions to support maintenance and repair.

With recent technological advancements, quantitative analysis has become an increasingly important area within professional sports. However, the manual process of collecting data on relevant match events like passes, goals and tacklings comes with considerable costs and limited consistency across providers, affecting both research and practice. In football, while automatic detection of events from positional data of the players and the ball could alleviate these issues, it is not entirely clear what accuracy current state-of-the-art methods realistically achieve because there is a lack of high-quality validations on realistic and diverse data sets. This paper adds context to existing research by validating a two-step rule-based pass and shot detection algorithm on four different data sets using a comprehensive validation routine that accounts for the temporal, hierarchical and imbalanced nature of the task. Our evaluation shows that pass and shot detection performance is highly dependent on the specifics of the data set. In accordance with previous studies, we achieve F-scores of up to 0.92 for passes, but only when there is an inherent dependency between event and positional data. We find a significantly lower accuracy with F-scores of 0.71 for passes and 0.65 for shots if event and positional data are independent. This result, together with a critical evaluation of existing methodologies, suggests that the accuracy of current football event detection algorithms operating on positional data is currently overestimated. Further analysis reveals that the temporal extraction of passes and shots from positional data poses the main challenge for rule-based approaches. Our results further indicate that the classification of plays into shots and passes is a relatively straightforward task, achieving F-scores between 0.83 to 0.91 ro rule-based classifiers and up to 0.95 for machine learning classifiers. We show that there exist simple classifiers that accurately differentiate shots from passes in different data sets using a low number of human-understandable rules. Operating on basic spatial features, our classifiers provide a simple, objective event definition that can be used as a foundation for more reliable event-based match analysis.

Data mining is a technique to discover useful patterns or patterns of special interest as explicit knowledge from a vast quantity of data. In video editing, there are a lot of editing patterns. According to the editor's preferences, different editing patterns give the opportunity to achieve a variety of effects. Discovering the editing patterns is required, because it is useful to find each editor's skills and to use them for editing new video material. In this paper, we propose the methods of extracting editing rules from video stream by introducing data mining technique. We can edit a video material by applying the extracted rules. The edited video may produce the same quality as the video from which we extracted the patterns.

Deep learning technology has been widely adopted in the research of automatic arrhythmia detection. However, there are several limitations in existing diagnostic models, e.g., difficulties in extracting temporal information from long-term ECG signals, a plethora of parameters, and sluggish operation speed. Additionally, the diagnosis performance of arrhythmia is prone to mistakes from signal noise. This paper proposes a smartphone-based m-health system for arrhythmia diagnosis. First, we design a cycle-GAN-based ECG denoising model which takes real-world noise signals as input and aims to produce clean ECG signals. In order to train its two generators and two discriminators simultaneously, we explore an unsupervised pre-training strategy to initialize the generator and accelerate the convergence speed during training. Second, we propose an arrhythmia diagnosis model based on the time convolution network (TCN). This model can identify 34 common arrhythmia events using eight-lead ECG signals, and we deploy such a model on the Android platform to develop an at-home ECG monitoring system. Experimental results have demonstrated that our approach outperforms the existing noise reduction methods and arrhythmia diagnosis models in terms of denoising effect, recognition accuracy, model size, and operation speed, making it more suitable for deployment on mobile devices for m-health monitoring services.

Simple SummaryThe accurate identification of individual animals is an important step in precision breeding. However, the traditional identification methods have large defects. With the continuous development of computer vision and deep learning technologies, it is possible to establish accurate biological recognition models. In this study, we built a lightweight sheep face recognition model using convolutional neural networks (CNNs) to face the challenges in sheep identity recognition. The model not only achieved high accuracy in recognition tasks but also was friendly for edge devices. The research results indicate that the proposed recognition model can identify sheep accurately and can be further deployed on the mobile end.The accurate identification of sheep is crucial for breeding, behavioral research, food quality tracking, and disease prevention on modern farms. As a result of the time-consuming, expensive, and unreliable problems of traditional sheep-identification methods, relevant studies have built sheep face recognition models to recognize sheep through facial images. However, the existing sheep face recognition models face problems such as high computational costs, large model sizes, and weak practicality. In response to the above issues, this study proposes a lightweight sheep face recognition model named LSR-YOLO. Specifically, the ShuffleNetv2 module and Ghost module were used to replace the feature extraction module in the backbone and neck of YOLOv5s to reduce floating-point operations per second (FLOPs) and parameters. In addition, the coordinated attention (CA) module was introduced into the backbone to suppress non-critical information and improve the feature extraction ability of the recognition model. We collected facial images of 63 small-tailed Han sheep to construct a sheep face dataset and further evaluate the proposed method. Compared to YOLOv5s, the FLOPs and parameters of LSR-YOLO decreased by 25.5% and 33.4%, respectively. LSR-YOLO achieved the best performance on the sheep face dataset, and the mAP@0.5 reached 97.8% when the model size was only 9.5 MB. The experimental results show that LSR-YOLO has significant advantages in recognition accuracy and model size. Finally, we integrated LSR-YOLO into mobile devices and further developed a recognition system to achieve real-time recognition. The results show that LSR-YOLO is an effective method for identifying sheep. The method has high recognition accuracy and fast recognition speed, which gives it a high application value in mobile recognition and welfare breeding.

Textile-based triboelectric nanogenerator (T-TENG) devices, particularly, narrow-gap mode, have been conceived and developed for obtaining energy harvesting and tactile sensing devices unaffected by the external environment. Enhancing the interfacial area of T-TENG materials offers exciting opportunities to improve the device output performance. In this work, a narrow-gap T-TENG was fabricated with a facile process, and a new strategy for improving the device output is proposed. The new structural sensor (polydimethylsiloxane (PDMS)-encapsulated electroless copper plating (EP-Cu) cotton) with multiple electricity generation mechanism was designed and fabricated for enhancing recognition accuracy. The result shows that only PDMS layer strain was established at an external stress of 1.24-12.4 kPa and the fibers laterally slip at a stress of 12.4-139 kPa; more importantly, the output performance of the TENG displayed a linear relationship under corresponding stress ranges. The as-fabricated device demonstrated the ability to convert different energies such as vibration, raindrops, wind and human motions into electrical energy with excellent sensitivity. Interestingly, the output signal of the as-fabricated TENG device is a combination of output signals from PDMS/EP-Cu and PDMS/recognition object devices. To be precise, there are two TENG devices (PDMS/EP-Cu and PDMS/recognition object) that work when the as-fabricated TENG device is under 12.4-139 kPa stress. Accompanied by unique characteristics, the generated TENG signals are capable of recognition of contact materials. Combining the TENG signal and deep learning technology, we explore a strategy that can enable the as-fabricated device to recognize 8 different materials with 99.48% recognition accuracy in the natural environment.

In the big data era, it is a great challenge to identify high-level abstract features out of a flood of sci-tech literature to achieve in-depth analysis of data. The deep learning technology has developed rapidly and achieved applications in many fields, but has rarely been utilized in the research of sci-tech literature data. This article introduced the presentation method of vector space of terminologies in sci-tech literature based on the deep learning model. It explored and adopted a deep AE model to reduce the dimensionality of input word vector feature. Also put forward is the methodology of correlation analysis of sci-tech literature based on deep learning technology. The experimental results showed that the processing of sci-tech literature data could be simplified into the computation of vectors in the multi-dimensional vector space, and the similarity in vector space could be used to represent similarity in text semantics. The correlation analysis of subject contents between sci-tech literatures of the same or different types can be made using this method.

In this paper, a novel occlusion invariant face recognition algorithm based on Mean based weight matrix (MBWM) technique is proposed. The proposed algorithm is composed of two phases—the occlusion detection phase and the MBWM based face recognition phase. A feature based approach is used to effectively detect partial occlusions for a given input face image. The input face image is first divided into a finite number of disjointed local patches, and features are extracted for each patch, and the occlusion present is detected. Features obtained from the corresponding occlusion-free patches of training images are used for face image recognition. The SVM classifier is used for occlusion detection for each patch. In the recognition phase, the MBWM bases of occlusion-free image patches are used for face recognition. Euclidean nearest neighbour rule is applied for the matching. GTAV face database that includes many occluded face images by sunglasses and hand are used for the experiment. The experimental results demonstrate that the proposed local patch-based occlusion detection technique works well and the MBWM based method shows superior performance to other conventional approaches.

An effective multi-layer occlusion algorithm is put forward, which is composed of basic point to point occlusion detecting algorithm, status combining algorithm, effective feature point detecting algorithm and continuity optimizing algorithm. The status combining algorithm is designed to increase occlusion detecting algorithm's robustness according to the property that the display status of adjacent points is spatio-temporal piecewise-constant. By selecting virtual edge points with obvious feature on mesh contour to do status detecting and combining computation, higher occlusion detecting accuracy can be gained. By applying continuity optimizing algorithm on virtual points' display statuses, the rendering quality can be improved much. Experiment result proved that this algorithm can realize reliable multi-layer occlusion detection in augmented reality system.