A Multi-Task Learning CNN for Image Steganalysis

Facial expression recognition and virtual animation character generation are crucial for animation production and human-computer interaction, but traditional models often perform poorly in complex scenes. This paper proposes a novel expression recognition and generation framework based on an improved Multi-Task Convolutional Neural Network (MTCNN), augmented by a High-Resolution Parallel Convolutional Network (HR-PCN) and Octave Convolution (OctConv). Specifically, HR-PCN enhances multi-scale feature extraction for facial keypoint detection, while OctConv improves frequency-aware representation learning. In terms of facial expression generation, Graph Convolutional Networks (GCNs) are adopted to model the semantic relationships between facial Action Units (AUs) and further enhanced with SE-ResNet50 for better spatial attention. The proposed MTCNN model was evaluated on the AFEW and CK+ datasets, achieving 89.70% and 93.50% accuracies, surpassing MTCNN’s 78.90% and 85.30% and SSD’s 85.40% and 90.10%. RMSE was reduced to 0.1 after 30 iterations, and inference time was kept within 40 ms/frame. For expression generation, the SE-ResNet50-GCN model attained a generation accuracy of up to 93.5%, significantly outperforming ResNet50-GCN (90.8%) and GCN (80.2%). These results validate the proposed framework’s effectiveness in improving both recognition accuracy and expression realism under complex conditions.

Accurate fault information is critical for optimal scheduling of production activities, improving system reliability, and reducing operation and maintenance costs. In recent years, many fault diagnosis methods for rolling element bearings have been developed based on deep learning. Most of them are totally data-driven and do not consider the domain knowledge that has been used in fault diagnosis for years. Meanwhile, operating conditions such as rotating speed and load that have great influence on vibration signals are also ignored. It may cause a decrease in accuracy when the bearing type or operating condition changes. To address these problems, this article proposes a rolling element bearing fault diagnosis and localization approach based on multitask convolutional neural network (CNN) with information fusion. In the proposed approach, domain knowledge, operating conditions, and vibration signals are fused into a three-dimensional input that can be processed well by CNN. Then, a multitask CNN with dynamic training rates is constructed to simultaneously accomplish two tasks, fault diagnosis and localization. Experimental results on two rolling element bearing test beds with different bearing types and operating conditions are presented and compared with existing state-of-the-art approaches to demonstrate the effectiveness and accuracy of the proposed approach.

Multi-task learning in Convolutional Neural Networks (CNNs) has led to remarkable success in a variety of applications of computer vision. Towards effective multi-task CNN architectures, recent studies automatically learn the optimal combinations of task-specific features at single network layers. However, they generally learn an unchanged operation of feature combination after training, regardless of the characteristic changes of task-specific features across different inputs. In this paper, we propose a novel Adaptive Feature Aggregation (AFA) layer for multi-task CNNs, in which a dynamic aggregation mechanism is designed to allow each task adaptively determines the degree to which the knowledge sharing or preserving between tasks is needed based on the characteristics of inputs. We introduce two types of aggregation modules to the AFA layer, which realize the adaptive feature aggregation by capturing the feature dependencies of different tasks along the channel and spatial axes, respectively. The AFA layer is a plug-and-play component with low parameter and computation overheads, and can be trained end-to-end along with backbone networks. For both pixel-level and image-level tasks, we empirically show that our approach substantially outperforms the previous state-of-the-art methods of multi-task CNNs. The code and models are available at https://github.com/zhenshen-mla/AFANet.

No-reference stereoscopic image quality assessment using a multi-task CNN and registered distortion representation

Recently, automatic vehicle identification system has great significance in practical life and with modern technology now widely available, the research on automatic vehicle recognition based on computer vision is becoming increasingly thorough and extensive. However, automatic vehicle identification also faces significant challenges, such as changes in lighting, weather conditions and distortion of captured images in natural road scenes. Since 2012, as a deep learning in image classification, the convolution neural network has become a major subject in many fields, so this study would focus on the vehicle identification in terms of the convolutional neural network. The Cars196 data set and an improved Convolutional Neural Network (CNN) network capable of multi-attribute recognition are applied to study the optimization network problem of the convolutional neutral network and to realize the multi-attribute classification and identification of automobile brands, models and colors. The data analysis shows that the improved CNN recognition network has a recognition accuracy in automatic vehicle recognition, but the overall recognition network is affected by many factors. According to the results of data analysis, in practice, alongside the improvement of traditional convolutional neural network, the improved new neural network structure should also be used for vehicle recognition, so as to improve the accuracy of vehicle identification to better serve the community.

BackgroundSimultaneous dual-tracer positron emission tomography (PET) imaging can observe two molecular targets in a single scan, which is conducive to disease diagnosis and tracking. Since the signals emitted by different tracers are the same, it is crucial to separate each single tracer from the mixed signals. The current study proposed a novel deep learning-based method to reconstruct single-tracer activity distributions from the dual-tracer sinogram.MethodsWe proposed the Multi-task CNN, a three-dimensional convolutional neural network (CNN) based on a framework of multi-task learning. One common encoder extracted features from the dual-tracer dynamic sinogram, followed by two distinct and parallel decoders which reconstructed the single-tracer dynamic images of two tracers separately. The model was evaluated by mean squared error (MSE), multiscale structural similarity (MS-SSIM) index and peak signal-to-noise ratio (PSNR) on simulated data and real animal data, and compared to the filtered back-projection method based on deep learning (FBP-CNN).ResultsIn the simulation experiments, the Multi-task CNN reconstructed single-tracer images with lower MSE, higher MS-SSIM and PSNR than FBP-CNN, and was more robust to the changes in individual difference, tracer combination and scanning protocol. In the experiment of rats with an orthotopic xenograft glioma model, the Multi-task CNN reconstructions also showed higher qualities than FBP-CNN reconstructions.ConclusionsThe proposed Multi-task CNN could effectively reconstruct the dynamic activity images of two single tracers from the dual-tracer dynamic sinogram, which was potential in the direct reconstruction for real simultaneous dual-tracer PET imaging data in future.

The Part 1 prequel to this review evaluated the evolution of modelling techniques used in evaluation of fruit quality over the past three decades and noted a progression towards the use of artificial neural networks (ANNs) and convolutional neural networks (CNNs). In this review, Part 2, the use of CNNs for NIR fruit quality evaluation is explored, given the success of CNNs in various other fields, such as image, video, speech, and audio processing, and the availability of large (open source) datasets of fruit spectra and reference quality attribute, which is required for the training of CNN models. The review provides an overview of deep learning and the CNN architectures and techniques used in NIR spectroscopy for regression modelling, with advantages and disadvantages identified. Studies using CNN for NIR based fruit quality evaluation are then critically examined. Eight publications have presented on models using the same open-source mango dry matter calibration and test set, enabling inter-method comparisons. CNN models have been demonstrated to be accurate, precise and robust. Techniques of transfer learning for CNN models offer an alternative solution to model updating and calibration transfer methods applied in traditional chemometrics. The review has highlighted crucial areas that require resolution and exploration in this application through future research, including, (i) data requirements for training a CNN (ii) optimal spectral pre-processing for CNN (iii) CNN architecture and hyper-parameter selection and tuning for fruit quality evaluation (iv) CNN model interpretability and explainability. Future studies must conduct clearer comparison to partial least squares (PLS) regression and shallow ANNs to better assess the prospective benefit of using CNN, a more complex model. The potential for visualisation of spectra relevance to the CNN model using techniques such as GradCam, currently employed in visualising 2D-CNN models, remains to be explored.

Microscopic analysis of breast tissues is necessary for a definitive diagnosis of breast cancer which is the most common cancer among women. Pathology examination requires time consuming scanning through tissue images under different magnification levels to find clinical assessment clues to produce correct diagnoses. Advances in digital imaging techniques offers assessment of pathology images using computer vision and machine learning methods which could automate some of the tasks in the diagnostic pathology workflow. Such automation could be beneficial to obtain fast and precise quantification, reduce observer variability, and increase objectivity. In this work, we propose to classify breast cancer histopathology images independent of their magnifications using convolutional neural networks (CNNs). We propose two different architectures; single task CNN is used to predict malignancy and multi-task CNN is used to predict both malignancy and image magnification level simultaneously. Evaluations and comparisons with previous results are carried out on BreaKHis dataset. Experimental results show that our magnification independent CNN approach improved the performance of magnification specific model. Our results in this limited set of training data are comparable with previous state-of-the-art results obtained by hand-crafted features. However, unlike previous methods, our approach has potential to directly benefit from additional training data, and such additional data could be captured with same or different magnification levels than previous data.

ObjectiveWe implement 2 different multitask learning (MTL) techniques, hard parameter sharing and cross-stitch, to train a word-level convolutional neural network (CNN) specifically designed for automatic extraction of cancer data from unstructured text in pathology reports. We show the importance of learning related information extraction (IE) tasks leveraging shared representations across the tasks to achieve state-of-the-art performance in classification accuracy and computational efficiency.Materials and MethodsMultitask CNN (MTCNN) attempts to tackle document information extraction by learning to extract multiple key cancer characteristics simultaneously. We trained our MTCNN to perform 5 information extraction tasks: (1) primary cancer site (65 classes), (2) laterality (4 classes), (3) behavior (3 classes), (4) histological type (63 classes), and (5) histological grade (5 classes). We evaluated the performance on a corpus of 95 231 pathology documents (71 223 unique tumors) obtained from the Louisiana Tumor Registry. We compared the performance of the MTCNN models against single-task CNN models and 2 traditional machine learning approaches, namely support vector machine (SVM) and random forest classifier (RFC).ResultsMTCNNs offered superior performance across all 5 tasks in terms of classification accuracy as compared with the other machine learning models. Based on retrospective evaluation, the hard parameter sharing and cross-stitch MTCNN models correctly classified 59.04% and 57.93% of the pathology reports respectively across all 5 tasks. The baseline models achieved 53.68% (CNN), 46.37% (RFC), and 36.75% (SVM). Based on prospective evaluation, the percentages of correctly classified cases across the 5 tasks were 60.11% (hard parameter sharing), 58.13% (cross-stitch), 51.30% (single-task CNN), 42.07% (RFC), and 35.16% (SVM). Moreover, hard parameter sharing MTCNNs outperformed the other models in computational efficiency by using about the same number of trainable parameters as a single-task CNN.ConclusionsThe hard parameter sharing MTCNN offers superior classification accuracy for automated coding support of pathology documents across a wide range of cancers and multiple information extraction tasks while maintaining similar training and inference time as those of a single task–specific model.

CAPTCHA serves as a vital tool in distinguishing between human users and automated bots attempting to access websites. The Turing test, a fundamental concept in this domain, aids in discerning robot involvement in web security breaches, thereby safeguarding against automated access and potential harm. CAPTCHA, encapsulated by the acronym Completely Automated Public Turing Test to Tell Computers and Humans Apart, is instrumental in preventing undesirable activities by posing tasks that humans find simple to solve yet prove exceedingly challenging for robots, representing a distinct category of challenges. Initiatives aimed at improving CAPTCHA systems have led to the development of models aimed at accurately recognizing characters within CAPTCHA images. While traditional methods require the segmentation of characters before recognition, proposed models eliminate this step by processing the entire image at once, resulting in improved accuracy. Convolutional Neural Networks (CNNs) exhibit enhanced accuracy with segmented characters, while Multi-Task Convolutional Neural Network (MTCNN) excels in achieving similar accuracy without pre-processing. Object detection algorithms, including Faster R-CNN, YOLO, and SSD, offer even greater potential for breaking CAPTCHA by detecting objects within images. Gesture-based CAPTCHA challenges, while promising, encounter usability issues related to precision, reaction speed, and perceived level of challenge. To address this, a novel approach is proposed, leveraging hand-based gestures that are easily solvable by humans yet challenging for robots to replicate. Additionally, dynamic game-based CAPTCHA designs offer an aesthetically appealing and engaging interface, potentially motivating users to solve CAPTCHA challenges with minimal annoyance. The objective of this study is to explore the influence of different CAPTCHA tests on user experience across diverse populations. It includes a comprehensive study of multitask learning convolutional neural networks and employs methods of object detection algorithms, including Faster R-CNN, YOLO, and SSD object detection for CAPTCHA character recognition. The research also encompasses the design of gesture-based and dynamic game-based CAPTCHA challenges and compares various deep learning CAPTCHA breaking techniques with SSD object detection methods, analyzing existing and designed CAPTCHAs on multiple parameters.

This article introduces an innovative approach for condition monitoring of rotating instruments using a piezoelectric sensor array and a multi-task convolutional neural network (CNN) framework. The method adopts a multi-input-single-output (MISO) diagnostic strategy to detect faults in bearings and gearboxes, even in the presence of multiple defects across different locations. The sensor array, composed of piezoelectric patches arranged in a mixed parallel-series configuration, enables spatially distributed monitoring through a single output voltage signal. To enhance voltage sensing, inductors are integrated with the array to form LC resonant circuits. A CNN classifier is initially designed to identify 12 combined health states, comprising four bearing states and three gear states, and achieves high accuracy under continuous machine operation. However, the model exhibits vulnerability to noise during machine restarts and struggles with data sparsity as the number of defect combinations increases. To address these limitations, a multi-task CNN architecture is proposed. It shares convolutional layers while employing separate fully connected layers for gear and bearing classification. This structure simplifies the classification task and improves generalization for spatial condition monitoring. Experiments show that multi-task learning achieves an overall accuracy of 93%, with at least 84% accuracy in the worst conditions, outperforming the 49% worst-case accuracy of single-task learning. This demonstrates the effectiveness of multi-task learning in multi-location defect monitoring using piezoelectric arrays.

MP34-06 MACHINE LEARNING USING A MULTI-TASK CONVOLUTIONAL NEURAL NETWORKS CAN ACCURATELY ASSESS ROBOTIC SKILLS

Deep learning is one of the most popular technologies and research areas in machine learning. Convolutional Neural Networks (CNNs) are a typical artificial neural network underpinning deep learning. They have been used in many fields including image recognition, natural language process, and through games such as AlphaGo. A CNN has many advantages such as efficient feature extraction, the simplicity of data format required and the small number of (hyper-)parameters that are required. This paper focuses on a particular application of deep learning: crowd counting. To address this, we apply a Multi-task, Cascaded Convolutional Neural Network (MTCNN). Compared to other models, this model has a good performance and requires a shorter inference time, with shallower network structure and smaller size. In order to demonstrate the value and feasibility of the technology and provide a friendly operating environment for users, the application was realised on both the iOS and Android platforms. A web platform was also developed to visualize the real-time data using a Firebase server.

In order to solve the problem of poor real-time performance and low accuracy of a single detection target of common driver fatigue driving detection method based on facial features in practical applications, a fatigue driving detection method based on lightweight convolutional neural network is proposed. First, the driver's facial feature point data set is made through MTCNN (multi task convolutional neural network). Then the data set is used to train a lightweight convolutional neural network to detect the accurate feature point position of the eyes and mouth. Finally, the open and close state of the driver's eyes and mouth is judged based on the feature point coordinates. According to the open and closed state of the eyes and mouth of the continuous multi-frame image, the driver is judged to be in the state of fatigue. The experimental results show that the processing time of the single frame image by the algorithm is 23.3 millisecond; the single detection accuracy is up to 99.4%, and the detection accuracy of fatigue driving can reach 95%. The algorithm is better real-time performance and higher accuracy, so it has certain engineering significance and application prospects.

The COVID-19 can be transmitted by air droplets, aerosols, and other carriers, the spread of the virus can be effectively prevented by wearing masks in public. Therefore, it is meaningful to identify whether a mask is worn in particular places. In this paper, a method based on multi-task convolutional neural networks (MTCNN) and MobileNet algorithms is proposed to implement mask recognition on human face. Firstly, MTCNN is used to detect facial contours. Then the output image is used to train MobileNet model. By comparing the extracted facial feature data, the human with mask or not can be marked. The method has been tested in a 1.8 GHz Intel Core machine with 160 × 160 static images. Average accuracy rate of 94.73% and detection speed of 1.9 s are achieved.