Effective Rating Prediction Using an Attention-Based User Review Sentiment Model

In recent years, with the development of deep learning, machine translation using neural network has gradually become the mainstream method in industry and academia. The existing Chinese-English machine translation models generally adopt the deep neural network architecture based on attention mechanism. However, it is still a challenging problem to model short and long sequences simultaneously. Therefore, a bidirectional LSTM model integrating attention mechanism is proposed. Firstly, by using the word vector as the input data of the translation model, the linguistic symbols used in the translation process are mathematized. Secondly, two attention mechanisms are designed: local attention mechanism and global attention mechanism. The local attention mechanism is mainly used to learn which words or phrases in the input sequence are more important for modeling, while the global attention mechanism is used to learn which layer of expression vector in the input sequence is more critical. Bidirectional LSTM can better fuse the feature information in the input sequence, while bidirectional LSTM with attention mechanism can simultaneously model short and long sequences. The experimental results show that compared with many existing translation models, the bidirectional LSTM model with attention mechanism can effectively improve the quality of machine translation.

Neighbor importance-aware graph collaborative filtering for item recommendation

We propose a novel method to improve the prediction accuracy on the rating prediction task by correcting the bias of user ratings. We demonstrate that the manner of user rating and review is biased and that it is necessary to correct this difference for more accurate prediction. Our proposed method comprises approaches based on the detection of each user value to ratings: The bias of the rating is detected using entropy of user rating and by updating word weights only when the words appear in the review, the problem of bias is reduced. We implement this idea by extending the Prank algorithm. We apply a review -- item matrix as a feature matrix instead of a user -- item matrix because of its volume of information. Our quantitative evaluation shows that our method improves the prediction accuracy (the Rank Loss measurement) significantly by 8.70 % compared with the normal Prank algorithm. Our proposed method helps users find out what they care about when buying something, and is applicable to newer variants of the Prank algorithm. Moreover, it is useful to most review sites because we use only rating and review data.

A new hybrid algorithm model for prediction of internal corrosion rate of multiphase pipeline

Heterogeneous graph embedding by aggregating meta-path and meta-structure through attention mechanism

The reliable operation of scroll compressors is crucial for the efficiency of rotating machinery and refrigeration systems. To address the need for efficient and accurate fault diagnosis in scroll compressor technology under varying operating states, diverse failure modes, and different operating conditions, a multi-branch convolutional neural network fault diagnosis method (SSG-Net) has been developed. This method is based on the Swin Transformer, the Global Attention Mechanism (GAM), and the ResNet architecture. Initially, the one-dimensional time-series signal is converted into a two-dimensional image using the Short-Time Fourier Transform, thereby enriching the feature set for deep learning analysis. Subsequently, the method integrates the window attention mechanism of the Swin Transformer, the 2D convolution of GAM attention, and the shallow ResNet's two-dimensional convolution feature extraction branch network. This integration further optimizes the feature extraction process, enhancing the accuracy of fault feature recognition and sensitivity to data variability. Consequently, by combining the global and local features extracted from these three branch networks, the model significantly improves feature representation capability and robustness. Finally, experimental results on scroll compressor datasets and the CWRU dataset demonstrate diagnostic accuracies of 97.44% and 99.78%, respectively. These results surpass existing comparative models and confirm the model's superior recognition precision and rapid convergence capabilities in complex fault environments.

Federated recommender systems have distinct advantages in terms of privacy protection over traditional recommender systems that are centralized at a data center. However, previous work on federated recommender systems does not fully consider the limitations of storage, RAM, energy and communication bandwidth in a mobile environment. The scales of the models proposed are too large to be easily run on mobile devices. And existing federated recommender systems need to fine-tune recommendation models on each device, making it hard to effectively exploit collaborative filtering information among users/devices. Our goal in this paper is to design a novel federated learning framework for rating prediction (RP) for mobile environments. We introduce a federated matrix factorization (MF) framework, named meta matrix factorization (MetaMF). Given a user, we first obtain a collaborative vector by collecting useful information with a collaborative memory module. Then, we employ a meta recommender module to generate private item embeddings and a RP model based on the collaborative vector in the server. To address the challenge of generating a large number of high-dimensional item embeddings, we devise a rise-dimensional generation strategy that first generates a low-dimensional item embedding matrix and a rise-dimensional matrix, and then multiply them to obtain high-dimensional embeddings. We use the generated model to produce private RPs for the given user on her device. MetaMF shows a high capacity even with a small RP model, which can adapt to the limitations of a mobile environment. We conduct extensive experiments on four benchmark datasets to compare MetaMF with existing MF methods and find that MetaMF can achieve competitive performance. Moreover, we find MetaMF achieves higher RP performance over existing federated methods by better exploiting collaborative filtering among users/devices.

Production defects caused by irresistible factors such as process design problems or differences in steel properties in strip production affect the economic benefits of the enterprise and threaten production safety. Traditional defect detection methods are difficult to achieve real-time and high-precision detection, so developing surface defect detection methods based on deep learning is of great significance for strip production. In order to effectively improve the accuracy of the deep learning model in detecting surface defects on hot-rolled strip, in this work we propose a real-time detection model for surface defects on strip steel based on the YOLOv8n model. Firstly, the newly convolutional layer Con5v is designed to replace the original convolutional layer in the neck, and an attention mechanism is added in front of each Con5v to improve the algorithm’s ability to extract small target information. Secondly, an additional set of upsampled feature extraction units is added to the neck in order to enhance the spatial information of the feature map. Subsequently, a set of feature fusion units is incorporated and the convolutional layers in it are improved to provide better feature maps. Thirdly, the number of decoupling detection heads is increased to receive more high-quality features. The final experimental results show that YOLOv8n-GAM (YOLOv8 Nano Model with Global Attention Mechanism) achieves 81.4mAP and 82.0FPS on the NEU-DET dataset and 71.2mAP and 55.0FPS on the GC10-DET dataset, which are 5.7% and 6.9% higher than those of YOLOv8n, respectively. The model proposed in this paper achieves a comprehensive performance improvement in strip steel.

As a novel type of automotive signal light, the running light is gradually being used in automobiles. To address the problem that there are few methods dedicated to detecting defects in automotive running lights and existing methods are inefficient, this paper proposes a YOLOv7-based method for detecting defects in automotive running lights. The accurate recognition of running light defects with different features is achieved by training a single model. Moreover, some improvements are made on the basis of the original network to improve the accuracy of recognition. To enhance the overall feature extraction capability and detail information retention capability of the algorithm, recursive gated convolution (gnconv) and global attention mechanism (GAM) are added to the network. Ghost convolution and parameter-free attention mechanism (SimAM) are introduced to reduce information redundancy and avoid the drastic increase in the number of parameters. In addition, a parallel structure ELAN module incorporating the attention mechanism is proposed. The experimental results show that the recognition accuracy of the method in this paper reaches 89.7% on the self-built dataset, while the image inference speed is maintained at 61 frames per second, which can meet the demand for efficient detection.

The accuracy of a logistic prediction model is degraded when it is transported to populations with outcome prevalences different from that of the population used to derive the model. The resultant errors can have major clinical implications. Accordingly, the authors developed a logistic prediction model with respect to the noninvasive diagnosis of coronary disease based on 1,824 patients who underwent exercise testing and coronary angiography, varied the prevalence of disease in various "test" populations by random sampling of the original "derivation" population, and determined the accuracy of the logistic prediction model before and after the application of a mathematical algorithm designed to adjust only for these differences in prevalence. The accuracy of each prediction model was quantified in terms of receiver operating characteristic (ROC) curve area (discrimination) and chi-square goodness-of-fit (calibration). As the prevalence of the test population diverged from the prevalence of the derivation population, discrimination improved (ROC-curve areas increased from 0.82 +/- 0.02 to 0.87 +/- 0.03; p < 0.05), and calibration deteriorated (chi-square goodness-of-fit statistics increased from 9 to 154; p < 0.05). Following adjustment of the logistic intercept for differences in prevalence, discrimination was unchanged and calibration improved (maximum chi-square goodness-of-fit fell from 154 to 16). When the adjusted algorithm was applied to three geographically remote populations with prevalences that differed from that of the derivation population, calibration improved 87%, while discrimination fell by 1%. Thus, prevalence differences produce statistically significant and potentially clinically important errors in the accuracy of logistic prediction models. These errors can potentially be mitigated by use of a relatively simple mathematical correction algorithm.

Personalized review-based rating prediction, a newly emerged research problem, aims at inferring users' ratings over their unrated items using existing reviews and corresponding ratings. While some researchers proposed to learn topic factor from review text to obtain interpretability for rating prediction, they often overlooked the fact that the learned topic factors are limited to review text and cannot fully reveal the complicated relations between reviews and ratings. Moreover, topic modeling based solutions for this problem usually utilize Gibbs sampling algorithms to learn topics and word distributions, resulting in non-negligible computational overload. To address the above challenges, we propose an integrated topic and latent factor model (ITLFM), which combines topic and latent factors in a linear way to make them complement each other for better accuracies in rating prediction tasks. In addition, ITLFM models review text through an additive topic model to reveal user's and item's topic factors simultaneously. To ensure high learning efficiency, we design a hybrid stochastic learning algorithm for ITLFM. We evaluate ITLFM on several standard benchmarks and compare with representative approaches. The experimental results demonstrate that the proposed ITLFM method is computationally efficient and accurate, as well as scalable for large scale applications.

With the rapid growth of information in the Internet era, people are in great need of recommendation methods to filter information. At present, recommendation methods which based on heterogeneous information network (HIN) have attracted wide attention. Recently, HIN-based recommendation methods need to be modeled from two aspects: node structural association and semantic association. To this end, we propose a graph contrastive learning model based on structural and semantic view for HIN recommendation (GCL-SS). GCL-SS utilizes U-I interactive view to obtain node structural embeddings, and utilizes U-I semantic view to obtain node semantic embeddings. Based on these two kinds of embeddings, we establish a self-supervised contrastive learning mechanism to effectively integrate structural information and semantic information of user (item) nodes in HIN, and finally learn a more discriminative user (item) embedding. In addition, in order to strengthen the semantic association between nodes, we innovatively utilize time sequence encoder (TSE), such as LSTM, to encode semantic homogeneous network decomposed by HIN in U-I semantic view. At last, based on the user and item embeddings, we adopt bilinear decoder to model the potential association between user and item, so as to realize rating prediction of user to item. The experimental results on three real datasets confirm that our GCL-SS model performs better than state-of-the-art recommendation methods in rating prediction task. In addition, the results of four ablation experiments indicate that our GCL-SS model can effectively improve the performance of rating prediction in recommendation.

Summary Production forecasting is usually performed by applying a single model from a classical statistical standpoint (point estimation). This approach neglects: (a) model uncertainty and (b) quantification of uncertainty of the model’s estimates. This work evaluates the predictive accuracy of rate-time models to forecast production from tight-oil wells using Bayesian methods. We apply Bayesian leave-one-out (LOO) and leave-future-out (LFO) cross-validation (CV) using an accuracy metric that evaluates the uncertainty of the models’ estimates: the expected log predictive density (elpd). We illustrate the application of the procedure to tight-oil wells of west Texas. This work assesses the predictive accuracy of rate-time models to forecast production of tight-oil wells. We use two empirical models, the Arps hyperbolic and logistic growth models, and two physics-based models: scaled slightly compressible single-phase and scaled two-phase (oil and gas) solutions of the diffusivity equation. First, we perform Bayesian inference to generate probabilistic production forecasts for each model using a Bayesian workflow in which we assess the convergence of the Markov chain Monte Carlo (MCMC) algorithm, calibrate, and evaluate the robustness of the models’ inferences. Second, we evaluate the predictive accuracy of the models using the elpd accuracy metric. This metric provides a measure of out-of-sample predictive performance. We apply two different CV techniques: LOO and LFO. The results of this study are the following. First, we evaluate the predictive performance of the models using the elpd accuracy metric, which accounts for the uncertainty of the models’ estimates assessing distributions instead of point estimates. Second, we perform fast CV calculations using an important sampling technique to evaluate and compare the results of the application of two CV techniques: leave-one-out cross-validation (LOO-CV) and leave-future-out cross-validation (LFO-CV). While the goal of LOO-CV is to evaluate the models’ ability to accurately resemble the structure of the production data, LFO-CV aims to assess the models’ capacity to predict future-time production (honoring the time-dependent structure of the data). Despite the difference in their prediction goals, both methods yield similar results on the set of tight-oil wells under study. The logistic growth model yields the best predictive performance for most of the wells in the data set, followed by the two-phase physics-based flow model. This work shows the application of new tools to evaluate the predictive accuracy of models used to forecast production of tight-oil wells using: (a) an accuracy metric that accounts for the uncertainty of the models’ estimates and (b) fast computation of two CV techniques, LOO-CV and LFO-CV. To our knowledge, the proposed approach is novel and suitable to evaluate and eventually select the rate-time model(s) with the best predictive accuracy of models to forecast hydrocarbon production.

Recently, click-through rate (CTR) prediction models have evolved from shallow methods to deep neural networks. Most deep CTR models follow an Embedding&MLP paradigm, that is, first mapping discrete id features, e.g. user visited items, into low dimensional vectors with an embedding module, then learn a multi-layer perception (MLP) to fit the target. In this way, embedding module performs as the representative learning and plays a key role in the model performance. However, in many real-world applications, deep CTR model often suffers from poor generalization performance, which is mostly due to the learning of embedding parameters. In this paper, we model user behavior using an interest delay model, study carefully the embedding mechanism, and obtain two important results: (i) We theoretically prove that small aggregation radius of embedding vectors of items which belongs to a same user interest domain will result in good generalization performance of deep CTR model. (ii) Following our theoretical analysis, we design a new embedding structure named res-embedding. In res-embedding module, embedding vector of each item is the sum of two components: (i) a central embedding vector calculated from an item-based interest graph (ii) a residual embedding vector with its scale to be relatively small. Empirical evaluation on several public datasets demonstrates the effectiveness of the proposed res-embedding structure, which brings significant improvement on the model performance.

The rapid growth of edge data generated by mobile devices and applications deployed at the edge of the network has exacerbated the problem of information overload. As an effective way to alleviate information overload, recommender system can improve the quality of various services by adding application data generated by users on edge devices, such as visual and textual information, on the basis of sparse rating data. The visual information in the movie trailer is a significant part of the movie recommender system. However, due to the complexity of visual information extraction, data sparsity cannot be remarkably alleviated by merely using the rough visual features to improve the rating prediction accuracy. Fortunately, the convolutional neural network can be used to extract the visual features precisely. Therefore, the end-to-end neural image caption (NIC) model can be utilized to obtain the textual information describing the visual features of movie trailers. This paper proposes a trailer inception probabilistic matrix factorization model called Ti-PMF, which combines NIC, recurrent convolutional neural network, and probabilistic matrix factorization models as the rating prediction model. We implement the proposed Ti-PMF model with extensive experiments on three real-world datasets to validate its effectiveness. The experimental results illustrate that the proposed Ti-PMF outperforms the existing ones.