Rating prediction using review texts with underlying sentiments

Personalized review-based rating prediction, a newly emerged research problem, aims at inferring users' ratings over their unrated items using existing reviews and corresponding ratings. While some researchers proposed to learn topic factor from review text to obtain interpretability for rating prediction, they often overlooked the fact that the learned topic factors are limited to review text and cannot fully reveal the complicated relations between reviews and ratings. Moreover, topic modeling based solutions for this problem usually utilize Gibbs sampling algorithms to learn topics and word distributions, resulting in non-negligible computational overload. To address the above challenges, we propose an integrated topic and latent factor model (ITLFM), which combines topic and latent factors in a linear way to make them complement each other for better accuracies in rating prediction tasks. In addition, ITLFM models review text through an additive topic model to reveal user's and item's topic factors simultaneously. To ensure high learning efficiency, we design a hybrid stochastic learning algorithm for ITLFM. We evaluate ITLFM on several standard benchmarks and compare with representative approaches. The experimental results demonstrate that the proposed ITLFM method is computationally efficient and accurate, as well as scalable for large scale applications.

Joint latent factors and attributes to discover interpretable preferences in recommendation

In current times, the importance of online hotel review sites has become more and more apparent. Users of these sites reference of reviews strongly influences their purchase behavior and as such, reviews are important to companies and researchers alike. The majority of review sites offer both text reviews and numerical hotel ratings, and both information sources are widely used by researchers as a representation of a customer's sentiment and opinion. However, an opinion is a difficult concept to measure, and as such, depending on the relation these two sources have, it would be apparent whether or not it is safe to consider them equally in research. In this study we utilize an entropy-based Support Vector Machine to classify positive and negative sentiments in hotel reviews from the site Ctrip, then calculating the ratio of positive and negative sentiment in each review and examine their correlation with said review's rating score using Spearman and Kendall Correlation coefficients and Maximal Information Coefficient (MIC).

Latent factor (LF) models are highly effective in extracting useful knowledge from High-Dimensional and Sparse (HiDS) matrices which are commonly seen in various industrial applications. An LF model usually adopts iterative optimizers, which may consume many iterations to achieve a local optima, resulting in considerable time cost. Hence, how to accelerate the training process of an LF model becomes a highly significant issue. To address it, this work innovatively proposes a randomized latent factor (RLF) model. It incorporates the principle of randomized learning techniques for neural networks into the LF analysis on HiDS matrices to alleviate the computational burden greatly. It also extends the standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly. Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models, RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data. More importantly, it provides a novel, effective, and efficient approach to LF analysis on HiDS matrices.

In today’s online services, users’ feedback such as numerical rating, textual review, time of purchase and so on for each product is often encouraged to provide. Many recommender systems predict the products that the users may like and purchase in the future using users’ historical ratings. With the increase of user data in the systems, more detailed and interpretable information about product features and user sentiments can be extracted from textual reviews that are relative to ratings. In this paper, we propose a novel topic and sentiment matrix factorization model, which leverages both topic and sentiment drawn from the reviews simultaneously. First, we conduct topic analysis and sentiment analysis on reviews using Latent Dirichlet Allocation (LDA) and a lexicon construction technique, respectively. Second, we combine the user consistency, which is calculated from his/her reviews and ratings, and helpful votes from other users on reviews to obtain a reliability measure to weight the ratings. Third, we integrate the obtained reliability measure and the results of the topic and sentiment analysis of reviews into the matrix factorization framework for prediction. Our experimental comparison using eight Amazon datasets indicates that the proposed method significantly improves performance compared to traditional matrix factorization up to 20.82%.

How to extract useful knowledge from a high-dimensional and sparse (HiDS) matrix efficiently is critical for many big data-related applications. A latent factor (LF) model has been widely adopted to address this problem. It commonly relies on an iterative learning algorithm like stochastic gradient descent. However, an algorithm of this kind commonly consumes many iterations to converge, resulting in considerable time cost on large-scale datasets. How to accelerate an LF model's training process without accuracy loss becomes a vital issue. To address it, this study innovatively proposes a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</u> ultilayered-and-randomized <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</u> atent <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</u> actor (MLF) model. Its main idea is two-fold: a) adopting randomized-learning to train LFs for implementing a ‘one-iteration’ training process for saving time; and 2) adopting the principle of a generally multilayered structure as in a deep forest or multilayered extreme learning machine to structure its LFs, thereby enhancing its representative learning ability. Empirical studies on six HiDS matrices from real applications demonstrate that compared with state-of-the-art LF models, an MLF model achieves significantly higher computational efficiency with satisfactory prediction accuracy. It has the potential to handle LF analysis on a large scale HiDS matrix with real-time requirements.

Recommender systems are essential engines to deliver product recommendations for e-commerce businesses. Successful adoption of recommender systems could significantly influence the growth of marketing targets. Collaborative filtering is a type of recommender system model that uses customers' activities in the past, such as ratings. Unfortunately, the number of ratings collected from customers is sparse, amounting to less than 4%. The latent factor model is a kind of collaborative filtering that involves matrix factorization to generate rating predictions. However, using only matrix factorization would result in an inaccurate recommendation. Several models include product review documents to increase the effectiveness of their rating prediction. Most of them use methods such as TF-IDF and LDA to interpret product review documents. However, traditional models such as LDA and TF-IDF face some shortcomings, in that they show a less contextual understanding of the document. This research integrated matrix factorization and novel models to interpret and understand product review documents using LSTM and word embedding. According to the experiment report, this model significantly outperformed the traditional latent factor model by more than 16% on an average and achieved 1% on an average based on RMSE evaluation metrics, compared to the previous best performance. Contextual insight of the product review document is an important aspect to improve performance in a sparse rating matrix. In the future work, generating contextual insight using bidirectional word sequential is required to increase the performance of e-commerce recommender systems with sparse data issues.

Recently, many e-commerce websites have encouraged their users to rate shopping items and write review texts. This review information has been very useful for understanding user preferences and item properties, as well as enhancing the capability to make personalized recommendations of these websites. In this paper, we propose to model user preferences and item properties using convolutional neural networks (CNNs) with dual local and global attention, motivated by the superiority of CNNs to extract complex features. By using aggregated review texts from a user and aggregated review text for an item, our model can learn the unique features (embedding) of each user and each item. These features are then used to predict ratings. We train these user and item networks jointly which enable the interaction between users and items in a similar way as matrix factorization. The local attention provides us insight on a user's preferences or an item's properties. The global attention helps CNNs focus on the semantic meaning of the whole review text. Thus, the combined local and global attentions enable an interpretable and better-learned representation of users and items. We validate the proposed models by testing on popular review datasets in Yelp and Amazon and compare the results with matrix factorization (MF), the hidden factor and topical (HFT) model, and the recently proposed convolutional matrix factorization (ConvMF+). Our proposed CNNs with dual attention model outperforms HFT and ConvMF+ in terms of mean square errors (MSE). In addition, we compare the user/item embeddings learned from these models for classification and recommendation. These results also confirm the superior quality of user/item embeddings learned from our model.

Recommender systems are designed to identify the items that a user will like or find useful based on the user's prior preferences and activities. These systems have become ubiquitous and are an essential tool for information filtering and (e-)commerce. Over the years, collaborative filtering, which derive these recommendations by leveraging past activities of groups of users, has emerged as the most prominent approach for solving this problem. This talk will present some of our recent work towards improving the performance of collaborative filtering-based recommender systems and understanding some of their fundamental limitations and characteristics. It will start by analyzing how the ratings that users provide to a set of items relate to their ratings of the set's individual items and, using these insights, will present rating prediction approaches that utilize distant supervision. It will then discuss extensions to approaches based on sparse linear and latent factor models that postulate that users' preferences are a combination of global and local preferences, which are shown to lead to better user modeling and as such improved prediction performance. Finally, the talk will conclude by discussing what can be accurately predicted by latent factor approaches and by analyzing the estimation error of sparse linear and latent factor models and how its characteristics impacts the performance of top N recommendation algorithms.

Online customer reviews provide valuable information about product quality, and recently some control chart-based schemes have been proposed to detect product performance anomalies from reviews. As a review outcome depends not only on inherent product quality but also on customer’s rating bias and latent preference, ignoring customer’s latent factors may lead to misjudgements about online product performance. Therefore, this study proposes a risk-adjusted control chart for monitoring the decrease of rating scores in reviews, by separating the personal risk factors of individual customers from the assignable causes with respect to online product performance. The proposed risk-adjustment model is fitted by a united latent factor model that learns user rating bias and preference factors by combining both review texts and corresponding ratings. According to the experimental results of a real-world case study on beer review corpus and extensive simulation studies, the proposed method shows superior performance in review shift detection, also with good diagnostic performance for explaining the reasons behind anomalies.

Profiling heterogeneity of Alzheimer's disease using white-matter impairment factors

Project Portfolio Management (PPM) has gained success in many projects due to its large number of features that covers effective scheduling, risk management, collaboration, and third-party software integrations to mention a few. A broad range of PPM software is available; however, it is essential to select the PPM with minimum usage issues over time. While many companies use surveys and market research to get users feedback, the PPM product software reviews carry the voice of users; the positive and negative sentiments of the PPM software reviews. This paper collected 4,775 reviews of ten PPM software from Capttera.com. Our approach has these phases- text preprocessing, sentiment analysis, summarization, and categorizations. The software reviews are filtered and cleaned, then negative sentiments of user reviews are summarized into a set of factors that identify issues of adopted PPM software. We report the most important issues of PPM software which were related to missing technological features and lack of training. Results using Latent Dirichlet Allocation (LDA) model showed that the top ten common issues are related to software complexity and lack of required features.

Mixture factor analysis is examined as a means of flexibly estimating nonnormally distributed continuous latent factors in the presence of both continuous and dichotomous observed variables. A simulation study compares mixture factor analysis with normal maximum likelihood (ML) latent factor modeling. Different results emerge for continuous versus dichotomous outcomes. For dichotomous outcomes, normal ML path estimates have bias that worsens as latent factor skew/kurtosis increases and does not diminish as sample size increases, whereas the mixture factor analysis model produces nearly unbiased estimators as sample sizes increase (500 and greater) and offers near nominal coverage probability. For continuous outcome variables, both methods produce factor loading estimates with minimal bias regardless of latent factor skew, but the mixture factor analysis is more efficient. The method is demonstrated using data motivated by a study on youth with cystic fibrosis examining predictors of treatment adherence. In summary, mixture factor analysis provides improvements over normal ML estimation in the presence of skewed/kurtotic latent factors, but due to variability in the estimator relating the latent factor to dichotomous outcomes and computational issues, the improvements were only fully realized, in this study, at larger sample sizes (500 and greater).

Latent factor models have been widely used for recommendation. Most existing latent factor models mainly utilize the rating information between users and items, although some recently extended models add some auxiliary information to learn a unified latent factor between users and items. The unified latent factor only represents the latent features of users and items from the aspect of purchase history. However, the latent features of users and items may stem from different aspects, e.g., the brand-aspect and category-aspect of items. In this paper, we propose a Neural network based Aspect-level Collaborative Filtering model (NeuACF) to exploit different aspect latent factors. Through modelling rich objects and relations in recommender system as a heterogeneous information network, NeuACF first extracts different aspect-level similarity matrices of users and items through different meta-paths and then feeds an elaborately designed deep neural network with these matrices to learn aspect-level latent factors. Finally, the aspect-level latent factors are effectively fused with an attention mechanism for the top-N recommendation. Extensive experiments on three real datasets show that NeuACF significantly outperforms both existing latent factor models and recent neural network models.

Tensor factorization method based on review text semantic similarity for rating prediction