Taming Ultra-Long Behavior Sequence in Session-wise Generative Recommendation

CTR (Click-Through Rate) prediction, crucial for recommender systems and online advertising, etc., has been confirmed to benefit from modeling long-term user behaviors. Nonetheless, the vast number of behaviors and complexity of noise interference pose challenges to prediction efficiency and effectiveness. Recent solutions have evolved from single-stage models to two-stage models. However, current two-stage models often filter out significant information, resulting in an inability to capture diverse user interests and build the complete latent space of user interests. Inspired by multi-interest and generative modeling, we propose DiffuMIN (Diffusion-driven Multi-Interest Network) to model long-term user behaviors and thoroughly explore the user interest space. Specifically, we propose a target-oriented multi-interest extraction method that begins by orthogonally decomposing the target to obtain interest channels. This is followed by modeling the relationships between interest channels and user behaviors to disentangle and extract multiple user interests. We then adopt a diffusion module guided by contextual interests and interest channels, which anchor users' personalized and target-oriented interest types, enabling the generation of augmented interests that align with the latent spaces of user interests, thereby further exploring restricted interest space. Finally, we leverage contrastive learning to ensure that the generated augmented interests align with users' genuine preferences. Extensive offline experiments are conducted on two public datasets and one industrial dataset, yielding results that demonstrate the superiority of DiffuMIN. Moreover, DiffuMIN increased CTR by 1.52% and CPM by 1.10% in online A/B testing. Our source code is available at https://github.com/laiweijiang/DiffuMIN.

Click-through rate (CTR) prediction is crucial in recommendation and online advertising systems. Existing methods usually model user behaviors, while ignoring the informative context which influences the user to make a click decision, e.g., click pages and pre-ranking candidates that inform inferences about user interests, leading to suboptimal performance. In this paper, we propose a Decision-Making Context Interaction Network (DCIN), which deploys a carefully designed Context Interaction Unit (CIU) to learn decision-making contexts and thus benefits CTR prediction. In addition, the relationship between different decision-making context sources is explored by the proposed Adaptive Interest Aggregation Unit (AIAU) to improve CTR prediction further. In the experiments on public and industrial datasets, DCIN significantly outperforms the state-of-the-art methods. Notably, the model has obtained the improvement of CTR+2.9%/CPM+2.1%/GMV+1.5% for online A/B testing and served the main traffic of Meituan Waimai advertising system.

Recommending fashion outfits requires learning a concept of style and fashionability that is typically human. There has been an increasing research effort into creating Machine Learning models able to learn such concepts, in order to distinguish between compatible and incompatible clothes and to select an item that would complete an outfit. However, most of the work done in literature tackles this problem from a pure Machine Learning point of view, disregarding real-case scenarios and the human interaction with systems able to generate outfits. This work tries to move the problem of generating outfits to the Recommender Systems domain by presenting as its main contribution a novel algorithm for a fashion-specific Recommender System that generates fashionable outfits, able to scale its inference time to be useful in real use case scenarios, and applies such algorithm on public and industrial datasets. In addition to this, this work shows preliminary results on how this algorithm can be employed in a real scenario and reports as preliminary results the evaluations provided by three professional stylists on the outfits generated by such algorithms.

Accurate Click-Through Rate (CTR) prediction is vital for search engines and recommendation systems, yet it is often hindered by the ''cold start problem'', which arises from insufficient historical data for new users. Recent approaches have sought to tackle this by training encoder-decoder networks on data from warm users to generate virtual behavior embeddings for cold users. However, these methods have shortcomings in terms of simplistic encoding techniques for warm user behaviors and direct utilization of virtual behavior embeddings, leading to limitations in user interest expression and generalization. To address these challenges, we propose a novel method that leverages Graph Isomorphism Networks (GIN) for cohort modeling within CTR prediction. GIN effectively captures high-order user-item interactions, providing a more nuanced understanding of users' diverse interests. Additionally, the cohort modeling strategy minimizes deviations in constructed embeddings, enhancing the model's generalization abilities. We validate our approach through experiments on public and industrial datasets, demonstrating significant improvements for both warm and cold users compared to existing methodologies. Furthermore, we implemented the GIN Cohort Modeling (GINCM) in a large-scale online advertising system, optimizing for both pre-computation and real-time processing to reduce latency. The implementation yields notable enhancements of 2.13% in CTR and Revenue Per Mille(RPM), showcasing the practical effectiveness and real-world applicability of our model.

This paper shows a method for building and publishing datasets in commercial services. Datasets contribute to the development of research in machine learning and recommender systems. In particular, because recommender systems play a central role in many commercial services, publishing datasets from the services are in great demand from the recommender system community. However, the publication of datasets by commercial services may have some business risks to those companies. To publish a dataset, this must be approved by a business manager of the service. Because many business managers are not specialists in machine learning or recommender systems, the researchers are responsible for explaining to them the risks and benefits. We first summarize three challenges in building datasets from commercial services: (1) anonymize the business metrics, (2) maintain fairness, and (3) reduce the popularity bias. Then, we formulate the problem of building and publishing datasets as an optimization problem that seeks the sampling weight of users, where the challenges are encoded as appropriate loss functions. We applied our method to build datasets from the raw data of our real-world mobile news delivery service. The raw data has more than 1,000,000 users with 100,000,000 interactions. Each dataset was built in less than 10 minutes. We discussed the properties of our method by checking the statistics of the datasets and the performances of typical recommender system algorithms.

In the wake of recent advancements in the field of AI, this paper investigates the impact of recommender systems and generative models on human decisional and creative autonomy. For this purpose, we adopt Dennett’s conception of autonomy as self-control. We show that recommender systems can play a double role in relation to decisional autonomy: as information filter, they can augment self-control in decision-making, but also act as mechanisms of remote control that clamp degrees of freedom. As for generative models in AI, we show that they can be seen as a powerful system of selection and suggestion (similar to standard recommender systems) but also as an instrument for information production. We suggest that the latter perspective opens new possibilities in terms of creative autonomy. Additionally, for both systems we propose a distinction between “extrinsic” and “intrinsic” mechanisms and effects. Through Dennett’s theory of self-control, this paper offers new insights into the relation between AI and human autonomy by framing it in terms of remote or self-control and by addressing the impact of generative models on creative autonomy.

Recommendation models typically follow a discriminative paradigm, predicting whether items should be retrieved. While effective, the expressive capabilities of these recommender systems are limited. Users can only passively browse the recommended items rather than actively express their needs and engage in an interactive experience. With recent advances in generative models such as large language models, a paradigm shift is happening in the study of recommender systems. Researchers propose building generative recommendation models either by aligning pre-trained generative models with user behaviors or designing recommendation models within a generative framework. These models enable the systems to receive and deliver more human-like content, such as natural language, images, and beyond. In this tutorial, we first provide an overview of the latest progress in generative recommendation models, covering approaches based on large language models, semantic IDs, diffusion models, and more. We then make an in-depth discussion on the challenges, open questions, and potential future directions in developing generative recommendation models.

Surface defect inspection based on deep learning has demonstrated outstanding performance in improving detection accuracy and model generalization. However, the small scale of defect datasets always limits the application of deep models in industry. Generative models can obtain realistic samples in a very cheap way, which can effectively solve this problem and thus has received widespread attention in recent years. This paper provides a comprehensive analysis and summary of the current studies of surface defect inspection methods proposed between 2022 and 2024. First, according to the use of generative models, these methods are classified into four categories: Variational Auto-Encoders (VAEs), Generative Adversarial Networks (GANs), Diffusion Models (DMs), and multi-models. Second, the research status of surface defect inspection based on generative models in recent years is discussed from four aspects: sample generation, detection objective, inspection task, and learning model. Then, the public datasets and evaluation metrics that are commonly used for surface defect inspection are discussed, and a comparative evaluation of defect inspection methods based on generative models is provided. Finally, this study discusses the existing challenges for the defect inspection methods based on generative models, providing insights for future research.

Extractive approaches have been the mainstream paradigm for identifying overlapping entity–relation extraction. However, limited by their inherently methodological flaws, which hardly deal with three issues: hierarchical dependent entity–relations, implicit entity–relations, and entity normalization. Recent advances have proposed an effective solution based on generative language models, which cast entity–relation extraction as a sequence-to-sequence text generation task. Inspired by the observation that humans learn by getting to the bottom of things, we propose a novel framework, namely GenRE, Generative multi-turn question answering with contrastive learning for entity–relation extraction. Specifically, a template-based question prompt generation first is designed to answer in different turns. We then formulate entity–relation extraction as a generative question answering task based on the general language model instead of span-based machine reading comprehension. Meanwhile, the contrastive learning strategy in fine-tuning is introduced to add negative samples to mitigate the exposure bias inherent in generative models. Our extensive experiments demonstrate that GenRE performs competitively on two public datasets and a custom dataset, highlighting its superiority in entity normalization and implicit entity–relation extraction. (The code is available at https://github.com/lovelyllwang/GenRE).

Deep generative models are promising in detecting novel cyber-physical attacks, mitigating the vulnerability of Cyber-physical systems (CPSs) without relying on labeled information. Nonetheless, these generative models face challenges in identifying attack behaviors that closely resemble normal data, or deviate from the normal data distribution but are in close proximity to the manifold of the normal cluster in latent space. To tackle this problem, this article proposes a novel unsupervised dual variational generative adversarial model named MST-DVGAN, to perform anomaly detection in multivariate time series data for CPS security. The central concept is to enhance the model's discriminative capability by widening the distinction between reconstructed abnormal samples and their normal counterparts. Specifically, we propose an augmented module by imposing contrastive constraints on the reconstruction process to obtain a more compact embedding. Then, by exploiting the distribution property and modeling the normal patterns of multivariate time series, a variational autoencoder is introduced to force the generative adversarial network (GAN) to generate diverse samples. Furthermore, two augmented loss functions are designed to extract essential characteristics in a self-supervised manner through mutual guidance between the augmented samples and original samples. Finally, a specific feature center loss is introduced for the generator network to enhance its stability. Empirical experiments are conducted on three public datasets, namely SWAT, WADI and NSL_KDD. Comparing with the state-of-the-art methods, the evaluation results show that the proposed MTS-DVGAN is more stable and can achieve consistent performance improvement.

Los sistemas de recomendación han tenido un rápido crecimiento tanto en investigación académica como en aplicaciones industriales en los últimos años. Las recomendaciones se nos proporcionan cuando elegimos elementos como música, películas, restaurantes, noticias, etc. Como parte de las técnicas de recomendación más prometedoras, los sistemas basados en recomendación de filtros colaborativos desempeñan un papel importante en las aplicaciones actuales. La aparición de nuevas fuentes de información requiere la combinación de sistemas de recomendación con técnicas avanzadas de otros campos, como el análisis de redes sociales, el procesamiento del lenguaje natural, el aprendizaje en profundidad, etc. Esta combinación mejora la calidad de las recomendaciones. Los sistemas de recomendación anteriores se centran principalmente en las mejoras de la precisión de la recomendación. Recientemente, surge una tendencia para la evaluación del rendimiento de los sistemas de recomendación que toma muchas propiedades nuevas en consideración. La fiabilidad es uno de los aspectos más notables y será profundamente estudiado en esta tesis. Se han propuesto algunas medidas para medir la fiabilidad de los sistemas de recomendación de filtrado colaborativo basados en la memoria. Sin embargo, observamos un área sin cubrir en el diseño de las medidas de fiabilidad adecuadas para los sistemas de recomendación de filtros colaborativos basados en modelos, especialmente aquellos basados en la factorización matricial. También encontramos que hay una falta de medidas de calidad de fiabilidad adecuadas. Esta tesis se concentra en proporcionar soluciones a ambas ausencias. Después de un estudio exhaustivo de trabajos de investigación del estado del arte, proponemos una nueva medida de fiabilidad que supera los desafíos de generalización y abstracción causados por los factores latentes de los métodos de factorización matricial. La medida propuesta se compara con con métodos y algoritmos representativos, usando bases de datos publicas, tanto académicas como comerciales. Los resultados experimentales muestran que nuestra nueva medida logra mejoras en la calidad de la recomendación. Además, la medida de fiabilidad propuesta también tiene un impacto muy positivo contra los "Shilling Attacks". Para obtener una mejora respecto a las medidas existentes de calidad de la fiabilidad, también proponemos, hasta donde sabemos, la primera medida de calidad de fiabilidad universal que es capaz de proporcionar valores numéricos para cualquier tipo de sistema de recomendación. Esta medida también se ha probado usando conjuntos de datos públicos reales y ha sido comparada con los métodos de calidad de la fiabilidad existentes. Basándonos en los resultados de los experimentos, concluimos que nuestra medida supera los últimos trabajos de investigación, proporcionando valores de calidad de fiabilidad más precisos e informativos. ----------ABSTRACT---------- Recommender systems have experienced a rapid growth in both academical research and industrial applications in latest years. Recommendations are provided to us when we choose items such as music, movies, restaurants, news, etc. As the most promising recommendation techniques, collaborative filtering recommender systems play an important role in current applications. The emergence of novel information sources requires to combine recommender systems with several advanced techniques coming from other fields like social network analysis, natural language processing, deep learning etc. This combination improves the quality of recommendations. Previous recommender systems research mainly focuses on the improvements of recommendation accuracy. Recently a new trend arises for the performance evaluation of recommender systems that takes many novel properties into consideration. Reliability is one of the most remarkable aspects, which will be deeply studied in this thesis. Some measures have been proposed for measuring reliability of memory based collaborative filtering recommender systems. However, we observe a gap in the design of proper reliability measures for model based collaborative filtering recommender systems, especially those based on matrix factorization. We also found that there is a lack in proper reliability quality measures. This thesis concentrates on providing solutions to both lacks. After a comprehensive study of state-of-the-art research works, we propose a novel reliability measure that overcomes the challenges of generality and abstraction caused by latent factors of matrix factorization methods. The proposed measure is tested against current baselines on public academic and commercial datasets; experiment results show that our novel measure achieves better improvements on recommendation quality. What’s more, the proposed reliability measure also has a very positive impact against shilling attacks. In order to fill in the blank of reliability quality measures, we also propose, as far as we know, the first universal reliability quality measure that is able to provide numerical values for any type of recommender systems. This measure has also been tested on public academic and commercial datasets using current methods. Based on the experiment results we conclude that our quality measure outperforms state-of-the-art research works by providing more accurate and informative reliability quality values.

This scientific article aims to develop a model based on generative artificial intelligence for the optimization of the geometric design of solar thermal collectors, with the goal of maximizing their thermal efficiency in residential and industrial applications. A methodological approach is followed, which employs generative neural networks to explore the design space of the collectors, producing a variety of novel and improved configurations. Through simulations, the performance of each design is evaluated in terms of efficiency, considering parameters such as collector area, geometric enhancements (e.g., reflectors or selective covers), and solar radiation conditions.The results demonstrate that the generative model is capable of increasing the thermal efficiency of collectors compared to baseline designs, identifying optimal configurations with efficiencies close to 75%. Simulated data graphs and tables are included, along with a flow diagram of the proposed process and pseudocode of the optimization algorithm.The literature review covers previous optimization techniques (heuristics, machine learning) and fundamentals of generative AI (GANs, transformers, diffusion models), justifying the choice of the technique used. The proposed generative approach proves to be a promising tool for the automated design of solar thermal collectors, capable of enhancing energy performance and providing innovative solutions in solar engineering.

The unprecedented acceleration in the adoption and development of artificial intelligence (AI) applications-spanning domains such as natural language processing, computer vision, recommendation systems, and generative modeling-

Movie recommendation systems have become ubiquitous in most sides of our lives. Currently, they are far from optimal. This paper presents a movielense recommendation system based on machine learning through utilizing the deep convolutional network and depending on generative modeling of public previous aspects mixtures. The objective of this paper is to introduce such a recommendation system to help users in selecting datasets of movies according to certain pre-specified measurements and data. The applied methodology is pivoted on implementing the system by using different sentimental analysis algorithms. These algorithms are keen to provide a solution for the full stack developers through using a trained model using their datasets. This will give suggestions based on their previous activity or recommended by other users’ interests demonstrated on their website. Thus to help users visualize their interest or to form the better scope of visualization. The presented system has proved better results concerning accuracy and efficiency in comparison with some other similar works. When experimentations on both real and synthetic datasets were conducted, the system showed percentile improvement of about 91.07%in the training dataset and 93.49%in the testing dataset respectively. This system is convenient for several application fields like time series network visualization, business process modeling, various data mining applications, e-commerce websites, besides most online platforms that people use including social media.

Target tracking is a hot topic in the field of computer vision. It uses the context information of video or image sequence to model the appearance and motion information of the target, so as to predict the motion state of the target and calibrate the position of the target. From the perspective of building models, target tracking algorithms can be divided into generative models and discriminant models; From the number of tracking targets, it can be divided into single target tracking and multi-target tracking. Target tracking integrates the theories and algorithms of image processing, machine learning, optimization and other fields. It is the premise and foundation to complete higher-level image understanding.The basic task of target tracking is to give the initial position of the target in a video sequence and continuously track and locate the target in each subsequent frame. In this process, a priori conditions such as the color, shape and size of the target will not be provided, that is, the tracking algorithm can track only by learning the target in the first frame. The traditional solution is to calculate the project similarity offline and save it in the system for algorithm call, but it can not make full use of the latest scoring data to reflect the change of user interest. To solve the above problems, a target tracking item similarity incremental update mechanism suitable for online applications is proposed, so that the recommendation system can update the similarity data between the corresponding item and other items in real time after the current user submits the item score, so that the recommendation system can recommend based on the latest item similarity data, To adapt to changes in user interests.