Modeling and Applications for Temporal Point Processes

Event Sequence Anomaly Detection (ESAD) plays a crucial role in domains such as healthcare, DevOps, and information security, where identifying deviations from normal patterns in event sequences is essential for ensuring stability and mitigating risks. Despite notable progress, existing ESAD methods often struggle when handling continuous-time event streams. Statistical models such as Poisson or Hawkes processes offer efficiency but fail to capture nonlinear temporal dependencies, while deep learning methods demand large-scale labeled data and frequently suffer from poor interpretability. These limitations hinder their deployment in high-stakes applications where reliability and transparency are critical. To address these challenges, we introduce TPP-LLMAD, a novel framework that integrates Temporal Point Processes (TPPs) with Large Language Models (LLMs) for interpretable anomaly detection. In TPP-LLMAD, neural TPPs model event sequences and estimate intensity functions, which are then transformed into structured triplets of timestamps, event marks, and intensities. These representations are embedded into tailored prompts that guide an LLM to assess deviations, assign anomaly labels, and generate humanreadable explanations. By combining the quantitative rigor of TPPs with the interpretive capacity of LLMs, the framework bridges the gap between mathematical modeling and naturallanguage reasoning. Extensive experiments on real-world datasets demonstrate that TPP-LLMAD achieves performance comparable to or exceeding state-of-the-art baselines while providing explanations that enhance interpretability and usability. This work represents the first systematic integration of TPP intensity modeling with LLM-based reasoning for ESAD, advancing the frontier of interpretable event sequence analysis.

Recent advancements in generative artificial intelligence (AI) have introduced novel opportunities for the application of AI within the financial sector. This chapter examines the various ways in which generative AI is contributing to the rapidly evolving landscape of finance. While traditional machine learning techniques have long been applied to financial tasks such as customer service, text summarization, risk prediction, algorithmic trading, and document processing, generative AI offers enhanced capabilities aimed at improving operational efficiency, elevating customer experience, and enabling responsible growth. Applications of generative AI in finance now extend to areas including decision-making, risk assessment, portfolio optimization, personalized financial advising, fraud detection, and predictive analytics. The field encompasses a diverse set of models, including generative adversarial networks (GANs), variational autoencoders (VAEs), temporal point processes (TPPs), transformers, and large language models (LLMs). This chapter explores how these models are applied in financial contexts, highlighting both theoretical foundations and practical implementations. A dedicated case study is presented, summarizing recent literature on the use of LLMs in financial tasks, including approaches such as zero-shot and few-shot learning with pretrained models, domain-specific fine-tuning, and training models from the ground up. Finally, the chapter discusses key opportunities and challenges associated with generative AI in FinTech, including the balance between innovation and responsibility, and the need for enhanced interpretability, robustness, and ethical compliance in financial applications, hence aiding towards goals of Industry 5.0.

Forecasting citations of scientific patents and publications is a crucial task for understanding the evolution and development of technological domains and for foresight into emerging technologies. By construing citations as a time series, the task can be cast into the domain of temporal point processes. Most existing work on forecasting with temporal point processes, both conventional and neural network-based, only performs single-step forecasting. In citation forecasting, however, the more salient goal is n -step forecasting: predicting the arrival of the next n citations. In this article, we propose Dynamic Multi-Context Attention Networks (DMA-Nets), a novel deep learning sequence-to-sequence (Seq2Seq) model with a novel hierarchical dynamic attention mechanism for long-term citation forecasting. Extensive experiments on two real-world datasets demonstrate that the proposed model learns better representations of conditional dependencies over historical sequences compared to state-of-the-art counterparts and thus achieves significant performance for citation predictions.

Recently, deep learning based approaches have emerged as indispensable tools to perform big data analytics. Normally, deep learning models are first trained with a supervised method and then deployed to execute various tasks. The supervised method involves extensive human efforts to collect and label the large-scale dataset, which becomes impractical in the big data era where raw data is largely un-labeled and uncategorized. Fortunately, the adversarial learning, represented by Generative Adversarial Network (GAN), enjoys a great success on the unsupervised learning. However, the distinct features of GAN, such as massive computing phases and non-traditional convolutions challenge the existing deep learning accelerator designs. In this work, we propose the first holistic solution for accelerating the unsupervised GAN-based Deep Learning. We overcome the above challenges with an algorithm and architecture co-design approach. First, we optimize the training procedure to reduce on-chip memory consumption. We then propose a novel time-multiplexed design to efficiently map the abundant computing phases to our microarchitecture. Moreover, we design high-efficiency dataflows to achieve high data reuse and skip the zero-operand multiplications in the non-traditional convolutions. Compared with traditional deep learning accelerators, our proposed design achieves the best performance (average 4.3X) with the same computing resource. Our design also has an average of 8.3X speedup over CPU and 6.2X energy-efficiency over NVIDIA GPU.

In recent years, Deep Learning (DL), such as the algorithms of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Generative Adversarial Networks (GAN), has been widely studied and applied in various fields including agriculture. Researchers in the fields of agriculture often use software frameworks without sufficiently examining the ideas and mechanisms of a technique. This article provides a concise summary of major DL algorithms, including concepts, limitations, implementation, training processes, and example codes, to help researchers in agriculture to gain a holistic picture of major DL techniques quickly. Research on DL applications in agriculture is summarized and analyzed, and future opportunities are discussed in this paper, which is expected to help researchers in agriculture to better understand DL algorithms and learn major DL techniques quickly, and further to facilitate data analysis, enhance related research in agriculture, and thus promote DL applications effectively. Keywords: deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition DOI: 10.25165/j.ijabe.20181104.4475 Citation: Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

Learning an effective distance measurement between sample pairs plays an important role in visual analysis, where the training procedure largely relies on hard negative samples. However, hard negative samples usually account for the tiny minority in the training set, which may fail to fully describe the data distribution close to the decision boundary. In this paper, we present a deep adversarial metric learning (DAML) framework to generate synthetic hard negatives from the original negative samples, which is widely applicable to existing supervised deep metric learning algorithms. Different from existing sampling strategies which simply ignore numerous easy negatives, our DAML aim to exploit them by generating synthetic hard negatives adversarial to the learned metric as complements. We simultaneously train the feature embedding and hard negative generator in an adversarial manner, so that adequate and targeted synthetic hard negatives are created to learn more precise distance metrics. As a single transformation may not be powerful enough to describe the global input space under the attack of the hard negative generator, we further propose a deep adversarial multi-metric learning (DAMML) method by learning multiple local transformations for more complete description. We simultaneously exploit the collaborative and competitive relationships among multiple metrics, where the metrics display unity against the generator for effective distance measurement as well as compete for more training data through a metric discriminator to avoid overlapping. Extensive experimental results on five benchmark datasets show that our DAML and DAMML effectively boost the performance of existing deep metric learning approaches through adversarial learning.

Entity resolution targets at identifying records that represent the same real-world entity from one or more datasets. A major challenge in learning-based entity resolution is how to reduce the label cost for training. Due to the quadratic nature of record pair comparison, labeling is a costly task that often requires a significant effort from human experts. Inspired by recent advances of generative adversarial network (GAN), we propose a novel deep learning method, called ErGAN, to address the challenge. ErGAN consists of two key components: a label generator and a discriminator which are optimized alternatively through adversarial learning. To alleviate the issues of overfitting and highly imbalanced distribution, we design two novel modules for diversity and propagation, which can greatly improve the model generalization power. We have conducted extensive experiments to empirically verify the labeling and learning efficiency of ErGAN. The experimental results show that ErGAN beats the state-of-the-art baselines, including unsupervised, semi-supervised, and unsupervised learning methods.

Deep Learning is a new generation of artificial neural networks that have transformed our daily lives and has impacted several industries and scientific disciplines in recent years. Recent development in deep learning provides various significant methods to obtain end-to-end learning models from complex data. Generative Adversarial Networks give the idea of learning the deep representations without extensively interpreting the training data. The generative adversarial network involves the generative modeling approach that uses the deep learning approach. The chapter is broadly divided into the following sections as (1) Insights of deep learning & the generative adversarial networks, (2) GAN’s representative variants, training methods, architecture, and mathematical representation, and (3) Efficacy of GAN in different applications. This chapter will gain the recent work done in GAN and implement this technique in the different deep learning applications for healthcare. Here, we will also analyze some of the future perspectives and trends in the forthcoming time.<br>

In recent years, machine learning and Deep Learning have increased and gathered epic success in traditional application domains and new areas of Artificial Intelligence. The performance using Deep Learning has dominated experimental results compared to conventional machine learning algorithms. This paper presents an overview of the progress that has occurred in Deep Learning (DL) concerning some application domains like Autonomous Driving, Healthcare, Voice Recognition, Image Recognition, Advertising, Predicting Natural Calamities, National Stock Exchange and many more. Additionally, deeper insights into several Deep Learning techniques, their working principles, and experimental results are scrutinized. The survey covers Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL).

Deep learning enables a computer to construct complex concepts from simpler ones by learning data representation with abstraction. However, deep learning requires a large number of samples for training to prevent over-fitting. Generative adversarial network (GAN) is a kind of unsupervised learning that can be used to synthesize images. Generating higher quality images using GAN is conducive to improving the training efficiency of deep learning. How to generate high quality and high-diversity samples using GAN to expand the sample size for deep learning training is quite appealing. In this paper, the concept of generative adversarial network including network structure and four popular loss functions is introduced. Furthermore, three typical GAN-based models to improve the image quality or diversity have been presented. Finally, the paper discusses the shortcomings of GANs to steer future research in the right direction. Generation of high quality images using GAN has great potential to help deep learning models to reduce the over-fitting problem in the near future.

Surface defect recognition (SDC) is essential in intelligent manufacturing. Deep learning (DL) is a research hotspot in SDC. Limited defective samples are available in most real-world cases, which poses challenges for DL methods. Given such circumstances, generating defective samples by generative adversarial networks (GANs) is applied. However, insufficient samples and high-frequency texture details in defects make GANs very hard to train, yield mode collapse, and poor image quality, which can further impact SDC. To solve these problems, this article proposes a new GAN called contrastive GAN, which can be trained to generate diverse defects with only extremely limited samples. Specifically, a shared data augmentation (SDA) module is proposed for avoiding overfitting. Then, a feature attention matching (FAM) module is proposed to align features for improving the quality of generated images. Finally, a contrastive loss based on hypersphere is employed to constrain GANs to generate images that differ from the traditional transform. Experiments show that the proposed GAN generates defective images with higher quality and lower variance between real defects compared to other GANs. Synthetic images contribute to pretrained DL networks with accuracies of up to 95.00%–99.56% for Northeastern University (NEU) datasets of different sizes and 91.84% for printed circuit board (PCB) cases, which proves the effectiveness of the proposed method.

Temporal point processes (TPP) are probabilistic generative models for continuous-time event sequences. Neural TPPs combine the fundamental ideas from point process literature with deep learning approaches, thus enabling construction of flexible and efficient models. The topic of neural TPPs has attracted significant attention in the recent years, leading to the development of numerous new architectures and applications for this class of models. In this review paper we aim to consolidate the existing body of knowledge on neural TPPs. Specifically, we focus on important design choices and general principles for defining neural TPP models. Next, we provide an overview of application areas commonly considered in the literature. We conclude this survey with the list of open challenges and important directions for future work in the field of neural TPPs.

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

Transfer learning can solve some problems that the lack of data and labeled data is unavailable in deep learning. The domain adaptation method in transfer learning can solve the problem of domain-shift, which is the main direction of current research. The generative adversarial network is popular in the generative model now, So the combination of them will obtain more respectable performance. While previous approaches only use it as a data augmentation tool. Even though the overall distribution is aligned, some samples will still be misclassified, and the corresponding classifier is trained on the source samples and cannot perfectly generalize to the target. We propose a novel generative discriminative domain adaptation method based on the reverse gradient by combining the generative adversarial method to reverse gradient. The method uses unlabeled target domain data to make the target and source domains distributions get closer in the feature space. At the same time, the auxiliary class's generation adversarial network is used to generate consistent-class representation, and the corresponding class's distribution is aligned across domains. Besides, we use circle loss as the classification function. Circle loss proposed a unified formula for deep feature learning features. It is useful for our classification task in our method. Our proposed method achieves better performance than others in most experimental settings and has shown to work well across different datasets such as digits and Office.

The Science of Deep Learning emerged from courses taught by the author that have provided thousands of students with training and experience for their academic studies, and prepared them for careers in deep learning, machine learning, and artificial intelligence in top companies in industry and academia. The book begins by covering the foundations of deep learning, followed by key deep learning architectures. Subsequent parts on generative models and reinforcement learning may be used as part of a deep learning course or as part of a course on each topic. The book includes state-of-the-art topics such as Transformers, graph neural networks, variational autoencoders, and deep reinforcement learning, with a broad range of applications. The appendices provide equations for computing gradients in backpropagation and optimization, and best practices in scientific writing and reviewing. The text presents an up-to-date guide to the field built upon clear visualizations using a unified notation and equations, lowering the barrier to entry for the reader. The accompanying website provides complementary code and hundreds of exercises with solutions.