Evolutionary Neural Architecture Search for Traffic Forecasting

ETNAS: An energy consumption task-driven neural architecture search

CPNet: Conditionally parameterized graph convolutional network for traffic forecasting

Decision of the model complexity is a significant challenge in contemporary data-driven modeling applications. Designing neural architecture involves the process of determining the optimal model complexity for deep neural networks (DNNs) models in order to uncover relationships in real-world data patterns. Consequently, optimizing DNN architectures is crucial for enhancing the practical approximation performance of DNN models to real-world data. This study implements a data-driven neuroevolution scheme for the optimal neural architecture search (NAS) and demonstrates a data-driven engineering application for cooling load estimation. The proposed neuroevolution scheme aims at evolving to the best generalizing DNN model that well suits the modeling complexity requirements of the dataset. To this end, the objective function for the neural architecture optimization process is simplified to the mean square error of the test dataset, which enables to reduce the risk of insufficient generalization during NAS. By employing this objective function for evolution field optimization (EFO), the proposed neuroevolution process can automatically achieve the optimal model complexity, preventing overfitting and underfitting cases, and thereby attaining almost the best generalization for the dataset. For this purpose, this approach combines parametric learning with the backpropagation algorithm and structural learning with EFO-based neural architecture search to address data-driven, optimal complexity DNN model generation problems. Effectiveness of the method is demonstrated in the cooling load estimation problem of residential buildings, and performances of the optimal DNN models with four objective functions are analyzed. The design of objective function for the best generalizing model is also elaborated.

PSTFormer: A novel parallel spatial-temporal transformer for remaining useful life prediction of aeroengine

In this paper, we tackle the challenging task of reconstructing Received Signal Strength (RSS) maps by harnessing location-dependent radio measurements and augmenting them with supplementary data related to the local environment. This side information includes city plans, terrain elevations, and the locations of gateways. The quantity of available supplementary data varies, necessitating the utilization of Neural Architecture Search (NAS) to tailor the neural network architecture to the specific characteristics of each setting. Our approach takes advantage of NAS’s adaptability, allowing it to automatically explore and pinpoint the optimal neural network architecture for each unique scenario. This adaptability ensures that the model is finely tuned to extract the most relevant features from the input data, thereby maximizing its ability to accurately reconstruct RSS maps. We demonstrate the effectiveness of our approach using three distinct datasets, each corresponding to a major city. Notably, we observe significant enhancements in areas near the gateways, where fluctuations in the mean received signal power are typically more pronounced. This underscores the importance of NAS-driven architectures in capturing subtle spatial variations. We also illustrate how NAS efficiently identifies the architecture of a Neural Network using both labeled and unlabeled data for Radio Map reconstruction. Our findings emphasize the potential of NAS as a potent tool for improving the precision and applicability of RSS map reconstruction techniques in urban environments.

As one of the most widely used applications of the Internet of Things (IoT), intelligent transportation system (ITS) is of great significance for urban traffic planning, traffic control, and traffic guidance. However, widespread traffic congestion occurs with the increased number of vehicles. The traffic flow prediction is a good idea for traffic congestion. Therefore, many schemes have been proposed for accurate and real-time traffic flow prediction, but there still exist many issues, including low accuracy, weak adaptability and inferior real-time. Meanwhile, the complex spatial and temporal dependencies in traffic flow are still challenging. To address the above issues, we propose a novel spatial-temporal Chebyshev graph neural network model (ST-ChebNet) for traffic flow prediction to capture the spatial-temporal features, which can ensure accurate traffic flow prediction. Concretely, we first add a fully connected layer to fuse the features of traffic data into a new feature to generate a matrix, and then the long short-term memory (LSTM) model is adopted to learn traffic state changes for capturing the temporal dependencies. Then, we use the Chebyshev graph neural network (ChebNet) to learn the complex topological structures in the traffic network for capturing the spatial dependencies. Eventually, the spatial features and the temporal features are fused to guarantee the traffic flow prediction. The experiments show that ST-ChebNet can make accurate and real-time traffic flow prediction compared with other eight baseline methods on real-world traffic data sets PeMS.

Human Activity Recognition (HAR) is essential for monitoring and analyzing human behavior, particularly in health applications such as fall detection and chronic disease management. Traditional methods, even those incorporating attention mechanisms, often oversimplify the complex temporal and spatial dependencies in sensor data by processing features uniformly, leading to inadequate modeling of high-dimensional interactions. To address these limitations, we propose a novel framework: the Temporal-Spatial Feature Decoupling Unit with Layer-wise Training Convolutional Neural Network (CNN-TSFDU-LW). Our model enhances HAR accuracy by decoupling temporal and spatial dependencies, facilitating more precise feature extraction and reducing computational overhead. The TSFDU mechanism enables parallel processing of temporal and spatial features, thereby enriching the learned representations. Furthermore, layer-wise training with a local error function allows for independent updates of each CNN layer, reducing the number of parameters and improving memory efficiency without compromising performance. Experiments on four benchmark datasets (UCI-HAR, PAMAP2, UNIMIB-SHAR, and USC-HAD) demonstrate accuracy improvements ranging from 0.9% to 4.19% over state-of-the-art methods while simultaneously reducing computational complexity. Specifically, our framework achieves accuracy rates of 97.90% on UCI-HAR, 94.34% on PAMAP2, 78.90% on UNIMIB-SHAR, and 94.71% on USC-HAD, underscoring its effectiveness in complex HAR tasks. In conclusion, the CNN-TSFDU-LW framework represents a significant advancement in sensor-based HAR, delivering both improved accuracy and computational efficiency, with promising potential for enhancing health monitoring applications.

Understanding crowd behavior is a pivotal step toward urban scene analysis. This is considered a very challenging task and has rarely been addressed to date due to the complexity of motion dynamics co-occurring across a given scene, which involves spatial and temporal dependencies. Unlike the mainstream research, which usually treats the temporal and spatial dependencies in a crowd separately, this paper presents a deep end-to-end approach that jointly considers the spatiotemporal information, leading to a rich understanding of crowd behavior. We first extract the displacement information describing crowd motion patterns from tracklets/trajectories. This information is subsequently fed into a convolutional layer to learn the underlying motion patterns and create high-level representations. The derived representations are used as inputs to a long short-term memory-based architecture to learn the underlying spatiotemporal cues in a single operation for the entire crowd in a given scene. We evaluate our approach on a widely used, large-scale benchmark datasets for three critical applications: pedestrian path forecasting, destination estimation, and holistic crowd behavior classification. The results show a drastic improvement compared to recently trending works.

A latent variable representation of count data models to accommodate spatial and temporal dependence: Application to predicting crash frequency at intersections

The generation of synthetic time series is important in contemporary water sciences for their wide applicability and ability to model environmental uncertainty. Hydroclimatic variables often exhibit highly skewed distributions, intermittency (that is, alternating dry and wet intervals), and spatial and temporal dependencies that pose a particular challenge to their study. Vine copula models offer an appealing approach to generate synthetic time series because of their ability to preserve any marginal distribution while modeling a variety of probabilistic dependence structures. In this work, we focus on the stochastic modeling of hydroclimatic processes using vine copula models. We provide an approach to model intermittency by coupling Markov chains with vine copula models. Our approach preserves first-order auto- and cross-dependencies (correlation). Moreover, we present a novel framework that is able to model multiple processes simultaneously. This method is based on the coupling of temporal and spatial dependence models through repetitive sampling. The result is a parsimonious and flexible method that can adequately account for temporal and spatial dependencies. Our method is illustrated within the context of a recent reliability assessment of a historical hydraulic structure in central Mexico. Our results show that by ignoring important characteristics of probabilistic dependence that are well captured by our approach, the reliability of the structure could be severely underestimated.

In the future, vehicles will be equipped with increasingly advanced interactive intelligent electronic devices, which will induce drivers to conduct secondary tasks, thereby leading to distractions. Therefore, the detection and early warning of driver distraction are essential for improving driving safety and pose an important challenge in intelligent transportation systems. Previous studies used traditional machine learning and deep learning transfer models, which have the disadvantages of complicated and time-consuming manual feature engineering, strong subjectivity, and weak generalization performance. In this paper, we propose a fine-grained detection method for driver distraction based on neural architecture search. First, we design an automatic construction algorithm for deep convolutional neural networks based on neural architecture search, which automatically searches for the optimal deep convolutional neural network architecture without human involvement. In addition, we fuse driver-related multisource perception information, use an automatically constructed deep convolutional neural network to extract high-dimensional mapping features, and implement fine-grained detection of various types of driver distraction states. The results on a large-scale multimodal driver distraction dataset demonstrate that our method can efficiently search an optimal deep convolutional neural network, which can quickly converge, and can accurately detect the considered types of driver distraction states, the average detection accuracy reaches 99.7796%; moreover, it has satisfactory robustness.

Traffic forecasting has emerged as an important task for developing intelligent transportation systems. Recent works focus on representing traffic as graph operation and using graph neural networks for spatial–temporal prediction. Most of the approaches assume a predefined graph structure based on node distances. However, spatial dependencies change over time in many scenarios of traffic flow. In this regard, this study takes an investigation capturing the spatial and temporal dependencies with no prior knowledge structure of traffic road networks. Specifically, we propose a multi-step prediction model named Dynamic Spatial Transformer WaveNet Network (DSTWN) to capture the dynamic conditions and directions of traffic flow in which a temporal convolution layer is adopted for the long time sequence and a spatial transformer layer is proposed to capture the dynamic spatial dependencies. Furthermore, we introduce a new traffic dataset, which is collected from the vehicle detection system in an urban area (UVDS). In particular, compared with existing benchmark traffic data, UVDS contains more complicated spatial information, which is similar to many real-world scenarios of traffic flow. Experiments on both benchmark traffic datasets indicate the promising results of DSTWN compared with state-of-the-art models in this research field.

Traditional resource (demand or supply) forecasting models mainly focus on modeling temporal dependency. However, spatio-temporal data include complex non-linear relational and spatial dependencies. In addition, dynamic contextual information also impacts resources. Methods that consider context assume that the impact of context on resources is fixed, which is not realistic. For example, in a bicycle-sharing system, bike supply in stations is affected by the weather, and that effect changes over time. We propose a novel graph-based context integrated relational model, Context Integrated Graph Neural Network (CIGNN), which models temporal, relational, spatial, and dynamic contextual dependencies for multi-step ahead resource forecasting. We define a resource graph, where nodes represent locations with associated resource time-series, and context graphs (one for each type of context), where nodes represent locations with associated contextual time-series. Assuming that various contexts have dynamic impact on resources, our proposed CIGNN model employs a novel fusion mechanism that jointly learns from multiple contextual time-series. To the best of our knowledge, CIGNN is the first approach that integrates dynamic contextual information using graph neural networks for resource forecasting. Empirical results on two real-world datasets demonstrate that CIGNN consistently outperforms state-of-the-art approaches.

The regional climate change index (RCCI) is employed to investigate hot-spots under 21st century global warming over East Asia. The RCCI is calculated on a 1-degree resolution grid from the ensemble of CMIP3 simulations for the B1, A1B, and A2 IPCC emission scenarios. The RCCI over East Asia exhibits marked sub-regional variability. Five sub-regional hot-spots are identified over the area of investigation: three in the northern regions (Northeast China, Mongolia, and Northwest China), one in eastern China, and one over the Tibetan Plateau. Contributions from different factors to the RCCI are discussed for the sub-regions. Analysis of the temporal evolution of the hot-spots throughout the 21st century shows different speeds of response time to global warming for the different sub-regions. Hot-spots firstly emerge in Northwest China and Mongolia. The Northeast China hot-spot becomes evident by the mid of the 21st century and it is the most prominent by the end of the century. While hot-spots are generally evident in all the 5 sub-regions for the A1B and A2 scenarios, only the Tibetan Plateau and Northwest China hot-spots emerge in the B1 scenario, which has the lowest greenhouse gas (GHG) concentrations. Our analysis indicates that subregional hot-spots show a rather complex spatial and temporal dependency on the GHG concentration and on the different factors contributing to the RCCI.

Predicting crop yield is a crucial factor in precision agriculture, especially under the conditions of increasing environmental uncertainty and climatic variability. This interest in predicting crop yield increased the necessity to develop advanced machine learning and dl models to refine the capabilities of accurate and efficient crop yield prediction. Existing models fail to handle real-time data and multiple types of data like weather conditions and IoT sensor readings. So it is crucial for us to implement a better crop yield prediction framework that can work well in all scenarios. In this paper, an Integrated Multi Stage Transformer (IMST) framework is discussed that uses an advanced Deep Learning (DL) approach. This IMST model uses a self-attention mechanism that captures complex spatial and temporal dependencies. The dataset contains different data like weather conditions, IoT sensor readings. The model is implemented and their performance is assessed using several evaluation metrics like accuracy, R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and F1 score. The transformer model results with an accuracy of 94.3%. It addresses the difficulties commonly found in real-world applications and exhibits good performance with real-time data.