A case study of MapReduce-based expressway traffic data analysis and service system

SUMMARYNetwork traffic monitoring and analysis are essential for effective network operation and resource management. In particular, multidimensional analysis for long‐term traffic data is necessary for comprehensive understanding of the traffic trend and effective quality‐of‐service provision considering the extremely dynamic behavior of the current Internet, where various types of traffic occur from high‐speed network links and greatly increasing number of applications. However, only limited analysis results are provided, as the existing network traffic analysis tools and systems are developed and deployed focusing on their own specialized analysis purposes. Consequently, it is difficult to understand the network comprehensively and deeply, which increases the necessity for multilateral analysis of long‐term traffic data. In this paper, we propose a novel traffic analysis model for large volumes of Internet traffic accumulated over a long period of time. TheNetCube, the proposed network traffic analysis model using online analytical processing (OLAP) on a multidimensional data cube, provides an easy and fast way to construct a multidimensional traffic analysis system for comprehensive and detailed analysis of long‐term traffic data by utilizing simple OLAP operations and powerful data‐mining techniques on various abstraction levels of traffic data to complete the analysis purpose. We validate the feasibility and applicability of the proposedNetCubetraffic analysis model by implementing a traffic analysis system and applying it to our campus network. Copyright © 2012 John Wiley & Sons, Ltd.

This paper designs and implements a network traffic collection and analysis system (NTCA). Network traffic is collected from multiple clients. According to the offline traffic analysis, network flows are analyzed and annotated with application protocol. At the same time, the results are stored in the system. NTCA also provides a visual interface to simplify the configuration process. With good flexibility and scalability, NTCA can be extended to a network traffic classification system, which is crucial to the research of network security technology.

Traffic volume data, especially average annual daily traffic (AADT), are important in transportation engineering. They are required in managing and maintaining existing facilities and in planning and designing new facilities. Many state highway agencies use the ramp counting procedure described in FHWA’s Traffic Monitoring Guide to estimate AADTs for freeways. The procedure involves counting all entrance and exit ramps between two established mainline counters (anchor points) and then reconciling the count data to estimate mainline AADT. The reconciling of count data includes three steps. First, AADTs for the ramps and the anchor points are estimated from the count data. Then AADT for each uncounted mainline link is calculated by addition or subtraction of ramp AADT to or from mainline AADT, starting from one anchor point. Finally, adjustments of the AADT are performed to achieve a match at the second anchor point if necessary. The process can be time-consuming and labor-intensive if it is done manually. A computer program to automate the process is required. The traffic count analysis system (TCAS) developed to automate the reconciling of count data in the ramp counting process is described. The TCAS was developed on the basis of data coaptation and data reconciliation techniques frequently used in the processing of network flow rate data. Data coaptation is used to calculate flow rates for uncounted links, and data reconciliation is used to adjust and balance the flow rates. The TCAS has been tested for the two longest freeways in Connecticut. The results are close to those from the ramp counting procedure. However, the TCAS significantly reduces the time and labor required for processing traffic volume data for freeways.

With the rapid development of cities, massive and complex traffic data is being generated and collected. The traffic data is not intuitive and cannot highlight key information about urban traffic conditions. However, traffic data visualization can directly correlate users with the data, and support users to interact with data in a convenient and visual way. Then realize the feedback of blending user wisdom and machine intelligence. This paper investigates a structured survey of the state of the art in the visualization of traffic data. First, we reviewed five representative traffic data visualization methods including WebVRGIS based traffic analysis and visualization system, TripMiner, IoV distributed architecture, SMASH architecture, and LDA-based topic modelling. Meanwhile, we analyzed the traffic datasets that applied in each method. Then we summarize these methods from seven aspects: scalability, data storage, data update, interactivity, reliability, data anomaly detection, and spatiotemporal visualization. In addition, we make a detailed comparative analysis of the key capabilities of five representative traffic data visualization methods in processing traffic big data. Finally, we conclude that the SMASH architecture performs better in processing high speed and large flow traffic data. Moreover, we propose a novel direction for optimizing traffic data visualization techniques.

The urban intelligent traffic decision analysis system is a system designed to solve urban road traffic problems, improve traffic safety and comfort, and improve the level of traffic services. The application of big data mining technology can improve the ability of various types of road traffic information analysis and processing, expand the scope of application, and provide decision-making reference for the application of intelligent transportation systems, so as to ensure the smoothness and safety of roads. This article will introduce the concept of big data mining technology and intelligent transportation system, and analyze the architecture of urban traffic intelligent system based on big data technology and the design and implementation of mining system.

The lack of current network dynamics studies that evaluate the effects of new application and protocol deployment or long-term studies that observe the effect of incremental changes on the Internet, and the change in the overall stability of the Internet under various conditions and threats has made network monitoring challenging. A good understanding of the nature and type of network traffic is the key to solving congestion problems. In this paper we describe the architecture and implementation of a scalable network traffic moni-toring and analysis system. The gigabit interface on the monitoring system was configured to capture network traffic and the Multi Router Traffic Grapher (MRTG) and Webalizer produces graphical and detailed traffic analysis. This system is in use at the Obafemi Awolowo University, IleIfe, Nigeria; we describe how this system can be replicated in another environment.

This paper proposes a method of applying a machine learning algorithm in order to improve the recognition rate of the video-based traffic information system. After applying the Error Backpropagation learning neural network algorithm to the traffic information it will be used for image recognition results. The training data is generated from the traffic information system, the noise of the generated data is removed by Gaussian smoothing. In this paper, we develop a machine learning based Traffic Condition analysis system was able to get an improved recognition rate than conventional vision-based system.

Traffic accidents cause enormous losses for our country and plenty of national assets drain away every year, and therefore the task of traffic management is weightier than Mount Tai, but the complexity of traffic accident analysis has brought many difficulties to traffic management and decision-making. A three-layer analysis system based on spatial data mining of GIS is proposed in this paper, this three-layer architecture displays the whole process of accident data extracting, preprocessing and mining and it applies spatial data mining to GIS, which will certainly further expand the width and depth of GIS application field. Finally, this paper introduces the method of developing traffic accident analysis system by using ArcGIS Engine and C#.NET and gives the class realization of system main functions.

Traffic Flow Analysis System is an important part of Intelligent Transportation Systems. It mainly contains two parts: vehicle detection part and vehicle tracking part. The efficient video object segmentation algorithm is the basis of vehicle detection part that aims to segment each vehicle correctly. In this paper, according to the street traffic environment, we adopt a multiple video object segmentation algorithm in vehicle detection. The algorithm consists of training section and segmentation section. At the training section, it uses codebook method to train the vehicle images: First, it uses Scale-invariant feature transform method to extract the features of vehicle image samples. Secondly, it segments the images into small patches to cluster. Third, it uses a new way to activate the codebook that can largely reduce the numbers of vehicle sample images. At the segmentation section, it uses an Implicit Shape Model to combine the recognition knowledge and the segmentation knowledge together. At last, we use this algorithm to implement a prototype of traffic flow analysis system, and achieve the expected results.

Traditional models of usability assume that usability is a quality that can be designed into a particular artifact. Yet constructivist theory implies that usability cannot be located in a single artifact; rather, it must be conceived as a quality of the entire activity in which the artifact is used. This article describes a distributed approach to usability, based on activity theory and genre theory. It then illustrates the approach with a four-decade examination of a traffic accident location and analysis system (ALAS). Using the theoretical framework of genre ecologies, the article demonstrates how usability is distributed across the many official and unofficial (ad hoc) genres employed by ALAS users.

In recent years, deep learning-based traffic anomaly detection has proven very promising. Although current methods achieve high accuracy in detecting anomalies, they struggle to accurately classify attack types of anomalies due to the imbalanced distribution of attack samples. To address the issue, we propose a highly intelligent system, ALM, which can simultaneously provide accurate traffic anomaly detection and attack types classification with the aid of the Large Language Model (LLM)'s few-shot learning ability. To tackle the challenges of training cost and inference efficiency associated with large models, ALM adopts a two-stage solution, i.e., AnomalyDetector and Anomaly Analyzer, that combines the fine-tuned LLM with small models. In the first stage, AnomalyDetector ensembles a set of lightweight models to handle high-concurrency real-time network traffic anomaly detection. In the second stage, Anomaly Analyzer leverages the LLM's powerful fitting and few-shot learning abilities for traffic anomaly analysis through three processes: LLM task adaption, traffic to sequence, and LLM fine-tuning. This allows Anomaly Analyzer to accurately identify the attack types and potential false positives. Experimental results indicate that ALM achieves over 90% Micro-F1 on four public datasets, with a maximum of 99.94 %, surpassing the baseline. Additionally, it requires minimal training costs while significantly improving inference efficiency compared to the pure LLM mode.

The shortest path between the start node and end node plays an important role in city’s road traffic network analysis system. The basic ant colony system algorithm which is a novel simulated evolutionary algorithm is studied to solve the shortest path problem. But the basic ant colony system algorithm is easy to run into the local optimum solution for shortest path. In order to solve the problem, the improved ant colony system algorithm is proposed. The improvement methods for selection strategy, local search, and information quantity modification of basic ant colony system are discussed in detail. The experiments are done in Beijing road network in China. The results of experiments show that comparing with the basic ant colony algorithm, the improved algorithm can easily converge at the global optimum for the shortest path.

Not only the leading countries but also Korea has been constructing Information Superhighway for national competitiveness and giving better quality of living to its citizens. Concerning this, many Korean Information Infrastructure (KII) pilot projects have been beginning to realize the vision of the information society. Especially, the government and Korea Telecom provided KII field trial services to the subscribers in local residential areas, in university and in the research institute in Taeduk Science Town. The applications that the pilot project are supporting include video phone, electronic news, high speed PC communications, high speed file transfer and digital library services, in which B-ISDN technology is applied. This paper presents the system analysis tasks, design issues for the service and traffic analysis system on this field trial.

Traffic monitoring is an increasingly important discipline for nowadays networking, as Accounting, Security and Traffic Engineering lay on it. Besides, traffic bandwidth has increased exponentially in the last few years, and high-speed network monitoring has become a challenging task. Performance requirements are highly relevant for passive QoS monitoring systems. A low-level study of the capturing and processing stages on a traffic analysis system (TAS) has shown room for improvement. We provide an architecture able to cope with high-speed traffic monitoring using commodity hardware. Our system is intended to exploit the parallelism available in up-to- date workstations, which also introduces constraints for multithreaded QoS analysis. This paper presents a kernel-level framework (ksensor) that, keeping the previous requirements, removes some issues from user-level processing and effectively integrates QoS algorithms, improving the overall performance.

The increasing complexity of urban transportation systems and the growing volume of vehicles have made traffic congestion a persistent challenge in modern cities. Efficient traffic flow prediction is essential for mitigating congestion, improving road safety, optimizing traffic signal control, and enhancing overall transportation efficiency. In recent years, artificial intelligence (AI) has emerged as a transformative tool in the field of traffic management, offering sophisticated algorithms capable of modeling, analyzing, and predicting complex traffic patterns with high accuracy. The application of AI in traffic flow prediction leverages vast amounts of real-time and historical data to generate precise forecasts, supporting data-driven decision-making by urban planners and traffic control authorities. The prediction of traffic flow involves analyzing time-series data that exhibit nonlinear, dynamic, and often stochastic behavior. Traditional statistical models, such as autoregressive integrated moving average (ARIMA), have proven to be limited in handling the high dimensionality and variability inherent in traffic systems. In contrast, AI algorithms possess the capacity to learn and adapt from complex data inputs without the need for explicit programming, making them particularly suitable for traffic-related applications. AI algorithms used in traffic flow prediction can be broadly categorized into machine learning (ML) and deep learning (DL) approaches. Machine learning algorithms such as k-nearest neighbors (KNN), support vector machines (SVM), decision trees, and random forests have demonstrated effectiveness in short-term traffic prediction tasks. These algorithms are capable of identifying hidden patterns in traffic data and adjusting to changes in traffic behavior over time. Ensemble methods, which combine the strengths of multiple learning models, further enhance prediction accuracy and robustness. Deep learning algorithms, a subfield of AI inspired by the human brain’s neural architecture, have shown exceptional performance in capturing spatial-temporal dependencies in traffic data. Recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks and gated recurrent units (GRUs), are widely used for their ability to process sequential data and retain information over extended time intervals. Convolutional neural networks (CNNs) are employed to extract spatial features from traffic sensor data or road network imagery. Hybrid models that integrate CNNs with RNNs have achieved high levels of predictive precision by simultaneously learning spatial and temporal correlations. In addition to supervised learning methods, unsupervised and reinforcement learning techniques are also applied in traffic flow prediction. Clustering algorithms, such as k-means and DBSCAN, assist in identifying traffic patterns, while reinforcement learning models optimize adaptive traffic signal control systems by learning optimal actions through environmental interaction. This study explores the different types of AI algorithms used in traffic flow prediction, examining their theoretical foundations, structural differences, and practical applications. It aims to evaluate the comparative advantages of various algorithms in addressing the challenges of real-time traffic prediction in increasingly complex transportation networks.