Filling the gaps

ABSTRACTIn recent years, call detail records (CDRs) have been widely used in human mobility research. Although CDRs are originally collected for billing purposes, the vast amount of digital footprints generated by calling and texting activities provide useful insights into population movement. However, can we fully trust CDRs given the uneven distribution of people’s phone communication activities in space and time? In this article, we investigate this issue using a mobile phone location dataset collected from over one million subscribers in Shanghai, China. It includes CDRs (~27%) plus other cellphone-related logs (e.g., tower pings, cellular handovers) generated in a workday. We extract all CDRs into a separate dataset in order to compare human mobility patterns derived from CDRs vs. from the complete dataset. From an individual perspective, the effectiveness of CDRs in estimating three frequently used mobility indicators is evaluated. We find that CDRs tend to underestimate the total travel distance and the movement entropy, while they can provide a good estimate to the radius of gyration. In addition, we observe that the level of deviation is related to the ratio of CDRs in an individual’s trajectory. From a collective perspective, we compare the outcomes of these two datasets in terms of the distance decay effect and urban community detection. The major differences are closely related to the habit of mobile phone usage in space and time. We believe that the event-triggered nature of CDRs does introduce a certain degree of bias in human mobility research and we suggest that researchers use caution to interpret results derived from CDR data.

A Call Detail Record (CDR) is a data record produced by telecommunication equipment consisting of detail of call transaction logs. It contains valuable information for many purposes of several domains such as billing, fraud detection and analytical purposes. However, in the real world, these needs face a big data challenge. Billions CDRs are generated every day and the processing systems are expected to deliver results in a timely manner. In our case, the system also has constraint that is running in limited computation resources. We found that our current production system was not enough to meet these needs. We had successfully analyzed the current system bottleneck and found the root cause. Based on this analysis, we designed and implemented a better performance system which is based on MapReduce and running on Hadoop cluster. This paper presents the analysis of previous system and the design and implementation of new system, called MS2. In this paper, we also provide empirical evidence demonstrating the efficiency and linearity of MS2. In a test case of telecommunication mediation system, our test has shown that MS2 reduces overhead by 44% and speedup performance by nearly twice compared to previous system. From benchmarking with several related technologies in large scale data processing, MS2 is also shown to perform better in case of CDR batch processing. Running on a cluster consists of eight core CPU and two conventional disks, MS2 is able to process 67,000 CDRs/second.

Billing Mediation Platform (BMP) in Telco is used to process real-time streams of Call Detail Records (CDRs) which can number tens of billions a day. The comprehensive records generated by BMPs can be used for billing and accounting, fraud detection, campaign management, spam filtering, traffic analysis, and churn prediction. Many of these applications are characterized by real-time processing requiring high throughput, low-latency analysis of CDRs. Testing such BMPs has different dimensions, stress testing of analytics for scalability, correctness of analytics, what-if scenarios, all of which require CDRs with realistic volumetric and contextual properties. We propose WLG, a framework for testing and benchmarking BMPs which involves generating high volumes of CDRs representative of real-world data. The framework is flexible in its ability to express and tune the workload generation to simulate CDRs from broad range of traffic patterns while preserving different spatio-temporal correlations and content-level information observed in real-world CDRs.

With the increasing use of mobile devices, now it is possible to collect different data about the day-to-day activities of personal life of the user. Call Detail Record (CDR) is the available dataset at large-scale, as they are already constantly collected by the mobile operator mostly for billing purpose. By examining this data it is possible to analyze the activities of the people in urban areas and discover the human behavioral patterns of their daily life. These datasets can be used for many applications that vary from urban and transportation planning to predictive analytics of human behavior. In our research work, we have proposed a hierarchical analytical model where this CDR Dataset is used to find facts on the daily life activities of urban users in multiple layers. In our model, only the raw CDR data are used as the input in the initial layer and the outputs from each consecutive layer is used as new input combined with the original CDR data in the next layers to find more detailed facts, e.g., traffic density in different areas in working days and holidays. So, the output in each layer is dependent on the results of the previous layers. This model utilized the CDR Dataset of one month collected from the Dhaka city, which is one of the most densely populated cities of the world. So, our main focus of this research work is to explore the usability of these types of dataset for innovative applications, such as urban planning, traffic monitoring and prediction, in a fashion more appropriate for densely populated areas of developing countries.

In the era of information explosion, huge amount of data are generated from various sensing devices continuously, which are often too low level for analytics purpose, and too massive to load to data-warehouses for filtering and summarizing with the reasonable latency. Distributed stream analytics for multilevel abstraction is the key to solve this problem. We advocate a distributed infrastructure for CDR (Call Detail Record) stream analytics in the telecommunication network where the stream processing is integrated into the database engine, and carried out in terms of continuous querying; the computation model is based on network-distributed (rather than clustered) Map-Reduce scheme. We propose the window based cooperation mechanism for having multiple engines synchronized and cooperating on the data falling in a common window boundary, based on time, cardinality, etc. This mechanism allows the engines to cooperate window by window without centralized coordination. We further propose the quantization mechanism for integrating the discretization and abstraction of continuous-valued data, for efficient and incremental data reduction, and in turn, network data movement reduction. These mechanisms provide the key roles in scaling out CDR stream analytics. The proposed approach has been integrated into the PostgreSQL engine. Our preliminary experiments reveal its merit for large-scale distributed stream processing.

The ubiquitous massive mobile phone data generation presents new opportunities to determine the requirements of transportation, disaster management and public health care systems. Currently, data from mobile phone records can help in identifying the location of the users while they are making trips. Generally, this estimation is achieved using traditional data collection methods; however, these methods are difficult to apply in developing countries with rapidly growing cities owing to the high population and limitation in conducting a survey. Call detail records (CDRs) are used as base data because they are valuable data sources and can reduce the cost and time limitations. The aim of this study is to estimate origin-destination (OD) trips from each zone by using the CDRs. The OD trips are estimated by using the CDRs of one week taken from Myanmar Post and Telecommunication mobile operator for over 1.9 million users per day in Yangon, the economic center of Myanmar. The OD trips are estimated from CDRs based on the location of the base station in a limited time window and time frame. If the same mobile users is observed in two different the ones within the time limit, it is assumed that the mobile user is coming out from the first zone and the trips represents an originating trip. This trip would be the destination trip for zone where the mobile user enters. In this study, the originating (outgoing) and destination trips (incoming) from each township on a weekday and weekend are determined. These data are useful for infrastructure development and urban transportation planning.

Machine learning techniques have been used to predict customer behavior in telecommunications industry. Typically, several data sources, including historical usage, billing, payment, network, customer service, and demographic data, can be used in a predictive model. However, in some cases, e.g., in the prepaid customer segment, there is often little data available except for the call detail record (CDR) data. In this paper, we tackle this challenging problem, using significantly delayed CDR data as the primary data source to predict customer behavior. We extract calling links, i.e., who called whom, from the CDR data, and propose several distance measures based on calling links. We demonstrate that, by using information derived from the calling links alone as inputs to a neural network model, an acceptable accuracy for predicting churn (customer switching from one service provider to another) can be achieved. Calling links can also be used to identify calling communities, which may be used for targeted marketing campaigns and help predict acceptance of marketing offers.

Existing predictions of human behaviour trajectories in mobile communication scenarios are mainly based on rules and Markov statistical methods. At the same time, less research has been conducted on CDR data using deep learning methods. This paper proposes the A-TPN (Adaptive Trajectory Prediction Network) algorithm to predict human trajectories for the large scale and sparsity of Call Detail Records (CDR) data. Experiments show that A-TPN achieves a prediction accuracy of 94.76% on real datasets, which is 8.7% better than the traditional Markov statistical computation method. This paper demonstrates the algorithm's effectiveness through rich comparative experiments and analysis.

This paper deals with two algorithms for monitoring the QoS and based on it to detect failures in voice communication systems. These algorithms are based on the analysis of data stored in Call Detail Records (CDR). For each call one CDR or ticket is generated. These tickets contain data related to the call describing the system elements involved, such as time and duration of the call, phone types and numbers, SS7 signaling, trunks, time slots, what happen to the call, etc. The tickets are generated in PSTN switches or over VoIP gateways, for the case of Internet Protocol Detail Record (IPDRs). These tickets can be used to monitor the QoS and to detect faults focusing in different aspects related to the call, such as technical, economic, or social. Our main goal is to analyze and classify these algorithms according to their performance and use.

Data supplied by mobile phones have become the basis for identifying meaningful places frequently visited by individuals. In this study, we introduce SAMPLID, a new Supervised Approach for Meaningful Place Identification, based on providing a knowledge base focused on the specific problem we aim to solve (e.g., home/work identification). This approach allows to tackle place identification from a supervised perspective, offering an alternative to unsupervised clustering techniques. These clustering techniques rely on data characteristics that may not always be directly related to classification objectives. Our results, using mobility data provided by call detail records (CDRs) from Milan, demonstrate superior performance compared to applying clustering techniques. For all types of CDRs, the best results are obtained with the 20 × 20 subgrid, indicating that the model performs better when supplied with information from neighboring cells with a close spatial relationship, establishing neighborhood relationships that allow the model to clearly learn to identify transitions between cells of different types. Considering that it is common for a place or cell to be labeled in multiple categories at once, this supervised approach opens the door to addressing the identification of meaningful places from a multi-label perspective, which is difficult to achieve using classical unsupervised methods.

Drive Test (DT) is a very important approach to measure the quality of the wireless networks. But with the rapid development of wireless networks, the network structure comes to more and more complicated, such as massive network, different systems and multiple manufacturers. The complexity of network leads to a mass expenditure in operation and maintenance which including DT. In this paper, we study a new method, called Virtual Drive Test (VDT) by using Measurement Report (MR) and Call Detail Record (CDR), to measure the quality of the wireless networks in more efficient and less costly ways.

This paper uses big data technologies to study base stations’ behaviors and activities and their predictability in mobile cellular networks. With new technologies quickly appearing, current cellular networks have become more larger, more heterogeneous, and more complex. This provides network managements and designs with larger challenges. How to use network big data to capture cellular network behavior and activity patterns and perform accurate predictions is recently one of main problems. To the end, first we exploit big data platform and technologies to analyze cellular network big data, i.e., Call Detail Records (CDRs). Our CDRs data set, which includes more than 1,000 cellular towers, more than million lines of CDRs, and several million users and sustains for more than 100 days, is collected from a national cellular network. Second, we propose our methodology to analyze these big data. The data pre-handling and cleaning approach is proposed to obtain the valuable big data sets for our further studies. The feature extraction and call predictability methods are presented to capture base stations’ behaviors and dissect their predictability. Third, based on our method, we perform the detailed activity pattern analysis, including call distributions, cross correlation features, call behavior patterns, and daily activities. The detailed analysis approaches are also proposed to dig out base stations’ activities. A series of findings are found and observed in the analysis process. Finally, a study case is proposed to validate the predictability of base stations’ behaviors and activities. Our studies demonstrates that big data technologies can indeed be utilized to effectively capture network behaviors and predict network activities so that they can help perform highly effective network managements.

Base on introducing the architecture of Next Generation Network (NGN)signaling monitoring system, the paper analyzes the Bearer Independent Call Control (BICC) protocol and signaling procedure. The paper innovativly puts forward an efficient synthetic method of Call Detail Records (CDR) by using Hash technology. At the same time, the scheme uses selectively storing the BICC's CDR messages. This application shows that it can work well in the real network.

Mobile phones are widely used in our day-to-day life. It's not only used by a common man but also used by antisocial elements and that's why it's not a surprise that today in almost every case the first step towards solving a Crime is to analyze the Call Records of the Suspects. Today in almost all the criminal cases, analysis of Mobile Phone calls of suspect's plays an important role in investigation of crime. In order to track the suspects/criminal, investigative agency need to analyze Call Detail Record (CDR) of the suspect's received in varied formats from the service providers. It is generally found that the anti-social elements have their own network and association with other criminals and anti-social elements. Many time they are associated together for committing a crime or they may be knowing about each other activities and crimes. Many of their associates might have been convicted or their name might have been recorded in old cases. Archiving the CDR in centralize database benefit the investigative agency for identification of possible suspects in other cases. This research paper discuss the implementation model for investigating the case using achieve CDR.

Predicting complex user behavior from CDR based social networks