Streaming workload generator for testing Billing Mediation Platform in telecom industry

A Call Detail Record (CDR) is a data record produced by telecommunication equipment consisting of detail of call transaction logs. It contains valuable information for many purposes of several domains such as billing, fraud detection and analytical purposes. However, in the real world, these needs face a big data challenge. Billions CDRs are generated every day and the processing systems are expected to deliver results in a timely manner. In our case, the system also has constraint that is running in limited computation resources. We found that our current production system was not enough to meet these needs. We had successfully analyzed the current system bottleneck and found the root cause. Based on this analysis, we designed and implemented a better performance system which is based on MapReduce and running on Hadoop cluster. This paper presents the analysis of previous system and the design and implementation of new system, called MS2. In this paper, we also provide empirical evidence demonstrating the efficiency and linearity of MS2. In a test case of telecommunication mediation system, our test has shown that MS2 reduces overhead by 44% and speedup performance by nearly twice compared to previous system. From benchmarking with several related technologies in large scale data processing, MS2 is also shown to perform better in case of CDR batch processing. Running on a cluster consists of eight core CPU and two conventional disks, MS2 is able to process 67,000 CDRs/second.

Machine learning techniques have been used to predict customer behavior in telecommunications industry. Typically, several data sources, including historical usage, billing, payment, network, customer service, and demographic data, can be used in a predictive model. However, in some cases, e.g., in the prepaid customer segment, there is often little data available except for the call detail record (CDR) data. In this paper, we tackle this challenging problem, using significantly delayed CDR data as the primary data source to predict customer behavior. We extract calling links, i.e., who called whom, from the CDR data, and propose several distance measures based on calling links. We demonstrate that, by using information derived from the calling links alone as inputs to a neural network model, an acceptable accuracy for predicting churn (customer switching from one service provider to another) can be achieved. Calling links can also be used to identify calling communities, which may be used for targeted marketing campaigns and help predict acceptance of marketing offers.

<title>Abstract</title> Telecommunication companies globally face the critical challenge of subscription fraud, which threatens both financial stability and national security. This research addresses this issue by developing an advanced fraud detection model specifically for Ethio Telecom. The model utilizes Ensemble and Adaptive Learning techniques to enhance detection accuracy by combining multiple classifiers. The study used a dataset of 1,000,000 Call Detail Records (CDRs) collected over two months known for increased fraudulent activity3. After filtering out irrelevant data and aggregating multiple call records per subscriber, the dataset was refined to 349,164 records. Initially, 16 features were analyzed, with four excluded for lacking relevance. The remaining 11 features, excluding the target variable, underwent preprocessing including data cleaning, transformation, and balancing4. Feature selection, utilizing Correlation Matrix and Random Forest importance analysis, led to the removal of four additional features, resulting in a final set of 8 key features, including INT_DIALLED, RATIO_INT_TOTAL, and RATIO_UNIQUE_TOTAL4. Three individual models, namely Decision Tree (DT), Logistic Regression (LR), and Artificial Neural Network (ANN), were implemented alongside ensemble methods such as Bagging, Boosting, Stacking, and Voting, and adaptive models like Hoeffding Tree and Adaptive Random Forest45. The findings of this research recommend Stacking and Adaptive Random Forest (ARF) as robust tools for subscription fraud detection.

Telecommunication companies globally face the critical challenge of subscription fraud, which threatens both financial stability and national security. This research addresses this issue by developing an advanced fraud detection model specifically for Ethio Telecom. The model utilizes Ensemble and Adaptive Learning techniques to enhance detection accuracy by combining multiple classifiers. The study used a dataset of 1,000,000 Call Detail Records (CDRs) collected over two months known for increased fraudulent activity3. After filtering out irrelevant data and aggregating multiple call records per subscriber, the dataset was refined to 349,164 records. Initially, 16 features were analyzed, with four excluded for lacking relevance. The remaining 11 features, excluding the target variable, underwent preprocessing including data cleaning, transformation, and balancing4. Feature selection, utilizing Correlation Matrix and Random Forest importance analysis, led to the removal of four additional features, resulting in a final set of 8 key features, including INT_DIALLED, RATIO_INT_TOTAL, and RATIO_UNIQUE_TOTAL4. Three individual models, namely Decision Tree (DT), Logistic Regression (LR), and Artificial Neural Network (ANN), were implemented alongside ensemble methods such as Bagging, Boosting, Stacking, and Voting, and adaptive models like Hoeffding Tree and Adaptive Random Forest45. The findings of this research recommend Stacking and Adaptive Random Forest (ARF) as robust tools for subscription fraud detection.

Companies are investing more in analytics to obtain a competitive edge in the market and decision makers are required better identification among their data to be able to interpret complex patterns more easily. Alluring thousands of new customers is worthless if an equal number is leaving. Business Intelligence (BI) systems are unable to find hidden churn patterns for the huge customer base. In this paper, a decision support system has been proposed, which can predict the churning behaviour of a customer efficiently. We have proposed a procedure to develop an analytical system using data mining as well as machine learning techniques C5, CHAID, QUEST, and ANN for the churn analysis and prediction for the telecommunication industry. Prediction performance can be significantly improved by using a large volume and several features from both Business Support Systems (BSS) and Operations Support Systems (OSS). Extensive experiments are performed; marginal increases in predictive performance can be seen by using a larger volume and multiple attributes from both Telco BSS and OSS data. From the results, it is observed that using a combination of techniques can help to figure out a better and precise churn prediction model.

Churn prediction is an important task for Customer Relationship Management (CRM) in telecommunication companies. Accurate churn prediction helps CRM in planning effective strategies to retain their valuable customers. However, churn prediction is a complex and challenging task. In this paper, a hybrid churn prediction model is proposed based on combining two approaches; Neighborhood Cleaning Rules (NCL) and Particle Swarm Optimization (PSO). NCL is applied in the preprocessing stage for handling the imbalanced churn data; and eliminating outliers and unrepresentative data. In the next stage, a Constricted PSO is applied for developing the final prediction model. The developed model is evaluated and compared with a baseline PSO model. The proposed hybrid model is compared also with Artificial Neural Networks (ANN) and Decision trees (DT) models which are traditional and common approaches used in the literature for churn prediction. The experimental results show that the proposed hybrid model outperforms the baseline PSO model, ANN and DT in terms of accuracy and actual churn rate.

Objective: Churn prediction process on telecom industry is performed using background knowledge such as user information and application oriented constraints with the aim of clustering churn items of telecom communication users. Methods: This paper proposes Semi-supervised Constraint based Churn Clustering (SCCC) method. Semi-supervised learning method identifies different types of churns for labelled data items in telecom industry. PCK Mean based Clusteringwith Expectation Maximization (EM) algorithm finds cluster labels and distance metric for easy grouping of faults set. Similar types of issues are mapped with fast traversal procedure. Constraint based Cluster Membership achieves effective churns clustering by considering relative pairing of users. Findings: The proposed Semi-supervised Constraint based Churn Clustering (SCCC) method is implemented by using JAVA language. This JAVA language uses the code to effectively identify the churns in telecom industry. The SCCC method uses a churn dataset for conducting the experimental work.Experiment is conducted on the factors such as execution time, accuracy, clustering efficiency and support rate on predicting the churns in telecom industry. Experimental results show that the proposed Semi-supervised Constraint based Churn Clustering (SCCC) method out performs than the existing methods. Improvement: Proposed method is able to minimize the execution time, improve accuracy and clustering efficiency and also increase the support rate on predicting churns in telecom industry.Keywords: Cluster Membership, Churn Prediction, Expectation Maximization, Semi-Supervised Clustering

Owing to saturated markets, fierce competition, dynamic criteria, along with introduction of new attractive offers, the considerable issue of customer churn was faced by the telecommunication industry. Thus, an efficient Churn Prediction (CP) model is required for monitoring customer churn. Therefore, this work proposes a novel framework to predict customer churn through a deep learning model namely Swish Recurrent Neural Network (S-RNN). Finally, SRNN is adapted to classify the Churn Customer (CC) and a normal customer. If the result is a churn customer, network utilisation history is analysed for retention process. Whereas, the number of churn customers based on the area network usage is not recognised in this frameworkOwing to saturated markets, fierce competition, dynamic criteria, along with introduction of new attractive offers, the considerable issue of customer churn was faced by the telecommunication industry. Thus, an efficient Churn Prediction (CP) model is required for monitoring customer churn. Therefore, this work proposes a novel framework to predict customer churn through a deep learning model namely Swish Recurrent Neural Network (S-RNN). Finally, S-RNN is adapted to classify the Churn Customer (CC) and a normal customer. If the result is a churn customer, network utilisation history is analysed for retention process.

Competitive markets have driven companies to engage in customer retention. Therefore, anticipating client churn (i.e., cancellation) became a significant challenge. Sequential pattern detection is one of the most popular approaches to the churn prediction problem. Although promising, initiatives based on this approach do not take into account information that can be useful to detect churn: the underlying sentiment in the interactions between client and company. This study raises the hypothesis that considering such information may improve churn prediction, since they may indicate the client's satisfaction level with services and products. Hence, the present study aims to evaluate the validity of the formulated hypothesis. For this purpose, it applies a method that generates churn prediction models from the combination of sequential pattern detection with sentiment extraction from customer-company interactions. Experimental results from telecom data confirm the adequacy of the generated models and of the raised hypothesis.

Churn prediction in telecom using Random Forest and PSO based data balancing in combination with various feature selection strategies

With the rising competition in business, particularly in the telecommunications industry, there has been a growing emphasis on churn prediction. This is attributed to the higher cost involved in attracting new customers than retaining the remaining ones. In telecom churn analysis, the primary goal is to accurately estimate churn behavior by identifying customers who are at risk of leaving. Another objective is to determine the primary reasons for customer churn. Manually predicting the churn in telecommunications is expensive, tedious, and time-consuming. To relieve the burden, machine learning algorithms are introduced to tackle this problem. This article examines several machine learning algorithms for predicting customer churn, including Random Forest, Decision Tree, and Naive Bayes. By constructing and comparing these classification models, the effectiveness of these algorithms is demonstrated. The aim of this work is to examine and compare the effectiveness of various models according to accuracy, which can be applied in real-life scenarios. The result shows that Naive Bayes outperforms the other models.

The Telecommunications (telecom) Industry is saturated and marketing strategies are focusing on customer retention and churn prevention. Churning is when a customer stops using a company’s service thereby opting for the next available service provider. This churn threat has led to various Customer Churn Prediction (CCP) studies and many models have been developed to predict possible churn cases for timely retention strategies. This review looks at the existing models in literature, using 30 selected CCP studies particularly from 2014 to 2017. Data preparation methods and churn prediction challenges have also been explored. This study reveals that Support Vector Machines, Naïve Bayes, Decision Trees and Neural Networks are the mostly used CCP techniques. Feature selection is the mostly used data preparation method followed by Normalization and Noise removal. Undersampling is the mostly preferred technique for handling telecom data class imbalances. Imbalanced, large and high dimensional datasets are the key challenges in telecom churn prediction. The issue of large telecom datasets is handled by use of new big data technologies such as Hadoop, Hbase and NoSQL (Not only Structured Query Language) though their adoption is still at a low rate.

Customer segmentation based on temporal variation of subscriber preferences is useful for communication service providers (CSPs) in applications such as targeted campaign design, churn prediction, and fraud detection. Traditional clustering algorithms are inadequate in this context, as a multidimensional feature vector represents a subscriber profile at an instant of time, and grouping of subscribers needs to consider variation of subscriber preferences across time. Clustering in this case usually requires complex multivariate time series analysis‐based models. Because conventional time series clustering models have limitations around scalability and ability to accurately represent temporal behaviour sequences (TBS) of users, that may be short, noisy, and non‐stationary, we propose a latent Dirichlet allocation (LDA) based model to represent temporal behaviour of mobile subscribers as compact and interpretable profiles. Our model makes use of the structural regularity within the observable data corresponding to a large number of user profiles and relaxes the strict temporal ordering of user preferences in TBS clustering. We use mean‐shift clustering to segment subscribers based on their discovered profiles. Further, we mine segment‐specific association rules from the discovered TBS clusters, to aid marketers in designing intelligent campaigns that match segment preferences. Our experiments on real world data collected from a popular Asian communication service provider gave encouraging results.

This article explores the transformative impact of Artificial Intelligence (AI) on subscription-based business models across various industries. It examines how AI is revolutionizing key aspects of subscription services, including personalization, customer retention, pricing strategies, customer support, operational efficiency, and fraud detection. The article highlights specific AI applications such as content recommendations, dynamic user interfaces, churn prediction, advanced customer segmentation, dynamic and usage-based pricing, AI-powered chatbots, and sentiment analysis. Additionally, it discusses how AI enhances operational efficiency through automated billing and inventory management, and improves security via anomaly detection and behavioral biometrics. Case studies of Adobe Creative Cloud and Amazon Web Services (AWS) are presented to illustrate real-world applications of AI in subscription services. The article concludes by emphasizing the paradigm shift AI represents in customer engagement, operational optimization, and revenue generation for subscription businesses, forecasting continued innovation and opportunities in this rapidly evolving sector.

The prevalence of fraud activities in telecommunication industry has reached a critical point so that efficient algorithms to identify such activities are greatly needed. In this article, we propose a new fraud detection algorithm using Gaussian mixed model (GMM), a probabilistic model successfully used in speech recognition problem. The expectation maximization algorithm is used to estimate the parameter of the model such that the initial values of the algorithm is determined using the kernel method. Using data obtained from one of the leading telecommunication companies in Malaysia, we show that the proposed algorithm has successfully not only detected fraud calls as suspected by the company, but also to identify suspicious calls which can be candidates of fraud call. The proposed algorithm is easy to implement with a great potential to be extended to detect (billed/outgoing) fraud calls and hence reduces the lost incurred by the telecommunication companies.