Big Data Analytics for Business Growth: Leveraging Large-Scale Data for Competitive Advantage

Nowadays, a large volume of data is generated in the form of text, voice, video, images, and sound. It is a very challenging job to handle and to get processed these different types of data. It is a very laborious process to analyze big data by using traditional data processing applications. Due to huge scattered file systems, a big data analysis is a difficult task. So, to analyze big data, a number of tools and techniques are required, Hadoop, Apache Spark, MongoDB, Cassandra are the tools used to handle big data . Keywords: Apache Spark, Cassandra, Hadoop, MongoDB

The sheer boom in data creation from multiple sources beginning from IoT sensors to social media has changed the technology landscape, rendering predictive analytics in big data environments more important than ever before. This work suggests a hybrid machine learning model for improving predictive accuracy, computational performance, and scalability in big data environments. The Model employs Random Forest (RF) for strong feature selection and Gradient Boosted Decision Trees (GBDT) for enhanced classification accuracy, both optimised using Apache Spark's MLlib in a distributed setting. The Model takes advantage of hybridisation to beat the single-algorithm model drawback in accommodating high-dimensional, heterogeneous data. The architecture is designed to handle data in real time and includes a dynamic pre-processing layer, parallel training pipeline, and continuous evaluation modules. Performance metrics like accuracy, precision, recall, F1 score, and AUC are used to validate performance against benchmark data sets. System-level metrics like latency, Throughput, and scalability are also monitored to validate usability for real-world deployments. Experiments using healthcare and e-commerce industry datasets yield better prediction power and operational effectiveness than single ML models. Visualisation using scatter plots and 3D graphs shows sharp jumps in model accuracy and processing time for different volumes of data. It is a scalable, adaptable, and precise predictive analytics tool for the big data era. Ease of use of the solution with Apache Hadoop and Spark deployment is a willingness to be utilised at the enterprise level.

Machine learning for Big Data analytics in plants.

The epitome of sustainable public procurement practices aligns with the principles of transparency, integrity, competitiveness, and efficiency, underpinned by the growing adoption of big data analytics capabilities. Big data analytics (BDA) enhances supply chain visibility, operational efficiency, and risk management; however, it has become evident that public supply chains in developing nations require robust digital infrastructure to effectively leverage the potential benefits of BDA. The research objective is to establish the effective role of Big Data Analytics (BDA) in enhancing sustainable public procurement practices (SPPP), aligning with the objectives of the Namibian Procurement Act 2015. The research further evaluates the factors impacting the technological, organisational and environmental context of public procurement principles through data-driven insights on the view of economic constraints and inequalities. The descriptive quantitative research design is rooted in a positivist perspective, focusing on an empirical case study of Namibia to assess the impact of BDA within the Technology-Organisation-Environment (TOE) framework. The research design employs a descriptive case study, enabling the exploration of factorial interrelationships. Diverse perspectives are integrated via stratified sampling, resulting in a sample size of 270 administered for data collection. Inferential statistics, including multivariate analysis, were used to analyse the data. The study revealed enhanced public procurement practices following the enactment of the Public Procurement Act 2015. ICT infrastructure, data quality, and system scalability are pivotal for BDA integration, despite hindrances such as leadership commitment, procurement capacity, procedural complexities, and resource constraints, which impede the shift towards sustainable procurement. These findings culminate in a TOE framework, which suggests that BDA's adoption offers significant strategic value, including improved decision-making, increased efficiency, enhanced transparency, and improved supplier performance in procurement. BDA's strategic effective role/value influences its adoption, leading to the transformation of sustainable public procurement practices in Namibia. This model systematically outlines the impact of BDA adoption on public procurement, showcasing how the TOE factors shape BDA's success in achieving SPPP. This research provides policymakers, public procurement professionals, and technology providers with practical insights to enhance the strategic value of BDA for sustainable procurement. This study contextualises the interplay of TOE factors for BDA adoption in public procurement for developing nations.

The research investigated the role of digital technology in enhancing sustainable business models, collective intelligence, and scaling to improve business process performance. Conducted as qualitative research, it employs a Systematic Literature Review (SLR) method, analyzing 20 Scopus-indexed journals ranked in Quartile 1 (Q1) to Quartile 2 (Q2) from 2019 to 2024. The data were processed using content analysis, focusing on digital technologies such as big data, the Internet of Things, cloud computing, cyber-physical systems, machine learning, artificial intelligence, and digital platforms. The findings demonstrate that digital technology supports businesses in understanding evolving customer needs, facilitating efficient and accurate decision-making, and enhancing customer engagement, which is crucial for business growth and scalability. Key benefits include improving customer experience, segmentation, retention, and engagement, reducing operational costs, increasing team collaboration, and evaluating environmental risks. These advantages give businesses a competitive edge by fostering sustainable practices and effective customer engagement. The advantages of implementing digital technologies can be felt from various economic, social, and environmental aspects. However, the research acknowledges limitations, including restricted data collection due to limited relevant research and variations in technological literacy across regions. Despite these challenges, the research underscores the transformative role of digital technology in advancing sustainable and competitive business processes.

Big data systems handle vast volumes of structured and unstructured information using distributed computing frameworks like Apache Hadoop, Spark, and Flink. These platforms rely heavily on computational resources, and CPU utilization becomes a critical factor in determining the performance and efficiency of data processing operations. Effective CPU usage directly impacts throughput, latency, and the ability to meet service-level objectives. In large-scale clusters, workloads vary from CPU-intensive tasks such as real-time analytics, data mining, and machine learning to more I/O-driven operations like data ingestion or storage management. Poor CPU distribution can lead to overloaded nodes while others remain underutilized, ultimately reducing system efficiency. Traditional scheduling mechanisms in big data platforms are often rule-based and may not respond well to dynamic or unpredictable workloads, which contributes to suboptimal CPU usage. When CPU utilization remains consistently high, it can trigger job slowdowns, task failures, and increased power consumption, negatively affecting overall performance. Conversely, low CPU utilization indicates underused resources, leading to waste in computational capacity and energy. Additionally, container technologies and control groups (cgroups) support fine-tuned CPU allocation and isolation, ensuring fair usage across concurrent users or tasks in multi-tenant environments. In more advanced implementations, machine learning algorithms are used to forecast CPU needs and schedule jobs more efficiently by learning from historical patterns and usage behavior. Reinforcement learning approaches have also shown potential in achieving balanced CPU usage by adapting policies based on feedback and environment changes. These intelligent mechanisms allow big data systems to optimize CPU allocation continuously, adapting to fluctuating demand without manual intervention. Efficient CPU management contributes significantly to faster job execution, reduced operational costs, and higher cluster reliability. As data volumes and complexity continue to grow, maintaining optimal CPU utilization remains a priority for organizations aiming to derive timely insights while maximizing infrastructure value. Basic CPU utilization approaches are exhibiting performance limitations. This paper addresses performance limitations related to CPU utilization.

Big data is the biggest challenges as we need huge processing power system and good algorithms to make an decision. We need Hadoop environment with pig hive, machine learning and hadoopecosystem components. The data comes from industries. Many devices around us and sensor, and from social media sites. According to McKinsey There will be a shortage of 15000000 big data professionals by the end of 2020. There are lots of technologies to solve the problem of big data Storage and processing. Such technologies are Apache Hadoop, Apache Spark, Apache Kafka, and many more. Here we analyse the processing speed for the 4GB data on cloudx lab with Hadoop mapreduce with varing mappers and reducers and with pig script and Hive querries and spark environment along with machine learning technology and from the results we can say that machine learning with Hadoop will enhance the processing performance along with with spark, and also we can say that spark is better than Hadoop mapreduce pig and hive, spark with hive and machine learning will be the best performance enhanced compared with pig and hive, Hadoop mapreduce jar.

The healthcare industry is undergoing a revolutionary transformation driven by the integration of distributed systems and big data analytics. This technological convergence enables real-time decision-making, advanced predictive capabilities, and personalized treatment plans as healthcare data grows exponentially. Traditional processing methods can no longer handle the scale and complexity of data from electronic health records, wearable devices, genomic sequencing, and medical imaging. Distributed computing frameworks like Apache Hadoop, Apache Spark, and cloud-based architectures provide the computational infrastructure to process and analyze this massive data effectively. These technologies enable breakthrough applications in early disease detection, personalized medicine, and operational optimization. Despite promising advancements, significant challenges remain in data integration, security, regulatory compliance, and algorithmic fairness. Emerging trends like edge computing, federated learning, and quantum computing will further expand predictive healthcare capabilities while addressing privacy concerns. The shift from reactive to proactive healthcare delivery promises improved patient outcomes and more efficient resource utilization across the healthcare ecosystem.

The rapid proliferation of big data across various industries, such as healthcare, finance, and social media, has created an urgent need for robust and efficient distributed neural network training frameworks. These frameworks must handle vast volumes of data while ensuring high performance and accuracy in machine learning tasks. As organizations increasingly rely on machine learning models to extract actionable insights from big data, selecting the right framework becomes critical. This paper provides a comprehensive comparison of two leading distributed neural network training frameworks: Apache Spark and TensorFlow. These frameworks are widely adopted due to their scalability and flexibility in handling complex data-driven tasks. The study evaluates their performance and accuracy in big data modeling and prediction, focusing on key metrics such as training speed, model accuracy, resource utilization, and scalability. Utilizing datasets that mirror real-world applications, the study includes thorough data preprocessing, model construction, and distributed training experiments across both frameworks. The findings reveal that while TensorFlow achieves superior model accuracy, Apache Spark demonstrates better scalability and resource efficiency, particularly in large-scale data environments. These insights offer valuable guidance for researchers and industry practitioners in selecting the most appropriate framework for their specific big data applications, ensuring optimal performance and resource management.

Digital transformation has made it necessary for corporate communication in organizations to shift in a bid to enhance customer engagement with brands. For companies providing essential services, such as electricity, like the Kenya Power and Lighting Company (KPLC), strategic and effective communication is not merely a branding exercise; it is a crucial element of ensuring operational efficiency, mitigating potential crises, and maintaining a positive relationship with the customers and communities they serve. The objectives of the study included: to examine the types of digital platforms used by KPLC and the influence they have on customer engagement on the Company’s products and services, to find out the impact of the digital communication strategies used by KPLC on customer satisfaction, and to assess the digital communication reach of KPLC communication channels and the influence it has on customer experience. The study adopted KPLC as a case study and was guided by the Uses and Gratifications Theory complemented by the Social Exchange Theory. The study adopted a quantitative methods approach of data collection and analysis. The study sample included 50 targeted customers who have engaged with the organization on digital media. The findings demonstrate that SMS remains a vital communication channel for Kenya Power customers, aligned with global trends in utility sectors where rapid transactional updates are essential. However, the uneven user experience across channels suggests a gap in integrated digital communication management, echoing literature advocating for a unified digital customer service framework to enhance satisfaction and trust. It is expected that the findings of this study will have far-reaching benefits for the Kenya Power and Lighting Company (KPLC) by providing it with data-driven insights into the effectiveness of its digital communication strategies, enabling the Company to refine its approach, improve customer engagement, and enhance overall operational efficiency as well as KPLC’s customer experience.

This in-depth investigation explores the revolutionary nexus of Java, Big Data, and Machine Learning (ML), clarifying the innovations and synergies that result from their integration. The voyage commences with a summary of the past, following the growth from Java's fundamental function in high-level development to the revolutionary influence of Big Data frameworks such as Apache Hadoop and Apache Spark. The story then delves into the fundamentals of machine learning in Java, highlighting its adaptability, rich library support, and crucial role in building strong ML pipelines. The investigation delves into the transformative power of Big Data frameworks, highlighting the distributed file system of Hadoop and the in-memory processing capabilities of Spark. We observe the significant effects of this convergence on a variety of industries through real-world case studies, from e-commerce personalized suggestions to fraud detection in banking. The insights gained from these implementations highlight how crucial it is to use ML models with ethical considerations, interdisciplinary cooperation, and ongoing learning. The following sections cover the nuances of data preprocessing, including the use of Java in ETL workflows, scalable feature engineering using Big Data frameworks, and data quality assurance via transformation and cleansing. ML model deployment is the major focus, along with an exploration of the Java runtime environment, micro services architecture, and crucial aspects of model robustness monitoring and maintenance. The investigation concludes with a focus on case studies and success stories that demonstrate the real-world effects of this convergence in sectors like e-commerce, finance, and healthcare. These real-world examples highlight the accomplishments of companies like Netflix, Uber, Airbnb, and others and provide insightful information about how well integration works to accomplish a range of business objectives.

With the advancement in technology, a huge amount of data containing useful information, called Big Data, is generated on a daily basis. For processing such tremendous volume of data, there is a need of Big Data frameworks such as Hadoop MapReduce, Apache Spark etc. Among these, Apache Spark performs up to 100 times faster than conventional frameworks like Hadoop Mapreduce. For the effective analysis and interpretation of this data, scalable Machine Learning methods are required to overcome the space and time bottlenecks. Partitional clustering algorithms are widely adopted by researchers for clustering large datasets due to their low computational requirements. Thus, we focus on the design of partitional clustering algorithm and its implementation on Apache Spark. In this paper, we propose a partitional based clustering algorithm called Scalable Random Sampling with Iterative Optimization Fuzzy c-Means algorithm (SRSIO-FCM) which is implemented on Apache Spark to handle the challenges associated with Big Data Clustering. Experimentation is performed on several big datasets to show the effectiveness of SRSIO-FCM in comparison with a proposed scalable version of the Literal Fuzzy c-Means (LFCM) called SLFCM implemented on Apache Spark. The comparative results are reported in terms of value of F-measure, ARI, Objective function, Run-time and Scalability. The reported results show the great potential of SRSIO-FCM for Big Data clustering.

PurposeThis paper aims to evaluate different approaches for the parallelization of compute-intensive tasks. The study compares a Java multi-threaded algorithm, distributed computing solutions with MapReduce (Apache Hadoop) and resilient distributed data set (RDD) (Apache Spark) paradigms and a graphics processing unit (GPU) approach with Numba for compute unified device architecture (CUDA).Design/methodology/approachThe paper uses a simple but computationally intensive puzzle as a case study for experiments. To find all solutions using brute force search, 15! permutations had to be computed and tested against the solution rules. The experimental application comprises a Java multi-threaded algorithm, distributed computing solutions with MapReduce (Apache Hadoop) and RDD (Apache Spark) paradigms and a GPU approach with Numba for CUDA. The implementations were benchmarked on Amazon-EC2 instances for performance and scalability measurements.FindingsThe comparison of the solutions with Apache Hadoop and Apache Spark under Amazon EMR showed that the processing time measured in CPU minutes with Spark was up to 30% lower, while the performance of Spark especially benefits from an increasing number of tasks. With the CUDA implementation, more than 16 times faster execution is achievable for the same price compared to the Spark solution. Apart from the multi-threaded implementation, the processing times of all solutions scale approximately linearly. Finally, several application suggestions for the different parallelization approaches are derived from the insights of this study.Originality/valueThere are numerous studies that have examined the performance of parallelization approaches. Most of these studies deal with processing large amounts of data or mathematical problems. This work, in contrast, compares these technologies on their ability to implement computationally intensive distributed algorithms.

Purpose: This research examines the intricate interplay between Business Intelligence (BI), Big Data Analytics (BDA), and Artificial Intelligence (AI) within the realm of Supply Chain Management (SCM). While the integration of these technologies has promised improved operational efficiency and decision-making capabilities, concerns about complexities and potential overreliance on technology persist. The study aims to provide insights into achieving a balance between data-driven insights and qualitative factors in SCM for sustained competitiveness. Design/methodology/approach: The research executed interviews with ten Arab Gulf-based consulting firms. These companies’ ability to successfully complete BI projects is well recognised. Findings: Through examining the interplay of human judgement and data-driven strategies, addressing integration challenges, and understanding the risks of excessive data reliance, the research enhances comprehension of the modern SCM landscape. It underscores BI’s foundational role, the necessity of balanced human input, and the significance of customer-centric strategies for lasting competitive advantage and relationships. Practical implications: The research provided information for organizations seeking to effectively navigate the complexities of integrating data-driven technologies in SCM. The research is a foundation for future studies to delve deeper into quantitative measurement methodologies and effective data security strategies in the SCM context. Originality: The research highlights the value of integrating BI, BDA, and AI in SCM for improved efficiency, cost reduction, and customer satisfaction, emphasising the need for a balanced approach that combines data-driven insights, human judgement, and customer-centric strategies to maintain competitiveness.

Fraud detection and strengthening of business strategy through transaction data analysis form the next crucial steps that arise with the development in e-commerce. Thus, building Big Data analytics for e-commerce solutions is the project that will incorporate PySpark, machine learning and statistical methods in transaction analysis, customer segmentation and demand forecasting. Synthetic transaction data with one million records consisting of user events, product transactions and price information has been created. Apache Spark provides effective large dataset processing to gain the pertinent insights of total sales, top-selling products and average transaction values. In this manner, K-Means clustering is applied for advanced BI by segmenting the customers. This will classify users according to their spending behavior. Demand forecasting will analyze ship-date sales trends that can be important for inventory planning. This also helps in identifying high-value customers and in monitoring products that are running low on stocks to facilitate data-driven decision-making. Such a system will feature scalability, near real-time processing, and actionable insights within most e-commerce platforms. By Big Data frameworks and machine learning, it provides a complete solution for transaction analytics and customer behavior observations and inventory management. It is a flexible framework for real-world applications, enabling companies to achieve data-driven insights for enhanced customer experience and operational efficiency.