Leveraging Data Engineering Techniques for Enhanced Business Intelligence

In today’s competitive business environment, data-driven marketing strategies are essential for successful campaign outcomes. This study presents a comprehensive analysis of marketing campaign data, emphasizing its role in enhancing customer engagement, improving decision-making, and increasing conversion rates. It explores the complexity of campaign dynamics and consumer behavior, demonstrating how business intelligence and data visualization techniques support informed marketing decisions and actionable insights. Advanced data science methods such as data cleaning, feature engineering, and cross-validation enhance predictive accuracy and campaign optimization. Visualization plays a central role in transforming raw data into interpretable insights, enabling businesses to identify trends in customer preferences and purchasing behavior. Key findings reveal that customers aged 51–70, particularly those with higher education and income levels, show the greatest purchasing power, especially for wine and meat products. These insights help align marketing strategies with data-driven understanding to design personalized campaigns that resonate with target audiences. By combining analytical methods with effective visualization, businesses can develop impactful campaigns that drive engagement, boost conversions, and foster revenue growth. The study concludes with directions for future research, including real-time data processing and automated decision-making systems to ensure continuous improvement in digital marketing strategies.

Microsoft Fabric is an end-to-end data analytical platform designed for achieve a Unified solutions in a Business Intelligence and Enterprises. It incorporates data movements, processing, ingestion, transformation, real-time event routing, and report building. It provide services like Data Engineering, Data Factory, Data Science, Real-Time Analytics, Data Warehouse, and Databases. This paper examines Microsoft Fabric and its features especially about a One Lake. Microsoft Fabric applied in a real-time analytics, data storage, advanced machine learning capabilities, and robust security protocols. Collection of files, folders and tables which represent the database is a data house . A Data lake is a cloud-based big data analytics service. Which allows storing, processing, and analyzing large amounts of structured and unstructured data. It integrates with other Azure services to provide a full data analysis solution[1].

Globally, enterprises are undergoing significant transformation in line with developments based on industrial revolution by leveraging extensive computing resources, data capture technologies, information processing systems, and advanced data science models that span analytics, optimization, and algorithmic intelligence. The global market's increasing demand, diverse resource supply options, global competition, and environmental protection needs are driving organizations to adopt sustainable strategies. These involve utilizing information technology and analytics more effectively, innovating manufacturing and service support systems, and employing novel problem-solving methods. The awareness of these challenges and opportunities inspired the theme of the joint event: 56th Annual Convention of ORSI (2023-ORSI) and the 10th International Conference on Business Analytics and Intelligence (2023-ICBAI), held at Indian Institute of Science, Bangalore, India, from December 18 to 20, 2023. The Operational Research Society of India (ORSI) Karnataka, the Department of Management Studies, IISc Bangalore, and the Analytics Society of India (ASI), DCAL, IIM Bangalore jointly organized this event. The joint event aimed to establish a premier platform for knowledge sharing among distinguished practitioners, academics, and researchers from industry and academia, focusing on the current applications of Operations Research (OR), Business Analytics (BA), and Business Intelligence (BI). The conference received over 655+ paper submissions, with 455 selected for presentation. 66 papers were deemed particularly interesting, and authors of 13 promising articles were invited to submit extended versions for a special issue of International Journal of Mathematical, Engineering and Management Sciences (IJMEMS). After rigorous peer review, eight papers were accepted for publication in this special issue, addressing common challenges in Operations Research, Business Analytics, and Business Intelligence. This Special Issue of the IJMEMS explores recent developments in Operations Research, Business Analytics, and Business Intelligence. It presents cutting-edge trends and substantial contributions to key areas such as Scheduling problems, Transshipment problems, E-commerce, Nanofluids, Blockchain Technology, Generative AI, Augmented Analytics, Machine Learning, and real-time Anomaly Detection. This Special Issue delves into following eight topics: Unmasking Content Clarity: Advancements in Defining, Measuring and Enhancing Readability: The authors present a novel method using natural language processing and Generative AI to quantitatively evaluate readability and comprehension. This approach surpasses traditional readability indices, offering substantial benefits for content creation and knowledge management in fields like education, business, technical support, and policy platforms. Strategic Insights into Blockchain Adoption in Automotive Supply Chains: A Comparative AHP-TOPSIS and TISM-MICMAC Analysis The authors explore blockchain adoption in the automotive industry using a multidisciplinary approach involving AHP, TOPSIS, TISM, and MICMAC analyses. This study identifies key enablers and their relationships, offering actionable insights and practical recommendations for automotive managers considering blockchain adoption. Avoid Maximum Cost Method for Solving Linear Fractional Transshipment Problem: The authors introduce a mathematical model for the linear fractional transshipment problem (LFTP) and suggest “Avoid Maximum Cost Method” to obtain an initial basic feasible solution for LFTP. This study conducts a comparative analysis with existing methods to demonstrate the efficiency of the proposed approach. Mathematical Study of Dispersion of Nano Biosensors in an Artery with Multiple Stenosis: This work examines nano-biosensors in a diseased artery with multiple stenoses, determining the temperature, velocity of nanofluid, and transport coefficients. The results lay the groundwork for developing nano-biosensors to diagnose, treat, and manage cardiovascular disease. The mathematical model had possible scope for target detection and drug delivery at stenosed sites. Integrating Generative AI in Business Intelligence: A Practical Framework for Enhancing Augmented Analytics: This study offers a practical framework for integrating generative AI (Gen AI) into Business Intelligence (BI). By adopting it, businesses can maximize GenAI and BI's potential, enhancing analytics, operations, and fostering a collaborative, data-driven culture. Data Monetization Through Cross Industry Collaboration in Retail Banking: This paper examines how data sharing between banks and e-commerce platforms, facilitated by data monetization, can improve banking customer experiences. This study proposes a framework using propensity models to identify promising customers and offer personalized products and promotions. Development of Dispatching Rule based Heuristic Algorithms for Real-Time Dynamic Scheduling of Non-identical Parallel Burn-in Ovens with Machine Eligibility Restriction: This study tackles a realistic problem in semiconductor manufacturing by scheduling non-identical parallel Burn-in ovens. The study proposes 25 heuristic algorithms for real-time dynamic scheduling with machine eligibility restrictions. Through empirical and statistical analysis, this study identifies top-performing algorithms. A Hybrid Framework for Real-Time Data Drift and Anomaly Identification Using Hierarchical Temporal Memory and Statistical Tests: This paper introduces a hybrid framework combining Hierarchical Temporal Memory and Sequential Probability Ratio Test for real-time data drift detection and anomaly identification. Retraining and false positives were minimized, outperforming traditional methods in experiments. Challenges and Future Directions 1. Enhanced Decision-Making: Addressing uncertainty and complexity in decision-making processes to improve outcomes. 2. Regulatory Frameworks: Conducting further research to establish flexible yet robust regulatory structures that can effectively adapt to the rapid evolution of blockchain technology. 3. Optimization Algorithms: Developing efficient algorithms to solve linear fractional transshipment problem with multi-objective linear fractional functions. 4. AI in Business Intelligence: Investigating long-term impacts of AI-enabled Business Intelligence tools on data-driven decision-making and organizational performance. 5. Ethical AI Considerations: Addressing ethical concerns such as data privacy, fairness, and biases in AI systems to ensure responsible use. 6. Digital Collaboration Frameworks: Developing frameworks that integrate digital footprints and cross-industry collaboration data to enhance strategic partnerships. 7. Advanced Scheduling Algorithms: Creating advanced meta-heuristic algorithms using efficient dispatching rule-based heuristics for dynamic scheduling of non-identical parallel batch processing machines with eligibility constraints. 8. Hybrid Anomaly Detection: Proposing a hybrid framework that combines multivariate extension of Hierarchical Temporal Memory with multivariate Sequential Probability Ratio Test for enhanced anomaly detection. 9. Advanced Text Generation: Utilizing advanced text generation techniques like prompt engineering and fine-tuning to produce more readable and engaging content. We extend our sincere appreciation to all contributing authors for their significant contributions and anonymous reviewers for their dedication and sincere evaluation of submissions. Their timely and excellent responses have been truly gratifying. Additionally, we would like to express our heartfelt thanks to Professor Mangey Ram, Editor-in-Chief of the International Journal of Mathematical, Engineering and Management Sciences, for his support in accepting this special issue and providing unwavering backing from its inception. Guest Editors

The U.S. healthcare system faces unprecedented financial challenges, requiring innovative approaches to financial management and resource allocation. This article examines the strategic role of Financial Planning and Analysis (FP&A) and Business Intelligence (BI) in enhancing healthcare resilience, with a specific focus on cost efficiency and resource optimization. Through a comprehensive literature review and analysis of case studies from hospitals and health systems, this research demonstrates how advanced financial forecasting and predictive analytics contribute to sustainable healthcare operations. Healthcare organizations that adopt robust FP&A practices together with BI tools achieve substantial cost reductions between 25% to 50% in specific areas without compromising their quality of care. Case studies highlight successful applications in sepsis prediction, fraud detection, demand forecasting, and resource allocation optimization. The integration of clinical and financial data emerges as a critical factor in developing accurate predictive models that support strategic decision-making. This article provides finance professionals with actionable insights on implementing effective FP&A and BI solutions to enhance cost management and resource allocation in healthcare settings. The research concludes that predictive analytics represents a transformative approach to healthcare financial management, enabling organizations to anticipate challenges, optimize resources, and build financial resilience in an increasingly complex environment

Business Intelligence (BI) solutions are among the popular applications that businesses demand in the Cloud. Although Business Intelligence functionality is already provided in the Cloud, vendors concentrate on providing existing products like Software as a Service, ignoring changing business and provisioning models of software. Business Intelligence functionality provided in the Cloud is lacking important integration features needed by other Cloud-based applications. While a simple upload of the data to be analyzed into a Cloud-based BI solution is already possible and well established, the visualization of the results and the interactive parts of the analysis are always provided inside the Cloud-based BI solution itself. However, as this chapter shows, common usage scenarios of integrated Cloud applications demand an integration of the analysis reports from BI applications into the user interface of other Cloud-based applications. This chapter describes how a fully integrated Cloud-based BI solution can collaborate with other Cloud-based applications. The technical concept illustrated with an example delineates how the required interface can be provided.

Financial institutions in emerging economies face significant cybersecurity challenges due to the increasing sophistication of cyber threats and limited resources for detection and prevention. This paper introduces Big Data-Driven Cybersecurity Models, designed to address gaps in threat detection and prevention by leveraging advanced analytics, machine learning, and real-time data processing. These models aim to strengthen the cybersecurity frameworks of financial institutions, particularly in emerging economies, by improving threat intelligence, detection accuracy, and response efficiency. The proposed models focus on the integration of big data technologies, such as distributed computing frameworks, data lakes, and advanced visualization tools, to handle large volumes of structured and unstructured data. By analyzing data from diverse sources, including transaction logs, network traffic, and customer interactions, the models enable the identification of anomalous patterns and potential security threats. Machine learning algorithms enhance predictive capabilities, allowing institutions to detect previously unknown threats and mitigate risks proactively. A key component of the models is their scalability and adaptability to resource-constrained environments. The framework includes cloud-based solutions to reduce infrastructure costs and ensure accessibility for smaller financial institutions. Additionally, the models emphasize the importance of fostering cybersecurity awareness and regulatory compliance to address human and organizational vulnerabilities. Validation through case studies and simulations demonstrates the effectiveness of these models in mitigating cybersecurity risks. Results show significant improvements in threat detection rates, response times, and cost-efficiency. These findings underscore the potential of big data-driven approaches to enhance the resilience of financial institutions against evolving cyber threats. This research provides actionable insights for policymakers, financial institutions, and technology developers seeking to strengthen cybersecurity in emerging economies. By adopting the proposed models, financial institutions can bridge critical gaps in threat detection and prevention, ensuring the security and trust necessary for sustainable economic growth.

The development of business intelligence (BI) has moved from static batch-based systems to dynamic, real-time and AI-enabled tools. Classic Microsoft BI products (e.g. SSIS, SSAS, SSRS) have been an excellent fit for enterprise on-premises analysis but had limitations for cloud-scale analytics, flexibility and AI integration. With organizations shifting towards cloud-based ecosystems, there is an increasing need to undergo ETL pipeline modernization and incorporate real-time analytics, without losing out on the gains of their existing MSBI investments. This article provides an end-to-end architecture for modern business intelligence (BI), such as Domo, using Microsoft Azure: SSIS Data Factory migration, PaaS instead of IaaS, and Power BI for full reporting capabilities, minus the data movement. I’ll describe the future of each cloud service; in this case, no fear. Following is a high-level view of the proposed hybrid cloud architecture to support phased deployment of cloud for scalable, real-time and intelligent decision making. It also shows scenarios such as real-time sales dashboard and churn prediction, which prove the capability of integration with Azure BI platform.

Integrating Business Intelligence (BI) and predictive analytics in banking has become a pivotal strategy for optimizing financial decision-making. This review paper explores this integration's theoretical foundations, current trends, and challenges while proposing a comprehensive framework to enhance decision-making processes. The proposed framework emphasizes the importance of a unified data architecture, advanced data integration techniques, and the synergistic use of BI tools and predictive models. By examining the implications for banking institutions, the paper highlights how this integration can transform strategic and operational decisions, improve risk management, and drive personalized customer engagement. Additionally, future trends such as the adoption of AI, real-time analytics, and enhanced data governance are discussed, along with recommendations for banks aiming to implement this integration successfully. The findings underscore the transformative potential of combining BI and predictive analytics to foster innovation, efficiency, and competitive advantage in the banking sector. Keywords: Business Intelligence (BI), Predictive Analytics, Banking Sector, Financial Decision-Making, Data Integration.

In today's tech landscape, business intelligence (BI) is essential, especially with advanced technologies emerging. This study delves into the transformative potential of artificial intelligence and machine learning in predictive analytics, offering crucial insights into data. The chapter provides an extensive overview, focusing on the future of predictive analytics in BI. It highlights real-time applications, featuring practical work and case studies. Key practices for integrating AI and predictive analytics are introduced, covering data preparation, cleaning, preprocessing, feature engineering, and exploratory data analysis. Integration of ML with BI facilitates visualization, report generation, interactive dashboards, and real-time monitoring. However, manual analysis of complex datasets poses challenges, risking valuable insights. This chapter explores how ML automates the identification of patterns, extracting insights. Key challenges and advantages in integrating machine learning into BI frameworks are also discussed.

The existing big data platforms take data through distributed processing platforms and store them in a data lake. The architectures such as Lambda and Kappa address the real-time and batch processing of data. Such systems provide real time analytics on the raw data and delayed analytics on the curated data. The data denormalization, creation and maintenance of a columnar dimensional data warehouse is usually time consuming with no or limited support for unstructured data. The system introduced in this paper automatically creates and dynamically maintains its data warehouse as a part of its big data pipeline in addition to its data lake. It creates its data warehouse on structured, semi-structured and unstructured data. It uses Machine Learning to identify and create dimensions. It also establishes relations among data from different data sources and creates the corresponding dimensions. It dynamically optimizes the dimensions based on the crowd sourced data provided by end users and also based on query analysis.

In this paper, we present a first of a kind system, called Business Intelligence from Voice of Customer (BIVoC), that can: 1) combine unstructured information and structured information in an information intensive enterprise and 2) derive richer business insights from the combined data. Unstructured information, in this paper, refers to Voice of Customer (VoC) obtained from interaction of customer with enterprise namely, conversation with call-center agents, email, and sms. Structured database reflect only those business variables that are static over (a longer window of) time such as, educational qualification, age group, and employment details. In contrast, a combination of unstructured and structured data provide access to business variables that reflect upto date dynamic requirements of the customers and more importantly indicate trends that are difficult to derive from a larger population of customers through any other means. For example, some of the variables reflected in unstructured data are problem/interest in a certain product, expression of dissatisfaction with the business provided, and some unexplored category of people showing certain interest/problem. This gives the BIVoC system the ability to derive business insights that are richer, more valuable and crucial to the enterprises than the traditional business intelligence systems which utilize onlystructured information. We demostrate the effectiveness of BIVoC system through one of our real-life engagements where the problem is to determine how to improve agent productivity in a call center scenario. We also highlight major challenges faced while dealing with unstructured information such as handling noise and linking with structured data.

NoSQL and relational databases have been integrated in response to the increasing demand for scalable and efficient data management in hybrid cloud environments. The differences in data structures and query processing methods between these databases present both challenges and opportunities when designing an optimized hybrid system. This study explores the integration of NoSQL and relational databases to maximize data storage, retrieval, and processing efficiency across multiple cloud systems. NoSQL databases excel in handling unstructured and semi-structured data, offering flexibility and scalability, whereas relational databases provide robust consistency and structured query capabilities. Despite the wide availability of relational databases, certain applications require the dynamic adaptability of NoSQL systems, making integration a viable solution. The research evaluates key performance parameters, including query execution speed, scalability, data consistency, and resource utilization. Cloudsim is used for simulation, allowing for an in-depth comparison between standalone and hybrid database models. Experimental results indicate that the proposed hybrid model improves query performance by 30%, reduces latency by 20%, and enhances scalability by 40% compared to using relational or NoSQL databases alone. The novelty of this approach lies in its ability to overcome the limitations of each database type by their strengths in a dynamic cloud environment. The results highlight the effectiveness of hybrid database integration in optimizing cloud-based data management while ensuring seamless operation, adaptability, and resource efficiency.

Clinical laboratory information systems (LIS) rely on new solutions such as Business Intelligence (BI) to help manage their environments. Without proper BI solution, decision-making is based on instinct rather than on data. Using current tools and processes in the LIS, it is often difficult to deliver valuable information to the right people at the right time. This systematic research conducted between the years 2007 and 2012 suggests that as BI solutions come in a wide variety of flavors, an actionable insight into 8 areas in addition to 11 key factors should be considered selecting and evaluating a BI platform.

Real time analysis has become a norm in today's fast and competitive business environment. Availability of recent data is a requirement to perform real time analysis. The data warehouse is meant to store historical data for analysis and these are populated from time to time through a process called as extract transform and load (ETL). The ETL process was done during nights or weekends to make the transactional data available at the data warehouse for analysis. But the delay of a day or weekend is not acceptable for real time analysis hence a concept of near real time ETL is developed to perform real time analysis. We explore in this study methods of how near real time ETL could be achieved. We focus on three ways of how near real time ETL could be performed. First is the Meta data management which if done effectively will greatly reduce the development time. Second is the concept of change data capture when implemented with some intelligent would greatly help in making the correct data available for analysis at the warehouse. The third is the use of parallel techniques to partition the flow of ETL and reduce the time window with which the data are available at the warehouse.

The personalized health care service utilizes the relational patient data and big data analytics to tailor the medication recommendations. However, most of the health care data are in unstructured form and it consumes a lot of time and effort to pull them into relational form. This study proposes a novel data lake architecture to reduce the data ingestion time and improve the precision of healthcare analytics. It also removes the data silos and enhances the analytics by allowing the connectivity to the third-party data providers (such as clinical lab results, chemist, insurance company, etc.). The data lake architecture uses the Hadoop Distributed File System (HDFS) to provide the storage for both structured and unstructured data. This study uses K-means clustering algorithm to find the patient clusters with similar health conditions. Subsequently, it employs a support vector machine to find the most successful healthcare recommendations for the each cluster. Our experiment results demonstrate the ability of data lake to reduce the time for ingesting data from various data vendors regardless of its format. Moreover, it is evident that the data lake poses the potential to generate clusters of patients more precisely than the existing approaches. It is obvious that the data lake provides an unified storage location for the data in its native format. It can also improve the personalized healthcare medication recommendations by removing the data silos.