Data Mining Techniques for Web Mining: A Survey

Usability evaluation is one of the essential aspects of software production. This evaluation should be done during the entire life cycle of a software application, from pre-production to production and post-production. However, the collection and evaluation of usability data can be a very challenging, time-consuming, and expensive task to be conducted manually, particularly for certain types of products and working conditions. These challenges may include the need to recruit participants fully engage and motivate them during evaluation, and factor in environmental conditions. Other challenges may include collecting data in real-world environments, especially when the users are geographically dispersed, minimizing evaluator and participant bias, and analyzing complex data sets, particularly when the volume of data is large. This research explores an alternative approach to automate the whole process of usability evaluation by utilizing data mining and machine learning techniques on data recorded in log files. The objective of this dissertation is to extract the value of usability-related metrics from user interactions and estimate usability measures in a quantitative manner. The use of log files for usability evaluation offers several advantages over traditional methods of evaluation, such as collecting data objectively without the need for subjective interpretation of the evaluator in some cases, creating a comprehensive view of user interactions with the possibility of identifying user behavioral patterns and trends that appear in large data sets, and collecting data from real-world scenarios instead of data from simulated scenarios. Other advantages include reducing evaluation costs, enabling remote data collection from anywhere in the world, which leads to the identification of location-dependent usability problems, and facilitating continuous monitoring over time, which leads to the identification of time-dependent usability issues. In this dissertation, Chapter II provides a comprehensive categorization, comparison, and summary of the pertinent usability evaluation techniques that utilize log files as input data in both academic and industrial research. Each method is examined carefully, and its respective strengths and weaknesses are highlighted to provide a systematic understanding of the advantages and limitations of each technique. Chapter III assesses the originality of the research questions and proposed solutions. Given the ongoing development and refinement of log file analysis tools and techniques by proficient teams in advanced corporations, and the substantial research already conducted in web usage mining, this dissertation compares its solution with prior works in these two domains. The similarities and disparities between them are evaluated to determine the uniqueness and value of

The escalating threat of fraud in financial institutions is a global issue, with the Malaysian sector being no exception. This study focuses on the implementation and efficacy of Data Mining and Machine Learning methodologies in identifying and mitigating fraudulent activities within these institutions. The paper critically reviews existing literature, bridging the gap between advanced technology application and fraud management. Fraudulent transactions in the financial sector are dynamic and sophisticated, requiring advanced detection techniques. Traditional approaches often struggle to manage this complexity effectively, demonstrating a need for more advanced and adaptive strategies. This is where Data Mining and Machine Learning techniques, renowned for their predictive and analytical prowess, can significantly contribute. Data Mining, the process of uncovering patterns and correlations within large datasets, is a useful tool for detecting anomalies that may suggest fraud. The study assesses various data mining techniques, such as clustering, classification, and association, and explores their application in detecting fraudulent transactions. Findings indicate that these techniques can substantially enhance fraud detection rates while minimizing false positives. Furthermore, Machine Learning, an artificial intelligence subset, has shown immense potential in fraud detection. Its ability to learn from and make decisions based on data makes it a viable solution for fraud detection. This paper explores both supervised and unsupervised learning algorithms and their efficacy in identifying fraud in the Malaysian financial sector. Results suggest that machine learning models, when correctly implemented, can significantly improve the accuracy of fraud detection. The review underscores the importance of employing advanced technologies like Data Mining and Machine Learning to combat financial fraud effectively. It also suggests future research directions, emphasizing the need for context-specific, localized models considering Malaysia's unique socio-economic environment. Moreover, the development of hybrid models, integrating both data mining and machine learning, could offer improved results. In conclusion, this study sets a precedent for further exploration into the application of advanced analytical tools in fraud detection in the Malaysian financial sector. The potential these technologies offer for improving accuracy and adaptability in fraud detection systems is substantial and warrants thorough investigation. Keywords: Fraud Detection, Malaysian Financial Institutions, Data Mining, Machine Learning, Financial Fraud Management

Data and web mining involves the use of data mining techniques to extract information from the web in order to gain insights into online behavior, customer preferences, and trends. The process requires data preprocessing to clean and prepare the data for analysis, and pattern recognition to identify relationships and patterns within the data. The benefits of data and web mining include improved decision-making and business growth, while challenges include the large volume of data and ensuring data quality. Web mining can be divided into three main categories: web content mining, web structure mining, and web usage mining. Web content mining involves extracting information from the content of web pages, such as text, images, and videos. Web structure mining involves analyzing the links between web pages to identify patterns and relationships. Web usage mining involves analyzing user behavior on the web, such as clickstream data and session logs, to extract useful information. KEYWORDS: data mining, web mining, information extraction, data preprocessing, pattern recognition, online behavior, customer preferences,

The world wide web is expanding, everyday huge amount of data is added to the web. Finding relevant information in the web is becoming a difficult task. Web Mining is the process of analysing and mining the web to find useful information. By web mining we extract information that are implicitly present in the web. Web Mining is classified into Web Content Mining (WCM), Web Structure Mining (WSM), Web Usage Mining (WUM) based on the type of data mined. Web Structure Mining analyses the structure of the web considering it as a graph. WSM can be used to rank pages present in the web, to improve the efficiency of search engines. This paper discusses about Web Mining, its types, and various ranking algorithms used in Web structure Mining.

Web mining is an emerging research area due to the rapid growth of websites. Web mining is classified into Web Content Mining (WCM), Web Usage Mining and Web Structure Mining. Extraction of required information from web page content available on World Wide Web (WWW) is WCM. The WCM is further classified into two categories first category is to directly mine the content on documents and second category is to mine the content using search engine. The mining method focuses on the information extraction and integration. The content of Web may be text, image, audio, video. Web pages typically contain a large amount of information that is not part of the main contents of the pages, like banner advertisements, navigation bars, copyright notices, etc. Such noises on Web pages usually lead to poor results in Web mining. This paper focuses on the problem of Noise free Information retrieval on web pages, which means the pre-processing of Web pages automatically to detect and eliminate noises. This paper proposes an approach for eliminating noises from web pages for the purpose of improving the accuracy and efficiency of web content mining. The main objective of removing noise from a Web Page is to improve the performance of the search. It is very essential to differentiate important information from noisy content that may misguide users' interest. This approach mainly concentrates on removing the following noises in stages: (1) Primary noises-Navigation bars, Panels and Frames, Page Headers and Footers, Copyright and Privacy Notices, Advertisements and other Uninteresting Data such as audio, video, multiple links. (2) Duplicate Contents and (3) Noise Contents according to block importance. The removal of these noises is done by performing three operations. Firstly, using the Block Splitting operation, primary noises are removed and only the useful text contents are partitioned into blocks. Secondly, using simhash algorithm, the duplicate blocks are removed to obtain the distinct blocks. For each block, three parameters namely Keyword Redundancy (KR), Linkword Percentage (LP) and Titleword Relevancy (TR) calculated. Using these three parameters block importance value (BI) is calculated, which is called Simhash algorithm. The importance of the block is then calculated using simhash algorithm. Based on a threshold value the important blocks are selected using sketching algorithm and the keywords are extracted from those important blocks.

The World Wide Web (WWW) has grown quickly in the past two decades from a small research community to the biggest and most popular infrastructure for communication, information dissemination, search, social interaction and commerce. The continuous growth in size and use of the WWW creates a need for methods to process these wicked volumes of data. Web Intelligence (WI), as a research direction, has a broad agenda to deal with the issues that arise around the WWW phenomenon. WI corresponds to research and development to explore the fundamental roles, artificial intelligence and advanced information technology on the web-empowered systems, services, and activities. In this context, WI is concerning of mining in web data and user behaviors. This creates a demand for using mining technologies to search large volume of data for gaining hidden knowledge. This hidden knowledge helps in gaining competitive advantages, better customers’ relationships, and even fraud detection. Achieving the intelligence to the web enhances Business Intelligence (BI) for the enterprises. Web Usage Mining (WUM) and Web Opinion Mining (WOM) are considered a leading mining approaches in mining the user behavior and reviews. Most of previous studies depended in building intelligent web on mining and analyzing user’s profiles or opinions separately. This becomes not fair enough and cause of limitations. This limitations could be narrowed if both of user preferences and opinions are considered in building recommendations. The paper proposes a framework for achieving the intelligence via using WUM and WOM. The proposed framework would contribute in solving the problem. The paper also surveys the background of using the intelligent web as an approach to enhance the Business Intelligence (BI) applications

The intersection of Artificial Intelligence, Machine Learning, and the Internet of Things is reshaping the contours of higher education. This chapter explores how these technologies are not merely enhancing but fundamentally transforming teaching, learning, and research practices. From AI-powered personalized learning systems to IoT-connected classrooms that sense and respond to student needs, the academic landscape is being redefined. Machine learning opens doors to predictive insights in both pedagogy and research, helping educators and scholars make data-informed decisions. Yet, amid the innovation, questions around equity, ethics, and institutional readiness remain. This chapter critically examines both the potential and the pitfalls, offering a grounded perspective on how AI, ML, and IoT are enabling a new era of collaborative, intelligent education.

World Wide Web is today the most important media for collecting, sharing and distributing information. Education is one of the fields where web-based technology has been quickly and successfully adopted. Web mining is active research area which involves transformation and interpretation of web data to find out the hidden information by using various data mining techniques which will be useful in enhancing E learning environment. Web usage mining for e learning environment can mine the server logs to find the user usage pattern so that users can be provided with more personalized services. WUM is one of the latest techniques in web data mining which can provide an effective platform to the learner. We focus on identifying learning pattern of learners. This paper introduces how by using WUM techniques we can make e learning environment more effective. Based on WUM technique how we can design an effective e learning environment and implementation of the model builds more intelligent system and can promote interest of learners or learning environment is modifying its behavior to reflect learning style.

Mining and analyzing social networks is now becoming a very popular research area not only for data mining and web mining but also social network analysis. Data mining is a technique that has the ability to process and analyze large amounts of data and by this to discover valuable information from the data. In recent years, due to the booming of social communications and social network-based web services, data mining has become a very important and powerful technique to process and analyze such large amounts of data. Recently, many researchers are focusing on developing new data mining techniques and algorithms, or devoting to improve traditional mining techniques for social network analysis. One key test of this discovered data is how it can be used in real-world decision-making situations. Social data are the aggregations of the communications, interactions and experiences of people, and it is useful to leverage this type of data for decision-making. Thus, is an important time to balance the research focus and real world applications, such as decision support and recommender systems. In this special issue, we invited authors to submit extended versions of their papers, which were deemed top-quality papers from MISNC 2015 (The 2nd Multidisciplinary International Social Networks Conference, Matsuyama, Japan), MSNDS 2015 (The 6th International Workshop on Mining and Analyzing Social Networks for Decision Support, Paris, France) and ASE SocialInformatics 2015 (Kaohsiung, Taiwan). In addition to the invited papers, we also called for public submissions. Each submission was reviewed by at least two experts and revised according to reviewers’ comments to ensure the quality of the papers. The first paper is entitled “Impersonate human decision making process: an interactive context-aware recommender system” and was authored by Chen-Shu Wang, Shiang-Lin Lin and Heng-Li Yang. The authors alter traditional context-aware-recommender systems (CAR) to interactive CAR (iCAR) in order to improve the recommendation accuracy. A car rental J Intell Inf Syst (2016) 47:193–194 DOI 10.1007/s10844-016-0425-4

World Wide Web is an extremely large collection of information, i.e. beyond our imagination. It provides enough information according to user's need. Web is rising dreadfully as approximately 70 million pages are added daily. Knowledge Discovery on web data is referred as Web Mining. Web Structure Mining based on the analysis of patterns from hyperlink structure in the web. Like as Data Mining, Web Mining has four stages i.e. Data Collection, Preprocessing, Knowledge Discovery and Knowledge Analysis. This paper based on the first two stages Data collection and Preprocessing. Data collection is to collect the data required for analysis. Data preprocessing is considered as an important stage of Web Structure mining because of data available on web is unstructured, heterogeneous and noisy.

A Critical Survey of Data Grid Replication Strategies Based on Data Mining Techniques

Data mining is a process of discovering useful information from a database and analysis of extracted information. Text mining uses many techniques of data mining. It primarily deals with unstructured data. Web mining is an extension of text mining since it deals with unstructured data. Data mining relates to find data from “static databases” which contains “structured” data where as, web mining plays with data that are “dynamic” and “unstructured”. In this papers our goal is to mine biomedical data from hypertext documents (e.g., mining data from web contents) using text mining techniques with the help of “biomedical ontology”. Web data repositories are the hypertext documents. Texts in the Hypertext documents are unstructured and they contain Hypertext Markup Language (HTML) tags, scripting languages, images, audios, videos, URLs etc. We collect a number of documents using Google crawler and preprocess the hypertext documents and extract the text data. Next, we identify whether a word is a biomedical entity or not by using a biomedical database the “Unified Medical Language System (UMLS) metathesaurus”. The mapping of biomedical entity from the metathesaurus will be done based on keyword query. Then we apply the result to re-rank the web documents to find most relevant documents. We conclude that the more occurrence of a biomedical entity in a page, the more relevant the page is, and thus, we can re-rank the documents to find the most relevant documents by using text mining technique.

An information retrieval system for college search is a crucial tool for students to find the best college for their academic needs. The system uses text mining, web mining, data mining, machine learning, and semantic web techniques to provide relevant and accurate results to users. The system collects data from various sources such as college websites, student reviews, and academic databases to create a comprehensive database of colleges and their relevant information. Users can search for colleges based on their preferences, such as location, academic program, admission requirements, tuition fees, and other factors. The system also provides personalized recommendations based on the user's search history and preferences. The proposed system aims to improve the college search process for students by providing a user-friendly interface and accurate information to make informed decisions about their education.

The reason at the back of data overloading dilemma faced by internet users on internet includes: excessive web information and billions of users around worldwide. Because of this, providing the internet users with more intended data is a challenging task in web applications. The lots of information available on internet are a fertile field for applying data mining techniques. This is what we call Web Mining (WM). The research in WM deals with research from many fields like database, Artificial Intelligence (machine learning [supervised, semi supervised, unsupervised and reinforcement], neural network and natural language processing (NLP)) and information retrieval. Here, research related to web mining and their categories is highlighted. We also situate comparison of most popular algorithms used from the field of data mining in pattern discovery phase of the WM.

The intersection of artificial intelligence (AI) and renewable energy systems is transforming how energy is generated, distributed, and consumed. This paper explores AI's applications in three key renewable energy sectors: solar, wind, and green hydrogen. AI technologies such as machine learning, neural networks, and predictive analytics address critical challenges such as the variability and intermittency of renewable resources. Additionally, AI-enhanced renewable systems have broader implications, such as democratizing energy access, fostering economic growth, and aligning with global climate goals like the Paris Agreement and the United Nations' Sustainable Development Goals. By leveraging the synergies between AI and renewable energy, society can achieve a cleaner, more resilient, and equitable energy future, critical for combating climate change and promoting global sustainability. This paper highlights the transformative potential of AI while offering insights into overcoming barriers to its widespread adoption in renewable energy systems.