A community detection algorithm for Web Usage Mining systems

The discipline of Web Usage Mining has grown rapidly in the past few years, despite the crash of the e-commerce boom of the late 1990s. Web Usage Mining is the application of data mining techniques to Web clickstream data in order to extract usage patterns. Yet, with all of the resources put into the problem, claims of success have been limited and are often tied to specific Web site properties that are not found in general. One reason for the limited success has been a component of Web Usage Mining that is often overlooked---the need to understand the content and structure of a Web site. The processing and quantification of a Web sites content and structure for all but completely static and single frame Web sites is arguably one of the most difficult tasks to automate in the Web Usage Mining process. This article shows that, not only is the Web Usage Mining process enhanced by content and structure, it cannot be completed without it. The results of experiments run on data from a large e-commerce site are presented to show that proper preprocessing cannot be completed without the use of Web site content and structure, and that the effectiveness of pattern analysis is greatly enhanced.

The data source of web usage mining is a web server access log file. This paper is focus on the user session identification that is the important step of data pre-processing. There are two useful methods to identification user and session: Session Time Thresholds (STT) and cookies identification. We identified user/session with cookies and STT methods and create different data pre-processing files. We will compare the user/session identification according to following levels of data pre-processing. By comparing these methods, we focused on the quality of extracted sequential rules. Sequence rules are an important output of the web usage mining process. Generated sequential rules from pre-processed files are classified into one of the following groups: useful, trivial, inexplicable. The aim of our paper is to compare the user/session identification using cookies with the identification of user/session using the STT. This comparison was performed with respect to the quality of the sequential rules generated, i.e., a comparison was made in terms of generation useful, trivial and inexplicable rules.

During their navigation, Web users leave many records of their activity. This huge amount of data can be a useful source of knowledge. Sophisticated data mining processes are needed for this knowledge to be extracted, understood and used. In this paper, we propose a Web usage mining (WUM) system, called SUGGEST, which dynamically generates suggested links to Web pages of potential interest for a user. SUGGEST is designed to efficiently integrate the WUM process with the ordinary Web server functionalities. It can provide useful information to make the user's Web navigation easier and to optimize the Web server's performance. Two quantities are introduced in order to give a measure of the quality of our WUM system.

With the number of Internet users and applications continues to grow, it becomes increasingly important to understand the users' behavior character for efficient network management and security monitoring. Web is one of the most popular applications, which can help users to obtain anything they want. In this paper, we propose a new framework to measure and monitor users' web access behavior character. Firstly, web services are divided into 12 types according to the content they provide. As user like to access different web services at different time instants, we proposed the behavior spectrum to describe the users' access character in an easily understandable way. Secondly, we employ three traffic features (number of connections, number of packets and number of bytes) to measure the intensity of users' access behavior. Based on the spectrums and features obtained, two kinds of user's access characters are analyzed: the characters at a particular time instant and the dynamic changing characters at continuous time points. Finally, we employ the Renyi entropy to cluster the users into different groups based on the spectrum and find the results are useful for network management. To verify the method, several actual traffic traces are collected from the Northwest Regional Center of CERNET (China Education and Research Network) and the experimental results prove the efficiency of the proposed methods.

Along with the deepening development of e-commerce, it becomes important to know the user access mode. Through Web usage mining, the server log, registration information and other relative information left by user access can be mined with the user access mode which will provide foundation for decision-making of organizations. This paper proposes a SQL Server2000 based Web usage mining solution and illustrates with authentic examples on how to use DTS, T-SQL and other tools under SQL Server2000 to realize data transfer, data cleansing, user recognition, session recognition and other data pre-processing work, and illustrates on how to use Online Analysis and Process (OLAP) as well as Data Mining (DM) under SQL Server2000 as to realize mode discovery and mode analysis. This thesis is aimed at exploring a visual Web usage mining method.

There has been an increased demand for understanding of Web-users due to the Web development and the increased number of Web-based applications. Informative knowledge extracted from Web user access patterns has been used for many applications, such as the prefetching of pages between clients and proxies. This paper presents an approach for measuring similarity of interests among Web users, based on the interest items collected from Web user's access logs. A matrix-based algorithm is then developed to cluster Web users such that the users in the same cluster are closely related with respect to the similarity measure. As an application example, a Web document prefetching technique is proposed that utilises the similarity measure and clusters obtained. Experiments have been conducted and the results have shown that our clustering method is capable of clustering Web users with similar interests, and the prefetching method is practical.

Web Usage Mining (WUM) includes all the Data Mining techniques used to analyze the behavior of a Web site‘s users (Cooley, Mobasher & Srivastava, 1999, Spiliopoulou, Faulstich & Winkler, 1999, Mobasher, Dai, Luo & Nakagawa, 2002). Based mainly on the data stored into the access log files, these methods allow the discovery of frequent behaviors. In particular, the extraction of sequential patterns (Agrawal, & Srikant, 1995) is well suited to the context of Web logs analysis, given the chronological nature of their records. On a Web portal, one could discover for example that “25% of the users navigated on the site in a particular order, by consulting first the homepage then the page with an article about the bird flu, then the Dow Jones index evolution to finally return on the homepage before consulting their personal e-mail as a subscriber”. In theory, this analysis allows us to find frequent behaviors rather easily. However, reality shows that the diversity of the Web pages and behaviors makes this approach delicate. Indeed, it is often necessary to set minimum thresholds of frequency (i.e. minimum support) of about 1% or 2% before revealing these behaviors. Such low supports combined with significant characteristics of access log files (e.g. huge number of records) are generally the cause of failures or limitations for the existent techniques employed in Web usage analysis. A solution for this problem consists in clustering the pages by topic, in the form of a taxonomy for example, in order to obtain a more general behavior. Considering again the previous example, one could have obtained: “70% of the users navigate on the Web site in a particular order, while consulting the home page then a page of news, then a page on financial indexes, then return on the homepage before consulting a service of communication offered by the Web portal”. A page on the financial indexes can relate to the Dow Jones as well as the FTSE 100 or the NIKKEI (and in a similar way: the e-mail or the chat are services of communication, the bird flu belongs to the news section, etc.). Moreover, the fact of grouping these pages under the “financial indexes” term has a direct impact by increasing the support of such behaviors and thus their readability, their relevance and significance. The drawback of using a taxonomy comes from the time and energy necessary to its definition and maintenance. In this chapter, we propose solutions to facilitate (or guide as much as possible) the automatic creation of this taxonomy allowing a WUM process to return more effective and relevant results. These solutions include a prior clustering of the pages depending on the way they are reached by the users. We will show the relevance of our approach in terms of efficiency and effectiveness when extracting the results.

Web sites are one of the most important tools for advertisements in international area for universities and other foundation. The quality of a website can be evaluated by analyzing user accesses of the website. To know the quality of a web site user accesses are to be evaluated by web usage mining. The results of mining can be used to improve the website design and increase satisfaction which helps in various applications. Log files are the best source to know user behavior. But the raw log files contains unnecessary details like image access, failed entries etc., which will affect the accuracy of pattern discovery and analysis. So preprocessing stage is an important work in mining to make efficient pattern analysis. To get accurate mining results user's session details are to be known.In our proposed work, user's browsing behavior on a web page is considered with different modes of actions on client side to calculate web page browsing time with more precision and accuracy. It helps us to maintain the accuracy in finding frequent web usage patterns and administrators to evaluate its website usage more effectively. In our proposed work we considered t, t1, t2 three time limits which are subjective to the website management. These values should be selected effectively because if we select small value it will increase load on client side and if we select large value it will not provide good results.

Web caching is intended to reduce network traffic, server load and user-perceived retrieval latency. Web pre-fetching, which can be considered as active caching, builds on regular Web caching, minimizing further a Web user's access delay. To be effective, however, the pre-fetching techniques must be able to predict subsequent Web access with minimum computational overheads. This paper presents a similarity-based mechanism to support similarity-aware Web document pre-fetching between proxy caches and browsing clients. We first define a set of measures to assess similarities between Web documents, and then propose a multi-cache architecture to cache Web documents based on those similarities. A predictor is developed to support the similarity-aware document pre-fetching algorithm. Preliminary experiments have shown that our predictor offers superior performance when compared with some existing prediction algorithms.

PurposeAs the number of web applications is increasing day by day web mining acts as an important tool to extract useful information from weblogs and analyse them according to the attributes and predict the usage of a website. The main aim of this paper is to inspect how process mining can be used to predict the web usability of hotel booking sites based on the number of users on each page, and the time of stay of each user. Through this paper, the authors analyse the web usability of a website through process mining by finding the web usability metrics. This work proposes an approach to finding the usage of a website using the attributes available in the weblog which predicts the actual footfall on a website.Design/methodology/approachPROM (Process Mining tool) is used for the analysis of the event log of a hotel booking site. In this work, authors have used a case study to apply the PROM (process mining tool) to pre-process the event log dataset for analysis to discover better-structured process maps than without pre-processing.FindingsThis article first provided an overview of process mining, then focused on web mining and later discussed process mining techniques. It also described different target languages: system nets (i.e. Petri nets with an initial and a final state), inductive miner and heuristic miner, graphs showing the change in behaviour of the dataset and predicting the outcome, that is the webpage having the maximum number of hits.Originality/valueIn this work, a case study has been used to apply the PROM (process mining tool) to pre-process the event log dataset for analysis to discover better-structured process maps than without pre-processing.

The tremendous growth in volume of web usage data results in the boost of web mining research with focus on discovering potentially useful knowledge from web usage data. This paper presents a new web usage mining process for finding sequential patterns in web usage data which can be used for predicting the possible next move in browsing sessions for web personalization. This process consists of three main stages: preprocessing web access sequences from the web server log, mining preprocessed web log access sequences by a tree-based algorithm, and predicting web access sequences by using a dynamic clustering-based model. It is designed based on the integration of the dynamic clustering-based Markov model with the Pre-Order Linked WAP-Tree Mining (PLWAP) algorithm to enhance mining performance. The proposed mining process is verified by experiments with promising results.

The endless amount of information on the web, known as “lost-in-hyper-space syndrome”, easily overwhelms users. User profiles are used as a means to support extracting relevant information by indicating user interests. In this paper, we propose a new method to develop and maintain a user profile by analyzing user's web access behavior. We propose an ontology-based semantic similarity measure and combine it with an importance measure to identify items that are of highest relevance to user interests. The proposed approach is used in a system for updating a user profile in music domain.

The purpose of this paper is to provide a more current evaluation and update of web mining research and techniques available. Current advances in each of the three different types of web mining are reviewed in the categories of web content mining, web usage mining, and web structure mining. For each tabulated research work, we examine such key issues as web mining process, methods/techniques, applications, data sources, and software used. Unlike previous investigators, we divide web mining processes into the following five subtasks: (1) resource finding and retrieving, (2) information selection and preprocessing, (3) patterns analysis and recognition, (4) validation and interpretation, and (5) visualization. This paper also reports the comparisons and summaries of selected software for web mining. The web mining software selected for discussion and comparison in this paper are SPSS Clementine, Megaputer PolyAnalyst, ClickTracks by web analytics, and QL2 by QL2 Software Inc. Applications of these selected web mining software to available data sets are discussed together with abundant presentations of screen shots, as well as conclusions and future directions of the research.

Over the last decade, we have witnessed an explosive growth in the information available on the Web. Today, Web browsers provide easy access to myriad sources of text and multimedia data. Search engines index more than a billion pages and finding the desired information is not an easy task. This profusion of resources has prompted the need for developing automatic mining techniques on Web, thereby giving rise to the term “Web mining” (Pal, Talwar, & Mitra, 2002). Web mining is the application of data mining techniques on the Web for discovering useful patterns and can be divided into three basic categories: Web content mining, Web structure mining, and Web usage mining. Web content mining includes techniques for assisting users in locating Web documents (i.e., pages) that meet certain criteria, while Web structure mining relates to discovering information based on the Web site structure data (the data depicting the Web site map). Web usage mining focuses on analyzing Web access logs and other sources of information regarding user interactions within the Web site in order to capture, understand and model their behavioral patterns and profiles and thereby improve their experience with the Web site. As citizens requirements and needs change continuously, traditional information searching, and fulfillment of various tasks result to the loss of valuable time spent in identifying the responsible actor (public authority) and waiting in queues. At the same time, the percentage of users who acquaint with the Internet has been remarkably increased (Internet World Stats, 2005). These two facts motivate many governmental organizations to proceed with the provision of e-services via their Web sites. The ease and speed with which business transactions can be carried out over the Web has been a key driving force in the rapid growth and popularity of e-government, e-commerce, and e-business applications. In this framework, the Web is emerging as the appropriate environment for business transactions and user-organization interactions. However, since it is a large collection of semi-structured and structured information sources, Web users often suffer from information overload. Personalization is considered as a popular solution in order to alleviate this problem and to customize the Web environment to users (Eirinaki & Vazirgiannis, 2003). Web personalization can be described, as any action that makes the Web experience of a user personalized to his or her needs and wishes. Principal elements of Web personalization include modeling of Web objects (pages) and subjects (users), categorization of objects and subjects, matching between and across objects and/or subjects, and determination of the set of actions to be recommended for personalization. In the remainder of this article, we present the way an e-government application can deploy Web mining techniques in order to support intelligent and personalized interactions with citizens. Specifically, we describe the tasks that typically comprise this process, illustrate the future trends, and discuss the open issues in the field.

Over the last decade, we have witnessed an explosive growth in the information available on the Web. Today, Web browsers provide easy access to myriad sources of text and multimedia data. Search engines index more than a billion pages and finding the desired information is not an easy task. This profusion of resources has prompted the need for developing automatic mining techniques on Web, thereby giving rise to the term “Web mining” (Pal, Talwar, & Mitra, 2002). Web mining is the application of data mining techniques on the Web for discovering useful patterns and can be divided into three basic categories: Web content mining, Web structure mining, and Web usage mining. Web content mining includes techniques for assisting users in locating Web documents (i.e., pages) that meet certain criteria, while Web structure mining relates to discovering information based on the Web site structure data (the data depicting the Web site map). Web usage mining focuses on analyzing Web access logs and other sources of information regarding user interactions within the Web site in order to capture, understand and model their behavioral patterns and profiles and thereby improve their experience with the Web site. As citizens requirements and needs change continuously, traditional information searching, and fulfillment of various tasks result to the loss of valuable time spent in identifying the responsible actor (public authority) and waiting in queues. At the same time, the percentage of users who acquaint with the Internet has been remarkably increased (Internet World Stats, 2005). These two facts motivate many governmental organizations to proceed with the provision of e-services via their Web sites. The ease and speed with which business transactions can be carried out over the Web has been a key driving force in the rapid growth and popularity of e-government, e-commerce, and e-business applications. In this framework, the Web is emerging as the appropriate environment for business transactions and user-organization interactions. However, since it is a large collection of semi-structured and structured information sources, Web users often suffer from information overload. Personalization is considered as a popular solution in order to alleviate this problem and to customize the Web environment to users (Eirinaki & Vazirgiannis, 2003). Web personalization can be described, as any action that makes the Web experience of a user personalized to his or her needs and wishes. Principal elements of Web personalization include modeling of Web objects (pages) and subjects (users), categorization of objects and subjects, matching between and across objects and/or subjects, and determination of the set of actions to be recommended for personalization. In the remainder of this article, we present the way an e-government application can deploy Web mining techniques in order to support intelligent and personalized interactions with citizens. Specifically, we describe the tasks that typically comprise this process, illustrate the future trends, and discuss the open issues in the field.