Literature Survey on Web Mining

This paper gives an introduction to Web mining, then describes Web Structure mining in detail, and explores the data structure used by the Web. This paper also explores different Page Rank algorithms and compare those algorithms used for Information Retrieval. In Web Mining, the basics of Web mining and the Web mining categories are explained. Different Page Rank based algorithms like PageRank (PR), WPR (Weighted PageRank), HITS (Hyperlink-Induced Topic Search), DistanceRank and DirichletRank algorithms are discussed and compared. PageRanks are calculated for PageRank and Weighted PageRank algorithms for a given hyperlink structure. Simulation Program is developed for PageRank algorithm because PageRank is the only ranking algorithm implemented in the search engine (Google). The outputs are shown in a table and chart format.

The world wide web is expanding, everyday huge amount of data is added to the web. Finding relevant information in the web is becoming a difficult task. Web Mining is the process of analysing and mining the web to find useful information. By web mining we extract information that are implicitly present in the web. Web Mining is classified into Web Content Mining (WCM), Web Structure Mining (WSM), Web Usage Mining (WUM) based on the type of data mined. Web Structure Mining analyses the structure of the web considering it as a graph. WSM can be used to rank pages present in the web, to improve the efficiency of search engines. This paper discusses about Web Mining, its types, and various ranking algorithms used in Web structure Mining.

The purpose of this paper is to provide a more current evaluation and update of web mining research and techniques available. Current advances in each of the three different types of web mining are reviewed in the categories of web content mining, web usage mining, and web structure mining. For each tabulated research work, we examine such key issues as web mining process, methods/techniques, applications, data sources, and software used. Unlike previous investigators, we divide web mining processes into the following five subtasks: (1) resource finding and retrieving, (2) information selection and preprocessing, (3) patterns analysis and recognition, (4) validation and interpretation, and (5) visualization. This paper also reports the comparisons and summaries of selected software for web mining. The web mining software selected for discussion and comparison in this paper are SPSS Clementine, Megaputer PolyAnalyst, ClickTracks by web analytics, and QL2 by QL2 Software Inc. Applications of these selected web mining software to available data sets are discussed together with abundant presentations of screen shots, as well as conclusions and future directions of the research.

This paper is based on e-commerce web sites how to use web mining technology for providing security on e-commerce web sites. The connection between web mining ,security and e-commerce analyzed based on user behavior on web. Different web mining algorithms and security algorithm are used to provided security on e-commerce web sites. Based on customer behavior different web mining algorithms like page rank algorithm and trust rank algorithm is used for developing web mining framework in e-commerce web sites. We have developed false hit database algorithm and nearest neighbor algorithm to provide security on e-commerce web site. In existing web mining framework is based on only web content mining, We have proposed Web mining framework system which is based on web structure mining analysis, Web Content Mining analysis, decision analysis and security analysis.KeywordsWeb miningSecurityE- commerceWeb usage mining

In the present day scenario the World Wide Web (WWW) is an important and popular information search tool. It provides convenient access to almost all kinds of information – from education to entertainment. The main objective of the chapter is to retrieve information from websites and then use the information for website quality analysis. In this chapter information of the website is retrieved through web mining process. Web mining is the process is the integration of three knowledge domains: Web Content Mining, Web Structure Mining and Web Usage Mining. Web content mining is the process of extracting knowledge from the content of web documents. Web structure mining is the process of inferring knowledge from the World Wide Web organization and links between references and referents in the Web. The web content elements are used to derive functionality and usability of the website. The Web Component elements are used to find the performance of the website. The website structural elements are used to find the complexity and usability of the website. The quality assurance techniques for web applications generally focus on the prevention of web failure or the reduction of chances for such failures. The web failures are defined as the inability to obtain or deliver information such as documents or computational results requested by web users. A high quality website is one that provides relevant, useful content and a good user experience. Thus in this chapter, all areas of website are thoroughly studied for analysing the quality of website design.

Applying Web usage mining for personalizing hyperlinks in Web-based adaptive educational systems

This survey report basically gives a comparison report on different page ranking algorithm on the basis of numerical analysis.Four popular algorithms used for this purpose PageRank, Weighted PageRank, PageRank using VOL and finally weighted PageRank using VOL.This report will give brief introduction of web mining concept because web mining is core content in page rank calculation .Many aspect related to the page ranking will be discussed by taking PageRank algorithm and its variation as a basis for this purpose.A detailed comparison table of PageRank, Weighted PageRank, PageRank using VOL and finally weighted PageRank using VOL will be used for better analysis.

Through the world wide web development, web mining becomes an important research topic in database technologies. Web usage mining is to discover useful information of web pages from user navigation patterns recorded in web logs. Web utility mining is not only discovering the frequent web navigation patterns, but also identifying the profit of web pages. For enhancing the web utility mining, this study proposes a novel preference utility mining algorithm to provide web designers and decision makers more useful and meaningful web information. The proposed algorithm can also be implemented on shared memory multiprocessing (SMP) architecture of parallel computation.

Web mining is an emerging research area due to the rapid growth of websites. Web mining is classified into Web Content Mining (WCM), Web Usage Mining and Web Structure Mining. Extraction of required information from web page content available on World Wide Web (WWW) is WCM. The WCM is further classified into two categories first category is to directly mine the content on documents and second category is to mine the content using search engine. The mining method focuses on the information extraction and integration. The content of Web may be text, image, audio, video. Web pages typically contain a large amount of information that is not part of the main contents of the pages, like banner advertisements, navigation bars, copyright notices, etc. Such noises on Web pages usually lead to poor results in Web mining. This paper focuses on the problem of Noise free Information retrieval on web pages, which means the pre-processing of Web pages automatically to detect and eliminate noises. This paper proposes an approach for eliminating noises from web pages for the purpose of improving the accuracy and efficiency of web content mining. The main objective of removing noise from a Web Page is to improve the performance of the search. It is very essential to differentiate important information from noisy content that may misguide users' interest. This approach mainly concentrates on removing the following noises in stages: (1) Primary noises-Navigation bars, Panels and Frames, Page Headers and Footers, Copyright and Privacy Notices, Advertisements and other Uninteresting Data such as audio, video, multiple links. (2) Duplicate Contents and (3) Noise Contents according to block importance. The removal of these noises is done by performing three operations. Firstly, using the Block Splitting operation, primary noises are removed and only the useful text contents are partitioned into blocks. Secondly, using simhash algorithm, the duplicate blocks are removed to obtain the distinct blocks. For each block, three parameters namely Keyword Redundancy (KR), Linkword Percentage (LP) and Titleword Relevancy (TR) calculated. Using these three parameters block importance value (BI) is calculated, which is called Simhash algorithm. The importance of the block is then calculated using simhash algorithm. Based on a threshold value the important blocks are selected using sketching algorithm and the keywords are extracted from those important blocks.

Data and web mining involves the use of data mining techniques to extract information from the web in order to gain insights into online behavior, customer preferences, and trends. The process requires data preprocessing to clean and prepare the data for analysis, and pattern recognition to identify relationships and patterns within the data. The benefits of data and web mining include improved decision-making and business growth, while challenges include the large volume of data and ensuring data quality. Web mining can be divided into three main categories: web content mining, web structure mining, and web usage mining. Web content mining involves extracting information from the content of web pages, such as text, images, and videos. Web structure mining involves analyzing the links between web pages to identify patterns and relationships. Web usage mining involves analyzing user behavior on the web, such as clickstream data and session logs, to extract useful information. KEYWORDS: data mining, web mining, information extraction, data preprocessing, pattern recognition, online behavior, customer preferences,

The discipline of Web Usage Mining has grown rapidly in the past few years, despite the crash of the e-commerce boom of the late 1990s. Web Usage Mining is the application of data mining techniques to Web clickstream data in order to extract usage patterns. Yet, with all of the resources put into the problem, claims of success have been limited and are often tied to specific Web site properties that are not found in general. One reason for the limited success has been a component of Web Usage Mining that is often overlooked---the need to understand the content and structure of a Web site. The processing and quantification of a Web sites content and structure for all but completely static and single frame Web sites is arguably one of the most difficult tasks to automate in the Web Usage Mining process. This article shows that, not only is the Web Usage Mining process enhanced by content and structure, it cannot be completed without it. The results of experiments run on data from a large e-commerce site are presented to show that proper preprocessing cannot be completed without the use of Web site content and structure, and that the effectiveness of pattern analysis is greatly enhanced.

A Segmented PageRank-Based Value Compensation Method for Personal Data in Alliance Blockchains

Internet is the biggest source of information which is easily accessible but unstructured. Web mining - is the application of data mining techniques to discover patterns from the Web. According to analysis targets, web mining can be divided into three different types, which are Web usage mining, Web content mining and Web structure mining. [3] Web usage mining reveals about user's navigation pattern at any website through analyzing web logs. In this study we have tried to analyze the users online job seeking tendency pattern through two available web-based tools viz. Google Trends and Google Zeitgeist.

this paper provides an in-depth survey of challenges in the design of intelligent information retrieval systems, pointing out some similarities and differences in the core data mining and web based search operations. The procedures for evaluation of search engine performance and implicit feedback of the user with respect to a search result are studied with references to the different algorithms. We have proposed a novel neural satisfaction based feedback vector in contrast to the existing activity pattern based feedback as a future research direction in Intelligent IR. We addressed the select research work in the area of soft information retrieval using fuzzy sets, artificial neural networks, genetic algorithms and probabilistic information retrieval. As an instance of information retrieval, web mining is a set of operations to retrieve relevant documents from preprocessed, crawled and indexed web, and it can be categorized into more specialized tasks of web content mining, web structure mining and web usage mining. We have given a survey of the important reviews on topic of web mining and its associated tasks. On the basis of Identified challenges in information retrieval in general, and web mining in particular, we have concentrated on applicability of soft computing techniques and their hybrids in web mining, the performance related issues of select solutions, future of web mining and the next generation user's expectations from a search engine.

Over the last decade, we have witnessed an explosive growth in the information available on the Web. Today, Web browsers provide easy access to myriad sources of text and multimedia data. Search engines index more than a billion pages and finding the desired information is not an easy task. This profusion of resources has prompted the need for developing automatic mining techniques on Web, thereby giving rise to the term “Web mining” (Pal, Talwar, & Mitra, 2002). Web mining is the application of data mining techniques on the Web for discovering useful patterns and can be divided into three basic categories: Web content mining, Web structure mining, and Web usage mining. Web content mining includes techniques for assisting users in locating Web documents (i.e., pages) that meet certain criteria, while Web structure mining relates to discovering information based on the Web site structure data (the data depicting the Web site map). Web usage mining focuses on analyzing Web access logs and other sources of information regarding user interactions within the Web site in order to capture, understand and model their behavioral patterns and profiles and thereby improve their experience with the Web site. As citizens requirements and needs change continuously, traditional information searching, and fulfillment of various tasks result to the loss of valuable time spent in identifying the responsible actor (public authority) and waiting in queues. At the same time, the percentage of users who acquaint with the Internet has been remarkably increased (Internet World Stats, 2005). These two facts motivate many governmental organizations to proceed with the provision of e-services via their Web sites. The ease and speed with which business transactions can be carried out over the Web has been a key driving force in the rapid growth and popularity of e-government, e-commerce, and e-business applications. In this framework, the Web is emerging as the appropriate environment for business transactions and user-organization interactions. However, since it is a large collection of semi-structured and structured information sources, Web users often suffer from information overload. Personalization is considered as a popular solution in order to alleviate this problem and to customize the Web environment to users (Eirinaki & Vazirgiannis, 2003). Web personalization can be described, as any action that makes the Web experience of a user personalized to his or her needs and wishes. Principal elements of Web personalization include modeling of Web objects (pages) and subjects (users), categorization of objects and subjects, matching between and across objects and/or subjects, and determination of the set of actions to be recommended for personalization. In the remainder of this article, we present the way an e-government application can deploy Web mining techniques in order to support intelligent and personalized interactions with citizens. Specifically, we describe the tasks that typically comprise this process, illustrate the future trends, and discuss the open issues in the field.