Noise removing from Web pages using neural network

Purpose In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Design/methodology/approach In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Findings In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Originality/value In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing.

Googling for Health Information

Local and global topics in text modeling of web pages nested in web sites

We describe the development of a computational cognitive model that explains navigation behavior on the World Wide Web. The model, called SNIF-ACT (Scent-based Navigation and Information Foraging in the ACT cognitive architecture), is motivated by Information Foraging Theory (IFT), which quantifies the perceived relevance of a Web link to a user's goal by a spreading activation mechanism. The model assumes that users evaluate links on a Web page sequentially and decide to click on a link or to go back to the previous page by a Bayesian satisficing model (BSM) that adaptively evaluates and selects actions based on a combination of previous and current assessments of the relevance of link texts to information goals. SNIF-ACT 1.0 utilizes the measure of utility, called information scent, derived from IFT to predict rankings of links on different Web pages. The model was tested against a detailed set of protocol data collected from 8 participants as they engaged in two information-seeking tasks using the World Wide Web. The model provided a good match to participants' link selections. In SNIF-ACT 2.0, we included the adaptive link selection mechanism from the BSM that sequentially evaluates links on a Web page. The mechanism allowed the model to dynamically build up the aspiration levels of actions in a satisficing process (e.g., to follow a link or leave a Web site) as it sequential assessed link texts on a Web page. The dynamic mechanism provides an integrated account of how and when users decide to click on a link or leave a page based on the sequential, ongoing experiences with the link context on current and previous Web pages. SNIF-ACT 2.0 was validated on a data set obtained from 74 subjects. Monte Carlo simulations of the model showed that SNIF-ACT 2.0 provided better fits to human data than SNIF-ACT 1.0 and a Position model that used position of links on a Web page to decide which link to select. We conclude that the combination of the IFT and the BSM provides a good description of user-Web interaction. Practical implications of the model are discussed.

A relevant consequence of the expansion of the web and e-commerce is the growth of the demand of new web sites and web applications. As a result, web sites and applications are usually developed without a formalized process, and web pages are directly coded in an incremental way, where new pages are obtained by duplicating existing ones. Duplicated web pages, having the same structure and just differing for the data they include, can be considered as clones. The identification of clones may reduce the effort devoted to test, maintain and evolve web sites and applications. Moreover, clone detection among different web sites aims to detect cases of possible plagiarism. In this paper we propose an approach. based on similarity metrics, to detect duplicated pages in web sites and applications, implemented with HTML language and ASP technology. The proposed approach has been assessed by analyzing several web sites and Web applications. The obtained results are reported in the paper with respect to some case studies.

An image-based opportunistic Web browser called "WebDriving" is described that automatically and continuously visualizes the "peripheral information space". It extracts the images from the current Web page, its link-destination pages, and other related pages and uses them to dynamically construct a 3D space. This enables the user to concurrently browse not only the images on the current Web page but also the images in the peripheral information space. The user browses these images by "driving a car through the constructed 3D world". The user thus becomes aware of other relevant Web pages while browsing the current page, which is a form of "opportunistic browsing". An experiment in which elementary school pupils used the WebDriving browser demonstrated that they could use it to effectively obtain information because it enabled them to intuitively understand a Web space, the functions of a Web browser, and Web search.KeywordsImage-based opportunistic web browserPeripheral information spaceInvestigative learning

An unsupervised method for joint information extraction and feature mining across different Web sites

With the phenomenal growth of World Wide Web and the huge number of web sites with so many web pages, finding relevant information becomes quite difficult for the web users. Web site promoters also need to ensure that their web site is on the top of the search results for better market share. Finding the right combination of keywords and placing them at the appropriate places may help in bringing the web site on the top of search results when a search engine searches the web sites and creates indexes. Over the years web site optimization research has reported encouraging results. Despite promising results, data mining techniques for web sites classification have hardly been applied. An appropriate implementation of classification of web pages may help designing the web pages accordingly. Here in this work data mining using R’s web scraping technique has been used for performing extraction of web pages and also pair of java tools called “website” and “webscrap” were developed to automatically get pages and for further analysis of the downloaded pages. This is the first attempt to analyze medical related nomenclature using.

In this paper, we propose a new approach to discover informative contents from a set of tabular documents (or Web pages) of a Web site. Our system, InfoDiscoverer, first partitions a page into several content blocks according to HTML tag in a Web page. Based on the occurrence of the features (terms) in the set of pages, it calculates entropy value of each feature. According to the entropy value of each feature in a content block, the entropy value of the block is defined. By analyzing the information measure, we propose a method to dynamically select the entropy-threshold that partitions blocks into either informative or redundant. Informative content blocks are distinguished parts of the page, whereas redundant content blocks are common parts. Based on the answer set generated from 13 manually tagged news Web sites with a total of 26,518 Web pages, experiments show that both recall and precision rates are greater than 0.956. That is, using the approach, informative blocks (news articles) of these sites can be automatically separated from semantically redundant contents such as advertisements, banners, navigation panels, news categories, etc. By adopting InfoDiscoverer as the preprocessor of information retrieval and extraction applications, the retrieval and extracting precision will be increased, and the indexing size and extracting complexity will also be reduced.

The behavioral web users’ access patterns help website administrator/web site owners to take major decisions in categorizing web pages of the web site as highly demanding pages and medium demanding pages. Human beings act as a spider surfing the web pages of the website in search of required information. Most of the traditional mining algorithms concentrate only on frequency/support of item sets (web pages set denoted as ps in a given web site), which may not bring considerably more amount of profit. The utility mining model focuses on only high utilities item sets (ps). General utility mining model was proposed to overcome weakness of the frequency and utility mining models. General utility mining does not encompass website topology. This limitation is overcome by a novel model called human behavioral patterns’ web pages categorizer (HBP-WPC) which considers structural statistics of the web page in addition to support and utility. The topology of the web site along with log file statistics plays a vital role in categorizing web pages of the web site. The web pages of the website along with log file statistics forms a population. Suitable auto optimization metric is defined which provides guidelines for website designers/owners to restructure the website based on behavioral patterns of web users.KeywordsWeb miningReorganizationLog fileWeb site topology

With the maturity of network technology, the deterministic factor of a 'successful' web site does not as much depend on its connectivity to the Internet, rather the content of web pages plays a more important role. The inclusion of dynamic data, for example, has become an increasingly desirable feature of a web page. Web servers that support dynamic web pages are capable of retrieving data at the time of user requests. Rather than being statically included in the web page, these dynamically retrieved data may physically reside in various file systems or database servers across the Internet.JAVA, a new object-oriented programming language, has become a buzzword since its introduction by Sun Microsystems in 1994 [1, 2, 3]. It was claimed that JAVA, via its ability to embed applets in a web page, would make the content of a web page alive and dynamic. Data and information on a web page can now be updated dynamically when users request them.In this paper, we discuss our experience of developing JAVA applets that use various types of JDBC (JAVA Database Connectivity) drivers to connect to and then manipulate data stored in SQL (Structured Query Language) database servers. We first start with the discussion of general client/server model involving JDBC, and then introducing the procedure of setting up programming projects employing this approach in an undergraduate database course.

At present, there are many security risks from the network, so it has practical significance for students in the university to learn the knowledge about Web pages and website security performance and have Web pages and web site security performance detection capability. It is described that the recent organization of a number of students to carry out a Web page and Web site security performance experiments. In experiments students’ security awareness on the Web page and website are trained through the use of security vulnerability detection tools and enable students to have basic security performance testing capabilities of web page and website

A total of 150 university Web sites were segregated into one of three groups: accredited residential, regionally accredited online, and nonaccredited online institutions. The promotional imagery, marketing messages and marketing themes found on the landing pages of each university program Web sites were analyzed for similarities and differences. A check sheet containing keywords was developed and as each Web page was analyzed, the results were recorded individually by two researchers. Intercoder reliability was confirmed with a Holsti correlation coefficient of 88%. At-test was used to assess variations of keywords across each category and a Chi-square was used to assess within group differences. The results show that accredited residential institutions are not leveraging clear advantages in order to differentiate themselves from online accredited and nonaccredited institutions. Benefits and themes featured on Web sites were remarkably similar, focusing on easily copied claims rather than building competitive advantages with emphasis on accreditation, faculty, resources, and quality of education.

We develop a novel framework that aims at automatically adapting previously learned information extraction knowledge from a source Web site to a new unseen target site in the same domain. Two kinds of features related to the text fragments from the Web documents are investigated. The first type of feature is called, a site-invariant feature. These features likely remain unchanged in Web pages from different sites in the same domain. The second type of feature is called a site-dependent feature. These features are different in the Web pages collected from different Web sites, while they are similar in the Web pages originating from the same site. In our framework, we derive the site-invariant features from previously learned extraction knowledge and the items previously collected or extracted from the source Web site. The derived site-invariant features will be exploited to automatically seek a new set of training examples in the new unseen target site. Both the site-dependent features and the site-invariant features of these automatically discovered training examples will be considered in the learning of new information extraction knowledge for the target site. We conducted extensive experiments on a set of real-world Web sites collected from three different domains to demonstrate the performance of our framework. For example, by just providing training examples from one online book catalog Web site, our approach can automatically extract information from ten different book catalog sites achieving an average precision and recall of 71.9% and 84.0% respectively without any further manual intervention.

Social book marking Web sites have emerged recently for collecting and sharing of interesting Web sites among users. People can add Web pages to such sites as bookmarks and allow themselves as well as others to manipulate them. One of the key features of the social book marking sites is the ability of annotating a Web page when it is being bookmarked. The annotation usually contains a set of words or phrases, which are collectively known as tags, that could reveal the semantics of the annotated Web page. Efficient and effective search of Web pages can then be achieved via such tags. However, spam tags that are irrelevant to the content of Web pages often appear to deceive other users for malicious or commercial purposes. Various techniques have been devised to tackle such tag spam detection problem. Most of these techniques were able to detect a user that always annotate spam tags. However, finer levels of detection are seldom discussed. In this work, we will propose a method based on a text mining approach to discover the relations between Web pages and there tag posts. These relations are then used to compute the similarity between a Web page and its tag post to decide if it is a spam post. Preliminary experiments show that the accuracy of the post-level spam detection task is 83%.