Functional Classification of Web Pages with Deep Learning

World Wide Web (WWW) is a platform that explores a wide range of information used for the development of web applications. Some examples of these applications include social network analysis, personalized item recommendations, and web page classification and ranking. Among these applications, search engines and web page ranking are particularly important as they consistently index and store billions of web pages on the internet. The main objective of this paper is to create an innovative framework for the classification and re-ranking of web pages using intelligent techniques. The framework is structured into two key phases: classification and re-ranking-based retrieval. In the initial classification phase, a series of pre-processing steps are implemented, including the elimination of HTML tags, punctuation, stop words, and the application of stemming. After pre-processing, a word-to-vector conversion is performed, followed by feature extraction utilizing Principal Component Analysis (PCA). This sequence of actions leads to optimal feature selection, which is vital for the precise classification of web pages. Given the multitude of features present in web pages that can compromise classification accuracy, this study employs a novel meta-heuristic algorithm, the Opposition Based-Tunicate Swarm Algorithm (O-TSA), to facilitate optimal feature selection. The refined features are subsequently processed through the Enhanced Convolutional-Recurrent Neural Network (E-CRNN), enhanced by O-TSA, resulting in the effective classification of diverse web page categories. In the second phase, the re-ranking process is executed using O-TSA, which establishes the objective function based on a similarity function (correlation) for URL matching, leading to optimal re-ranking of web.

The World Wide Web is growing at an uncontrollable rate. Hundreds of thousands of web sites appear every day, with the added challenge of keeping the web directories up-to-date. Further, the uncontrolled nature of web presents difficulties for Web page classification. As the number of Internet users is growing, so is the need for classification of web pages with greater precision in order to present the users with web pages of their desired class. However, web page classification has been accomplished mostly by using textual categorization methods. Herein, we propose a novel approach for web page classification that uses the HTML information present in a web page for its classification. There are many ways of achieving classification of web pages into various domains. This paper proposes an entirely new dimension towards web page classification using Artificial Neural Networks (ANN). Index Terms: World Wide Web, Web page classification, textual categorization, HTML, Artificial Neural Networks, ANN.

Information filtering and information retrieving applications are based on web page classification methods. Usually, web pages serve different functionalities or develop different topics or subjects. The diversity of web page content increases the need for automatic web page classification, making it a challenging task at the same time. Considering that the main component of the content of a web page is most often represented by the text and the classification of the text is a problem intensively studied in the last years, with researchers reporting state-of-the-art results for various methods, the idea of applying these methods on the text extracted from web pages could lead to important results. In this work, we revisit our experimental study on convolutional neural networks for multi-label multi-language web page classification with a new approach that consists of dividing the classification problem into functional classification and subject classification of web pages. From the experimental evaluation, one may conclude that the separation of the functional and subject classification of web pages leads to an improvement of the overall results.

Automatic web page classification can be used in domain-specific search engines to help users get the specific information more conveniently and precisely on Internet. The semantic similarity and noisy data in domain-specific web pages make traditional classifier perform poorly on them. In this paper, a dictionary-based multilingual web page classification method is proposed to try to improve the classification performance. A domain-specific dictionary is constructed in the method to intensify the domain-specific knowledge in the pages. An automatic encoding detection and integration method is also introduced in the classifier to extract Chinese and English information precisely from the multilingual pages. After verified in the experiments, the method is integrated into a real domain-specific search engine where it shows good effectiveness. Keywords-Web page classification, Search engine, Domainspecific knowledge, Dictionary

Googling for Health Information

We propose a generic multiple classifier system based solely on pairwise classifiers to classify web pages. Web page classification is getting huge attention now because of its use in enhancing the accuracy of search engines and in summarizing web content for small-screen handheld devices. We have used a Support Vector Machine (SVM) as our core pair-wise classifier. The proposed system has produced very encouraging results on the problem web page classification. The proposed solution is totally generic and should be applicable in solving a wide range of multiple class pattern recognition problems.KeywordsSupport Vector MachineHandwritten CharacterSupport Vector ClassifierMultiple Classifier SystemStructure Support Vector MachineThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

These days, the Internet is developing at an exponential rate and can cover just about any data required. Nonetheless, the immense measure of web pages makes it more difficult to effectively discover the target data by a user. Therefore, an efficient method, for classifying this huge amount of data is essential if the web pages are to be exploited to its full potential. In the domain of automatic web page classifier many approaches have been tried to solve this problem using different Machine learning-based algorithms including Support Vector Machine (SVM), Naive Bayes, Decision Tree, K-Nearest Neighbor (K-NN) and Neural Networks. However, there is a lack of comparison between these algorithms to find a better framework for the classification and analysis of health related web pages. In this research study, we compare two commonly used supervised Machine Learning algorithms; Support Vector Machines (SVM) and Naive Bayes to classify web pages which provide drugs related information of patients for example side effects, patient action and follow-up information for patients. We use Unified Medical Language System (UMLS) to annotate the health related concepts in Web pages and train SVM and Naive Bayes classifiers in General Architecture for Text Engineering to classify health related and non-health related Web pages. The evaluation was performed using K-fold cross validation using four runs on a data set of fifty Web pages. Results found that SVM performed better to classify health and non-health related pages in terms of precision, recall and F-measure.

Hidden Markov models (HHMs), while well applied in fields such as speech recognition and optical character recognition, have not been used in post-classification for search engines. We explore the use of HMMs for optimization of search engines tasks, specifically focusing on how to construct a new model structure to improve the classification of web pages. We show that a manually constructed new structure model that contains only two states and two classes of observations per field can produce good classification results, and discuss strategies for learning the model structure automatically from data. We also demonstrate that the use of new structure model to classify the search results using some search engines and some different search keywords provide a significant improvement in search accuracy. Our models are applied to the task of post-classifying the web pages selected by the search engine Google, and achieve a classification accuracy of 93.4.

The lack of sufficient labeled Web pages in many languages, especially for those uncommonly used ones, presents a great challenge to traditional supervised classification methods to achieve satisfactory Web page classification performance. To address this, we propose a novel Nonnegative Matrix Tri-factorization (NMTF) based Dual Knowledge Transfer (DKT) approach for cross-language Web page classification, which is based on the following two important observations. First, we observe that Web pages for a same topic from different languages usually share some common semantic patterns, though in different representation forms. Second, we also observe that the associations between word clusters and Web page classes are a more reliable carrier than raw words to transfer knowledge across languages. With these recognitions, we attempt to transfer knowledge from the auxiliary language, in which abundant labeled Web pages are available, to target languages, in which we want classify Web pages, through two different paths: word cluster approximations and the associations between word clusters and Web page classes. Due to the reinforcement between these two different knowledge transfer paths, our approach can achieve better classification accuracy. We evaluate the proposed approach in extensive experiments using a real world cross-language Web page data set. Promising results demonstrate the effectiveness of our approach that is consistent with our theoretical analyses.

The rapid growth of web has made it a huge source of information which will make the availability of data easier and more efficient if its content is well organized. Automatic classification of web pages is one of the major methods in the Web Content Mining (WCM) which can be of great value in the development and maintenance of web directories. Based on the analysis done, CMAC neural network showed faster learning in high dimensional problems, but considering the heavy data on the web, the main challenge in web page classification is how to deal with high dimensional feature space which increase the memory required by CMAC neural network. In the present paper a Classifier-CMAC neural network model is proposed for use in content based web page classification which requires less memory. The results reveal that the proposed model is more useful than any other algorithms.

In 2020, Most Filipinos are using the internet due to COVID-19 pandemic lockdowns. The internet is not limited to adults and children might be exposed to online adult content and abuse. The Philippine Internet Service providers fail to capture pornographic web pages that are not for child viewing. A Web Page classifier would help in detecting and classifying web pages. In this study, a total of 12000 web pages with adult content and academic web pages were collected using scrapy and existing datasets from DMOZ were used to create a Support Vector Machine (SVM) multi-class classifier. To improve the accuracy of the SVM model, data preprocessing was performed to remove noisy and irrelevant data from the dataset. The text in the web pages was used to train the SVM classifier by using Term Frequency and Inverse Document Frequency, Count vectorizer, and Word2vec Skip-gram embedding with TF-IDF as a multiplier. A series of experiments were conducted using multiple word embedding techniques. The SVM model built using word2vec with TF-IDF multiplier outperforms the SVM model built using TF-IDF and Count Vectorizer. The word embedding generated using word2vec was generated with a window size of 9 and a vector dimension of 900. The SVM model built using word2vec shows an S6% accuracy. The SMV model is deployed in the Django framework and a chrome plugin was created to use the SVM model using REST API.

Integrating a web application into an automated business process requires to design wrappers that get user queries as input and map them onto the search forms that the application provides. Such wrappers build on automatic navigators which are responsible for navigating to the pages that provide the information required to answer the original user queries. A navigator relies on a web page classifier that discerns which pages provide the information and which do not. In the literature, there are many proposals to classify web pages, but none of them fulfills the requirements for a web page classifier in a navigator context. We address the problem of designing an unsupervised web page classifier that builds solely on the information provided by the URLs and does not require extensive crawling of the site being analysed. Our contribution is CALA, a new automated proposal to generate URL-based web page classifiers. Its salient features are that it does not need to previously crawl the complete web site, it is unsupervised, it does not require to download a page before classifying it, and it is computationally tractable. It has been validated by a number of experiments using real-world, top-visited web sites.

In the modern era, web data retrieval and data analytics play a crucial role for taking intelligent decisions in Internet of Things (IoT) environment. In IoT environment, various objects perform the tasks of sensing and computation for providing uninterrupted services (e.g., e-health, e- transportation, security access, etc.) to the end users. However, accessing the relevant and accurate information with reduced delay is still a challenging task in IoT environment. Although this aspect has been explored in the literature, the existing proposals have high complexity and require long time for accessing the relevant information from different IoT objects located across the globe. The information may be located across different web pages, which are linked together irrespective of their geographical locations. So, this paper addresses the issues such as accuracy, context- aware, reduced delay with low complexity in accessing the information from a remote device by the end users. In the proposed scheme, the strength of a web page which contains the relevant information to be fetched is judged by the quality of content and the inter- connections between different web pages. The proposed scheme simplifies the rank score calculation of these web pages and provides quality web pages at the top of the search result pages by demoting spam web pages. Bias connected web pages are verified by the linkage information of spam web pages. The Quality Assessment for Information Retrieval (QAIR) algorithm is proposed for the classification of web pages. The proposed algorithm computes the QAIR score by evaluating the web page quality. Microsoft Learning to Rank dataset has been used for the experiments, which consists of 239092 query-url pairs. By using this dataset, the computed QAIR score is compared with the PageRank score. This comparison determines the category of web page, i.e., either the page is strong or weak. The proposed scheme has been validated with decision tree followed by ten- fold cross validation, which results in an accuracy of 92.4%.

Visual content-based web page categorization with deep transfer learning and metric learning

Chinese Web page classification (WPC) has been considered as a hot research area in data mining. In order to effectively classify Web pages, we present a Web page categorization based on a least square support vector machine (LS-SVM) with latent semantic analysis (LSA). LSA uses singular value decomposition (SVD) to obtain latent semantic structure of original term-document matrix solving the polysemous and synonymous keywords problem. LS-SVM is an effective method for learning the classification knowledge from massive data, especially on condition of high cost in getting labeled classical examples. We adopt a novel method of Web page expression, and make use of summarization algorithm to reduce the noise of Web pages. A preliminary experimental comparison is made showing encouraging results.