AutoRM: An effective approach for automatic Web data record mining

A Web page classification system based on a genetic algorithm using tagged-terms as features

The main issue for effective Web information extraction is how to recognize similar patterns in a Web page. Traditionally, it has been shown that pattern matching by using the HTML DOM tree is more efficient than the simple string matching approach. Nonetheless, previous tree-based pattern matching methods have problems by assuming that all HTML tags have the same values, assigning the same weight to each node in HTML trees. This paper proposes an enhanced tree matching algorithm that improves the tree edit distance method by considering the characteristics of HTML features. We assign different values to different HTML tree nodes according to their weights for displaying the corresponding data objects in the browser. Pattern matching of HTML patterns is done by obtaining the maximum mapping values of two HTML trees that are constructed with weighted node values from HTML data objects. Experiments are done over several Web commerce sites to evaluate the effectiveness of the proposed HTML tree matching algorithm.

Data logger is a key component in building a telemetry system. Using the telemetry system, the activity measurements of an object and retrieval of measurement data can be done remotely (remote). Universal Data Logger (UniDalog), the data logger is a device that has ability to store measurement data, download the data remotely, as well as setting of data logger and remote sensor calibration. By utilizing the Global System for Mobile communication (GSM) network and facilities provided by the operator for data communications. This paper will discuss the design software application that is used to access any Unidalog remotely (from a distance) as Remote Terminal Unit (RTU) by using the GSM network as a medium for data communication. Processing software application design models using evolutionary development models; specification, remote application development and validation methodology to get the system iteratively to create modules are part of the telemetry monitoring.

Conventional Web search engines rank their searched results page by page. That is, conventionally, the information unit for both searching and ranking is a single Web page. There are, however, cases where a set of searched pages shows a better similarity (relevance) to a given (keyword) query than each individually searched page. This is because the information a user wishes to have is sometimes distributed on multiple Web pages. In such cases, the information unit used for ranking should be a set of pages rather than a single page. In this paper, we propose the notion of a “page set ranking”, which is to rank each pertinent set of searched Web pages. We describe our new algorithm of the page set ranking to efficiently construct and rank page sets. We present some experimental results and the effectiveness of our approach.

With advances in artificial intelligence and semantic technology, search engines are integrating semantics to address complex search queries to improve the results. This requires identification of well-known concepts or entities and their relationship from web page contents. But the increase in complex unstructured data on web pages has made the task of concept identification overly complex. Existing research focuses on entity recognition from the perspective of linguistic structures such as complete sentences and paragraphs, whereas a huge part of the data on web pages exists as unstructured text fragments enclosed in HTML tags. Ontologies provide schemas to structure the data on the web. However, including them in the web pages requires additional resources and expertise from organizations or webmasters and thus becoming a major hindrance in their large-scale adoption. We propose an approach for autonomous identification of entities from short text present in web pages to populate semantic models based on a specific ontology model. The proposed approach has been applied to a public dataset containing academic web pages. We employ a long short-term memory (LSTM) deep learning network and the random forest machine learning algorithm to predict entities. The proposed methodology gives an overall accuracy of 0.94 on the test dataset, indicating a potential for automated prediction even in the case of a limited number of training samples for various entities, thus, significantly reducing the required manual workload in practical applications.

Web page clustering is a focal task in Web Mining to organize the content of websites, understanding their structure and discovering interactions among web pages. It is a tricky task since web pages have multiple dimension based on textual, hyperlink and HTML formatting (i.e. HTML tags and visual) properties. Existing algorithms use this information almost independently, mainly because it is difficult to combine them. This paper makes a contribution on clustering of web pages in a website by taking into account a distributional representation that combines all these features into a single vector space. The approach first crawls the website by using web pages’ HTML formatting and web lists in order to identify and represent the hyperlink structure by means of an adapted skip-gram model. Then, this hyperlink structure and the textual information are fused into a single vector space representation. The obtained representation is used to cluster websites using simultaneously their hyperlink structure and textual information. Experiments on real websites show that the proposed method improves clustering results.

<p>Web data extraction is the process of extracting user required information from web page. The information consists of semi-structured data not in structured format. The extraction data involves the web documents in html format. Nowadays, most people uses web data extractors because the extraction involve large information which makes the process of manual information extraction takes time and complicated. We present in this paper WEIDJ approach to extract images from the web, whose goal is to harvest images as object from template-based html pages. The WEIDJ (Web Extraction Image using DOM (Document Object Model) and JSON (JavaScript Object Notation)) applies DOM theory in order to build the structure and JSON as environment of programming. The extraction process leverages both the input of web address and the structure of extraction. Then, WEIDJ splits DOM tree into small subtrees and applies searching algorithm by visual blocks for each web page to find images. Our approach focus on three level of extraction; single web page, multiple web page and the whole web page. Extensive experiments on several biodiversity web pages has been done to show the comparison time performance between image extraction using DOM, JSON and WEIDJ for single web page. The experimental results advocate via our model, WEIDJ image extraction can be done fast and effectively.</p>

In this study we propose a genetic algorithm to select best features for Web page classification problem to improve accuracy and run time performance of the classifiers. The increase in the amount of information on the Web has caused the need for accurate automated classifiers for Web pages to maintain Web directories and to increase search engines' performance. To determine whether a Web page belongs to a specific class (e.g., a graduate student homepage, a course page, etc.) or not, a classifier needs to have “good” features extracted from the Web pages. As every component in a Web page such as HTML tags and terms can be taken as a feature, dimension of the classification problem becomes too high to be solved by well known classifiers like decision trees, support vector machines, etc. To decrease the feature space, we developed a genetic algorithm that determines the best features for a given set of Web pages. It is found that when features selected by our genetic algorithm are used and a kNN classifier is employed, the accuracy improves up to 96%.

Mobile devices have made the world a small place. They are also being used widely, to access the Web. However, because most available web pages are designed for desktop PC in mind, it is inconvenient to browse these large web pages on a mobile device with a small screen. With the enormous increase in mobile users since last decade, to such an extent that it even exceeds the number of desktop users in many developing countries, it becomes crucial to design systems for Web Manipulation, there by providing easier and greater accessibility and device independence. In this paper, we propose a new browsing technique to facilitate efficient web access and provide a better user experience on a device with rather small screen as compared to the usual desktop. It also proposes a novel technique to handle multimedia content, especially the images and video on a web page. In order to solve the problem of incomplete tags in HTML, an equivalent XML is generated for the desired webpage. For efficient browsing, the web page is represented as a multi level hierarchy, with the index of major sections on the first level and the sections indexed again in to sub-sections on the second level and so on, until we reach sub-sections that can be represented as sub-pages that fit in the small screen. We also propose an efficient algorithm for navigating through the sub-pages, thereby providing better user experience. The proposed technique provides better and efficient web browsing on mobile small displays up to user satisfaction.

In web classification, web pages from one or more web sites are assigned to pre-defined categories according to their content. Since web pages are more than just plain text documents, web classification methods have to consider using other context features of web pages, such as hyperlinks and HTML tags. In this paper, we propose the use of Support Vector Machine (SVM) classifiers to classify web pages using both their text and context feature sets. We have experimented our web classification method on the WebKB data set. Compared with earlier Foil-Pilfs method on the same data set, our method has been shown to perform very well. We have also shown that the use of context features especially hyperlinks can improve the classification performance significantly.

Domain-independent web information extraction can be addressed as a structured prediction problem where we learn a mapping function from an input web page to the structured and interdependent output variables, labeling each block on the page. In this paper, built upon an HTML parser of Internet Explorer that parses and renders a web page based on HTML tags and visual appearance, we propose a max margin learning approach for web information extraction. Specifically, the output of the parser is a vision tree, which is similar to a DOM tree but with visual information, i.e., how each node is displayed. Based on this hierarchical structure, we develop a max margin learning method for labeling each of its nodes. Due to the rich connections between blocks on the web page, we further introduce edges that connect spatially adjacent nodes on the vision tree, complicating the problem into a cyclic graph labeling task. A max margin learning method on cyclic graphs is developed for this problem, where loopy belief propagation is used for approximate inference. Experimental results on web data extraction show the feasibility and promise of our approach.

To evaluate the listening conditions of bilateral cochlear implant (CI) users in their daily living environment. Retrospective study. University hospital. Thirty-one adult CI users 16 years or older; 18 underwent sequential surgery, and 13 underwent simultaneous surgery. Inclusion criteria included the availability of CI data logging features and a minimum duration of binaural use of 6 months. Retrospective analysis of data obtained from the automatic scene classifier data logging system. Comparison of data logging and maximum speech discrimination scores of the two surgery groups (sequential vs. simultaneous) to investigate the potential influence of these factors on the listening conditions of CI users. The maximum speech discrimination score of the second CI in the sequential group was significantly worse than that of any other CI in the sequential and simultaneous CI groups. Additionally, the longer the interval between surgeries, the more significant the difference in "time on air" between the first and the second CIs. The second CI in sequential CI surgery had a shorter "time on air" than the first or the bilateral simultaneous CIs; the second CI was also used more frequently in noisy and speech with background noise environments. A second CI may be more frequently used in challenging listening environments because of its binaural auditory effect, despite its lower speech discrimination performance. The timing of sequential implantation and the potential impact of binaural hearing should be considered when developing rehabilitation strategies for individuals with bilateral CIs.

Data extraction in Web is to obtain the desired information to users in Web pages. For a more accurately valuable data extraction, this paper proposes a new method called data extraction based on index path in Web (DEIP) . This approach establishes the index path for each text node using XML DOM; defines the prefix of data-rich by keywords in the index path; generate extraction rule and obtain a wrapper according. The wrapper can extract data automatically in the same domain from a Website. It does relevant to the continuity, the structural similarity, and the location relations of the useful information in Web pages, but not the HTML tag, Experiments indicate that this method is efficient in the recall and the precision of data extraction.

HTML tags are designed to support only the display of Web page content, but the study quantifies their feasibility as proxies for semantic content as well. More specifically, the author examined a number of Web pages to try to quantify whether, consciously or otherwise, authors use headings or the highlighted text accompanying a link to indicate the subject matter of the corresponding page. The investigations focused on empirically evaluating the feasibility of using key Web page elements as proxies to indicate page contents. Additionally, the author compared text retrieval by keyword. The author describes the methods he used and summarizes results. The empirical results suggest that text in HTML headings and in anchor texts is useful for indicating Web page content for logic programming.

A Web information retrieval algorithm based on Web page segment is designed, the key idea of which is to segment each Web page into different topic areas or segments according to its HTML tags and contents since Web pages are semi-structure. First, the algorithm builds a HTML tag tree, and then it combines nodes in the tree under the rule of content similarity and visual similarity. During the process of retrieval and ranking, the algorithm makes full use of the segmentation information to sequence the relevant pages. The experimental results show that this method is able to improve the precision in search significantly and it is also a good reference for the design of the future search engines.