Automatic Web News Extraction Based on DS Theory Considering Content Topics

Situation estimation and threat assessment in air combat are crucial issues. Due to the limitations of the sensor’s precision, terrible battlefield environment and the deliberately interfere of the enemy, the information of threat assessment is often ambiguous and uncertain. Belief function theory has unique and significant advantages in uncertain information processing and fusion, so it is suitable for threat assessment. Our proposed approach handles information begins with feature extraction using fuzzy sets, then interspecific fusion and time domain fusion are carried out by belief function theory and evidence discount. Finally, the data is processed into a definite number to measure the threat index of the target. In the fusion process, evidence discount improves the quality and accuracy of convergence. The simulation example verifies that our proposed method is not only able to accurately analyze and convey the sensor information, but also has better real-time performance and the ability to process uncertain information.

IntroductionPatients increasingly rely on online health information to understand and manage their diseases. Concerns about the quality and readability of these materials have been reported in the literature. Poor quality and difficult-to-read information lead to delayed diagnoses and adverse outcomes. We assessed the quality and readability of online health information about iron deficiency anemia (IDA) on Google.MethodWe searched for online web pages using the term "iron deficiency anemia"on Google. One hundred and twelve out of 200 web pages were included. We assessed web page typology, readability, the Journal of the American Medical Association (JAMA) score, the DISCERN score, and the Health on the Net Foundation certification (HONcode). Statistical analysis was performed with R version 4.2.2.ResultNon-profit and scientific journal web pages were the most common. Scientific journal web pages were of the highest quality. News web pages were the most readable. The first Google Page web pages did not have greater JAMA scores or lower Flesch-Kinkaid Reading Grade Level (FKGL) and Simple Measure of Gobbledygook (SMOG) scores. Forty-six percent of all web pages were high-quality. Web pages on the first Google page were more likely to have HONCode certification.ConclusionWe highlight gaps in the readability and quality of online information about IDA. Online web pages exceeded the recommended reading level for patients. Most web pages were low quality; only a quarter were HONcode-certified; and the first Google page web pages were not higher in quality than the later web pages on search.

As the web endures to develop, there is an enormous amount of information which is typically designed for its users, which makes it difficult to extract relevant data from numerous sources. In this paper, we propose an approach for extracting the main content from news web pages. Our approach is based on the concept of tokenisation of HTML page, these tokens construct the tag tree; web pages from different websites are parsed into Tag tree and generated a template from each web pages and discover matching patterns and multiple sequence alignment. It finds and removed shared token sequences from the web pages until the relevant information is extracted from them. We perform experiments on 500 web pages from ten different news websites. Experimental results show that our approach efficiently extracts the relevant information.

The problem of soft information fusion is one of the key challenges in both fields of information fusion and situation awareness. However uncertainty modelling in the context of semantic information fusion is not well-researched.In this paper we present the current state-of-the-art regarding fusion of uncertain semantic information using simple conceptual graphs and propose methods to improve its handling of partial conflict, reliability and expressiveness regarding modal uncertainty. Particular attention is paid to the issues of scalability and tractability. Similarities to other uncertainty handling frameworks such as possibility theory and belief functions are discussed. Potential avenues are identified for further research regarding uncertainty representation in semantic information fusion.

ABSTRACTDempster–Shafer (D–S) evidence theory is very efficient and widely used mathematical tool for uncertain and imprecise information fusion for decision making. D–S rule is criticised by many researchers as it gives illogical and counterintuitive results especially when the series of evidence provided by various experts are in a high degree of conflict. Various attempts have been made and several alternatives proposed to this rule. In this paper, a new alternative is proposed which considers the possibility of an error made by experts while providing evidence, calculates the error and incorporates in the revised masses. The validity and efficiency of the proposed approach have been demonstrated with numerous examples and the results are compared with already existing methods.HighlightsAn alternative method is proposed to handle the conflicting evidence.An Error In Judgement while gathering evidence is considered and incorporated before combining evidence.The method is simple and gives better and reasonable results than other previous methods when evidence conflicts

A Web page typically contains many information blocks. Apart from the main content blocks, it usually has such blocks as navigation panels, copyright and privacy notices, and advertisements. We call these blocks the noisy blocks. The noises in Web pages can seriously harm Web data mining. To the question of climinating these noises, we intro duce a new tree structure, called Style Tree, and study an algorithm how to construct a site style tree. The Style Tree Model is employed to detect and climinate noises in any Web pages of the site. An information based measure to determine which element node is noisy is also constructed. In addition, the applications of this method are discussed in detail. Experimental results show that our noises climination technique is able to improve the mining results significantly.

In this paper, we propose a new approach to discover informative contents from a set of tabular documents (or Web pages) of a Web site. Our system, InfoDiscoverer, first partitions a page into several content blocks according to HTML tag in a Web page. Based on the occurrence of the features (terms) in the set of pages, it calculates entropy value of each feature. According to the entropy value of each feature in a content block, the entropy value of the block is defined. By analyzing the information measure, we propose a method to dynamically select the entropy-threshold that partitions blocks into either informative or redundant. Informative content blocks are distinguished parts of the page, whereas redundant content blocks are common parts. Based on the answer set generated from 13 manually tagged news Web sites with a total of 26,518 Web pages, experiments show that both recall and precision rates are greater than 0.956. That is, using the approach, informative blocks (news articles) of these sites can be automatically separated from semantically redundant contents such as advertisements, banners, navigation panels, news categories, etc. By adopting InfoDiscoverer as the preprocessor of information retrieval and extraction applications, the retrieval and extracting precision will be increased, and the indexing size and extracting complexity will also be reduced.

In this paper we propose a new dataset for information extraction from news web pages. Accurate collection of news articles is necessary to build systems that aggregate and analyze data from a large number of news sources. However, existing news page datasets are designed for the problem of article content extraction and do not consider other metadata such as dates or authors, while modern generic approaches for structured web data extraction are evaluated on other domains, mostly e-commerce products websites. Our dataset contains 724 web pages from 114 Russian news websites. On each page we manually annotated title, text, publication date, tags and other article attributes. We describe data collection and annotation process and demonstrate evaluation results for open source tools and neural network based approaches. The dataset is publicly available on GitHub.

There are various approaches for web news extraction, including tree-edit distance approach that needs to assume the existence of web templates, visual wrapper based approach that requires large training sets and statistical approach whose flexibility is low. In this paper, a blocking tag based Web news extraction approach is proposed, which automatically detects the blocking tags that break the web page down into functional areas and then analyzes the web page according to the blocking tags to find out the news content. We have implemented the proposed news extraction approach in a news search engine which has been applied in business of an intelligence enterprise. Compared with related work, our approach does not require the web page templates or large training sets, and the complexity is lower.

Over the years, wireless communication networks have been widely used by a large community of users in wide variety of applications such as intelligent transportation systems, energy management, safety, and security etc. But, during this era, due to large number of user's request, there may be a performance bottleneck in some part of the network with respect to various QoS parameters such as congestion and network delay. Hence, there is a requirement of an efficient classification technique to reduce congestion in the network so that throughput of various applications can be increased. Classification helps in searching, sorting, retrieval, and querying of a document for the wireless networks. World Wide Web (WWW) contains huge repository of information in the form of web pages. However, size of Internet is growing day-by-day. The huge repository of information poses challenge to collect and process the relevant related information of a particular domain. So, traditional text classification techniques are difficult to apply on the rapidly growing web-based contents. Hence, novel approaches and techniques need to be devised to reduce the manual efforts in web page classification. Keeping focus on these points, this paper proposes a novel approach for multiclass classifier based on unique personality features of the web page of particular domain category for the next generation wireless networks. Personality features are collected and assigned weights in the proposed scheme. Then, the proposed classifier is trained based on these special features. Results obtained depict that proposed classifier successfully classified news domain pages, education, resume, online shopping, and research web pages from large database repository. Accuracy of the proposed classifier is found to be satisfactory from a large data set of different categories. Also, there is a 10---15 % overall performance gain using the proposed scheme in comparison to the other existing schemes.

In order to realize a stable and effective gearbox fault diagnosis system, an uncertain information fusion method based on BP neural network and modified DS evidence theory is proposed. The common gearbox fault diagnosis system consists of two dominating parts: signal feature extraction and fusion decision. Actually, the main task of most papers is to extract the characteristics of gearbox fault with various algorithms, whereas the fusion decision part is still absent. To manage the integration of gearbox fault diagnosis in the fusion decision part, this paper puts forward a novel fusion method by the introduction of two-layer fusion. In this paper, BP neural network is used to establish the local-layer fusion, while an improved DS evidence theory is taken to build decision-layer fusion. It can be seen from the definition and derivation of the two-layer fusion, the original method settles the uncertainty of multi-sensor information fusion in gearbox fault diagnosis system. According to the experiment results and analyses, the innovative algorithm improves the accuracy of fault identification and diagnosis, in addition, reduces the uncertainty of fault diagnosis system.

Quantifying uncertainty is a hot topic for uncertain information processing in the framework of evidence theory, but there is limited research on belief entropy in the open world assumption. In this paper, an uncertainty measurement method that is based on Deng entropy, named Open Deng entropy (ODE), is proposed. In the open world assumption, the frame of discernment (FOD) may be incomplete, and ODE can reasonably and effectively quantify uncertain incomplete information. On the basis of Deng entropy, the ODE adopts the mass value of the empty set, the cardinality of FOD, and the natural constant e to construct a new uncertainty factor for modeling the uncertainty in the FOD. Numerical example shows that, in the closed world assumption, ODE can be degenerated to Deng entropy. An ODE-based information fusion method for sensor data fusion is proposed in uncertain environments. By applying it to the sensor data fusion experiment, the rationality and effectiveness of ODE and its application in uncertain information fusion are verified.

The article considers some methodological aspects of the concept of “virtual reality”, used in the research area of computer technologies and their interaction with the mental world of the individual. In the paper the virtual online reality is defined as the picture of the world presented in the totality of cognitive-semiotic tools on the web pages with informational and news content. The aim of this work is to describe the categorical features of the phenomenon. Theoretical background of the research consists of the works in the field of Internet linguistics and psychological virtualistics, upon those a brief overview of the approaches to the description of the properties and characteristics of virtual reality is provided. In the framework of the proposed cognitive-semiotic approach the author offers a working list of the categorical features of online virtual reality: tessellation, visual format, carnival character, mobility. The paper provides the substantiation of the described features and outlines possible directions of further linguistic and interdisciplinary research of this phenomenon.

Most information in the world exists in the format of text, such as news articles and web pages. Different lines of research have been conducted to discover, understand and access knowledge about real-world entities and relations from text. However, the application of these word oriented methods to ontology integration tasks has not been yet explored. In this paper, we apply these word oriented methods to ontology integration tasks in which we analyze a noun phrase (NP) to identify its head noun, which is useful to avoid wrong relations between entities. We also propose a collaborative acquisition algorithm that combines WordNet-based and Text corpus.

In view of the deficiencies in the studies of the multi-media teaching resource management method and resource management evaluation model, a multi-media teaching resource management evaluation model in the big data environment is proposed in this paper based on the existing work. The teaching resource management evaluation method of the proposed model is further studied based on the elaboration of the component elements of the model. Guided by the multimedia teaching resource management evaluation model, the construction method for the multimedia teaching resource management evaluation model is studied in the open big data environment. And the multimedia remote teaching data of 360 encyclopedia and news web pages are used as the basis for experimental verification. The results show that the proposed model and method can effectively evaluate the multimedia teaching resource management.