Exploration and mining of web repositories

The purpose of the workis to explore the current problems and prospects of mining solution, big web data in real time, as well as the possibility of practical implementation of Web Mining technology for big web data on a practical example.Materials and methods.The study included a review of bibliographic sources on big data mining. We used Web Mining technology for associative analysis of large web data, as well as computer modeling of the practical task of transaction analysis using a general-purpose scripting language (PHP).Results.During the work, the specifics of the Data Mining technology were described, and a modern approach to the analysis of large web data –Web Mining was analyzed. A brief classification of tasks solved using Web Mining technology is given. The problem of data mining of large web data in a general-purpose scripting language (PHP) has been solved: the lack of libraries for data mining, the difficult normalization of data to the form necessary for associative analysis, interaction with the database management system. Also, an example showing an approach to the mining of large web data was implemented. Based on the understanding of Web Mining technology and the described difficulties of analyzing web data in the PHP language, methods for effectively solving the practical problem of analyzing web data based on transactions committed in a dynamic web application have been proposed. A module for associative analysis of customer transactions in the programming language PHP was developed. The module includes an intelligent data processing class. The structural scheme of the module and system architecture were developed. The constructed module allows us to solve the main part of the problem of associative analysis of large web data using Web Mining technology in order to solve the problem of identifying patterns in a large array of web data. Associative analysis of web data is much faster because of the combination of a general-purpose scripting language and an object-oriented approach.Conclusion.According to the results of the study, it can be argued that the current state of the technology for the analysis of large web data allows efficiently process data objects, identify patterns, obtain hidden data and receive complete statistical data in real time. The results can be used both for the purpose of the initial research of technologies for analyzing large web data, and as an addition to the content management system for the intelligent analysis of web data. The usage of the technology of associative analysis and the created universal handler class makes the created module flexible, while the possibility of manual integration makes this module universal. With manual integration, the database management system is not important. Algorithm methods work with selected data. This factor greatly simplifies the further development of program code.

In this paper we explore some of the opportunities and challenges for machine learning on the Semantic Web. The Semantic Web provides standardized formats for the representation of both data and ontological background knowledge. Semantic Web standards are used to describe meta data but also have great potential as a general data format for data communication and data integration. Within a broad range of possible applications machine learning will play an increasingly important role: Machine learning solutions have been developed to support the management of ontologies, for the semi-automatic annotation of unstructured data, and to integrate semantic information into web mining. Machine learning will increasingly be employed to analyze distributed data sources described in Semantic Web formats and to support approximate Semantic Web reasoning and querying. In this paper we discuss existing and future applications of machine learning on the Semantic Web with a strong focus on learning algorithms that are suitable for the relational character of the Semantic Web's data structure. We discuss some of the particular aspects of learning that we expect will be of relevance for the Semantic Web such as scalability, missing and contradicting data, and the potential to integrate ontological background knowledge. In addition we review some of the work on the learning of ontologies and on the population of ontologies, mostly in the context of textual data.

The purpose of the work is to study the current problems and prospects of the solution for processing big data received or stored in the Internet (web data), as well as the possibility of practical realization of Data Mining technology for big web data on practical example. Materials and methods. The study included a review of bibliographic sources on big data analysis problems.Data Mining technology was used to analyze large web data, as well as computer modeling of a practical problem using the C # programming language and creating a DDL database structure for accumulating web data.Results. In the course of the work, the specifics of big data were described, the main characteristics of big data were highlighted, and modern approaches to processing big data were analyzed. A brief description of the horizontal-scalable architecture and the BI-solution architecture for big data processing is given. The problems of processing large web data are formulated: limiting the speed of access to data, providing access via network protocols through general-purpose networks.An example showing the approach to processing large web data was also implemented. Based on the idea of big data, the described complexities of web data processing and the methods of Data Mining, techniques were proposed for effectively solving the practical problem of processing and searching patterns in a large data array.The following classes have been developed in the C # programming language:Class of receiving web data via the Internet; Data conversion class;Intelligent data processing class;Created DDL script that creates a structure for the accumulation of web data.A single UML class diagram has been developed.The constructed system of data and classes allows to solve the main part of the problems of processing large web data and perform intelligent processing using Data Mining technology in order to solve the problem posed of identifying certain records in a large array. The combination of object-oriented approach, neural networks and BI-analysis to filter data will speed up the process of data processing and obtaining the result of the studyConclusion. According to the results of the study, it can be argued that the current state of technology for analyzing large web data allows you to efficiently process data objects, identify patterns, get hidden data and get full-fledged statistical data.The obtained results can be used both for the purpose of the initial study of big data processing technologies, and as a basis for developing an already real application for analyzing web data. The use of neural networks and the created universal classes-handlers makes the created architecture flexible and self-learning, and the class declarations and the base DDL structure will greatly simplify the development of program code.

IntroductionDrug utilization is currently assessed through traditional data sources such as big electronic medical records (EMRs) databases, surveys, and medication sales. Social media and internet data have been reported to provide more accessible and more timely access to medications' utilization.ObjectiveThis review aims at providing evidence comparing web data on drug utilization to other sources before the COVID-19 pandemic.MethodsWe searched Medline, EMBASE, Web of Science, and Scopus until November 25th, 2019, using a predefined search strategy. Two independent reviewers conducted screening and data extraction.ResultsOf 6,563 (64%) deduplicated publications retrieved, 14 (0.2%) were included. All studies showed positive associations between drug utilization information from web and comparison data using very different methods. A total of nine (64%) studies found positive linear correlations in drug utilization between web and comparison data. Five studies reported association using other methods: One study reported similar drug popularity rankings using both data sources. Two studies developed prediction models for future drug consumption, including both web and comparison data, and two studies conducted ecological analyses but did not quantitatively compare data sources. According to the STROBE, RECORD, and RECORD-PE checklists, overall reporting quality was mediocre. Many items were left blank as they were out of scope for the type of study investigated.ConclusionOur results demonstrate the potential of web data for assessing drug utilization, although the field is still in a nascent period of investigation. Ultimately, social media and internet search data could be used to get a quick preliminary quantification of drug use in real time. Additional studies on the topic should use more standardized methodologies on different sets of drugs in order to confirm these findings. In addition, currently available checklists for study quality of reporting would need to be adapted to these new sources of scientific information.

World Wide Web is today the most important media for collecting, sharing and distributing information. Education is one of the fields where web-based technology has been quickly and successfully adopted. Web mining is active research area which involves transformation and interpretation of web data to find out the hidden information by using various data mining techniques which will be useful in enhancing E learning environment. Web usage mining for e learning environment can mine the server logs to find the user usage pattern so that users can be provided with more personalized services. WUM is one of the latest techniques in web data mining which can provide an effective platform to the learner. We focus on identifying learning pattern of learners. This paper introduces how by using WUM techniques we can make e learning environment more effective. Based on WUM technique how we can design an effective e learning environment and implementation of the model builds more intelligent system and can promote interest of learners or learning environment is modifying its behavior to reflect learning style.

Internet search is done by exploring the link and keyword frequency. In 2012, Google released Knowledge Graph --Semantic Web. The human reasoning can be enhanced by the use semantic web an emerging area. Most of the current applications link open data views due to which there is huge flow of data in semantic web, particularly Resource Description Framework (RDF) data. In the semantic web research community this leads to design and development of scalable data processing techniques for RDF data. The aim of semantic web is to make available semantically connected data across the globe. This is a review paper giving analysis of techniques implemented to achieve the aim of semantic web, various approaches to processes RDF data. Within the semantic web community, RDF is a common acronym because it forms one of the basic building blocks for forming the web of semantic data, called a graph database. This paper compares various methodologies followed by different researchers along with the results analysis of implemented techniques over different datasets.

This chapter aims to master web mining and Information Retrieval (IR) in the digital age, thus describing the overviews of web mining and web usage mining; the significance of web mining in the digital age; the overview of IR; the concept of Collaborative Information Retrieval (CIR); the evaluation of IR systems; and the significance of IR in the digital age. Web mining can contribute to the increase in profits by selling more products and by minimizing costs. Web mining is the application of data mining techniques to discover the interesting patterns from web data in order to better serve the needs of web-based multifaceted applications. Mining web data can improve the personalization, create the selling opportunities, and lead to more profitable relationships with customers in global business. Web mining techniques can be applied with the effective analysis of the clearly understood business needs and requirements. Web mining builds the detailed customer profiles based on the transactional data. Web mining is used to create the personalized search engines which can recognize the individuals' search queries by analyzing and profiling the web user's search behavior. IR is the process of obtaining relevant information from a collection of informational resources. IR has considerably changed with the expansion of the Internet and the advent of modern and inexpensive graphical user interfaces and mass storage devices. The effective IR system, including an active indexing system, not only decreases the chances that information will be misfiled but also expedites the retrieval of information. Regarding IR utilization, the resulting time-saving benefit increases office efficiency and productivity while decreasing stress and anxiety. Most IR systems provide the advanced searching capabilities that allow users to create the sophisticated queries. The chapter argues that applying web mining and IR has the potential to enhance organizational performance and reach strategic goals in the digital age.

Web Data Mining is the application of data mining technology in the Web environment collections, and potentially useful patterns or information found in the relevant data to browse the site from a large number of Web documents. E-commerce is becoming increasingly important in modern business activities, business data processing, highlighting the importance of data mining. This article discusses the main methods of Web data mining in detail and the function and application of data mining techniques in e-commerce trading platform. Web Usage mining excavation left in the server-side access records when the user accesses the Web. The paper presents the building of e-commerce trading platform based on Web data mining.KeywordsE-commerceData miningTrading platformWeb mining

It's easy to see how, in post-Soviet Eastern Europe, a five-day discussion of connectivity and information sharing could be read as a metaphor for a new era of social openness and understanding. That said, the Twelfth International World Wide Web Conference held in May in Budapest, Hungary nonetheless focused largely on nuts and bolts, with tracks devoted to rising trends in electronic security, data mining, and e-commerce. But the most intriguing of the tracks might well have been those about the Semantic Web, Semantic Web services, and the connections between them. The Semantic Web remains in its infancy, but the technologies behind it could still promise to dramatically rewrite the way the Internet operates in the coming years.

The prolific growth of web-based applications and the enormous amount of data involved therein led to the development of techniques for identifying patterns in the web data. Web mining refers to the application of data mining techniques to the World Wide Web. Web usage mining is the process of extracting useful information from web server logs based on the browsing and access patterns of the users. The information is especially valuable for business sites in order to achieve improved customer satisfaction. Based on the user’s needs, Web Usage Mining discovers interesting usage patterns from web data in order to understand and better serve the needs of the web based application. Web Usage Mining is used to discover hidden patterns from weblogs. It consists of three phases like Preprocessing, pattern discovery and Pattern analysis. In this paper, we present each phase in detail, the process of extracting useful information from server log files and some of application areas of Web Usage Mining such as Education, Health, Human-computer interaction, and Social media.

Nowadays, the process of data mining is one of the most important topics in scientific and business problems. Grid Computing can be used as infrastructures to provide an effective computational support for distributed data mining applications. The paper proposed a data mining architecture from large-scale distributed and semantically heterogeneous data resources in order to efficiently support knowledge acquisition. Grid technology and semantic web technology were combined to solve the difficult problem of distributed data mining. Data mining in the Grid environment can provide highly efficient and powerful data analysis and knowledge discovery solution and semantic web can effectively solve the heterogeneity of data sources and make them available for data mining.

Semantic mashup of biomedical data

Web mining is one of the mining technologies, which applies data mining techniques in large amount of web log data. Web navigational mining discovers users’ access patterns from web logs. This information can be used to identify the behavior of the web user. However, the web data will grow rapidly in the short time, and some of the web data may be antiquated. The user behavior may be changed when the new web data is inserted into and the old web data is deleted from web logs. Therefore, the user behavior must be re-discovered from the updated web logs. However, it is very time-consuming to re-find the users’ access patterns. Hence, many researchers pay attention to the incremental mining, which utilizes the previous mining results and finds new patterns just from the inserted or deleted part of the web logs such that the mining time can be reduced. The present paper proposes an efficient incremental web navigational mining algorithm for discovering web navigational patterns when the user sequences are inserted into and deleted from original database. It avoids re-finding the original web navigational patterns and re-counting the original candidate sequences. It uses lattice structure to keep the previous mining results such that just new candidate sequences need to be computed. Hence, the web navigational patterns can be obtained rapidly when the navigational sequence database is updated. Besides, maximal web navigational patterns can also be obtained easily by traversing the lattice structure. The experimental results show that the present algorithm is more efficient than the other approaches.

Semantic web offers information for both individuals and computers to preserve large data scale semantically and provide a meaningful content of unstructured data. It offers new benefits for big-data research and applications. Big Data and Semantic Web are the epitome of computer sciences latest trend study subjects. Big data is a new tendency relates to a huge set of datasets including structured, semi-structured and unstructured data collected from different sources. Their integration faces many issues, as it is difficult to process this information using traditional databases and software methods. Recent works on the incorporation of both these technologies have provided a scalable approach in Data Analytics. This article attempts to give a comparative study of methods in integrating Big Data with Semantic Web, describing how Semantic Web makes Big Data smarter, revisits the difficulties and possibilities of Big Data and Semantic Web, and lastly summarizes the future direction of this inclusion.

Web usage mining is the application of data mining techniques to discover usage patterns and behaviors from web data (clickstream, purchase information, customer information, etc.) in order to unde...