Cheshire 3 Framework White Paper: Implementing Support for Digital Repositories in a Data Grid Environment

Over the last decade data warehousing and data mining tools have evolved from research into a unique and popular applications, ranging from data warehousing and data mining for decision support to business intelligence and other kind of applications. The chapter presents and discusses data warehousing methodologies along with the main components of data mining tools and technologies and how they all could be integrated together for knowledge management in a broader sense. Knowledge management refers to the set of processes developed in an organization to create, extract, transfer, store and apply knowledge. The chapter also focuses on how data mining tools and technologies could be used in extracting knowledge from large databases or data warehouses. Knowledge management increases the ability of an organization to learn from its environment and to incorporate knowledge into the business processes by adapting to new tools and technologies. Knowledge management is also about the reusability of the knowledge that is being extracted and stored in the knowledge base. One way to improve the reusability is to use this knowledge base as front-ends to case-based reasoning (CBR) applications. The chapter further focuses on the reusability issues of knowledge management and presents an integrated framework for knowledge management by combining data mining (DM) tools and technologies with CBR methodologies. The purpose of the integrated framework is to discover, validate, retain, reuse and share knowledge in an organization with its internal users as well as its external users. The framework is independent of application domain and would be suitable for uses in areas, such as data mining and knowledge management in e-government.

The Special Interest Group (SIG) on Text Mining (or BioLINK — Biological Literature, Information and Knowledge; http://www.pdg.cnb.uam.es/BioLINK/) was created to address the need for communication and interchange of ideas in the field of text mining and information extraction applied to biology and biomedicine. Information extraction (IE) is an outgrowth of work in automated natural language processing, which began in the 1950s with work on transformational grammar by Zellig Harris [5,6] and later Noam Chomsky [3,4]. Information extraction technology made rapid progress starting in the late 1980s, thanks to a series of conferences focused on evaluation of IE: the Message Understanding Conferences [1]. There is also a long history of research on applications in medicine. Applications to the medical field focus on two distinct sub-problems: improved access to the medical literature and extraction of information from patient records. Despite these successes in other fields, natural language processing (NLP) techniques were not introduced in biology until the late 1990s. Even today, there are two distinct groups: on the one hand, researchers with a background in computer science, and on the other hand, their colleagues with a background in the life sciences, with only limited interaction between the two groups. To improve this situation, the BioLINK group holds regular open meetings to bring together researchers developing text data mining tools and related language processing methods to manage the information explosion in the biomedical field. They include invited and contributed papers, with a focus on developing shared infrastructure (tools, corpora, ontologies) and challenge evaluations, in the style of the KDD Challenge Cups [2]. This year, the BioLINK SIG meeting focused on resources and tools for text mining, with special emphasis on the evaluation of these tools. Speakers from the following areas were invited:

Data Mining refers to a set of methods applicable to large and complex databases to eliminate the randomness and discover the hidden pattern. Datamining (DM), also known as knowledge discovery from databases (KDD), is the extraction of new knowledge from huge databases. Data mining involves the use of sophisticated data analysis tools to discover previously unknown, valid patterns and relationships in large data sets. Data mining tools can forecast the future trends and activities to support the decision of people. The scope of datamining is associated with Uncovering trends and patterns are a great power for the businesses of all sectors and industries. Modern intrusion detection applications are confronted with a variety of issues. These applications must be reliable, extensible, manageable, and minimal in maintenance costs. Data mining-based intrusion detection systems (IDSs) have shown high accuracy, good generalisation to novel types of intrusion, and stable behaviour in a changing environment in recent years. The number of hidden layers in various neural network topologies is evaluated in order to discover the best neural network. The technique of attempting to discover instances of network attacks by comparing current behaviour to the expected actions of an intruder is known as misuse detection. Artificial neural networks have the ability to detect and classify network activity using data that is limited, incomplete, and nonlinear. The main purpose of this work is to identify privacy and security concerns among cloud computing participants and consumers in a distributed environment. Techniques like Machine Learning, Natural Language Processing (NLP), and Data Mining are combined to automatically identify and uncover patterns from many sorts of materials. Predictive analytics is capable of dealing with both continuous and discontinuous changes. Classification, prediction, and to some extent, affinity analysis constitute the analytical methods employed in predictive analytics. The contemporary study in text or document mining is focusing on syntactic components and the semantic environment. In order to accomplish this, and with the motivation gained from our previous research contributions, we investigated a mining model to classify documents based on the Order of Context, Concept, and Semantic Relations (OCCSR). The use of data mining techniques based on Cloud computing will enable users to retrieve meaningful information from virtually integrated data warehouses, lowering infrastructure and storage costs. Data mining can extract useful and potentially useful information from the cloud. Big Data is typically defined by three characteristics known as the 3Vs (Volume, Velocity and Variety). The surveys approaches, environments, and technologies in key areas for Big Data analytics capabilities and discusses how they aid in the development of analytics solutions for Clouds. The clustering technique belongs to an unsupervised learning and it is used to discover a new set of categories. Grid-based clustering has the shortest processing time, which is typically determined by the size of the grid rather than the data. We compare the performance of three clustering algorithms: hierarchical clustering, density-based clustering, and K Means clustering. The majority of current approaches to detecting misuse involve the use of rule-based expert systems to identify indicators of known attacks. We provide a brief overview of the use of various Artificial Intelligence techniques and their advancements in the design, development, and application of Intrusion Detection Systems (IDS) for protecting computer and communication networks from intruders. The goal of Knowledge Discovery in Data (KDD) is to extract information that is not obvious by using careful and detailed analysis and interpretation. To drive decisions and actions, analytics employs KDD, data mining, text mining, statistical and quantitative analysis, explanatory and predictive models, and advanced and interactive visualisation techniques.

Computational scientists must understand results from experimental, observational and computational simulation generated data to gain insights and perform knowledge discovery. As systems approach the petascale range, problems that were unimaginable a few years ago are within reach. With the increasing volume and complexity of data produced by ultra-scale simulations and high-throughput experiments, understanding the science is largely hampered by the lack of comprehensive I/O, storage, acceleration of data manipulation, analysis, and mining tools. Scientists require techniques, tools and infrastructure to facilitate better understanding of their data, in particular the ability to effectively perform complex data analysis, statistical analysis and knowledge discovery. The goal of this work is to enable more effective analysis of scientific datasets through the integration of enhancements in the I/O stack, from active storage support at the file system layer to MPI-IO and high-level I/O library layers. We propose to provide software components to accelerate data analytics, mining, I/O, and knowledge discovery for large-scale scientific applications, thereby increasing productivity of both scientists and the systems. Our approaches include 1) design the interfaces in high-level I/O libraries, such as parallel netCDF, for applications to activate data mining operations at the lower I/O layers; 2) Enhance MPI-IO runtime systems to incorporate the functionality developed as a part of the runtime system design; 3) Develop parallel data mining programs as part of runtime library for server-side file system in PVFS file system; and 4) Prototype an active storage cluster, which will utilize multicore CPUs, GPUs, and FPGAs to carry out the data mining workload.

Powerful, Flexible Tools for a Data-Driven WorldAs the data deluge continues in todays world, the need to master data mining, predictive analytics, and business analytics has never been greater. These techniques and tools provide unprecedented insights into data, enabling better decision making and forecasting, and ultimately the solution of increasingly complex problems. Learn from the Creators of the RapidMiner Software Written by leaders in the data mining community, including the developers of the RapidMiner software, RapidMiner: Data Mining Use Cases and Business Analytics Applications provides an in-depth introduction to the application of data mining and business analytics techniques and tools in scientific research, medicine, industry, commerce, and diverse other sectors. It presents the most powerful and flexible open source software solutions: RapidMiner and RapidAnalytics. The software and their extensions can be freely downloaded at www.RapidMiner.com. Understand Each Stage of the Data Mining ProcessThe book and software tools cover all relevant steps of the data mining process, from data loading, transformation, integration, aggregation, and visualization to automated feature selection, automated parameter and process optimization, and integration with other tools, such as R packages or your IT infrastructure via web services. The book and software also extensively discuss the analysis of unstructured data, including text and image mining. Easily Implement Analytics Approaches Using RapidMiner and RapidAnalytics Each chapter describes an application, how to approach it with data mining methods, and how to implement it with RapidMiner and RapidAnalytics. These application-oriented chapters give you not only the necessary analytics to solve problems and tasks, but also reproducible, step-by-step descriptions of using RapidMiner and RapidAnalytics. The case studies serve as blueprints for your own data mining applications, enabling you to effectively solve similar problems.

Today’s buildings are adequately informative because of the wide implementation of the building automation system (BAS). In this regard, data mining (DM) tools have an excellent ability to interpret and discover interesting, unknown information from raw data collected from BAS. The present work is a case study, in which DM tools (such as clustering and decision tree) are applied to understand the operational pattern of a hot water storage tank (sanicube) used in a residential building, located in Scotland. The main objective of this study is to explore the correlation among the important features (storage tank temperature, solar collector, the operation of the sanicube heating element, etc.) in the chosen building. Interesting correlations and patterns between the solar collector, storage tank, sanicube-heating element are explored. The clustering results show the existence of different charging patterns of the storage tank and it highly influences the space heating (SH) and domestic hot water (DHW) temperature. The main inference from the result is that the charging of the storage tank is not automatic, and it is influenced by the occupancy behaviour. To maintain the temperature of the storage tank and subsequently to meet the SH and DHW demand, the operation of both solar collector and sanicube heating element (during night time) is recommended rather than operating only the solar collector.

Pardi et al., (2024). pyQCM-BraTaDio: A tool for visualization, data mining, and modelling of Quartz crystal microbalance with dissipation data. Journal of Open Source Software, 9(99), 6831, https://doi.org/10.21105/joss.06831

In many databases, biocuration primarily involves literature curation, which usually involves retrieving relevant articles, extracting information that will translate into annotations and identifying new incoming literature. As the volume of biological literature increases, the use of text mining to assist in biocuration becomes increasingly relevant. A number of groups have developed tools for text mining from a computer science/linguistics perspective, and there are many initiatives to curate some aspect of biology from the literature. Some biocuration efforts already make use of a text mining tool, but there have not been many broad-based systematic efforts to study which aspects of a text mining tool contribute to its usefulness for a curation task. Here, we report on an effort to bring together text mining tool developers and database biocurators to test the utility and usability of tools. Six text mining systems presenting diverse biocuration tasks participated in a formal evaluation, and appropriate biocurators were recruited for testing. The performance results from this evaluation indicate that some of the systems were able to improve efficiency of curation by speeding up the curation task significantly (∼1.7- to 2.5-fold) over manual curation. In addition, some of the systems were able to improve annotation accuracy when compared with the performance on the manually curated set. In terms of inter-annotator agreement, the factors that contributed to significant differences for some of the systems included the expertise of the biocurator on the given curation task, the inherent difficulty of the curation and attention to annotation guidelines. After the task, annotators were asked to complete a survey to help identify strengths and weaknesses of the various systems. The analysis of this survey highlights how important task completion is to the biocurators’ overall experience of a system, regardless of the system’s high score on design, learnability and usability. In addition, strategies to refine the annotation guidelines and systems documentation, to adapt the tools to the needs and query types the end user might have and to evaluate performance in terms of efficiency, user interface, result export and traditional evaluation metrics have been analyzed during this task. This analysis will help to plan for a more intense study in BioCreative IV.

Foundations of SQL Server 2008 R2 Business Intelligence introduces the exciting gamut of business intelligence tools included with SQL Server 2008. Microsoft has designed SQL Server 2008 to be more than just a database. Its a complete business intelligence (BI) platform. The database is at the core, and surrounding the database are tools for data mining, modeling, reporting, analyzing, charting, and integration with other, enterprise-level software packages. SQL Server 2008 puts an incredible amount of BI functionality at your disposal. But how do you take advantage of it? Thats what this book is all about. Authors Guy Fouch and Lynn Langit show how to implement end-to-end BI solutions using SQL Server Analysis Services (SSAS), SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), and other tools in the Microsoft BI toolkit. Youll learn about all-new features such as PowerPivot and Report Builder 3.0. Also provided are clear examples of predictive analysis made possible through powerful data mining features in SQL Server. If youre an analyst or developer working with SQL Server 2008 who is charged with delivering results that drive business success, you cant afford to be without this book; you cant afford to ignore the powerful BI suite that Microsoft has placed at your disposal. Provides the big picture of Microsofts BI tool suite Covers PowerPivot and other game-changing technologies introduced alongside SQL Server 2008 Release 2 Gives a practical analysis of features based on real-world practices What youll learn Apply dimensional- and fact-based modeling to create OLAP schemas Optimize data extraction and transformation with SQL Server Integration Services Develop OLAP and data-mining solutions using SQL Server Analysis Services Create compelling reports using the new Report Builder 3.0 in SQL Server Reporting Services Deliver self-service business-intelligence via Microsoft PowerPivot Manage and maintain control over your BI environment Who this book is for Foundations of SQL Server 2008 R2 Business Intelligence is for business intelligence analysts and developers wanting to employ the full stack of business intelligence tools that are part of SQL Server. The book also appeals to database administrators managing databases in support of business intelligence efforts. Table of Contents What Is Business Intelligence? OLAP Modeling Concepts Introducing OLAP Modeling with SSAS Intermediate OLAP Modeling with SSAS Advanced OLAP Modeling with SSAS Cube Storage and Aggregation Introducing SSIS Intermediate SSIS Advanced SSIS Reporting Tools Data Mining with Excel Introducing PowerPivot Introduction to MDX Introduction to Data Mining

of works held in research and academic libraries and other relevant institutions, for research and private study; * clarify the current EU exception permitting the use of works that were made to be permanently located in the public space (the 'panorama exception'), to take into account new dissemination channels.

There is global attention on new data analytic methods. Data scraping (a typical first step for advanced data analytics), text and data mining (TDM, the extraction of knowledge from data) and machine learning (ML, often also simply referred to as Artificial Intelligence or AI) are seen as critical technologies. The legal issues involved in the regulation of data range from privacy and data protection (such as the GDPR) to proprietary approaches (such as copyright, database rights, or proposed new rights in data themselves). This paper focusses on one specific intervention, the introduction of two exceptions for text and data mining in the Directive on Copyright in the Digital Single Market (CDSM). Art. 3 is a mandatory exception for text and data mining (TDM) for the purposes of scientific research; Art. 4 permits text and data mining by anyone but with rightsholders able to “contract-out” (Art. 4), for example preventing TDM use of publicly available online content by “machine-readable means”. We trace the context of using the lever of copyright law to enable emerging technologies and support innovation. Within the EU copyright intervention, elements that may underpin a transparent legal framework for AI are identified, such as the possibility of retention of (permanent) copies for further verification. On the other hand, we identify several pitfalls, including an excessively broad definition of TDM which makes the entire field of data-driven AI development dependent on an exception. We analyse the implications of limiting the scope of the exceptions to the right of reproduction (which leaves the communication of research results in a grey zone). We also argue that the limitation of Art. 3 to certain beneficiaries remains problematic; and that the requirement of lawful access is difficult to operationalize. In conclusion, we argue that there should be no need for a TDM exception for the act of extracting informational value from protected works. The EU’s CDSM provisions paradoxically may favour the development of biased AI systems due to price and accessibility conditions for accessing training data that offer the wrong incentives. We also identify some old and new areas of the EU acquis which will play a crucial role in the future relationship of EU copyright law with technological innovation.

The term “social construction of knowledge” is introduced and a system is proposed that facilitates this process. Knowledge is based on information that may have to be retrieved from vast, dynamic and unknown information spaces. The goal of knowledge construction may be vague, such that the tasks to achieve it evolve incrementally as information is gathered and insights increase. The collaborative construction of knowledge introduces further challenges and stresses aspects such as validity, trust, relevance, and popularity of knowledge. A combination of groupware, text and data mining tools, software agents and a knowledge building component is proposed to address all these issues involved.

In this paper is presented an experimental study with Data Mining (DM) tools, such as, CMSR Data Miner, KnowledgeSTUDIO and RapidMiner. It is goal to discover at that problems (classification, regression, clustering, association, among others) may be used, including, to verify if can treat Big Data, these tools were applied to three databases: Breast Cancer, Iris and Wine (obtained from the Machine Learning repository from University of California). In the experiments with the CMSR Data Miner tool, the Decision Tree and SOM (Self Organizing Maps) Neural Network algorithms of Data Mining were used. After that, with the KnowledgeSTUDIO tool, Logistic Regression, Deep Learning and K-Means algorithms were used. And finally, with the RapidMiner tool, Decision Tree, Naive Bayes and Deep Learning algorithms of Data Mining were applied. Additionally, it is also with this research aim to decision making by organizational managers, helping them to choose tools from DM more appropriate to their daily work.

Diabetes is one of the major health issues among youngsters nowadays because of poor diet, family history, lifestyle habits. The medical diagnosis is important based on complicated risks that should be performed accurately and effectively. Based on the result and reports further investigation is needed through diagnosis and treatment are given to the patient. Many numbers of the data can be available in the healthcare systems. The availability and necessary medical data can be analyzed for data analysis tools to extract useful information through pattern recognition. Both data mining and knowledge discovery data are infinite applications including business and research scientific fields. Diabetes is one of the major applications nowadays where data mining tools are very useful to easily diagnose and find the solution. This paper, diagnoses diabetes through data mining tools such as SVM, association rule, clustering, and association. So, Data mining helps to predict and diagnose diseases with a low occurrence of risks. In this paper, the main focus is to make a present detailed survey of various data mining techniques and approaches that have been put to use for the prognosis of diabetes.

Information explosion and availability of information in various forms has changed the shape of information centres and nature of information profession and professionals. Information profession and professionals have been impacted by the exponentially increasing volumes of information available — as well as with changing attitudes and behaviour of information seeker toward electronic resources. It is very difficult to find the required pieces of information in the bundled of scattered information. The task becomes more challenging, when we find that over 90% of the information available is in unstructured and semi-structured forms, which is very difficult to search. Here text mining has come as a tool to help Information professionals to find the relevant information and deliver to its users. Text mining is used as a technology for analyzing large volumes of structured/unstructured textual documents. Text mining has very high knowledge and commercial values. The aim of Text mining is generally to strengthen decision making and internal operations processes of any organisation and generation of new domain of knowledge. These technologies help to increase the utilization of Knowledge Management (KM) systems and pro-actively help information professionals to improve their competencies and thus productivity of the organization. This article discusses the basic concept of text mining, its framework and text mining products and tools. It also discusses the benefits and challenges of text mining and examines the role of Information Professionals in Text Mining.