Discovering Unknown Patterns in Free Text

The storage of data is a key issue of information systems, which is an important foundation for data query and data mining. Relational database model has been proven to be a very useful data-storage technique. As information is stored as data in relational databases, the induction of concepts from data is a pivotal topic in the data mining field. Formal Concept Analysis (FCA) turns out to be a perfect instrument for a meaningful and conceptual exploration of the stored data. In FCA, conceptual scaling provides a complete framework for transforming any many-valued context (i.e., relation/table) into a context (called a derived context), in which each many valued attribute is given a scale. The attributes in a scale basically describe meaningful features of the values of the initial attribute. From the logical point of view, complement operation plays a very important role in relational databases and data query systems. In this paper, we provide the connections between the concepts of binary relations and those of complementary binary relations, and propose an approach toward normalizing (complementary) scales, i.e., each (complementary)scale can be represented by a set of statements. One advantage of normalizing scales is to avoid generating huge derived relations, and hence this approach reduces storage cost. By the normalization, the concept lattice of the complement of a derived relation is reduced to a combination of the concept lattice of the derived relation and a set of statements.

of works held in research and academic libraries and other relevant institutions, for research and private study; * clarify the current EU exception permitting the use of works that were made to be permanently located in the public space (the 'panorama exception'), to take into account new dissemination channels.

There is global attention on new data analytic methods. Data scraping (a typical first step for advanced data analytics), text and data mining (TDM, the extraction of knowledge from data) and machine learning (ML, often also simply referred to as Artificial Intelligence or AI) are seen as critical technologies. The legal issues involved in the regulation of data range from privacy and data protection (such as the GDPR) to proprietary approaches (such as copyright, database rights, or proposed new rights in data themselves). This paper focusses on one specific intervention, the introduction of two exceptions for text and data mining in the Directive on Copyright in the Digital Single Market (CDSM). Art. 3 is a mandatory exception for text and data mining (TDM) for the purposes of scientific research; Art. 4 permits text and data mining by anyone but with rightsholders able to “contract-out” (Art. 4), for example preventing TDM use of publicly available online content by “machine-readable means”. We trace the context of using the lever of copyright law to enable emerging technologies and support innovation. Within the EU copyright intervention, elements that may underpin a transparent legal framework for AI are identified, such as the possibility of retention of (permanent) copies for further verification. On the other hand, we identify several pitfalls, including an excessively broad definition of TDM which makes the entire field of data-driven AI development dependent on an exception. We analyse the implications of limiting the scope of the exceptions to the right of reproduction (which leaves the communication of research results in a grey zone). We also argue that the limitation of Art. 3 to certain beneficiaries remains problematic; and that the requirement of lawful access is difficult to operationalize. In conclusion, we argue that there should be no need for a TDM exception for the act of extracting informational value from protected works. The EU’s CDSM provisions paradoxically may favour the development of biased AI systems due to price and accessibility conditions for accessing training data that offer the wrong incentives. We also identify some old and new areas of the EU acquis which will play a crucial role in the future relationship of EU copyright law with technological innovation.

Public science is critical to the economy and to society. However, much of the beneficial impact of scientific research only occurs when scientific knowledge is disseminated broadly and is used by others. This thesis examines the emerging policy, law, and practice of facilitating open access to scientific research data. One particular focus is to examine the open data policies recently introduced by research funders and publishers, and the potential in these for driving the practice of open scientific data into the future. This thesis identifies five major stumbling blocks to sustainable open scientific data. Firstly, the prevailing ‘mindset’ that facilitating open access to data is analogous to facilitating open access to publications and, therefore, research data can easily be shared, with research funders and librarians effectively leading the process. Secondly, the unclear meaning of the term ‘data’, which causes confusion among stakeholders. Thirdly, ‘misunderstood incentives’ for data sharing and the additional inputs required from researchers. Fourthly, ‘data privacy’—an issue that only applies to selected research datasets, and yet appears to dominate the discussion about open research data. Finally, there is ‘copyright law’, which poses challenges at different stages of data release and reuse. In this thesis, I argue that the above problems can be addressed using a staged model for open scientific data. I draw specifically on the practice with open scientific data at CERN (the European Organization for Nuclear Research) and the practice of sharing clinical trial data to argue that open data can be shared at various stages of processing and diversification. This model is supplemented by recommendations proposing changes to existing open data mandates and the introduction of a text and data mining exemption into Australian copyright law.

BackgroundVast numbers of domestic violence (DV) incidents are attended by the New South Wales Police Force each year in New South Wales and recorded as both structured quantitative data and unstructured free text in the WebCOPS (Web-based interface for the Computerised Operational Policing System) database regarding the details of the incident, the victim, and person of interest (POI). Although the structured data are used for reporting purposes, the free text remains untapped for DV reporting and surveillance purposes.ObjectiveIn this paper, we explore whether text mining can automatically identify mental health disorders from this unstructured text.MethodsWe used a training set of 200 DV recorded events to design a knowledge-driven approach based on lexical patterns in text suggesting mental health disorders for POIs and victims.ResultsThe precision returned from an evaluation set of 100 DV events was 97.5% and 87.1% for mental health disorders related to POIs and victims, respectively. After applying our approach to a large-scale corpus of almost a half million DV events, we identified 77,995 events (15.83%) that mentioned mental health disorders, with 76.96% (60,032/77,995) of those linked to POIs versus 16.47% (12,852/77,995) for the victims and 6.55% (5111/77,995) for both. Depression was the most common mental health disorder mentioned in both victims (22.25%, 3269) and POIs (18.70%, 8944), followed by alcohol abuse for POIs (12.19%, 5829) and various anxiety disorders (eg, panic disorder, generalized anxiety disorder) for victims (11.66%, 1714).ConclusionsThe results suggest that text mining can automatically extract targeted information from police-recorded DV events to support further public health research into the nexus between mental health disorders and DV.

News is new information or information about something that is currently happening, presented in the form of print media, broadcasts, social media or word of mouth to third parties or multiple people. The spread of this news is now very much happening both from social media or messages. However, the problem is that no one can guarantee the truth of the news received. Text mining is the application of data mining concepts and techniques to look for patterns in text, the process of analyzing text to find useful information for a particular purpose. Another definition related to text mining is that text mining is data mining in the form of text where the data source is usually obtained from documents and the goal is to find words that can represent the contents of the document so that an analysis of the connectivity between documents can be carried out. To make it easier to identify news hoax requires a text mining algorithm, using a text mining algorithm can be useful to get real news. Based on the calculations of the TF-IDF algorithm, it can be concluded that the similarity of the meaning of the news whose truth is not yet known is closest to the news ofmerahputih.com with a confidence level of 23.11%.

DEFINING TEXT MINING Text mining can be broadly defined as a knowledge-intensive process in which a user interacts with a document collection over time by using a suite of analysis tools. In a manner analogous to data mining, text mining seeks to extract useful information from data sources through the identification and exploration of interesting patterns. In the case of text mining, however, the data sources are document collections, and interesting patterns are found not among formalized database records but in the unstructured textual data in the documents in these collections. Certainly, text mining derives much of its inspiration and direction from seminal research on data mining. Therefore, it is not surprising to find that text mining and data mining systems evince many high-level architectural similarities. For instance, both types of systems rely on preprocessing routines, pattern-discovery algorithms, and presentation-layer elements such as visualization tools to enhance the browsing of answer sets. Further, text mining adopts many of the specific types of patterns in its core knowledge discovery operations that were first introduced and vetted in data mining research. Because data mining assumes that data have already been stored in a structured format, much of its preprocessing focus falls on two critical tasks: Scrubbing and normalizing data and creating extensive numbers of table joins. In contrast, for text mining systems, preprocessing operations center on the identification and extraction of representative features for natural language documents.

Structured Query Language (SQL) is commonly used in Relational Database Management Systems (RDBMS) and is currently one of the most popular data definition and manipulation languages. Its core functionality is implemented, with only some minor variations, throughout all RDBMS products. It is an effective tool in the process of managing and querying data in relational databases. This paper describes a method to effectively automate the conversion of a data query from a Natural Language Query (NLQ) to Structured Query Language (SQL) with Online Analytical Processing (OLAP) cube data warehouse objects. To obtain or manipulate the data from relational databases, the user must be familiar with SQL and must also write an appropriate and valid SQL statement. However, users who are not familiar with SQL are unable to obtain relevant data through relational databases. To address this, we propose a Natural Language Processing (NLP) model to convert an NLQ into an SQL query. This allows novice users to obtain the required data without having to know any complicated SQL details. The model is also capable of handling complex queries using the OLAP cube technique, which allows data to be pre-calculated and stored in a multi-dimensional and ready-to-use format. A multi-dimensional cube (hypercube) is used to connect with the NLP interface, thereby eliminating long-running data queries and enabling self-service business intelligence. The study demonstrated how the use of hypercube technology helps to increase the system response speed and the ability to process very complex query sentences. The system achieved impressive performance in terms of NLP and the accuracy of generating different query sentences. Using OLAP hypercube technology, the study achieved distinguished results compared to previous studies in terms of the speed of the response of the model to NLQ analysis, the generation of complex SQL statements, and the dynamic display of the results. As a plan for future work, it is recommended to use infinite-dimension (n-D) cubes instead of 4-D cubes to enable ingesting as much data as possible in a single object and to facilitate the execution of query statements that may be too complex in query interfaces running in a data warehouse. The study demonstrated how the use of hypercube technology helps to increase system response speed and process very complex query sentences.

Ontology provides a shared and reusable piece of knowledge about a specific domain, and has been applied in many fields, such as semantic Web, e-commerce and information retrieval, etc. However, building ontology by hand is a very hard and error-prone task. Learning ontology from existing resources is a good solution. Because relational database is widely used for storing data and OWL is the latest standard recommended by W3C, this paper proposes an approach of learning OWL ontology from data in relational database. Compared with existing methods, the approach can acquire ontology from relational database automatically by using a group of learning rules instead of using a middle model. In addition, it can obtain OWL ontology, including the classes, properties, properties characteristics, cardinality and instances, while none of existing methods can acquire all of them. The proposed learning rules have been proven to be correct by practice.

In Article 3 of the “Proposal for a Directive on copyright in the Digital Single Market COM(2016) 593 final” the European Commission suggests an exception for text and data mining (TDM). While, in principle, a clear legal framework for TDM is to be welcomed, the proposed provisions are to be criticized regarding their scope and the applied regulatory method. This Position Statement develops an alternative proposal: Since TDM is to be seen as a normal use of works and other protected subject-matter, a field exemption is suggested allowing everyone to carry out TDM related to lawfully accessible works or other subject-matter. This includes the permission to extract contents of databases and to make reproductions for the sole purpose of TDM. Moreover, research organizations also need to carry out TDM regarding content to which they do not have lawful access. The proposal includes a specific provision obliging rightholders who market works or other subject-matter primarily for research purposes to provide datasets suitable for TDM only, for which they may request a reasonable payment.

Text mining applies the same analytical functions of data mining to the domain of textual information, relying on sophisticated text analysis techniques that distill information from free-text documents. IBM’s Intelligent Miner for Text provides the necessary tools to unlock the business information that is “trapped” in email, insurance claims, news feeds, or other document repositories. It has been successfully applied in analyzing patent portfolios, customer complaint letters, and even competitors’ Web pages. After defining our notion of “text mining”, we focus on the differences between text and data mining and describe in some more detail the unique technologies that are key to successful text mining.

In the database field, the requirement of high level facilities for retrieval/update operations is now increasing rapidly. Our approach from the relational database point of view for contribution to this problem is to provide 1) efficient processing of relational retrieval/ update operations, and 2) a high level user interface. In order to achieve this goal, a new concept concerning "abstracted characteristics" is presented. Abstracted characteristics are defined to be characteristics abstracted from sets of tuples in relations stored in the database. A classification of abstracted characteristics is presented. Functional dependencies, which play an important role in relational database design, and time-dependent functional dependencies are pointed out to be useful in processing retrieval/update operations. Some important applications of abstracted characteristics are discussed. Among them: 1) efficient processing of retrieval/update operations, 2) powerful view update checking facilities, 3) providing some rough meanings of null responses and 4) a high level user interface.

This article focuses on testing a path-oriented querying approach to hierarchical data in relational databases. The authors execute a user study to compare the path-oriented approach and traditiona...

Hierarchical part-of relationships/aggregation structures and related queries are essential parts of information systems. However, relational database query languages do not explicitly support hierarchical relationships and queries. A hierarchical query may require a great number of join operations, which increases the effort in query formulation. Therefore, we propose path expressions in formulating hierarchical views over relational data because path expressions are a conventional and compact way to represent hierarchical relationships. We embed path expressions within SQL queries and compile them to standard SQL. This ensures that the path expressions can straightforwardly be implemented on the top of standard relational database systems. The compilation of a path expression is given by an attribute grammar, a conventional formalism to define the semantics of a language.

Data is becoming one of the world’s most valuable resources and it is suggested that those who own the data will own the future. However, despite data being an important asset, data owners struggle to assess its value. Some recent pioneer works have led to an increased awareness of the necessity for measuring data value. They have also put forward some simple but engaging survey-based methods to help with the first-level data assessment in an organisation. However, these methods are manual and they depend on the costly input of domain experts. In this paper, we propose to extend the manual survey-based approaches with additional metrics and dimensions derived from the evolving literature on data value dimensions and tailored specifically for our use case study. We also developed an automatic, metric-based data value assessment approach that (i) automatically quantifies the business value of data in Relational Databases (RDB), and (ii) provides a scoring method that facilitates the ranking and extraction of the most valuable RDB tables. We evaluate our proposed approach on a real-world RDB database from a small online retailer (MyVolts) and show in our experimental study that the data value assessments made by our automated system match those expressed by the domain expert approach.