The problem of noise in classification: Past, current and future work

Research in machine learning, data mining and related areas has produced a wide variety of algorithms for cost-sensitive (CS) classification, where instead of maximizing the classification accuracy, minimizing the misclassification cost becomes the objective. However, these methods assume that training sets do not contain significant noise, which is rarely the case in real-world environments. In this paper, we systematically study the impacts of class noise on CS learning, and propose a cost-guided class noise handling algorithm to identify noise for effective CS learning. We call it cost-guided iterative classification filter (CICF), because it seamlessly integrates costs and an existing classification filter (C. Brodley and M. Friedl, 1999) for noise identification. Instead of putting equal weights to handle noise in all classes in existing efforts, CICF puts more emphasis on expensive classes, which makes it especially successful in dealing with datasets with a large cost-ratio. Experimental results and comparative studies from real-world datasets indicate that the existence of noise may seriously corrupt the performance of CS classifiers, and by adopting the proposed CICF algorithm, we can significantly reduce the misclassification cost of a CS classifier in noisy environments.

At present, prime numbers are an essential part of online security for secure key generation and digital signatures due to their unpredictabil-‎ity and resistance to factorization attacks, which has led to considerable interest in research on distributions of primes, especially prime gaps ‎and hidden patterns. Traditional prime analytical methods lack structural insight at the scale of cryptographic systems. This has led to greater ‎interest in applying machine learning (ML) techniques to prime analysis. However, due to a lack of publicly available high-quality datasets, ‎the evaluation of prime behavior using ML is often impeded. Furthermore, current prime generation techniques tend to become computa‎tionally inefficient when scaled to produce large quantities of high-bit-length prime numbers. In response, this paper proposes a novel generative pipeline, GANLPN, that leverages Generative Adversarial Networks (GANs) combined with the Learning Parity with Noise (LPN) ‎problem to generate bulk primes that are cryptographically secure. In this way, a publicly accessible dataset of 1,115,000 1024-bit primes ‎has been generated. The generated sequences passed all NIST SP800-22 statistical randomness tests with better inference time and throughput. The present work outlines a path for future ML-assisted studies of prime patterns, gaps, distributions, and the presence of weak keys in ‎cryptography.

The escalating threat of fraud in financial institutions is a global issue, with the Malaysian sector being no exception. This study focuses on the implementation and efficacy of Data Mining and Machine Learning methodologies in identifying and mitigating fraudulent activities within these institutions. The paper critically reviews existing literature, bridging the gap between advanced technology application and fraud management. Fraudulent transactions in the financial sector are dynamic and sophisticated, requiring advanced detection techniques. Traditional approaches often struggle to manage this complexity effectively, demonstrating a need for more advanced and adaptive strategies. This is where Data Mining and Machine Learning techniques, renowned for their predictive and analytical prowess, can significantly contribute. Data Mining, the process of uncovering patterns and correlations within large datasets, is a useful tool for detecting anomalies that may suggest fraud. The study assesses various data mining techniques, such as clustering, classification, and association, and explores their application in detecting fraudulent transactions. Findings indicate that these techniques can substantially enhance fraud detection rates while minimizing false positives. Furthermore, Machine Learning, an artificial intelligence subset, has shown immense potential in fraud detection. Its ability to learn from and make decisions based on data makes it a viable solution for fraud detection. This paper explores both supervised and unsupervised learning algorithms and their efficacy in identifying fraud in the Malaysian financial sector. Results suggest that machine learning models, when correctly implemented, can significantly improve the accuracy of fraud detection. The review underscores the importance of employing advanced technologies like Data Mining and Machine Learning to combat financial fraud effectively. It also suggests future research directions, emphasizing the need for context-specific, localized models considering Malaysia's unique socio-economic environment. Moreover, the development of hybrid models, integrating both data mining and machine learning, could offer improved results. In conclusion, this study sets a precedent for further exploration into the application of advanced analytical tools in fraud detection in the Malaysian financial sector. The potential these technologies offer for improving accuracy and adaptability in fraud detection systems is substantial and warrants thorough investigation. Keywords: Fraud Detection, Malaysian Financial Institutions, Data Mining, Machine Learning, Financial Fraud Management

Separating models of learning with faulty teachers

Modern information technology makes it possible today to collect, store, transfer, and combine huge amounts of data at very low costs. Thus more and more companies, and scientific and governmental institutions build up large archives of all kinds of data like numbers, tables, documents, images, sounds, etc. Although users often have a vague understanding of their data and can usually formulate hypotheses and guess dependencies, turning these – often abundantly available – data into useful information turns out to be rather difficult. In response to these challenges a new area of research has emerged, called ‘‘knowledge discovery in databases’’ or ‘‘data mining’’, that tries to provide tools to extract valid, useful, understandable, unknown, and unexpected relationships from large databases [1]. This current form of data analysis is an interdisciplinary field, and employs methods from statistics, soft computing, artificial intelligence and machine learning. The stress lies on the development of techniques that produce human-understandable results, and are suited for large, real world datasets. The data in real world applications has in most cases characteristics that challenge classical analyzing approaches. Besides being heterogeneous – which in its simplest form can mean that we have to deal with numerical and symbolic features – , the data is often of low quality. Algorithms must thus be able to deal with uncertainty and imprecision. The characteristics of the data sources – their quantity, complexity, dimensionality and imperfection – , the essential of extracting understandable patterns from these, and the need to incorporate available background knowledge in that process, makes us assume that (neuro-) fuzzy techniques will play a considerable role in the future of data mining [3]. It is the ability of fuzzy sets to transform between computer representations and (naturally linguistic) human concepts that makes them so valuable to meet the advanced data mining demands, and that Zadeh meant, when he promoted his idea of computing with words [4]. The inherent imprecision of words is not necessarily a weakness, but, on the contrary, can be crucial to model complex systems. From our own experience we observed that many practical applications have this certain robustness where full precision is not necessary. In such cases, exaggerated precision can be a waste of resources, and solutions obtained using fuzzy approaches might be easier to understand and to apply, and gain their strengths by explicitly taking into account vagueness, imprecision or uncertainty. One prominent way to use fuzzy systems in data analysis, is to induce fuzzy if-then rules from data by neurofuzzy techniques. The use of linguistic variables eases the readability and interpretability of the rule base. If we apply such techniques, we must be aware of the trade-off between precision and interpretability. However, the results in data mining are not only judged for their accuracy, but also for their interpretability, as the ultimate goal is to extract human understandable patterns [2]. The papers in this special issue of the Soft Computing Journal deal with quite different aspects of neuro-fuzzy techniques, with a stress on data analysis and rule based systems. Although the intention of this issue cannot be to give a survey of this wide area, it shall give an insight in current trends and research topics in this field. The papers can roughly be divided into three groups:

Data mining and knowledge discovery aim at finding useful information from typically massive collections of data, and then extracting useful knowledge from the information. To date a large number of approaches have been proposed to find useful information and discover useful knowledge; for example, decision trees, Bayesian belief networks, evidence theory, rough set theory, fuzzy set theory, kNN (k-nearest-neighborhood) classifier, neural networks, and support vector machines. However, these approaches are based on a specific data type. In the real world, an intelligent system often encounters mixed data types, incomplete information (missing values), and imprecise information (fuzzy conditions). In the UCI (University of California – Irvine) Machine Learning Repository, it can be seen that there are many real world data sets with missing values and mixed data types. It is a challenge to enable machine learning or data mining approaches to deal with mixed data types (Ching, 1995; Coppock, 2003) because there are difficulties in finding a measure of similarity between objects with mixed data type attributes. The problem with mixed data types is a long-standing issue faced in data mining. The emerging techniques targeted at this issue can be classified into three classes as follows: (1) Symbolic data mining approaches plus different discretizers (e.g., Dougherty et al., 1995; Wu, 1996; Kurgan et al., 2004; Diday, 2004; Darmont et al., 2006; Wu et al., 2007) for transformation from continuous data to symbolic data; (2) Numerical data mining approaches plus transformation from symbolic data to numerical data (e.g.,, Kasabov, 2003; Darmont et al., 2006; Hadzic et al., 2007); (3) Hybrid of symbolic data mining approaches and numerical data mining approaches (e.g.,, Tung, 2002; Kasabov, 2003; Leng et al., 2005; Wu et al., 2006). Since hybrid approaches have the potential to exploit the advantages from both symbolic data mining and numerical data mining approaches, this chapter, after discassing the merits and shortcomings of current approaches, focuses on applying Self-Organizing Computing Network Model to construct a hybrid system to solve the problems of knowledge discovery from databases with a diversity of data types. Future trends for data mining on mixed type data are then discussed. Finally a conclusion is presented.

Data mining and machine learning (ML) methods are used more than ever in cyber security. The use of machine learning (ML) is one of the potential solutions that may be successful against zero-day attacks, starting with categorising IP traffic and filtering harmful traffic for intrusion detection. In this field, certain published systematic reviews were taken into consideration. Recent systematic reviews may incorporate older and more recent works in the topic of investigation.. Both security professionals and hackers use data mining capabilities. Applications for data mining may be used to analyze programme activity, surfing patterns, and other factors to identify potential cyber-attacks in the future. The new study uses statistical traffic features, ML, and data mining approaches. This research performs a concentrated literature review on machine learning and its usage in cyber analytics for email filtering, traffic categorization, and intrusion detection. Each approach was identified, and a summary was provided based on the relevancy and quantity of citations. Some well-known datasets are also discussed since they are a crucial component of ML techniques. On when to utilize a certain algorithm is also offered some advice. Four ML algorithms have been evaluated on MODBUS data gathered from a gas pipeline. Using ML algorithms, other assaults have been categorized, and then the effectiveness of each approach has been evaluated. This study demonstrates the use of ML and data mining for threat research and detection, focusing on malware detection with high accuracy and short detection times.

This focus issue of Physiological Measurement follows the 38th Annual International Computing in Cardiology (CinC) Conference, hosted in Hangzhou, China in September 2011 by Zhejiang University. Each year, the NIH-sponsored PhysioNet resource (http://physionet.org/) runs an open competition lasting several months, aimed at encouraging the development of solutions to an unsolved or poorly solved problem in biomedicine, in most cases making use of relevant clinical and experimental data provided freely by PhysioNet. Participants in these annual challenges discuss their diverse approaches to the Challenge problems during dedicated scientific sessions at CinC. The topics of these PhysioNet/CinC Challenges range from physiologic signal processing and analysis to forecasting and modelling clinically important events and processes.

Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature‐based classifiers and nonlinear predictive models. Training neural networks involves the optimization of nonconvex objective functions, and usually, the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counterintuitive, alternative is to randomly assign a subset of the networks’ weights so that the resulting optimization task can be formulated as a linear least‐squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favorable benefits, including (1) simplicity of implementation, (2) faster learning with less intervention from human beings, and (3) possibility of leveraging overall linear regression and classification algorithms (e.g.,ℓ1norm minimization for obtaining sparse formulations). This class of neural networks attractive and valuable to the data mining community, particularly for handling large scale data mining in real‐time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims to provide a self‐contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems, and most importantly, foster the exchanges of well‐known results throughout different communities.WIREs Data Mining Knowl Discov2017, 7:e1200. doi: 10.1002/widm.1200This article is categorized under:Technologies > Machine Learning

Medical data includes clinical trials and clinical data such as patient-generated health data, laboratory results, medical imaging, and different signals coming from continuous health monitoring. Some commonly used data analysis techniques are text mining, big data analytics, and data mining. These techniques can be used for classification, clustering, and machine learning tasks. Machine learning could be described as an automatic learning process derived from concepts and knowledge without deliberate system coding. However, finding a suitable machine learning architecture for a specific task is still an open problem. In this work, we propose a machine learning model for the multi-class classification of medical data. This model is comprised of two components—a restricted Boltzmann machine and a classifier system. It uses a discriminant pruning method to select the most salient neurons in the hidden layer of the neural network, which implicitly leads to a selection of features for the input patterns that feed the classifier system. This study aims to investigate whether information-entropy measures may provide evidence for guiding discriminative pruning in a neural network for medical data processing, particularly cancer research, by using three cancer databases: Breast Cancer, Cervical Cancer, and Primary Tumour. Our proposal aimed to investigate the post-training neuronal pruning methodology using dissimilarity measures inspired by the information-entropy theory; the results obtained after pruning the neural network were favourable. Specifically, for the Breast Cancer dataset, the reported results indicate a 10.68% error rate, while our error rates range from 10% to 15%; for the Cervical Cancer dataset, the reported best error rate is 31%, while our proposal error rates are in the range of 4% to 6%; lastly, for the Primary Tumour dataset, the reported error rate is 20.35%, and our best error rate is 31%.

The field of Smart Healthcare has undergone a revolution due to the development of machine learning technologies. Artificial intelligence (AI) has a subfield called machine learning that allows computers to learn from data and develop over time without explicit programming. Healthcare workers can now discover trends, diagnose diseases, and make decisions based on more precise and timely data thanks to machine learning's capacity to comprehend and analyze enormous amounts of data. Finding patterns and links in massive datasets is known as data mining. Data mining is used in Smart Healthcare to locate patterns in patient data that can be utilized to comprehend a patient's health status better, diagnose specific disorders, and forecast the most effective course of therapy. Data mining can create novel cures and treatments customized to a person's genetic profile. Data mining in Smart Healthcare is now considerably more effective and efficient, thanks to machine learning. Using complex algorithms, machine learning models can analyze vast amounts of data markedly faster and more correctly than conventional methods. Healthcare experts may not immediately notice hidden patterns and connections in data that machine learning algorithms can find. As a result, machine learning can assist healthcare practitioners in making decisions that are better informed, improving patient outcomes.

Addressing label noise in leukemia image classification using small loss approach and pLOF with weighted-average ensemble

The field of sentiment mining and text summarization has evoked the interest of many scientists and researchers over the last few years, as the textual data has become useful for many real-world applications and challenges. Sentiment Analysis and Opinion Mining is the most popular field for analyzing and discovering insights from text data from various sources, such as Facebook, Twitter and Amazon, Zomato, etc. It involves a computational study of an individual's behavior in terms of buying interest and then extracting his opinions on the business entity of the company. This entity can be viewed as an event, individual, blog post or product experience. Scholars in the fields of natural language processing, data mining, machine learning and others have tested a variety of methods for automating sentiment analysis. These reviews are increasing on a daily basis, as a result of which the summarization of the reviews plays a role where the text is summarized as needed, which provides useful information from a large number of reviews. It is very difficult for a human being to extract and interpret useful data from a very large file. In the text analysis, the value of sentences is decided on the basis of the linguistic characteristics of sentences. This paper provides a comprehensive review of current and past work on sentiment analysis and text description. In this research work, a new hybrid classification system is proposed based on coupling classification methods using arcing classifiers and their quality is evaluated within terms of accuracy. The Classifier Collection was constructed using Naïve Bayes (NB), Support Vector Machine (SVM) and Genetic Algorithm (GA). The proposed work consists of a comparative study of the efficacy of the ensemble technique for sentiment classification. The feasibility and benefits of the proposed approaches are demonstrated by a restaurant review that is widely used in the field of sentiment classification. A wide range of comparative studies is performed and, ultimately, some in-depth analysis is addressed and conclusions are drawn on the efficacy of the ensemble technique for sentiment classification.

Use of machine learning (ML) and data mining (DM) algorithms has surfaced more often in the recent years for disease risk prediction problems in the healthcare communities. Several traditional feature selection models are combined with the DM and ML algorithms to improve accuracy of the disease risk prediction. In this study, a new Bio-inspired Ensemble Feature Selection (BEFS) model is introduced which is applied with the DM and ML algorithms. In the BEFS model, the most relevant and highly contributing features in the prediction are determined with a bio-inspired algorithm i.e., genetic algorithm, and an ensemble algorithm i.e., random forest algorithm. These important features obtained from the proposed model are then combined in various combinations and applied with the DM and ML algorithms, here logistic regression (LR) and random forest (RF), and the results obtained are promising. The experiment is executed using the famous ML language R. To accomplish this objective, the Breast Cancer Wisconsin (Diagnostic) dataset of UCI (University of California, Irvine) ML repository is utilized. In the experimental outcomes, the highest accuracy attained with the BEFS model is 96.49%, the AUC (Area Under Curve) achieved is 96%, and the sensitivity is 98.11%. These results, which greatly improve the disease risk prediction, are higher than several other existing works, while utilizing only six most relevant features out of the thirty two features of the dataset.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.