Outlier Detection Based on Rough Membership Function

<abstract><p>There is a close analogy and similarity between topology and rough set theory. As, the leading idea of this theory is depended on two approximations, namely lower and upper approximations, which correspond to the interior and closure operators in topology, respectively. So, the joined study of this theory and topology becomes fundamental. This theory mainly propose to enlarge the lower approximations by adding new elements to it, which is an equivalent goal for canceling elements from the upper approximations. For this intention, one of the primary motivation of this paper is the desire of improving the accuracy measure and reducing the boundary region. This aim can be achieved easily by utilizing ideal in the construction of the approximations as it plays an important role in removing the vagueness of concept. The emergence of ideal in this theory leads to increase the lower approximations and decrease the upper approximations. Consequently, it minimizes the boundary and makes the accuracy higher than the previous. Therefore, this work expresses the set of approximations by using new topological notions relies on ideals namely $ \mathcal{I} $-$ {\delta_{\beta}}_{J} $-open sets and $ \mathcal{I} $-$ {\bigwedge_{\beta}}_{J} $-sets. Moreover, these notions are also utilized to extend the definitions of the rough membership relations and functions. The essential properties of the suggested approximations, relations and functions are studied. Comparisons between the current and previous studies are presented and turned out to be more precise and general. The brilliant idea of these results is increased in importance by applying it in the chemical field as it is shown in the end of this paper. Additionally, a practical example induced from an information system is introduced to elucidate that the current rough membership functions is better than the former ones in the other studies.</p></abstract>

In many Knowledge Discovery applications, finding outliers is more interesting than finding inliers in a dataset.The perception of outliers is rare cases in dataset in which is being described as abnormal data in the information table.Outliers detections are applied in many important applications like fraud detection systems to uncover the suspicious objects which may have important knowledge hidden in the system.A new outlier detection technique based on Rough Sets Theory (RST) is hereby proposed.RSetOF is a new measure for the outlier factor based on RST.By employing this factor, a new formulation for detecting outlier is established.The outlyingness of outliers objects in a dataset using this measurement is identified.To detect outliers, two measurements which are the top n ratio and the coverage ratio are presented.Finding top n outliers from all objects allow searching of outliers from top ranked records based on the least outlier factor value.The capability in detecting outliers at top n number of outliers will indicate how fast the detection is.The efficiency of this technique by obtaining the coverage ratio value is then tested.The maximum percentage of coverage obtained shows the maximum number of outliers detected belonging to rare cases.A comparison is hence carried out to examine the performance of the RSetAlg with a selective outlier detection method, the Frequent Pattern method referred to as FindFPOF.Ten benchmark datasets for assessing the outlier detection technique are used for this purpose.The experimental result shows that the proposed technique is competitive and proven to be better in speed of detection than the other technique.The fast and efficient detection of outliers has proven its potential as a new outliers detection technique based on RST.

Outlier detection shows its increasingly high practical value in many application areas such as intrusion detection, fraud detection, discovery of criminal activities in electronic commerce and so on. Many techniques have been developed for outlier detection, including distribution-based outlier detection algorithm, depth-based outlier detection algorithm, distance-based outlier detection algorithm, density-based outlier detection algorithm and clustering-based outlier detection. Spectral clustering receives much attention as a competitive clustering algorithms emerging in recent years. However, it is not very well scalable to modern large datasets. To partially circumvent this drawback, in this paper, we propose a new outlier detection method inspired by spectral clustering. Our algorithm combines the concept of kNN and spectral clustering techniques to obtain the abnormal data as outliers by using the information of eigenvalues and eigenvectors statistically in the feature space. We compare the performance of our methods with distance-based outlier detection methods and density-based outlier detection methods. Experimental results show the effectiveness of our algorithm for identifying outliers.

Soft rough fuzzy sets and soft fuzzy rough sets

Rough set theory for the interval-valued fuzzy information systems

Theories of fuzzy sets and rough sets are powerful mathematical tools for modelling various types of uncertainty. Dubois and Prade investigated the problem of combining fuzzy sets with rough sets. Soft set theory was proposed by Molodtsov as a general framework for reasoning about vague concepts. The present paper is devoted to a possible fusion of these distinct but closely related soft computing approaches. Based on a Pawlak approximation space, the approximation of a soft set is proposed to obtain a hybrid model called rough soft sets. Alternatively, a soft set instead of an equivalence relation can be used to granulate the universe. This leads to a deviation of Pawlak approximation space called a soft approximation space, in which soft rough approximations and soft rough sets can be introduced accordingly. Furthermore, we also consider approximation of a fuzzy set in a soft approximation space, and initiate a concept called soft---rough fuzzy sets, which extends Dubois and Prade's rough fuzzy sets. Further research will be needed to establish whether the notions put forth in this paper may lead to a fruitful theory.

To decrease the influence of outlier on the measurement of tooth profiles, this paper proposes a method of outlier detection and correction based on the grey system theory. After studying the characteristics of outliers from the deviations of tooth profiles, this paper proposes a preprocessing method for the modeling data which include abnormal value, and establishes an outlier detection and correction model for the deviations of tooth profiles. Simulation results show that the precision of ONDGM(1,1)(one order and one variable non-homogenous discrete grey model whose outlier is processed by the preprocessing method proposed in this paper) is higher than that of NDGM(1,1)(one order and one variable non-homogenous discrete grey model), and the ONDGM(1,1) is more suitable than the NDGM(1,1) for dealing with the outliers from the deviations of tooth profiles. The experiment results show that the outlier detection and correction model detects and corrects the outliers from the deviations of tooth profiles, and the correction value of the outlier is basically in accordance with the actual deviation. Therefore, the method of outlier detection and correction decreases the influence of outlier and improves the precision in the measurement of tooth profiles.

The goal of outlier detection is to pinpoint data points that exhibit notable deviations from the rest of the observed values. It has found successful application in numerous fields, including process inspection, anti-terrorist operations, and public security. However, the majority of existing outlier algorithms rely on methods involving filling or deleting missing data, with few directly addressing incomplete data. This article studies outlier detection for incomplete real-valued data based on class-consistent technology, rough set theory, and granular computing. First, a tolerance relation founded on class-consistent technology is presented to illustrate the similarity among information values within an incomplete real-valued information system (IRVIS). Then, the tolerance classes are established based on the tolerance relation and utilized for computing approximate accuracy and other metrics. Next, an outlier factor is defined, considering both the degree of outlierness and the weight function assigned to each object within an IRVIS, elucidating the uncertainty and degree of outlier. Finally, an outlier detection algorithm (ODIRG) for an IRVIS based on class-consistent technology, rough set theory, and granular computing is devised. Numerical experiments on seven UCI datasets are undertaken to evaluate the stability of the ODIRG algorithm. The proposed method is demonstrated to exhibit strong effectiveness and adaptability for categorical data when compared with five other algorithms. It is notable that for comprehensive comparison, precision, recall, F F 1-measure, and receiver operating characteristic curve are employed to delineate the benefits of the proposed approach.

A close similarity and analogy between rough set theory and topology is attributed to the corresponding behavior of lower and upper rough approximations with interior and closure topological operators, respectively. This relation motivates joint studies between topology and this theory. We endeavor by rough set theory to enlarge the knowledge we obtain from the information systems, for this reason, we apply the abstract concept of ideal structures to build new generalized approximation spaces with less vagueness. In the present work, we employ a novel type of nearly open sets in topology so-called ``${\mathcal{L}}$-${\theta\beta}_{\lambda}$-open" with an ideal structure to introduce novel approximation spaces satisfying the desired properties concerning shrinking the boundary region of uncertainty and expanding the domain of confirmed information. We set up the fundamentals of the proposed rough paradigms and demonstrate their superiority over the preceding paradigms induced by some nearly open sets. Two algorithms are furnished to illustrate the way of specifying the family of ${\mathcal{L}}$-${\theta\beta}_{\lambda}$-open sets and exploring whether a subset is ${\mathcal{L}}$-${\theta\beta}_{\lambda}$-definable or ${\mathcal{L}}$-${\theta\beta}_{\lambda}$-rough. Then, we put forward the concepts of rough membership relations and functions and uncover their core characterizations. Finally, we examine the proposed models to model a real situation in the Chemistry field and clarify how our models improve the outcomes of generalized approximation spaces over the previous models.

The outlier detection technique is widely used in the data analysis for the clustering of data. Many techniques have been applied in the outlier detection to increase the efficiency of the data analysis. The Local Projection based Outlier Detection (LPOD) method effectively identifies neighbouring values of data, but this has the drawback of random selection of the cluster centre that affects the overall clustering performance of the system. In this study, the Adaptive Clustering by Fast Search and Find of Density Peak (ACFSFDP) is proposed to select the clustering centre and density peak. This ACFSFDP method is implemented with the min-max algorithm to find the number of categories that measured the local density and distance information. The density and distance are used to select the cluster centre, but density is not calculated on the existing distance based clustering techniques. The ACFSFDP method calculates cluster centre based on the density and distance during the clustering process, whereas the existing techniques randomly select the data centre. The results indicated that the ACFSFDP method is provided effective outlier detection compared with existing Clustering by Fast Search and Find of Density Peak (CFSFDP) methods. The ACFSFDP is tested on two datasets Pen-digits and waveform datasets. The experiment results proved that Area Under Curve (AUC) of the ACFSFDP is 99.08% on the Pen-Digit dataset, while the existing distance classifier method k-Nearest Neighbour has achieved 68.7% of AUC.

Theories of fuzzy sets and rough sets are powerful mathematical tools for modelling various types of uncertainty. Molodtsov (Comput Math Appl 37:19–31, 1999 [6]) initiated a novel concept called soft sets, a new mathematical tool for dealing with uncertainties. It has been found that fuzzy sets, rough sets, and soft sets are closely related concepts (Aktas and Cagman in Inf Sci 1(77):2726–2735, 2007 [1]). Research works on soft sets are very active and progressing rapidly in these years. In 2001, Maji et al. (J Fuzzy Math 9(3):589–602, 2001 [5]) proposed the idea of intuitionistic fuzzy soft set theory and established some results on them. Based on an equivalence relation on the universe of discourse, Dubois and Prade (Int J Gen Syst 17:191–209, 1990 [3]) introduced the lower and upper approximation of fuzzy sets in a Pawlak approximation space and obtained a new notion called rough fuzzy sets. Feng et al. (Soft Compt 14:899–911, 2009 [4]) introduced lower and upper soft rough approximation of fuzzy sets in a soft approximation space and obtained a new hybrid model called soft rough fuzzy sets which is the extension of Dubois and Prade’s rough fuzzy sets. The aim of this chapter is to consider lower and upper soft rough intuitionistic fuzzy approximation of intuitionistic fuzzy sets in intuitionistic fuzzy soft approximation space (IF soft approximation space) and obtain a new hybrid model called soft rough intuitionistic fuzzy sets which can be seen as extension of both the previous work by Dubois and Prade and Feng et al.

ObjectivesBenchmarking is common in clinical registries to support the improvement of health outcomes by identifying underperforming clinician or health service providers. Despite the rise in clinical registries and interest in...

Outliers detection has wide application for financial surveillance. The Traditional outlier detection method is based on statistical models, such as ARMA, ARCH and GARCH, which require special hypotheses, and they are inappropriate to apply to complex financial data, such as high frequency data. This paper introduces a new data mining method to detect outliers for analysis of share index fluctuation. Based on the Voronoi diagram, we propose a novel outlier detection method, which called Voronoi based Outlier Detection (VOD). Experiments show the VOD method performs more efficient and effective against the existing method in outlier detection for financial data.

<abstract><p>In data analysis, the choice of an appropriate regression model and outlier detection are both very important in obtaining reliable results. Gamma regression (GR) is employed when the distribution of the dependent variable is gamma. In this work, we derived new methods for outlier detection in GR. The proposed methods are based upon the adjusted and standardized Pearson residuals. Furthermore, a comparison of available and proposed methods is made using a simulation study and a real-life data set. The results of simulation and real-life application the evidence better performance of the adjusted Pearson residual based outlier detection approach.</p></abstract>

Measurement errors commonly occur in 24-h hormonal data and may affect the outcomes of such studies. Measurement errors often appear as outliers in such data sets; however, no well-established method is available for their automatic detection. In this study, we aimed to compare performances of different methods for outlier detection in hormonal serial data. Hormones (glucose, insulin, thyroid-stimulating hormone, cortisol, and growth hormone) were measured in blood sampled every 10 min for 24 h in 38 participants of the Leiden Longevity Study. Four methods for detecting outliers were compared: (1) eyeballing, (2) Tukey’s fences, (3) stepwise approach, and (4) the expectation-maximization (EM) algorithm. Eyeballing detects outliers based on experts’ knowledge, and the stepwise approach incorporates physiological knowledge with a statistical algorithm. Tukey’s fences and the EM algorithm are data-driven methods, using interquartile range and a mathematical algorithm to identify the underlying distribution, respectively. The performance of the methods was evaluated based on the number of outliers detected and the change in statistical outcomes after removing detected outliers. Eyeballing resulted in the lowest number of outliers detected (1.0% of all data points), followed by Tukey’s fences (2.3%), the stepwise approach (2.7%), and the EM algorithm (11.0%). In all methods, the mean hormone levels did not change materially after removing outliers. However, their minima were affected by outlier removal. Although removing outliers affected the correlation between glucose and insulin on the individual level, when averaged over all participants, none of the 4 methods influenced the correlation. Based on our results, the EM algorithm is not recommended given the high number of outliers detected, even where data points are physiologically plausible. Since Tukey’s fences is not suitable for all types of data and eyeballing is time-consuming, we recommend the stepwise approach for outlier detection, which combines physiological knowledge and an automated process.