• Cite Icon14
  • https://doi.org/10.1145/2063576.2063616Copy DOI Icon

Frequency-aware similarity measures

  • Oct 24, 2011
  • Dustin Lange +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Measuring the similarity of two records is a challenging problem, but necessary for fundamental tasks, such as duplicate detection and similarity search. By exploiting frequencies of attribute values, many similarity measures can be improved: In a person table with U.S. citizens, Arnold Schwarzenegger is a very rare name. If we find several Arnold Schwarzeneggers in it, it is very likely that these are duplicates. We are then less strict when comparing other attribute values, such as birth date or address. We put this intuition to use by partitioning compared record pairs according to frequencies of attribute values. For example, we could create three partitions from our data: Partition 1 contains all pairs with rare names, Partition 2 all pairs with medium frequent names, and Partition 3 all pairs with frequent names. For each partition, we learn a different similarity measure: we apply machine learning techniques to combine a set of base similarity measures into an overall measure. To determine a good partitioning, we compare different partitioning strategies. We achieved best results with a novel algorithm inspired by genetic programming.

Similar Papers
  • Research Article
  • Citations45

Retrieval, alignment, and clustering of computational models based on semantic annotations

  • Jan 01, 2011
  • Molecular Systems Biology
  • Marvin Schulz +4
  • Book Chapter

A Meta Graph-Based Top-k Similarity Measure for Heterogeneous Information Networks

  • Jan 01, 2020
  • Xiangtao Chen +4
  • Research Article

Risk factors for hematuria during indwelling urinary catheterization in acute myocardial infarction: a comparative analysis using logistic regression and decision tree

  • Jan 01, 2025
  • Frontiers in Cardiovascular Medicine
  • Jia Zhang +5
  • Research Article
  • Citations510

A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees

  • Feb 12, 2018
  • European Journal of Operational Research
  • Arno De Caigny +2
  • Research Article

Environmental Factors Prediction in Preterm Birth Using Comparison between Logistic Regression and Decision Tree Methods:  An Exploratory Analysis

  • Jan 01, 2021
  • SSRN Electronic Journal
  • Rakesh Kumar Saroj +2
  • Research Article
  • Citations9

Building Multiclass Classification Model of Logistic Regression and Decision Tree Using the Chi-Square Test for Variable Selection Method

  • Apr 30, 2022
  • Journal of Hunan University Natural Sciences
  • Waego H Nugroho +3
  • Research Article
  • Citations5

Application and Comparison of Laboratory Parameters for Forecasting Severe Hand-Foot-Mouth Disease Using Logistic Regression, Discriminant Analysis and Decision Tree.

  • Jan 01, 2016
  • Clinical laboratory
  • Meili Sui +13
  • Supplementary Content
  • Citations24

Ensemble Classification Approach for Sarcasm Detection.

  • Nov 22, 2021
  • Behavioural Neurology
  • Jyoti Godara +3
  • PDF
  • Research Article
  • Citations54

Implementing Machine Learning for Smart Farming to Forecast Farmers’ Interest in Hiring Equipment

  • Feb 11, 2022
  • Journal of Food Quality
  • Manik Rakhra +5
  • Book Chapter
  • Citations4

Lung Cancer Stages Classification Based on Differential Gene Expression

  • Jan 01, 2023
  • Moshira S Ghaleb +2
  • PDF
  • Research Article
  • Citations19

Correlates of cognitive impairment in the elderly in China: A cross-sectional study.

  • Oct 20, 2022
  • Frontiers in Public Health
  • Yuan-Yuan Wang +4
  • Research Article

Spammer Detection and Fake User Identification on Social Media

  • Mar 14, 2025
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • T Meghana, +3
  • Research Article
  • Citations6

Discovering and Recognizing of Imbalance Human Activity in Healthcare Monitoring using Data Resampling Technique and Decision Tree Model

  • Nov 01, 2023
  • Journal of Advanced Research in Applied Sciences and Engineering Technology
  • Raihani Mohamed +5
  • Research Article

A unified network embedding algorithm for multi-type similarity measures

  • Jan 01, 2023
  • AI Open
  • Rui Feng +5
  • PDF
  • Research Article
  • Citations7

Application of kernel functions for accurate similarity search in large chemical databases.

  • Apr 01, 2010
  • BMC Bioinformatics
  • Xiaohong Wang +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.