• Home
  • Search
  • Predicting Classification Accuracy of Unlabeled Datasets Using Multiple Deep Neural Networks
  • Open Access IconOpen Access
  • Cite Icon8
  • https://doi.org/10.1109/access.2022.3169279Copy DOI Icon

Predicting Classification Accuracy of Unlabeled Datasets Using Multiple Deep Neural Networks

Show More
  • Abstract
  • Highlights & Summary
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In machine learning problems, we usually assume that the validation accuracy is a good estimation of prediction accuracy for datasets without ground truth. In reality, this assumption may not hold. Therefore, we propose an approach to estimate the prediction accuracy of a target model on unlabeled datasets. The proposed approach uses multiple target homogeneous models to assign each unlabeled sample a confidence value, based on the number of models agreeing on the predicted label. With the confidence values, the prediction accuracy of the target model on the datasets can be estimated. In the experiments, the target model is a convolutional neural network (CNN) model, and the homogeneous models only differ in initial weights. The experiments are conducted with datasets from a wide variety of music genres. The estimation performance of the proposed approach is compared with the reversed testing qualities (RTQ) and the ensemble average qualities (EAQ) approaches. The RTQ approach was proposed to estimate the prediction accuracy of trained models, and the EAQ approach was originally designed for estimating the predictive uncertainty of individual samples. We apply all three compared models to estimate prediction accuracy of datasets by using a linear model. The parameters of the linear model are either computed by using multiple labeled datasets or one labeled dataset. The experimental results show that when compared with the RTQ approach, the proposed approach has much lower estimation errors for some datasets. When compared with the EAQ approach, the proposed approach is more robust for datasets with large distribution shifts. Finally, we show an additional benefit of the proposed approach. In case that the estimated accuracy is unsatisfactory, we may re-train the target model with a new training set, which contains the original training samples plus new training samples with manual labeling from the unlabeled dataset. The experimental results confirm that it is more effective to select (and label) new samples from those with low confidence values than those randomly selected. Overall, the proposed approach is a promising approach for estimating prediction accuracy on unlabeled datasets.

Similar Papers
  • Research Article
  • Citations2

국립공원에 서식하는 생물종의 분포모델 평가

  • Jun 30, 2023
  • Korea National Park Research Institute
  • Soon Jik Kwon +1
  • PDF
  • Research Article
  • Citations2

Research on Community Risk Prediction Model and Management Based on Deep Learning

  • Mar 11, 2022
  • Wireless Communications and Mobile Computing
  • Weida Yin
  • Research Article

The effect of training data size on real-time respiration prediction using long short-term memory model

  • Jun 06, 2025
  • Radiation Oncology
  • Wenzheng Sun +9
  • Research Article
  • Citations2

Application value of machine learning algorithms for predicting recurrence after resection of early-stage hepatocellular carcinoma

  • Aug 01, 2021
  • Zhonghua wai ke za zhi [Chinese journal of surgery]
  • X C Li +4
  • Research Article
  • Citations45

Quantitative analysis modeling for the ChemCam spectral data based on laser-induced breakdown spectroscopy using convolutional neural network

  • Aug 25, 2020
  • Plasma Science and Technology
  • Xueqiang Cao +5
  • Research Article

Multimodal data-driven intelligent prediction of sinter ore FeO content: A KanFuzz-Trans model approach

  • Aug 04, 2025
  • Ironmaking & Steelmaking: Processes, Products and Applications
  • Yu-Han Gong +5
  • Research Article
  • Citations12

A heterogeneous tissue model for treatment planning for magnetic resonance-guided laser interstitial thermal therapy

  • Feb 05, 2018
  • International Journal of Hyperthermia
  • Drew Mitchell +9
  • PDF
  • Research Article
  • Citations5

Research on Short-term Ionospheric Prediction Combining with EOF and ARIMA Model Over Guangxi Area

  • Feb 08, 2020
  • The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
  • C Li +4
  • Research Article
  • Citations11

BRR-Net: A tandem architectural CNN-RNN for automatic body region localization in CT images.

  • Aug 20, 2020
  • Medical Physics
  • Vibhu Agrawal +3
  • Research Article
  • Citations32

EVALUATING RISK-PREDICTION MODELS USING DATA FROM ELECTRONIC HEALTH RECORDS.

  • Mar 01, 2016
  • The Annals of Applied Statistics
  • Le Wang +4
  • Conference Article
  • Citations2

Oil Temperature Prediction of Long-Distance Hot-Crude-Oil Pipeline Based on GA-BP Optimization

  • Sep 28, 2020
  • Volume 3: Operations, Monitoring, and Maintenance; Materials and Joining
  • Tao Yu +4
  • Research Article

Abstract B059: Cancer Recurrence and Anxiety/Depression: High Accuracy Artificial Intelligence Predictive Modeling

  • Jul 10, 2025
  • Clinical Cancer Research
  • Nabil R Adam +2
  • Book Chapter
  • Citations16

Effects of Sample Size on Accuracy and Stability of Species Distribution Models: A Comparison of GARP and Maxent

  • Jan 01, 2012
  • Xinmei Chen +1
  • PDF
  • Research Article

The Impact of Short Video Live Broadcast on the Sales of Sports Machinery and Equipment

  • May 17, 2024
  • International Journal of e-Collaboration
  • Chunyue Huang +1
  • PDF
  • Research Article
  • Citations11

Dual-Hybrid Modeling for Option Pricing of CSI 300ETF

  • Jan 13, 2022
  • Information
  • Kejing Zhao +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.