SUPER Learning: A Supervised-Unsupervised Framework for Low-Dose CT Image Reconstruction

Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?

In recent years, deep neural networks have become the most powerful supervised learning method. Several advanced neural networks, such as AlexNet, ZFNet, Inception, ResNet, and DenseNet, have achieved excellent performance on image recognition tasks. However, deep neural networks rely heavily on huge training sets to obtain good performance. Many applications, such as medical image analysis, do not allow for such large training sets, and it is difficult to train such networks on small-scale training sets. Magnifying narrow band imaging (M-NBI) is widely used to assist doctors in diagnosing gastric cancer, but relatively few of these images are available, compared with the number of general images. In this paper, we propose to use a Siamese network architecture to learn discriminative feature representations based on pairs of images. Then, we use a micro neural network to recognize these features and classify the input images. Our experimental results show that the proposed network can effectively learn discriminative features from a limited number of training images, and also that it can successfully recognize gastric cancer in M-NBI images.

Supervised deep learning (DL) models are becoming ubiquitous in computational chemistry because they can efficiently learn complex input-output relationships and predict chemical properties at a cost significantly lower than methods based on quantum mechanics. The central challenge in many DL applications is the need to invest considerable computational resources in generating large ( N>1×105 ) training sets such that the resulting DL model can be generalized reliably to unseen systems. The lack of better alternatives has encouraged the use of low-cost and relatively inaccurate density-functional theory (DFT) methods to generate training data, leading to DL models that lack accuracy and reliability. In this article, we describe a robust and easily implemented approach based on property-specific atom-centered potentials (ACPs) that resolves this central challenge in DL model development. ACPs are one-electron potentials that are applied in combination with a computationally inexpensive but inaccurate quantum mechanical method (e.g. double-ζ DFT) and fitted against relatively few high-level data ( N≈1×103 – 1×104 ), possibly obtained from the literature. The resulting ACP-corrected methods retain the low cost of the double-ζ DFT approach, while generating high-level-quality data in unseen systems for the specific property for which they were designed. With this approach, we demonstrate that ACPs can be used as an intermediate method between high-level approaches and DL model development, enabling the calculation of large and accurate DL training sets for the chemical property of interest. We demonstrate the effectiveness of the proposed approach by predicting bond dissociation enthalpies, reaction barrier heights, and reaction energies with chemical accuracy at a computational cost lower than the DFT methods routinely used for DL training data set generation.

Bag-of-Words (BoW) model has been widely used for feature representation in multimedia search area, in which a key step is to vector-quantize local image descriptors and generate a visual vocabulary. Popular visual vocabulary construction schemes generally perform a flat or hierarchical clustering operation using a very large training set in their original description space. However, these methods usually suffer from two issues: (1) A large training set is required to construct a large visual vocabulary, making the construction computationally inefficient; (2) The generated visual vocabularies are heavily biased towards the training samples. In this work, we introduce a partitioned k-means clustering (PKM) scheme to efficiently generate a large and unbiased vocabulary using only a small training set. Instead of directly clustering training descriptors in their original space, we first split the original space into a set of subspaces and then perform a separate k-means clustering process in each subspace. Sequentially, we can build a complete visual vocabulary by combining different cluster centroids from multiple subspaces. Comprehensive experiments demonstrate that the proposed method indeed generates unbiased vocabularies and provides good scalability for building large vocabularies.

Compressed image restoration via deep deblocker driven unified framework

Knowledge transfer is widely held to be a primary mechanism that enables humans to quickly learn new complex concepts when given only small training sets. In this paper, we apply knowledge transfer to deep convolutional neural nets, which we argue are particularly well suited for knowledge transfer. Our initial results demonstrate that components of a trained deep convolutional neural net can constructively transfer information to another such net. Furthermore, this transfer is completed in such a way that one can envision creating a net that could learn new concepts throughout its lifetime. The experiments we performed involved training a Deep Convolutional Neural Net (DCNN) on a large training set containing 20 different classes of handwritten characters from the NIST Special Database 19. This net was then used as a foundation for training a new net on a set of 20 different character classes from the NIST Special Database 19. The new net would keep the bottom layers of the old net (i.e. those nearest to the input) and only allow the top layers to train on the new character classes. We purposely used small training sets for the new net to force it to rely as much as possible upon transferred knowledge as opposed to a large and varied training set to learn the new set of hand written characters. Our results show a clear advantage in relying upon transferred knowledge to learn new tasks when given small training sets, if the new tasks are sufficiently similar to the previously mastered one. However, this advantage decreases as training sets increase in size.

BERT is a pre-trained language model that achieves state-of-the-art performance on natural language processing (NLP) tasks. Once it was published, it quickly became one of the most popular models in the NLP field. The official recommended method for applying BERT to downstream tasks is fine-tuning. However, we argue that transfer learning is also a very practical approach to applying BERT, and this approach has its own advantages compared to fine-tuning. In this paper, we explore the advantages of the transfer learning approach through the method that uses transfer learning and fine-tuning approaches separately to apply BERT to the same eight GLUE benchmark tasks and compare the training time spent and model performance scores, such as accuracy or F1 score, of the two approaches. Finally, this paper finds that among all eight GLUE tasks and on small training sets, transfer learning saves 30% to 50% of the time compared to a fine-tuning approach to train the models on the data of downstream tasks and can achieve very similar performance. Besides, for the same amount of time, transfer learning can obtain higher performance scores than fine-tuning on larger training sets. In conclusion, on small training sets, transfer learning has a huge advantage in time consumption under the premise of approximate performance, and it also performs better on large training sets under the same length of time, compared to fine-tuning.

Statistical shape priors can be crucial in segmenting objects when the data differentiates poorly between the object and its surroundings. For reliable learning, while some methods need high-quality expert segmentations, other methods need large training sets, both of which can often be difficult to obtain in clinical deployment or scientific studies. We propose to couple deep neural networks with a pointset-based shape prior that can be learned effectively despite training sets having small size and imperfections in expert curation. The prior relies on sparse Riemannian modeling in Kendall shape space. Results on clinical brain magnetic resonance imaging data show that our framework improves over the state of the art in segmenting the thalamus and the caudate.

Inference of m-NLP data using radial basis function regression with center-evolving algorithm

The probabilistic mapping of landslide occurrence at a high spatial resolution and over a large geographic extent is explored using random forests (RF) machine learning; light detection and ranging (LiDAR)-derived terrain variables; additional variables relating to lithology, soils, distance to roads and streams and cost distance to roads and streams; and training data interpreted from high spatial resolution LiDAR-derivatives. Using a large training set and all predictor variables, an area under the receiver operating characteristic (ROC) curve (AUC) of 0.946 is obtained. Our findings highlight the value of a large training dataset, the incorporation of a variety of terrain variables and the use of variable window sizes to characterize the landscape at different spatial scales. We also document important variables for mapping slope failures. Our results suggest that feature selection is not required to improve the RF modeling results and that incorporating multiple models using different pseudo absence samples is not necessary. From our findings and based on a review of prior studies, we make recommendations for high spatial resolution, large-area slope failure probabilistic mapping.

Woody plant encroachment (WPE) is transforming grasslands globally, yet accurately mapping this process remains challenging. State-funded, publicly available high-resolution aerial imagery offers a potential solution, including the USDA’s National Agriculture Imagery Program (NAIP) and NSF’s National Ecological Observatory Network (NEON) Aerial Observation Platform (AOP). We evaluated the accuracy of land cover classification using NAIP, NEON, and both sources combined. We compared two machine learning models—support vector machines and random forests—implemented in R using large training and evaluation data sets. Our study site, Konza Prairie Biological Station, is a long-term experiment in which variable fire and grazing have created mosaics of herbaceous plants, shrubs, deciduous trees, and evergreen trees (Juniperus virginiana). All models achieved high overall accuracy (>90%), with NEON slightly outperforming NAIP. NAIP underperformed in detecting evergreen trees (52–78% vs. 83–86% accuracy with NEON). NEON models relied on LiDAR-based canopy height data, whereas NAIP relied on multispectral bands. Combining data from both platforms yielded the best results, with 97.7% overall accuracy. Vegetation indices contributed little to model accuracy, including NDVI (normalized digital vegetation index) and EVI (enhanced vegetation index). Both machine learning methods achieved similar accuracy. Our results demonstrate that free, high-resolution imagery and open-source tools can enable accurate, high-resolution, landscape-scale WPE monitoring. Broader adoption of such approaches could substantially improve the monitoring and management of grassland biodiversity, ecosystem function, ecosystem services, and environmental resilience.

BackgroundThe success of genomic selection depends mainly on the extent of linkage disequilibrium (LD) between markers and quantitative trait loci (QTL), the number of animals in the training set (TS) and the heritability (h2) of the trait. The extent of LD depends on the genetic structure of the population and the density of markers. The aim of this study was to calculate accuracy of direct genomic estimated breeding values (DGEBV) using best linear unbiased genomic prediction (GBLUP) for different marker densities, heritabilities and sizes of the TS in simulated populations that mimicked previously reported extent and pattern of LD in beef cattle.ResultsThe accuracy of DGEBV increased significantly (p < 0.05) with the increase in the number of bulls in the TS (480, 960 or 1920), trait h2 (0.10, 0.25 or 0.40) and marker densities (40 k or 800 k). Increasing the number of animals in the TS by 4-fold and using their phenotypes to estimate marker effects was not sufficient to maintain or increase the accuracy of DGEBV obtained using estimated breeding values (EBVs) when the trait h2 was lower than 0.40 for both marker densities. Comparing to expected accuracies of parent average (PA), the gains by using DGEBV would be of 27%, 13% and 10% for trait h2 equal to 0.10, 0.25 and 0.40, respectively, considering the scenario with 40 k markers and 1920 bulls in TS.ConclusionsAs reported in dairy cattle, the size of the TS and the extent of LD have major impact on the accuracy of DGEBV. Based on the findings of this simulation study, large TS, as well as dense marker panels, aiming to increase the level of LD between markers and QTL, will likely be needed in beef cattle for successful implementation of genomic selection.

We apply our general-purpose algorithm for visual category recognition using bag-of-visual-words and other visual features and fusion of SVM classifiers to the recognition of indoor locations. This is an important application in many emerging fields, such as mobile augmented reality and autonomous robots. We evaluate the proposed method with other location recognition systems in the ImageCLEF 2010 RobotVision contest. The results show that given a large enough training set, a purely appearance-based method can perform very well - ranked first for one of the contest's training sets.

Quantitative structure–activity relationship (QSAR) methods have been widely applied in drug discovery, lead optimization, toxicity prediction, and regulatory decisions. Despite major advances in algorithms and software, QSAR models have inherent limitations associated with a size and chemical-structure diversity of the training set, experimental error, and many characteristics of structure representation and correlation algorithms. Whereas excellent fit to the training data may be readily attainable, often models fail to predict accurately chemicals that are outside their domain of applicability. A QSAR’s utility and, in the case of regulatory decisions, justification for usage increasingly depend on the ability to quantify a model’s potential for predicting unknown chemicals with some known degree of certainty. It is never possible to predict an unknown chemical with absolute certainty. Here we report on two QSAR models based on different data sets for classification of chemicals according to their ability to bind to the estrogen receptor. The models were developed by using a novel QSAR method, Decision Forest, which combines the results of multiple heterogeneous but comparable Decision Tree models to produce a consensus prediction. We used an extensive cross-validation process to define an applicability domain for model predictions based on two quantitative measures: prediction confidence and domain extrapolation. Together, these measures quantify the accuracy of each prediction within and outside of the training domain. Despite being based on large and diverse training sets, both QSAR models had poor accuracy for chemicals within the domain of low confidence, whereas good accuracy was obtained for those within the domain of high confidence. For prediction in the high confidence domain, accuracy was inversely proportional to the degree of domain extrapolation. The model with a larger training set of 1,092, compared with 232 for the other, was more accurate in predicting chemicals at larger domain extrapolation, and could be particularly useful for rapidly prioritizing potential endocrine disruptors from large chemical universe.

We present a new supervised deep-learning approach to the problem of the extraction of smeared spectral densities from Euclidean lattice correlators. A distinctive feature of our method is a model-independent training strategy that we implement by parametrizing the training sets over a functional space spanned by Chebyshev polynomials. The other distinctive feature is a reliable estimate of the systematic uncertainties that we achieve by introducing several ensembles of machines, the broad audience of the title. By training an ensemble of machines with the same number of neurons over training sets of fixed dimensions and complexity, we manage to provide a reliable estimate of the systematic errors by studying numerically the asymptotic limits of infinitely large networks and training sets. The method has been validated on a very large set of random mock data and also in the case of lattice QCD data. We extracted the strange-strange connected contribution to the smeared R-ratio from a lattice QCD correlator produced by the ETM Collaboration and compared the results of the new method with the ones previously obtained with the HLT method by finding a remarkably good agreement between the two totally unrelated approaches.