A novel in-depth analysis approach for domain-specific problems based on multidomain data

The difficulty of feature extraction and the small sample size are two challenges in the field of mechanical fault diagnosis for a long time. Here we propose an intelligent mechanical fault diagnosis method for scenario with small sample datasets. This method can not only diagnose bearing faults but also gear faults, and has strong generalization performance. We use convolutional neural network to realize automatic feature extraction. Through sliding window scanning, one sample set is expanded to three sub-sample sets with different scales to meet the needs of deep learning training. Three convolutional networks are used to extract the features of the subsets respectively to ensure that their useful features are fully extracted. After feature extraction, the feature is reconstructed through feature splicing. Because of the unique advantages of SVM in dealing with small sample sets, we use SVM to classify the reconstructed features. We use the bearing data set collected by Case Western Reserve University in the United States, the bearing fault data set collected by Xi'an Jiaotong University in China, and the gearbox fault data collected by the University of Connecticut in the United States to conduct experiments. The experimental results show that the accuracy of training, validation and testing of the proposed method on the three data sets all reach 100%. This proves that our method can not only tackle the two challenges, but also has high fault diagnosis accuracy and strong generalization performance. It is hoped that our proposed method can contribute to the development of mechanical fault diagnosis.

The class imbalance problem is encountered in real-world applications of machine learning and results in a classifier's suboptimal performance. Researchers have rigorously studied the resampling, algorithms, and feature selection approaches to this problem. No systematic studies have been conducted to understand how well these methods combat the class imbalance problem and which of these methods best manage the different challenges posed by imbalanced data sets. In particular, feature selection has rarely been studied outside of text classification problems. Additionally, no studies have looked at the additional problem of learning from small samples. This paper presents a first systematic comparison of the three types of methods developed for imbalanced data classification problems and of seven feature selection metrics evaluated on small sample data sets from different applications. We evaluated the performance of these metrics using area under the receiver operating characteristic (AUC) and area under the precision-recall curve (PRC). We compared each metric on the average performance across all problems and on the likelihood of a metric yielding the best performance on a specific problem. We examined the performance of these metrics inside each problem domain. Finally, we evaluated the efficacy of these metrics to see which perform best across algorithms. Our results showed that signal-to-noise correlation coefficient (S2N) and Feature Assessment by Sliding Thresholds (FAST) are great candidates for feature selection in most applications, especially when selecting very small numbers of features.

Neural networks have been constructed to analyze artificial time series derived from the Tent and Henon map as well as population data of the Canadian lynx. Simple three-layer forwardfeed networks, trained on a small sample data set, provided reasonably good fit to the data and performed well on short-term predictions. Simple neural network models trained on small data sets can perform quite well with synthetic chaotic series as well as population data. The accuracy of the predictions is comparable to that obtained using embedding techniques with three embedding dimensions or nonlinear regression models. In relation to embedding techniques, the current forecasting method using neural networks may be regarded as a global rather than a local method using d embedding dimensions with unit delay time. However, once the network has been trained, the calculation of predicted values is very fast and straightforward, without any searching as required by embedding methods.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

To solve the problem that it is difficult to identify the overvoltage in high-voltage electrical system of EMUs, a method for identifying overvoltage types of high-voltage electrical system of multiple units based on ShuffleNet lightweight convolutional neural network (CNN) is proposed. Six overvoltage types are mapped into gray images by B2G algorithm, which is input into ShuffleNet network to identify all kinds of gray images, and different overvoltages are classified into their families by training. This method analyzes the accuracy of the model under different parameters, and obtains the optimal parameter combination of the model through the training of the model under different parameters. Six shallow machine learning models are built and compared. Experimental results show that this method has higher accuracy in small sample data sets. Compared with traditional machine learning, it avoids the complexity of manual feature extraction, improves the generalization ability of the model, and verifies that this method has better recognition and classification performance.

Real driving environment EEG-based detection of driving fatigue using the wavelet scattering network

The machine learning method, now widely used for predicting well performance from unconventional reservoirs in the industry, generally needs large data sets for model development and training. The large data sets, however, are not always available, especially for newly developed unconventional plays. The objective of this work is to develop an innovative machine learning method for predicting well performance in unconventional reservoirs with a relatively small data set. For a small training data set, the corresponding machine learning model can significantly suffer from so-called overfitting meaning that the model can match the training data but has poor predictivity. To overcome this, our new method averages predictions from multiple models that are developed with the same model input, but different initial guesses of model parameters that are unknowns in a machine learning algorithm and determined in the model training. The averaged results are used for the final model prediction. Unlike traditional ensemble learning methods, each prediction in the new method uses all the input data rather than its subset. We mathematically prove that the averaged prediction provides less model uncertainty and under certain conditions the optimum prediction. It is also demonstrated that the method practically minimizes the overfitting and gives relatively unique prediction. The usefulness of the method is further confirmed by its successful application to the data set collected from less than 100 wells in an unconventional reservoir. Sensitivity results with the trained machine learning model show that the model results are consistent with the domain knowledge regarding the production from the reservoir.

Improving classification accuracy using data augmentation on small data sets

High-quality internal experimental materials data are often “small data.” Using external datasets, e.g., data from the literature and other groups, to expand the “small data” to relatively “big data” is particularly important for improving the prediction accuracy of machine learning models. However, the most critical issue is how to use a small amount of internal reliable data to filter external data extracted from multiple sources and obtain optimal datasets with a distribution similar to that of internal data. This is called the multi-source materials data problem. This question was addressed by designing an active learning-based data screening (ALDS) model that is suitable for small material samples. This study used negative expansion materials as subjects. The results show that ALDS can be used to screen the external multi-source dataset and far exceeds traditional outlier filtering methods. The average mean absolute percentage error (MAPE) of the predictive key target property of the negative thermal expansion coefficient (NTEC) was reduced from 4.301 to 0.056. Furthermore, reverse design experiments were conducted on anti-perovskite manganese nitride (AMN), a type of negative thermal expansion material, to prove that the ALDS model can guide the reverse design on a small AMN dataset using the MAPE between ALDS prediction and the ground truth of samples’ three property indicators. This achieves high confidence levels with values of 0.203,0.126, and 0.115. This verifies that the ALDS proposed method improves the effect of ”materials small internal data” in guiding material design and property prediction.

Two significant obstacles hinder the advancement of Radiology AI. The first is the challenge of overfitting, where small training data sets can result in unreliable outcomes. The second challenge is the need for more generalizability, the lack of which creates difficulties in implementing the technology across various institutions and practices. A recent innovation, deep neuroevolution (DNE), has been introduced to tackle the overfitting issue by training on small data sets and producing accurate predictions. However, the generalizability of DNE has yet to be proven. This paper strives to overcome this barrier by demonstrating that DNE can achieve satisfactory results in diverse external validation sets. The main innovation of the work is thus showing that DNE can generalize to varied outside data. Our example use case is predicting brain metastasis from neuroblastoma, emphasizing the importance of AI with limited data sets. Despite image collection and labeling advancements, rare diseases will always constrain data availability. We optimized a convolutional neural network (CNN) with DNE to demonstrate generalizability. We trained the CNN with 60 MRI images and tested it on a separate diverse collection of images from over 50 institutions. For comparison, we also trained with the more traditional stochastic gradient descent (SGD) method, with the two variants of (1) training from scratch and (2) transfer learning. Our results show that DNE demonstrates excellent generalizability with 97% accuracy on the heterogeneous testing set, while neither form of SGD could reach 60% accuracy. DNE's ability to generalize from small training sets to external and diverse testing sets suggests that it or similar approaches may play an integral role in improving the clinical performance of AI.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

Small sample image recognition using improved Convolutional Neural Network

A novel interpolation and difference optimized (IDO) machine learning model to predict the depth of silicon etching is proposed, which is particularly well-suited to addressing small sample problems. Our approach involves dividing both experimental and simulation data obtained from the Technology Computer-Aided Design (TCAD) software into training and testing sets. Both experimental data and TCAD simulation data are used as inputs to machine learning module 1 (ML1), while ML2 takes the actual experimental data as inputs and then learns the difference between the experimental data and the TCAD simulation data, outputting the difference. The outputs generated by ML1 and ML2 serve as input parameters to machine learning module 3 (ML3), and the weights of ML3 are updated through its own learning process to produce the final prediction results. We demonstrate that our IDO model, which contains three basic ML algorithms, achieves higher prediction accuracy compared to the basic ML algorithm alone. Moreover, through ablation studies, we establish that the three components of the IDO prediction model are inseparable. The IDO model exhibits improved generalization performance, making it particularly suitable for small sample datasets in the semiconductor processing domain.

Radio frequency (RF) fingerprint representing the inherent hardware characteristics of mobile devices has been employed to classify and identify wireless devices for the security of Internet of Things (IoT). Existing works on RF fingerprinting are usually based on the amplitude or phase of RF signal envelope, which leads to relatively coarse features. Moreover, the classification performance over small sample dataset is poor. To solve the problem, a novel device identification method based on RF fingerprinting with on deep learning is proposed. In particular, the RF signal are transformed into two dimensional representations by image preprocessing. Then the gray images representing the RF fingerprints are classified by employing classical CNN. To verify the performance of the proposed approach, a testbed is constructed by using MATLAB build framework of gray image preprocessing. Extensive experiment results show that the identification accuracy can reach at least 90%. Even with the sample rate of 20Gsps. Particularly, the accuracy of iPhone can reach 100%. It is verified that the proposed method can effectively classify mobile devices even with small sample RF fingerprints represented two dimensional gray images,

In pavement engineering, the data sets that are typically obtained from experiments are small and cannot be classified as big data. The effective use of machine learning techniques such as artificial neural networks (ANN) for small data is a challenge because of poor accuracy of models. This paper presents a method of multiple structure multiple run and ranging to optimize ANN to produce models with small data sets with high accuracy. In this method, a large number of data fitting ANNs, with different number of neurons, layers, training and validation ratios, and randomized layer weights and biases are run in parallel, and the most accurate ANN is filtered out on the basis of the lowest MSE or highest R. The process is demonstrated with weather and pavement temperature data for a hot mix asphalt (HMA) and an open graded friction course (OGFC) pavement. Models are generated to predict the temperature at a depth of 12.5 mm below the surface. For the HMA pavement, an accuracy of 99.73% was obtained and an optimum structure was found to be with 4 layers, 11 neurons, 70% training ratio, 15% validation ratio. In the case of the OGFC pavement, an accuracy of 99.75% was obtained for an optimum structure with 3 layers, 11 neurons, 75% training ratio, 15% validation ratio. Furthermore, the fitting/regression problem was converted to a classification problem with different ranges, and then ANNs were utilized to develop very accurate classification models with small datasets.

Microplastic particle ingestion or inhalation by humans is a problem of growing concern. Unfortunately, current research methods that use machine learning to understand their potential harms are obstructed by a lack of available data. Deep learning techniques in particular are challenged by such domains where only small or imbalanced data sets are available. Overcoming this challenge often involves oversampling underrepresented classes or augmenting the existing data to improve model performance. This paper proposes GANsemble: a two-module framework connecting data augmentation with conditional generative adversarial networks (cGANs) to generate class-conditioned synthetic data. First, the data chooser module automates augmentation strategy selection by searching for the best data augmentation strategy. Next, the cGAN module uses this strategy to train a cGAN for generating enhanced synthetic data. We experiment with the GANsemble framework on a small and imbalanced microplastics data set. A Microplastic-cGAN (MPcGAN) algorithm is introduced, and baselines for synthetic microplastics (SYMP) data are established in terms of Fréchet Inception Distance (FID) and Inception Scores (IS). We also provide a synthetic microplastics filter (SYMP-Filter) algorithm to increase the quality of generated SYMP. Additionally, we show the best amount of oversampling with augmentation to fix class imbalance in small microplastics data sets. To our knowledge, this study is the first application of generative AI to synthetically create microplastics data.