ENSEMBLE METHOD BASED ON AVERAGING SHAPES OF OBJECTS USING THE PYRAMID METHOD

To obtain the spatial relationship between three or more pixels in the texture image, bispectrum is choosen to extract texture features of the image, and it contains amplitude information and phase information of the image. Due to some problems in neural network, such as unstable classifier design, configuration, training, the research based on the ensemble of neural networks appears. Compared with a single neural network, an ensemble of neural networks has better fault tolerance and generalisation ability. In this paper, bispectrum is used to extract texture features and the neural network ensembles are used to recognize the texture images. The experimental results demonstrate that the ensemble of BP neural networks can effectively improve correct recognition rate of texture images.

Purpose. At the present stage, network attack detection systems based on the following neural networks are most often offered: multilayer perceptron, Kohonen network or self-organizing map and their combinations. The efficiency problem of two approaches to detect attacks on a computer network using neural network technology based on the normalized data of the open NSL-KDD database is considered. Methodology. As an architectural solution to the network attack detection system, it is proposed to consider the following approaches: based on one neural network determining the attack class (first approach) and an ensemble of five neural networks (second approach), which at the first stage determines the attack category (DoS, Probe, U2R , R2L), and in the second stage, the attack class belonging to a certain category. Findings. Based on the neural networks created in the MatLAB program, a study was conducted of their error on the length of the training sample using various training algorithms: Levenberg-Marquardt; Bayesian Regularization; Scaled Conjugate Gradient with different numbers of hidden neurons (minimum, average and maximum). Certain optimal parameters of neural networks with two approaches were determined. Originality. In the course of conducting experiments with various approaches, the results obtained were: TP (True Positive); FP (False Positive); FN (False Negative); TN (True Negative), based on which the following indicators were calculated for assessing the quality of solutions: correct determination of network attacks; false positives; reliability; accuracy and completeness, which prove the feasibility of using an ensemble of neural networks (second approach). Practical value. On the created neural networks with various approaches, studies were conducted: the operating time of neural networks; errors of the first kind; errors of the second kind. According to the results of the first study, the average operating time of an ensemble of neural networks is 0.92 s, and the operating time of a neural network (according to the first approach) is 2.21 s. According to the results of the second study, the error of the first kind using an ensemble of neural networks is 2.17%, and using the neural network (the first approach) – 7.39%. According to the results of the third study, the error of the second kind using an ensemble of neural networks is 3.91%, and using the neural network (the first approach) – 6.96%, which is confirmed by the efficiency of using an ensemble of neural networks (second approach).

Damage in structures can negatively affect their functionality and safety and leads to failure. Thus it is very important to monitor structures for occurrence, location and severity of damage. Structural health monitoring techniques provide information on the life expectancy of structures simultaneously detects and locates structural damage. Damage identification of structures based on vibration has always been important subjects and are being rapidly used to damage and location of structures. Artificial Neural Networks (ANNs) as a soft computing method using dynamic parameters of structures have been utilized increasingly for structural damage detection due to their excellent pattern recognition capability. Dynamic parameters of structure are easy to implement for damage identification and can directly linked to the topology of structure. This study presents the application of ANN for damage identification in steel beams using dynamic parameters. For identification of severity and location of damage, at first, five individual neural networks corresponding to mode 1 to mode 5 are considered. At the second step, a method based on neural network ensemble is proposed to combine the outcomes of the individual neural networks to a single network. Ensemble results were evaluated and discussed according to the differences between predicted output by ANN and desire data (target data) obtained from experimental modal analysis of structure.KeywordsArtificial neural networksMean square errorModal analysisDamage detectionNeural network ensemble

Background and Purpose:Accurate automated infarct segmentation is needed for acute ischemic stroke studies relying on infarct volumes as an imaging phenotype or biomarker that require large numbers of subjects. This study investigates whether an ensemble of convolutional neural networks (CNN) trained on multiparametric DWI maps outperforms single networks trained on solo DWI parametric maps.Materials and Methods:CNNs were trained on combinations of DWI, ADC, and low b-value-weighted images from 116 subjects. The performances of the networks (measured by Dice score, sensitivity and precision) were compared to one another and to ensembles of 5 networks. To assess the generalizability of the approach, the best performing model was applied to an independent evaluation cohort of 151 subjects. Agreement between manual and automated segmentations for identifying patients with large lesions volumes was calculated across multiple thresholds (21 cm3, 31 cm3, 51 cm3, and 70 cm3).ResultsAn ensemble of CNNs trained on DWI, ADC and low b-value-weighted images produced the most accurate acute infarct segmentation over individual networks (p<0.0001). Automated volumes correlated with manually measured volumes (Spearman’s ρ=0.91, p<0.0001) for the independent cohort. For the task of identifying patients with large lesion volumes, agreement between manual outlines and automated outlines was high (Cohen’s κ 0.86 to 0.90, p<0.0001).ConclusionAcute infarcts are more accurately segmented using ensembles of CNNs trained with multi-parametric maps than using a single model trained with a solo map. Automated lesion segmentation can perform with high agreement with manual techniques for identifying patients with large lesion volumes.

Melanoma is a rare skin cancer that constitutes only 1% of skin cancer cases. However, its ability to spread to other organs makes it deadliest among the four major cancer types. Early diagnosis of melanoma is essential, as it prevents cancer from spreading to other body parts, therefore significantly reducing mortality rates. In this study, we presented a forward selection‐based ensembling strategy for deep neural networks to aid the diagnosis of melanoma in dermoscopy images. The proposed approach uses an ensemble of neural networks with varying input sizes to effectively capture size‐related various properties of dermoscopy images. To this end, EfficientNet models B3–B7 are used with input resolutions of 256, 384, 512, and 768. Training and validation are carried out in a triple stratified cross‐validation style with folds providing patient isolation, balance in the percentage of classes and balanced patient count distribution. Ensembles are formed by a modified form of forward selection algorithm. Experimental results show that the AUC for classification is increased by 2.01% using the proposed ensembling scheme.

Training an ensemble of convolutional neural networks requires much computational resources for a large set of high-resolution medical 3D scans because deep representation requires many parameters and layers. In this study, 100 3D late gadolinium-enhanced (LGE)-MRIs with a spatial resolution of 0.625 mm × 0.625 mm × 0.625 mm from patients with atrial fibrillation were utilized. To contain cost of the training, down-sampling of images, transfer learning and ensemble of network’s past weights were deployed. This paper proposes an image processing stage using down-sampling and contrast limited adaptive histogram equalization, a network training stage using a cyclical learning rate schedule, and a testing stage using an ensemble. While this method achieves reasonable segmentation accuracy with the median of the Dice coefficients at 0.87, this method can be used on a computer with a GPU that has a Kepler architecture and at least 3 GB memory.

Predicting stock returns by classifier ensembles

Multi-model ensemble is an important fundamental technique of practical value for many artificial intelligence applications. However, the usage for multi-model ensemble has been limited when it is combined with deep neural networks to construct ensemble of deep neural networks. Due to the big time and computing resources required to train and to integrate multiple deep neural networks for the achievement of multi-model ensemble, the engineering application field where developing time and computing resources are usually restricted, has not yet widespreadly benefited from ensemble of deep neural networks. To alleviate this situation, we present a new multi-model ensemble approach entitled feature transferring based multi-model ensemble (FTBME), for ensemble of deep neural networks. Primarily, we propose a feature transferring based multi-model training strategy to more affordably find multiple extra models based on a given previously optimized deep neural network model. Sequentially, to develop better ensemble solutions, we design a more effective random greedy based ensemble selection strategy to filter out models non-positive to ensemble generalization. Finally, inspired by the idea of averaging parameter points, we propose to fuse the obtained models in weight space which eventually reduces the expense of ensemble at the testing stage to a single deep neural network model while retaining the generalization. These three advances constitute the resulting technique FTBME. We conducted extensive experiments using deep neural networks, from light weight to complex, on ImageNet, CIFAR-10 and CIFAR-100. Results show that, given a deep neural network model which has been well-optimized and reaching its limit, FTBME can obtain better generalization with minor extra training expense while maintaining the expense to a single model at ensemble testing. This promising property of FTBME make us believe that it could be leveraged to broaden the usage for ensemble of deep neural networks, alleviating the situation that the engineering application field has not yet widespreadly benefited from ensemble of deep neural networks.

The paper describes the approach to automatic face recognition for access control application area using wavelet transform and neural network's ensemble. Wavelet transform implements the compression of the face images and thus accelerates the classifier's work, while neural network's ensemble with proposed decision rule provides low recognition error. Proposed ensemble's decision rule carries out a high level of unknown people's access rejection, which is the most significant requirement for the access control systems, and gives a good balance between known and unknown people's recognition errors.

Accurate tumor segmentation plays an important role in radiosurgery planning and the assessment of radiotherapy treatment efficacy. In this paper we propose a method combining an ensemble of 2D convolutional neural networks for doing a volumetric segmentation of magnetic resonance images. The segmentation is done in three steps; first the full tumor region, is segmented from the background by a voxel-wise merging of the decisions of three networks learned from three orthogonal planes, next the segmentation is refined using a cellular automaton-based seed growing method known as growcut. Finally, within-tumor sub-regions are segmented using an additional ensemble of networks trained for the task. We demonstrate the method on the MICCAI Brain Tumor Segmentation Challenge dataset of 2014, and show improved segmentation accuracy compared to an axially trained 2D network and an ensemble segmentation without growcut. We further obtain competitive Dice scores compared with the most recent tumor segmentation challenge.

Artificial neural networks apply complex non-linear functions to pattern recognition problems. An ensemble is a 'committee' of neural networks that usually outperforms single neural networks. Bronchiolitis is a common manifestation of viral lower respiratory tract infection in infants and toddlers. To train artificial neural network ensembles to predict the disposition and length of stay in children presenting to the Emergency Department with bronchiolitis. A specifically constructed database of 119 episodes of bronchiolitis was used to train, validate, and test a neural network ensemble. We used EasyNN 7.0 on a 200 Mhz pentium PC with a maths co-processor. The ensemble of neural networks constructed was subjected to fivefold validation. Comparison with actual and predicted dispositions was measured using the kappa statistic for disposition and the Kaplan-Meier estimations and log rank test for predictions of length of stay. The neural network ensembles correctly predicted disposition in 81% (range 75-90%) of test cases. When compared with actual disposition the neural network performed similarly to a logistic regression model and significantly better than various 'dumb machine' strategies with which we compared it. The prediction of length of stay was poorer, 65% (range 60-80%), but the difference between observed and predicted lengths of stay were not significantly different. Artificial neural network ensembles can predict disposition for infants and toddlers with bronchiolitis; however, the prediction of length of hospital stay is not as good.

Niche-based cooperative co-evolutionary ensemble neural network for classification

GUDU: Geometrically-constrained Ultrasound Data augmentation in U-Net for echocardiography semantic segmentation

In this paper, we propose an effective method of $$\hbox {PM}_{2.5}$$ prediction based on image contrast-sensitive features and weighted bagging based neural network (WBBNN). Three types of image contrast-sensitive features are first extracted from the images and fuzzified. Next, a weighted bagging strategy combining the ensemble fuzzy neural network (FNN) and ensemble radial basis function neural network (RBFNN) is established. The ensemble neural network (NN), regardless of FNN and RBFNN, is obtained by simply averaging the outputs of component neural networks. And these component neural networks are trained by the improved gradient descent algorithm and samples acquired by bootstrap sampling. Finally, the WBBNN is used to forecast $$\hbox {PM}_{2.5}$$ concentration by extracting three types of image contrast-sensitive features. Results of experiments demonstrate that our prediction method is more reliable, practical and efficient than FNN, RBFNN, the ensemble NNs, and state-of-the-art quality assessment method in terms of predicting the concentration of $$\hbox {PM}_{2.5}$$. More importantly, an improved gradient descent algorithm is developed to accelerate the convergence speed and ensure the prediction accuracy of WBBNN and the fuzzy features acquired by the feature fuzzification method can greatly improve the robustness and precision of WBBNN.

Currently, ensemble approaches based, among other things, on the use of non-network models are powerful tools for solving data analysis problems in various practical applications. An important problem in the formation of ensembles of models is ensuring the synergy of solutions by using the properties of a variety of basic individual solutions; therefore, the problem of developing an approach that ensures the maintenance of diversity in a preliminary pool of models for an ensemble is relevant for development and research. This article is devoted to the study of the possibility of using a method for the probabilistic formation of neural network structures developed by the authors. In order to form ensembles of neural networks, the influence of parameters of neural network structure generation on the quality of solving regression problems is considered. To improve the quality of the overall ensemble solution, using a flexible adjustment of the probabilistic procedure for choosing the type of activation function when filling in the layers of a neural network is proposed. In order to determine the effectiveness of this approach, a number of numerical studies on the effectiveness of using neural network ensembles on a set of generated test tasks and real datasets were conducted. The procedure of forming a common solution in ensembles of neural networks based on the application of an evolutionary method of genetic programming is also considered. This article presents the results of a numerical study that demonstrate a higher efficiency of the approach with a modified structure formation procedure compared to a basic approach of selecting the best individual neural networks from a preformed pool. These numerical studies were carried out on a set of test problems and several problems with real datasets that, in particular, describe the process of ore-thermal melting.