Convex-Concave Programming: An Effective Alternative for Optimizing Shallow Neural Networks

Purpose In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Design/methodology/approach In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Findings In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Originality/value In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing.

Whether using a shallow neural network with one hidden layer, or a deep network with many hidden layers, the training data must represent subgroups of the deposit type being explored to be useful. Published examples of neural networks have mostly been limited to one individual mineral deposit for training. Variation of geologic features among deposits within a type are so large that a single deposit cannot provide proper information to train a neural net to generalize and guide exploration for other deposits. Models trained with only one deposit tend to be academic successes but are not of practical value in exploration for other deposits. This is why it takes much experience examining many deposits to properly train an economic geologist—a neural network is not any different. Two examples of shallow neural networks are used to demonstrate the power of neural networks to possibly locate undiscovered deposits and to provide some suggestions of how to deal with missing data. The training data needs to include information spatially related to known deposits and hopefully information from many different deposits of the type. Lessons learned from these and other examples point to a proposed sampling plan for data that could lead to a generalized neural network for exploration. In this plan, 10 or more well-explored gold-rich porphyry copper deposits from around the world with 100 or more sample sites near and some distance from each deposit would probably capture important variability among such deposits and provide proper data to train and test a shallow neural network to predict locations of undiscovered deposits.

Peptide-based vaccine development needs accurate prediction of the binding affinity between major histocompatibility complex I (MHC I) proteins and their peptide ligands. Nowadays more and more machine learning methods have been developed to predict binding affinity and some of them have become the popular tools. However most of them are designed by the shallow neural networks. Bengio said that deep neural networks can learn better fits with less data than shallow neural networks. In our case, some of the alleles only have dozens of peptide data. In addition, we transform each peptide into a characteristic matrix and input it into the model. As we know when dealing with the problem that the input is a matrix, convolutional neural network (CNN) can find the most critical features by itself. Obviously, compared with the traditional neural network model, CNN is more suitable for predicting binding affinity. Different from the previous studies which are based on blocks substitution matrix (BLOSUM), we used novel feature to do the prediction. Since we consider that the order of the sequence, hydropathy index, polarity and the length of the peptide could affect the binding affinity and the properties of these amino acids are key factors for their binding to MHC, we extracted these information from each peptide. In order to make full use of the data we have obtained, we have integrated different lengths of peptides into 15mer based on the binding mode of peptide to MHC I. In order to demonstrate that our method is reliable to predict peptide-MHC binding, we compared our method with several popular methods. The experiments show the superiority of our method.

In this work, we propose Wasserstein distributionally robust shallow convex neural networks (WaDiRo-SCNNs) to provide reliable nonlinear predictions when subject to adverse and corrupted data sets. Our approach is based on the reformulation of a new convex training program for rectified linear unit–based shallow neural networks, and this allows us to cast the problem into the order-1 Wasserstein distributionally robust optimization framework. Our training procedure is conservative, has low stochasticity, is solvable with open-source solvers, and is scalable to large industrial deployments. We provide out-of-sample performance guarantees, show that hard convex physical constraints can be enforced in the training program, and propose a mixed-integer convex posttraining verification program to evaluate model stability. WaDiRo-SCNN aims to make neural networks safer for critical applications, such as in the energy sector. Finally, we numerically demonstrate our model’s performance through both a synthetic experiment and a real-world power system application, namely, the prediction of hourly energy consumption in nonresidential buildings within the context of virtual power plants, and evaluate its stability across standard regression benchmark data sets. The experimental results are convincing and showcase the strengths of the proposed model. Funding: This work was possible thanks to funding from the Fonds de recherche du Québec, the Natural Sciences and Engineering Research Council of Canada, Mitacs Accelerate, and Hilo by Hydro-Québec [Grants RGPIN-2023-04235, IT35303, and IT38517; Scholarships CGRS-M and B1X].

In this work, we propose ShallowDeepNet, a novel system architecture that includes a shallow and a deep neural network. The shallow neural network has the duty of data preprocessing and generating adversarial samples. The deep neural network has the duty of understanding data and information as well as detecting adversarial samples. The deep neural network gets its weights from transfer learning, adversarial training, and noise training. The system is examined on the biometric (fingerprint and iris) and the pharmaceutical data (pill image). According to the simulation results, the system is capable of improving the detection accuracy of the biometric data from 1.31% to 80.65% when the adversarial data is used and to 93.4% when the adversarial data as well as the noisy data are given to the network. The system performance on the pill image data is increased from 34.55% to 96.03% and then to 98.2%, respectively. Training on different types of noise can benefit us in detecting samples from unknown and unseen adversarial attacks. Meanwhile, the system training on the adversarial data as well as noisy data occurs only once. In fact, retraining the system may improve the performance further. Furthermore, training the system on new types of attacks and noise can help in enhancing the system performance.

Trustworthy volumetric flow measurements are essential in many applications such as power plant controls or district heating systems. Flow metering under disturbed flow conditions, such as downstream of bends, is a challenge and leads to errors of up to 20 %. In this paper, an algorithm based on a shallow neural network (SNN) is developed, leading to a significant error reduction for strongly disturbed flow profiles. To cover a wide range of disturbances, the training dataset was chosen to consist of three base types of elbow configurations. For 83 % of the test data, the SNN produces a smaller error than the state-of-the-art approach. The average error is reduced from 2.25 % to 0.42 %. For the SNN, an error of less than 1 % can be achieved for downstream distances greater than 10 pipe diameters. The SNN demonstrated robustness to various reductions of the training dataset, as well as to noisy input data. Additionally, simulation data of a realistic pipe system with a significantly different geometry compared with the training data was used for testing. In this strong extrapolation, the mean error of the SNN was always smaller than the state-of-the-art approach and an error of less than 1 % could be achieved for more than 10 pipe diameters downstream of the last disturbance.

A Robust DNS Flood Attack Detection with a Hybrid Deeper Learning Model

This paper presents an odour sensing device with machine learning algorithms that can classify urine odour to aid incontinent individuals. The device contains custom made metal oxide sensors that are controlled by a Teensy 3.6 microcontroller. The gas classification experiment was performed with an automatic test rig on four compounds, acetone, ammonia, ethyl acetate and synthetic urine; at five concentration levels and three humidity levels. The collected data were processed employing three classifier methods, k-nearest neighbour (KNN), a shallow neural network (MLP) and a convolutional neural network (CNN). The overall classification accuracies of these three models are 93.5%, 92.6% and 95.4%, respectively. More importantly, both KNN and CNN have 100% success rate in urine classification, and only one misclassification of synthetic urine occurred with the shallow neural network.

This paper investigates within-scene path loss prediction at 1.8 GHz in a smart-campus micro-urban environment using multivariate machine-learning (ML) models. We leverage an open measurement campaign from Covenant University (Nigeria) comprising three routes with per-sample geospatial predictors—longitude, latitude, altitude, elevation, Tx–Rx distance, and clutter height—and train Random Forests (RF), Gradient Boosting (GB), Support Vector Regression (SVR), Gaussian Processes (GP), and a shallow neural network (NN). A unified pipeline with 5-fold cross-validation (CV), seeded reproducibility, and Optuna-driven hyperparameter search is adopted; performance is reported as RMSE/MAE/R2 (mean ± sd). To contextualize feature reliability, we include Pearson correlation heatmaps and Variance Inflation Factors (VIFs), a systematic ablation of predictors, and TreeSHAP beeswarm analyses on held-out splits. We also evaluate spatially aware validation (blocked CV within route and leave-one-route-out checks) to mitigate optimism due to spatial autocorrelation. Results show that multivariate ML consistently outperforms classical empirical formulas (COST-231, ECC-33) in this campus setting, with RF achieving the lowest errors across routes (RMSE ≈ 2.14/2.16/2.95 dB for X/Y/Z, respectively), while GB ranks second and kernel methods (SVR/GP) and the NN trail closely behind. Ablation confirms that distance plus coordinates drive the largest gains, with terrain/clutter providing route-dependent refinements. SHAP analyses align with these findings, highlighting stable, interpretable contributions of geospatial covariates. Spatial CV increases absolute errors moderately but preserves model ranking, supporting the robustness of conclusions. Overall, scenario-aware, multivariate ML yields material accuracy gains for smart-campus planning at 1.8 GHz.

Renewable sources such as solar PV and wind are stochastic in nature, hence their integration with emerging isolated microgrid (MG) is challenging especially with regards to stability issues. An accurate prediction model of wind and solar sources is necessary to analyze the uncertainty in MG system and to encourage the reliable participation of wind and solar power in the energy market. The advancement in deep learning methods has made it possible to develop a multi-step forecasting model unlike shallow neural networks (SNNs). The time series forecasting using SNN and Recurrent Neural Network (RNN) suffers from the problem of vanishing/exploding gradient while training. To eliminate this problem the long short-term memory (LSTM) RNN has been used in this study for wind speed and solar irradiance prediction. The forecasted solar and wind power is applied to analyze the load frequency behavior and the response of nonrenewable sources for sudden rise and fall in load power demand and PI controller is used to mitigate frequency deviation to ensure the stability of the MG power system.

As for the problem of too long training time of convolution neural network (CNN), this paper proposes a fast training method for CNN in SAR automatic target recognition (ATR). The CNN is divided into two parts: one that contains all the convolution layers and sub-sampling layers is considered as convolutional auto-encoder (CAE) for unsupervised training to extract high-level features; the other that contains fully connected layers is regarded as shallow neural network (SNN) to work as a classifier. The experiment based on MSATR database shows that the proposed method can tremendously reduce the training time with little loss of recognition rate.

The accumulation of electronic waste (e-waste) is becoming a problem in society. Old parts and components are conveniently discarded instead of being recycled. Economic and environmental measures should be taken by individuals and organizations to enhance sustainability. This could include desoldering and reusing parts from electronic circuit boards. Hence, the purpose of the dataset presented in this paper is for the classification of used electronic parts in linear voltage regulator power supply circuits. The dataset presented in this paper comprises low-resolution (30 × 30 pixels) grayscale images of major reusable electronic parts from a typical adjustable regulated linear voltage power supply kitset. The three major reusable parts are capacitors, potentiometers, and voltage regulator ICs. These are typically the most relatively expensive components. Data representing the parts are extracted from 960 × 720 pixel workspace images containing multiple parts. This permits the dataset to be used with multiple types of classifiers, such as lightweight shallow neural networks (SNNs), support vector machines (SVMs), or convolutional neural networks (CNNs). Classification accuracies of 93.5%, 94.9%, and 98.4% were achieved with SNNs, SVMs, and CNNs, respectively. Successful detection and classification of parts will permit a Niryo Ned robotic arm to pick and place parts in the desired locations. The dataset can be used by other academics and researchers working with the Niryo Ned robot and Matlab to handle electronic parts. It can be expanded to include relatively expensive components from other types of electronic circuit boards.

Object detection using deep neural networks (DNNs) involves a huge amount of computation which impedes its implementation on resource/energy-limited user-end devices. The reason for the success of DNNs is due to having knowledge over all different domains of observed environments. However, we need a limited knowledge of the observed environment at inference time which can be learned using a shallow neural network (SHNN). In this paper, a systemlevel design is proposed to improve the energy consumption of object detection on the user-end device. An SHNN is deployed on the user-end device to detect objects in the observing environment. Also, a knowledge transfer mechanism is implemented to update the SHNN model using the DNN knowledge when there is a change in the object domain. DNN knowledge can be obtained from a powerful edge device connected to the user-end device through LAN or Wi-Fi. Experiments demonstrate that the energy consumption of the user-end device and the inference time can be improved by 78% and 40% compared with running the deep model on the user-end device.

In this paper we investigate the ability of Shallow Neural Networks i.e. neural networks with one hidden layer, to solve Laplace’s equation on the half space. We are interested in answering the question if it is possible to fit the boundary value using a neural network then is it possible to learn the solution to the PDE in the entire region using the same network? Our analysis is done primarily in Barron Spaces, which are function spaces designed to include neural networks with a single hidden layer and infinite width. Our results indicate in general the solution is not in the Barron space even if the boundary values are. However, the solution can be approximated to ∼ ε^2 accuracy with functions of a low Barron norm. We implement a Physics Informed Neural Network with a custom loss function todemonstrate some of the theoretical results shown before.

Calculus equation is an important tool for mathematical research and plays an important role in most natural science research. Since the beginning of the eighteenth century, people have gradually used differential and integral equations to solve physical problems. In general, several different aspects of differential equations in the field of mathematics are concerned and studied by most scholars. However, this paper studies and establishes the optimal model for numerical solution of differential equations through deep learning and genetic algorithm. In this paper, the solution of ordinary differential equations is solved through the use of polynomial function space, while the linear combination of simple function x and its product nx can obtain multinomial function space. The space function form of polynomial is very simple, and the operation ability is very strong. Almost all functions can be approximated, and the function space can be transformed by a simple function. Through data simulation test results, it can be found that the oscillation of neural network output is stronger and stronger with the increase in depth, that is to say, the deeper depth endows the neural network with stronger oscillation properties, so for the oscillation function, the depth neural network fitting effect is better than the shallow neural network. Therefore, by combining deep learning and genetic algorithm, this paper studies and establishes the optimal model for numerical solution of differential equations, and finds that the deep neural network can largely complete data simulation.