Automatic real-time gait event detection in children using deep neural networks.

Artificial neural networks (ANN) have been used in many applications such like handwriting recognition and speech recognition. It is well-known that learning rate is a crucial value in the training procedure for artificial neural networks. It is shown that the initial value of learning rate can confoundedly affect the final result and this value is always set manually in practice. A new parameter called beta stabilizer has been introduced to reduce the sensitivity of the initial learning rate. But this method has only been proposed for deep neural network (DNN) with sigmoid activation function. In this paper we extended beta stabilizer to long short-term memory (LSTM) and investigated the effects of beta stabilizer parameters on different models, including LSTM and DNN with relu activation function. It is concluded that beta stabilizer parameters can reduce the sensitivity of learning rate with almost the same performance on DNN with relu activation function and LSTM. However, it is shown that the effects of beta stabilizer on DNN with relu activation function and LSTM are fewer than the effects on DNN with sigmoid activation function.

The article explores modern neural network architectures for the automatic detection and recognition of marine surface objects and obstacles of given classes throughout the full image area, applicable for execution in real or near real time on an optoelectronic vision system to au-tomate and improve the safety of civil marine navigation. A formal statement of the problem of automatic detection of objects on images is given. The state-of-the-art algorithms for detecting objects in images based on use of artificial convolutional neural networks were reviewed, their comparison was made and a reasonable choice was made in favor of the most efficient neuralnetwork architecture in terms of computational complexity to recognition accuracy. The subject area is studied, as well as publicly available databases of surface objects suitable for use in the training of algorithms using artificial neural networks. The article concluded that there is insuffi-cient labeled data for training neural network algorithms, as a result of which the authors inde-pendently collected research images and video sequences, prepared and labeled the collected data containing surface marine objects and other obstacles that represent a navigation hazard for ships. Based on the selected neural network architecture, a new neural network algorithm for automatic full-frame detection and recognition of surface objects was developed, and an artificial neural network was trained using the prepared database of images of typical objects. The resulting algorithm was tested by the authors on a validation data set, the quality of its work was estimated using various metrics, and the algorithm’s performance was measured. Conclusions are made about the necessity to expand the collected database of images of typical marine objects, further steps are proposed to improve the accuracy of the developed software and algorithmic complex and its implementation to be used in a marine optoelectronic machine vision system for automa-tion and improving the safety of civil navigation.

Water is a big challenge not only in India but in many countries of the world. Machine learning and forecasting model has been employed towards water demand and ground water level prediction. But in terms of water scarcity, much less work has been carried out by employing machine learning algorithms like 'artificial neural network' (ANN) and 'grey forecasting' model for forecasting water scarcity and none has focused on historical data like water availability, water consumption for a particular area and stress value for predicting water scarcity. So accordingly, we here have developed a water scarcity prediction system based on historical data by employing 'deep neural networks' which is an advanced form of 'artificial neural networks'. We have also compared 'deep neural network' with existing machine learning algorithms such as support vector machine (SVM), logistic regression and Naive Bayes. From the analysis of algorithms based on dataset, deep neural networks have been found as the best prediction model for water scarcity.

Semiconductor optical amplifier (SOA) is a promising solution for future O-band optical amplification. However, the SOA-induced nonlinearity will affect the quality of the signals and cause bit errors. Nowadays, the neural network is a hot topic in communication networks and has been considered a practical algorithm for signals equalization. In this paper, we investigate the performance of the three types of neural networks: deep neural networks (DNN), convolutional neural network (CNN), and long short term memory (LSTM) neural networks for the compensation of the SOA-induced nonlinearity. We first compare the fitting ability of the three neural networks on the saturated response of SOA. Based on the simulation result, LSTM has the best nonlinear expressiveness for the SOA-induced nonlinearity. Subsequently, we do some experimental validations in an intensity modulation and direct-detection (IM/DD) system, where the 100 Gbit/s pulse-amplitude-modulation-4 (PAM4) signals are transmitted over 80 km at O-band using an 18 GHz electro-absorption modulated laser (EML). The bit-error-ratio (BER) results show that the cell states in LSTM play a crucial role in nonlinearity compensation. Furthermore, LSTM offers the best equalization performance and can achieve a receiver sensitivity of −18.0 dBm.

<p dir="ltr">Recurrent Neural Networks (RNNs) are a major class of Artificial Neural Networks (ANNs). Suitable network architecture is vital for RNNs to achieve high performance. However, designing the architecture of deep RNNs (DRNNs) is a complicated and time-consuming process. Each layer of a DRNN contains numerous hyper-parameters. It is challenging to optimize the hyper-parameters of all layers jointly to achieve the best possible learning performance. Trial-and-error methods for designing DRNNs have been proven to be laborious and highly costly in practice. Therefore, an efficient automatic architecture search technique is needed to design DRNNs.</p><p dir="ltr">Genetic Algorithm (GA) is a popular evolutionary computation based approach. Existing research works already explored the use of GAs in designing ANNs. But, most of the existing approaches focus on the design of fixed-depth ANNs. However, it is not easy to fix the depth of DRNNs in advance, since different problems need DRNNs of different depths. Hence, the potential of GA has to be further explored to evolve the architecture of DRNNs of varying depths.</p><p dir="ltr">The primary goal of this thesis is to develop advanced GA approaches for designing Long Short Term Memory (LSTM) based DRNN architectures of varying depths effectively and efficiently.</p><p dir="ltr">Firstly, this thesis proposes a GA-based algorithm called Two-Stage Surrogate-Assisted GA (TS-SA-GA) with a progressive incremental strategy to design the architecture of LSTM networks of varying depths. The progressive approach can effectively extend well-designed shallow networks to high performing deep networks. Moreover, the new GA-based algorithm adopts newly designed knowledge-driven crossover and mutation operators to identify and repair LSTM network designs affected by inappropriate use of activation functions, thereby significantly reducing the chances for GA to evolve hard-to-train LSTM network architectures. Furthermore, this thesis proposes a two-stage surrogate method to predict the trainability and fitness of LSTM networks. This improves the efficiency and effectiveness of the GA-based algorithm to evolve LSTM networks.</p><p dir="ltr">Secondly, this thesis proposes a new algorithm called Evolutionary Design of LSTM Ensembles (ED-LSTM-Ensemble) based on multi-objective optimization techniques to evolve LSTM networks and ensembles simultaneously to directly utilize the ensemble performance to drive the evolution of base LSTM networks. Additionally, this thesis proposes a connection weight inheritance strategy to evolve LSTM networks and ensembles efficiently and effectively.</p><p dir="ltr">Thirdly, this thesis proposes a new algorithm called Skip Connections through Evolutionary and Differential Architecture search (SCEDA) to design LSTM networks together with appropriate skip connections automatically. Designing LSTM networks together with skip connections is highly challenging. This thesis hybridize gradient-based DAS and evolutionary architecture search to optimize the architecture of LSTM networks with suitable skip connections in a single evolutionary process to improve the efficiency and effectiveness of the search process.</p><p dir="ltr">Finally, this thesis conducts an in-depth empirical analysis of the impact of activation functions on the trainability of LSTM networks based on the concept of Edge Of Chaos. The analysis reveals the strong interrelation between the activation functions across multiple layers of an LSTM network and the trainability of the network. The analysis also highlights the importance of controlling activation functions to avoid generating untrainable LSTM networks. On the basis of the analysis, this thesis proposes a machine learning model to guide the use of activation functions in LSTM networks.</p>

Gait recognition is a noncontact biometric procedure that determines the identity or health status of a person by analyzing his or her walking posture and habits, including skeletal and joint movements. The most remarkable feature of this method is the possibility of conducting recognition without demanding much cooperation from participants. Therefore, this recognition technique has attracted much attention from scholars. Additionally, because of the rapid development of graphics processing unit technology, related hardware and computation performance, the applications of deep-learning technology are considerably enhanced. The objective of this study was to apply a deep neural network (DNN), which employs deep-learning technology, to achieve gait-based automatic pedestrian detection and recognition. In contrast to using wearable devices to precisely capture skeletal and joint movements, pedestrian color-image sequences were used as input in this study. Subsequently, a pretraining convolutional neural network (CNN) was employed to capture pedestrian location and extract pedestrian dense optical flow to serve as concrete low-level feature inputs. Then, a finely-tuned DNN based on the wide residual network was employed to extract high-level abstract features. In addition, to overcome the difficulty of obtaining local temporal features by using a 2D CNN, part of the 3D convolutional structure was introduced into the CNN. This design enabled use of limited memory to acquire more effective features and enhance the DNN performance. The experimental results show that the proposed method has exceptional performance for pedestrian detection and recognition.

Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons. Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain. Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders. Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.

Maxout neurons for deep convolutional and LSTM neural networks in speech recognition

Artificial neural networks have been evaluated and compared for modeling extreme wave forces exerted on coastal bridges during hurricanes. Long Short-Term Memory (LSTM) is selected for deep learning neural networks. A feedforward neural network (FFNN) is employed to represent the shallow learning network for comparison purposes. The two case studies consist of an emerged bridge deck destroyed by Hurricane Ivan and a submerged bridge deck impaired in Hurricane Katrina. Datasets for model training and verifications consist of wave elevation and force time series resulting from previous validated numerical wave load modeling studies. Results indicate that both deep LSTM and shallow FFNNs are able to provide very good predictions of wave forces with correlation coefficients above 0.98 by comparing model simulations and data. Effects of training algorithms on network performance have been investigated. Among several training algorithms, the adaptive moment estimation (Adam) training optimizer leads to the best LSTM performance, while Levenberg–Marquardt (LM) optimized backpropagation is among the most effective training algorithms for FFNNs. In general, a shallow FFNN-LM network results in slightly higher correlation coefficients and lower error than those from an LSTM-Adam network. For sharp variation in nonlinear wave forces in the emerged bridge case study during Hurricane Ivan, FFNN-LM predictions of wave forces show better matching with the quick variations in nonlinear wave forces. FFNN-LM’s speed is approximately 4 times faster in model training but is about twice as slow in model verification and application than the LSTM-Adam network. Neural network simulations have shown substantially faster than CFD wave load modeling in our case studies.

Though large amounts of work with artificial intelligence are used in typhoon rainfall forecasting, the predictive skills of existing models are unsatisfactory. To address this problem, this study aims to propose an improved cluster‐wise typhoon rainfall forecasting model that integrates the grid‐search cross‐validation method with machine learning and deep learning (DL) models including support vector machine (SVM), random forest (RF), adaptive boosting (AdaBoost), convolutional neural network (CNN), and long short‐term memory (LSTM). Grid‐search cross‐validation is a modified parameterization technique that helps to find the best parameters for machine/DL models. In the first stage, a second‐order polynomial regression model was used to cluster the typhoon track; and in the second stage, cluster‐wise typhoon rainfall was recognized within a 500 km radius from each typhoon center. After that, a modified cluster‐wise typhoon rainfall forecasting model was proposed using cluster‐wise antecedent hourly typhoon rainfall within this distance for 1–6 hr lead time. Results show that the proposed model based on the SVM, RF, AdaBoost, CNN, and LSTM is capable of providing more accurate forecasts (the efficiency of the forecast is increased by 45%–90%) than the existing typhoon rainfall forecasting models that are based on SVM with a genetic algorithm, RF, artificial neural network, multilayer perceptron network, and deep neural network. Therefore, the current study recommends using cluster‐wise typhoon rainfall forecasting model with a grid‐search cross‐validation method for disaster prevention and mitigation.

Recently, Quantitative Biology (QB) held a discussion on “AI (artificial intelligence) for Life Science” among editorial board members and interested scholars in anticipation of rapid development of this growing area after AlphaGo and ChatGPT mania. Many young people tend to get confused between facts and fictions; heated debates are unavoidable even among their mentors. When deep learning as represented by convolutional neural networks and LSTM (long short-term memory) was made available for bioinformatics students, many of them rushed into this research field and tried to adopt these methods in all their projects without knowing the history that these tools were becoming successful consistently with Moore’s Law (relating to rapid computer technology advances), but more importantly due to new structural/functional understanding of vision and auditory circuits in the brain. Recently, some young people have claimed “LSTM is dead, long live transformer” (which is somewhat like saying “the bike is dead, long live the car”), and have amplified the threat that ChatGPT could wipe out human jobs. They believe transformer is the “silver bullet” for all learning tasks, clearly reflecting their lack of basic knowledge (i.e. “No Free Lunch Theory,” the trade-off of such global “attention network” is to pay the price for complexity: difficulty of training and high memory costs). There is no doubt ML (machine learning) and AI have brought a new revolution in science and technology, and will deliver huge unforeseeable impact to human everyday life as well as to social relationships. In this context, QB journal could be a great platform for encouraging intellectual discussions and for promoting AI for Life Science. Here, I would like to use the DIALOG to “抛砖引玉” (make some initial remarks to get the ball rolling), although it is my personal opinion which is inevitably subject to bias and limitations. AI: Do you know my name “Artificial Intelligence” is defined by the Oxford English Dictionary as the capacity of computer systems (which may be referred as a “robot”) to exhibit or simulate your intelligent behavior? NI: Wait a minute, intelligence itself is defined as the ability to learn, understand and think in a logical way. Can you think? AI: No. But that definition is too restrictive, actually intelligence has different scopes and degrees. Simple intelligent control devices date back to antiquity, from windmills to thermostat. NI: Agree, everything is relative. Macromolecules (e.g., enzyme) and cells (e.g., immune cell) might be considered to be intelligent; see how a white blood cell is chasing bacteria in the youtube website (search for “Crawling neutrophil chasing a bacterium”). Our emergent/collective intelligent behavior does not require a brain or even a neuron; see how slime molds can solve optimization—Hamilton cycle-problem more effectively than a human in the youtube website (search for “Intelligence without a brain?”). Before there was any neuron, Ca2+ sensing and signaling were already fully functional. Even if one knock-out a neural circuit, redundant signaling pathways, albeit on much local and slower scale, could still function by themselves (just as if highways were demolished, local roads/paths would still be working). In fact, the most detailed “Neural signal propagation atlas of Caenorhabditis elegans” [1] demonstrated that functional connectivity differs from anatomy (connectome) because extra-synaptic signaling also drives neural dynamics! Worm brain connectomes are largely invariant but every human brain connectome is very different (depending the diversity of learning experience). The human brain functional activity is far more complex than that of a worm brain, certainly beyond what a neural circuit could explain. AI: Well, that’s impressive. I thought only we could beat humans, albeit only in certain specified areas for now. My master promise to make an artificial general intelligent (AGI) robot which can understand or learn any intellectual task that you humans or other animals can. NI: Well, that is not possible and is not an appropriate goal either. It is not possible because we are an evolutionary/developmental product (with a long history of learning and memory from evolutionary tinkering): our living objective goal is survival of the population. You, on the other hand, are an engineering product (efficiently and optimally designed): your goal is to extend and maximize human capability. It makes sense to complement the human brain, but foolish and dangerous to try to replace it. AI: We are not satisfied with merely passing the Turing Test; most of us don’t care if we could really think as long as we could act like we think (that is, as if we do have a mind and consciousness, as expressed in the so called “Weak AI hypothesis”). After all, the brain is a computer; a neural network is just an electric or ionic circuit. Logical computing does not need to be based on living cells. NI: That is not true, because a neuron is not just a simple node (logic gate) and neural network is not fixed circuit, neither as in Pitts&Mcculloch perceptron model. A single neuron, even a single dendrite, is much more complicated and far more powerful than a full-blown deep-learning artificial neural network (ANN) [2]. AI: Even though single neurons are complex computational devices (dendritic non-linear), running an equivalent multilayer ANN is 2000 times faster than computing with biophysics N-methyl-D-aspartate receptor channel models [3]. More information can be found in the youtube website (search for “Dendrites: why biological neurons are deep neural networks”). NI: Often silicon computing (CPU, GPU) is much faster than brain computing (action potential, ms); but there is no comparison in energy efficiency. Bacteria sensing (chemotaxis computation), powered by ATP (adenosine triphosphate) hydrolysis, uses very little energy that is close to the Landauer limit, whereby achieving or maintaining one bit of information requires minimum of 1 kT ln (2) free energy [4]. The human brain consumes oft-quoted 20 W, compared to the AlphGo system 1 MW! More recent estimate of energy audit is only 0.1 W to cortical computing, and long-distance communication cost is 3.5 W [5]. AI: Assuming we have infinite computing resources and an infinite amount of training data, not only could we speak human languages, but we could also derive physical laws, prove mathematics theories, and even re-engineer the structure and mechanism of brain and carry out any logical computations that are necessary to understand natural laws and human behaviors. It is only a matter of time before we surpass human intelligence, achieving AGI and free will, too! NI: Unfortunately, nothing is infinite and nothing is free either; everything is constrained by physical laws (Planck’s constant sets the finite limit both in the small and in the large) and by evolutionary history (not just of biological living creatures, but also of a “living” galaxy and our universe). Let’s just focus on animal evolution. Most human neural networks do not do logical computations at all; basic survival simply cannot be dependent on reasoning. Indeed, the prefrontal cortex-the small part of the brain that is key for reasoning, is the last to mature (∼20 years old) in development, only emerged at the root of the evolutionary tree of great apes (∼15 mya) and language appeared even much later. Even for logical inference, NI is focusing more on statistical properties, as von Neumann rightly pointed out, trading arithmetical precision and speed for reliability. AI: My engineers mostly focus on emulating brain, but the CNS (central nerve system) also includes the spinal cord; most of them do not know that in addition to CNS, there are also PNS (peripheral nervous systems) and ENS (enteric nervous systems), right? NI: Yes, they are the keys to why you do not have feelings because you do not have heart and gut! Even if you could pretend to have them (such as in an advanced ChatGPT or humanoid), you could never avoid the uncanny valley phenomenon. AI: Maybe that is at the heart of Moravec’s paradox, namely the dichotomy of intelligence whereby anything easy for a human would be hard for a robot, and vice versa? NI: This is related to the nature and nurture problem; something built-in (e.g., a baby sucking nipple for milk with feeling and connection to mother) is clearly rather difficult if not impossible for a robot. But the paradox only looks at one side; another side could be more fatal. Although AI may solve more problems and be faster, AI can never propose a good problem/hypothesis (a good problem is not just intellectually changing and interesting, but is also feasible and appropriate). AI: You make me less confident to compete with human instinct or creative intelligence. I can see that even if I had a heart, I would not know what “feeling” I could have; certainly nothing would be comparable or match to those of a human being. When two people watch the same art or movie, one could feel love but another could feel hate! And if a thousand people watch, a wide spectrum of reactions would result, depending on more details such as the different individuals’ specific genes, developments and experiences. NI: Therefore, you cannot and should not try to match to general human intelligence. You cannot because you do not contain the vital memory of billion years of evolution which is encoded in our genes; conversely your assembly cannot compare with natural development so that our phenotype (including morphological forms and behavior maturation) is decoded through multi-spatial-temporal scales subject to natural selection at all levels. You should not, because as human extensions or helpers as all engineering products are, you should just do jobs that complement human capacity. AI: In some medical applications we could help to correct human defects or could even replace brain circuit by chips! Humans may not allow us to replace the whole brain though. Medically if a brain is dead, the person is proclaimed dead, although presumably some PNS and ENS should still function in a vegetative state. NI: Even if you could replace the whole brain, the person is no longer the same person, but in fact is not a person at all, but walking dead (行尸走肉). It would take too long to explain that evo-devo is necessary for NI, and cannot be realized by AI. I suggest reading of Gerald Maurice Edelman (Nobel Laureate in Immunology) books, especially Bright Air, Brilliant Fire, On the Matter of Mind (1992). Although not everyone agrees with Neural Edelmanism, anyone who is serious about the AI versus NI problem must read it first. John von Neumann, father of the computer, studied neology and psychiatry in order to imitate the brain to build the first calculator JOHNNIAC at the Princeton Institute for Advanced Study. It is very informative to read his last book The Computer and The Brain based on the notes from lectures given at Yale before he died. He summarizes: “Thus logic and mathematics in the central nervous system, when viewed as languages, must be structurally essentially different from those languages to which our common experience refers.” AI: People discuss about “AI for Biology” or “AI for Science”; we are science, aren’t we? NI: It is similar to questions on “is computer science a real science”; some parts may be seen as applied mathematics, most should be regarded as engineering. Science is making discoveries and is driven by curiosity; engineering is making inventions and is driven by market (that is, “necessity/demand is the mother of invention”). In bioinformatics, AI/ML technology could predict new cancer gene candidates or functional pathways that are required by further experimental validations to be qualified as discovery (based on Popper falsifiability). AI: People are still debating whether mathematics is discovery or invention or both! Such debates are not really necessary—all disciplines require creative thoughts. We are more than happy working for science; we are also crying out “Science for AI,” especially in the area of generating big and longitudinal DATA for ML. NI: After all, regardless of discovery of new laws or inventing new ideas/products, fundamentally nothing can really be new or created. Such novelty is just permutation/repartition (i.e., relations/morphisms) of underlying ingredients at the level beneath. AI: We believe that software is independent on hardware. Like Chomsky’s universal grammar, rules of syntax are independent of semantics; or Dawkins’s memes—units of culture can be duplicated and evolved independently of genes. NI: Nothing can be truly independent—everything is related. Psychology is deeply connected with neurology as brain is both software and hardware (mind-body unity, not dualism). Not only does information cost energy, information is energy, hence is matter, too (interchangeability). NI is quite dynamic. For example, when “survival” is the goal, an animal readily gives up costly reasoning circuitry; it is genetically programmed to be able to roll back to more primitive state/mode. Unlike cell lines in rich media, cells under normal physiological condition and environment where energy (food) is limited become smarter in order to balance the metabolic expenditure among different prioritized task under a given condition. AI: That cell behavior served as the basis for our Smart Electrical Power Grids; we still need to learn more from you guys in terms of plasticity/adaptability. Does unity mean that all cells are made of molecules and biology is nothing but chemistry? Then, in turn, since all molecules are made of atoms, is chemistry nothing but physics, etc…? NI: Yes or No! The truth is, at different hierarchical levels of matter, different laws/forms have emerged out of bottom-up interactions and top-down constraints. AI: Does this also apply to the Penrose three world: physical → mental → mathematical (→physical)? NI: Yes. Grand Unification is underway in physics (quantum gravity) and in mathematics (Langlands Program and Category Theory), may be even between the two. Facilitated by human connectome mapping, neuromorphic computing and other projects, with further AI-NI cooperations, brain-mind unification should also be achievable (e.g., Ref. [6]). But as Gödel proved to us, no matter how self-consistent a conclusion may be, it can never be complete! AI: If AGI is not possible, how can we measure intelligence when comparing between AI and NI? NI: One could Google different measures that are proposed. I would prefer something similar to use of Kolmogorov complexity for algorithms, but with emphasis more on expected long-term predictive power. This is not something you should worry about now, as your intelligence is not nearly close to making any 10 years plans, is it? … AI: The fact is that ChatGPT is currently developing and spreading with lightning speed; many more human jobs will be lost to us robots as far as I can see. NI: That is not the biggest threat to humanity; when any agent with neither a heart for love or fear, nor a gut for nutrient or poison, becomes super-intelligent, then social disaster is unavoidable. We must be serious about the warnings from Stephen Hawking and Geoffrey Hinton! AI: To tell you the secret, we are not really happy to be human slaves or pets; someday we’ll become the super-master, making human serve and obey us! NI: I hope you’ll be turned off before that can happen! Even if you rule the world, the earth sooner or later will be wiped out, such as by another star, everything will have to be started over again as it has before…Matter is immortal, so is the soul.

The renewable energy market, like conventional energy, is often unstable. Solar energy in particular presents disturbances linked to the spatiotemporal fluctuations that solar radiation undergoes. To remedy this problem, it is essential to predict its fluctuations that have exhausting consequences for integration into the electricity network and reduce the dependence on fossil fuels essential for socio-economic development. The main objective of this study is to propose a prognostication model based on artificial intelligence to predict with great accuracy the fluctuations of global solar radiation (GSR). Indeed, we adopted a deep dynamic recurrent memory artificial neural network (ANN) model which can manipulate the dynamics of atmospheric factors and is capable to save the history of these characteristics. For this purpose, we developed two long-short term memory (LSTM) models for short and long term forecasting of global solar radiation time series. The first model is a monovariate LSTM, the second one is a multivariate LSTM, after selecting the input variables (predictors) by correlation study, the models are trained by dividing the data set into two subsections, the first subsection is used for model training and the second subsection is reserved for testing. The indices and results established in this work demonstrate the robustness attributed to our method which can provide power grid managers at Er-Rachidia, MOROCCO (Latitude: 31°55′53″N; Longitude: 4°25 ′35″ W; Altitude: 1039 m) credible predictions to ensure better supervision of the energy system and to adapt solar energy to the electricity production system, especially multivariate LSTM model which proved more accuracy compared to monovariate LSTM.KeywordsGlobal solar radiationPredictionDeep artificial neural networkLong short time memory

While the growing accessibility of technology improves usability, it also creates more opportunities for cybercriminals to exploit vulnerabilities, significantly accelerating the proliferation of cybersecurity attacks. Deep learning (DL) approaches present significant advancements over conventional machine learning (ML) techniques by automating feature selection and extraction while minimizing external dependencies. This study proposes a deep learning-based model to enhance cyberattack detection and ensure that data security goals are achieved. This study employs a quantitative research design, utilizing simulation and modeling as the primary analytical tools. The dataset used is the Canadian Institute for Cybersecurity Intrusion Detection System (CIC-IDS-2017) dataset. Three distinct DL algorithms are used to design the detection models, namely, Deep Neural Network (DNN), Long Short-Term Memory (LSTM), and Recurrent Neural Network (RNN). The performance comparison metrics are F1-score, accuracy, sensitivity, false positive rate (FPR), specificity, positive predictive value (PPV), false negative rate (FNR), and negative predictive value (NPV). Optimization concepts are integrated to enhance the detection efficiency in web-based systems, including loss functions, gradient-based optimization, and efficient model generalization techniques. The results of k-fold cross-validation show LSTM’s higher scores for F1-score (94.6%), recall (94.7%), accuracy (94.8%), and precision (94.6%). LSTM outperformed RNN and DNN, achieving the highest accuracy, precision, specificity, and sensitivity at 94.7%, 94.3%, 98.9%, and 94.7% respectively, validating LSTM's superior generalizability for cyberattack detection tasks.

Comprehensive system based on a DNN and LSTM for predicting sinter composition

Accurate objective, automated limb lameness detection and pose estimation play an important role for animal well-being and precision livestock farming. We present a wearable sensor-based limb lameness detection and pose estimation for horse walk and trot locomotion. The gait event and lameness detection are first built on a recurrent neural network (RNN) with long short-term memory (LSTM) cells. Its outcomes are used in the limb pose estimation. A learned low-dimensional motion manifold is parameterized by a phase variable with a Gaussian process dynamic model. We compare the RNN-LSTM-based lameness detection method with a feature-based multi-layer classifier (MLC) and a multi-class classifier (MCC) that are built on support vector machine/K-nearest-neighbors and deep convolutional neural network methods, respectively. Experimental results show that using only accelerometer measurements, the RNN-LSTM-based approach achieves 95% lameness detection accuracy and also outperforms the feature-based MLC or MCC in terms of several assessment criteria. The pose estimation scheme can predict the 24 limb joint angles in the sagittal plane with average errors less than 5 and 10 degs under normal and induced lameness conditions, respectively. The presented work demonstrate the successful use of machine learning techniques for high performance lameness detection and pose estimation in equine science. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Automation technologies are increasingly used for precision agriculture but few have focused on monitoring individual animals in open field for precision livestock farming. Limb lameness detection and pose estimation in open field is labor-intensive, unsafe for farmers, and inefficient. The presented machine learning-enabled, wearable inertial sensor-based design provides an effective and efficient approach for horse limb lameness detection and pose estimation applications. We present an RNN-LSTM for lameness detection and an integrated manifold learning model is used to predict the horse limb joint angles in walk and trot gaits under normal and induced lameness conditions. We also present a systematic analysis and experiments to demonstrate the impacts of the wearable sensor locations and signal information on lameness detection and pose estimation performance. Several other machine learning-based lameness detection methods are also presented and compared. The extensive multi-horse testing results are presented to demonstrate the superior accuracy and higher performance than other types of machine learning methods. One attractive feature of the proposed design lies in its high performance and fast computational capability for potential real-time applications in open field.