Convergence analysis of efficient online learning in Bayesian spiking neurons

Bayesian spiking neurons (BSNs) provide a probabilistic interpretation of how neurons perform inference and learning. Online learning in BSNs typically involves parameter estimation based on maximum-likelihood expectation-maximization (ML-EM) which is computationally slow and limits the potential of studying networks of BSNs. An online learning algorithm, fast learning (FL), is presented that is more computationally efficient than the benchmark ML-EM for a fixed number of time steps as the number of inputs to a BSN increases (e.g., 16.5times faster run times for 20 inputs). Although ML-EM appears to converge 2.0 to 3.6times faster than FL, the computational cost of ML-EM means that ML-EM takes longer to simulate to convergence than FL. FL also provides reasonable convergence performance that is robust to initialization of parameter estimates that are far from the true parameter values. However, parameter estimation depends on the range of true parameter values. Nevertheless, for a physiologically meaningful range of parameter values, FL gives very good average estimation accuracy, despite its approximate nature. The FL algorithm therefore provides an efficient tool, complementary to ML-EM, for exploring BSN networks in more detail in order to better understand their biological relevance. Moreover, the simplicity of the FL algorithm means it can be easily implemented in neuromorphic VLSI such that one can take advantage of the energy-efficient spike coding of BSNs.

A fast learning algorithm is proposed for training of multilayer feedforward neural networks, based on a combination of optimal linear Kalman filtering theory and error propagation. In this algorithm, all the information available from the start of the training process to the current training sample is exploited in the update procedure for the weight vector of each neuron in the network in an efficient parallel recursive method. This innovation is a massively parallel implementation and has better convergence properties than the conventional backpropagation learning technique. Its performance is illustrated on some examples, such as a XOR logical operation and a nonlinear mapping of two continuous signals. >

The paper proposes a fast online learning method for neural network structures by using genetic algorithm (GA) and dynamic back propagation algorithm (BP) jointly. GA is used in the coarse tuning process which adjusts interconnection weights of the neural network. The dynamic back propagation algorithm is subsequently applied to achieve fine adjusting of the network weights. The fitness function, based on the squared error between the teaching signal and the network output value, is redefined at every time step and the proposed GA based algorithm solves a nonstationary function optimization task. At every time step the solution with the best fitness function is used for current representation of the neural network weights and biases. It is shown through the simulations and real time temperature control of drying oven that this learning algorithm has faster convergence ability and better performance on reducing mapping error in the online learning neural network structures. This leads to an improvement of the transient response of neuro adaptive systems. The proposed method has the potential to be applied to many practical areas such as system modeling and control, signal processing and pattern recognition.

The iterative maximum-likelihood expectation-maximization (ML-EM) algorithm is an excellent algorithm for image reconstruction and usually provides better images than the filtered backprojection (FBP) algorithm. However, a windowed FBP algorithm can outperform the ML-EM in certain occasions, when the least-squared difference from the true image, that is, the least-squared error (LSE), is used as the comparison criterion. Computer simulations were carried out for the two algorithms. For a given data set the best reconstruction (compared to the true image) from each algorithm was first obtained, and the two reconstructions are compared. The stopping iteration number of the ML-EM algorithm and the parameters of the windowed FBP algorithm were determined, so that they produced an image that was closest to the true image. However, to use the LSE criterion to compare algorithms, one must know the true image. How to select the optimal parameters when the true image is unknown is a practical open problem. For noisy Poisson projections, computer simulation results indicate that the ML-EM images are better than the regular FBP images, and the windowed FBP algorithm images are better than the ML-EM images. For the noiseless projections, the FBP algorithms outperform the ML-EM algorithm. The computer simulations reveal that the windowed FBP algorithm can provide a reconstruction that is closer to the true image than the ML-EM algorithm.

This paper proposes the implementation of on-line learning neural controllers in the autopilot control laws of a modern high-performance aircraft. A first advantage of this design philosophy consists in avoiding the precomputation, storing, and interpolation between thousands of feedback gains of a typical flight control system. Another advantage is the ability to compensate for nonlinearities and model uncertainties. In addition, an on-line learning time-varying neural architecture will avoid the time-consuming gain recalculation following any modification to the aircraft or to its control system during its operative life. The implementation of these types of alternative controllers is made possible by the recent simultaneous advances in neural-network technology with regard to the availability of efficient and fast learning algorithms, along with progress in digital microprocessors. The approach is shown using a six-degree-of-freedom nonlinear simulation code. The traditional gain-scheduling-based control laws for typical autopilot functions are replaced by on-line learning neural architectures trained with the extended back-propagation algorithm. This algorithm has shown substantial improvements over the conventional back-propagation method in learning speed and accuracy. The results of the simulation with and without a man in the loop are presented and discussed.

The empty nest syndrome has become a significant problem in many countries. Caregivers can help the elderly to cope with some of the issues related to this syndrome, but a tremendous shortage of caregivers is expected due to the low income and high workload. The growing popularity of mobile device use by the elderly has created a great potential for mobile virtual companions. Studies have shown that virtual companions that can show empathy towards its users are perceived as more caring, likeable, trustworthy and supportive. Therefore, we believe that an empathetic companion can help to improve the elderly’s emotional wellness and promote an active independent lifestyle. In this work, we propose an empathetic elderly companion based on the psychological theory of empathy in the form of a smart chat bot. The empathetic companion records the elderly’s speech during their conversations, extracts phonetic features, and employs a fast online learning algorithm, namely Online Sequential Extreme Learning Machine (OS-ELM), to recognize emotion. Based on the elderly’s current emotion, the companion will provide context-aware empathetic responses. The experimental results show that OS-ELM is superior in classifying emotions from phonetic information.

Three-dimensional (3D) iterative maximum likelihood expectation maximization (ML-EM) algorithms for single-photon emission tomography (SPET) are capable of correcting image-degrading effects of non-uniform attenuation, distance-dependent camera response and patient shape-dependent scatter. However, the resulting improvements in quantitation, resolution and signal-to-noise ratio (SNR) are obtained at the cost of a huge computational burden. This paper presents a new acceleration method for ML-EM: dual matrix ordered subsets (DM-OS). DM-OS combines two acceleration methods: (a) different matrices for projection and back-projection and (b) ordered subsets of projections. DM-OS was compared with ML-EM on simulated data and on physical thorax phantom data, for both 180 degrees and 360 degrees orbits. Contrast, normalized standard deviation and mean squared error were calculated for the digital phantom experiment. DM-OS resulted in similar image quality to ML-EM, even for speed-up factors of 200 compared to ML-EM in the case of 120 projections. The thorax phantom data could be reconstructed 50 times faster (60 projections) using DM-OS with preservation of image quality. ML-EM and DM-OS with scatter compensation showed significant improvement of SNR compared to ML-EM without scatter compensation. Furthermore, inclusion of complex image formation models in the computer code is simplified in the case of DM-OS. It is thus shown that DM-OS is a fast and relatively simple algorithm for 3D iterative scatter compensation, with similar results to conventional ML-EM, for both 180 degrees and 360 degrees acquired data.

We investigated the spatial resolution and quantitative properties of SPECT images reconstructed using filteredbackprojection (FB) and maximum likelihood-expectation maximization (ML-EM) algorithms. We studied ML-EM as a function of iteration number and projectorhackprojector (php) model, and FE3 as a function of filter. To quantify the image resolution for a shift-variant process we defined a local point response function (PRF). To justify the use of the local PRF to quantify resolution for nonlinear ML-EM we tested the regime over which the algorithm behaved linearly with respect to input perturbations. Using Fourier domain analogs of the local PRF we demonstrated that ML-EM with an accurate php can lead to images of better resolution than FB with a Butterworth or Metz filter or ML-EM with a php that does not fully model the imaging process. We also showed that the resolution improves with increasing MLEM iteration number, and the extent of the improvement depends on the accuracy of the php model. We conclude that the local PRF can be used to characterize spatial resolution in SPECT images, and that ML-EM offers the possibility of superior resolution, particularly if the algorithm contains a good model of the imaging process.

A statistical method called maximum likelihood expectation maximization (MLEM) is quite attractive, especially in PET/SPECT. However, the convergence rate of the iterative scheme of MLEM is quite slow. This study aims to develop and test a new method to speed up the convergence rate of the MLEM algorithm. We introduce a relaxation parameter in the conventional MLEM iterative formula and propose the relaxation strategy on the condition that the spectral radius of the derived iterative matrix from the iterative scheme with the accelerated parameter reaches a minimum value. Experiments with Shepp-Logan phantom and an annual tree image demonstrate that the new computational strategy effectively accelerates computation time while maintains reasonable image quality. The proposed new computational method involving the relaxation strategy has a faster convergence speed than the original method.

Most of the intrusion detection models (IDM) are constructed with off-line training data. Time-variance characteristic of the practical network system cannot be embodied in the off-line constructed IDM. On-line updating of the off-line IDM with the valued new samples is very necessary. In this paper, a new on-line instruction detection model based on approximate linear dependent (ALD) condition with linear latent feature extraction is proposed to address this problem. Specifically, the valued samples which can represent drift of the practical network are indentified with ALD and prior knowledge. Then, these selected samples are used to update the off-line IDM based on on-line latent feature extraction method and fast machine learning algorithm with sample-based updating strategy. Experiments based on KDD99 data are used to validate the proposed approach.

We have seen in the past decade a sharp increase in the extent that companies use data to optimize their businesses. Variously called the `Big Data' or `Data Science' revolution, this has been characterized by massive amounts of data, including unstructured and nontraditional data like text and images, and the use of fast and flexible Machine Learning (ML) algorithms in analysis. With recent improvements in Deep Neural Networks (DNNs) and related methods, application of high-performance ML algorithms has become more automatic and robust to different data scenarios. That has led to the rapid rise of an Artificial Intelligence (AI) that works by combining many ML algorithms together – each targeting a straightforward prediction task – to solve complex problems. We will define a framework for thinking about the ingredients of this new ML-driven AI. Having an understanding of the pieces that make up these systems and how they fit together is important for those who will be building businesses around this technology. Those studying the economics of AI can use these definitions to remove ambiguity from the conversation on AI's projected productivity impacts and data requirements. Finally, this framework should help clarify the role for AI in the practice of modern business analytics and economic measurement.

A fast and flexible algorithm for microstructure reconstruction combining simulated annealing and deep learning

The maximum likelihood expectation maximisation (ML-EM) algorithm can be used to reduce reconstruction artefacts produced by filtered backprojection (FBP) methods in pinhole single-photon emission tomography (SPET). However, ML-EM suffers from noise propagation along iterations, which leads to quantitatively unpleasant reconstruction results. To avoid this increase in noise, the median root prior (MRP) algorithm for pinhole SPET was implemented. Projection data of a line source and Picker's thyroid phantom were collected using a single-head gamma camera with a pinhole collimator. MRP was added to existing pinhole ML-EM reconstruction algorithm and the phantom studies were reconstructed using MRP, ML-EM and FBP for comparison. Coefficients of variation, contrasts and full-widths at half-maximum were calculated and showed a clear reduction in noise without significant loss of resolution or decrease in contrast when MRP was applied. MRP also produced visually pleasing images even with high iteration numbers, free of the checkerboard-type noise patterns which are typical of ML-EM images.

The rapid development of online learning networks has resulted in the widespread use of recorded educational contents. While the community structure of those networks may have an influence on the use of contents, research on detecting online learning communities and investigating their structures using social network analysis (SNA) methods is scarce. The purpose of the research presented here is to investigate the structure of online learning networks and their users’ engagement patterns. In this study, Khan Academy, a widely used video learning repository, will be used as a case. Community detection algorithms are used to detect the development of online learning communities and network performance and effectiveness measures are applied to assess the network structure, effectiveness, and efficiency of a large dataset consisting of 359,163 users that interacted with Khan Academy's videos with over 3M questions and answers. The results demonstrate that different community detection algorithms can be implemented on learning networks and produce good learning communities which are not necessarily related to a domain or a topic. Measures such as density can be used to measure social presence while centrality measures are used to define central users and hubs in the communities. This study complements previous research that shed the light on the power and potential of SNA measures to structurally evaluate and detect online learning communities.

An agent must learn and adapt quickly when playing against other agents. This process is challenging in particular when playing in stochastic environments against other learning agents. In this paper, we introduce a fast and adaptive learning algorithm for repeated stochastic games (FAL-SG). FAL-SG utilizes lossy game abstraction to reduce the state space of the game and facilitate learning and adapting rapidly. We analyze FAL-SG's performance by proving bounds on the abstraction loss and prediction mistakes and show that FAL-SG satisfies three criteria prescribed for multiagent learning algorithms. We successfully establish the robustness and scalability of FAL-SG with extensive theoretical and experimental results.