Simultaneous probabilistic localization and learning: A new algorithm for online learning

Online learning is a machine learning paradigm that is useful when data is not available all at once. In this paper we focus in real time applications for which data is being collected as the algorithm executes. The forgetron Algorithm [1] is an online learning algorithm that works under a limited memory constraint while guaranteeing a bound on the number of total mistakes. We have proposed a specific architecture for the forgetron algorithm using hardware/software based design in order to improve computation time in the training process. Experiments on real world data show the advantages of this implementation compared to an exclusive software implementation using a Xilinx Virtex II Pro FPGA with an embedded Power PC. These experiments validate the performance of the proposed architecture.

Understanding the visual quality of a feature map plays a significant role in many active vision applications. Previous works mostly rely on object-level features, such as compactness, to estimate the quality score of a feature map. However, the compactness is leveraged on feature maps produced by salient object detection techniques where the maps tend to be compact. As a result, the compactness feature fails when the feature maps are blurry (e.g., fixation maps). In this paper, we regard the process of estimating the quality score of feature maps, specifically fixation maps, as a regression problem. After extracting several local, global, geometric, and positional characteristic features from a feature map, a model is learned using a random forest regressor to estimate the quality score of any unseen feature map. Our model is specifically tailored to estimate the quality of three types of maps: bottom-up, target, and contextual feature maps. These maps are produced for a large benchmark fixation data set of more than 900 challenging outdoor images. We demonstrate that our approach provides an accurate estimate of the quality of the abovementioned feature maps compared to the groundtruth data. In addition, we show that our proposed approach is useful in feature map integration for predicting human fixation. Instead of naively integrating all three feature maps when predicting human fixation, our proposed approach dynamically selects the best feature map with the highest estimated quality score on an individual image basis, thereby improving the fixation prediction accuracy.

Kalman filtering and online learning are two approaches to estimate the state of a system in the presence of inaccurate (e.g., noisy) measurements. While many online learning algorithms are model-free and data-driven, two recently developed online learning algorithms, Dynamic Mirror Descent (DMD) and Dynamic Fixed Share (DFS), incorporate dynamic models, similarly to Kalman filtering algorithms. Our previous work showed that DMD can be constructed to produce state estimates that are identical to those produced by a discrete-time Kalman filter. This work extends our previous work by exploring connections between a multiple model Kalman filter (MMKF) and DFS, which both incorporate a set of candidate models to address situations in which the underlying model is unknown. We show that the functions/parameters used within DFS can be constructed to produce the same estimates as a MMKF. We then modify DFS to include several heuristics that are used to improve the performance of a MMKF in order to assess whether they can also be used to improve the performance of DFS. Finally, we investigate the performance of the algorithms and their variations in a simulation study. Specifically, we seek to estimate the time-varying power consumption of an aggregation of electric loads, which could be used as the feedback signal within a demand response algorithm. The simulation results empirically show that DFS implementations generally perform better than comparable MMKF implementations since we are able to tune the functions/parameters used within DFS.

This paper presents results for an approach for indoor location estimation that integrates received signal strength (RSS) data from both WiFi and GSM networks. Previous work has focused on relatively small indoor environments. In many potential applications, getting approximate location information, such as in which room the mobile user is, is adequate. A hierarchical clustering method is used to partition the RSS space. To choose the best transmitters in a partition, we assess the amount of RSS variance that is attributable to different base stations (BSs) or access points (APs) by transforming the RSS tuples into principal components (PCs). This allows us to retain most of the useful information of detectable transmitters in fewer dimensions. In our experiments, we collected WiFi and cellular RSS on the 2nd and 3rd-floor electronic engineering (EE) building in Queen Mary campus. The experiment results show that the proposed method can provide a good accuracy of room prediction, especially when we integrate WiFi RSS with GSM RSS together to do the positioning.

Article Online Learning of Bayesian Classifiers with Nonstationary Data Streams Peng Wu 1,2,*, and Ning Xiong 1 1 School of Innovation, Design and Engineering Mälardalen University, Västerås 721 23, Sweden 2 Department of Computer Engineering, Taiyuan Institute of Technology, Taiyuan 030008, China * Correspondence: 14112078@bjtu.edu.cn Received: 31 March 2023 Accepted: 13 July 2023 Published: 26 September 2023 Abstract: The advancement in Internet of things and sensor technologies has enabled data to be continuously generated with a high rate, i.e., data streams. It is practically infeasible to store streaming data in a hard disk, and apply a traditional batch learning method to extract a relevant knowledge model from these data. This paper studies online incremental learning with data streams, in which one sample is processed at each time to update the existing model. For the learning target, the Bayesian classifier is adopted which is a computationally economical model of easy deployment for online processing in edges or devices. By using the individual new example, we first present an online learning algorithm to incrementally update classifier parameters in a way equivalent to the offline learning counterpart. In order to adapt to concept drifts in nonstationary environments, the proposed online learning algorithm is improved to enable recent examples to be more impactful during the sequential learning procedure. Preliminary simulation tests reveal that the improved online learning algorithm can lead to faster model adaption than the unimproved online algorithm when the data drift occurs. In case of presumed stationary data streams without drifts, the improved online algorithm is proved to be competent by performing at least as good as (sometimes, even better than) the unimproved algorithm.

Detecting cyberattacks in industrial control systems using online learning algorithms

The convergence of online learning algorithms is analyzed using the tools of the stochastic approximation theory, and proved under very weak conditions. A general framework for online learning algorithms is first presented. This framework encompasses the most common online learning algorithms in use today, as illustrated by several examples. The stochastic approximation theory then provides general results describing the convergence of all these learning algorithms at once.

Phase change memory (PCM), one of the most mature emerging non-volatile memories, has gained considerable attention over the years for use as electronic synapses in biologically inspired neuromorphic systems. The resistance drift of PCM devices, nonetheless, has long been identified as one of the biggest challenges toward realizing many areas of applications. Although this drawback has been extensively studied for memory development and many methods were proposed to mitigate the drift effect, its impact, if any, on online learning has not been fully explored yet. In this letter, we investigate the impact of resistance drift and variations in resistance drift parameters during unsupervised online learning. We use the resistance drift characteristics measured from experiments and incorporate them into the spiking neural network (SNN) for MNIST handwritten digits classification. Our results show that resistance drift, considered as a non-ideality for PCM devices, can be exploited to boost accuracy for online learning of handwritten digits in the SNN.

Due to the effective guidance of prior information, feature map-based pruning methods have emerged as promising techniques for model compression. In the previous works, the undifferentiated treatment of all information on feature maps amplifies the negative impact of noise and background information. To address this issue, a novel filter pruning strategy called Filter Pruning via Attention Consistency (FPAC) is proposed, and a simple and effective implementation method of FPAC is presented. FPAC is inspired by the notion that the attention of feature maps on one layer is in high consistency of spatial dimension. The experiments also show that feature maps with lower consistency are less important. Hence, FPAC measures the importance of filters by evaluating the attention consistency on the feature maps and then prunes the filters corresponding to feature maps with lower consistency. The present experiments on various datasets further confirm the effectiveness of FPAC. For instance, applying VGG-16 on CIFAR-10, the classification accuracy even increases from 93.96% to 94.03% with 58.1% FLOPs reductions. Furthermore, applying ResNet-50 on ImageNet achieves 45% FLOPs reductions with only 0.53% accuracy loss.

Despite the importance of mutual adaption in human relationships, online learning is not yet used during most successful human-robot interactions. The lack of online learning in HRI to date can be attributed to at least two unsolved challenges: random exploration (a core component of most online-learning algorithms) and the slow convergence rates of previous online-learning algorithms. However, several recently developed online-learning algorithms have been reported to learn at much faster rates than before, which makes them candidates for use in human-robot interactions. In this paper, we explore the ability of these algorithms to learn to interact with people. Via user study, we show that these algorithms alone do not consistently learn to collaborate with human partners. Similarly, we observe that humans fail to consistently collaborate with each other in the absence of explicit communication. However, we demonstrate that one algorithm does learn to effectively collaborate with people when paired with a novel cheap-talk communication system. In addition to this technical achievement, this work highlights the need to address AI and HRI synergistically rather than independently.

An on-line unsupervised learning mechanism is proposed for unlabeled data which is polluted by noises. By using a similarity threshold and local error based insertion criterion, the system is able to grow incrementally and to accommodate input patterns of on-line non-stationary data distribution. The definition of a utility parameter - "error-radius" enables this system to learn the number of nodes needed to solve a current task. The usage of a new technique for removing nodes in low probability density regions can separate the clusters with low-density overlaps and dynamically eliminate the noise in the input data. The design of two-layer neural network makes it possible for this system to represent the topological structure of unsupervised on-line data, report the reasonable number of clusters and give typical prototype patterns of every cluster without any priori conditions such as suitable number of nodes or a good initial codebook.

Advances in miniaturized wireless and sensing technologies have enabled the construction of cheap, low-powered, portable wireless devices capable of forming ad hoc networks. While these networks have shown enormous potential in applications such as remote sensing and target tracking, these applications require the devices to determine their own location. Additionally, devices capable of self-localization can also be used to implement location-based services or to improve coordination between first-responders to disaster sites or infantry in tactical situations. Existing techniques such as GPS may not be available due to design or environmental constraints, so other methods need to be devised. Previous works have proposed methods for wireless devices to self-localize based on received signal strength (RSS), but these methods offer limited accuracy due to the large error in RSS measurements. Recognizing the trend for these portable wireless devices to contain acceleration sensors, we propose an algorithm to combine these acceleration measurements with RSS readings to achieve accurate localization. We apply a distributed extended Kalman filter to track position based on these two measurements and a kinematic node movement model. This algorithm is able to take advantage of correlations between successive location estimates to improve estimation accuracy. We calculate the posterior Cramer-Rao bound for this algorithm and analyze it through simulation. Our analysis shows that by utilizing the acceleration information, the network is able to self-localize despite the large inaccuracy in RSS readings.

Monitoring a large population of dynamic processes with limited resources presents a significant challenge across various industrial sectors. This is due to 1) the inherent disparity between the available monitoring resources and the extensive number of processes to be monitored and 2) the unpredictable and heterogeneous dynamics inherent in the progression of these processes. Online learning approaches, commonly referred to as bandit methods, have demonstrated notable potential in addressing this issue by dynamically allocating resources and effectively balancing the exploitation of high-reward processes and the exploration of uncertain ones. However, most online learning algorithms are designed for 1) a centralized setting that requires data sharing across processes for accurate predictions or 2) a homogeneity assumption that estimates a single global model from decentralized data. To overcome these limitations and enable online learning in a heterogeneous population under a decentralized setting, we propose a federated collaborative online monitoring method. Our approach utilizes representation learning to capture the latent representative models within the population and introduces a novel federated collaborative UCB algorithm to estimate these models from sequentially observed decentralized data. This strategy facilitates informed monitoring of resource allocation. The efficacy of our method is demonstrated through theoretical analysis, simulation studies, and its application to decentralized cognitive degradation monitoring in Alzheimer’s disease.

This study addresses the challenge of extracting valuable information and selecting key variables from large datasets, essential across statistics, computational science, and data science. In the age of big data, where safeguarding personal privacy is paramount, this study presents an online learning algorithm that leverages differential privacy to handle large‐scale data effectively. The focus is on enhancing the online group lasso approach within the differential privacy realm. The study begins by comparing online and offline learning approaches and classifying common online learning techniques. It proceeds to elucidate the concept of differential privacy and its importance. By enhancing the group‐follow‐the‐proximally‐regularized‐leader (GFTPRL) algorithm, we have created a new method for the online group lasso model that integrates differential privacy for binary classification in logistic regression. The research offers a solid validation of the algorithm’s effectiveness based on differential privacy and online learning principles. The algorithm’s performance was thoroughly evaluated through simulations with both synthetic and actual data. The comparison is made between the proposed privacy‐preserving algorithm and traditional non‐privacy‐preserving counterparts, with a focus on regret bounds, a measure of performance. The findings underscore the practical benefits of the differential privacy‐preserving algorithm in tackling large‐scale data analysis while upholding privacy standards. This research marks a significant step forward in the fusion of big data analytics and the safeguarding of individual privacy.

Over the last couple of decades the demand for high precision and enhanced performance of physical systems has been steadily increasing. This demand often results in miniaturization and complex design, thus increasing the need for complex nonlinear control methods. Some of the state of the art nonlinear methods are stymied by the requirement of full state information, model and parameter uncertainties, mathematical complexity, etc. For many scenarios it is nearly impossible to consider all the uncertainties during the design of a feedback controller. Additionally, while designing a modelbased nonlinear control there is no standard mechanism to incorporate performance measures. Some of the mentioned issues can be addressed by using online learning. Animals and humans have the ability to share, explore, act or respond, memorize the outcome and repeat the task to achieve a better outcome when they encounter the same or a similar scenario. This is called learning from interaction. One instance of this approach is reinforcement learning (RL). However, RL methods are hindered by the curse of dimensionality, non-interpretability and non-monotonic convergence of the learning algorithms. This can be attributed to the intrinsic characteristics of RL, as it is a modelfree approach and hence no standard mechanism exists to incorporate a priori model information. In this thesis, learning methods are proposed which explicitly use the available system knowledge. This can be seen as a new class of approaches that bridge model-based and model-free methods. These methods can address some of the hurdles mentioned earlier. For example, i) a prior system information can speed up the learning, ii) new control objectives can be achieved which otherwise would be extremely difficult to attain using only model-based methods, iii) physical meaning can be attributed to the learned controller. The developed approach is as follows: themodel of the given physical system is represented in the port-Hamiltonian (PH) form. For the system dynamics in PH form a passivity-based control (PBC) law is formulated, which often requires the solution to a set of partial differential equations (PDEs). Instead of finding an analytical solution, the PBC control law is parameterized using an unknown parameter vector. Then, by using a variation of the standard actor-critic learning algorithm, the unknown parameters can be learned online. Using the principles of stochastic approximation theory, a proof of convergence for the developed method is shown. The proposedmethods are evaluated for the stabilization and regulation ofmechanical and electro-mechanical systems. The simulation and experimental results show comparable learning curves. In the final part of the thesis a novel integral reinforcement learning approach is developed to solve for the optimal output tracking control problem for a set of linear heterogeneous multi-agent systems. Unlike existing methods, this approach does not need to solve either the output regulator equation or requires a p-copy of the leader’s dynamics in the agent’s control law. A detailed numerical evaluation has been conducted to show the feasibility of the developed method.