Intelligent Camera Selection Decisions for Target Tracking in a Camera Network

Not having access to compact and meaningful representations is known to significantly increase the complexity of reinforcement learning (RL). For this reason, it can be useful to perform state representation learning (SRL) before tackling RL tasks. However, obtaining a good state representation can only be done if a large diversity of transitions is observed, which can require a difficult exploration, especially if the environment is initially reward-free. To solve the problems of exploration and SRL in parallel, we propose a new approach called XSRL (eXploratory State Representation Learning). On one hand, it jointly learns compact state representations and a state transition estimator which is used to remove unexploitable information from the representations. On the other hand, it continuously trains an inverse model, and adds to the prediction error of this model a k-step learning progress bonus to form the maximization objective of a discovery policy. This results in a policy that seeks complex transitions from which the trained models can effectively learn. Our experimental results show that the approach leads to efficient exploration in challenging environments with image observations, and to state representations that significantly accelerate learning in RL tasks.

Multi-Agent Deterministic Policy Gradient (MADDPG) is a very useful algorithm in multi-agent domains. We analyze the algorithm and find that MADDPG uses deep neural networks (DNNs) as a Q function model. One advantage of DNNs is that they can build very complex processing functions to handle high-dimensional input. However, the disadvantage of this end-to-end learning is that it usually requires a lot of data, which is not always available for real-world control applications. In this paper, a new algorithm, State Representation Learning Multi-Agent Deep Deterministic Policy Gradient (SRL-MADPPG), is proposed that combines MADDPG with state representation learning which uses DNNs as a function fitting. i.e., model learning network is used to pre-train the first layer of the actor and critic networks, then the actor and critic learn from the state representation instead of the raw observations. Simulation result shows that the SRL-MADDPG algorithm improves the final performance in comparison with the end-to-end learning.

In offline reinforcement learning-based recommender systems (RLRS), learning effective state representations is crucial for capturing user preferences that directly impact long-term rewards. However, raw state representations often contain high-dimensional, noisy information and components that are not causally relevant to the reward. Additionally, missing transitions in offline data make it challenging to accurately identify features that are most relevant to user satisfaction. To address these challenges, we propose Policy-Guided Causal Representation (PGCR), a novel two-stage framework for causal feature selection and state representation learning in offline RLRS. In the first stage, we learn a causal feature selection policy that generates modified states by isolating and retaining only the causally relevant components (CRCs) while altering irrelevant components. This policy is guided by a reward function based on the Wasserstein distance, which measures the causal effect of state components on the reward and encourages the preservation of CRCs that directly influence user interests. In the second stage, we train an encoder to learn compact state representations by minimizing the mean squared error (MSE) loss between the latent representations of the original and modified states, ensuring that the representations focus on CRCs. We provide a theoretical analysis proving the identifiability of causal effects from interventions, validating the ability of PGCR to isolate critical state components for decision-making. Extensive experiments demonstrate that PGCR significantly improves recommendation performance, confirming its effectiveness for offline RL-based recommender systems.

Real-time automatic activity recognition is an important area of research in the field of Computer Vision with plenty of applications in surveillance, gaming, entertainment and automobile safety. Because of advances in wireless networks and camera technologies, distributed camera networks are becoming more prominent. Distributed camera networks offer complimentary views of scenes and hence are better suited for real-time surveillance applications. They are robust to camera failures and in-complete field of views.;In a camera network, fusing information from multiple cameras is an important problem, especially when one doesn't have knowledge of subjects orientation with respect to the camera and when arrangement of cameras is not symmetric. The objective of this dissertation is to design a information fusion technique for camera networks and to apply them in the contenxt of surveillance and safety applications (in coal-mines). (Abstract shortened by ProQuest.).

Within a camera network, the contribution of a camera to the observations of a scene depends on its viewpoint and on the scene configuration. This is a dynamic property, as the scene content is subject to change over time. An automatic selection of a subset of cameras that significantly contributes to the desired observation of a scene can be of great value for the reduction of the amount of transmitted and stored image data. We propose a greedy algorithm for camera selection in practical vision networks where the selection decision has to be taken in real time. The selection criterion is based on the information from each camera sensor's observations of persons in a scene, and only low data rate information is required to be sent over wireless channels since the image frames are first locally processed by each sensor node before transmission. Experimental results show that the performance of the proposed greedy algorithm is close to the performance of the optimal selection algorithm. In addition, we propose communication protocols for such camera networks, and through experiments, we show the proposed protocols improve latency and observation frequency without deteriorating the performance.

In Internet Protocol Television (IPTV) systems, Quality-of-Service (QoS) is a critical factor for user satisfaction. In this paper, we first propose a queueing model for IPTV systems and discuss how to ensure the QoS of IPTV. We then use the model to analyze the effects of IPTV traffic on other applications in home networks. We find that TCP congestion control may not work well under this circumstance and propose a new approach to improve the performance of TCP. We also verify our results throughns2simulations.

The learning of dialogue strategies in spoken dialogue systems using reinforcement learning is a promising approach to acquire robust dialogue strategies. However, the trade-off between available dialogue data and information in the dialogue state either forces information to be excluded from the state representations or requires large amount of training data. In this paper, we propose to use dynamic state aggregation to efficiently learn dialogue policies using less data. State aggregation reduces the size of the problem to be solved. Experimental results show that the proposed method converges faster and that in case of data sparseness, the proposed method is less sensitive to atypical training examples.

Wireless multimedia sensor networks (WMSNs) are interconnected devices that allow retrieving video and audio streams, still images, and scalar data from the environment. In a densely deployed WMSN, there exists correlation among the visual information observed by cameras with overlapped field of views. In this paper, the correlation characteristics of visual information are used to address two issues: 1) how to measure the amount of visual information provided by multiple cameras in the network, and 2) how to select a group of cameras to report their information to the sink under distortion constraints. An entropy-based analytical framework is developed to measure the amount of visual information provided by multiple correlated cameras first. Based on this framework, a correlation-based camera selection scheme is introduced. Simulation results show that, given a distortion bound at the sink, the correlation-based selection scheme requires fewer cameras to report to the sink than the random selection scheme.

Multiple target tracking is an important problem in analysing video data in camera networks. Distributed processing is a promising scheme to deal with huge volume of video data in camera networks. This paper addresses the problem of distributed multiple target tracking in camera networks. Each camera shares measurements with its immediate neighbours and performs inter-camera measurement-to-measurement association in distributed manner. The measurements are assigned to the targets using the nearest neighbourhood principle. To update the target state we use probabilistic data association with sigma point information filters. This filter is integrated with a consensus algorithm to develop distributed multi target tracking algorithm. We evaluated the proposed algorithm on various real world datasets and show that our algorithm outperforms the other related state-of-art distributed algorithms.

In computer science, reinforcement learning is a powerful framework with which artificial agents can learn to maximize their performance for any given Markov decision process (MDP). Advances over the last decade, in combination with deep neural networks, have enjoyed performance advantages over humans in many difficult task settings. However, such frameworks perform far less favorably when evaluated in their ability to generalize or transfer representations across different tasks. Existing algorithms that facilitate transfer typically are limited to cases in which the transition function or the optimal policy is portable to new contexts, but achieving “deep transfer” characteristic of human behavior has been elusive. Such transfer typically requires discovery of abstractions that permit analogical reuse of previously learned representations to superficially distinct tasks. Here, we demonstrate that abstractions that minimize error in predictions of reward outcomes generalize across tasks with different transition and reward functions. Such reward-predictive representations compress the state space of a task into a lower dimensional representation by combining states that are equivalent in terms of both the transition and reward functions. Because only state equivalences are considered, the resulting state representation is not tied to the transition and reward functions themselves and thus generalizes across tasks with different reward and transition functions. These results contrast with those using abstractions that myopically maximize reward in any given MDP and motivate further experiments in humans and animals to investigate if neural and cognitive systems involved in state representation perform abstractions that facilitate such equivalence relations.

A comprehensive, intuitive, task-oriented three-dimensional coverage model for cameras and multi-camera networks using fuzzy sets is presented. The model captures the vagueness inherent in the concept of visual coverage. At present, the model can be used to evaluate, given a scene model and an objective, the coverage performance of a given camera or multi-camera network configuration, as a single numerical metric. Plans to use the model for optimal camera placement and other problems involving coverage are discussed. Examples of qualitative experimental validation of the coverage model are presented.

This paper introduces a simple method to achieve full-field and real-scale reconstruction using a movable binocular vision system (MBVS). The MBVS is composed of two cameras, one is called the tracking camera, and the other is called the working camera. The tracking camera is used for tracking the positions of the MBVS and the working camera is used for the 3D reconstruction task. The MBVS has several advantages compared with a single moving camera or multi-camera networks. Firstly, the MBVS could recover the real-scale-depth-information from the captured image sequences without using auxiliary objects whose geometry or motion should be precisely known. Secondly, the removability of the system could guarantee appropriate baselines to supply more robust point correspondences. Additionally, using one camera could avoid the drawback which exists in multi-camera networks, that the variability of a cameras’ parameters and performance could significantly affect the accuracy and robustness of the feature extraction and stereo matching methods. The proposed framework consists of local reconstruction and initial pose estimation of the MBVS based on transferable features, followed by overall optimization and accurate integration of multi-view 3D reconstruction data. The whole process requires no information other than the input images. The framework has been verified with real data, and very good results have been obtained.

With the widespread application of automatic fire alarm systems in various types of buildings, the problem of fire false alarms has gradually become prominent, which not only causes resource waste, but also may reduce users’ trust in the alarm system, thereby affecting the efficiency of emergency response in actual fires. According to data from a certain fire cloud platform, 99.85% of the suspected fires predicted by its system are false alarms. Although existing models can recognize most fire accidents, the accuracy of fire alarm recognition is only 0.15%, due to loose judgment logic, which still requires a large amount of manpower to verify alarms. This article analyzes a large amount of false alarm data and explores the main causes of false alarms, including environmental interference, equipment failure, and improper human operation. By using a fire dynamics simulator (FDS) to establish fire simulation models under different data settings, horizontal and vertical multi-scene fire simulation data are obtained. The study combines simulation and platform data to form a fire and false alarm dataset using a one-dimensional convolutional neural network (1D-CNN) and deep neural network (DNN) deep learning techniques to learn the deductive rules of the fire scene, establish a two-stage judgment model, and gradually, accurately, judge the results. By quantifying the precision, recall, and F1 score of the model, a deep learning model designed to accurately identify genuine fire alarms while filtering out false ones is proposed that can significantly reduce the false alarm rate. The results indicate that the model can identify 1705 false alarms out of 2255 highly suspected true alarms identified by existing systems in multiple practical scenarios and eliminate 75.61% of false positive alarms. On the premise of ensuring an authenticity recognition rate greater than 98%, the accuracy of fire alarm recognition increased from 0.15% to 28.85%, which will significantly reduce the workload of staff verifying alerts, and has good practical value.

Autonomous multisensor tracking surveillance system

With the advancement of technology and the increasing use of wearable devices, information security have become a necessity. Although many biometrics authentication methods have been studied on these devices to ensure information security, an activity-aware deep learning (DL) model that is compatible with different device types and uses only electrocardiogram signals has not been studied. Our objective is to investigate DL models that exclusively use ECG signals during several physical activities, facilitating their implementation on various devices. Through this research, we aim to contribute to the advancement of wearable devices for the purpose of biometric verification. In this context, this study investigates the application of adaptive techniques that rely on prior activity classification to potentially improve biometric performance using DL models. In this study, we compare three time-frequency representations to generate images for activity classification using GoogleNet, ResNet50 and DenseNet201, and for biometric verification using ResNet50 and DenseNet201. Despite employing various convolutional neural network (CNN) models, we could not achieve high accuracy in activity classification. Consequently, manually classified samples were used for activity-aware biometric verification. We also provide a detailed comparison of various DL parameters. We use a public dataset simultaneously collected from both medical and wearable devices to offer a cross-device comparison. The results demonstrate that our method can be applied to both wearable and medical devices for activity classification and biometric verification. Besides, although it is known that DL requires a large amount of training data, our model, which was created using a small amount of training data and a real-life biometric verification scenario, achieved comparable results to studies using a large amount of data. The model was achieved 0.16% to 30.48% better results when classified according to their physical activities.