Occlusion handling framework for tracking in smart camera networks by per-target assistance task assignment

Smart Camera Network (SCN) mainly has different applications based on the surveillance, security and safety with limited energy, power and memory. Normally cameras operate under limited power resources. To extend the lifetime of smart camera, it is required to reduce the energy consumption in an effective way. This paper mainly presents adaptive energy consumption of smart camera network, by assigning distributed and adaptive utility function for vision tasks. The switching of vision tasks between active, idle and sleep modes takes place based on the frequency and duty cycle. Switching of tasks is mainly done on the basis of availability of energy resources. Algorithm for Multitask allocation in smart camera network is assigned in such a way that results in the reduction of energy consumption without minute changes in the performance of the target tracing and representation. Algorithms that are assigned for each task must be effective and the weight is varied in accordance with the threshold values for energy resources. This paper represents, how a little change in the task allotment, algorithm and energy resources for smart camera results a major role in the grown of implementation of vision task and life time.

Smart cameras integrate sensing, processing, and communication on embedded platforms and serve as key components at the sensory edge of many Internet of Things (IoT) and cyber-physical systems (CPS) applications. Although impressive technological progress has been achieved over the last decade, smart cameras still face resource limitations, particularly when used for advanced networked data processing. This position paper briefly summarizes trends in pervasive smart cameras, presents general strategies for improving resource efficiency, and discusses the resource challenges of person re-identification; a prototypical example of surveillance applications for smart cameras based on advanced machine learning techniques. Lastly, we present our concept for resource-efficient decentralized person re-identification in smart camera networks as a work in progress.

Distributed data association in smart camera network via dual decomposition

Smart cameras allow pre-processing of video data on the camera instead of sending it to a remote server for further analysis. Having a network of smart cameras allows various vision tasks to be processed in a distributed fashion. While cameras may have different tasks, we concentrate on distributed tracking in smart camera networks. This application introduces various highly interesting problems. Firstly, how can conflicting goals be satisfied such as cameras in the network try to track objects while also trying to keep communication overhead low? Secondly, how can cameras in the network self adapt in response to the behavior of objects and changes in scenarios, to ensure continued efficient performance? Thirdly, how can cameras organise themselves to improve the overall network's performance and efficiency? This paper presents a simulation environment, called CamSim, allowing distributed self-adaptation and self-organisation algorithms to be tested, without setting up a physical smart camera network. The simulation tool is written in Java and hence allows high portability between different operating systems. Relaxing various problems of computer vision and network communication enables a focus on implementing and testing new self-adaptation and self-organisation algorithms for cameras to use.

This paper presents an innovative concept for a distributed system that combines a smart camera network with semantic reasoning. The proposed system is context sensitive and combines the information extracted by the smart camera with logic rules and knowledge of what the camera observes, building information and events that may occurred. The proposed system is a justification for the use of smart cameras, and it can improve the classical visual sensor networks (VSN) and enhance the standard computer vision approach. The main application of our system is smart building management, where we specifically focus on increasing the services of the building users.

Smart camera networks are becoming a fundamental piece of our intelligent cities, buildings and homes, progressively inserting themselves in our lives. From smart surveillance systems composed of a multitude of smart camera nodes to small wearable cameras able to render a visual log of our daily experience. These devices interact with each other and with a wealth of other smart stuff, and of course the internet. This rapid development is being possible thanks to the convergence of several technologies. From advanced image sensors and vision chips to embedded vision systems capable of efficient feature extraction, image encoding and wireless transmission of the relevant visual content. This opens the door to new application domains, where video analytics and the extraction of semantic information from the scene is performed in a distributed fashion, implementing a new type of cooperative and/or collaborative vision, and even fusing visual information with other sensory data.

For efficient wide area surveillance, smart cameras should provide large observed data and many intelligent analysis functions. In order to extend the surveillance area and overcome occlusion, integration of data from multiple cameras and post analysis of it are required. The surveillance data includes low level multimedia data, recognized feature data, event alarms and others. Integration of those data needs interaction between cameras. This paper presents the design of smart camera network and the distributed data management scheme such as data structuring, interchanging and deployment. An overwhelming amount of surveillance data is generated by smart cameras. The data contained in them if properly structured, integrated and analyzed, can induce useful context information. This work builds a hierarchical data deployment structure and exchange related data arriving to and from multiple smart cameras. Proactive data deploying provides the main concepts and properties to model a hierarchical data structure. Our simulation and implementation of the strategy shows efficiency and usefulness.

With the introduction of battery-powered wireless embedded smart cameras, it has now become viable to deploy large numbers of spatially-distributed cameras with more flexibility in terms of camera locations. However, many challenges remain to be addressed to build operational, battery-powered, wireless smart-camera networks. Battery life is limited, and video processing tasks, such as foreground detection and tracking, consume considerable amount of energy. Thus, it is essential to design and implement lightweight algorithms and methods to increase the energy efficiency of each camera node, and thus the overall life-time of the camera network. We present an adaptive method based on tracking that significantly decreases the energy consumption of the embedded camera. The microprocessor on the camera board is sent to an idle state depending on the amount of activity in the scene. The amount of time the camera remains in idle mode is adaptively changed based on the speeds of tracked objects. Instead of continuously capturing and processing every frame, the camera drops frames during idle mode while preserving the tracking performance and thus system reliability at the same time. We present experimental results showing the energy-efficiency of the proposed method, and the gain in battery life. The proposed methodology provides 25% to 37% savings in the energy consumption, and 45:83% to 65% increase in the battery life depending on the number of objects in the scene and their speeds.

Smart camera networks have recently emerged as a new class of sensor network infrastructure that is capable of supporting high-power in-network signal processing and enabling a wide range of applications. In this article, we provide an exposition of our efforts to build a low-bandwidth wireless camera network platform, called CITRIC, and its applications in smart camera networks. The platform integrates a camera, a microphone, a frequency-scalable (up to 624 MHz) CPU, 16 MB FLASH, and 64 MB RAM onto a single device. The device then connects with a standard sensor network mote to form a wireless camera mote. With reasonably low power consumption and extensive algorithmic libraries running on a decent operating system that is easy to program, CITRIC is ideal for research and applications in distributed image and video processing. Its capabilities of in-network image processing also reduce communication requirements, which has been high in other existing camera networks with centralized processing. Furthermore, the mote easily integrates with other low-bandwidth sensor networks via the IEEE 802.15.4 protocol. To justify the utility of CITRIC, we present several representative applications. In particular, concrete research results will be demonstrated in two areas, namely, distributed coverage hole identification and distributed object recognition.

Abstract-With the fast advance of mobile chips technologies, a node in a smart camera network can afford sophisticated processing via on-board multicore CPUs and GPUs, e.g., face detection. The performance of a general purpose face detector, however, may degrade seriously under specific situations with unexpected challenges such as facial coverage or bad illumination. This degradation is due to the difference of probability distributions between training data and testing data, known as source data domain and target data domain, respectively. To better adapt a smart camera network to a specific situation, some form of target domain adaptation is needed, which usually requires both the source domain data and as much as possible target domain data at each node, which may strain storage capacity and bandwidth. In this paper, we propose an adaptation method which fuses the source specific hypotheses (SSHs) and target specific hypotheses (TSHs) - requiring only a pre-trained face detector and a few target data to be shared by peer-to-peer communications, thus relieving the storage and bandwidth constraints. The method uses the accuracy-regularization objective as the adaptation model, to fuse SSHs and TSHs, and tries to minimize the misclassification error on target data. With an existing frontal face detector, we conduct experiments to verify our algorithm, covering cases of video surveillance, extreme pose challenge, and different illumination spectra. Significant performance gains are observed with only dozens of target data in all the experiments, demonstrating the effectiveness of the proposed adaptation model. Therefore, the proposed adaption can be applied to a smart camera network with peer-to-peer communications to improve the network's overall face detection performance.

Smart camera networks are becoming increasingly popular in a number of application domains. In many applications, cameras are required to collaboratively track objects (e.g., habitat monitoring, or surveillance). In smart networks, camera coverage control is necessary to allow automatic tracking of targets without human intervention, allowing these systems to scale. In this paper, we consider the problem of automatic control of the cameras to maximize coverage of a set of targets. We formulate an optimization problem with the goal of maximizing the number of covered targets. Since the optimization problem is NP-hard, even for static targets, we propose a computationally efficient heuristic to reach near-optimal solution. Centralized solutions achieve excellent coverage, and can work well for small-scale networks, however they require significant communication cost for large scale networks. As a result, we propose an algorithm that spatially decomposes the network and computes optimal solutions for individual partitions. By decomposing the partitions in a way that minimizes dependencies between them, this approach results in coverage quality close to the centralized optimal solution, with an overhead and reaction time similar to those of distributed solutions.

Smart camera networks (SCNs) are increasingly used in applications such as homeland security, border control and traffic monitoring. The cameras are often wireless with an organically growing structure, to reduce the overhead of deployment. We frame a new and important problem in SCNs: how to transmit videos from multiple cameras with overlapping coverage given the limited available wireless bandwidth to maximize the quality of the received videos. We call this problem Coordinated Transport of Correlated Videos (CTCV). CTCV is a more general version of 3D video transport: in that problem, highly correlated videos from two cameras (that provide the 3D perspective) are jointly encoded exploiting their pre-defined and known overlap. In contrast, in CTCV there is an arbitrary number of cameras whose overlap is not known apriori and that require transmission as multiple video streams. To effectively support CTCV, we propose a video delivery protocol that consists of two primary components: (1) Consolidation of correlated videos from multiple cameras which removes spatially redundant fields-of-view; and (2) Network and coverage aware bandwidth allocation to optimize coverage quality cooperatively among the different video streams to match the available bandwidth. We formulate the problem of optimal bandwidth allocation for maximizing coverage. We propose and investigate different heuristic policies for bandwidth allocation. We evaluate CTCV using data from a small camera testbed as well as topologies from realistic deployments. Experiments show that CTCV achieves around 10 dB gains in video quality in the scenarios we consider.

In this work, we propose the cubature Kalman filter (CKF) based distributed object tracking algorithm in a visual sensor network (VSN). A VSN consists of several distributed smart cameras having the ability to process and analyze the retrieved data locally. The first objective is to optimize the tracking process within the VSN through the CKF. Under the conditions of non-linear motion and observation model, the CKF based method features a considerably better tracking accuracy than the extended Kalman filter (EKF) based method in terms of the mean square error (MSE). Although, the particle filter (PF) based method shows better performance than the CKF, it is computationally very complex. The second objective is to optimize the object tracking by aggregating the tracking results from multiple cameras. Assuming the VSN is a multi-camera network with overlapping field of views (FOVs), cameras having the same object in their FOV exchange their local estimates of the object's position and velocity. During the estimation process, each of the participating cameras aggregates the received states via a consensus algorithm. Thus, the object's real state is more accurately predicted by the resulting joint state than it would be by processing only a single camera's observation.

Visual attention modelling characterises the scene to segment regions of visual interest and is increasingly being used as a pre-processing step in many computer vision applications including surveillance and security. Smart camera architectures are an emerging technology and a foundation of security and safety frameworks in modern vision systems. In this paper, we present a dataflow design of a visual saliency based camera architecture targeting a heterogeneous CPU+FPGA platform to propose a smart camera network infrastructure. The proposed design flow encompasses image processing algorithm implementation, hardware & software integration and network connectivity through a unified model. By leveraging the properties of the dataflow paradigm, we iteratively refine the algorithm specification into a deployable solution, addressing distinct requirements at each design stage: from algorithm accuracy to hardware-software interactions, real-time execution and power consumption. Our design achieved real-time run time performance and the power consumption of the optimised asynchronous design is reported at only 0.25 Watt. The resource usages on a Xilinx Zynq platform remains significantly low.

An embedded smart camera is a stand-alone unit that not only captures images, but also includes a processor, memory and communication interface. With battery-powered and embedded smart cameras, it has become viable to install many spatially-distributed cameras interconnected by wireless links. Not requiring to have access to electrical outlets and have wired links increase system flexibility. However, wireless and battery-powered smart-camera networks introduce many additional challenges since they have very limited resources, such as power, memory and bandwidth. The algorithms running on the camera boards should be lightweight and efficient. In addition, the frequency of communication between camera nodes, and the content of the message packets should be carefully designed, since communication consumes power. In this paper, we present a wireless embedded smart-camera system that performs peer-to-peer object tracking and event detection. We analyze the power consumption and performance of this system during different parts of the algorithm execution and for different message exchanges between camera nodes. We also present a graph of the energy consumption for different tasks performed in a camera's processor. The number of instructions are also presented. The results demonstrate the importance of the careful choice of when and what data to transfer between cameras, and also the necessity of having lightweight algorithms in these resource-constrained systems.