An Object Recognition Scheme Based on Visual Descriptors for a Smart Home Environment

The home automation controls the electronic devices remotely and automatically with the help of context aware behaviour. Context awareness is the process where data from various sensors and input from user or user action must be interpreted and corresponding commands to actuators must be generated. This context aware automation and decision support process is done with the help of adaptive rules which is defined for the automatic behaviour of devices in the smart home environment. The consistent behaviour of the devices in the smart environment will capture the customer satisfaction. This work describes the process of checking the consistent behaviour of the context aware system in smart home environment using formal modelling and verification method. We have discussed a mechanism to check the consistent behaviour of the adaptive mechanism of smart home environment. The result ensures the consistent behaviour of the context aware system in the smart environment.

This paper addresses the problem of learning situation models for providing context-aware services. Context for modeling human behavior in a smart environment is represented by a situation model describing environment, users, and their activities. A framework for acquiring and evolving different layers of a situation model in a smart environment is proposed. Different learning methods are presented as part of this framework: role detection per entity, unsupervised extraction of situations from multimodal data, supervised learning of situation representations, and evolution of a predefined situation model with feedback. The situation model serves as frame and support for the different methods, permitting to stay in an intuitive declarative framework. The proposed methods have been integrated into a whole system for smart home environment. The implementation is detailed, and two evaluations are conducted in the smart home environment. The obtained results validate the proposed approach.

Event Abstract Back to Event Logarithmic Hybrid Optical Neural Network-type systems: towards digital-optical cognitive models of robust object recognition Ioannis Kypraios1* 1 Oxford University, Centre for Innovation & Enterprise, APEM Computing Lab, United Kingdom Abstract - By combining the complex logarithmic r-theta mapping of a space-variant imaging sensor [1, 2] with the hybrid digital-optical neural network filter together with a window unit multiple objects of the same class or of different classes can robustly be recognized. The resulted logarithmic r-theta mapping for hybrid digital-optical neural network system or briefly referred to as logarithmic hybrid optical neural network system is shown to exhibit with a single pass over the input data simultaneously in-plane rotation, out-of-plane rotation, scale, log r-θ map translation and shift invariance, and good clutter tolerance by recognizing correctly the different objects within the cluttered scenes. Here, we study the biologically-inspired knowledge learning and knowledge representation [3] achieved by the logarithmic hybrid optical neural network-type of object recognition systems (see Fig. 1 and Fig. 2). We investigate the effects that altering the knowledge representation can have on the problem’s learned knowledge and the problem solving process, in overall. Further, we study the logarithmic unconstrained-, logarithmic constrained-, and logarithmic modified-hybrid optical neural network systems architectures’ designs [4, 5]. We show how the logarithmic unconstrained-hybrid optical neural network object recognition system applies an unconstrained representation of the problem’s knowledge to maximize the search of solutions, how the logarithmic constrained-hybrid optical neural network object recognition system uses a constrained representation of the problem’s knowledge to guide the search towards certain solutions over others in the multidimensional search space, and how the logarithmic modified-hybrid optical neural network object recognition system uses memory-like masks to recall certain solutions over others in the multidimensional search space. Fig. 1. (a) Simplified human retina and visual cortex model used for description purposes only; (b) A digital-optical computational model for the cognitive interaction between the retina and the human visual cortex with the general logarithmic hybrid optical neural network architercture. Fig. 2. Biologically-inspired knowledge learning and knowledge representation with the logarithmic hybrid optical neural network-type of object recognition systems. Figure 1 Figure 2 References [1] C. F. R. Weiman, “Video compression via log-polar mapping”, Real-Time Image Processing II, SPIE Symposium on OE/Aerospace Sensing, Vol. 1295, 266-277 (1990) [2] C. G. Ho et al., “Sensor geometry and sampling methods for space-variant image processing”, Pattern Analysis and Applications, Vol. 5, 369-384 (2002) [3] I. Lee, and B. Portier, "An empirical study of knowledge representation and learning within conceptual spaces for intelligent agents", 6th IEEE/ACIS International Conference Computer and Information Science, 463-468, Melbourne, Australia (2007) [4] I. Kypraios et al., “Fully invariant complex Logarithmic r- map for the Hybrid Optical Neural Network filter for object recognition within cluttered scenes”, 50th Anniversary International Symposium ELMAR 2008, IEEE Region 8/IEEE Croatia Section/EURASIP, Zadar-Croatia, Vol. 1, 141-146 (2008) [5] I. Kypraios et al., “Logarithmic r-θ mapping for hybrid optical neural network filter for object recogntion within cluttered scenes”, Recent Advances in Multimedia Signal Processing and Communications, SCI231, Springer-Verlag Berlin Heidelberg, 91-120 (2009) Keywords: digital-optical, cognitive, object recognition, Robust and Adaptive Systems, Knowledge representation, learning and memory Conference: Neuroinformatics 2013, Stockholm, Sweden, 27 Aug - 29 Aug, 2013. Presentation Type: Poster Topic: Neuromorphic engineering Citation: Kypraios I (2013). Logarithmic Hybrid Optical Neural Network-type systems: towards digital-optical cognitive models of robust object recognition. Front. Neuroinform. Conference Abstract: Neuroinformatics 2013. doi: 10.3389/conf.fninf.2013.09.00066 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 09 Apr 2013; Published Online: 11 Jul 2013. * Correspondence: Dr. Ioannis Kypraios, Oxford University, Centre for Innovation & Enterprise, APEM Computing Lab, Begbroke, Oxfordshire, OX5 1PF, United Kingdom, ioanniskyp@yahoo.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Ioannis Kypraios Google Ioannis Kypraios Google Scholar Ioannis Kypraios PubMed Ioannis Kypraios Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Unobtrusive and multiple-sensor interfaces enable observation of natural human behavior in smart environments. Being able to detect, analyze and interpret this activity allows for the implementation of various applications, including real-time surveillance and real-time support in smart home and office environments. We focus on smart meeting environments, where nonverbal behavioral cues sometimes tell more about issues such as discussion participation, involvement and contribution, than information obtained from verbal contributions. An important aspect of this behavior is the interaction between meeting participants. We regard the special case where some of the participants are at physically different locations, which hinders natural interaction. We discuss how we can exploit the ability of a sensor-equipped environment to detect nonverbal interaction cues and use these to allow and improve natural interaction between collaborating participants at distributed locations. We focus on research efforts to detect nonverbal interaction cues by looking at various modalities. Also, we discuss how information obtained from the fusion of these modalities allows us to generate and display behavioral cues that allow remote participants to take part in a distributed meeting in a natural way.

Smart environments are becoming ubiquitous despite many potential security and privacy issues. But, do people understand what consequences could arise from using smart environments? To answer this research question, we conducted a survey with 575 participants from three different countries (Germany, Spain, Romania) considering smart home and health environments. Less than half of all participants mentioned at least one security and privacy issue, with significantly more German participants mentioning issues than the Spanish ones and the Spanish participants in turn mentioning significantly more security and privacy issues than the Romanian participants. Using open coding, we find that among the 275 participants mentioning security and privacy issues, 111 only expressed abstract concerns such as “security issues” and only 34 mentioned concrete harms such as “Burglaries (physical and privacy)”, caused by security and privacy violations. The remaining 130 participants who mentioned security and privacy issues named only threats (i.e. their responses were more concrete than just abstract concerns but they did not mention concrete harming scenarios).

This article details the development and testing of an empirical data capture system with the ability to collect spatial-frequency statistics relating to the movement behaviour of a smart home inhabitant. This is achieved using a greyscale normalised cross-correlation pattern matching algorithm. Environmental obstructions on the floor space can also be inferred from a visual representation of the accumulated data. Whilst this methodology itself is not novel, its application to person tracking specifically within a smart home environment does not appear in the literature and is considered a novel approach. The results of tests performed on the pattern matching technique show a tracking competency rate of 94.45% with a standard deviation of 0.009027, indicating high fidelity across a wide variety of environmental factors.

Stakeholders in disaster management settings often find the effective and efficient utilisation of emerging technologies quite a challenging process but very frequently a critical computational inclusion to the intelligence required for protecting individuals, communities, their environment and critical infrastructures. The special issue is dedicated to the dissemination of original contributions that are related to the theories, practices and concepts of smart environments and emerging collective computational and collaborative technologies for the specific purpose of improving intelligence when managing natural or man-made disasters. In particular, the special issue is focused on the applicability—for the purpose of managing disasters—of technologies that can support autonomous adaptation to complex and messy situations caused from these, highly dynamic contexts. Various methods and technologies including simulations, multi-dimensional decision modeling, data mining, swarm intelligence, smart spaces and sensors, contextaware, situated and pervasive computing, geographical information systems, ad-hoc mobile networks, wireless communications, grid and cloud computing, social networks and crowd sourcing are a smart technology means to enable a more informed decision making to mitigate and prepare for, respond to and recover from growing occurrences of disasters. This special issue comprises six articles, which have been selected after a careful review process. Specifically, this is a collection of extended versions of papers presented either at the Smart Environments for Disaster Management (SEDM-2011) workshop which run in conjunction with the 6th 3PGCIC-2011, held on October 26–28, 2011 in Barcelona, Spain or at the Computational Intelligence for Disaster Management (CIDM-2011) workshop which run in conjunction with the 3rd INCoS 2011, held on November 30–December 2, 2011 in Fukuoka, Japan. In the first article, Li explains that mobility is an important issue that deters people living in remote areas and elderlies to visit health clinics on a regular basis; this is also the case for individuals with disabilities. Advances in information and telecommunication technologies have been made telemedicine possible. By using the latest sensor technologies, a person’s vital data can be collected in a smart home environment. The bio-information can then be transferred wirelessly or via the Internet to medical databases and the healthcare professionals. The article surveys and compiles the state-of-the-art smart home technologies and telemedicine systems. In the second article, Sakanushi et al. discuss concerns with triage tags, which are widely used at disaster scenes. Some of these concerns include the inability to show the current priorities of casualties and a failure to collect the physiological conditions of the casualties. In their article, authors propose an electronic triage which continuously monitor the vital signs of casualties and transmit them to an electronic triage server. Experimental results show that their proposed electronic triage system can save more lives than the paper triage tags version. In the third article, Furey et al. explain that the use of Wi-Fi signals is an attractive and reasonably affordable option to deal with the currently unsolved problem of widespread tracking in an indoor environment. In their article, authors present a system which aims at overcoming N. Bessis (&) E. Asimakopoulou School of Computing and Mathematics, University of Derby, Kedleston Road, Derby DE22 1GB, UK e-mail: n.bessis@derby.ac.uk

In this work we will examine and develop a system that can assist people in Activities of Daily Life (ADL). This study focuses on resolving conflicts for the requests from different users' profiles, for instance - elderly, adult and young. The objective of the system is to present a dialogue manager which is able to detect multi-user semantic conflict and to resolve the conflict for improved dialogue informing about its decisions using a system interface avatar. The system is also able to prioritize requests that occurred among the services of multiple home appliances, as well as to deal with conflicting entities involving a single device. We investigated whether the multi-user context awareness by a Virtual Assistant adds value to the Smart Home concept in recognizing multi-user conflicts dynamically. This work has proposed a preference based method for resolving conflict and evaluated the developed system in a smart home environment.

Object recognition based on convex hull alignment

Smart homes can provide complementary information to assist home service robots. We present a robotic misplaced item finding (MIF) system, which uses human historical trajectory data obtained in a smart home environment. First, a multi-sensor fusion method is developed to localize and track a resident. Second, a path-planning method is developed to generate the robot movement plan, which considers the knowledge of the human historical trajectory. Third, a real-time object detector based on a convolutional neural network is applied to detect the misplaced item. We present MIF experiments in a smart home testbed and the experimental results verify the accuracy and efficiency of our solution.

We propose a new scheme for object recognition in robot vision. The proposed scheme uses the HSI model as the input. A gradient algorithm is used to obtain the edge estimation of objects. The edge points are regarded as stimulus and the other points as nonstimulus. The stimulus context of all the possible positions of objects is coded in stimulus vectors. Then a HMM-based vision system for scene analyse and object recognition is presented. This method has been tested using our mobile service robot called ARNOLD and real data of Columbia Object Image Library (COIL-20). Through these experiments, we have demonstrated the generalisation capabilities and robustness of object recognition and classification.

Detecting multiple salient objects in complex scenes is a challenging task. In this paper, we present a novel method to detect salient objects in images. The proposed method is based on the general ‘center-surround’ visual attention mechanism and the spatial frequency response of the human visual system (HVS). The saliency computation is performed in a statistical way. This method is modeled following three biologically inspired principles and compute saliency by two ‘scatter matrices’ which are used to measure the variability within and between two classes, i.e., the center and surrounding regions, respectively. In order to detect multiple salient objects of different sizes in a scene, the saliency of a pixel is estimated via its saliency support region which is defined as the most salient region centered at the pixel. Compliance with human perceptual characteristics enables the proposed method to detect salient objects in complex scenes and predict human fixations. Experimental results on three eye tracking datasets verify the effectiveness of the method and show that the proposed method outperforms the state-of-the-art methods on the visual saliency detection task.

This paper presents an asyn-chronous electrooculography (EOG)-based human-machine interface (HMI) for smart home environmental control with the purpose of providing daily assistance for severe spinal cord injury (SCI) patients. The proposed HMI allows users to interact with a smart home environment through eye blinking. Specifically, several buttons, each corresponding to a control command, randomly flash on a graphical user interface. Each flash of the buttons functions as a visual cue for the user to blink. To issue a control command, the user can blink synchronously with the flashes of the corresponding button. Through detecting blinks based on the recorded EOG signal, the target button and its corresponding control command are determined. Seven SCI patients participated in an online experiment, during which the patients were required to control a smart home environment including household electrical appliances, an intelligent wheelchair, as well as a nursing bed via the proposed HMI. The average false operation ratio in the control state was 4.1%, whereas during the idle state, no false operations occurred. All SCI patients were able to control the smart home environment using the proposed EOG-based HMI with satisfactory performance in terms of the false operation ratio in both the control and the idle states. The proposed HMI offers a simple and effective approach for patients with severe SCIs to control a smart home environment. Therefore, it is promising to assist severe SCI patients in their daily lives.

Smart homes are an emerging paradigm of Internet of Things (IoT) in which users can remotely control various home devices via the internet anytime and anywhere. However, smart home environments are vulnerable to security attacks because an attacker can inject, insert, intercept, delete, and modify transmitted messages over an insecure channel. Thus, secure and lightweight authentication protocols are essential to ensure useful services in smart home environments. In 2021, Kaur and Kumar presented a two-factor based user authentication protocol for smart homes using elliptic curve cryptosystems (ECC). Unfortunately, we demonstrate that their scheme cannot resist security attacks such as impersonation and session key disclosure attacks, and also ensure secure user authentication. Moreover, their scheme is not suitable in smart home environments because it utilizes public-key cryptosystems such as ECC. Hence, we design a secure and lightweight three-factor based privacy-preserving authentication scheme for IoT-enabled smart home environments to overcome the security problems of Kaur and Kumar’s protocol. We prove the security of the proposed scheme by using informal and formal security analyses such as the ROR model and AVISPA simulation. In addition, we compare the performance and security features between the proposed scheme and related schemes. The proposed scheme better provides security and efficiency compared with the previous schemes and is more suitable than previous schemes for IoT-enabled smart home environments.

Smart environments enhance user capabilities and comfort by providing new services and automated service execution based on sensed user activity. Though, when becoming really “smart”, such environments should not only provide some service automation, but further learn and adapt their behavior during use. This article motivates and investigates learning of situation models in a smart environment, covering knowledge acquisition from observation as well as evolving situation models during use. An integral framework for acquiring and evolving different layers of a situation model is detailed. Different learning methods are presented as part of this framework: role detection per entity, unsupervised extraction of situations from multimodal data, supervised learning of situation representations, and the evolution of a predefined situation model with feedback. The situation model serves as frame and support for the different methods, permitting to stay in an intuitive declarative framework. An implementation of the whole framework for a smart home environment is described, and the results of several evaluations are depicted.