An automated energy management framework for smart homes

Smart homes are permeating our everyday lives. Different fast-evolving smart home technologies, various usage contexts, and heterogeneous users make it challenging to anticipate all requirements for smart homes in advance. User feedback gathered from the crowd of smart home technologies users is a valuable information source for verifying and eliciting requirements for system evolution. However, research on user feedback currently does not consider the smart home domain. Thus, we know only little about whether and how users of smart home technologies could provide feedback to support the elicitation of requirements and hence system evolution. In this paper, we present preliminary findings from our explorative interview study with smart home users on their attitudes towards feedback communication. Early results suggest that users are willing to provide feedback about their smart homes but have different needs regarding the feedback communication process. We assume the characteristics of smart home technologies (e.g., ubiquity, multimodality, connectivity) provide opportunities but also raise challenges for developing a user feedback approach in the context of smart homes.

In this paper we will talk about the smart home concept. Even if smart home domain is not like the robotics domain, we can notice resemblances. In V. Ricquebourg et al. (2005), we shown that a smart home can be seen as a robot that turn over itself. A robot needs sensors to discover its environment. In the same way, a smart home needs sensors to watch in itself. This paper deals with a service oriented architecture of context perception for the smart home. In our approach, the smart house must be able to provide services adapted to the needs for its inhabitants. To answer to the needs, the environment, which will be described as smart, must take into account the context around the inhabitant: it will make it possible to provide contextualised services. Here, we present our service oriented architecture which integrates the various layers necessary to the determination of this context. Four layers are considered. Perception layer to obtain necessary information on the context, the context layer which makes it possible to give a sense to the perceived data, the inference layer which makes it possible to elaborate an artificial intelligence layer and to reason on the contextual data and the action layer which makes it possible to act following the decisions induced by the inference layer

There are high hopes for smart home technology to deliver Home Energy Management (HEM) solutions, including through smart thermostats, plugs, lights, switches, and appliances. However, adoption of these technologies is lagging behind expectations. Moreover, it is unclear how energy management features in the smart home consumer adoption process. We know there is technical potential for the smart home to support energy management, but we know little about the degree to which energy benefits interest consumers and motivate them to adopt smart home technologies relative to non-energy benefits such as security, comfort, and convenience, which could have implications for increasing rather than decreasing energy consumption. To that point, we know little about whether and how the energy management functionalities of smart home products are actually used by adopters. The present research investigated consumers’ knowledge of, attitudes toward, and experiences with smart home technologies that have energy management functionalities (smart HEMS), in order to assess barriers to adoption and to achieving purported energy benefits. Specifically, we studied shoppers at smart home retailers to gauge their existing awareness of and attitudes toward smart HEMS, and we analyzed Amazon customer reviews of smart HEMS to better understand early adopters’ motivations and experiences. Results revealed challenges to achieving energy benefits with existing products and marketing strategies, and implications for shaping the future of these technologies to achieve energy demand reductions and load shifting capabilities at scale for the smart home and smart grid of the future.

The rapid development of Internet of Things (IoT) technology makes smart homes a reality, where many user-centered service scenarios are yet to be built. For such applications, it is of utmost importance to let end users express their needs easily on the one hand, and these users' expectations can be interpreted by the smart home systems accurately on the other hand. Unfortunately, existing requirements languages are not for end users to express their needs in daily terms. An easy but expressive requirements language is required. This paper carries out a survey on suitable and expressive requirements description language for smart homes. Based on the results, a user requirements description language Smart3E is designed for enabling end user expressions. This paper also puts forward the challenges that are needed to be addressed when allowing end users to express their needs freely in smart home domain.

Triboelectric based smart ceramic tiles

The advancement in wireless telecommunication technologies like 3G, 4G and now 5G enrich the demand of high performance portable devices, operating at high frequencies i.e. from few hundred MHz to several GHz. High performance can only be achieved at the cost of high power dissipation. Nowadays, the use of FPGAs is rapidly increasing due to its low cost and smaller time to market. Today’s high end FPGAs are capable of implementing high performance designs such as transceivers for RF application, signal processing and image processing applications etc. This is only possible due to their complex designs that includes on chip dedicated multipliers, Dual RAM blocks, Ethernet and DCMs etc. Since FPGAs consume large amount of power, it becomes a challenge for the designers to implement low power battery operated communication systems. Several power estimation models are available in literature for early stage power estimation of FPGAs in the synthesis design flow but they provide less accuracy (high percentage error) for the designs incorporated with low power reduction technique. This paper presents the limitation of existing power estimation model proposed by Deng. The given model is converted into a script using MATLAB. The power values estimated from the power model (script) are compared with the high level synthesis tool: Xpower Analyzer targeted to Virtex FPGA. For estimation of power values, an embedded multiplier is synthesized for two different configurations: one with ‘CE’ (low power) and another without ‘CE’ for different I/O vector length. The results suggested that the given low level power estimation models provide less accuracy (high % error) for the designs incorporated with low power reduction technique i.e. with ‘CE’. Further, there is a scope of remodel the existing model for the observed shortcoming.

Nowadays, the load monitoring system is an important element in smart buildings to reduce energy consumption. Nonintrusive load monitoring (NILM) is utilized to determine the power consumption of each appliance in smart homes. The main problem of NILM is how to separate each appliance's power from the signal of aggregated consumption. In this regard, this paper presents a combination between particle swarm optimization (PSO) and artificial neural networks (ANNs) to identify electrical appliances for demand‐side management. ANN is applied in NILM as a load identification task, and PSO is used to train the ANN algorithm. This combination enhances the NILM technique's accuracy, which is further verified by experiments on different datasets like Reference Energy Disaggregation Dataset, UK Domestic Appliance‐Level ElectricityUK‐DALE, and Indian data for Ambient Water and electricity Sensing. The high accuracy of the proposed algorithm is verified by comparisons with state of the art methods. Compared with other approaches, the total mean absolute error has decreased from 39.3566 to 18.607. Also, the normalized root mean square error (NRMSE) method has been used to compare the measured and predicted results. The NRMSE is in the range of 1.719%–16.514%, which means perfect consistency. This demonstrates the effectiveness of the proposed approach for home energy management. Furthermore, customer behavior has been studied, considering energy costs during day hours.

The high penetration of distributed energy resources and heavy flexible loads, such as electric vehicles, has changed the operating conditions of the distribution network. In particular, the adoption of green energy innovations has caused an increase in electricity consumption. Such an increase can result in thermal overloading due to power flow exceeding a network asset's transfer capability, possibly damaging devices such as distribution transformers and feeders. It is very challenging to design a congestion management scheme given the uncertainty of flexible loads consumption and electricity prices. Obtaining stochastic models for such loads may not be easily available in practice. In this article, a deep deterministic policy gradient (DDPG) reinforcement learning (RL) scheme is proposed to alleviate congestion. DDPG RL is a model free technique that does not require explicit probabilistic models of the controllable loads to determine the change in electricity prices needed, in the form of tariffs and/or subsidies. The DDPG RL technique is compared with the existing model-based congestion management scheme using an IEEE 33 bus system and the results obtained demonstrate the out performance of the proposed technique in terms of the electricity cost and peak-to-average ratio of the load profiles.

Despite the fact that deep reinforcement learning (RL) has surpassed human-level performances in various tasks, it still has several fundamental challenges. First, most RL methods require intensive data from the exploration of the environment to achieve satisfactory performance. Second, the use of neural networks in RL renders it hard to interpret the internals of the system in a way that humans can understand. To address these two challenges, we propose a framework that enables an RL agent to reason over its exploration process and distill high-level knowledge for effectively guiding its future explorations. Specifically, we propose a novel RL algorithm that learns high-level knowledge in the form of a finite reward automaton by using the L* learning algorithm. We prove that in episodic RL, a finite reward automaton can express any non-Markovian bounded reward functions with finitely many reward values and approximate any non-Markovian bounded reward function (with infinitely many reward values) with arbitrary precision. We also provide a lower bound for the episode length such that the proposed RL approach almost surely converges to an optimal policy in the limit. We test this approach on two RL environments with non-Markovian reward functions, choosing a variety of tasks with increasing complexity for each environment. We compare our algorithm with the state-of-the-art RL algorithms for non-Markovian reward functions, such as Joint Inference of Reward machines and Policies for RL (JIRP), Learning Reward Machine (LRM), and Proximal Policy Optimization (PPO2). Our results show that our algorithm converges to an optimal policy faster than other baseline methods.

There is a growing need for accurate power models at the system level. Memory structures such as caches, Branch Target Buffers (BTBs), and register files occupy significant area in contemporary SoC designs and are the main contributors to system leakage power dissipation. Existing models for leakage power estimation in array structures typically use coefficients derived from elaborate SPICE simulations. However, these methodologies are not applicable to array designs in a newer technology, that require power estimates early in the design cycle. In this paper, we propose analytical models for array structures that are based only on high level design parameters. Assuming typical circuit implementation styles, we identify the transistors that contribute to the leakage power in each array sub-circuit and develop models as a function of the operation (read/write/idle) on the array and organizational parameters of the array. The developed models are validated by comparing their estimates against the leakage power measured using SPICE simulations on industrial array designs belonging to the e5001 processor core. The comparison shows that the models are accurate with an error margin of less than 21.5% and thus can be used in high-level power-performance exploration. Interestingly, in array designs with dual threshold voltage technology, we observed that contrary to the general expectation, the array memory core contributes to just 9% and the address decoder contributes to as much as 62% of the total leakage power.

Thermal management systems (TMSs) are crucial for driving safety, mileage, and comfort in battery electric vehicles. To maximize the potential of the integrated TMSs in terms of temperature control and energy saving, the deep deterministic policy gradient (DDPG) is utilized to design a learning thermal management methodology (TMM) for it. Considering the key challenges faced, linear mapping trick and gated recurrent unit are employed to improve the original DDPG. The former empowers the DDPG agents with the ability to make decisions in the discrete-continuous hybrid action space while helping it to avoid the ’curse of dimensionality’. The latter provides the DDPG agents with beneficial historical information, which enhances its decision-making quality. Simulation results show that the proposed TMM decreases the convergence episode to 77 while increasing the convergence reward to 553.4. Further adaptive tests demonstrate that the suggested TMM promptly stabilizes the motor and battery temperatures at the desired values. Simultaneously, energy consumption decreases by 14.71% and 11.45% for two cases compared to the conventional rule-based TMM. In conclusion, the proposed method provides a theoretical foundation for addressing the hybrid action space optimization problem in integrated TMSs.

Internet of Things (IoT) has introduced new applications and environments. Smart Home provides new ways of communication and service consumption. In addition, Artificial Intelligence (AI) and deep learning have improved different services and tasks by automatizing them. In this field, reinforcement learning (RL) provides an unsupervised way to learn from the environment. In this paper, a new intelligent system based on RL and deep learning is proposed for Smart Home environments to guarantee good levels of QoE, focused on multimedia services. This system is aimed to reduce the impact on user experience when the classifying system achieves a low accuracy. The experiments performed show that the deep learning model proposed achieves better accuracy than the KNN algorithm and that the RL system increases the QoE of the user up to 3.8 on a scale of 10.

Reinforcement learning (RL) presents numerous benefits compared to rule-based approaches in various applications. Privacy concerns have grown with the widespread use of RL trained with privacy-sensitive data in IoT devices, especially for human-in-the-loop systems. On the one hand, RL methods enhance the user experience by trying to adapt to the highly dynamic nature of humans. On the other hand, trained policies can leak the user's private information. Recent attention has been drawn to designing privacy-aware RL algorithms while maintaining an acceptable system utility. A central challenge in designing privacy-aware RL, especially for human-in-the-loop systems, is that humans have intrinsic variability and their preferences and behavior evolve. The effect of one privacy leak mitigation can be different for the same human or across different humans over time. Hence, we can not design one fixed model for privacy-aware RL that fits all. To that end, we propose adaPARL, an adaptive approach for privacy-aware RL, especially for human-in-the-loop IoT systems. adaPARL provides a personalized privacy-utility trade-off depending on human behavior and preference. We validate the proposed adaPARL on two IoT applications, namely (i) Human-in-the-Loop Smart Home and (ii) Human-in-the-Loop Virtual Reality (VR) Smart Classroom. Results obtained on these two applications validate the generality of adaPARL and its ability to provide a personalized privacy-utility trade-off. On average, for the first application, adaPARL improves the utility by $57\%$ over the baseline and by $43\%$ over randomization. adaPARL also reduces the privacy leak by $23\%$ on average. For the second application, adaPARL decreases the privacy leak to $44\%$ before the utility drops by $15\%$.

One of the most important topics in the field of wireless sensor networks is the development of approaches to improve network lifetime. In this paper, an energy-efficient control and routing protocol for wireless sensor networks is presented. This algorithm is based on reinforcement learning for energy management in the network. This protocol seeks to optimize routing policies to maximize the long-term reward received by each node, using reinforcement learning, which is a machine learning approach. In order to improve the lifetime of wireless sensor network, three energy management approaches have been proposed. The first approach is to navigate correctly using reinforcement learning to reduce the length of the routes and to improve energy consumption. The second approach is to exploit a sleep scheduling technique to improve node energy consumption. The last approach is used to restrict data transmission of each node based on the received data change rate. Simulation results show that in terms of network lifespan, the proposed method significantly outperforms previous reported methods.

Attitude control of a novel regional truss-braced wing (TBW) aircraft with low stability characteristics is addressed in this paper using Reinforcement Learning (RL). In recent years, RL has been increasingly employed in challenging applications, particularly, autonomous flight control. However, a significant predicament confronting discrete RL algorithms is the dimension limitation of the state-action table and difficulties in defining the elements of the RL environment. To address these issues, in this paper, a detailed mathematical model of the mentioned aircraft is first developed to shape an RL environment. Subsequently, Q-learning, the most prevalent discrete RL algorithm, will be implemented in both the Markov Decision Process (MDP) and Partially Observable Markov Decision Process (POMDP) frameworks to control the longitudinal mode of the proposed aircraft. In order to eliminate residual fluctuations that are a consequence of discrete action selection, and simultaneously track variable pitch angles, a Fuzzy Action Assignment (FAA) method is proposed to generate continuous control commands using the trained optimal Q-table. Accordingly, it will be proved that by defining a comprehensive reward function based on dynamic behavior considerations, along with observing all crucial states (equivalent to satisfying the Markov Property), the air vehicle would be capable of tracking the desired attitude in the presence of different uncertain dynamics including measurement noises, atmospheric disturbances, actuator faults, and model uncertainties where the performance of the introduced control system surpasses a well-tuned Proportional–Integral–Derivative (PID) controller.