Non-Intrusive Load Monitoring (NILM) for Energy Disaggregation Using Soft Computing Techniques

Non-intrusive load monitoring (NILM) is to estimate individual appliance’s power consumption from aggregated smart meter data, which is useful for optimized energy management and provisioning of customized services. While deep learning (DL) has achieved state-of-the-art NILM performance, it is still constrained by the dependency on large amounts of data and intensive computations on training. In this paper, we propose a pre-training approach to address the generalization of DL models for NILM. We develop a meta-learning based approach and an ensemble learning based approach, which pre-train a base model and then fine-tune it with few-short learning when applied to an unknown dataset. The models are validated with two real-world datasets and shown to achieve a superior transferability performance compared with traditional DL and transfer learning methods.

This paper presents a complete real-time implementation of a Non-Intrusive Appliance Load Monitoring (NIALM) system that, is robust under residential voltage level fluctuations. Existing NIALM techniques rely on multiple measurements taken at high sampling rates, but, only have been proven in simulated environments without even considering the effect of residential voltage level fluctuations — which is a severe problem in power systems of most developing countries like Sri Lanka. In contrast, through the NIALM method proposed in this paper, accurate load monitoring results were obtained in realtime using only smart meter measurements taken at a low sampling rate from a real appliance setup under residential voltage level fluctuations. In the proposed NIALM method, initially in the learning phase, a properly constructed MATLABTM Graphical User Interface (GUI) was used to acquire signals of each appliance active power consumption and voltage levels. Then, obtained active power measurements were separated into subspace components (SCs) via the Karhunen Loeve' Expansion (KLE) while also taking the voltage variations into account. Using those SCs, a unique information rich appliance level signature database was constructed and it was then used to obtain the signatures for all possible device combinations. Next, a separate GUI was designed to identify the turned ON appliance combination in the current time window using the pre-constructed signature databases, after reading the total residential active power consumption and the supply voltage. To validate the proposed real-time NIALM implementation, data from a laboratory arrangement consisting of ten household appliances was used. From the results, it was found that the proposed method is capable of accurately identifying the turned on appliances even under severe residential supply voltage level fluctuations.

Renewable energy sources are slowly but surely supplementing conventional sources of energy. These energy sources create a large number of micro-grids which become a part of the main grid. Smart meters are essential components in smart micro-grids and smart grids. Together they form a complex system which has become today an attractive area of research world-wide in view of enormous benefits of reliability and safety of power supply economically. Several studies in smart meter applications have shown that in residential buildings, energy consumption can be reduced up to 15% with better management. For this, a non-intrusive appliance load monitoring (NIALM) technique has been advocated which enables appliance identification to help better energy management. The identification is done using load signatures with appropriate algorithm. Several features of load signatures ranging from voltage and current profiles (steady states and transients) to a plethora of features such as active power (P) and reactive power (Q), power factor (pf), harmonic components (h), magnitudes of h (mh), current waveform (CW), eigen values, switching transient waveform, instantaneous admittance waveform, etc have been used in NIALM. The algorithms used vary from a `single algorithm' to `multiple algorithms', the latter used in `Committee Decision Mechanism' (CDM) in which majority of `votes' (i.e., agreement), `cast' by each of the algorithms, decides the final result of appliances identified. This scenario presents a serious difficulty with regard to choice of features and algorithms, besides the possibility of increasing the hardware and software costs of smart meters. The motivation for the study in this paper is provided by this scenario. The basic objective of this paper is to investigate the possibility of including only a limited number of features as load signature of appliances, and the use of only a single algorithm to effectively implement NIALM without any ambiguity. Results of our study so far indicate that generally only four features of P, Q, h and mh are sufficient for detection of appliances in operation; however, in some cases (~>10%), the additional feature, CW, may be needed for final identification.. The experiments conducted so far yielded unambiguous identification of appliances and the results are encouraging enough for possible use in practice for residential loads only.

What we measure, we can improve. In accordance to this approach, Indian Institute of Science (IISc), Bangalore has developed a Micro-grid Monitoring System in the campus through the installation of Smart Meters, covering almost 250 nodes including substations, centers, departments, administration, hostels and other utilities. This will help the institute in various ways such as capacity planning, substation loading, phase imbalance correction, over-voltage monitoring, billing and so on. Smart Meters measure the power consumption at a single point in the building giving a picture of the energy consumption of the building as a whole. It is necessary to also understand the scenario of the constituent loads at the point where the smart meter is installed so that ways could be found to reduce consumption. Personalised, concise and reliable feedback providing appliance level breakdown of energy consumption in the premises is the key in implementing energy efficiency programs. Taking this into consideration the area of Non Intrusive Load Monitoring (NILM) was explored. In NILM the aggregate smart meter data is separated into constituent loads by machine learning techniques. The NILM system is trained through previous data sets and then the algorithm will disaggregate the total power into individual appliances based on its experience. A benchmark NILM algorithm called Factorial Hidden Markov Model was used for proper load disaggregation. Finally an attempt was made to develop a Smartphone app to visualize results and bring the data to the people.

Flexibility characterization of residential electricity consumption: A machine learning approach

Large-scale smart energy metering deployment worldwide and integration of smart meters within the smart grid will enable two-way communication between the consumer and energy network, thus ensuring improved response to demand. Energy disaggregation or non-intrusive load monitoring (NILM), namely disaggregation of the total metered electricity consumption down to individual appliances using purely algorithmic tools, is gaining popularity as an added-value that makes the most of meter data. However, NILM remains a challenging problem since NILM is susceptible to sensor noise, unknown load noise, transient spikes, and fluctuations. In this paper, we tackle this problem using novel graph signal processing (GSP) concepts, applied at both, physical signal level via graph-based filtering and data level, via effective semi-supervised GSP-based feature matching. The proposed GSP-based method is generic and can be used to improve results of various event-based NILM approaches. We demonstrate significant improvement in performance using three state-of-the-art NILM methods, both supervised and unsupervised, and real-world active power consumption readings from the REDD and REFIT1 data sets, sampled at 1 and 8 s, respectively.

The availability of smart meters and IoT technology has opened new opportunities, ranging from monitoring electrical energy to extracting various types of information related to household occupancy, and with the frequency of usage of different appliances. Non-intrusive load monitoring (NILM) allows users to disaggregate the usage of each device in the house using the total aggregated power signals collected from a smart meter that is typically installed in the household. It enables the monitoring of domestic appliance use without the need to install individual sensors for each device, thus minimizing electrical system complexities and associated costs. This paper proposes an NILM framework based on low frequency power data using a convex hull data selection approach and hybrid deep learning architecture. It employs a sliding window of aggregated active and reactive powers sampled at 1 Hz. A randomized approximation convex hull data selection approach performs the selection of the most informative vertices of the real convex hull. The hybrid deep learning architecture is composed of two models: a classification model based on a convolutional neural network trained with a regression model based on a bidirectional long-term memory neural network. The results obtained on the test dataset demonstrate the effectiveness of the proposed approach, achieving F1 values ranging from 0.95 to 0.99 for the four devices considered and estimation accuracy values between 0.88 and 0.98. These results compare favorably with the performance of existing approaches.

Non-intrusive load monitoring (NILM) considers different approaches for disaggregating energy consumption in residential, tertiary, and industrial buildings to enable smart grid services. The main feature of NILM is that it can break down the bulk electricity demand, as recorded by conventional smart meters, into the consumption of individual appliances without the need for additional meters or sensors. Furthermore, NILM can identify when an appliance is in use and estimate its real-time consumption based on its unique consumption patterns. However, NILM is based on machine learning methods and its performance is dependent on the quality of the training data for each appliance. Therefore, a common problem with NILM systems is that they may not generalize well to new environments where the appliances are unknown, which hinders their widespread adoption and more significant contributions to emerging smart grid services. The main goal of the presented research is to apply a domain adversarial neural network (DANN) approach to improve the generalization of NILM systems. The proposed semi-supervised algorithm utilizes both labeled and unlabeled data and was tested on data from publicly available REDD and UK-DALE datasets. The results show a improvement in generalization performance on highly uncorrelated data, indicating the potential for real-world applications.

Millions of smart meters have been deployed worldwide, collecting the total power consumed by individual households. Based on these data, electricity suppliers offer their clients energy monitoring solutions to provide feedback on the consumption of their individual appliances. Historically, such estimates have relied on statistical methods that use coarse-grained total monthly consumption and static customer data, such as appliance ownership. Non-Intrusive Load Monitoring (NILM) is the problem of disaggregating a household's collected total power consumption to retrieve the consumed power for individual appliances. Current state-of-the-art (SotA) solutions for NILM are based on deep-learning (DL) and operate on subsequences of an entire household consumption reading. However, the non-stationary nature of real-world smart meter data leads to a drift in the data distribution within each segmented window, which significantly affects model performance. This paper introduces NILMFormer, a Transformer-based architecture that incorporates a new subsequence stationarization/de-stationarization scheme to mitigate the distribution drift and that uses a novel positional encoding that relies only on the subsequence's timestamp information. Experiments with 4 real-world datasets show that NILMFormer significantly outperforms the SotA approaches. Our solution has been deployed as the backbone algorithm for EDF's (Electricité De France) consumption monitoring service, delivering detailed insights to millions of customers about their individual appliances' power consumption.

Beyond privacy and security: Exploring ethical issues of smart metering and non-intrusive load monitoring

Energy disaggregation algorithms disintegrate aggregate demand into appliance-level demands. Among various energy disaggregation approaches, non-intrusive load monitoring (NILM) algorithms requiring a single sensor have gained much attention in recent years. Various machine learning and optimization-based NILM approaches are available in the literature, but bulk training data and high computational time are their respective drawbacks. Considering these drawbacks, we devised an event matching energy disaggregation algorithm (EMEDA) for NILM of multistate household appliances using smart meter data. Having limited training data, K-means clustering was employed to estimate appliance power states. These power states were accumulated to generate an event database (EVD) containing all combinations of appliance operations in their various states. Prior to matching, the test samples of aggregate demand events were decreased by event-driven data compression for computational effectiveness. The compressed test events were matched in the sorted EVD to assess the contribution of each appliance in the aggregate demand. To counter the effects of transient spikes and/or dips that occurred during the state transition of appliances, a post-processing algorithm was also developed. The proposed approach was validated using the low-rate data of the Reference Energy Disaggregation Dataset (REDD). With better energy disaggregation performance, the proposed EMEDA exhibited reductions of 97.5 and 61.7% in computational time compared with the recent smart event-based optimization and optimization-based load disaggregation approaches, respectively.

A promising approach toward efficient energy management is non-intrusive load monitoring (NILM), that is to extract the consumption of appliances by analyzing the aggregated consumption signal. The large number of appliances and the presence of appliances with close consumption values are known to limit the performance of event-based NILM methods. To tackle these challenges, one could enhance the feature space which in turn results in extra hardware costs, installation complexity, and concerns regarding the consumer’s privacy. This has led to the emergence of an alternative approach, namely semi-intrusive load monitoring, where appliances are partitioned into blocks and the consumption of each block is monitored via separate power smart meters. While a greater number of smart meters can result in more accurate disaggregation, it increases the monetary cost of load monitoring, indicating a trade-off that represents an important gap in this field. In this paper, we take a comprehensive approach to close this gap by establishing a so-called notion of “disaggregation difficulty metric (DDM),” which quantifies how difficult it is to monitor the events of any given group of appliances based on both their power values and the consumer’s usage behavior. Thus, DDM in essence quantifies how much is expected to be gained in terms of disaggregation accuracy of a generic event-based algorithm by installing smart meters on the blocks of any partition of the appliances. Experimental results based on the REDD dataset illustrate the practicality of the proposed approach in addressing the aforementioned trade-off.

Smart meters are required to identify home appliances to fulfill various tasks in the smart grid environment. On the other hand, techniques using non-intrusive appliance load monitoring (NIALM) are yet to result in meaningful practical implementation. Our experimental setup, on the recommended specifications of the internal electrical wiring in Indian residences, used common household appliances' load signatures of active and reactive powers, harmonic components and their magnitudes. We have introduced a new approach of ‘multi point sensing’ and ‘group control’ rather than the ‘single point sensing’ and ‘individual control’, used so far in NIALM techniques. The disaggregation system based on Central Public Works Department of India (CPWD). One feature i.e. amplitude of first 8 odd harmonics of current signature of home appliances were selected for the classification. Further principle component analysis (PCA) is used. A comparison between these classification algorithms has been done and it is revealed that artificial neural network classifier and Bayes classifier provides 99.18% and 98.08% accuracy respectively for the experimental data.

Prudent and meaningful performance evaluation of algorithms is essential for the progression of any research field. In the field of Non-Intrusive Load Monitoring (NILM), performance evaluation can be conducted on real-world aggregate signals, provided by smart energy meters or artificial superpositions of individual load signals (i.e., denoised aggregates). It has long been suspected that testing on these denoised aggregates provides better evaluation results mainly due to the fact that the signal is less complex. Complexity in real-world aggregate signals increases with the number of unknown/untracked loads. Although this is a known performance reporting problem, an investigation into the actual performance gap between real and denoised testing is still pending. In this paper, we examine the performance gap between testing on real-world and denoised aggregates with the aim of bringing clarity into this matter. Starting with an assessment of noise levels in datasets, we find significant differences in test cases. We give broad insights into our evaluation setup comprising three load disaggregation algorithms, two of them relying on neural network architectures. The results presented in this paper, based on studies covering three scenarios with ascending noise levels, show a strong tendency towards load disaggregation algorithms providing significantly better performance on denoised aggregate signals. A closer look at the outcome of our studies reveals that all appliance types could be subject to this phenomenon. We conclude the paper by discussing aspects that could be causing these considerable gaps between real and denoised testing in NILM.

Nonintrusive disaggregation of residential air-conditioning loads from sub-hourly smart meter data