Electrical Energy Prediction in Residential Buildings for Short-Term Horizons Using Hybrid Deep Learning Strategy

Abstract

Smart grid technology based on renewable energy and energy storage systems are attracting considerable attention towards energy crises. Accurate and reliable model for electricity prediction is considered a key factor for a suitable energy management policy. Currently, electricity consumption is rapidly increasing due to the rise in human population and technology development. Therefore, in this study, we established a two-step methodology for residential building load prediction, which comprises two stages: in the first stage, the raw data of electricity consumption are refined for effective training; and the second step includes a hybrid model with the integration of convolutional neural network (CNN) and multilayer bidirectional gated recurrent unit (MB-GRU). The CNN layers are incorporated into the model as a feature extractor, while MB-GRU learns the sequences between electricity consumption data. The proposed model is evaluated using the root mean square error (RMSE), mean square error (MSE), and mean absolute error (MAE) metrics. Finally, our model is assessed over benchmark datasets that exhibited an extensive drop in the error rate in comparison to other techniques. The results indicated that the proposed model reduced errors over the individual household electricity consumption prediction (IHEPC) dataset (i.e., RMSE (5%), MSE (4%), and MAE (4%)), and for the appliances load prediction (AEP) dataset (i.e., RMSE (2%), and MAE (1%)).

1. Introduction

The electric power industry plays an important role in the economic development of a country, and its decisive operation provides significant societal wellbeing. As reported in [1] the global energy consumption is increasing for sustainable advancement in society; therefore, the effectiveness of electricity consumption prediction needs to be improved [2]. As reported by the World Energy Outlook in 2017, the compound annual growth rate (CAGR) of electricity demand will have a global incremental rise of 1.0% in the period of 2016–2040 [3]. Another report presented in [4] described that residential buildings generally consume 27% of the total energy consumption, whereas buildings in the United States (US) consume 40% of their national energy [5]. Owing to high energy consumption levels in residential buildings, efficient management of their consumption is essential. Therefore, proper planning of energy is vital for energy saving, which is possible through effective energy consumption prediction models.

The electricity prediction strategies for short-term horizons are categorized into four types: very-short, short, medium, and long-term [6,7]. Short and very short-term predictions refer to minutely ahead predictions up to several days. Medium-term load prediction means one week ahead prediction or up to a year, whereas annual or several years’ ahead prediction is called very long-term prediction. Each electricity prediction horizon has its own applications; however, in this study, we focus on short-term and very-short-term predictions [6,8,9,10]. The major applications of short-term electric load prediction are power plant’s reliable and secure operation, reliability and economic dispatch, and power system generation scheduling. Short-term load prediction ensures power system security, and it is an essential tool for the determination of the optimal operational state. Reliability and economic dispatch are also important applications of power systems in short-term horizons, wherein the abrupt variation of load demand fluctuates its reliability. This causes a power supply shortage if the load demand is underestimated, which makes it difficult to manage overload conditions and the quality of the overall power supply system. Another application of short-term load prediction is generation scheduling, which can be achieved through accurate load prediction to verify the allocation of operational limitations, generation resources, equipment usage, and environmental constraints. In the literature, several studies have been conducted for short-term load prediction, electricity load forecasting, electricity demand forecasting, electricity storage, and occupant behavior [11,12,13,14,15]. The mainstream electricity load prediction models are mainly grouped into two categories: statistical models and artificial intelligence models [16,17]. Statistical models such as Auto Regressive Integrated Moving Average (ARIMA) [18], linear regression [19], Kalman filtering [20], and clustering [21,22] etc. were used for load prediction in the early days. These models are effective in learning linear data but inadequate to learn the nonlinear complex electricity load. Besides this, the artificial intelligence models can learn nonlinear complex and linear electricity loads, which are further divided into shallow and deep structure methods. Shallow based methods include random forest [23], wavelet neural networks [24], support vector machines (SVMs) [25], artificial neural networks (ANN) [26], and extreme learning machines [27]. Shallow based methods perform well compared to statistical methods but perform poorly in feature mining. Hence, these methods require more features and selects strong features to enhance the prediction accuracy. Obtaining optimal feature extraction is a challenging task for these methods. Further, these methods have insufficient generalization ability over different datasets due to small hypothesis owing to less number of parameters. Deep structure methods have the ability to address the aforementioned concerns of shallow-based methods using multi-layer processing and hierarchical feature learning from electricity historical data. Recently, recurrent neural networks (RNNs) and convolutional neural networks (CNNs) are two powerful architecture proposed in the literature for the analysis of time series data. For instance, Amarasinghe et al. [28] developed a CNN-based methodology for electricity load forecasting and compared their results with a factored restricted Boltzmann machine, sequence-to-sequence long short term memory (LSTM), support vector regressor (SVR), and ANN. Another study presented in [29] proposed a deep CNN network for day-ahead load forecasting and compared the results with an extreme learning machine, ARIMA, CNN, and RNN. Several studies also used RNN models for electricity load prediction, whereas Tokgoz et al. [30] used RNN, gated recurrent unit (GRU), and LSTM models for electricity load prediction in Turkey and extensively decreased the error. Furthermore, the authors of [31] developed an LSTM-based model for periodic energy prediction and compared their results with other models. Another study presented in [32] developed an RNN-based model for medium- and long-term electricity load prediction.

The electricity consumption data is time-series data, which comprises spatial and temporal information. The CNN models perform well for spatial information extraction, but insufficient for temporal information, whereas the RNN models are insufficient for spatial information and can learn temporal information. Therefore, to develop an optimal model for electricity load prediction, hybrid models are introduced in the recent literature. For instance, Kim et al. [33] developed a hybrid model combining CNN with LSTM for short-term load prediction and compared their results with GRU, attention LSTM, LSTM, and bidirectional LSTM. Ullah et al. [34] also developed a hybrid model with a combination of CNN and multi-layer bidirectional LSTM and compared their results with bidirectional LSTM, LSTM, and CNN-LSTM. Similarly, another study presented in [17] integrated a CNN with an LSTM auto-encoder and compared the final results with LSTM, LSTM autoencoder, and CNN-LSTM. Moreover, Sajjad et al. [35] and Afrasiabi et al. [36] presented the performance of a CNN-GRU based model for electricity forecasting. The performance of hybrid models is quite promising and has achieved state-of-the-art accuracy; however, further improvements are needed for optimal electricity load prediction. Therefore, in the current study, we established a two-step framework for predicting the electricity load that includes data preprocessing and proposed a hybrid model. In the first step, the historical data of electricity is refined to remove abnormalities that are then passed to the next step CNN along with MB-GRU model for learning. To extract the spatial information, we used CNN layers where MB-GRU is used to learn the temporal information. The contributions of the proposed research are summarized below:

• The electricity consumption data are gathered from smart meter sensors, which include missing values, redundant values, outlier values, etc., due to several reasons, such as faults in meter sensors, variable weather conditions, abnormal customer consumption patterns, etc., which need to be refined before training the model. Therefore, in this work, the input raw datasets are refined before training to fill the missing values and remove the outlier values from the dataset. Similarly, the electricity consumption patterns are of very diverse nature where the neural networks are sensitive to it, so a data normalization technique is applied to bring the dataset into a standard range.
• The mainstream methods use solo models for electricity consumption prediction, which are unable to precisely extract spatiotemporal patterns and have high error rates. Therefore, in this study, we proposed a hybrid model with a combination of CNN and MB-GRU that helps to improve the accuracy of electricity consumption prediction.
• The performance of the model was evaluated using the root mean square error (RMSE), mean square error (MSE), and mean absolute error (MAE). The experimental results show that the proposed model extensively decreases the error rate when compared to baseline models.

The main goal of this work is to improve the prediction accuracy for short-term electrical load prediction in residential buildings, which reduces customer consumption and provides economic benefits. The experimental section shows the effectiveness of the proposed method, which ensures the best performance of the proposed method as compared to other baseline models.

The remainder of the paper is arranged as follows: Section 2 provides a detailed explanation of the proposed method; Section 3 includes the experimental results of the proposed method and comparison with other state-of-the-art models. Finally, the manuscript is concluded in Section 4.

2. Proposed Framework

Accurate electricity load prediction is very important for electricity saving and vital economic implications [37]. As reported by [38] a 1% decrease in the error rate of the electricity prediction model can profit 1.6M dollars and can save 10K MW of electricity annually. For accurate load prediction, an appropriate learning methodology is required. The electricity load prediction models are learned from historical data generated from smart meter sensors. However, some times, due to weather conditions, meter faults, etc., they generate some abnormal data that should be refined before training. Therefore, this work represents a two-step framework that includes data preprocessing and the proposed hybrid model. The preprocessing step refines the input raw data of electricity consumption and then passes it to the proposed model to learn it, as demonstrated in Figure 1, where the details of each step are further discussed in the following sections.

2.1. Data Preprocessing

For better performance of the electricity load prediction models, the training data should be analyzed before training. As previously mentioned, the historical data of electricity consumption include abnormalities that affect the model performance. In this study, we used individual household electricity consumption prediction (IHEPC) and appliances load prediction (AEP) datasets, which consist of missing and outlier values. These abnormalities are removed from the data in the preprocessing step of the proposed framework. For missing values filling, NAN interpolation techniques are used, whereas for outlier values, three sigma rules of thumb [39] are applied. After the outlier reduction and filling missing values, the datasets are normalized using the min–max normalization technique to transform the dataset into a particular range which the neural network can learn easily.

2.2. Proposed CNN and MBGRU Architecture

This work combines a CNN with a multilayered bidirectional GRU (MB-GRU) for short-term electricity load prediction, where the CNN layers are incorporated to extract features from the preprocessed input data, and the MB-GRU model is used to learn the sequences between them. CNN is a neural network architecture that learns in a hierarchical manner whereby each layer learns more and more abstract features. The first layers learn atomic/primitive representations, while the intermediate-level layers learn intermediate abstract representations, and finally, fully connected layers learn high-level patterns. Therefore, the depth of the network is defined by the number of such layers. The higher the layer count, the deeper the network and can learn tiny representations. A CNN is a particular type of deep neural network that employs alternating layers of convolutions and pooling. It contains trainable filter banks per layer. Each individual filter in a filter bank, which is called a kernel and has a fixed receptive field (window) that is scanned over a layer below it, to compute an output feature map. The kernel performs a simple dot product and bias computation as it scans the layer below it and then feeds the result through an activation function, a rectifier, for example, to compute the output map. The output map is then subsampled using sum or max pooling, the latter being more common in order to reduce sensitivity to distortions in the upper layers. This process is alternated up to some point when the features become specific to the problem at hand. Thus, the CNN is a deep neural network for learning increase and more compact features that can later be used for recognition problems. The last few layers in a typical CNN comprise a typical fully connected neural network or support vector machines in order to recognize different combinations of features from the convolutional layers. The CNN architecture is used in different domains, such as image and video recognition [17,35,40,41], language processing [42,43], electricity load forecasting [44,45], crowed counting [46], etc. In the time series domain, the CNN layers are used to extract spatial information and then pass the output into sequential learning algorithms such as RNN LSTM and GRU.

RNN [47] is a sequence learning architecture with backward connections among hidden layers that include some kind of memory and is extensively used in several domains such as natural language processing [48], time series analysis [49], and speech recognition [50], visual data processing [51,52,53], etc. The RNN models generate output at each time stamp from the input data, which leads to the vanishing gradient problem. The RNN model forgets the long sequence of electricity data, such as 60-min resolution, which leads to loss of important information.

$${ʄ}_{Ʈ}=ʘ\left({\mathsf{ώ}}_{ʄ}\cdot \left[{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{ʄ}\right)$$

(1)

$${𝒾}_{Ʈ}=ʘ\left({\mathsf{ώ}}_i\cdot \left[{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{𝒾}\right)$$

(2)

$${𝒸}_{Ʈ}=\tan 𝒽\left({\mathsf{ώ}}_{𝒸}\cdot \left[{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{𝒸}\right)$$

(3)

$${\mathcal{C}}_{Ʈ}={ʄ}_{Ʈ}\times {\mathcal{C}}_{Ʈ-1}+{𝒾}_{Ʈ}\times {𝒸}_{Ʈ}$$

(4)

$${\mathcal{O}}_{Ʈ}=ʘ\left({\mathsf{ώ}}_{\mathcal{O}}\cdot \left[{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{\mathcal{O}}\right)$$

(5)

$${𝒽}_{Ʈ}={\mathcal{O}}_{Ʈ}\times \tan 𝒽(ʘ\left({\mathcal{C}}_{Ʈ}\right).$$

(6)

The problem of losing long sequence information is addressed by LSTM using the three-gate mechanism input, output, and forget. The mathematical representation of each gate is shown in Equations (1)–(6). In these equations the output of each gates is represented through “𝒾”, “ʄ” and “$𝒪$” where “ʘ” represents the activation function. The weights of the gates are represented through “ώ’’, whereas” 𝒽_Ʈ-1” refers to the output of previous LSTM block and “𝒷” represents the bias of the gates. The LSTM structure is complex and computationally expensive due to these gates’ units and memory cells. To overcome the concern of LSTM, another lightweight architecture is developed called GRU [54] which comprises the reset and update gates. The mathematical representation of GRU gates are shown in Equations (7)–(10) where the update gate examines the earlier cell memory to remain active,

$${𝓏}_{Ʈ}=ʘ\left({\mathsf{ώ}}_{𝓏}.\left[{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{𝓏}\right)$$

(7)

$${𝓇}_{Ʈ}=ʘ\left({\mathsf{ώ}}_{𝓇}.\left[{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{𝓇}\right)$$

(8)

$${𝓀}_t=\tan 𝒽\left({\mathsf{ώ}}_{𝒽}.\left[{𝓇}_{Ʈ}·{𝒽}_{Ʈ-1},{𝒶}_{Ʈ}\right]+{𝒷}_{𝒽}\right)$$

(9)

$${𝒽}_{Ʈ}=\left(1-{𝓏}_{Ʈ})·{𝒽}_{Ʈ-1}+{𝓏}_{Ʈ}·{𝓀}_{Ʈ}\right)$$

(10)

and the reset gate merges the next cell input sequence with previous cell memory. In this study, we used MB-GRU, which processes the sequence of input data in both backward and forward directions [55]. The bidirectional RNN models perform better in several domains such as classification, summarization [56], and load forecasting [57]. Therefore, in this study, we incorporate bidirectional GRU layers that contain both backward and forward layers, where the output sequence of the foreword layer is iteratively calculated through input in the positive sequence. The output of the backward layer is calculated through the reverse of the input.

The electricity consumption patterns include spatial and temporal features. Some researchers deployed a solo model that is insufficient to extract both types of features at a time. Therefore, in this work, we established a hybrid model that combines CNN with MB-GRU, as shown in Figure 1b. The proposed hybrid model includes an input layer, CNN layers, and bidirectional GRU layers. Two CNN layers are incorporated after several experiments over different layers and different parameters. Finally, we select filters of 8 and 4 for the first and second CNN layers with a kernel size of 3.1, respectively and used ReLU as an activation function in these layers. After convolutional layers, two bidirectional GRU layers are incorporated to learn the temporal information of the electricity historical data. Finally, the fully connected layers are integrated for the final output prediction.

3. Results and Discussion

In this section, we provide a detailed description of the dataset, evaluation metrics, and experimentation over the IHEPC and AEP datasets, and compare them with other baseline models. The model was trained over a GeFore GTX 2060 GPU with 64 GB RAM using the Keras framework with backend TensorFlow.

3.1. Datasets

The model’s performance is assessed on two benchmark datasets, AEP and IHEPC [58,59]. The AEP dataset was recorded in 4.5 months in a residential house in 10 min resolution. This dataset comprises 29 various parameters of weather information (wind speed, humidity, dew point, temperature, and pressure), light, and appliance energy consumption, as presented in

Table 1. AEP dataset variables, a short description and its units.

S.no	Data Features	Units
1	Appliances: total energy consumption by appliances.	Wh
2	Light: total energy consumption by lights.	Wh
3	T1: demonstrate the temperature of kitchen.	C
4	RH1: demonstrate the humidity in kitchen.	%
5	T2: demonstrate the temperature of living room.	C
6	RH2: demonstrate the humidity of living room.	%
7	T3: demonstrate the temperature of laundry room.	C
8	RH3: demonstrate the humidity of laundry room.	%
9	T4: demonstrate the temperature of office room.	C
10	RH4: demonstrate the humidity of office room.	%
11	T5: demonstrate the temperature of bathroom.	C
12	RH5: demonstrate the humidity of bathroom.	%
13	RH6: demonstrate the outside temperature of building.	C
14	RH6: demonstrate the outside humidity of building.	%
15	T7: demonstrate the temperature of ironing room.	C
16	RH7: demonstrate the humidity in ironing room.	%
17	T8: demonstrate the temperature of teenager room.	C
18	RH8: demonstrate the humidity of teenager room.	%
19	T9: demonstrate the temperature of parent room.	C
20	RH9: demonstrate the humidity of parent room.	%
21	To: demonstrate the outside temperature which are collected from Chievres Weather Station (CWS).	C
22	Pressure: outside pressure which are collected from CWS.	Mm Hg
23	Rho: demonstrate the outside humidity from CWS.	%
24	Wind speed: outside wind speed which are collected from CWS.	m/s
25	Visibility: outside visibility from CWS.	Km
26	Tdewpoint: outside Tdewpoint from CWS.	C

. The data samples were collected from both indoor and outdoor environments through a wireless sensor network. The outdoor data are collected from nearby airport. The building includes 9 indoor and 1 outdoor temperature sensors, 9 humidity sensors in which 7 are integrated with indoor environment and one is in outdoor environment. The outdoor pressure, visibility, temperature, humidity, and dew point are recorded nearby airport region. The IHEPC dataset includes 9 parameters which are; date, time, voltage, global-active-power (GAP), intensity, global-reactive-power (GRP) and three sub-metering as shown in

Table 2. Individual household electricity consumption prediction (IHEPC) dataset variables description and units.

S.no	Data Features	Units
1	Date	dd/mm/yyyy
2	Time	hh: mm: ss
3	GAP	Kw
4	GRP	Kw
5	Voltage	V
6	Global intensity	Amp
7	Submeetring-1	W
8	Submeetring-2	W
9	Submeetring-3	W

. The dataset was recorded in a residential house in France during 2006 and 2010 for one-minute resolution.

3.2. Metrics of Evaluation

To evaluate the performance of the model, we used RMSE, MSE, and MAE metrics. The mathematical representation of these metrics is depicted in Equations (11)–(13). RMSE calculates the difference between all predicted data points and the actual data point, then compute the mean of these square errors and finally calculate the square root of the mean values. The MSE calculates the mean disparity between the actual and model output values. The MAE calculates the mean absolute difference between the actual and predicted values.

$$RMSE=\sqrt{\frac{1}{n}{\sum }_1^n(y-\widehat{y}{)}^2}$$

(11)

$$MSE=\frac{1}{n}{\sum }_1^n(y-\widehat{y}{)}^2$$

(12)

$$\mathrm{MAE}=\frac{1}{n}{\sum }_1^n\left|y-\widehat{y}\right|$$

(13)

3.3. Experimentations over IHEPC, AEP Dataset and Comparison with other Models

In this section, we evaluate the performance comparison of the proposed model with existing models for short-term load prediction (one hour ahead) over the IHEPC and AEP datasets. For the IHEPC dataset, the proposed model achieved 0.42, 0.18, and 0.29 RMSE, MSE, and MAE, respectively. The performance prediction over the test data is displayed in Figure 2a. A comparison of the proposed model over IHEPC dataset for short-term load prediction with other baseline models is shown in Figure 3. For more detail the performance of the proposed model is compared with [1,17,33,34,35,60,61] in the short-term horizon. In [60] the authors used deep learning methodology for residential load prediction and obtained 0.79 RMSE and 0.59 MAE, whereas [33] used a CNN-LSTM hybrid network for short-term residential load prediction and achieved 0.59, 0.35, and 0.33 RMSE, MSE, and MAE, respectively. 0.47, 0.19 and 0.31 RMSE, MSE and MAE was reported in [17] whereas [34] reports 0.56, 0.31 and 0.34 values for these metrics. In [61] the authors achieved 0.38 MSE and 0.39 MAE, whereas [1] reported 0.66 RMSE. Another strategy presented in [35] attained 0.47, 0.22, and 0.33 RMSE, MSE, and MAE, respectively. Among these results, the proposed model achieved the lowest error rate for short-term electric load prediction.

Furthermore, the effectiveness of the proposed model is evaluated over the AEP dataset for a short-term horizon. For the AEP dataset, the proposed model attained 0.31, 0.10, 0.33 RMSE, MSE, and MAE, respectively, whereas the prediction results are shown in Figure 2b. Similarly, the effectiveness of the proposed model over the AEP dataset is also compared with other baseline models, as shown in Figure 4. For instance, the results are compared with [34,62,63,64]. For further details, Ref. [62] achieved 0.59 RMSE and 0.26 MSE, Ref. [63] achieved 0.35 RMSE and 0.66 MSE, and [64] achieved a 0.59 RMSE. In [35], authors achieved 0.31, 0.09, and 0.24 scores for RMSE, MSE, and MAE, respectively. Compared to these models, the proposed model performed better in reducing the RMSE and MAE error rate, while only the results of Sajjad et al. [35] in terms of MSE is better than the proposed model in the short-term horizon.

4. Conclusions

In this study, we established a two-step methodology for short-term load prediction. In the first step, we performed data preprocessing over raw data to refine it for training. The refinement of the data is important because the historical data of energy consumption is generated from smart meter sensors, which include abnormalities such as outliers, missing values etc. These abnormalities from the raw data are extracted in this step, and finally, the normalization technique is applied to transform the data into a specific range. The second step is the hybrid model, which is a combination of CNN and multilayered bi-directional GRU (MB-GRU). The CNN layers are incorporated to extract important features from the refined data, while the MB-GRU layers are used to learn the temporal information of electricity consumption data. The proposed methodology is tested over two challenging datasets and achieves better performance when compared to other methods, as demonstrated in the results section.