As energy demand grows globally, the energy management system (EMS) is becoming increasingly important. Energy prediction is an essential component in the first step to create a management plan in EMS. Conventional energy prediction models focus on prediction performance, but in order to build an efficient system, it is necessary to predict energy demand according to various conditions. In this paper, we propose a method to predict energy demand in various situations using a deep learning model based on an autoencoder. This model consists of a projector that defines an appropriate state for a given situation and a predictor that forecasts energy demand from the defined state. The proposed model produces consumption predictions for 15, 30, 45, and 60 minutes with 60-minute demand to date. In the experiments with household electric power consumption data for five years, this model not only has a better performance with a mean squared error of 0.384 than the conventional models, but also improves the capacity to explain the results of prediction by visualizing the state with t-SNE algorithm. Despite unsupervised representation learning, we confirm that the proposed model defines the state well and predicts the energy demand accordingly.
Keywords: electric energy; energy prediction; energy management system; deep learning; autoencoder; explainable AI
關鍵詞：電能；能量預測；能源管理系統；深度學習；自動編碼器；可解釋的人工智能
1. Introduction
As industrialization has progressed globally and the industry has developed, the demand for energy has become so high that energy has become an important topic in national policy [1]. In addition, energy use is rapidly increasing due to economic growth and human development [2]. The causes of these phenomena can be attributed to uncontrolled energy use such as overconsumption, poor infrastructure, and wastage of energy [3]. Among the demanders of various energy sources, Streimikiene estimates that residential energy consumption will account for a large proportion by 2030 [4]. According to Zuo, 39 % of the United States’ total energy use is referred to as building energy consumption [5]. An energy management system (EMS) like a smart grid has been proposed to control the demand for soaring energy.
One work cycle of the EMS is the Plan-Do-Check-Act (PDCA) cycle as depicted in Figure 1 [6]. Formulating an energy plan is the first thing to do. This is the decision of the initial energy baseline, the energy performance indicators, the strategic and operative energy objectives, and the action plans. In the “do” phase, planning and action take place. The plans conducted in the previous phase have to be checked to ensure that they are effective. In the last phase, the results are reviewed, and a new strategy is established. Among the four stages, the “plan” phase is very important because it is the stage of establishing an energy use strategy and it includes an energy demand forecasting step. Therefore, it is necessary to study the energy prediction model to construct an efficient EMS.
Many researchers have conducted studies with various methods to predict energy demand. In the past, machine learning techniques such as the support vector machine (SVM) and linear regression (LR) have been widely used. However, as shown in Figure 2a, energy demand values over time are complex and noisy, which limits performance. As depicted in Figure 2b, the Fourier transform to analyze patterns of energy demand reveals that it has complex features. For quantitative analysis, t-test and ANOVA were performed on the dataset used in this paper as shown in Table 1. In the statistical analysis using the t-test, two groups (e.g., two different months in monthly demand) are chosen randomly and computed p-values, and we compute the average of all possible sampling. In the case of using ANOVA, p-value is computed from all the groups in each month, date, and hour.
許多研究人員已經用各種方法進行了研究，以預測能源需求。過去，諸如支持向量機（SVM）和線性迴歸（LR）的機器學習技術已被廣泛使用。但是，如圖2a所示，能源需求值隨時間變化既複雜又嘈雜，從而限制了性能。如圖2b所示，對能源需求模式進行分析的傅立葉變換表明它具有複雜的功能。爲了進行定量分析，對本文使用的數據集進行了t檢驗和ANOVA（方差分析(Analysis of Variance，ANOVA)），如表1所示。在使用t檢驗的統計分析中，隨機選擇了兩組（例如，每月需求量不同的兩個月），計算p值，然後計算所有可能採樣的平均值。在使用ANOVA的情況下，將從每個月，日期和小時中的所有組計算p值。
As the result of our analysis that the characteristics of energy demand are complex, we have conducted the research with deep learning to extract difficult characteristics and work out tasks. Studies conducted using deep learning have contributed a lot to improving prediction performance, but they have not led to more utility from an energy management system perspective. If the energy demand forecasting model in the EMS copes with various situations and predicts the demand, it will be able to build a more efficient system. Therefore, in this paper, we propose a model that predicts future energy demand with that until now, considers various situations, and predicts energy demand according to different situations. This model consists of a projector that defines the state based on the energy demand to date and a predictor that predicts future energy demand from the defined state. We can adjust the automatically learned state in the middle to predict the energy demand with the consideration of various situations. The summary of the main contribution is as follows.
⚫ We propose a novel predictive model that can be explained by not only predicting future demand for electric power but also defining current demand pattern as state. ⚫ Our model predicts a very complex power demand value with stable and high performance compared with previous studies. ⚫ We analyze the state defined in latent space by the proposed model and investigate a model that predicts the power demand by assuming various explanations.
The rest of the paper is as follows. Section 2 introduces the previous studies for forecasting energy demand and addresses the limitations. To overcome these shortcomings, we propose our model in Section 3. Section 4 shows the results of the energy demand forecasting with the proposed model and also shows the result of forecasting the demand by considering various situations. In the final section, conclusions and discussion are presented.
Several studies have been conducted to predict energy demand mentioned in Section 1. Table 2 summarizes the previous studies. In the past, statistical techniques were used mainly to predict energy demand. Munz et al. predicted a time series of irregular patterns using k-means clustering [7]. Kandananond used different forecasting methods—autoregressive integrated moving average (ARIMA), artificial neural network (ANN), and multiple linear regression (MLR) —to predict energy consumption [8]. Cauwer et al. proposed a method to predict energy consumption using a statistical model and its underlying physical principles [9].
However, due to the irregular patterns of energy demand, statistical techniques have limited performance and many models of prediction using machine learning methods have been investigated. Dong et al. predicted the demand of building energy using SVM with consumption and weather information [10]. Gonzalez and Zamarreno forecasted the next temperature from the temperature to date using a feedforward neural network (NN) and predicted the requirement with the difference of them [11]. Ekici and Aksoy predicted the building energy needs with properties of buildings without weather conditions [12]. Li et al. estimated the annual energy demand using SVM with the building’s transfer coefficient [13]. However, these studies only constructed models to predict correct value corresponding to the input so as to lack the basis for influence of the input features. To solve this problem, Xuemei et al. set the state for forecasting energy consumption through fuzzy c-means clustering and predicted demand with fuzzy SVM [14]. Ma forecasted energy consumption with specific population activities or unexpected events, as well as weather condition as inputs of the MLR model [15]. Although the above studies set the state and forecasted future consumption based on it, they lacked the mechanism to identify the state accurately.
As mentioned in Section 1, the energy consumption data contain large noise. Deep learning, which is a rising method to solve complex tasks of late, is efficient for predicting energy demand because it solves tasks by modeling complex characteristics of data well [16]. Ahmad et al. forecasted energy demand by constructing a deep NN and inputting the information of weather and building usage rate [17]. Lee et al. estimated environmental consumption by using a temporal model like recurrent neural network (RNN) with energy consumption data and temporal features [18]. Li et al. proposed a method to predict energy demand with autoencoder, one of the methods to represent data [19]. However, Li et al.’s model included only fully-connected layers, so that temporal features were ignored, and it is hard to control the conditions because latent space where the features of data are represented is not defined in that model.
Although some of the above studies provided novel research directions, other features such as information of weather and building are used in addition to the energy demand value, which is costly to construct the model for energy consumption prediction. Besides, they lack the explanation capability on the predicted value, because there was no study on the state to analyze the results of prediction. However, studies that analyze and explain the predictive results are essential for practical use of the predictive model. In this paper, we propose a model that visualizes a state by defining the state based on the current usage pattern and date information, so as to be able to explain the results of prediction. Our model, like any other prediction models, takes the energy demand up to now as input and predicts consumption in the future. However, in order to overcome the limitations of the end-to-end system, which cannot analyze the internal prediction process, we add a step to define the state of the demand pattern in the middle.
As mentioned in Sections 1 and 2, nonlinear approaches, including those based on fuzzy and neural net, have demonstrated successful performance in many applications [20–23]. In this paper, we have contributed to the field of application by solving the power demand forecasting problem using the deep neural network-based method. Compared to the previous work, the overall architecture of our model consists of a projector f and a predicter g, similar to an auto-encoder consisting of an encoder and a decoder as shown in Figure 3 [24]. There are many ways to deal with time series data, but f and g are based on long short-term memory (LSTM), one of the RNN’s, to handle time series data [25–28]. Predictor uses the output value of each time-step as the input of the next [29]. The projector defines the state by compressing the energy demand and transferring it to the latent space representing the demand information. Predictor predicts future energy demand based on the defined state. It can be done by end-to-end learning of the projector and the predictor, and the process of defining state is trained automatically. In the variational autoencoder (VAE), the state defined on the latent space contains the feature of the produced data, and also contains the information of the expected energy consumption, as well as features of the input values [30,31]. Ma and Lee predicted the energy consumption by adding more information of the surrounding environment while learning [15,18]. However, unlike them, after learning to predict the consumption with only demand to date, our model can predict future consumption by adjusting the state on the latent space with the condition of the surrounding environment.
The state of the projector is located on the latent space where patterns and features of input energy consumption are shown. Therefore, by controlling the state transferred to the latent space, it is possible to predict the future consumption as well as to analyze the current consumption situation.
This section presents how to use the state set by the projector for forecasting future demand. First, as shown in Equations (10) and (11), the predictor predicts a single consumption value immediately after inputting the state. Recursively, predictor forecasts the next single demand value with the predicted value.
To verify the proposed model, we use a dataset on household electric power consumption [32]. There are about two million minutes of electric energy demand data from 2006 to 2010, and they are divided into training and test data as a 9:1 ratio. It consists of eight attributes including date, global active power (GAP), global reactive power (GRP), global intensity (GI), voltage, sub metering 1, 2, and 3 (S1, 2, and 3), and the model predicts the GAP. S1 corresponds to the kitchen, containing mainly a microwave, an oven, and a dishwasher. S2 corresponds to the laundry room, containing a refrigerator, a tumble-drier, a light, and a washing-machine. S3 corresponds to an air-conditioner and an electric water-heater. The statistical summary of each feature is described in Table 3.
To train the time series model, we use a backpropagation through time (BPTT) algorithm, and the Adam optimizer with default hyper parameters in the keras library of python [33,34]. All weights are initialized with Glorot initialization [35]. The operating system of the computer used in our experiments was Ubuntu 16.04.2 LTS and the central processing unit of the computer was an Intel Xeon E5-2630V3. The random-access memory of the computer was Samsung DDR4 16 GB × 4, and the graphic processing unit of the computer was GTX Titan X D5 12 GB. The number of hidden units in the deep learning approach, including our model (i.e., the size of the state s) was set at 64.
To verify the performance of the proposed model, we show the energy demand forecasting result using our model and compared with other conventional methods. Figure 4 is the result showing real and predicted energy demand values at the same time. The model predicts energy demand for 15, 30, 45, and 60 minutes with actual energy demand for 60 minutes. Although the model could not predict the energy demand perfectly, we confirm that the energy demand pattern predicted well. We show the convergence of the learning algorithm experimentally by showing the change of loss value as learning progresses in Figure 5.
Figure 4. The predicted electric energy consumption and the actual demand. We show the prediction results for (a) 15, (b) 30, (c) 45, and (d) 60 minutes.
Our model is compared with conventional machine learning methods such as linear regression (LR), decision tree (DT), random forest (RF) and multilayer perceptron (MLP), and with deep learning methods such as LSTM, stacked LSTM, and the autoencoder model proposed by Li. Stacked LSTM is a model including two LSTM layers similar to our model but does not set the state. A model proposed by Li has one hundred hidden units and four hidden layers. The MSE measure of the experimental results for each model is shown in Figure 6 as box plot. The results of the comparison with other models show that the proposed model outperforms other models. We can confirm that the conventional machine learning methods (LR, DT, RF, and MLP) show a large variation in prediction performance, but the deep learning methods (LSTM, Stacked LSTM, the Li’s model, and ours) are trained in stable. Some of the deep learning methods are worse than machine learning methods, but our model yields the best performance.
To examine the performance of the prediction model, we use three evaluation metrics—the mean squared error (MSE), the mean absolute error (MAE), and the mean relative error (MRE), which can be calculated respectively as follows.
We empirically verify whether our model can automatically learn a capacity to define state as described in Section 3. We extract the output of the projector to get states and visualize them as shown in Figure 7. We use the t-SNE algorithm to visualize the state [36]. We confirm that the consumption data are not separated clearly by month, but they are clustered by month on the latent space even with the unsupervised representation learning. Approximately the distribution of data can be divided into right (January, February, May, October, and November), left (August and September), top (December), center (March), center-right (June), center-left (July), and center-top (April). We mark plotted points of each month with annotations to figure and illustrate it monthly to show the state for each month, resulting in twelve plots. It can be seen that the defined state is well clustered on a monthly basis, achieving low intra-class variability.
Our model is also empirically confirmed that not only defines the state well but also has the capacity to adjust the prediction by controlling the state on latent space. This method is effective for EMS, for example, because the electricity demand prediction can be made flexible according to the climate or economic situation. The experiment to control the condition of the month is conducted, and other conditions are left for future study. An example of a state transition that controls a state on the latent space is shown in Figure 8. If we want to forecast the energy demand in October with only consumption in April, we just project the demand in April into the latent space to extract the state. After extracting the state for the electric energy consumption at one point in April, we add the average value of the states for October v(xOCT) and subtract the average value of the states for the April v(xAPR) and put it into the predictor to get the predicted values.
Table 5 shows the average value of the predicted electric power consumption for one hour in minutes to determine whether the demand pattern for April is changed to the consumption pattern for October after the state transition. Each column shows the month of the input electric energy demand to date, and each row shows an output month of state transition to predict the desired pattern for the specified month. The ground truth (GT) is the average electric energy consumption for each month. It can adjust the state on the latent space because the predicted consumption after conditioning is similar to GT.
Table 5. The results of experiments on state transition. Each column shows the month of the input electric energy demand and each row shows an output month after state transition. The ground truth (G.T.) is the average electric energy consumption for each month.
We have addressed the importance of energy demand prediction and proposed a model to solve them. It attempts to predict electric energy consumption through defining the state unlike the conventional machine learning or deep learning models. Divided into two parts (projector and predictor), each part interacts with the other and learns to automatically set a state without any supervision. We achieve the best forecasting performance compared to others, analyze the state, and peek the basis for the predicted consumption value. In addition, the state transition method shows that our model can be more efficient because we can control the predicted electric energy consumption values according to various situations by adjusting conditions. For example, if we add several conditions to the state such as information of weather or economy, we can predict electricity demand accordingly.
In this paper, we have conducted experiments with several conditions of the state only for the month. We will experiment with various conditions such as weather, economy, or any other events in the future works. In this paper, only the energy consumption of one individual household is predicted, so the demand of several buildings will be collected, and we will add the information about building into the state and have a plan to propose a model which can predict energy consumption of various buildings. Finally, we will construct an efficient energy management system including the proposed prediction model.
