key: cord-0425399-0ps9bedf authors: Wang, Ziyun; Wang, Hao title: Improving Load Forecast in Energy Markets During COVID-19 date: 2021-10-01 journal: nan DOI: 10.1145/3486611.3486648 sha: 042376fd68a41662f703906d7b428c213ba55e89 doc_id: 425399 cord_uid: 0ps9bedf The abrupt outbreak of the COVID-19 pandemic was the most significant event in 2020, which had profound and lasting impacts across the world. Studies on energy markets observed a decline in energy demand and changes in energy consumption behaviors during COVID-19. However, as an essential part of system operation, how the load forecasting performs amid COVID-19 is not well understood. This paper aims to bridge the research gap by systematically evaluating models and features that can be used to improve the load forecasting performance amid COVID-19. Using real-world data from the New York Independent System Operator, our analysis employs three deep learning models and adopts both novel COVID-related features as well as classical weather-related features. We also propose simulating the stay-at-home situation with pre-stay-at-home weekend data and demonstrate its effectiveness in improving load forecasting accuracy during COVID-19. At the end of 2019, a novel infectious coronavirus (named was identified, which quickly developed into a worldwide pandemic in 2020 [22] . Many governments issued stay-at-home (or shelter-in-Place) orders to contain the spread of the coronavirus, asking people to self-quarantine until they had the disease under control [14, 21] . The sudden changes in daily life had an impact on the electricity industry around the world, and many countries have observed significant changes in electricity consumption patterns [11] . Researchers in [1, 7, 17] have attempted to record and review the impact of COVID-19 on the electricity industry, but there was a research gap on load forecasting during COVID-19. It is not well understood how the load forecast performs during an emergency event like the COVID-19 pandemic and how to improve the performance of load forecasting. We are thus motivated to bridge the research gap by systematically evaluating models and features that can be used to improve load forecasting amid COVID- 19 . During the pandemic, decreasing electricity consumption was observed worldwide [11] . Amid COVID-19, a study in Ontario, Canada revealed that the overall electricity demand has declined by 14% in April 2020 [1] , Spain displayed a decreased power demand of 13.49% due to the lockdown measures [17] , while in the US, overall electricity generation declined after the issue of stay-at-home advisories [7] . The limit on mobility adds further complexity to electricity consumption behaviors. Under the full lockdown scenario, increased residential consumption and decreased industrial electricity consumption were observed in a lot of places in the world, such as Lagos, Nigeria [6] , Kragujevac, Serbia [5] , China [4] , and New York, US [2] . The study on the residential electricity consumption in Kragujevac, Serbia also identified an increase in consumption for heating and electrical devices brought about by the increased presence of residents in their households [5] . Despite having such volatile load demand profiles during the pandemic, only a few studies were conducted on improving load forecasting accuracy. Among them was a study by Chen et al. [3] , according to which mobility data forms a useful feature for load forecasting during COVID-19, which will be examined in this paper as well. Short-term load forecasting is an essential tool for operating reserve [18] , which is an extra generating power prepared for any form of supply disruption. Since the operating reserve is at least the capacity of the largest supplier plus a fraction of the peak load [19] , underestimation of peak load can be disastrous, while overestimation will cause utility companies to waste extra resources in larger operating reserve [13] . Load forecasting accuracy is crucial for utility companies to avoid further loss with the fickle energy consumption during the pandemic. This paper aims to systematically analyze load forecasting models and relevant features that can improve load forecasting in an emergency event like COVID-19. We take the energy market of New York Independent System Operator (NYISO) in the US as our case study, as the New York state was at the epicenter of the pandemic during the first wave in 2020 [12] . Since we target the wholesale market instead of regional systems, the mix of electric and gas heating is not changing significantly and is not explicitly considered in our model. The main contributions of this paper are the follows. • This paper is among the first to systematically evaluates the effectiveness of COVID-19 related features and deep learning models for load forecasting during COVID-19. • We propose using pre-stay-at-home weekend data as extra training data, which effectively simulates post-stay-at-home situations and improves load forecasting accuracy when COVID-19 hits. Our work suggests a significant improvement of load forecasting methods in an emergency like the pandemic. Insights such as simulating stay-at-home situations with weekend data and adopting mobility features for load forecasting during COVID-19 can help stakeholders, such as system operators and utilities, better prepare for similar contingencies in the future. The data used for load forecasting in this paper is imported from COVID-EMDA+ data hub [16] and COVID-19 Community Mobility Reports (CCMR) [8] , and we introduce the data in detail as follows. Load data is at the core of load forecasting, where indicates time step. We extract load data from the COVID-EMDA+ data hub, which contains hourly load profiles measured in MW for the NYISO energy market. Weather and COVID case data in Section 2.2 are also from the same data hub and will be introduced later. The stay-at-home order on March 22nd, 2020 in New York state [14] splits into pre-stay-at-home and post-stay-at-home . Figure 1 shows lower energy demand in NYISO after the stay-at-home order compared to the same period in 2019. Since the load data dates back to the earliest in 2017, we use data from Jan 1st, 2017 to May 31st, 2020. The major impact of stay-at-home order causing people not reporting to work resembles the situation of weekends. Hence, we also propose using pre-stay-at-home weekend data to simulate the pattern of in Section 3.2. Traditional load forecasting utilized weather as auxiliary information. We acquire hourly weather data , including air temperature, dew point temperature, wind speed, and relative humidity. To mimic the effect of day-ahead weather forecasts, we shift the weather data backward by 24 hours. We evaluate the effectiveness of COVID case data and mobility data in assisting load forecast. includes daily new confirmed and death cases, signaling the severity of the pandemic. contains daily workplace and residential mobility, indicating the impact of quarantine and work-from-home policies. is obtained from [8] , measuring percentage change in visitors to workplaces or time spent in residential areas with respect to the median value from a 5-week period, from Jan 3rd to Feb 6th, 2020. While and are daily records, and are measured hourly. For consistency in format, each data entry in and is repeated 24 times to simulate hourly measurements. Moreover, the first non-zero data entry of and is in Jan and Feb 2020, respectively, later than that of and . Therefore, we fill the empty data entries in and with zeros so that their sizes match that of and . Load forecasting can employ statistical models for time series analysis, or deep learning models that are promising in tackling sequential data [9] . We apply both statistical models, such as Auto-Regressive Integrated Moving Average (ARIMA) and Deep Neural Networks, including FCDNN, LSTM, and GRU, to load forecasting and evaluate their performance. We also evaluate the performance of pre-stay-at-home weekend training data as suggested in Section 2.1 against that of ordinary training data . We need to build a benchmark model by mimicking the scenario, in which no changes are made to improve load forecasting accuracy in the face of COVID-19. We train the aforementioned models with a classical feature set, including load and weather , to produce day-ahead load forecasts for for 10 weeks. The forecast results are then evaluated against actual values using the Mean Absolute Percentage Error (MAPE), where and are actual and forecast data points at time step , respectively, and is the number of fitted points. FCDNN, LSTM, and GRU exhibit comparable performance as demonstrated in Figure 2 , achieving an overall MAPE of 4.48%, 5.32%, and 4.85% for the 10-week period after the stay-at-home order, respectively. Although the overall accuracy of the three models seems to remain in the acceptable range, they are much worse than their performance under the ordinary pre-stay-at-home situation, in which they achieved 2.97%, 3.22%, and 2.78%, respectively. In Figure 2 , the daily forecasting MAPEs also fluctuate drastically for all three models, reaching high peaks > 10% around day 40 to 60, suggesting volatile performance forecasting unusual data. Among all the evaluated models, ARIMA is unable to produce meaningful results as the resultant accuracy is much lower than that of the other three models. Thus we will not consider ARIMA in the actual analysis. Since LSTM has been demonstrated to be effective in load forecasting per [10, 15, 20] , we choose the prevalent LSTM model as our benchmark result for later comparisons. As mention in Section 2.1, we propose to use to capture the impact of the Stay-at-home policy. We believe that improves the efficiency of training by removing the influence of weekday data which are more dissimilar to . To verify this hypothesis, we trained FCDNN, LSTM, and GRU with and and used them to perform day-ahead load forecasts for for 10 weeks, and the resultant MAPEs are compared against our benchmark result in Section 3.1. As shown in Figure 3 and Table 1 , FCDNN, LSTM, and GRU models trained with and showed comparable performance with a significant improvement in accuracy from the benchmark results, achieving an overall MAPE of 4.04%, 3.85%, and 4.13% respectively forecasting 10 weeks of . This shows that simulating with is an effective measure in improving load forecasting accuracy during COVID-19. Therefore, in the following sections, will be used instead of as training data. Using the NYISO energy market as a case study, we aim to systematically compare the effectiveness of different feature sets, , and , and different models, including FCDNN, LSTM, and GRU, in load forecasting during COVID-19. We propose using COVID-related features such as COVID cases and mobility as indicators of notable changes during the pandemic, and compare their effectiveness against as auxiliary features for the day-ahead forecasts of . Since and are blank for the pre-stay-at-home period, we have to provide more information of and in order to train the models. Therefore, we employ a rolling forecast technique, such that is added to the training data as stated in Section 3.2, where denotes the week index after the stay-at-home order and = { | 7( −1) ≤ ≤ 7 }, to produce day-ahead load forecasts for +1 . Both and experience drastic changes immediately after stay-at-home order, so day-ahead forecasts for =1 using and will reasonably display high inaccuracies. This gives and a natural disadvantage, which is their disability to be used immediately after stay-at-home order. As shown in Figure 4 , the rolling forecast technique is vulnerable to distortion during day 40 to 50 (in Week 7) after the stay-at-home order, across all auxiliary features used. Compared to Figure 3 , rolling forecast trained with the same feature performs comparably well for the undistorted days, but the additional information from =6 hindered the judgment of the models, causing soaring forecasting MAPEs. Moving on, as an auxiliary feature has displayed a certain level of effectiveness in load forecasting during COVID-19. Compared to benchmark results, shows significant improvement in load forecasting accuracy except for =7 and =10 and has comparable performance to that of during the same period. This coincides with the insights from [3] that mobility data can be an effective feature in load forecasting during COVID-19. On the other hand, is rather baleful than helpful in load forecasting for , showing a higher MAPE than benchmark result across almost the entire 10-week period. Even if the COVID case data is capable of indicating latent changes during the pandemic, it is less directly related to load demand profiles. It hence is less effective than and when used in load forecasting. Using the NYISO market as a case study, we evaluated models and features that can be used to improve the accuracy of load forecasting amid COVID-19. We provided insights into improving load forecast in future contingencies similar to COVID-19 as follows. • We used the pre-stay-at-home weekend data to simulate the stay-at-home situation, and showed that it effectively improved load forecasting accuracy compared to using ordinary pre-stay-at-home training data. • We then adopted auxiliary features, including COVID cases and mobility data, and compared their performance with the classical weather feature. In comparison, the mobility feature showed improvements in forecast accuracy though slightly, the COVID feature was clearly not as helpful. • Across all scenarios analyzed as stated above, we evaluated the performance of deep learning models, including FCDNN, LSTM, and GRU, which showed comparative performance. Our key finding is that pre-stay-at-home weekend load data led to significant improvement in load forecast during COVID-19 and can be easily obtained compared to COVID cases and mobility data. In our future work, we plan to explore data from more energy markets with distinct energy consumption behaviors and evaluate the various combinations of different feature sets. We are also interested in exploring potential causes for the distorted rolling forecasts as shown in Section 4.1. This work is in part supported by the FIT Academic Staff Funding of Monash University. The authors thank anonymous reviewers for their helpful comments and suggestions and Dr. Yize Chen for informative discussions. Analysis of the electricity demand trends amidst the COVID-19 coronavirus pandemic Coronavirus comes home? Energy use, home energy management, and the social-psychological factors of COVID-19 Using Mobility for Electrical Load Forecasting During the COVID-19 Pandemic COVID-19 and household energy implications: what are the main impacts on energy use? Heliyon Impact of people's behavior on the energy sustainability of the residential sector in emergency situations caused by COVID-19 Energy transition in a lockdown: An analysis of the impact of COVID-19 on changes in electricity demand in Lagos Nigeria Assessment of the COVID-19 pandemic effect on regional electricity generation mix in NYISO, MISO, and PJM markets Google. 2021. COVID-19 Community Mobility Report Load forecasting using deep neural networks Short-Term Load Forecasting Using an LSTM Neural Network Covid-19 impact on electricity coronavirus cases: Tracking deaths, confirmed cases by state Short term electricity load forecasting for institutional buildings Timing of State and Territorial COVID-19 Stay-at-Home Orders and Changes in Population Movement -United States Short-Term Load Forecasts Using LSTM Networks A Cross-Domain Approach to Analyzing the Short-Run Impact of COVID-19 on the US Electricity Sector Electricity demand during pandemic times: The case of the COVID-19 in Spain Load forecasting, the importance of the probability "tails" in the definition of the input vector Operating reserve model in the power market A Short-Term Residential Load Forecasting Model Based on LSTM Recurrent Neural Network Considering Weather Features COVID-19 pandemic in China: Context, experience and lessons A pneumonia outbreak associated with a new coronavirus of probable bat origin