Siyuan Lu

Machine learning based multi-physical-model blending for enhancing renewable energy forecast - improvement via situation dependent error correction

With increasing penetration of solar and wind energy to the total energy supply mix, the pressing need for accurate energy forecasting has become well-recognized. Here we report the development of a machine-learning based model blending approach for statistically combining multiple meteorological models for improving the accuracy of solar/wind power forecast. Importantly, we demonstrate that in addition to parameters to be predicted (such as solar irradiance and power), including additional atmospheric state parameters which collectively define weather situations as machine learning input provides further enhanced accuracy for the blended result. Functional analysis of variance shows that the error of individual model has substantial dependence on the weather situation. The machine-learning approach effectively reduces such situation dependent error thus produces more accurate results compared to conventional multi-model ensemble approaches based on simplistic equally or unequally weighted model averaging. Validation over an extended period of time results show over 30% improvement in solar irradiance/power forecast accuracy compared to forecasts based on the best individual model.

PAIRS: A scalable geo-spatial data analytics platform

Geospatial data volume exceeds hundreds of Petabytes and is increasing exponentially mainly driven by images/videos/data generated by mobile devices and high resolution imaging systems. Fast data discovery on historical archives and/or real time datasets is currently limited by various data formats that have different projections and spatial resolution, requiring extensive data processing before analytics can be carried out. A new platform called Physical Analytics Integrated Repository and Services (PAIRS) is presented that enables rapid data discovery by automatically updating, joining, and homogenizing data layers in space and time. Built on top of open source big data software, PAIRS manages automatic data download, data curation, and scalable storage while being simultaneously a computational platform for running physical and statistical models on the curated datasets. By addressing data curation before data being uploaded to the platform, multi-layer queries and filtering can be performed in real time. In addition, PAIRS offers a foundation for developing custom analytics. Towards that end we present two examples with models which are running operationally: (1) high resolution evapo-transpiration and vegetation monitoring for agriculture and (2) hyperlocal weather forecasting driven by machine learning for renewable energy forecasting.

A Methodology for Quantifying Reliability Benefits from Improved Solar Power Forecasting in Multi-Timescale Power System Operations

Solar power forecasting improvements are mainly evaluated by statistical and economic metrics, and the practical reliability benefits of these forecasting enhancements have not yet been well quantified. This paper aims to quantify reliability benefits from solar power forecasting improvements. To systematically analyze the relationship between solar power forecasting improvements and reliability performance in power system operations, an expected synthetic reliability (ESR) metric is proposed to integrate multiple state-of-the-art independent reliability metrics. The absolute value and standard deviation of area control errors (ACEs), and the North American Electric Reliability Corporation Control Performance Standard 2 (CPS2) score are calculated through a multi-timescale scheduling simulation, including the day-ahead unit commitment, real-time unit commitment, real-time economic dispatch, and automatic generation control sub-models. The absolute ACE in energy, CPS2 violations, CPS2 score, and standard deviation of the raw ACE are all calculated and combined as the ESR metric. Numerical simulations show that the reliability benefits of multi-timescale power system operations are significantly increased due to the improved solar power forecasts.

Toward large-scale crop production forecasts for global food security

Predicting crop production plays a critical role in food price forecasting and mitigating potential food shortages. Crop models may require parameters from, for example, weather, crop genotype, farm management, and soil. Sources for these data are often found in very different places. Researchers spend a significant amount of time to collect and curate them. In addition, in order to scale yield forecasts from the single-farm level up to the continental scale, crop models have to be coupled with a geospatial big data platform to provide the required data inputs. In a proof-of-concept case study, we investigate the coupling of a scalable geospatial big data platform, Physical Analytics Integrated Repository and Services (PAIRS), to the Decision Support System for Agrotechnology Transfer (DSSAT) crop model. We envision running this system on a global scale. For geospatial analytics, PAIRS provides curation of heterogeneous data sources to simulate crop models using hundreds of terabytes of data.

On the usefulness of solar energy forecasting in the presence of asymmetric costs of errors

Because of the weather-associated variability of renewable energy generation, forecasting is an inherent component of an overall solution to reduce the grid integration cost of renewable energy. Accuracy of a forecast is characterized typically by metrics such as root mean square error (RMSE) or mean absolute error (MAE). Such metrics, however, may not comprehensively capture the usefulness of a forecast. Use cases of forecasts are usually complex and are connected to how energy producers, balancers, or traders may apply the forecast to minimize some cost functions in order to improve performance. Often a cost function is asymmetric, unlike RMSE or MAE. Here, we treat complex cost functions as asymmetric perturbations to the symmetric RMSE metric and ask how a forecast can be statistically corrected to minimize the cost function. The analysis leads to an analytical expression for one aspect of a forecasts usefulness, which characterizes the capability of a user to benefit from the knowledge of the asymmetry of the cost function. As a case study, we present a comparison of solar forecasts derived from a number of numerical weather predictions at a test site in Rutland, Vermont.

Baseline and target values for PV forecasts: Toward improved solar power forecasting

Accurate solar power forecasting allows utilities to get the most out of the solar resources on their systems. To truly measure the improvements that any new solar forecasting methods can provide, it is important to first develop (or determine) baseline and target solar forecasting at different spatial and temporal scales. This paper aims to develop baseline and target values for solar forecasting metrics. These were informed by close collaboration with utility and independent system operator partners. The baseline values are established based on state-of-the-art numerical weather prediction models and persistence models. The target values are determined based on the reduction in the amount of reserves that must be held to accommodate the uncertainty of solar power output.

Bottom-Up Estimation and Top-Down Prediction: Solar Energy Prediction Combining Information from Multiple Sources

Accurately forecasting solar power using the data from multiple sources is an important but challenging problem. Our goal is to combine two different physics model forecasting outputs with real measurements from an automated monitoring network so as to better predict solar power in a timely manner. To this end, we propose a new approach of analyzing large-scale multilevel models with great computational efficiency requiring minimum monitoring and intervention. This approach features a division of the large scale data set into smaller ones with manageable sizes, based on their physical locations, and fit a local model in each area. The local model estimates are then combined sequentially from the specified multilevel models using our novel bottom-up approach for parameter estimation. The prediction, on the other hand, is implemented in a top-down matter. The proposed method is applied to the solar energy prediction problem for the U.S. Department of Energy’s SunShot Initiative.

A Solar Time-based Analog Ensemble Method for Regional Solar Power Forecasting

This paper presents a new analog ensemble method for day-ahead regional photovoltaic (PV) power forecasting with hourly resolution. By utilizing open weather forecast and power measurement data, this prediction method is processed within a set of historical data with similar meteorological data (temperature and irradiance), and astronomical date (solar time and earth declination angle). Furthermore, clustering and blending strategies are applied to improve its accuracy in regional PV forecasting. The robustness of the proposed method is demonstrated with three different numerical weather prediction models, the North American mesoscale forecast system, the global forecast system, and the short-range ensemble forecast, for both region level and single site level PV forecasts. Using real measured data, the new forecasting approach is applied to the load zone in Southeastern Massachusetts as a case study. The normalized root mean square error has been reduced by 13.80% to 61.21% when compared with three tested baselines.

A machine-learning approach for regional photovoltaic power forecasting

This paper presents a machine-learning approach for regional photovoltaic (PV) power forecasting of up to 2 days ahead with hourly resolution. Physical PV power model is aggregated by geographical clusters and then summed for an entire ISO load zone. Numerical weather prediction (NWP) forecasts provide parameters, such as irradiance, temperature, barometric pressure, and wind speed, which are used as inputs to calculate plane of array (POA) irradiance and PV output power. A machine-learning approach is then developed. Bias correction for calculated power is conducted using linear regression method. During this procedure, categorization in accordance to critical parameters is employed to obtain a fine approximation. With optimized blending coefficients, adaptive mixture of correction results following different NWP methods is introduced to obtain an intelligent and adaptable output power PV forecast. A case study for the period from June 12, 2014 to January 24, 2015 of Southeastern Massachusetts (SEMA) load zone is carried out. Normalized Root Mean Square Error (NRMSE) is 5.28% for day-ahead forecast horizon, which is reduced by 30.6% compared to the baseline that the best individual model is used.

Solar radiation forecast with machine learning

Renewable energy forecasting becomes increasingly important as the contribution of solar/wind power production to the electrical power grid constantly increases. Significant improvement in forecasting accuracy has been demonstrated by developing more sophisticated solar irradiance forecasting models using statistics and/or numerical weather predictions. In this presentation, we report the development of a machine-learning based multi-model blending approach for statistically combing multiple meteorological models to improve the accuracy of solar power forecasting. The system leverages upon multiple existing physical models for prediction including numerous atmospheric and cloud prediction models based on satellite imagery as well as numerical weather prediction (NWP) products.