Deep Learning for Inferring the Surface Solar Irradiance from Sky Imagery
Mehdi Zakroum, Mounir Ghogho, Mustapha Faqir, Mohamed Aymane Ahajjam
DDeep Learning for Inferring the Surface SolarIrradiance from Sky Imagery
Mehdi Zakroum ∗ , Mounir Ghogho ∗‡ , Mustapha Faqir ∗ and Mohamed Aymane Ahajjam ∗∗ International University of Rabat, TICLab, Morocco ‡ University of Leeds, School of IEEE, UK { mehdi.zakroum,mounir.ghogho,mustapha.faqir,aymane.ahajjam } @uir.ac.ma Abstract —We present a novel approach to perform ground-based estimation and prediction of the surface solar irradiancewith the view to predicting photovoltaic energy production. Wepropose the use of mini-batch k -means clustering to extractfeatures, referred to as per cluster number of pixels (PCNP),from sky images taken by a low-cost fish eye camera. Thesefeatures are first used to classify the sky as clear or cloudy usinga single hidden layer neural network; the classification accuracyachieves 99.7%. If the sky is classified as cloudy, we propose touse a deep neural network having as input features the PCNPto predict intra-hour variability of the solar irradiance. Towardthis objective, in this paper, we focus on estimating the deepneural network model relating the PCNP features and the solarirradiance, which is an important step before performing theprediction task. The proposed deep learning-based estimationapproach is shown to have an accuracy of 95%. Index Terms —photovoltaic, solar irradiance, sky imaging,machine learning, k-means clustering, deep learning, neuralnetworks.
I. I
NTRODUCTION
With the proliferation of photovoltaic (PV) energy systems,it is important to develop a reliable energy management systemfor operators to optimize the integration of PV energy intothe electrical grid. Predicting the energy generated by the PVinstallation is a key feature in such a system, since it allows todetect faulty performance, to perform load scheduling, and ingeneral to make better operational decisions [1]. Forecastingsurface solar irradiance (SSI) is the basis of forecasting thePV energy because of the direct relation between these two.If the sky is clear, physics-based prediction models performwell. Since some of the parameters in these models may bedifficult to obtain, data-driven methods have been proposed topredict the SSI. For example, the Global Horizontal Irradiance(GHI) was shown to be accurately predicted with the nonlinearauto-regressive with exogenous inputs (NARX) model, havingas input past GHI values [2].Predicting the SSI in the case of a cloudy sky is much morechallenging than in the case of clear sky conditions. This isdue to the non-stationarity of the clouds introduced by thestochastic behaviour of the wind in the spatial (e.g. heightof the clouds) and temporal dimensions. To address this issue,two categories of methods have been proposed in the literature:ground-based methods and geostationary satellite-based meth-ods. Satellite-based prediction is primarily carried out using numerical weather prediction and satellite cloud monitoring.However, this approach only provides rough estimation dueto the spatial and temporal resolution limitations of satelliteimages [3]. Ground-based methods, using sky imaging, havethe potential to provide better performance, particularly for theprediction of intra-hour variability of the irradiance [4]. Thisis the approach adopted in this paper.Toward the objective of forecasting the SSI, in this paper,we propose to use deep learning to accurately estimate themapping function between the sky image and the correspond-ing irradiance, which is an important step before addressingthe forecasting task. The data set used in this paper consistsof sky images, taken with a low-cost fish eye camera, andtheir corresponding Global Horizontal Irradiance (GHI). First,the size of the images is reduced by clustering the colors ofthe images’ pixels into a relatively small number of colorclusters; this is carried out using the mini-batch k -meansclustering algorithm. The resulting segmented sky image isused to classify the sky as either clear or cloudy; the classifierwas learned using manually labeled images and a single hiddenlayer neural network. Using the cloudy sky image subset,a deep neural network is trained to model the relationshipbetween the sky images and the corresponding irradiances.II. D ATA SET
We use a data set consisting of RGB hemispheric skyimages of size × pixels, taken with a Vivotek FE8174fish-eye camera, and their corresponding GHI, recorded withKipp & Zonen CMP11 pyranometer. The data set was recordedduring a period of three months (October, November and De-cember of 2016) in Folsom, CA, USA. A record is taken everyminute. More details about this data set can be found in [4].Figure 1 shows a sample of three sky images correspondingto different weather conditions.III. S KY IMAGE SEGMENTATION AND FEATUREEXTRACTION
Segmenting an image consists of reducing it to salient pixelsthat reflect its global perceptual aspects. These representativepixels, also called super-pixels, define the segments of areduced image. Each segment has a label, usually a color,and is present in one or more regions in the image. This taskprovides a convenient representation of the image for furtheranalysis. It also serves to lower the computational cost because a r X i v : . [ c s . C V ] D ec a) Clear sky (b) Cloudy sky (c) Overcast skyFig. 1. A sample of sky images with different weather conditions of the reduced number of values taken by the channels of thecolor model in use.Our goal is to simplify the representation of the RGB skyimages without losing much information about the three skycomponents, which are the clear sky, the clouds and the solardisk. These three sky components are represented by differentsets of a relatively small number of RGB colors, with eachone of them contributing to the GHI with a certain weight.For instance, clouds might have different levels of brightness,depending on the light they receive from the sun, going fromwhite near the solar disk to dark gray in the surroundings.The intuition behind extracting different segments of the skycomponent is that we could infer the GHI by knowing thenumber of pixels of each segments of the segmented skyimage. Indeed, a decrease in the GHI due the presence ofclouds, particularly when (partially) obscuring the sun, ismanifested in the segmented sky image by a reduction of thenumber of pixels present in the sun segments, an increase ofthe number of pixels in the cloud segments, and a reductionof the number of pixels representing the clear-sky.Clustering is one of the commonly used methods to segmentan image. The most widely used clustering algorithm is k -means. The statement of the problem is as follows: given n pixels in the -dimensional space defined by the R, G and Bchannels (the axes of the space), and an integer k , find a set of k super-pixels, so as to minimize the mean squared distancebetween each pixel and its nearest super-pixel. The caughtsuper-pixels should be representative of all the colors presentin the sky images of the whole data set, knowing that thecolors vary with the time of day and the weather conditions,as depicted in Figure 1.The standard version of the k -means algorithm, first pro-posed by Lloyd [5], is suitable for relatively small data sets(which is not our case), because it requires to load all theimages in memory before processing. A workaround to thismemory limitation problem is to train the k -means algorithmon a manually selected set of sky images that catch most of thetypical colors representing the sky components. A better solu-tion is to use the stochastic gradient descent implementation ofthe k -means algorithm. To reduce the stochastic noise and thusallow convergence to better centroids, a mini-batch version of the stochastic gradient descent k -means algorithms wasproposed in [6]. Figure 2 shows clustered sky images usinga trained k -means model with k = 64 . It is worth pointingout that before running the k -means algorithm, a mask hidingthe surrounding components which are not part of the skyis applied to the images in order to intercept exclusively skypixels.After training the mini-batch k -means algorithm on theentire set of sky images, we perform the segmentation usingthe resultant centroids. Then, for each segmented image, weextract the number of pixels present in each segment. Werefer to these features in the following sections as per clusternumber of pixels (PCNP).IV. S KY IMAGE CLASSIFICATION
Many studies in the literature used NARX neural networksto forecast the PV energy [1], [2]. Even though this modelshowed excellent ability in predicting the solar irradiance (anddirectly the PV energy) under clear sky conditions, there areno studies, to our knowledge, that cover the problem of thenoise generated by the clouds that affects the solar irradiance,making the NARX model performance to degrade. The aimof this study is to enhance the prediction capabilities of sucha model by taking into account the configuration of the sky.In order to decide about the prediction strategy, we first needa classifier that separates clear sky images from cloudy ones.We use a data set consisting of sky images labeledmanually, from which 75% serve as a training set and theremaining sky images as a testing set. The classifier is a singlehidden layer neural network that takes as inputs the PCNPfeatures extracted from the raw sky images and outputs twoprobability scores corresponfding to the two labels. We use thesigmoid function as neurons’ activations. The neural networkis trained using Limited-memory BFGS algorithm which isknown to converge to better solutions on small data sets.Table I outlines information about the classifier.V. GHI
ESTIMATION
In this section, we estimate the transfer function betweenthe extracted PCNP features and the measured GHI. The dataset we use consists of 256 PCNP features (i.e. k = 256 , with a) Clear sky (b) Cloudy sky (c) Overcast skyFig. 2. Segmented versions of the sky images shown in Figure 1 using a trained k -means model (64 clusters). The images belong to the testing set and werenever visited during the training phase. The black pixels depict the applied mask.TABLE IN EURAL NETWORK PARAMETERS SUMMARY FOR THE SKY IMAGECLASSIFIER
Parameter Value
Features PCNP , PCNP , . . . , PCNP Target sky class: “clear” or “cloudy”Number of hidden layers 1Number of hidden units 27Activation function sigmoid f ( x ) = e − x l regularization parameter . Optimizer Limited-memory BFGS each feature representing the count of one of the super-pixels,and a target variable which is the GHI. The data set counts records after removing elements having a GHI of .Our modeling approach utilizes a deep neural network.The model’s architecture consists of five hidden layers L , L . . . , L having , , , and hidden units,respectively. We use as activation function the Rectified Lin-ear Unit ReLU ( x ) = max(0 , x ) which gives better resultsthan the hyperbolic tangent tanh( x ) = e x − e − x e x + e − x usually usedin regression problems. Dropout regularization is applied toeach hidden layer of the network during the training phase,which helps prevent over-fitting by reducing hidden units co-adaptation. This technique gives major improvement to ourmodel over penalizing the weights with the l regularization.We also apply batch normalization to each hidden layer inorder to maintain the mean activation close to and theactivation standard deviation close to . This has the effect ofaccelerating the training phase by allowing the use of a higherlearning rate. It should be pointed out that the PCNP featureswere preprocessed by scaling them to zero mean and unitstandard deviation. Figure 3 depicts our deep neural network’sarchitecture.The model is trained using Stochastic Gradient Descentalgorithm with Nesterov momentum, which is better suited forour large data set. The hyper-parameters of the deep neuralnetwork such as the number of hidden layers and neurons,the activation function and the regularization parameters aredetermined using 10-fold cross validation. Table II summarizes TABLE IID
EEP NEURAL NETWORK PARAMETERS SUMMARY
Parameter Value
Features PCNP , PCNP , . . . , PCNP
Target GHINumber of hidden layers 5Number of hidden units , , , and Activation function ReLU ( x ) = max (0 , x ) Dropout rates . and . for the last hidden layerOptimizer Stochastic gradient descentOptimizer learning rate . Optimizer momentum Nesterov with parameter . the parameters of our model.VI. R ESULTS AND DISCUSSION
The estimation of the transfer function between the PCNPfeatures and the GHI shows on the testing set a coefficient ofdetermination R of . . The corresponding mean absoluteerror on GHI estimation is about W/m , where themeasured GHI ranges from to W/m . These scoresare validated using -fold cross validation.Figure 4 shows the GHI output by the deep neural networkversus the real values acquired by the pyranometer for differentweather conditions. The entries are randomly selected from thetesting set.Concerning the neural network classifier, the metric we useto measure the model’s performance is the accuracy (i.e. thefrequency with which predictions matches labels). Accuracy = True positives + True negativesNumber of predictions
The classifier shows an accuracy of . % on the testing setand is able to distinguish between clear sky images having theglare surrounding the center (c.f. Figure 1a) and sky imageshaving partial clouds. It is worth mentioning that the modelmay be improved by making it able to detect overcast skyconditions as well. ig. 3. Architecture of the deep neural network VII. C
ONCLUSION AND FUTURE WORK
In this paper, we presented a new method for modelingthe relationship between hemispheric sky images and theircorresponding surface solar irradiances. We first used mini-batch k -means clustering in order to reduce the size of skyimages. The extracted PCNP features were used an inputs totrain a deep learning neural network to predict the irradianceassociated with each sky image. A real dataset was used toillustrate the merits of the proposed method. As a future work,we will investigate how the extracted PCNP features can beused to forecast intra-hour variability of the irradiance in thecase of cloudy and overcast sky conditions.A CKNOWLEDGMENT
This research work was supported by the Institute forResearch in Solar Energy and New Energies (IRESEN) and theUnited States Agency for International Development (USAID).The authors would like to thank Yinghao Chu, Hugo T. C.Pedro, Lukas Nonnenmacher, Rich H. Inman, Zhouyi Liaoand Carlos F. M. Coimbra for providing the data set used inthis paper. G H I ( W / m ) MeasuredEstimated (a) Clear sky conditions G H I ( W / m ) MeasuredEstimated (b) Cloudy sky conditions G H I ( W / m ) MeasuredEstimated (c) Overcast sky conditionsFig. 4. The estimated GHI (dark) versus the measured GHI (gray) in
W/m for randomly selected entries from the testing set. R EFERENCES[1] M. Cococcioni, E. D’Andrea, and B. Lazzerini, “24-hour-ahead forecast-ing of energy production in solar PV systems,” in , Nov 2011,pp. 1276–1281.[2] A. E. Hendouzi and A. Bourouhou, “Forecasting of PV power applicationto PV power penetration in a microgrid,” in , May 2016, pp.468–473.[3] C. W. Chow, B. Urquhart, M. Lave, A. Dominguez, J. Kleissl, J. Shields,and B. Washom, “Intra-hour forecasting with a total sky imager at theUC san diego solar energy testbed,”
Solar Energy , vol. 85, no. 11, pp.2881 – 2893, 2011.[4] Y. Chu, H. T. C. Pedro, L. Nonnenmacher, R. H. Inman, Z. Liao, andC. F. M. Coimbra, “A smart image-based cloud detection system forintrahour solar irradiance forecasts,”
Journal of Atmospheric and OceanicTechnology , vol. 31, no. 9, pp. 1995–2007, 2014.[5] S. Lloyd, “Least squares quantization in PCM,”
IEEE Transactions onInformation Theory , vol. 28, no. 2, pp. 129–137, March 1982.[6] D. Sculley, “Web-scale k-means clustering,” in