Assessing the Sharpness of Satellite Images: Study of the PlanetScope Constellation
AASSESSING THE SHARPNESS OF SATELLITE IMAGES:STUDY OF THE PLANETSCOPE CONSTELLATION
Jérémy Anger (cid:63)
Carlo de Franchis (cid:63), † Gabriele Facciolo (cid:63)(cid:63)
CMLA, ENS Cachan, CNRS, Université Paris-Saclay, France † Kayrros, France
ABSTRACT
New micro-satellite constellations enable unprecedented sys-tematic monitoring applications thanks to their wide coverageand short revisit capabilities. However, the large volumes ofimages that they produce have uneven qualities, creating theneed for automatic quality assessment methods. In this work,we quantify the sharpness of images from the PlanetScopeconstellation by estimating the blur kernel from each image.Once the kernel has been estimated, it is possible to computean absolute measure of sharpness which allows to discard lowquality images and deconvolve blurry images before any fur-ther processing. The method is fully blind and automatic, andsince it does not require the knowledge of any satellite speci-fications it can be ported to other constellations.
1. INTRODUCTION
The usability of satellite images for interpretation, or objectdetection and reconstruction purposes highly depends on theimage quality, which can be characterized by a large numberof measures, e.g. contrast, brightness, noise variance, radio-metric resolution, sharpness, etc. Among those measures, im-age sharpness is one of the most important for characterizingimages as it evaluates image blur, which limits the visibilityof details. Image blur is introduced by both the optical systemand potential motion during the acquisition time [1].Assuming a stationary blur kernel k (or Point Spread Func-tion (PSF)) that combines the optical and motion blur we canformulate the image formation model as v = u ∗ k + n, (1)where v is the blurry image, u is the latent sharp image, and n is acquisition noise. Then, sharpness can be objectivelymeasured by estimating the point spread function k or its am-plitude spectrum, the Modulation Transfer Function (MTF).In remote sensing, most sharpness studies [2, 3] focus onthe in-flight characterization of the camera system. These ap-proaches usually rely on the presence of on-ground targets
Work partly financed by Office of Naval research grant N00014-17-1-2552, DGA Astrid project « filmer la Terre » n ◦ ANR-17-ASTR-0013-01,MENRT; DGA PhD scholarship jointly supported with FMJH. such as edges, lines, or point reflectors. Some methods esti-mate the blur kernel assuming a parametric model, for exam-ple Gaussian, and try to fit its parameters [4]. Another groupof methods estimate cross sections of the MTF, usually byapplying some variant of the slanted edge method over cali-bration sites [2]. These methods allow to periodically assessthe sharpness of the system, however they cannot account formotion blur of a particular acquisition, an artifact that has be-come more common within modern fleets of micro-satellites.Other methods [5, 6, 7] seek to estimate the MTF by re-lying on the detection of straight edges naturally present inthe scene. This allows to estimate sharpness (mostly on urbanscenes) without the need of a calibration target. Note howeverthat the MTF is only useful for characterizing the sharpnessand, while it allows a simple frequency enhancement, it can-not be used to restore the image as the phase of the kernel isnot estimated.Estimating the blur kernel from a single image is an activefield of research, especially for natural images since it is anecessary step of most blind deblurring methods [8, 9, 10].These methods rely only on the presence of contrasted edges(not necessarily straight lines), which allows to apply them tovirtually any scene. In addition to estimating the kernel forquality assessment, one may want to restore the sharp image u . Indeed, if two images have different blur kernels, it mightbe difficult to compare and analyze, both visually and auto-matically by advanced image processing techniques.We focus our experiments on PlanetScope [11] images.The PlanetScope constellation is made of approximately 130small satellites (form factor of × × cm) imagingthe entire Earth’s landmass every day. The satellites carry3-band or 4-band frame cameras and fly at 475 km on sun-synchronous orbits whose constant local solar time is between9:30 and 11:30 am. The Ground Sample Distance (GSD) atnadir is between 3.5 m and 4 m. The images are availableas either individual basic scenes, ortho scenes, or ortho tiles.We focus here on the basic and ortho scenes, respectively fornon-orthorectified and orthorectified images. Exploiting bothproducts allows us to infer the influence of the orthorectifica-tion on the image sharpness. Contributions.
We propose a criteria to assess the sharp- a r X i v : . [ c s . C V ] A p r ess of satellite images through the estimation of their blurkernels. This criteria allows to sort images by quality, thusgiving an absolute threshold to discard low quality imagesand allowing to increase the quality of blurry images using adeblurring step. Our method is fully blind and is designed forconsumers of PlanetScope images. Indeed it does not requirethe precise specifications of the satellites and could also beused for images from other sources. We validate our method-ology through a study of the PlanetScope constellation. Inparticular we show the effect of the orthorectification on thesharpness and we study the per-satellite sharpness.
2. SHARPNESS ASSESSMENT AND DEBLURRING
In this section we detail our methodology by first explaininghow to blindly estimate the blur kernel and compute a sharp-ness score from it, then describe the deblurring step.
In this work, we use the kernel estimation method of Panet al. [10] originally developed to deblur text images. Thismethod is based on the (cid:96) gradient prior which restores themain structures of the image, including dominant edges. Oncethe edges are restored, the blur kernel can be estimated. Themethod iterates between the estimation of the sharp image us-ing the previous kernel and the re-estimation of the blur kernel u ( t +1) = arg min u (cid:107) u ∗ k ( t ) − v (cid:107) + λ (cid:107)∇ u (cid:107) (2) k ( t +1) = arg min k (cid:107)∇ u ( t +1) ∗ k − ∇ v (cid:107) + γ (cid:107) k (cid:107) . (3)We use the efficient implementation of Anger et al. [12].Even if the (cid:96) gradient prior kernel estimation method wasdesigned for text and natural images, we argue that it is appli-cable to satellite images without any adaptation. Indeed, theassumption behind this prior is that non-blurry images containcontrasted edges, which is valid for satellite images. Further-more, satellite image are more likely to respect the stationaryconvolution model than natural images since the scene is faraway from the camera, which results in less parallax, a mostlytranslational motion and low optical distortion. Existing quality assessment metrics for satellite images in-clude measures on the PSF or on the MTF, please refer toBlanc et al. [2] for a comprehensive study. We design oursharpness score so that the maximal score of is achievedfor a perfectly sharp image (delta kernel) and it decreases forblurrier images (spread out kernels). Let us note that the ker-nel is assumed to be normalized so that (cid:107) k (cid:107) = 1 . The sim-plest measure satisfying these criteria is the (cid:96) norm S = (cid:107) k (cid:107) = (cid:113)(cid:80) x | k ( x ) | . (4) (a) 0.019 (b) 0.022 (c) 0.026 (d) 0.032 (e) 0.033 Fig. 1 : Samples of estimated blur kernels. The top rowsshows crops of orthorectified images with their associatedsharpness score on the bottom row. Clouds mislead the es-timation towards a low sharpness score
Fig. 2 : Deblurring of an orthorectified image of Tokyo. Theinput image (left, S = 0 . ) contains motion blur that isremoved after deconvolution (right, S = 0 . ).An advantage of using the blur kernel to assess sharpnessis that it is independent of the image content, which is notthe case for measures estimated based on properties of theimage itself. Thus the resulting S score is absolute and can becompared across scenes and/or satellites. While very simple,this measure is sufficient to characterize the quality of satelliteimages for many applications.Figure 1 shows five crops of PlanetScope orthorectified im-ages and their associated kernel and S score. We observethat ordering the images by their estimated sharpness indeedcorrelates well with our perception of the blur introduced bythe respective kernel. Furthermore, we observe that whenthe scene contains a majority of clouds, the kernel estima-tion tends to give a very spread out kernel. This phenomenonoccurs because clouds do not have sparse gradients and thushinder the estimation. Fortunately, most of the time the pre-dicted sharpness of cloudy scenes is very low, allowing to sortsuch images as low quality and discard them. Having estimated a blur kernel, it is possible to inverse theproblem (1) using non-blind deconvolution methods [13, 14].Since satellite images usually contain low noise in nominalconditions, a simple prior is enough to recover a high quality
Fig. 3 : Histogram of sharpness for the dataset of im-ages. The hatched region represents low quality images orvery cloudy scenes.image. In this work, we use the total variation prior (TV) [13]already well studied for satellite image deblurring [15], lead-ing to the following optimization problem arg min u (cid:107) u ∗ k − v (cid:107) + α (cid:107)∇ u (cid:107) . (5)We solve problem (5) using the fast method from Krishnanet al. [14]. Figure 2 shows a deconvolution result on an or-thorectified image. Notice that the input image contains ananisotropic blur, successfully removed by deblurring.
3. STUDY OF THE PLANETSCOPECONSTELLATION
In this section we apply our sharpness measure on Plan-etScope images to show that it captures the variability presentin the images from this constellation. As dataset, we collected basic and ortho
PlanetScopes images at 29 differ-ent locations for a total of images.
Sharpness distribution.
Figure 3 shows two histograms rep-resenting the distribution of sharpness for basic and ortho im-ages. We note that the sharpness of ortho images is on averagelower and less variable than the one of basic images. Indeed,the sharpness after orthorectification decreases significantly,with an average of S = 0 . versus S = 0 . for basic images. This indicates that the orthorectified images are in-deed less sharp and our measure does quantify the amount ofsharpness lost due to the resampling.We also observe that both distributions have a second modenear S = 0 . . This mode correspond to invalid kernelswhich can occur on very cloudy scenes for example (as shownon Figure 1a), or when the signal to noise ratio (SNR) is lowdue to poor atmospheric conditions. Quality thresholds.
From these histograms and our obser-vations of the data, we found that for orthorectified images,the threshold S > . indicates high sharpness imageswhereas S < . corresponds to highly blurred images Fig. 4 : Example of significant blur difference from two con-secutive basic (non-orthorectified) images from two differentsatellites. Sharpness scores are . and . . The verticalblur is likely due to an hazardous stabilization of the satelliteduring the acquisition.leaving little hope for a high quality restoration (in red on Fig-ure 3). Otherwise, the image can be sharpened using the pre-viously described deblurring algorithm in order to increase itsquality before visualization or processing. For basic images, S < . provides a similar threshold while accounting forthe increase of sharpness compared to ortho images. Presence of motion blur.
Sharpness metrics using MTF isusually calibrated using on-ground targets [2]. While thisallows for very precise estimations, it cannot consider allsources of blur. In particular, our methodology takes into ac-count resampling as well as blur due to motion during theintegration time. Figure 4 shows two non-orthorectified im-ages taken on two consecutive days from different satellites.The right image shows an example of motion blur. The sharp-ness score allows to automatically filter out such poor qualityimages using a simple threshold.
Per-satellite sharpness.
As previously explained, the Plan-etScope constellation is composed of a hundred satellites.Here, we study the correlation between the sharpness of theimages and the satellite that acquired them. Our dataset of images represent 153 distinct satellites. In order tohave large enough sample sizes, we kept only the satellitesfor which there are at least 50 images. Then, we computedthe S score of each image and removed from the dataset allinvalid images, that is having a sharpness score below . and . respectively for basic and ortho images. Sharp-ness averages and standard deviations for each satellite arereported in Figure 5. We first notice that the average sharp-ness is not uniform across the constellation, which would in-dicate that each satellite produces images with slightly differ-ent blur than others. Indeed, an analysis of variance (one-wayANOVA test) rejects the hypothesis of equal averages and in-dicates a statistically significant difference in the per-satellitesharpness averages. Moreover, the clear correlation betweenthe sharpness of basic and ortho images per satellite confirmsthat our measure is reliable. Finally, it is important to note S h a r p n e ss Sharpness per satellite basicortho
Fig. 5 : Sharpness mean and variance across the constella-tion. Satellite ids (in abscissa) were sorted by mean sharp-ness. Dash-lines indicates the standard deviation for eachsatellite. The two series correspond to basic and ortho im-ages.that the standard deviation is large and thus the satellite is notthe only factor responsible for the variation of sharpness, andother factors such as motion during acquisition also introducevariance to the effective sharpness of the images.
4. CONCLUSION
In this study, we quantified the variability of blur fromPlanetScope images using an efficient blur kernel estimationmethod and automatically assigning a measure of sharpnessto each blur kernel. The method is blind and does not requirethe specifications of the optical system. We also demonstratedthat it is possible to apply blind deblurring methods to satel-lite images in order to equalize quality across time or improvevisualization.Our study of the constellation indicated variation across theimages. We observed that the images can contain significantmotion blur. Furthermore, we showed that the orthorectifica-tion provided by Planet does decrease the average sharpnessof the images. We also showed correlation between a givensatellite and its average sharpness. Finally, we proposed sim-ple thresholds that allow to discard unsatisfactory images.However, our method has a few limitations we would liketo overcome in future works. First, as explained in Sec-tion 2.2, the method is affected by clouds. One way to solvethis issue would be to apply a cloud detector on the imagesand mask out detected regions during the kernel estimation.The second limitation is noise which can be present in someimages due to atmospheric conditions and degrades the per-formance of both kernel estimation and non-blind deconvolu-tion. We would also like to tackle this problem and provide anadditional measure to indicate the noise level and the amountof details we can expect from the restoration. Finally, satu-rated region in the image tends to mislead the kernel estima-tion towards a delta, and further work is required to handlethis degradation.
5. REFERENCES [1] T. Choi and D. L. Helder, “Generic sensor modeling for mod-ulation transfer function (mtf) estimation,”
Pecora 16 , pp. 23–27, 2005.[2] P. Blanc and L. Wald, “A review of earth-viewing methods forin-flight assessment of modulation transfer function and noiseof optical spaceborne sensors,” 2009.[3] M. Pagnutti, S. Blonski, M. Cramer, D. Helder, K. Holekamp,E. Honkavaara, and R. Ryan, “Targets, methods, and sites forassessing the in-flight spatial resolution of electro-optical dataproducts,”
Canadian Journal of Remote Sensing , vol. 36, no.5, pp. 583–601, 2010.[4] A. Jalobeanu, L. Blanc-Féraud, and J. Zerubia, “Estimationof blur and noise parameters in remote sensing,” in
ICASSP ,2002, vol. 4, pp. 3580–3583.[5] K. Kohm, “Modulation transfer function measurement methodand results for the orbview-3 high resolution imaging satellite,”2004, pp. 12–23.[6] M. Crespi and L. De Vendictis, “A procedure for high resolu-tion satellite imagery quality assessment,”
Sensors , vol. 9, no.5, pp. 3289–3313, 2009.[7] M. Luxen and W. Forstner, “Characterizing image quality:Blind estimation of the point spread function from a single im-age,”
ISPRS , vol. 34, pp. 205–210, 2002.[8] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, “Un-derstanding and evaluating blind deconvolution algorithms,”
CVPR Workshops , pp. 1964–1971, 2009.[9] Q. Shan, J. Jia, and A. Agarwala, “High-quality motion deblur-ring from a single image,”
ACM Transactions on Graphics , vol.27, no. 3, pp. 1, 2008.[10] J. Pan, Z. Hu, Z. Su, and M. H. Yang, “L0-Regularized In-tensity and Gradient Prior for Deblurring Text Images and be-yond,”
TPAMI , vol. 39, no. 2, pp. 342–355, 2015.[11] Planet Team, “Planet Application Program Interface: In Spacefor Life on Earth,” San Francisco, CA., 2017, https://api.planet.com .[12] J. Anger, G. Facciolo, and M. Delbracio, “Blind image deblur-ring using the l0 gradient prior,”
Image Processing On Line ,2019.[13] L. I. Rudin and S. Osher, “Total variation based image restora-tion with free local constraints,”
ICIP , vol. 1, no. April, pp.31—-35 vol.1, 1994.[14] D. Krishnan and R. Fergus, “Fast image deconvolution usinghyper-laplacian priors,”