Certifiable Risk-Based Engineering Design Optimization
Anirban Chaudhuri, Boris Kramer, Matthew Norton, Johannes O. Royset, Karen Willcox
CCertifiable Risk-Based Engineering Design Optimization
Anirban Chaudhuri ∗ Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
Boris Kramer † University of California San Diego, CA, 92093, USA
Matthew Norton ‡ , Johannes O. Royset § Naval Postgraduate School, Monterey, CA, 93943, USA
Karen E. Willcox ¶ University of Texas at Austin, Austin, TX, 78712, USA
Abstract
Reliable, risk-averse design of complex engineering systems with optimized performance requiresdealing with uncertainties. A conventional approach is to add safety margins to a design that was ob-tained from deterministic optimization. Safer engineering designs require appropriate cost and constraintfunction definitions that capture the risk associated with unwanted system behavior in the presence ofuncertainties. The paper proposes two notions of certifiability. The first is based on accounting forthe magnitude of failure to ensure data-informed conservativeness. The second is the ability to provideoptimization convergence guarantees by preserving convexity. Satisfying these notions leads to certi-fiable risk-based design optimization (CRiBDO). In the context of CRiBDO, risk measures based onsuperquantile (a.k.a. conditional value-at-risk) and buffered probability of failure are analyzed. CRiBDOis contrasted with reliability-based design optimization (RBDO), where uncertainties are accounted forvia the probability of failure, through a structural and a thermal design problem. A reformulation ofthe short column structural design problem leading to a convex CRiBDO problem is presented. TheCRiBDO formulations capture more information about the problem to assign the appropriate conserva-tiveness, exhibit superior optimization convergence by preserving properties of underlying functions, andalleviate the adverse effects of choosing hard failure thresholds required in RBDO.
The design of complex engineering systems requires quantifying and accounting for risk in the presence ofuncertainties. This is not only vital to ensure safety of designs but also to safeguard against costly designalterations late in the design cycle. The traditional approach is to add safety margins to compensate foruncertainties after a deterministic optimization is performed. This produces a sense of security, but is atbest an imprecise recognition of risk and results in overly conservative designs that can limit performance.Properly accounting for risk during the design optimization of those systems could allow for more efficientdesigns. For example, payload increases for spacecraft and aircraft could be possible without sacrificing safety .The financial community has long recognized the superiority of specific risk measures in portfolio optimization(most importantly the conditional-value-at-risk (CVaR) pioneered by Rockafellar and Uryasev [1]), see [1,2, 3]. In the financial context, it is understood that exposure to tail risk—rather rare events—can lead tocatastrophic outcomes for companies, and adding too many “safety factors” (insurance, hedging) reduces ∗ Research Scientist, Department of Aeronautics and Astronautics, [email protected]. † Assistant Professor, Department of Mechanical and Aerospace Engineering, [email protected]. ‡ Assistant Professor, Department of Operations Research, [email protected]. § Professor, Department of Operations Research, [email protected]. ¶ Director, Oden Institute for Computational Engineering and Sciences, [email protected] a r X i v : . [ m a t h . O C ] J a n rofit. Analogously, in the engineering context, the problem is to find safe engineering designs withoutunnecessarily limiting performance and limiting the effects of the heuristic guesswork of choosing thresholds.In general, there are two main issues when formulating a design optimization under uncertainty problem:(1) what to optimize and (2) how to optimize. The first issue involves deciding the design criterion, which inthe context of decision theory could boil down to what type of utility function to use. What is a meaningfulway of making design decisions under uncertainty? One would like to have a framework that can reflectstakeholders’ preferences, but at the same time is relatively simple and can be explained to the public, to agovernor, to a CEO, etc. The answer for what to optimize directly influences how you optimize. If the “whatto optimize” was chosen poorly, the second issue becomes much more challenging. Design optimization ofa real-world system is difficult, even in a deterministic setting, so it is essential to manage complexity aswe formulate the design-under-uncertainty problem. Thus, any design criterion that preserves convexityand other desirable mathematical properties of the underlying functions is preferable as it simplifies thesubsequent optimization.This motivates us to incorporate specific mathematical measures of risk, either as a design constraint orcost function, into the design optimization formulation. To this end, we focus on two particular risk mea-sures that have potentially superior properties: (i) superquantile/CVaR [4, 5], and (ii) buffered probabilityof failure (bPoF) [6]. Three immediate benefits of using these risk measures arise. First, both risk measuresrecognize extreme (tail) events which automatically enhances resilience. Second, they preserve convexity ofunderlying functions so that specialized and provably convergent optimizers can be employed. This dras-tically improves optimization performance. Third, superquantile and bPoF are conservative risk measuresthat add a buffer zone to the limiting threshold by taking into account the magnitude of failure. This can behandled by adding safety factors to the threshold; however, it has been shown before that probabilistic ap-proaches lead to safer designs with optimized performance compared to the safety factor approach [7, 8, 9].Superquantile/CVaR has been recently used in specific formulations in civil [10, 11], naval [12, 13] andaerospace [14, 15] engineering, as well as general PDE-constrained optimization [16, 17, 18, 19]. The bPoFrisk measure has been shown to possess beneficial properties when used in optimization [6, 20, 21, 22], yethas been seldom used in engineering to-date [23, 24, 25, 26]. We contrast these above risk-based engineeringdesign methods with the most common approach to address parametric uncertainties in engineering design,namely reliability-based design optimization (RBDO) [27, 28] which uses the probability of failure (PoF) asa design constraint. We discuss the specific advantages of using these ways of measuring risk in the designoptimization cycle and their effect on the final design under uncertainty.In this paper, we define two certifiability conditions for risk-based design optimization that can certifydesigns against near-failure and catastrophic failure events, and guarantee convergence to the global optimumbased on preservation of convexity by the risk measures. We call the optimization formulations usingrisk measures satisfying any of the certifiability conditions as C ertifiable Ri sk- B ased D esign O ptimization(CRiBDO). Risk measures satisfying both certifiability conditions lead to strongly certifiable risk-baseddesign. We analyze superquantile and bPoF, which are examples of risk measures satisfying the certifiabilityconditions. We discuss how the nature of probabilistic conservativeness introduced through superquantileand bPoF makes practical sense since it is data-informed and based on the magnitude of failure. The data-informed probabilistic conservativeness of superquantiles and bPoF circumvents the guesswork associatedwith setting safety factors (especially, for the conceptual design phase) and transcends the limitations ofsetting hard thresholds for limit state functions used in PoF. This helps us move away from being conservativeblindly to being conservative to the level dictated by the data . We compare the different risk-based designoptimization formulations using a structural and a thermal design problem. For the structural design of ashort column problem, we show a convex reformulation of the objective and limit state functions that leadsto a convex CRiBDO formulation.The remainder of this paper is organized as follows. We summarize the widely-used RBDO formulationin Section 2. The different risk-based optimization problem formulations along with the risk measures usedin this work are described in Section 3. Section 4 explains the features of different risk-based optimizationformulations through numerical experiments on the short column problem with a convex reformulation.Section 5 explores the different risk-based optimization formulations for the thermal design of a cooling finproblem with non-convex limit state. Section 6 presents the concluding remarks.2 Reliability-based Design Optimization
In this section, we review the RBDO formulation, which uses PoF to quantify uncertainties. Let the quantityof interest of an engineering system be computed from the model f : D × Ω (cid:55)→ R as f ( d , Z ), where the inputsto the system are the n d design variables d ∈ D ⊆ R n d and the n z random variables Z with the probabilitydistribution π . The realizations of the random variables Z are denoted by z ∈ Ω ⊆ R n z . The space of designvariables is denoted by D and the space of random samples is denoted by Ω. The failure of the system isdescribed by a limit state function g : D × Ω (cid:55)→ R and a critical threshold t ∈ R , where, without loss ofgenerality, g ( d , z ) > t defines failure of the system. For a system under uncertainty, g ( d , Z ) is also a randomvariable given a particular design d . The limit state function in most engineering applications requires thesolution of a system of equations (such as ordinary differential equations or partial differential equations).The most common RBDO formulation involves the use of a PoF constraint asmin d ∈D E [ f ( d , Z )]subject to p t ( g ( d , Z )) ≤ − α T , (1)where α T ∈ [0 ,
1] is the targeted reliability and the PoF is defined via the limit state function g and thefailure threshold t as p t ( g ( d , Z )) := P [ g ( d , Z ) > t ]. The RBDO problem (1) designs a system with optimalmean characteristics, in terms of f ( d , Z ), such that it maintains a reliability of at least α . Note, however,that PoF has no information about the magnitude of the failure event as it is merely a measure of the set { g ( d , Z ) > t } , see Figure 1. p t = P [ g ( d , Z ) > t ] t g ( d , Z ) P r o b a b ili t y d e n s i t y Figure 1: Illustration for PoF indicated by the area of the shaded region.For our upcoming discussion, it is helpful to point out that a constraint on the PoF is equivalent to aconstraint on the α -quantile. The α -quantile, also known as the value-at-risk at level α , is defined in termsof the inverse cumulative distribution function of the limit state function F − g ( d ,Z ) as Q α [ g ( d , Z )] := F − g ( d ,Z ) ( α ) . (2)PoF and Q α are natural counterparts that are measures of the tail of the distribution of g ( d , Z ). Whenone knows that the largest 100(1 − α )% outcomes are the ones of interest, the quantile is a measure of thebest-case scenario within the set of these tail events. When one knows that outcomes larger than a giventhreshold t are of interest, PoF provides a measure of the frequency of these “large” events. This equivalenceof PoF and Q α risk constraints is illustrated in Figure 2. In the context of our optimization problem, usingthe same value of t and α T , (1) can be written equivalently asmin d ∈D E [ f ( d , Z )]subject to Q α T [ g ( d , Z )] ≤ t. (3)The most elementary method (although, inefficient) for estimating PoF is Monte Carlo (MC) simulationwhen dealing with nonlinear limit state functions. The MC estimate of the PoF for a given design d isˆ p t ( g ( d , Z )) = 1 m m (cid:88) i =1 I G ( d ) ( z i ) , (4)3 t > − αt Q α g ( d , Z ) P r o b a b ili t y d e n s i t y (a) p t < − αQ α t g ( d , Z ) P r o b a b ili t y d e n s i t y (b) p t = 1 − αQ α = t g ( d , Z ) P r o b a b ili t y d e n s i t y (c) Figure 2: Illustration of equivalence of PoF (shown by the shaded region) and Q α showing that the twoquantities converge at the constraint threshold when the reliability constraint is active.where z , . . . , z m are m samples distributed according to π , G ( d ) = { z | g ( d , z ) > t } is the failure set, and I G ( d ) : Ω → { , } is the indicator function defined as I G ( d ) ( z ) = (cid:26) , if z ∈ G ( d )0 , else. (5)The MC estimator is unbiased with the variance being p t (1 − p t ) /m . The PoF estimation requires samplingfrom the tails of the distribution, which can often make MC estimators expensive. A wealth of literatureexists for methods that have been developed to deal with the computational complexity of PoF estimationand the RBDO problem. First, reliability index methods (e.g., FORM, SORM, etc. [29, 30]) geometricallyapproximate the limit state function to reduce the computational effort of PoF estimation. However, whenthe limit state function is nonlinear, the reliability index method could lead to inaccuracies in the estimate.Second, MC variance reduction techniques such as importance sampling [31, 32, 33], adaptive importancesampling [34, 35, 36, 37, 38, 39], and multifidelity approaches [40, 41, 42] offer computational advantages.While the decay rate of the MC estimate cannot be improved upon, the variance of the MC estimatorcan be reduced, which offers computational advantages in that fewer (suitably chosen) MC samples areneeded to obtain accurate PoF estimates. Third, adaptive data-driven surrogates for the limit state failureboundary identification can improve computational efficiency for the RBDO problem [43, 44, 45]. Fourth,bi-fidelity RBDO methods [46, 47] and recent multifidelity/multi-information-source methods for the PoFestimate [48, 49, 44] and the RBDO problem [50, 51] have led to significant computational savings.Although significant research has been devoted to PoF and RBDO, PoF as a risk measure does not factorin how catastrophic is the failure and thus, lacks resiliency. In other words, PoF neglects the magnitude offailure of the system and instead encodes a hard threshold via a binary function evaluation. We describebelow this drawback of PoF. Remark 1 (Limitations of hard-thresholding)
To motivate the upcoming use of risk measures, we takea closer look at the limit state function g and its use to characterize failure events. In the standard setting, afailure event is characterized by a realization of Z for some fixed design d that leads to g ( d , z ) > t . However,this hard-threshold characterization of system failure potentially ignores important information quantified bythe magnitude of g ( d , z ) and PoF fails to promote resilience, i.e., no distinction between bad and very bad.For example, there may be a large difference between the event g ( d , z ) = t + . and g ( d , z ) = t + 100 , thelatter characterizing a catastrophic system failure. This is not captured when considering system failure onlyas a binary decision with a hard threshold. Similarly, one could also consider events g ( d , z ) = t − . and g ( d , z ) = t − . A hard-threshold assessment deems both of these events as non-failure events, even though g ( d , z ) = t − . is clearly a near-failure event compared to g ( d , z ) = t − . A hard-threshold characterizationof failure would potentially overlook these important near-failure events and consider them as safe realizationsof g . In reality, failure events do not usually occur using a hard-threshold rule. Even if they do, determinationof the true threshold will also involve uncertainty, blending statistical estimation, expert knowledge, andsystem models. Therefore, the choice of threshold should be involved in any discussion of measures of failurerisk and we analyze later in Remark 6, the advantage of the data-informed thresholding property of certainrisk measures as compared to hard-thresholding. As we show in the next section, superquantile and bPoF donot have this deficiency.
4n the engineering community, PoF has been the preferred choice. Using PoF and RBDO offers somespecific advantages starting with the simplicity of the risk measure and the natural intuition behind formu-lating the optimization problems, which is a major reason leading to the rich literature on this topic as notedbefore. Another advantage of PoF is the invariance to nonlinear reformulation for the limit state function.For example, let z be a random load and z be a random strength of a structure. Then the PoF would bethe same regardless if the limit state function is defined as z − z or z /z −
1. However, there are severalpotential issues when using PoF as the risk measure for design optimization under uncertainty as notedbelow.
Remark 2 (Optimization considerations)
While there are several advantages of using PoF and RBDO,there are several potential drawbacks. First, PoF is not necessarily a convex function w.r.t. design variables d even when the underlying limit state function is convex w.r.t. d . Thus, we cannot formulate a convexoptimization problem even when underlying functions f and g are convex w.r.t. d . This is important becauseconvexity guarantees convergence of standard and efficient algorithms to a globally optimal design underminimal assumptions since every local optimum is a global optimum in that case. Second, the computation ofPoF gradients can be ill-conditioned, so traditional gradient-based optimizers that require accurate gradientevaluations tend to face challenges. While PoF is differentiable for the specific case when d only containsparameters of the distribution of Z , such as mean and standard deviation, PoF is in general not a differen-tiable function. Consequently, PoF gradients may not exist and when using approximate methods, such asfinite difference, the accuracy of the PoF gradients could be poor. Some of these drawbacks can be addressedby using other methods for estimating the PoF gradients, but they have been developed under potentiallyrestrictive assumptions [52, 53, 54], which might not be easily verifiable for practical problems. Third, PoFcan suffer from sensitivity to the failure threshold due to it being a discontinuous function w.r.t. threshold t .As mentioned in Remark 1, the choice of failure threshold is often uncertain; thus, one would ideally preferto have a measure of risk that is less sensitive to small changes in t . Design optimization with a special class of risk measures can provide certifiable designs and algorithms. Wefirst present two notions of certifiability in risk-based optimization in Section 3. 3.1. We then discuss twospecific risk measures, superquantile in Section 3. 3.2 and bPoF in Section 3. 3.3, that satisfy these notionsof certifiability.
Risk in an engineering context can be quantified in several ways and the choice of risk measure, and itsuse as a cost or constraint, influences the design. We focus on a class of risk measures that can satisfy thefollowing two certifiability conditions :1.
Data-informed conservativeness:
Risk measures that take the magnitude of failure into account todecide the level of conservativeness required can certify the designs against near-failure and catastrophicfailure events leading to increased resilience. The obtained designs can overcome the limitations ofhard thresholding and are certifiably risk-averse against a continuous range of failure modes. In typicalengineering problems, the limit state function distributions are not known and the information aboutthe magnitude of failure is encoded through the generated data, thus making the conservativenessdata-informed.2.
Optimization convergence and efficiency:
Risk measures that preserve the convexity of underlyinglimit state functions (and/or cost functions) lead to convex risk-based optimization formulations. Theresulting optimization problem is better behaved than a non-convex problem and can be solved moreefficiently. Thus, one can find the design that is certifiably optimal in comparison with all alternatedesigns at reduced computational cost. In general, the risk measure preserves the convexity of thelimit state function, such that the complexity of the optimization under uncertainty problem remainssimilar to the complexity of the deterministic optimization problem using the limit state function.5e denote the risk-based design optimization formulations that use risk measures satisfying any of thetwo certifiability conditions as C ertifiable Ri sk- B ased D esign O ptimization (CRiBDO). Note that designsobtained through RBDO do not satisfy either of the above conditions since using PoF as the risk measurecannot guard against near-threshold or catastrophic failure events, see Remark 1, and cannot certify thedesign to be a global optimum, see Remark 2. The optimization formulations satisfying both the conditionslead to strongly certifiable risk-based designs. In general engineering applications, the convexity conditionis difficult to satisfy but encapsulates an ideal situation, highlighting the importance of research in creating(piece-wise) convex approximations for physical problems. In Sections 3. 3.2 and 3. 3.3, we discuss theproperties of two particular risk measures, superquantile and bPoF, that lead to certifiable risk-based designsand have the potential to be strongly certifiable when underlying functions are convex. Although we focuson these two particular risk measures in this work, other measures of risk could also be used to producecertifiable risk-based designs, see [55, 56, 10]. This section describes the concept of superquantiles and associated risk-averse optimization problem formu-lations. Superquantiles emphasize tail events, and from an engineering perspective it is important to managesuch tail risks.
Intuitively, superquantiles can be understood as a tail expectation, or an average over a portion of worst-caseoutcomes. Given a fixed design d and a distribution of potential outcomes g ( d , Z ), the superquantile at level α ∈ [0 ,
1] is the expected value of the largest 100(1 − α )% realizations of g ( d , Z ). In the literature, severalother terms, such as CVaR and expected shortfall, have been used interchangeably with superquantile. Weprefer the term superquantile because of its inherent connection with the long existing statistical quantityof quantiles and it being application agnostic.The definition of α -superquantile is based on the α -quantile Q α [ g ( d , Z )] from Equation (2). The α -superquantile Q α can be defined as Q α [ g ( d , Z )] := Q α [ g ( d , Z )] + 11 − α E (cid:104) [ g ( d , Z ) − Q α [ g ( d , Z )]] + (cid:105) , (6)where d is the given design and [ c ] + := max { , c } . The expectation in the second part of the right handside of Equation (6) can be interpreted as the expectation of the tail of the distribution exceeding the α -quantile. The α -superquantile can be seen as the sum of the α -quantile and a non-negative term and thus, Q α [ g ( d , Z )] is a quantity higher (as indicated by “super”) than Q α [ g ( d , Z )]. It follows from the definitionthat Q α [ g ( d , Z )] ≥ Q α [ g ( d , Z )]. When the cumulative distribution of g ( d , Z ) is continuous for any d ,we can also view Q α [ g ( d , Z )] as the conditional expectation of g ( d , Z ) with the condition that g ( d , Z ) isnot less than Q α [ g ( d , Z )], i.e., Q α [ g ( d , Z )] = E [ g ( d , Z ) | g ( d , Z ) ≥ Q α [ g ( d , Z )]] [5]. We also note that bydefinition [4] for α = 0 , Q [ g ( d , Z )] = E [ g ( d , Z )] , andfor α = 1 , Q [ g ( d , Z )] = ess sup g ( d , Z ) , (7)where ess sup g ( d , Z ) is the lowest value that g ( d , Z ) doesn’t exceed with probability 1.Figure 3 illustrates the Q α risk measure for two differently shaped, generic distributions of the limitstate function. The figure shows that the magnitude of Q α − Q α (or the induced conservativeness) changeswith the underlying distribution. Algorithm 1 describes standard MC sampling for approximating Q α . Thesecond term on the right hand side in Equation (8) is a MC estimate of the expectation in Equation (6). As noted before, the PoF constraint of the RBDO problem in (1) can be viewed as a Q α constraint (as seenin (3)). The PoF constraint (and thus the Q α constraint) does not consider the magnitude of the failureevents, but only whether they are larger than the failure threshold. This could be a potential drawback6 − αQ α Q α g ( d , Z ) P r o b a b ili t y d e n s i t y (a) α − αQ α Q α g ( d , Z ) P r o b a b ili t y d e n s i t y (b) Figure 3: Illustration for Q α on two generic distributions: expectation of the worst-case 1 − α outcomesshown in blue is Q α [ g ( d , Z )]. Algorithm 1
Sampling-based estimation of Q α and Q α . Input: m i.i.d. samples z , . . . , z m of random variable Z , design variable d , risk level α ∈ (0 , g ( d , Z ). Output:
Sample approximations (cid:98) Q α [ g ( d , Z )], (cid:98) Q α [ g ( d , Z )]. Evaluate limit state function at the samples to get g ( d , z ) , . . . , g ( d , z m ). Sort values of limit state function in descending order and relabel the samples so that g ( d , z ) > g ( d , z ) > · · · > g ( d , z m ) . Find the index k α = (cid:100) m (1 − α ) (cid:101) to estimate (cid:98) Q α [ g ( d , Z )] ← g ( d , z k α ). Estimate (cid:98) Q α [ g ( d , Z )] = (cid:98) Q α [ g ( d , Z )] + 1 m (1 − α ) m (cid:88) j =1 (cid:104) g ( d , z j ) − (cid:98) Q α [ g ( d , Z )] (cid:105) + . (8)for engineering applications. On the other hand, a Q α constraint considers the magnitude of the failureevents by specifically constraining the expected value of the largest 100(1 − α )% realizations of g ( d , Z ).Additionally, depending upon the actual construction of g ( d , z ) and the accuracy of the sampling procedure,the Q α constraint may have numerical advantages over the Q α constraint when it comes to optimization asdiscussed later. In particular, we have the optimization problem formulationmin d ∈D E [ f ( d , Z )]subject to Q α T [ g ( d , Z )] ≤ t, (9)where α T is the desired reliability level given the limit state failure threshold t . The Q α -based formulationtypically leads to a more conservative design than when PoF is used. This can be observed by noting that Q α T [ g ( d , Z )] ≤ t = ⇒ Q α T [ g ( d , Z )] ≤ t ⇐⇒ p t ( g ( d , Z )) ≤ − α T . Therefore, if the design satisfies the Q α T constraint, then the design will also satisfy the related PoF constraint. Additionally, since the Q α T constraint ensures that the average of the (1 − α T ) tail is no larger than t , it is likely that the probabilityof exceeding t (PoF) is strictly smaller than 1 − α T and is thus a conservative design for target reliability of α T . Intuitively, this conservatism comes from the fact that Q α T considers the magnitude of the worst failureevents.The formulation with Q α T as the constraint is useful when the designer is unsure about the failureboundary location for the problem but requires a certain level of reliability from the design. For example,consider the case where the failure is defined as maximum stress of a structure not exceeding a certain value.However, the designers cannot agree on the cut-off value for stress but can agree on the desired level ofreliability they want. One can use this formulation to design a structure with a given reliability (1 − α T )7hile constraining a conservative estimate of the cut-off value ( Q α T ) on the stress. Remark 3 (Convexity in Q α -based optimization) It can be shown that Q α can be written in the formof an optimization problem [5] as Q α [ g ( d , Z )] = min γ ∈ R γ + 11 − α E (cid:104) [ g ( d , Z ) − γ ] + (cid:105) , (10) where d is the given design, γ is an auxiliary variable, and [ c ] + := max { , c } . At the optimum, γ ∗ = Q α [ g ( d , Z )] . Using Equation (10) , the formulation (9) can be reduced to an optimization problem involvingonly expectations as given by min γ ∈ R , d ∈D E [ f ( d , Z )]subject to γ + 11 − α T E (cid:104) [ g ( d , Z ) − γ ] + (cid:105) ≤ t. (11) The formulation (11) is a convex optimization problem when g ( d , Z ) and f ( d , Z ) are convex in d since [ · ] + is a convex function and preserves the convexity of the limit state function. Another advantage of (11) ,as outlined in Ref. [5], is that the nonlinear part of the constraint, E (cid:104) [ g ( d , Z ) − γ ] + (cid:105) , can be reformulatedas a set of convex (linear) constraints if g ( d , Z ) is convex (linear) in d and has a discrete (or empirical)distribution with the distribution of Z being independent of d . Specifically, consider a MC estimate where z i , i = 1 , . . . , m are m samples from probability distribution π . Then, using auxiliary variables b i , i = 1 , . . . , m to define b = { b , . . . , b m } , we can reformulate (11) as min γ ∈ R , b ∈ R m , d ∈D E [ f ( d , Z )]subject to γ + 1 m (1 − α T ) m (cid:88) i =1 b i ≤ t,g ( d , z i ) − γ ≤ b i , i = 1 , . . . , m,b i ≥ , i = 1 , . . . , m. (12) The formulation (12) is a linear program when g ( d , Z ) and f ( d , Z ) are linear in d . As noted in Remark 3, the formulations in (11) and (12) are convex (or linear) only when the underlyingfunctions g ( d , Z ) and f ( d , Z ) are convex (or linear) in d . However, the advantages and possibility of suchformulations indicates that one can achieve significant gains by investing in convex (or linear) approximationsfor the underlying functions. The α -superquantile Q α naturally arises as a replacement for Q α in the constraint, but it can also beused as the objective function in the optimization problem formulation. For example, in PDE-constrainedoptimization, superquantiles have been used in the objective function [16, 17]. The optimization formulationis min d ∈D Q α T [ g ( d , Z )]subject to Q β T [ f ( d , Z )] ≤ C T , (13)where α T and β T are the desired risk levels for g and f respectively, and C T is a threshold on the quantityof interest f . This is a useful formulation when it is easier to define a threshold on the quantity of interestthan deciding a risk level for the limit state function. For example, if the quantity of interest is the costof manufacturing a rocket engine, one can specify a budget constraint and use the above formulation. Thesolution of this optimization formulation would result in the safest rocket engine design such that the expectedbudget is does not exceed the given budget. In the case where π depends upon d , one can perform optimization by using sampling-based estimators for the gradient of Q α [57, 58]. .2.4 Discussion on superquantile-based optimization From an optimization perspective, an important feature of Q α is that it preserves convexity of the functionit is applied to, i.e., the limit state function or cost function. Q α -based formulations can lead to well-behaved convex optimization problems that allows one to provide convergence guarantees as described inRemark 3. The reformulation offers a major advantage, since an optimization algorithm can work directlyon the limit state function without passing through an indicator function. This preserves the convexity andother mathematical properties of the limit state function. Q α also takes the magnitude of failure into account,which makes it more informative and resilient compared to PoF and builds in data-informed conservativeness.As noted in [59], Q α estimators are less stable than estimators of Q α since rare, large magnitude tailsamples can have large effect on the sample estimate. This is more prevalent when the distribution of therandom quantity is fat-tailed. Thus, there is a need for more research to develop efficient algorithms for Q α estimation. Despite offering convexity, a drawback of Q α is that it is non-smooth, and a direct Q α -basedoptimization would require either non-smooth optimization methods, for example variable-metric algorithms[60], or gradient-free methods. Note that smoothed approximations exist [16, 23], which significantly improveoptimization performance. In addition, the formulation (12) offers a smooth alternative.As noted in Remark 3, Q α -based formulations can be further reduced to a linear program. The formula-tion in (12) increases the dimensionality of the optimization problem from n d +1 to n d + m +1, where m is thenumber of MC samples, which poses an issue when the number of MC samples is large. However, formulation(12) has mostly linear constraints and can also be completely converted into a linear program by using alinear approximation for g ( d , z i ) (following similar ideas as reliability index methods described in Section 1).There are extremely efficient methods for finding solutions to linear programs even for high-dimensionalproblems. Buffered probability of failure was first introduced by Rockafellar and Royset [6] as an alternative to PoF. Thissection describes bPoF and the associated optimization problem formulations. When used as constraints,bPoF and superquantile lead to equivalent optimization formulations but bPoF provides an alternativeinterpretation of the Q α constraint that is, arguably, more natural for applications dealing with constraintsin terms of failure probability instead of constraints involving quantiles. When considered as an objectivefunction, bPoF and superquantile lead to different optimal design solutions. The bPoF is an alternate measure of reliability which adds a buffer to the traditional PoF. The definition ofbPoF at a given design d is based on the superquantile as given by p t ( g ( d , Z )) := (cid:8) − α | Q α [ g ( d , Z )] = t (cid:9) , if Q [ g ( d , Z )] < t < Q [ g ( d , Z )]0 , if t ≥ Q [ g ( d , Z )]1 , otherwise . (14)The domains of the threshold t in Equation (14) can interpreted in more intuitive terms using Equation (7)for Q [ g ( d , Z )] and Q [ g ( d , Z )]. The relationship between superquantiles and bPoF in the first condition inEquation (14) can also be viewed in the same way as that connecting α -quantile and PoF by recalling that Q α [ g ( d , Z )] ≤ t ⇐⇒ p t ( g ( d , Z )) ≤ − α and here, Q α [ g ( d , Z )] ≤ t ⇐⇒ p t ( g ( d , Z )) ≤ − α. (15)To make the concept of buffer concrete, we further analyze the case in the first condition in Equa-tion (14) when t ∈ (cid:0) Q [ g ( d , Z )] , Q [ g ( d , Z )] (cid:1) and g ( d , Z ) is a continuous random variable, which leads to p t ( g ( d , Z )) = (cid:8) − α | Q α [ g ( d , Z )] = t (cid:9) . Using the definition of quantiles from Equation (2) and its connec-tion with superquantiles (see Equation (6) and Figure 3), we can see that 1 − α = P [ g ( d , Z ) ≥ Q α [ g ( d , Z )]].This leads to another definition of bPoF in terms of probability of exceeding a quantile given the conditionon α as p t ( g ( d , Z )) = P [ g ( d , Z ) ≥ Q α [ g ( d , Z )]] = 1 − α, where α is such that Q α [ g ( d , Z )] = t. (16)9e know that superquantiles are conservative as compared to quantiles (Section 3. 3.2. 3.2.1), which leadsto Q α ≤ t since Q α = t . Thus, Equation (16) can be split as a sum of PoF and the probability of near-failure as p t ( g ( d , Z )) = P [ g ( d , Z ) > t ] + P [ g ( d , Z ) ∈ [ Q α [ g ( d , Z )] , t ]] = p t ( g ( d , Z )) + P [ g ( d , Z ) ∈ [ λ, t ]] , (17)where α is such that t = Q α [ g ( d , Z )] leading to λ = Q α [ g ( d , Z )]. The value of λ is affected by the conditionon α through superquantiles can be seen as the tail expectation beyond α -quantile is equal to t and takesinto account the frequency and magnitude of failure. Thus, the near-failure region [ λ, t ] is determined bythe frequency and magnitude of tail events around t and can be intuitively seen as the buffer on top of thePoF. An illustration of the bPoF risk measure is shown in Figure 4. Algorithm 2 describes standard MCsampling for estimating bPoF. buffer p t = P [ g ( d , Z ) > t ]¯ p t = P [ g ( d , Z ) ∈ [ λ, t ]] + p t = 1 − αQ α = λ Q α = t g ( d , Z ) P r o b a b ili t y d e n s i t y Figure 4: Illustration for bPoF: for a given threshold t , PoF equals the area in red while bPoF equals thecombined area in red and blue . Algorithm 2
Sampling-based estimation of bPoF.
Input: m i.i.d. samples z , . . . , z m of random variable Z , design variable d , failure threshold t , and limitstate function g ( d , Z ). Output:
Sample approximation (cid:98) p t ( g ( d , Z )). Evaluate limit state function at the samples to get g ( d , z ) , . . . , g ( d , z m ). Sort values of limit state function in descending order and relabel the samples so that g ( d , z ) > g ( d , z ) > . . . > g ( d , z m ) . c = g ( d , z ) (cid:46) Initialize superquantile estimate k = 1 while c ≥ t do (cid:46) Check if superquantile estimate equals threshold k ← k + 1 c = k (cid:80) ki =1 g ( d , z k ) (cid:46) Update superquantile estimate end while Estimate bPoF as (cid:98) p t ( g ( d , Z )) ≈ k − m (cid:46) Estimate bPoF as 1 − α when c ≈ t In general, we can see that for any design d , p t ( g ( d , Z )) ≥ p t ( g ( d , Z )) . (18)Through Equation (17), we can see that the conservatism of bPoF comes from the data-dependent mechanismthat selects the conservative threshold λ ≤ t , which acts to establish a buffer zone. If realizations of g ( d , Z )beyond t are very large (potentially catastrophic failures), λ will need to be smaller (making bPoF bigger)to drive the expectation beyond λ to t . Thus, the larger bPoF serves to account for not only the frequencyof failure events, but also their magnitude. The bPoF also accounts for the frequency of near-failure events10hat have magnitude below, but very close to t . If there are a large number of near-failure events, bPoF willtake this into account, since it will be included in the λ -tail which must have average equal to t . Thus, thebPoF is a conservative estimate of the PoF for any design d and carries more information about failure thanPoF since it takes into consideration the magnitude of failure. Remark 4 (Continuity of bPoF w.r.t. threshold)
In practice, thresholds are sometimes set by regula-tory commissions, informed by industry standards, without a full analysis the consequences. As discussedbefore, the data-informed conservativeness of bPoF reduces the adverse effects of poorly chosen thresholdsby building a buffer around the threshold t . Another issue with poorly set thresholds is that the values couldchange as one learns more about the system. In such cases, continuity of the risk measure w.r.t. the thresholdbecomes important. bPoF is continuous w.r.t. the threshold but PoF is not. Consequently, if an engineermakes small changes to the threshold t , then it can have significant effects on the resulting design when PoFis used in the optimization formulation. On the other hand, small changes in t will only have small effecton the bPoF-based optimal design due to bPoF being continuous w.r.t. t .The following example illustrates the continuity of bPoF vs PoF w.r.t. the threshold. Let X be a randomvariable with finite distribution probability mass function given by P ( X = x ) = . , if x = − . , if x = 00 . , if x = 1 , which is visualized in Figure 5(a). For this simple distribution, one can derive the PoF and bPoF analyticallyfor any given threshold t . The PoF values for different values of t are p t = , if t < − . , if t ∈ [ − , . , if t ∈ [0 , , if t ≥ , which is clearly not continuous in t . The bPoF values for different values of t are p t = , if t < − . . / ( t + 1) , if t ∈ [ − . , . . /t, if t ∈ [0 . , , if t ≥ , which is continuous in t on the interval (cid:0) −∞ , Q (cid:1) = ( −∞ , . The PoF and bPoF values as a function of thethreshold t are plotted in Figure 5(b) showing the continuity of bPoF in t . In a similar way, superquantiles Q α are continuous in α but quantiles Q α are not. One of the advantages of bPoF, which provides data-informed conservativeness, is the intuitive relatability tothe widely used PoF. This helps in easy transition from PoF-based formulations to bPoF-based formulations.Consider the optimization problem (1) with the PoF constraint replaced by the bPoF constraint,min d ∈D E [ f ( d , Z )]subject to p t ( g ( d , Z )) ≤ − α T . (19)Just as a PoF constraint is equivalent to a Q α constraint, it can be shown that the bPoF constraint for-mulation described above is equivalent to a Q α constraint (see Equation (15)). It can be observed that thebPoF-based formulation (19) is equivalent to the Q α -based optimization formulation (9) by noting that thebPoF constraint being active implies that the Q α constraint is also active, i.e., p t ( g ( d , Z )) = 1 − α T = ⇒ Q α T [ g ( d , Z )] = t . However, formulation (19) is useful when considered in the context of interpretabilityw.r.t. the originally intended PoF reliability constraint along with the data-informed conservative bufferprovided by bPoF. In engineering applications, the exact failure threshold is often uncertain and chosen bya subject matter expert. Thus, it is beneficial that bPoF can provide a reliability constraint that is robustto uncertain or inexact choices of failure threshold. 11 .20.40.60.81-1.5 -1 -0.5 0 0.5 1 1.5 (a) -1.5 -1 -0.5 0 0.5 1 1.50.20.40.60.81 (b) Figure 5: Illustrating (a) the probability mass function of X and (b) the continuity of bPoF w.r.t. changingthreshold values as compared to discontinuous nature of PoF. Remark 5 (Convexity in bPoF-based optimization)
It can be shown that bPoF can be written in theform of a convex optimization problem, similar to Q α , as [61, 20] p t ( g ( d , Z )) = min λ The problem consists of designing a short column with rectangular cross-section of dimensions w and h ,subjected to uncertain loads (axial force F and bending moment M ). The yield stress of the material, Y ,is also considered to be uncertain. The random variables are Z = [ F, M, Y ] (cid:62) with a joint distribution π .Table 1 describes the random variables used in the short column design. The correlation coefficient between F and M is 0.5. The design variables, d = [ w, h ] (cid:62) , are the length and width of the cross-section as shownin Table 2. The objective function is the cross-sectional area given by wh . Along with a failure threshold t = 1, the limit state function is defined as g ( d , z ) = 4 Mwh Y + F w h Y . (25)13able 1: Random variables used in the short column application.Randomvariable Units Distribution Mean Standard deviation F kN Normal 500 100 M kNm Normal 2000 400 Y MPa Log-normal 5 0.5Table 2: Design variables used in the short column application.Design variable Lower bound (m) Upper bound (m) w h 15 25 This section provides the optimization formulations based on PoF and bPoF for the short column structuraldesign. We show a convex reformulation of the bPoF-based optimization short column problem to emphasizethe specific advantage of bPoF risk measure making it strongly certifiable. For each case, we solve multipleoptimization problems each with a different fixed value of desired reliability level 1 − α T . The RBDO problem is given by min w,h wh subject to p t ( g ( d , Z )) ≤ − α T ,(cid:96) w ≤ w ≤ u w ,(cid:96) h ≤ h ≤ u h , (26)where ( (cid:96) w , (cid:96) h , u w , u h ) denote the lower and upper bounds on w and h as defined in Table 2. The bPoF-based optimization problem for the short column design ismin λ Remark 6 (Desirable data-informed conservativeness) We take a closer look at the conservativenessinduced by the bPoF-based CRiBDO for the same desired reliability level as compared to PoF-based optimiza-tion (as shown in Figure 6(c)) and why this type of conservativeness would be desirable and resilient. Thereare other ways of introducing conservativeness, such as safety factors, basis values, and stricter reliabilitylevels. Typically, using safety factors leads to overly conservative designs. When inappropriate safety factorvalues are used, the deterministic optimization setup could also potentially lead to unreliable designs. Thisis because converting to deterministic optimization using just safety factors (or basis values) to account forthe uncertainty in the system does not take into account the distribution of the limit state function and lackssufficient information to make good decisions. These well-known issues with safety-factor- and basis-values-based deterministic optimization formulations have progressively led us to consider risk-based optimizationunder uncertainty. Another way to introduce conservativeness in risk-based optimization is by using lowervalues of − α T , which leads to stricter reliability constraints. However, this will just lead to overly reliabledesigns without any information about the distribution of the limit state function. he bPof-based CRiBDO can be seen as a better way to induce conservativeness because it encodes moreinformation about the underlying limit state function through the data on the magnitude of failure (as seenfrom Equations (16) and (18) ). In Section 5, we highlight a similar observation on conservativeness for Q α -based CRiBDO through the thermal design problem. These CRiBDO formulations lead to a probabilisticdata-informed way of achieving a conservative design, which can be seen as more desirable in practice. Figure 7 compares the desired reliability levels versus the estimated PoF or bPoF for the optimal designsobtained through PoF- and bPoF-based optimization. For these plots, we use 5 × samples to getaccurate estimates of the PoF or bPoF at the optimum. Figure 7(a) shows the desired reliability level andthe PoF/bPoF for the optimum design obtained using the RBDO problem. We can see that since the MCerror for PoF estimate in the RBDO problem was always ensured to be below 1%, the desired PoF and thePoF at the optimum overlap. The figure also shows the conservative property of bPoF for the same desiredreliability level.A key observation is illustrated by Figure 7(b), which compares the results for optimal designs obtainedusing bPoF-based CRiBDO for different a priori sample sizes, i.e. the value of m in Equation (30). We makethis comparison to analyze the effect of fixing the sample set for all optimization iterations before startingthe optimization, which is required to obtain the convex optimization formulation shown in Equation (30).We can see that for lower sample sizes of 10 and 10 , the bPoF at the optimum and the desired bPoFdo not overlap reflecting inaccurate MC estimates of bPoF. However, it should be noted that the bPoFformulation is still effective in controlling the PoF, even when sample size is small. In other words, evenwhen small number of samples are used within the optimization, the nature of bPoF yields an optimal designwith desirable conservativeness and thus, an acceptably low PoF. Additionally, even when formulated witha small sample size, the bPoF-based convex optimization problem is still considerably stable leading to goodoptimal designs. One of the primary drawbacks of RBDO is the potential fragility of the optimization,particularly when sample sizes are small, where the estimates of PoF and/or gradients (if a gradient-basedsolver is used) are unstable and produce poor or inconsistent optimization results. The bPoF formulationdoes not seem to suffer in the same way for the short column design as illustrated here. (a) RBDO designs (b) bPoF-based CRiBDO designs Figure 7: Comparing (a) bPoF estimates at the optimal designs obtained through RBDO, and (b) PoFestimates at the optimal designs obtained through bPoF-based CRiBDO using different a priori samplesizes for different desired reliability levels. These samples are not used in the optimization, but only to estimate at the optimal design after the optimization iscompleted. The a priori sample sizes m used for the bPoF CRiBDO are indicated in the legend using bPoF- m . Thermal Design: Cooling Fin Problem In this section, we compare the properties of PoF- and Q α -based optimization formulations for the thermaldesign of a cooling fin problem. We consider a cooling fin with fixed geometry as shown in Figure 8, consisting of a vertical post withhorizontal fins attached. We briefly review the problem here and refer to [66] for more details. The fin arrayconsists of four horizontal sub-fins with width 2.5 and thickness 0.25, as well as a fin post with unit widthand height four. The thermal design is parametrized by the fin conductivities k i , i = 1 , . . . , k , as well as the Biot number Bi , which is a non-dimensionalized heat transfer coefficient forthermal transfer from the fins to the surrounding air. The design variables, d = [ k , k , k , k ] are the thermalconductivities of the four fins as shown in Table 3. The post conductivity is k = 5 and the Biot numberis Bi = 0 . 5. We introduce manufacturing and operational uncertainties in all the parameters through therandom variable Z = [ ξ , ξ , ξ , ξ , ξ , ξ Bi ] (cid:62) with a joint distribution π given in Table 4. The randomvariables ξ i model the additive uncertainty for the respective thermal conductivities k i , i = 0 , . . . , ξ Bi models the additive uncertainty for the Biot number Bi . The system is governed by Poisson’s equationin two spatial dimensions denoted by x whose solution is the temperature field y ( x , d , Z ). The PDE issemi-discretized with the finite element method and yields a system with 4 , 760 degrees of freedom. k k k k k Γ root . . Figure 8: Fin geometry and model parameters.The fin conducts heat away from the root Γ root , so the lower the root temperature, the more effective thecooling fin. Thus, our objective function depends on the measure of the average temperature at the root,i.e., Y ( d , Z ) = (cid:90) Γ root y ( x , d , Z )d x. (31)We also include a quantity proportional to the cost of the material based on the area and material thermalconductivity in the objective function as shown in Section 5.5.2. The limit state function for the cooling finproblem is based on the maximum temperature and is defined as g ( d , Z ) = max x y ( x , d , Z ) . (32)We choose t = 0 . 35 as the constraint on the limit state function to define the maximum allowable temperatureof the system. This section provides the optimization formulations based on PoF and Q α for the cooling fin thermal design.18able 3: Design variables used in the cooling fin application.Design variable Lower bound Upper bound k i , i = 1 , . . . µ Standard deviation σξ i , i = 0 , . . . µ − σ, µ + 2 σ ]) 0 0.1 ξ Bi The RBDO problem is given by min d ∈D Y ( d , µ Z ) + (cid:80) i =0 A i k i A + (cid:80) i =1 A i subject to p t ( g ( d , Z )) ≤ − α T , (33)where A i denotes the area for the material with thermal conductivity of k i , i = 0 , . . . , A i k i representsa quantity proportional to the cost of the material. Here, the fin post area is A = 4 and sub-fin areasare A i = 1 . , i = 1 . . . , 4. The cost part is normalized by the maximum proportionate cost. We use twodifferent values of 1 − α T ∈ { . , . } . Q α -constrained CRiBDO The Q α -constrained CRiBDO formulation for the cooling fin design ismin d ∈D Y ( d , µ Z ) + (cid:80) i =0 A i k i A + (cid:80) i =1 A i subject to Q α T [ g ( d , Z )] ≤ t, (34)where we find the optimal designs for two different values of 1 − α T ∈ { . , . } . In this case, theunderlying limit state function is not known to be convex making the CRiBDO formulation (34) certifiablein one condition, which is the data-informed conservativeness. Q α -constrained CRiBDO We compare the optimal results obtained through RBDO and Q α -constrained CRiBDO formulations underthe same α T values. We solve the RBDO and CRiBDO problems using the gradient-free COBYLA optimizer.We estimate the PoF in each RBDO iteration by iteratively adding samples until the MC error reaches below1% with the maximum number of samples capped at 10 . We estimate the Q α in each CRiBDO iterationby using 10 MC samples.Table 5 shows the optimal designs obtained from the different optimization formulations. We start theoptimization with an initial design that is feasible for all the optimization formulations. The optimal designsobtained using Q α -constrained CRiBDO for a given α T are more conservative than the RBDO designs. Thishighlights one of the major advantages of using Q α -constrained CRiBDO that certifies designs through thedata-informed conservativeness. The conservative nature of the Q α -constrained CRiBDO can be clearly seenby comparing the limit state function distributions at the optimal designs as shown in Figure 9. As discussedin Remark 6, this conservativeness is desirable and required to prevent catastrophic failures. Figure 10compares the specified hard thresholds with the Q α for the different optimal designs to further highlight thefact that Q α considering the magnitude of failure and not using hard thresholding leads to appropriatelyconservative designs. The data-informed nature of the conservativeness is a significant advantage since the19agnitude of conservativeness induced automatically changes according to the data from the underlyinglimit state function distribution for a particular design, i.e., Q α is more conservative only when it is requiredas dictated by the underlying distribution. We explicitly show the data-informed nature of conservativenessin the next section.Table 5: Optimal designs obtained from RBDO and Q α -constrained CRiBDO.Designvariable/Outputstatistic Initialdesign RBDO Q α -constrained CRiBDO1 − α T =0 . 001 1 − α T = 0 . 05 1 − α T =0 . 001 1 − α T = 0 . k k k k . 001 0.0486 . × − Q α T − α T =0 . − α T = 0 . (a) 1 − α T = 0 . (b) 1 − α T = 0 . Figure 9: Histograms comparing limit state function distributions for optimal designs obtained throughdifferent optimization formulations. In this section, we demonstrate the data-informed nature of the conservativeness induced by Q α as describedin Remark 6. The magnitude of conservativeness is naturally adjusted for different limit state functiondistributions. Since, Q α and Q α are natural counterparts, we quantify the magnitude of conservativenessby the percentage difference ( Q α − Q α ) /Q α %. We fix the design and the α value for the comparison in thissection. We use the optimal design obtained by RBDO with 1 − α T = 0 . 05 given in Table 5 as the fixed design d = [3 . , . , . , MC samples. 20 .31 0.32 0.33 0.34 0.35 0.36 0.3701000200030004000 (a) RBDO (1 − α T = 0 . (b) RBDO (1 − α T = 0 . (c) Q α CRiBDO (1 − α T = 0 . (d) Q α CRiBDO (1 − α T = 0 . Figure 10: Comparing specified thresholds and Q α for optimal designs obtained through different optimiza-tion formulations.Figure 12 shows the different levels of conservativeness of Q α when compared to Q α for different limitstate function distributions. We can see that Q α is always conservative when compared to Q α . Furthermore,it can be seen that the magnitude of conservativeness depends on the distribution, which exemplifies thedata-informed nature of the induced conservativeness, i.e., Q α is as conservative as required by the underlyingdistribution. Specifically, the magnitude of conservativeness for superquantile is higher for the fatter-taileddistribution as seen in Figure 12 (a). For fat-tailed distributions, i.e., distributions with significant tail risk,superquantile provides additional conservativeness by nature of being a tail-integral. In this work, we propose two certifiability conditions that lead to certifiable risk-based design optimization(CRiBDO): (a) data-informed conservativeness: the resulting designs should be certifiably risk-averse againstnear-failure and catastrophic failure events, and (b) optimization convergence and efficiency: the resultingdesigns should be certifiably optimal in comparison with all alternate designs at reduced computationalcost for the optimization. The risk measures satisfying either of the certifiability conditions are classifiedunder CRiBDO while satisfying both conditions makes the resulting optimal designs strongly certifiable. Wecompare and contrast the existing RBDO formulation based on probability of failure (PoF) with risk-basedoptimization formulations using the buffered probability of failure (bPoF) and the superquantile (a.k.a.conditional value-at-risk) risk measures. We show that RBDO does not satisfy either of the certifiabilityconditions while superquantiles and bPoF lead to CRiBDO formulations. An additional advantage of bPoF21 .31 0.32 0.33 0.34 0.35 0.36 0.370100020003000400050006000 Figure 11: Histograms for different limit state function distributions generated through modifying inputuncertainty truncation range for a fixed design d = [3 . , . , . , (a) [ µ − σ, µ + 2 σ ] (b) [ µ − σ, µ + 2 σ ] (c) [ µ − σ, µ + σ ] Figure 12: Conservativeness induced by Q α compared to Q α for different limit state function dis-tributions generated through modifying input uncertainty truncation range for a fixed design d =[3 . , . , . , 1] and 1 − α = 0 . cknowledgement This work has been supported in part by the Air Force Office of Scientific Research (AFOSR) MURI on man-aging multiple information sources of multi-physics systems award numbers FA9550-15-1-0038 and FA9550-18-1-0023, and Air Force Center of Excellence on Multi-Fidelity Modeling of Rocket Combustor Dynamicsaward FA9550-17-1-0195. The fourth author acknowledges the support from Office of Naval Research underMIPR N0001420WX00519. References [1] Rockafellar, R. T. and Uryasev, S., “Conditional value-at-risk for general loss distributions,” Journal ofBanking & Finance , Vol. 26, No. 7, 2002, pp. 1443–1471.[2] Krokhmal, P., Palmquist, J., and Uryasev, S., “Portfolio optimization with conditional value-at-riskobjective and constraints,” Journal of Risk , Vol. 4, No. 2, 2002, pp. 11–27.[3] Mansini, R., Ogryczak, W., and Speranza, M. G., “Conditional value at risk and related linear program-ming models for portfolio optimization,” Annals of Operations Research , Vol. 152, 2007, pp. 227–256.[4] Rockafellar, R. T. and Royset, J. O., “Superquantiles and their applications to risk, random variables,and regression,” Theory Driven by Influential Applications , Informs, 2013, pp. 151–167.[5] Rockafellar, R. T. and Uryasev, S., “Optimization of conditional value-at-risk,” Journal of Risk , Vol. 2,2000, pp. 21–42.[6] Rockafellar, R. T. and Royset, J. O., “On buffered failure probability in design and optimization ofstructures,” Reliability Engineering & System Safety , Vol. 95, No. 5, 2010, pp. 499–510.[7] Roland, H. E. and Moriarty, B., System safety engineering and management , John Wiley & Sons, 1990.[8] M¨oller, N. and Hansson, S. O., “Principles of engineering safety: Risk and uncertainty reduction,” Reliability Engineering & System Safety , Vol. 93, No. 6, 2008, pp. 798–805.[9] Suzuki, Y. and Haftka, R. T., “Analytical benchmark example for risk allocation in structural optimiza-tion,” Structural and Multidisciplinary Optimization , Vol. 50, No. 1, 2014, pp. 1–7.[10] Rockafellar, R. T. and Royset, J. O., “Engineering decisions under risk averseness,” ASCE-ASMEJournal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering , Vol. 1, No. 2,2015, pp. 04015003.[11] Zhang, W., Rahimian, H., and Bayraksan, G., “Decomposition algorithms for risk-averse multistagestochastic programs with application to water allocation under uncertainty,” INFORMS Journal onComputing , Vol. 28, No. 3, 2016, pp. 385–404.[12] Royset, J. O., Bonfiglio, L., Vernengo, G., and Brizzolara, S., “Risk-adaptive set-based design andapplications to shaping a hydrofoil,” Journal of Mechanical Design , Vol. 139, No. 10, 2017, pp. 101403.[13] Bonfiglio, L. and Royset, J. O., “Multidisciplinary Risk-Adaptive Set-Based Design of SupercavitatingHydrofoils,” AIAA Journal , Vol. 57, No. 8, 2019, pp. 3360–3378.[14] Yang, H. and Gunzburger, M., “Algorithms and analyses for stochastic optimization for turbofan noisereduction using parallel reduced-order modeling,” Comput. Methods Appl. Mech. Engrg. , Vol. 319, 2017,pp. 217–239.[15] Chaudhuri, A., Peherstorfer, B., and Willcox, K., “Multifidelity Cross-Entropy Estimation of Condi-tional Value-at-Risk for Risk-Averse Design Optimization,” AIAA Scitech 2020 Forum , 2020, p. 2129.[16] Kouri, D. P. and Surowiec, T. M., “Risk-averse PDE-constrained optimization using the conditionalvalue-at-risk,” SIAM Journal on Optimization , Vol. 26, No. 1, 2016, pp. 365–396.2317] Zou, Z., Kouri, D. P., and Aquino, W., “A locally adapted reduced basis method for solving risk-averse PDE-constrained optimization problems,” , 2018.[18] Heinkenschloss, M., Kramer, B., Takhtaganov, T., and Willcox, K., “Conditional-value-at-risk estima-tion via reduced-order models,” SIAM/ASA Journal on Uncertainty Quantification , Vol. 6, No. 4, 2018,pp. 1395–1423.[19] Heinkenschloss, M., Kramer, B., and Takhtaganov, T., “Adaptive reduced-order model construction forconditional value-at-risk estimation,” SIAM/ASA Journal on Uncertainty Quantification , Vol. 8, No. 2,2020, pp. 668–692.[20] Mafusalov, A. and Uryasev, S., “Buffered probability of exceedance: mathematical properties andoptimization,” SIAM Journal on Optimization , Vol. 28, No. 2, 2018, pp. 1077–1103.[21] Norton, M., Khokhlov, V., and Uryasev, S., “Calculating CVaR and bPOE for common probabilitydistributions with application to portfolio optimization and density estimation,” Annals of OperationsResearch , 2019, pp. 1–35.[22] Rockafellar, R. T. and Uryasev, S., “Minimizing buffered probability of exceedance by progressivehedging,” Mathematical Programming , 2020, pp. 1–20.[23] Basova, H. G., Rockafellar, R. T., and Royset, J. O., “A computational study of the buffered failureprobability in reliability-based design optimization,” Proceedings of International Conference on Ap-plications of Statistics and Probability in Civil Engineering (ICASP), Zurich, Switzerland , Citeseer,2011.[24] Minguez, R., Castillo, E., and Lara, J., “Iterative scenario reduction technique to solve reliability basedoptimization problems using the buffered failure probability,” Proceedings of ICOSSAR , 2013.[25] Harajli, M. M., Rockafellar, R. T., and Royset, J. O., “Importance sampling in the evaluation andoptimization of buffered failure probability,” Proceedings of International Conference on Applicationsof Statistics and Probability in Civil Engineering (ICASP), Vancouver, Canada, July 12-15 , 2015.[26] Royset, J. O., G¨unay, S., and Mosalam, K. M., “Risk-Adaptive Learning of Seismic Response usingMulti-Fidelity Analysis,” Proceedings of the International Conference on Applied Statistics and Proba-bility in Civil Engineering (ICASP), Seoul, Korea , 2019.[27] Yao, W., Chen, X., Luo, W., van Tooren, M., and Guo, J., “Review of uncertainty-based multidisci-plinary design optimization methods for aerospace vehicles,” Progress in Aerospace Sciences , Vol. 47,No. 6, 2011, pp. 450–479.[28] Aoues, Y. and Chateauneuf, A., “Benchmark study of numerical methods for reliability-based designoptimization,” Structural and Multidisciplinary Optimization , Vol. 41, No. 2, 2010, pp. 277–294.[29] Sobieszczanski-Sobieski, J., Morris, A., and van Tooren, M., Multidisciplinary design optimization sup-ported by knowledge based engineering , John Wiley & Sons, 2015.[30] Du, X. and Chen, W., “A most probable point-based method for efficient uncertainty analysis,” Journalof Design and Manufacturing Automation , Vol. 4, No. 1, 2001, pp. 47–66.[31] Rubinstein, R. Y. and Kroese, D. P., Simulation and the Monte Carlo method , Vol. 10, John Wiley &Sons, 2016.[32] Melchers, R., “Importance sampling in structural systems,” Structural Safety , Vol. 6, No. 1, 1989,pp. 3–10.[33] Owen, A. B., Monte Carlo theory, methods and examples , 2013.2434] Au, S.-K. and Beck, J. L., “A new adaptive importance sampling scheme for reliability calculations,” Structural Safety , Vol. 21, No. 2, 1999, pp. 135–158.[35] Dey, A. and Mahadevan, S., “Ductile structural system reliability analysis using adaptive importancesampling,” Structural Safety , Vol. 20, No. 2, 1998, pp. 137–154.[36] De Boer, P.-T., Kroese, D. P., Mannor, S., and Rubinstein, R. Y., “A tutorial on the cross-entropymethod,” Annals of Operations Research , Vol. 134, No. 1, 2005, pp. 19–67.[37] Papaioannou, I., Betz, W., Zwirglmaier, K., and Straub, D., “MCMC algorithms for subset simulation,” Probabilistic Engineering Mechanics , Vol. 41, 2015, pp. 89–103.[38] Depina, I., Papaioannou, I., Straub, D., and Eiksund, G., “Coupling the cross-entropy with the linesampling method for risk-based design optimization,” Structural and Multidisciplinary Optimization ,Vol. 55, No. 5, 2017, pp. 1589–1612.[39] Kurtz, N. and Song, J., “Cross-entropy-based adaptive importance sampling using Gaussian mixture,” Structural Safety , Vol. 42, 2013, pp. 35–44.[40] Li, J., Li, J., and Xiu, D., “An efficient surrogate-based method for computing rare failure probability,” Journal of Computational Physics , Vol. 230, No. 24, 2011, pp. 8683–8697.[41] Peherstorfer, B., Kramer, B., and Willcox, K., “Combining multiple surrogate models to acceleratefailure probability estimation with expensive high-fidelity models,” Journal of Computational Physics ,Vol. 341, 2017, pp. 61–75.[42] Peherstorfer, B., Kramer, B., and Willcox, K., “Multifidelity preconditioning of the cross-entropymethod for rare event simulation and failure probability estimation,” SIAM/ASA Journal on Uncer-tainty Quantification , Vol. 6, No. 2, 2018, pp. 737–761.[43] Bichon, B. J., Eldred, M. S., Mahadevan, S., and McFarland, J. M., “Efficient global surrogate mod-eling for reliability-based design optimization,” Journal of Mechanical Design , Vol. 135, No. 1, 2013,pp. 011009.[44] Chaudhuri, A., Marques, A. N., and Willcox, K. E., “mfEGRA: Multifidelity Efficient Global ReliabilityAnalysis through Active Learning for Failure Boundary Location,” Structural and MultidisciplinaryOptimization , 2020.[45] Moustapha, M. and Sudret, B., “Surrogate-assisted reliability-based design optimization: a survey anda unified modular framework,” Structural and Multidisciplinary Optimization , Vol. 60, No. 5, 2019,pp. 1–20.[46] Gano, S. E., Renaud, J. E., Agarwal, H., and Tovar, A., “Reliability-based design using variable-fidelityoptimization,” Structures and Infrastructure Engineering , Vol. 2, No. 3-4, 2006, pp. 247–260.[47] Li, X., Qiu, H., Jiang, Z., Gao, L., and Shao, X., “A VF-SLP framework using least squares hybridscaling for RBDO,” Structural and Multidisciplinary Optimization , Vol. 55, No. 5, 2017, pp. 1629–1640.[48] Marques, A., Lam, R., and Willcox, K., “Contour location via entropy reduction leveraging multipleinformation sources,” Advances in Neural Information Processing Systems , 2018, pp. 5217–5227.[49] Kramer, B., Marques, A., Peherstorfer, B., Villa, U., and Willcox, K., “Multifidelity probability esti-mation via fusion of estimators,” Journal of Computational Physics , Vol. 392, 2019, pp. 385–402.[50] Chaudhuri, A., Marques, A. N., Lam, R., and Willcox, K. E., “Reusing Information for MultifidelityActive Learning in Reliability-Based Design Optimization,” AIAA Scitech 2019 Forum , 2019, p. 1222.[51] Chaudhuri, A., Kramer, B., and Willcox, K. E., “Information Reuse for Importance Sampling inReliability-Based Design Optimization,” Reliability Engineering & System Safety , 2020, pp. 106853.2552] Uryasev, S., “Derivatives of probability functions and some applications,” Annals of Operations Re-search , Vol. 56, No. 1, 1995, pp. 287–311.[53] Royset, J. and Polak, E., “Extensions of stochastic optimization results to problems with system failureprobability functions,” Journal of Optimization Theory and Applications , Vol. 133, No. 1, 2007, pp. 1–18.[54] Tretiakov, G. L., “Star-shaped approximation approach for stochastic programming problems withprobability function,” Optimization , Vol. 47, No. 3-4, 2000, pp. 303–317.[55] Artzner, P., Delbaen, F., Eber, J.-M., and Heath, D., “Coherent measures of risk,” MathematicalFinance , Vol. 9, No. 3, 1999, pp. 203–228.[56] Rockafellar, R. T. and Uryasev, S., “The fundamental risk quadrangle in risk management, optimizationand statistical estimation,” Surveys in Operations Research and Management Science , Vol. 18, No. 1-2,2013, pp. 33–53.[57] Lan, G. and Zhou, Z., “Algorithms for stochastic optimization with expectation constraints,” arXivpreprint arXiv:1604.03887 , 2016.[58] Tamar, A., Glassner, Y., and Mannor, S., “Optimizing the CVaR via sampling,” Twenty-Ninth AAAIConference on Artificial Intelligence , 2015, pp. 2993–2999.[59] Yamai, Y., Yoshiba, T., et al., “Comparative analyses of expected shortfall and value-at-risk: theirestimation error, decomposition, and optimization,” Monetary and Economic Studies , Vol. 20, No. 1,2002, pp. 87–121.[60] Uryasev, S., “New variable-metric algorithms for nondifferentiable optimization problems,” Journal ofOptimization Theory and Applications , Vol. 71, No. 2, 1991, pp. 359–388.[61] Norton, M. and Uryasev, S., “Maximization of AUC and buffered AUC in binary classification,” Math-ematical Programming , Vol. 174, No. 1-2, 2018, pp. 575–612.[62] Zhang, T., Uryasev, S., and Guan, Y., “Derivatives and subderivatives of buffered probability of ex-ceedance,” Operations Research Letters , Vol. 47, No. 2, 2019, pp. 130–132.[63] Kouri, D., “Higher-moment buffered probability,” Optimization Letters , Vol. 13, No. 6, 2019, pp. 1223–1237.[64] Powell, M. J., “A direct search optimization method that models the objective and constraint functionsby linear interpolation,” Advances in optimization and numerical analysis , Springer, 1994, pp. 51–67.[65] Diamond, S. and Boyd, S., “CVXPY: A Python-embedded modeling language for convex optimization,” Journal of Machine Learning Research , Vol. 17, No. 83, 2016, pp. 1–5.[66] Prud’homme, C., Rovas, D. V., Veroy, K., Machiels, L., Maday, Y., Patera, A. T., and Turinici, G.,“Reliable Real-Time Solution of Parametrized Partial Differential Equations: Reduced-Basis OutputBound Methods,”