Optimizing for Aesthetically Pleasing Quadrotor Camera Motion
OOptimizing for Aesthetically Pleasing Quadrotor Camera Motion
CHRISTOPH GEBHARDT,
AIT Lab, ETH Zürich
STEFAN STEVŠIĆ,
AIT Lab, ETH Zürich
OTMAR HILLIGES,
AIT Lab, ETH Zürich
Fig. 1. Quadrotor camera tools generate trajectories based on user-specified keyframes in time and space. Reasoning about spatio-temporal distances is hardfor users and can lead to visually unappealing results and fluctuating camera velocities.
Top row: user-specified keyframes (blue) are positioned in time, suchthat the camera first moves too slow and then needs to accelerate drastically to reach the specified end-point.
Bottom row: results of our method whichautomatically positions keyframes (blue) in time such that the camera moves smoothly over the entire trajectory (illustrative example).
In this paper we first contribute a large scale online study ( N ≈ jointly . This allows to generate globally smooth trajectories, while retaininguser control over reference timings. The formulation is posed as a variable,infinite horizon, contour-following algorithm. Finally, a comparative labstudy indicates that our optimization scheme outperforms the state-of-the-art in terms of perceived usability and preference of resulting videos. Fornovices our method produces smoother and better looking results and alsoexperts benefit from generated timings. Authors’ addresses: Christoph Gebhardt, AIT Lab, ETH Zürich; Stefan Stevšić, AIT Lab,ETH Zürich; Otmar Hilliges, AIT Lab, ETH Zürich, [email protected] to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected].© 2018 Copyright held by the owner/author(s). Publication rights licensed to Associationfor Computing Machinery.0730-0301/2018/8-ART90 $15.00https://doi.org/10.1145/3197517.3201390
CCS Concepts: •
Computing methodologies → Computer graphics ; Motion path planning ; Robotic planning ; •
Computer systems orga-nization → External interfaces for robotics;Additional Key Words and Phrases: computational design, aerial videography,quadrotor camera tools, trajectory optimization
ACM Reference Format:
Christoph Gebhardt, Stefan Stevšić, and Otmar Hilliges. 2018. Optimizing forAesthetically Pleasing Quadrotor Camera Motion.
ACM Trans. Graph.
37, 4,Article 90 (August 2018), 11 pages. https://doi.org/10.1145/3197517.3201390
Camera quadrotors have become a mainstream technology but fine-grained control of such camera drones for aerial videography is ahigh-dimensional and hence difficult task. In response several toolshave been proposed to plan quadrotor shots by defining keyframesin virtual environments [Gebhardt et al. 2016; Gebhardt and Hilliges2018; Joubert et al. 2015; Roberts and Hanrahan 2016]. This input isthen used in an optimization algorithm to automatically generatequadrotor and camera trajectories. Intuitively, smooth camera mo-tion is an obvious factor impacting the visual quality of a shot. Thisintuition alongside expert-feedback [Joubert et al. 2015] and litera-ture on (aerial) cinematography [Arijon 1976; Audronis 2014; Hen-nessy 2015] forms the basis for most existing quadrotor tools. Thesetake a spline representation, connecting user specified keyframes,and optimize higher derivatives of these splines, such as jerk.
ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. a r X i v : . [ c s . G R ] J un Balasubramanian et al. [2015] define global smoothness as “aquality related to the continuity or non-intermittency of a move-ment, independent of its amplitude and duration”. However, becausekeyframe timings are kept fixed in current quadrotor camera opti-mization schemes [Gebhardt et al. 2016; Joubert et al. 2015], or closeto the user input [Roberts and Hanrahan 2016], smooth motion canonly be generated subject to these hard-constraints. This can causestrong variation of camera velocities across different trajectory seg-ments and result in visually unpleasant videos.Consider popular fly-by-shots, such as the one illustrated in Fig-ure 1, where an object is filmed first from one direction and thengradually the camera yaws around it’s own z-axis by 180 ◦ as thequadrotor flies past the object until it is filmed from the opposingdirection. To achieve visually pleasing footage both the quadrotormotion and the camera’s angular velocity need to be smooth. Usersgenerally struggle with this or similar problems in which they placethe keyframes in the correct spatial location but too close (or toofar) to each other temporally (see Figure 1, top and [video]). Thisis indeed a difficult task because keyframes are specified in 5D (3Dposition and camera pitch and yaw) and imagining the resultingtranslational and rotational velocities is cognitively demanding.Although existing work provides UI tools (i.e. progress curves,timelines) to cope with this problem, it has been shown that users,especially novices, struggle to create smooth camera motion overa sequence of keyframes [Gebhardt and Hilliges 2018]. While op-timizing for global smoothness may address this issue for novices,an interesting tension arises when looking at experienced users.Experts explicitly time the visual progression of a shot in order toachieve desired compositional effects [Joubert et al. 2015] (e.g. ease-in, ease-out behavior). Our first contribution is a large online study( N = soft-constraints . This extends the state-of-the-art in that it allows usersto trade off path-following fidelity with temporal fidelity. Such aformulation poses significant technical difficulties. Prior methodsincorporate keyframe timings as hard-constraints , yielding a qua-dratic and hence convex optimization formulation (depending onthe dynamical model), allowing for efficient implementation. In con-trast, we formulated the quadrotor camera trajectory generationproblem as a variable, infinite horizon, contour-following algorithmapplicable to linear and non-linear quadrotor models. Our formula-tion has to discretize the model at each solver iteration accordingto the optimized trajectory end time. Although this formulationis no-longer convex, it is formulated as well-behaved non-convexproblem and our implementation runs at interactive rates. Finally, we show the benefit of our method compared to the state-of-the-art in a lab study in which we compare different variantsof our method with [Gebhardt et al. 2016]. It can be shown thatour method positively effects the usability of quadrotor cameratools and improves the visual quality of video shots for experts andnon-experts. Both benefit from using an optimized timing initially,fine-tuning it according to their intention. In addition, the userstudy revealed that timing control does not need to be precise butis rather used to control camera velocity in order to create a certaincompositional effect. Camera Control in Virtual Environments:
Camera placement [Linoand Christie 2012], path planning [Li and Cheng 2008; Yeh et al. 2011]and automated cinematography [Lino et al. 2011] have been studiedextensively in virtual environments (VE), for a survey see [Christieet al. 2008]. These works share our goal of assisting users in thecreation of camera motion (e.g., [Drucker and Zeltzer 1994; Lino andChristie 2015]). Nevertheless, it is important to consider that VEsare not limited by real-world physics and robot constraints, hencemay yield trajectories that can not be flown by a quadrotor.
Character Animation:
In character animation, a variety of meth-ods exist which are capable of trading-off positional and temporalreference fit to optimize for smoother character motion. In [Liuet al. 2006], the authors specify constraints in warped time and thenoptimize the mapping between warped and actual time accordingto their objective function. For an original motion, [McCann et al.2006] find the convex hull of all physically valid motions attainablevia re-timing. Plausible new motions are then found by performinggradient descent and penalizing distance between possible solutionsand the feasible hull. Like [Liu et al. 2006], our formulation is basedon a time-free parameterization of a reference path. In contrast tothe character animation methods, we adjust timings by optimizingthe progress of the quadrotor camera on the reference according toa objective favoring smoothness. Unlike [McCann et al. 2006] ourformulation does not require nested optimization.
Trajectory Generation:
Trajectory generation for dynamical sys-tems is a well studied problem in computer graphics [Geijtenbeekand Pronost 2012] and robotics [Betts 2009]. Approaches that encodethe system dynamics as equality constraints to solve for the controlinputs along a motion trajectory are referred to as spacetime con-straints in graphics [Witkin and Kass 1988] and direct collocationin robotics [Betts 2009]. Used out-of-the box such approaches canlead to slow convergence time especially with long time horizons(cf. [Roberts and Hanrahan 2016]).With the commoditization of quadrotors, the generation of dronetrajectories shifted into the focus of research. Exploiting the differen-tial flatness of quadrotors in the output space, [Mellinger and Kumar2011] generated physically feasible minimal snap trajectories. Sev-eral methods exist for the generation of trajectories for aggressivequadrotor flight [Bry et al. 2015; Mellinger et al. 2012]. Tradition-ally, these methods convert a sequence of input positions into atime-dependent reference and based on a dynamical model gener-ate a trajectory which follows this reference. For [Bry et al. 2015;
ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. ptimizing for Aesthetically Pleasing Quadrotor Camera Motion • 90:3
Mellinger and Kumar 2011], time optimization is done in a cascadedmanner where an approximated gradient descent for keyframe tim-ings is calculated based on the original optimization problem. Theseformulations suffer from very long runtimes as the original problemneeds to be called once for each keyframe to calculate the gradientapproximation. In contrast, our method optimizes keyframe timingsand trajectory jointly reducing optimization runtime and allowingto trade-off temporal and positional fit. In [Mellinger et al. 2012],sequentially composed controllers are used to optimize the timing ofa trajectory such that physical limits are not violated given desiredfeed-forward terms. Our work does not only ensure physical fea-sibility but is also capable of generating trajectories with differentdynamics (smooth and more aggressive) for the same spatial input.
Computational Support of Aerial Videography:
A number of toolsfor the planning of aerial videography exists. Commercially avail-able applications and consumer-grade drones often place waypointson a 2D map [APM 2016; DJI 2016; Technology 2016] or allowto interactively control the quadrotor’s camera as it tracks a pre-determined path [3D Robotics 2015]. These tools generally do notprovide means to ensure feasibility of the resulting plans and donot consider aesthetic or usability objectives in the video compo-sition task. The planning of physically feasible quadrotor cameratrajectories has recently received a lot of attention. Such tools allowfor planning of aerial shots in 3D virtual environments [Gebhardtet al. 2016; Gebhardt and Hilliges 2018; Joubert et al. 2015; Robertsand Hanrahan 2016] and employ optimization to ensure that bothaesthetic objectives and robot modeling constraints are considered.In [Joubert et al. 2015] and [Gebhardt et al. 2016], users spec-ify keyframes in time and space. These are incorporated as hard-constraints into an objective function. Solving for the trajectory onlyoptimizes camera dynamics and positions. This causes the genera-tion of locally smooth camera motion (between keyframes) but canlead to varying velocities across keyframes. Joubert et al. [2015] de-tect violations of the robot model constraints. However, correctingthese violations is offloaded to the user. In contrast, by generatingtimings or incorporating them as soft-constraints our optimizationreturns the closest feasible fit of the user-specified inputs, subject toour robot model, and generates globally smooth quadrotor cameratrajectories. [Gebhardt and Hilliges 2018] re-optimizes keyframetimings in a cascaded optimization scheme. Here an approximatedgradient on the keyframe times produced by the optimization for-mulation of [Gebhardt et al. 2016] is calculated and used to improvevisual smoothness. However, this approach is relatively slow andthe paper reports that users therefore did not make significant useof it in the evaluation. In contrast, our method runs at interactiverates optimizing trajectories of different duration within seconds(avg. 2 . s ). Roberts and Hanrahan [2016] take physically infeasibletrajectories and compute the closest possible feasible trajectory byre-timing the trajectories subject to a non-linear quadrotor model.In contrast, we prevent trajectories from becoming infeasible atoptimization time. Although the method of [Roberts and Hanrahan2016] theoretically can be used to adjust timings based on a jerkminimization objective, our method can also trade-off the positionalfit of a reference path in order to achieve even smoother motion. Recently, several works address the generation of quadrotor cam-era trajectories in real-time to record dynamic scenes. [Galvane et al.2016; Joubert et al. 2016] plan camera motion in a lower dimensionalsubspace to attain real-time performance. Using a Model PredictiveController (MPC), [Naegeli et al. 2017] optimizes cinematographicconstraints, such as visibility and position on the screen, subject torobot constraints for a single quadrotor. [Nägeli et al. 2017] extendsthis work for multiple drones and allows actor-driven tracking ona geometric path. Focusing on dynamic scenes, this work does notcover the global planning aspects of aerial videography. Online Path Planning:
Approaches that address trajectory opti-mization and path following have been proposed in the controltheory literature. They allow for optimal reference following givenreal world influences. Methods like MPC [Faulwasser et al. 2009]optimize the reference path and the actuator inputs simultaneouslybased on the system state. MPC has been successfully use for the real-time generation of quadrotor trajectories [Mueller and D’Andrea2013]. Nevertheless, [Aguiar et al. 2008] show that the tracking errorfor following timed-trajectories can be larger than for following ageometric path only. Motivated by this observation Model PredictiveContouring Control (MPCC) [Lam et al. 2013] has been proposed tofollow a time-free reference, optimizing system control inputs fortime-optimal progress. MPCC approaches have been successfullyapplied in industrial contouring [Lam et al. 2013] and RC racing[Liniger et al. 2014]. Recently, [Nägeli et al. 2017] extended theMPCC-framework to allow for real-time path following in 3D spacewith quadrotors. We propose a trajectory generation method thatis conceptually related to MPCC formulations in that it optimizestimings for a quadrotor camera trajectory based on a time-optimalpath-following objective. Our formulation treats keyframes, userspecified reference timings and velocities as well as smoothnessacross the entire trajectory jointly in a soft-constrained formulationand allows users to produce aesthetically more pleasing videos.
We propose a new method to generate globally smooth quadrotorcamera trajectories. Our aim is to allow even novice users to designcomplex shots without having to explicitly reason about 5D spatio-temporal distances. Our central hypothesis is that smoothness acrossthe entire trajectory matters and hence is the main objective of ouroptimization formulation. We first introduce the model of the systemdynamics in Section 3.1 and discuss our optimization formulationin Section 3.2-3.5. See Appendix A for a table of notations.
We use the approximated quadrotor camera model of [Gebhardt et al.2016]. This discrete first-order dynamical system is incorporated asequality constraint into our optimization problem: x i + = A x i + B u i + д , u min ≤ u i ≤ u max , x i = [ r , ψ q , ψ д , ϕ д , (cid:219) r , (cid:219) ψ q , (cid:219) ψ д , (cid:219) ϕ д , (cid:220) r , (cid:220) ψ q , (cid:220) ψ д , (cid:220) ϕ д , (cid:221) r , (cid:221) ψ q , (cid:221) ψ д , (cid:221) ϕ д ] T , u i = [ F , M ψ q , M ψ д , M ϕ д ] T , (1)where x i ∈ R are the quadrotor camera states and u i ∈ R are theinputs to the system at stage i . Furthermore, r ∈ R is the position of ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. the quadrotor, ψ q is the quadrotor’s yaw angle and ψ д and ϕ д are theyaw and pitch angles of the camera gimbal. The matrix A ∈ R x propagates the state x forward, the matrix B ∈ R x defines theeffect of the input u on the state and the vector д ∈ R that ofgravity for one time-step. F is the the force acting on the quadrotor, M ψ q is the torque along its z-axis and M ψ д , M ϕ д are torques actingon pitch and yaw of the gimbal.Please note that our formulation is agnostic to the dynamicalmodel of the quadrotor. We verified this by incorporating the non-linear model of [Nägeli et al. 2017]. Qualitatively this does not impactresults, yet the computational cost increases (see Figure 5). In space-time optimization, the horizon length is defined by dividingthe timing of the last keyframe by the discretization step ∆ t . How-ever, one key idea in our formulation is that we treat trajectories, atleast initially, as time free. In particular, our method does not taketimed keyframes as input and therefore traditional approaches todetermining the horizon length are not applicable.Taking inspiration from MPC literature [Michalska and Mayne1993], we make the length of the horizon an optimization variableitself by adding the trajectory end time T into the state space of ourmodel ( x = [ x , T ] T ∈ R with δTδt = ∆ t = TN . Here N is the number of stages in thehorizon spanning the entire trajectory. The forward propagationmatrices A and B are also recalculated based on the current ∆ t . We require a time-free parameterization of the reference to opti-mize the timing of keyframes. We use a chord length parameterized,piecewise cubic polynomial spline in hermite form (PCHIP) to in-terpolate the user-defined keyframes [Fritsch and Carlson 1980].The resulting chord length parameter θ describes progress on thespatial reference path defined as f d ( θ ) = [ r d ( θ ) , ψ d ( θ ) , ϕ d ( θ )] ∈ R .To prevent sudden changes of the progress parameter, we add θ into our model and formulate its dynamics with the following lineardiscrete system equation: Θ i + = C Θ i + Dv i , ≤ v i ≤ v max , (2)where Θ i = [ θ i , (cid:219) θ i ] is the state and v i is the input of θ at step i and C ∈ R x , D ∈ R x are the discrete system matrices. Intuitively, v i approximates the quadrotor’s acceleration as θ is an approximationof the trajectory length.With this extension of the dynamic model in place, we now formu-late an objective to minimize the error between the desired quadro-tor position r d ( θ ) and the current quadrotor position r . With respectto the time optimization we want the quadrotor to follow r d ( θ i ) as closely as possible in time (no lag) but allow deviations from itscontour for smoother motion. This distinction is not possible whenminimizing the 2-norm distance to the reference point. For this rea-son, we differentiate between a lag ϵ l and a contour error ϵ c similarto MPCC approaches (e.g., [Lam et al. 2013]). We approximate thetrue error from the spline by using the 3D-space approximation oflag ˆ ϵ l and a contour error ˆ ϵ c of [Nägeli et al. 2017] (see Figure 2, Fig. 2. The position (x,y,z) and orientation (yaw, pitch) over time are afunction of path progress f d ( θ ) . Inset: to advance along the path we optimizefor smooth progress (cid:219) θ via minimization of lag ˆ ϵ l and contour ˆ ϵ c error. inset). The approximated lag error is defined as,ˆ ϵ l ( θ , r i ) = e T n (3)where e = r d ( θ ) − r i is the relative vector between desired andactual positions and n = (cid:219) r d ( θ )| |(cid:219) r d ( θ )| | is the normalized tangent vectorof r d ( θ ) at θ . The resulting contour error approximation is givenby: ˆ ϵ c ( θ , r i ) = || e − ˆ ϵ l ( θ , r i ) n || . (4)Both error terms are then inserted into the cost term, c p ( θ , r i ) = (cid:20) ˆ ϵ l ( θ ) ˆ ϵ c ( θ ) (cid:21) T Q (cid:20) ˆ ϵ l ( θ ) ˆ ϵ c ( θ ) (cid:21) , (5)where Q is a diagonal positive definite weight matrix. Minimizing c p will move the quadrotor along the user defined spatial reference.Our experiments have shown that distinguishing between lagand contour error is important for the temporal aspects of the opti-mization. Trajectories generated by minimizing || e || , depending onthe weighting of the term, either lag behind the temporal referenceor cannot trade-off positional fit for smoother motion. With appro-priate weights for lag and contour error this behavior is avoided.To give users fine grained control over the target framing wefollow user-specified viewing angles in an analogous fashion. Toattain the camera yaw and pitch we minimize the 2-norm discrep-ancy between desired and actual orientation of the quadrotor andcamera gimbal. Given by the following cost terms: c ψ ( θ , ψ q , i , ψ д , i ) = || ψ d ( θ ) − ( ψ q , i + ψ д , i )|| (6) c ϕ ( θ , ϕ д , i ) = || ϕ d ( θ ) − ϕ д , i || , (7)where ψ q , i , ψ д , i , ϕ д , i are the current yaw and pitch angles. Further-more, we preprocess every keyframe by adding a multiple of 2 π to yaw and pitch such that the absolute distance to the respectiveangle of the previous keyframe has the smallest value. For the camera to smoothly follow the path, we need to ensure that θ progresses. By specifying an initial θ and demanding θ to reachthe end of the trajectory in the terminal state θ N , the progress of ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. ptimizing for Aesthetically Pleasing Quadrotor Camera Motion • 90:5 θ can be forced with an implicit cost term. We simply penalize thetrajectory end time by minimizing the state space variable T , c end ( T ) = T . (8)Minimizing the end-time can be interpreted as optimizing trajecto-ries to be as short as possible temporally (while respecting smooth-ness and limits of the robot model). This forces θ to make progresssuch that the terminal state θ N is reached within time T .To ensure that the generated motion for the quadrotor is smooth,we introduce a cost term on the model’s jerk states, c j ((cid:221) r , (cid:221) ψ q , (cid:221) ψ д , (cid:221) ϕ д ) = || j i || , (9)where j i = [(cid:221) r , (cid:221) ψ q , (cid:221) ψ д , (cid:221) ϕ д ] T is jerk and angular jerk. We minimize jerksince it provides a commonly used metric to quantify smoothness[Hogan 1984] and is known to be a decisive factor for the aestheticperception of motion [Bronner and Shippen 2015]. This cost termagain implicitly effects θ by only allowing it to progress such thatthe quadrotor motion following the reference path f d ( θ ) is smoothaccording to (9). This is illustrated in Figure 2, left. The blue dot ( θ i )progresses on the reference path such that the generated motion ofthe quadrotor following f d ( θ i ) is smooth.To still be able to specify the temporal length of a video shot withthis formulation, we define the cost term, c len ( N , ∆ t ) = || t len − T || , (10)where we minimize the 2-norm discrepancy between the trajectoryend time T and a user-specified video length t len . In case a trajectoryis optimized for Eq. (10), the weight for Eq. (8) is set to zero. We construct our overall objective function by linearly combin-ing the cost terms from Eq. (5), (6), (7), (8), (9), (10) and a 2-normminimization of v . The final cost is: J i = w p c p ( θ i , r i ) + w ψ c ψ ( θ i , ψ q , i , ψ д , i ) + w ϕ c ϕ ( θ i , ϕ д , i ) (11) + w j c j ((cid:221) r , (cid:221) ψ q , (cid:221) ψ д , (cid:221) ϕ д ) + w end c end ( T ) + w len c len ( N , ∆ t ) + w v || v || , where the scalar weight parameters w p , w ψ , w ϕ , w j , w end , w len , w v > x , u , Θ , v N (cid:213) i = J i (12)subject to x = k (initial state) Θ = (initial progress) Θ N = L (terminal progress) x i + = A x i + B u i + д (dynamical model) Θ i + = C Θ i + Dv i (progress model) x min ≤ x i ≤ x max , (state bounds) u min ≤ u i ≤ u max , (input limits) ≤ Θ i ≤ Θ max (progress bounds) ≤ v i ≤ v max (progress input limits) , This also prevents solutions of infinitely long trajectories in time where adding stepswith u i ≈ is free wrt. to Eq. (9)). where J i is quadratic in x , u , v and linear in Θ . When flying agenerated trajectory we follow the optimized positional trajectory r with a standard LQR-controller and use velocity and accelerationsstates of x as feed-forward terms. We implemented the above optimization problem with MATLAB andsolve it with the FORCES Pro software [Domahidi and Jerez 2017]which generates fast solver code, exploiting the special structure inthe non-linear program. We set the horizon length of our problem tobe N =
60. The solver requires a continuous path parametrization.To attain a description of the reference spline across the piecewisesections of the PCHIP spline, we need to locally approximate it.Therefore, we implemented an iterative programming scheme ableto generate trajectories at interactive rates. For further details on theIP-scheme and the empirically derived weights of the optimizationproblem, we refer the interested reader to Appendix B.
To evaluate the efficacy of our method in creating smooth cameramotion even on problematic (for hard-constrained methods) inputs,we designed a challenging shot and generated two trajectories, onewith [Gebhardt et al. 2016] and the other one with our method.Figure 3 plots the resulting positions in x , y , z and the correspond-ing camera angles. Our method adjusts the timing of the originalkeyframes ( k - k ) to attain smoother motion over time. This is visi-ble when comparing the x-dimension of ours and [Gebhardt et al.2016]. The need to trade-off timing and spatial location is illustratedby the orientation plot (Figure 3, bottom). The keyframes k , k have been moved very close to each other which would cause ex-cessive yaw velocities since the quadrotor would need to perform a Fig. 3. Position (x,y,z in m ) and orientation (yaw, pitch in ◦ ) over time (in s ) for the same user-specified keyframes ( k - k ) for [Gebhardt et al. 2016](top) and our method (bottom). ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018.
Fig. 4. Qualitative comparison of video frames as well as jerk (in ms ) and angular jerk (in ◦ s ) profiles of two trajectories generated with [Gebhardt et al. 2016](top row) and our method (bottom row). Fig. 5. Left: comparing avg. squared jerk (in m s ) and angular jerk (in ◦ s ) perhorizon stage of different trajectories for our method, [Gebhardt et al. 2016]and [Joubert et al. 2015] (note that latter uses a different model). Right: ourmethod’s optimization runtime for different trajectories is plotted againsttheir temporal length (both in sec ). We differentiate between using a linearand a non-linear quadrotor model for trajectory generation. quadrotor model of Sec. 3.1 and the non-linear model of [Nägeliet al. 2017]. We measured runtime on a standard desktop machine(Intel Core i7 4GHz CPU, Forces Pro NLP-solver). The computationtime for the trajectories are shown in Figure 5. In average, it took2.41 s (SD = 2.50 s) to generate a trajectory with the linear modeland 14.79 s (SD = 15.50 s) with the non-linear model. Our technical evaluation shows that the proposed method generatessmoother trajectories. However, it has not been validated that thetrajectories generated with our method result in aesthetically morepleasing video. To this end, we conduct an online survey compar-ing videos which follow user-specified timings, generated with themethods of [Gebhardt et al. 2016; Joubert et al. 2015], with videosgenerated by our method. Therefore, we compare user-designedtrajectories from prior work [Gebhardt and Hilliges 2018; Joubertet al. 2015]. For each question we take the user-specified keyframesof the original trajectory and generated a time-optimized trajectoryof the same temporal duration (via Equation 10) using our method.We then render videos for the original and time-optimized trajectoryusing Google Earth (based on GPS-coordinates and camera angles).The two resulting videos are placed side-by-side, randomly assignedto the left or right, and participants state which video they prefer ona forced alternative choice 5-point Likert scale. The five responsesare: "shot on the left side looks much more pleasing", "shot on theleft side looks more pleasing", "both the same", "shot on the rightside looks more pleasing", and "shot on the right side looks muchmore pleasing". Each participant had to compare 14 videos.
In total, 424 participants answered the online survey. Assumingequidistant intervals, we mapped survey responses onto a scale from-2 to 2, where negative values mean that the original, timed video isaesthetically more pleasing, 0 indicates no difference and a positivevalue indicates a more aesthetically pleasing time-optimized video.
ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. ptimizing for Aesthetically Pleasing Quadrotor Camera Motion • 90:7
In order to attain interval data, our samples are build by takingthe mean of the Likert-type results of the expert and non-expertdesigned videos per participant. Visual inspection of residual plotsdid not reveal any obvious deviations from normality.Evaluating all responses of the survey, we try to attain a meanwhich compensates random participant effects. Therefore, we con-struct a linear mixed model using the participant as random in-tercept, the video as fixed intercept and introducing a fixed-effectintercept to represent the overall mean. The adjusted mean of thedata has a positive value with a high confidence (see Figure 6). Atype III ANOVA showed that there is a significant effect of ourmethod on the aesthetics of videos ( F ( , ) = . , p < . non-expert users (data from [Gebhardt andHilliges 2018]) and expert users (i.e. cinematographers, data from[Joubert et al. 2015]). Analyzing the results for significance, we per-form a one sample t-test on the averaged Likert ratings for expert -and non-expert -designed videos. The effect of both conditions andtheir confidence intervals are shown in Figure 6. While they aresignificant for both conditions ( expert : t ( ) , p < . non-expert : t ( ) , p < . non-expert designed videos and negative for expert designed videos. Fig. 6. Mean and 95% confidence interval of the effect of optimizationscheme on all , non-expert designed and expert designed videos. The perceptual study provides strong evidence that our method hasa positive effect on the aesthetic perception of aerial videos. Fur-thermore, it has shown that this effect is even stronger for videosby non-experts . This supports our hypothesis that non-experts bene-fit from generating trajectories according to global smoothness asmain criteria. Looking at expert created videos the picture is differ-ent. These videos were rated as more pleasant when generated withmethods which respect user-specified timings. This can be explainedby the fact that experts explicitly leverage shot timings to createparticular compositional effects. Optimizing for global smoothnessremoves this intention from the result. However, the significantpositive effect of our method on all responses and a larger effectsize for the positive effect of non-expert- compared to the negativeeffect of expert designed videos indicate that smooth motion is amore important factor for the aesthetic perception of aerial videosthan timing. This suggests that users, especially experts, could ben-efit from a problem formulation which allows for soft-constrainedinstead of hard-constrained timings. In this way, users could stillemploy shot timings to create compositional effects, while the opti-mization scheme generates trajectories trading-off user-specifiedtimings and global smoothness. Based on these results, we formulate three requirements forquadrotor camera generation schemes: 1) smoothness should bethe primary objective of quadrotor camera trajectory generation,2) methods should auto generate or adjust keyframe timings tobetter support non-experts, 3) while providing tools for expertsto specify soft-constrained timings. The proposed method alreadyfull-fills requirement 1) and 2). In the next section, we propose howour method can be extended such that all requirements are met.
Recognizing the need to provide both global smoothness and explicituser control over camera timings, we present two method extensionsto control camera motion: an approach based on "classic" keyframetimings and a further approach based on velocity profiles.
We augment our objective function with an additional term forsoft-constraint keyframe timings. The original formulation does notallow for the setting of timing references based on horizon stages:due to the variable horizon we lack a fixed mapping between timeand stage. To be able to map timings with the spatial reference,we use the θ -parameterization of the reference spline. Referencetimings hence need to be specified strictly monotonically increasingin θ . Based on the reference timings and the corresponding θ -valueswe interpolate a spline through these points, which results in tim-ing reference function t d ( θ ) which can be followed analogously tospatial references by minimizing the cost, c t ( θ , i , ∆ t ) = || t d ( θ ) − ( i ∗ ∆ t )|| , (13)where i is the current stage of the horizon and ∆ t is the currentdiscretization of the model. We add this cost into (11) and assign aweight to specify its importance w t . By setting the value of w t to avery large number, quasi hard-constrained keyframes are attainable. The above extension enables mimicry of timing control in priormethods. However, the actual purpose of specifying camera timingsin a video is to control or change camera velocity to achieve adesired effect (recall the fly-by example). Since determining thetiming of the shot explicitly is difficult, we propose a way for usersto directly specify camera velocities. We extend the formulation ofour method to accept reference velocities as input. Again, we usethe θ -parameterization to assign velocities to the reference spline f d .To minimize the difference between the velocity of the quadrotorand the user-specified velocity profile v d ( θ ) , we specify the cost, c v ( θ , i , ∆ t ) = || v d ( θ ) − (cid:219) r Ti n || , (14)where we project the current velocity of the quadrotor (cid:219) r i on thenormalized tangent vector of the positional reference function n .We add this cost term and a weight w v to (11). To understand whether our final method has the potential to im-prove the usability of quadrotor camera tools, whether soft-constrainedtiming methods produce videos of similar perceived aesthetics thenhard-constrained timing methods and whether experts can benefit
ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018.
Fig. 7. Progress curve (a) and velocity profile (b). from our method, we conduct an additional user study. In this study,we compare different variants of our method with the method of[Gebhardt et al. 2016] as representative for quadrotor camera opti-mization schemes which use hard-constrained keyframe timings.
User Interface:
In our experiment we used the tool of [Gebhardtand Hilliges 2018] and extended the UI with a toolbar. This toolbarcontains a slider to specify w p (see Equation 11) and dependingon the condition, a progress curve or a velocity profile. A progresscurve allows for the editing of the relative progress on a trajectoryover time (see Figure 7, a). A velocity profile enables editing of thecamera speed over the progress on the trajectory (see Figure 7, b). Experimental conditions:
We investigate four different conditions:1) In timed , participants work with the optimization method of [Geb-hardt et al. 2016] and a progress curve (see Figure 7, a). 2)
Soft-timed uses our optimizer and the progress curve. Participants can decidewhether they want to specify keyframe timings (see Equation 13)or use the auto-generated timings. 3) In auto participants workwith our optimization and the keyframe timings it provides. Theycan choose to fix the end time of a trajectory (see Equation 10).4)
Velocity uses our method and a velocity profile (see Figure 7,b). Participants can decide whether they want to specify cameravelocity (see Equation 13) or use the auto-generated speed.
Tasks:
The study comprises two tasks: 1) Participants were askedto design a free-form video of a building in a virtual environment(T1). We asked participants to keep the spatial trajectory as similar aspossible across conditions whereas the dynamics of camera motionwere allowed to differ. They performed the task in the conditions timed , soft-timed and auto . 2) Participants were asked to faithfullyreproduce an aerial video shot with varying camera velocity (T2).Participants should try to reproduce camera path and dynamics ofthe reference video. This task was performed with the conditions timed , soft-timed and velocity to investigate the level of controlover timing afforded by the different conditions. We use a within-subjects design and counterbalance order of conditions within atask to compensate for learning effects. Procedure:
Participants were introduced to the system and thefour conditions and were given time to practice using the tool in atutorial taks. Participants then solved T1 and T2 in the respectiveconditions. Tasks were completed when participants reported tobe satisfied with the designed shot (T1) or the similarity to the reference (T2). For each task and condition participants completedthe NASA-TLX and a questionnaire on levels of satisfaction with theresult. Finally, a short exit interview was conducted. A session tookon average approximately 70 min (introduction ≈ ≈ ≈
22 min, T2 ≈
23 min).
Participants:
We recruited 12 participants (5 female, 7 male). Wepurposely included 3 experts: an avid hobby quadrotor videogra-pher, a professional videographer, experimenting with quadrotorvideography in his free-time, and a professional quadrotor videog-rapher. The remaining participants reported no experience in aerialor normal photo- or videography.
We analyze the effect of the conditions on the usability of the tooland the aesthetics of the resulting videos. For significance testing,we ran a one-way ANOVA if the normality assumption holds and aKruskal-Wallis test when it is violated. Analyzing the data of expertsand non-experts separately, we found no significant differences inresults and thus will not differentiate between them in this section.
Usability.
To asses the effect of our method on the usability ofthe tool, we asked participants to fill out NASA-TLX and collectedinteraction logs (e.g. task execution time). In T1, auto has the low-est median in terms of task load, followed by soft-timed and timed (see Figure 8). This ranking remains the same for all interactionlogging measures of T1 (see task execution time (TET), time updatesand number of generations). Although there is no significant ef-fect of conditions in T1 on task load ( F ( , ) = . , p = . H ( ) = . , p < .
03; time updates: H ( ) = . , p < . H ( ) = . , p < . auto is signifi-cantly different to timed (TET: p < .
01; time updates: p < . auto and soft-timed significantly dif-fer to timed ( auto - timed : p < . soft-timed - timed : p < . Auto automatically generatestimings and thereby camera velocities. This explains the condi-tion’s first rank in terms of task load and interaction logs as itsimplifies the task drastically. For T2, velocity and soft-timed yielda lower task load compared to timed , indicating a slight advantageof our method in terms of usability (differences are not significant: H ( ) = . , p = . soft-timed and velocity perform similar and are followedby timed . The number of generations between conditions differsstatistically significantly ( H ( ) = . , p < . velocity and soft-timed significantly differfrom timed ( velocity - timed : p < . soft-timed - timed : p < . H ( ) = . , p = . F ( , ) = . , p = . Aesthetics.
We are also interested in participants’ perceived dif-ference in aesthetics of the generated videos. We asked participantsin both tasks to assess the visual quality of the video they designedon a scale ranging from 1 (not at all pleasing) to 7 (very pleasing, seeFigure 8). Although differences are not significant (aesthetics in T1: H ( ) = . , p = . H ( ) = . , p = . ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. ptimizing for Aesthetically Pleasing Quadrotor Camera Motion • 90:9
Fig. 8. Boxplots for the results of the user study (T1 on upper row, T2 on lower row). The investigated conditions are auto (a.), velocity (v.), soft-timed (s.-t.), and timed (t.). Task execution time is depicted in minutes. Aesthetics, smoothness and task load are shown in the respective scales of the questionnaire items. Timeupdates and number of generations are counts. In case the data of a measure is normally distributed the mean is displayed (red box). the variants of our method are perceived to produce aestheticallymore pleasing videos in both tasks. For T1, we also asked users torate the smoothness of videos on a scale from 1 (non-smooth) to 7(very smooth). Figure 8 summarizes the results which do not differsignificantly between conditions ( H ( ) = . , p = . Despite the small sample size of our experiment, the results indicatea positive effect of our method on both, the perceived aestheticsof results and the usability of the tool.
Auto caused the lowest taskload among conditions and participants where satisfied with thegenerated results. Although soft-timed and timed allow to specifythe timing of a shot in the same manor (using the progress-curve orthe timeline), soft-timed performed better than timed in terms of taskload (T2) and aesthetics (T1 and T2). We think that this preferencecan be explained by two factors. First, participants in soft-timed generally used a workflow in which they initially take generatedtimings and then adjust keyframe times to create a desired visual.This workflow was implemented by experts but also by non-experts(if they used keyframe timings at all). Second, in soft-timed keyframetimings are specified as soft-constraints, allowing the optimizer totrade-off the temporal fit for a smoother trajectory. This makes soft-timed more forgiving than timed wrt to the space-time-ratioin-between keyframes, reducing adjustments participants had to doin order to solve a task in this condition (see time/velocity updatesand no. of generations in Figure 8). In addition, soft-constrainedtimings allow the optimizer to still generate feasible trajectorieseven if the underlying user input would not yield a feasible resultThe preference for soft-constrained keyframe timings is also anindication for our general assumption that timing control is notused to precisely specify the time a camera should be at a certainposition. Instead users employ timing to specify the velocity of thecamera along the path. This is also suggested by looking at theresults of the velocity condition. In T2, it performed similar to soft-timed and better than timed for task load and aesthetics, indicatingthat specifying camera dynamics via a velocity profile is an intuitivealternative for providing keyframe timings.
In this paper, we addressed the dichotomy of smoothness and tim-ing control in current quadrotor camera tools. According to design requirements in literature [Joubert et al. 2015] their optimizers in-corporate keyframes timing as hard constraints, providing precisetiming control. A recent study [Gebhardt and Hilliges 2018] hasshown that this causes users to struggle when specifying smoothcamera motion over an entire trajectory. The current optimizationformulations needs to have matching distances between the 5 di-mensions of a keyframe (position, yaw and pitch of camera angle)with its temporal position. This poses a particular hard interactionproblem for users, especially novices. In this paper, we proposea method which generates smooth quadrotor camera trajectoriesby taking keyframes only specified in space and optimizing theirtimings. We formulated this non-linear problem as a variable hori-zon trajectory optimization scheme which is capable of temporallyoptimizing positional references.In a large-scale online survey we compared videos generated withour method to videos generated with [Gebhardt et al. 2016] and[Joubert et al. 2015]. The results indicate a general preference forvideos generated according to a global smoothness objective, butalso highlight that videos of experts are aesthetically more pleasingwhen provided timing control. Based on these insights, we extendour method such that users can specify keyframe timings as soft-constraints but still globally smooth trajectories are attained. Inaddition, we allow users to specify camera reference velocities setas soft-constraints in the optimization.We test the efficacy and usability of our optimization in a com-parative user study (baseline is [Gebhardt et al. 2016]). The resultsindicate that our method positively effects the usability of quadro-tor camera tools and improves the visual quality of video shots forexperts and non-experts. Both benefit from using an optimized tim-ing initially and having the possibility of fine-tuning it accordingto their intention. In addition, the user study revealed that timingcontrol does not need to be precise but is rather used to controlcamera velocity in order to create a desired compositional effect.
ACKNOWLEDGMENTS
We thank Yi Hao Ng for his work in the exploratory phase of theproject, Chat Wacharamanotham for helping with the statisticalanalysis of the perceptual study and Velko Vechev for providing thevideo voice-over. We are also grateful for the valuable feedback ofTobias Nägeli on problem formulation and implementation. Thiswork was funded in parts by the Swiss National Science Foundation( UFO 200021L_153644).
ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018.
REFERENCES
3D Robotics. 2015. 3DR Solo. (2015). Retrieved September 13, 2016 from http://3drobotics.com/soloA. Pedro Aguiar, Joao P. Hespanha, and Petar V. Kokotovic. 2008. Performance limita-tions in reference tracking and path following for nonlinear systems.
Automatica
Journal of neu-roengineering and rehabilitation
12, 1 (2015), 112. https://doi.org/10.1186/10.1186/s12984-015-0090-9John T. Betts. 2009.
Practical Methods for Optimal Control and Estimation Using NonlinearProgramming (2nd ed.). Cambridge University Press, New York, NY, USA.Shaw Bronner and James Shippen. 2015. Biomechanical metrics of aesthetic perceptionin dance.
Experimental Brain Research
TheInternational Journal of Robotics Research
34, 7 (2015), 969–1002. https://doi.org/10.1177/0278364914558129Marc Christie, Patrick Olivier, and Jean-Marie Normand. 2008. Camera Control inComputer Graphics.
Computer Graphics Forum
In Proceedings of Graphics Interface ’94 . 190–199.T. Faulwasser, B. Kern, and R. Findeisen. 2009. Model predictive path-following forconstrained nonlinear systems. In
Proceedings of the 48h IEEE Conference on Decisionand Control (CDC) . 8642–8647. https://doi.org/10.1109/CDC.2009.5399744F. N. Fritsch and R. E. Carlson. 1980. Monotone Piecewise Cubic Interpolation.
SIAM J. Numer. Anal.
17, 2 (1980), 238–246. https://doi.org/10.1137/0717021arXiv:https://doi.org/10.1137/0717021Q. Galvane, J. Fleureau, F. L. Tariolle, and P. Guillotel. 2016. Automated Cinematographywith Unmanned Aerial Vehicles. In
Proceedings of the Eurographics Workshop onIntelligent Cinematography and Editing (WICED ’16) . Eurographics Association,Goslar Germany, Germany, 23–30. https://doi.org/10.2312/wiced.20161097Christoph Gebhardt, Benjamin Hepp, Tobias Nägeli, Stefan Stevšić, and Otmar Hilliges.2016. Airways: Optimization-Based Planning of Quadrotor Trajectories Accordingto High-Level User Goals. In
Proceedings of the 2016 CHI Conference on HumanFactors in Computing Systems (CHI ’16) . ACM, New York, NY, USA, 2508–2519.https://doi.org/10.1145/2858036.2858353Christoph Gebhardt and Otmar Hilliges. 2018. WYFIWYG: Investigating Effective UserSupport in Aerial Videography. (2018). arXiv:1801.05972T. Geijtenbeek and N. Pronost. 2012. Interactive Character Animation Using SimulatedPhysics: A State-of-the-Art Review.
Computer Graphics Forum
31, 8 (2012), 2492–2515. https://doi.org/10.1111/j.1467-8659.2012.03189.xJohn Hennessy. 2015. 13 Powerful Tips to Improve Your Aerial Cinematog-raphy. (2015). Retrieved August 29, 2017 from https://skytango.com/13-powerful-tips-to-improve-your-aerial-cinematography/Neville Hogan. 1984. Adaptive control of mechanical impedance by coactivation ofantagonist muscles.
IEEE Trans. Automat. Control
29, 8 (1984), 681–690.Niels Joubert, Dan B Goldman, Floraine Berthouzoz, Mike Roberts, James A Landay,Pat Hanrahan, et al. 2016. Towards a Drone Cinematographer: Guiding QuadrotorCameras using Visual Composition Principles. (2016). arXiv:1610.01691Niels Joubert, Mike Roberts, Anh Truong, Floraine Berthouzoz, and Pat Hanrahan. 2015.An Interactive Tool for Designing Quadrotor Camera Shots.
ACM Trans. Graph.
Internat. J. Control
86, 8 (2013), 1410–1424. https://doi.org/10.1080/00207179.2013.770170Tsai-Yen Li and Chung-Chiang Cheng. 2008.
Real-Time Camera Planning for Navigationin Virtual Environments . Springer Berlin Heidelberg, Berlin, Heidelberg, 118–129.https://doi.org/10.1007/978-3-540-85412-8_11Alexander Liniger, Alexander Domahidi, and Manfred Morari. 2014. Optimization-based autonomous racing of 1:43 scale RC cars.
Optimal Control Applications andMethods (2014). https://doi.org/10.1002/oca.2123 Christophe Lino and Marc Christie. 2012. Efficient Composition for Virtual CameraControl. In
Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Com-puter Animation (SCA ’12) . Eurographics Association, Aire-la-Ville, Switzerland,Switzerland, 65–70. https://doi.org/10.1145/1409060.1409068Christophe Lino and Marc Christie. 2015. Intuitive and Efficient Camera Controlwith the Toric Space.
ACM Trans. Graph.
34, 4, Article 82 (July 2015), 12 pages.https://doi.org/10.1145/2766965Christophe Lino, Marc Christie, Roberto Ranon, and William Bares. 2011. The Director’sLens: An Intelligent Assistant for Virtual Cinematography. In
Proceedings of the 19thACM International Conference on Multimedia (MM ’11) . ACM, New York, NY, USA,323–332. https://doi.org/10.1145/2072298.2072341C. Karen Liu, Aaron Hertzmann, and Zoran Popović. 2006. Composition of Com-plex Optimal Multi-character Motions. In
Proceedings of the 2006 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation (SCA ’06) . EurographicsAssociation, Aire-la-Ville, Switzerland, Switzerland, 215–222. http://dl.acm.org/citation.cfm?id=1218064.1218093James McCann, Nancy S Pollard, and Siddhartha Srinivasa. 2006. Physics-based motionretiming. In
Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium onComputer animation . Eurographics Association, 205–214.D. Mellinger and V. Kumar. 2011. Minimum snap trajectory generation and controlfor quadrotors. In .2520–2525. https://doi.org/10.1109/ICRA.2011.5980409D Mellinger, N Michael, and V Kumar. 2012. Trajectory Generation and Control forPrecise Aggressive Maneuvers with Quadrotors.
The International Journal of RoboticsResearch
31, 5 (2012), 664–674. https://doi.org/10.1177/0278364911434236H. Michalska and D. Q. Mayne. 1993. Robust receding horizon control of constrainednonlinear systems.
IEEE Trans. Automat. Control
38, 11 (Nov 1993), 1623–1633.https://doi.org/10.1109/9.262032M.W. Mueller and R. D’Andrea. 2013. A model predictive controller for quadrocopterstate interception. In
Proceedings of the European Control Conference (ECC), 2013 .1383–1389. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6669415T. Naegeli, J. Alonso-Mora, A. Domahidi, D. Rus, and O. Hilliges. 2017. Real-timeMotion Planning for Aerial Videography with Dynamic Obstacle Avoidance andViewpoint Optimization.
IEEE Robotics and Automation Letters
PP, 99 (2017), 1–1.https://doi.org/10.1109/LRA.2017.2665693Tobias Nägeli, Lukas Meier, Alexander Domahidi, Javier Alonso-Mora, and OtmarHilliges. 2017. Real-time Planning for Automated Multi-view Drone Cinematogra-phy.
ACM Trans. Graph.
36, 4, Article 132 (July 2017), 10 pages. https://doi.org/10.1145/3072959.3073712Mike Roberts and Pat Hanrahan. 2016. Generating Dynamically Feasible Trajectoriesfor Quadrotor Cameras.
ACM Trans. Graph.
35, 4, Article 61 (July 2016), 11 pages.https://doi.org/10.1145/2897824.2925980VC Technology. 2016. Litchi Tool. (2016). Retrieved September 13, 2016 from https://flylitchi.com/Andrew Witkin and Michael Kass. 1988. Spacetime Constraints. In
Proceedings of the15th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’88) . ACM, New York, NY, USA, 159–168. https://doi.org/10.1145/54852.378507I-Cheng Yeh, Chao-Hung Lin, Hung-Jen Chien, and Tong-Yee Lee. 2011. Efficientcamera path planning algorithm for human motion overview.
Computer Animationand Virtual Worlds
22, 2-3 (2011), 239–250. https://doi.org/10.1002/cav.398
A NOTATION
For completeness and reproducibility of our method we provide asummary of the notation used in the paper in Table 1.
B IMPLEMENTATION DETAILS
In this section, we provide details on the weights we use in the objec-tive function, the iterative programming scheme we implementedto attain a continuous path parametrization and its performance.
B.1 Optimization Weights
The values for the weights of the objective function we used in theuser study and the online survey are listed in Table 2.
B.2 Iterative Programming Scheme
The solver requires a continuous path parametrization. To attain adescription of the reference spline even in-between the piecewisesections of the PCHIP-spline, we need to locally approximate it.
ACM Transactions on Graphics, Vol. 37, No. 4, Article 90. Publication date: August 2018. ptimizing for Aesthetically Pleasing Quadrotor Camera Motion • 90:11
Symbol Description r , (cid:219) r , (cid:220) r , (cid:221) r Quadrotor position, velocity, acceleration and jerk ψ q , (cid:219) ψ q , (cid:220) ψ q , (cid:221) ψ q Quad. yaw and angular velocity/acceleration/jerk ψ д , (cid:219) ψ д , (cid:220) ψ д , (cid:221) ψ д Gimbal yaw and angular velocity/acceleration/jerk ϕ д , (cid:219) ϕ д , (cid:220) ϕ д , (cid:221) ϕ д Gimbal pitch and angular velocity/acceleration/jerk x , u Quadrotor states and inputs A , B System matrices of quadrotor д Gravity T Trajectory end time N Horizon length θ Progress parameter f d ( θ ) Reference spline ( R ) r d ( θ ) Positional reference ( R ) ψ d ( θ ) Pitch reference ϕ d ( θ ) Yaw reference t d ( θ ) Time reference v d ( θ ) Velocity reference Θ , v Progress state and input C , D System matrices of progressˆ ϵ l , ˆ ϵ c Approximate lag and contour error
Table 1. Summary of notation used in the body of the paper
ValueWeight (layed on) Online survey User study w p (position) 1 [ . , ] (user-specified) Q (lag, contour err.) diaд ( , ) diaд ( , ) w ψ (yaw) 1 [ . , ] (user-specified) w ϕ (pitch) 1 [ . , ] (user-specified) w j (jerk) 10 100, 10 (if w t , w v > w T (end-time) 0 1, 0 (if w t , w end > w end (length in t.) 1 1 w t (timing) 0 100 w v (velocity) 0 100 Table 2. Values for weights used in Equation 11.
Therefore, we implement an iterative programming scheme wherewe compute a quadratic approximation of the reference splinearound the θ i -value of each stage in the horizon. This process isdescribed in Figure 9. In the first iteration of the scheme we ini-tialize all θ i to zero and fit the entire reference trajectory (blackspline) with a single quadratic approximation (blue spline). By solv-ing the optimization problem of Equation 12, the progression of θ -values will be decided based on the quadratic approximation (yel-low dots). For the next iterations, we always take the value of θ i from the last run of the solver, project it on the reference spline(green dots) and fit a local quadratic approximation (red splines).Based on these fits the progress of θ -values again is optimized. Westop the optimization when the difference of the θ -values for allstages in-between iterations is smaller than a pre-defined threshold, θ last − θ current < δ . Fig. 9. Illustration of the iterative programming scheme, showing the refer-ence path (black) as well as the quadratic approximations of the first (blue)and second (red) iteration with their respective θ -values (yellow, green).-values (yellow, green).