FAKIR: An algorithm for revealing the anatomy and pose of statues from raw point sets
FFAKIR : An algorithm for estimating the pose and elementary anatomyof archaeological statues
TONG FU,
LIRIS, CNRS, Université de Lyon
RAPHAËLLE CHAINE,
LIRIS, CNRS, Université de Lyon
JULIE DIGNE,
LIRIS, CNRS, Université de Lyon
Input CompletionPart adaptationSkeletonregistration Pose change
Fig. 1. Application of our method to pose change and shape completion on the Prince Paris statue. Through skeleton regression on statues, we identify theirpose and anatomy. If the input statue has missing data, we propose to adapt parts from other statues and use them to complete the input. Once the statue iscomplete, we can also change its pose: here we bring it to the pose of
The Thinker by Auguste Rodin.
The digitization of archaeological artefacts has become an essential part ofcultural heritage research be it for purposes of preservation or restoration.Statues, in particular, have been at the center of many projects. In thispaper, we introduce a way to improve the understanding of acquired statuesby registering a simple and pliable anatomical model to the raw point setdata. Our method performs a Forward And bacKward Iterative Registration(FAKIR) which proceeds joint by joint, needing only a few iterations toconverge. Furthermore, we introduce a simple detail-preserving skinningapproach working directly on the point cloud, without needing a mesh. Bycombining FAKIR with our skinning method we are able to detect the poseand the elementary anatomy of a sculpture and modify it, paving the way forpose-independent style comparison and statue restoration by combinationof parts belonging to statues with different poses.CCS Concepts: •
Computing methodologies → Shape modeling ; Point-based models .Additional Key Words and Phrases: Skeleton registration, point set analysis,anatomy detection, shape editing
With the progress of 3d scanning techniques, it is now commonto create digital replicas of artworks, which will remain foreverintact, while the real-world counterparts will slowly decay due totime damage or human activity. However when the digitizationis performed, the statues are often already degraded. Restorationsbeing costly, invasive and sometimes even risky, museums are often
Authors’ addresses: Tong Fu, [email protected], CNRS, Université de Lyon;Raphaëlle Chaine, [email protected], CNRS, Université de Lyon; JulieDigne, [email protected], CNRS, Université de Lyon. reluctant to carry out such processes. Hopefully, digitization offersanother huge advantage: the possibility to build and test restorationhypotheses, whether or not they are applied to the real model af-terwards. In the statue’s case, one of the first steps towards virtualrestoration is to be able to identify anatomical parts, in order toguide the restoration. While this task can be performed manually,it is often long and tedious. In this paper, we are thus interestedin recovering the anatomy of a statue, using a simple anatomicalmodel, while keeping user interventions to a minimum. We focuson human statues with no or few garments, since those will benefitdirectly from a registered anatomical model. Many Roman or Gallo-Roman statues fall within this scope. Furthermore, we consider thatthe digitized statues are provided as point sets.Our objective is to identify the elementary anatomy of a statue al-lowing in turn to change the statue’s pose. To do so, we first proposea method for calibrating and registering a simple anatomical modelto a point set. This step is achieved directly on the point cloud, avoid-ing thus the tedious meshing step and preserving the accuracy ofthe initial sampling. To perform the calibration and registration, weintroduce the Forward And bacKward Iterative Registration (FAKIR)algorithm, inspired by recent inverse kinematics approaches. FAKIRpermits to efficiently register the anatomical model in only a fewiterations. Once the model is registered, the point set surface islocally represented as a residual heightfield above the registeredanatomical model. It is then possible to change the pose of the statueand combine parts of different statues after they are brought to a a r X i v : . [ c s . G R ] J u l • Tong Fu, Raphaëlle Chaine, and Julie Digne common pose. To do this, we propose a skinning approach that isapplied directly to the point cloud, without using a mesh.To summarize, our contributions are the following: • A simple anatomical model efficiently representing a statuepose. • An efficient calibration and registration process based oninverse kinematics. • A point set skinning process permitting to modify the pointset following an anatomy and/or pose change.
Anatomical Model.
Designing anatomical models for human shapeshas raised a lot of interest. The most common representation con-sists in a more or less detailed graph of bones such as the ones usedin the MakeHuman framework [5]. While some methods go beyondthe human skeleton representation and model every single muscleto increase realism [21], we will focus here on skeletons, whichare simple and efficient enough for our purpose. Among skeleton-based models, the sphere-mesh model [30], a variant of convolutionsurfaces [6], has been introduced for representing mesh models bypacking sphere into it and encoding its structure. Conceptually, thesphere-mesh model can be seen as a piecewise linear simplificationof the computational geometry skeleton [29]. Although the originalsphere-mesh construction proposed to discover entirely the shapestructure from an input mesh, the sphere-mesh model can be usedto represent an anatomical model by imposing constraints on it. Assuch, it has been used successfully for representing hand skeletons[28, 31]. This model is light and pliable and we will also rely on it.
Skeleton rigging and skinning.
Once a skeleton model is chosen,the next problem for animation purposes is to position it inside aninput mesh and to attach surface points to it, processes that are called rigging and skinning respectively. Skeleton rigging can be performedmanually, but a few methods have investigated automatic processes.In particular, the Pinocchio algorithm [4] adapts a skeleton to astatic mesh by defining an objective function and maximizing it. Itworks by packing spheres into the mesh and by considering theircenters, gathered in a graph, as the admissible joint positions. Thispre-computation makes the skeleton pose estimation tractable. If theinput data is dynamic, it is possible to infer, or track, a skeleton fromit. Most tracking approaches focus on the direct independent andsimultaneous capture of the positions of the joints, using a temporalsequence and prior constraints. The pose parameters (angles) andintrinsic parameters ( e.g. bone lengths) are then inferred from it.Many of such tracking methods work with depth streams or videosstarting from a previously calibrated skeleton but the calibrationitself can be performed from a depth video and a set of knownadmissible poses [28, 31]. Such methods require a dynamic scene andcannot apply to the static mesh rigging problem. It is also possibleto rely on a whole database of people scans to learn the pose anddeformation of human bodies [2, 14, 15, 23]. Finally, some methods[13, 35] aim at finding a person’s pose despite its sometimes looseclothing, but this is outside the scope of our paper. For the sake ofcompleteness it is also important to mention that many methodsperform human tracking without requiring a model using multipleview acquisitions with or without markers. These methods are only remotely linked to our problem and we refer the reader to [7] foran excellent overview.Once the skeleton is correctly positioned, skinning methods aimat attaching the surface model to it by using weights that define theinfluence of the bones on the position of surface points. Then, whenchanging the pose of the skeleton, the attached surface should de-form accordingly. Setting the right weights is an important question:while the profile of the weights is generally sketched by graphicdesigners [25], there exist automatic weighting techniques that, forexample, use heat diffusion [4]. As far as skinning techniques areconcerned, Linear Blend Skinning [24] is one of the most popularones. A mesh surface point is transformed by a linearly weightedcombination of the motion of the bones it is attached to. Severalmethods attempt to fix the well-known collapsing problem of LinearBlend Skinning, such as Pose Space Deformation [22], Multi-WeightEnveloping [34], spherical Skinning [19] and Dual-QuaternionsSkinning [18]. A recent skinning method [20] corrects artefacts oflinear blend skinning by locally estimating the rigid transformationthat best restores the relative position of a vertex with respect toits neighbors using Laplacian differential coordinates. This method,designed for meshes, involves a definition of details in terms ofLaplacian differences. In our approach, we rather define the detailas the residual over our anatomical model. Taking a different per-spective on the problem, Implicit skinning [32] uses an implicitformulation of the surface that better supports pose changes andreprojects skinned vertices on the implicit model after each posechange. In this paper, we also use a proxy model but it is explicit.
Pose change.
When a model is rigged and skinned, it is possibleto change its pose manually by interacting with some joints of theskeleton. Through the skinning weights, the mesh surface shoulddeform accordingly. However, it is often tedious to design everysingle motion of each joint for each frame of an animation. As aconsequence, research has focused on inferring the motion fromsome key joints and frames with given skeleton positions. In thisinverse kinematics context, the Fabrik [3] and CCD [33] algorithmsdefine kinematic chains and aim at transforming each chain fromits input pose to its target pose by updating pose parameters oneafter the other alternatively forward and backward along the chain.Our registration method will also use kinematic chains in a forwardand backward approach, but the similarity ends here, since ourgoal is to estimate not only the pose but also length and widthof the model limbs using data-attachment constraints in a staticframework. Quite differently, some approaches transfer the volumedelimited by a mesh to the interior of another mesh by minimizingsome harmonic energy [1]. To do this, a deformation field is set upbetween the two meshes, using the correspondence with the nearestpoint in the current iteration [12].
Shape Synthesis.
Shape synthesis is a very active research area. Itis often done by reusing parts of existing models. [11] proposed anexample-based modeling system, where new models are generatedby stitching together statue parts from a database constructed bysegmenting a set of input models. For completing a 3D model, ageneral idea is to retrieve suitable models from the database andwarp the retrieved models to conform with the incomplete model [9,27]. More recently, a probabilistic representation for the components
AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 3 of a shape has been developed to suggest relevant componentsduring an interactive assembly-based modeling session [8, 17]. Wewill propose an application to statue synthesis by part combination,but the choice of the parts is done manually while the part adaptationis automatic.
Our work focuses on artworks representing human beings withoutor with only a few garments. A challenge of artistic data comparedto human scans lies in the difference in aesthetic perception. Indeedmany sculptors favored the perceived beauty of their work over therealism of human proportions [16, 26]. Figure 2 shows examplesof such unrealistic statues of the Roman and Gallo-Roman eras.In this context, it is necessary to devise a human model with fewconstraints, allowing to fit a sculpture which does not follow thehuman proportion beauty canons. The existing fully detailed humantemplates modeling every single limb in a very realistic way are tooconstrained for our purpose. In particular, in our model we avoidmodeling muscles. After we have registered our simplified humanmodel to the statue point cloud, a further step could be to add newflexible muscle models, but this goes beyond the scope of this paperand is unnecessary for statues where the sculptor uses muscles asstylistic elements, in the same way that he could use arabesques,yielding a possibly unrealistic result (Figure 2b). (a) A Gallo-Roman statue of a Gallicwarrior (Avignon, France, picture: F.Philibert-Caillat ) (b) Statue of Heracles and Cacus(1530-1534) by Baccio Bandinelli,(Florence, Italy, picture: Cyberuly)Fig. 2. Examples of statues with unrealistic anatomies.
We introduce an anatomical model inspired by the sphere-meshmodel [30], already successfully used for hand tracking [28, 31],using only one-dimensional elements. In this model, each bone isrepresented by a sphere-mesh B corresponding to the envelope ofthe union of a set of spheres centered on a segment and with alinearly varying radius (Figure 3b). Each bone is defined by two endsphere centers c and c with associated radii r and r respectively.The segment [ c c ] is the medial axis of the bone. For each point c ∈ [ c c ] , the radius of the sphere centered at c is r ( c ) = ( − τ ) r + τr ,with τ = ∥ c c ∥∥ c c ∥ .The sphere-mesh model is controlled by the length l = ∥ c c ∥ and the pair of sphere radii r = { r , r } . Consequently, we denotethe sphere-mesh model for one bone as B ( l , r ) . We also denote by α the angle of the conic part of the bone, as illustrated on Figure 3a.Importantly enough, the bones we are defining do not correspondto anatomical bones, but more to limbs ( i.e. it includes a coarsedescription of the flesh volume around the anatomical bone). Byanalogy to inverse kinematics, we keep the word bone instead of limb . c c c r(c)r r l (a) Cross-section of a bone (b) Sphere-mesh of a 3D bone.Fig. 3. The sphere-mesh of a bone is the union of the spheres centered onsegment [ c c ] , with radius varying linearly between the two extremities. With this type of bone element, we construct a simple humanbody template with a very coarse respect of human proportions asan initial body shape (Figure 4). Our human body template contains22 bones { B k } k = .. . Three of those correspond to the pelvis andhave no relative motion: their length is fixed up to a common scaleparameter that will be determined during the registration, alongwith the orientation of the triplet. Additionally, a special bone isused to connect the spine bone to the neck, and its length andorientation directly depend on the adjacent spine bone. The otherbones have no constraint on their relative proportions. The bonesare organized into 5 chains, depicted in different colors in figure 4a:the spine chain, the right arm chain, the left arm chain, the rightleg chain and the left leg chain. These chains are independent withthe only constraint that some extremities must remain anchoredto the spine. The chain organization is used to define the notion ofpredecessor and successor for one bone in a chain, and this orderingwill be extensively used in our kinematic registration. In particular,we will reverse the ordering of the chain to process it forward andbackward several times along the process. Each bone is thus fullydefined by its intrinsic parameters (length and two radii) and byits extrinsic parameter (rotation with respect to its predecessor).Furthermore, two successive bones share a common radius. Becauseof the simplicity of the sphere-mesh bone model, the distance from apoint to the model can be easily computed. In contrast, using a meshmodel would make these computations much more demanding. To capture the anatomy and the pose of a statue, we need a distancefunction to measure how the sphere-mesh model fits a point set P ,even if the points are far from the limbs they should be attached to.We assume that the coordinates of the points are provided with acoarse approximation of the normal so that we can locally distin-guish the inside from the outside of the sampled surface. If normals • Tong Fu, Raphaëlle Chaine, and Julie Digne (a) Control skeleton (b) Sphere-mesh modelFig. 4. Our anatomical human model: the bones are organized into 5 chainsshown in different colors. additional bones are drawn in black: the pelviswhich is a constrained triplet of bones, and the connection bone betweenthe spine and the neck. are unavailable, in most cases, they can be roughly estimated andorientated using viewing directions from the 3D sensor for example. Distance from a point to a one-bone model.
We start by defining thenormal-constrained projection of a sampled point p on a single bone B by using the normal vector n p to disambiguate the choice betweenseveral orthogonal projection possibilities. The sphere-mesh modelcalibration and registration strives to reduce the distance betweenthe sampled points and their corresponding points on the sphere-mesh and using the normal orientation helps this process. Given apoint p in the ambient space with oriented normal n p , we considerthe lines passing through p and orthogonally intersecting the sphere-mesh surface (possibly crossing its interior) at some points. Weabusively call these intersection points orthogonal projections (seeappendix A for the detailed cases). From all these possibilities, weselect the projection ˜ p whose normal n ˜ p has positive scalar productwith n p . Considering this normal-constrained orthogonal projectionallows to better handle the case where the bone is not initializedclose to the point set. Since each point p has a normal-constrainedprojection on several bones, we refer to its normal-constrainedprojection on bone B k as ˜ p k . If no subscript is provided, ˜ p refersto the normal-constrained projection of p on the associated closestbone. Distance from a point set to a sphere-mesh model.
Given a point set P and a sphere-mesh model of K bones, we first need to approximatethe subset of corresponding points for each bone. In the following,we define the point set P k as the subset of points p ∈ P which areclosest to bone B k using the distance d k = ∥ p − ˜ p k ∥ , k = · · · K .However, we try to favor the assignation to the conical part of a bone.Therefore, if p projects on the spherical part of the closest bone andif it also projects on the conical part of another bone with a similardistance, we assign it to this second bone. Once the assignment iscomputed, the one-bone distance function E k is defined as the sum of squared distances from points of P k to bone B k : E k ( P k , B k ( l k , r k ) , θ k ) = (cid:213) p ∈ P k ∥ p − ˜ p k ∥ (1)Importantly enough, the subset P k and the one-bone energy E k depend on the position of the initial extremity of the chain of bonesinvolving B k , as well as the parameters of the previous bones in thechain.In the following, the sum of one-bone distance functions is con-sidered as an energy that we aim to minimize in order to capturethe anatomy and the pose of the sphere-mesh that best correspondto our point set: E = K (cid:213) k = (cid:213) p ∈ P k ∥ p − ˜ p k ∥ . (2)In the next sections, we will also be interested in the distancerestricted to two adjacent bones B k and B k + , which we call two-bones distance: E k , k + = (cid:213) p ∈ P k ∥ p − ˜ p k ∥ + (cid:213) p ∈ P k + ∥ p − ˜ p k + ∥ . (3) To register our anatomical model to the acquired static point cloud,we propose a kinematic approach taking into account the articulatedproperty of the skeleton. Contrarily to many tracking methods thatexploit the redundancy of information between several frames orseveral views [28, 31], our method requires only one joint of theskeleton to be close to its optimal position, the rest of the skeletonpose being arbitrary, except for a mild assumption that in practicemeans using oversized initial lengths for the bones. It is one moreassumption than the Pinocchio algorithm [4], which also works inthe static case and starts with no prerequisite and we will compareour algorithm to it. Inspired by the FABRIK [3] and CCD [33] algo-rithms, our registration algorithm successively loops forward andbackward through the chains of bones so as to rotate and rescalethem to match the data, refining the parameters while temporarilyfixing the extremities of some bones. Hence our algorithm is namedForward And bacKward Iterative Registration (FAKIR). An origi-nality of our method is that bones are not only considered one byone but also by consecutive pairs, which allows for a more robustestimation of the pose and skeleton parameters along a chain.
Registration process for a chain of bones.
If the bone B k is close tothe points to which it should be associated, the estimation of the one-bone energy function becomes meaningful and the minimization ofthis energy can be used to coarsely optimize the position and radii ofthat bone with respect to the data. Therefore, our approach amountsto iteratively updating the rotation and intrinsic parameters of B k until we catch a coarse estimation of P k and B k , one extremity of B k being kept fixed. Our algorithm gradually rotates the current bone B k with respect to its predecessor, updating P k after each rotation, sothat P k gradually contains more relevant points. Once the positionof B k has been approximately found, the algorithm turns to thecoarse estimation of the position of B k + . All these computationsare driven by the minimization of the one-bone energy. However,the one-bone energy alone might be inefficient to approximate the AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 5 full length of a bone accurately and we use this energy only toobtain a first approximation of the registration for one bone. Inan intertwined manner, a finer local registration procedure of ouralgorithm is performed each time two consecutive bones B k and B k + have been processed. Its goal is to optimize the common jointposition and radius while fixing the two other joint extremities.This optimization is performed by minimizing a two-bones energyfunction, defined above. Once a chain of K bones has been positionedand scaled over its entire length, we repeat the process forward andbackward in the chain in order to further refine the joints positionsand radii between pairs of consecutive bones, only using two-bonesenergies. Finally, the position of the last extremity of a chain isalso optimized between each forward and backward step of theloop by optimizing the one-bone energy function. Each parameteroptimization is thus made by minimizing an energy related to eitherone or two bones. The full process is summarized in Algorithm 1and illustrated with a chain of three bones in Figure 5.Note that during the process, two consecutive bones should notbe enclosed in a same limb of the model point set, otherwise a jointmight remain enclosed inside this limb, trapping the optimization ina local minimum. This can be avoided by introducing a condition onthe initial bone lengths ensuring that the sum of the initial lengthsof two consecutive bones is below the optimal length of each of thelimbs they are close to. A practical trick is to slightly oversize thebones before starting the registration process. Optimization of a single bone.
As stated above, the optimizationof the one-bone energy is only used for estimating the parametersof the extremities of a chain, or for the very first forward pass inAlgorithm 1). After this step, the optimization of the position ofa joint using a bone pair should be preferred because it is moreprecise.In the single-bone case, the aim is to estimate the 3D rotationof the bone, its length and the radius of its free extremity by mini-mizing the one-bone energy function E k ( P k , B k ( l k , r k ) , θ k ) , where θ k are the angles of rotation with respect to the predecessor bone.This optimization is performed using the Levenberg-Marquardtalgorithm for each parameter. In particular, the rotation can be de-composed into two rotations around two axes that are orthogonalto c c , indeed the rotation around c c is not considered since itleaves the bone unchanged. In the case of the minimization withrespect to the vector of rotation angles θ k , we iteratively try to re-place the current angles θ k with an update θ k + δ θ k . At a minimum, ∇ δ θ k E k ( P k , B k ( l r , r k ) , θ k + δ θ k ) = , and the value for δ θ k follows.The details for the damped least-squares estimation are provided inAppendix B. Optimization for the joint between two consecutive bones.
The op-timization of the position and radius of the joint between two con-secutive bones ( B k , B k + ) is performed by optimizing a set of fourparameters in a loop (an angle, two lengths and a radius) minimiz-ing the two-bones energy function. The two end-sphere centersbeing fixed ( c k and c k + in Figure 6), we first compute the optimalrotation of the two bones around axis c k c k + , all other parametersbeing fixed. We then optimize the bone lengths ˆ l k = l k + δl k andˆ l k + = l k + + δl k + and, finally, the radius of the common joint is Algorithm 1
Forward and backward iterative registration
Input:
A point set P and a sphere-mesh chain of K bones with onechain extremity close to its optimal position Output:
The registered sphere-mesh chain. Initialization: Fix the center of the first extremity of the chain. Rotate the firstbone and adjust its radii and length by minimizing the one-boneenergy function; for k : = to K − do Consider the pair of bones B k , B k + : Fix the position of the joint common to B k and B k + ; Alternate between the optimization of B k + ’s rotation w.r.t B k , optimization of B k + ’s intrinsic parameters and updateof P k + ; Fix the positions of the 2 joints that B k and B k + do not share,and free their common joint; Compute the position and the radius of the common joint byusing the two-bones energy function. end for Compute the length of the last bone and the radius of the lastsphere.
Forward and Backward registration loop: repeat
Reverse the order of the bones in the chain; for k : = to K − do Consider the pair of bones B k , B k + : Fix the positions of the 2 joints that B k and B k + do notshare; Compute the position and the radius of the common jointby using the two-bones energy function. end for
Compute the length of the last bone and the radius of the lastsphere with the one-bone energy function. until convergencecomputed as ˆ r k + = r k + + δr . The parameters optimization alter-nates with a recomputation of point sets P k and P k + , which refinesthe point-to-bone assignment. Similarly to the single-bone case,the optimization is also performed using the Levenberg-Marquardtalgorithm whose details are provided in Appendix C. Full Skeleton Registration.
Our anatomical model is composed of5 chains, one of which is of particular importance: the spine chainwhich connects all other chains (the arms, and the legs throughthe pelvis block). We assume that the pelvis part of our modelis initialized near the corresponding part of the point set, whichrequires a very limited user interaction - basically only one point andclick. Each chain is then registered in turn using FAKIR yielding aregistered skeleton both in terms of intrinsic parameters and pose inonly a couple of iterations. The position of the pelvis is then revisedduring the registration of the spine chain. The registration order isthe following: first the spine chain is registered, followed by each ofthe two leg chains and each of the two arm chains. When registeringthe arms and legs chains, the position for the joint attached to thespine or the pelvis remains fixed. • Tong Fu, Raphaëlle Chaine, and Julie Digne c c c c B B B (a) c c c c B B B (b) c c c c B B B (c) c c c c B B B (d) c c c c B B B (e) c c c c B B B (f) c c c B B B c (g)Fig. 5. Overview of the forward and backward iterative registration for a 3-bone chain. From an initial position (a), the chain extremity c is fixed and the firstbone B is rotated and scaled to roughly calibrate its dimensions and pose through the optimization of a one-bone energy function (b); the bone B is fixedand the parameters of the second bone B are roughly calibrated in turn (c); joint c which is common to the first two bones is scaled and its position isoptimized, by using a two-bones energy function, the other joints being fixed (d); The position and length of the third bone B are then coarsely calibratedthrough one-bone optimization and the process continues by alternating single bone optimization and two-bones optimization, until the last bone of the chain(e). After this first coarse calibration forward pass finishes, a backward pass using only two-bone optimizations is performed (f) permitting to refine the poseand skeleton parameters and solve for the chain extremity position. With few forward and backward pass involving two-bone optimization only, the model isregistered (g).(a) Rotate B k and B k + (b) Refine length l k of B k (c) Refine length l k + of B k + (d) Refine radius r k + Fig. 6. Pairwise Optimization. With fixed extremities c k and c k + , the pair ofbones B k and B k + is first rotated around axis c k c k + in order to minimizethe two-bones energy. Then the lengths of the bones B k and B k + and theircommon radius r k + are optimized successively. After these updates, thepoint-to-bone assignment is recomputed. As the process is repeated thedistances are more accurate since the point-to-bone assignment becomesmore meaningful. Once the anatomical model is registered to the statue point set, itis possible to exploit the assignment between the model and thepoint set to change the pose and the elementary anatomy of astatue by modifying the extrinsic and intrinsic parameters of thebones respectively. To do so we attach the point set to its registeredanatomical model so that deforming the model will deform the pointset adequately. This process, well known in the animation researchcommunity, is called
Skinning . It allows to deform the skin followingan underlying skeletal animation, a widespread method to animate3D models. In most skinning methods, the skin is a 3D mesh and theskeleton is a tree whose nodes represent joints of the skeleton andedges represent bones. An originality of our approach is to avoidusing a mesh with fixed connectivity whose triangles quality maybe altered by deformations related to pose and anatomy changes, possibly creating triangle slivers and self-intersections. We ratherpropose to develop a point set skinning process.In our case, since after applying FAKIR, the anatomical modelis registered to the point set, it can serve as the skeleton of usualanimation methods. We introduce a skinning method working onpoint sets and taking a better account of the twisting and bendingrotations, alleviating known artefacts of Linear Blend Skinning andDual Quaternions. We further propose to consider the skin of themodel as a set of details, encoded as a heightfield on top of ourelementary anatomical and transfered back on the model after poseor intrinsic parameters changes.
Local heightfield over the sphere-mesh model.
From now on, let usassume that the sphere-mesh model is registered to the point set. Theheightfield value h of a point p is defined as a signed distance from p to its projection on the sphere-mesh base point ˜ p : p = ˜ p + h ( p ) n ˜ p ,where n ˜ p is the normal to the sphere-mesh surface at ˜ p . Importantlyenough, ˜ p is the regular orthogonal projection on the bone p isassigned to, in opposition to the normal-constrained projectiondefined in section 3.2 and appendix A. Hence the surface pointset is decomposed into the set of base points on the sphere-meshand the residual orthogonal heightfield. The pose and anatomychanges modify, through skinning, the position of the base pointon the sphere-mesh model. Then the heightfield is added back tothe modified projection yielding the final point set, as described inalgorithm 2.Under pose change, a point p remains affected to the same bonebut its evolution depends on both the evolution of its base point˜ p , the normal at ˜ p and the possible scaling of its the heightfieldvalue h ( p ) . The evolution of the base point ˜ p is driven by the bone p is assigned to and its adjacent bones in the chain. The motioninduced by our skinning model is continuous. Indeed, the positionof a base point ˜ p continuously depends on a set of one to threeconsecutive bones. Although the point-to-bone assignation is fixedduring deformation, the base point can slide along the bone andmove from the conic part to the sphere part of the bone and con-versely. Since the conic part is tangent to the sphere, both the basepoint and its normal evolve continuously even in case of cone tosphere or sphere to cone base point motion. Since the heightfieldof a point is either preserved or continuously scaled, the wholemotion is continuous throughout the deformation. Naturally, some AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 7 points may become hidden because of self-intersections betweenbones caused by the rotation. In this work, we choose to keep theself-intersections, since the resulting outer envelop of the model isvisually satisfying. Compared to other heightfield decompositionmethods, the heightfield is carried by the points themselves andnot by the base surface. Hence it can encode folded surface sheetsover the sphere-mesh model (see the Dancer with Crotales exampleon Figure 21). In the following we describe our proposed point setskinning method to be applied to the base point sets before liftingthem with the heightfields (Algorithm 2). Since the skinning can beapplied independently of the heightfield decomposition, we describethe method in a generic point set context for notation simplicity.Note however that heightfield skinning should be preferred to di-rect point set skinning, since it provides a much better result asillustrated on Figure 7.
Fig. 7. Heightfield skinning. From left to right: original point set; posechange with direct point set skinning; pose change with heightfield skinning.
Algorithm 2
Height-field skinning
Input:
Sphere-mesh model registered to the input point set P .Target pose and anatomy. Output:
Point set P ′ corresponding to the target changes. Compute base point set ˜ P corresponding to P on the registeredsphere-mesh and compute the set of height values h ( P ) Apply point set skinning on the base points ˜ P Deform ˜ P to the target pose and anatomy, yielding ˜ P sk Re-project points of ˜ P sk on the sphere-mesh yielding ˜ P ′ Lift points ˜ P ′ to the skin surface using the - possibly scaled -initial height of each point, yielding P ′ . The general idea of skinning is to attach each point to one or morebones with weights measuring the influence of each bone on it.The position of the point after a deformation is a weighted linearcombination of the positions relative to its influencing bones. Lin-ear Blend Skinning [24], one of the most popular skinning method,causes some well-known collapsing problems at joints, in particu-lar for large rotations. For example, the volume at the joint is notpreserved when it is bent around the axis orthogonal to the twoadjacent bones (Figure 8). Similarly, if we apply a twist around the axis of a bone while keeping its predecessor fixed, Linear Blend Skin-ning produces a folding of the joint around a singular point (Figure12a). These flaws are avoided by Dual Quaternions Skinning [18]which interprets a combination of rigid transformations as a rigidtransformation, however artefacts still appear in the concavities(Figures 11 and 14).We propose to deal with the pose change in a different way thatbreaks down the movement into its natural components. We con-sider that the motion between two bones at a joint is the combinationof a twisting rotation around the axis of one of the bones and a bend-ing rotation around an axis perpendicular to both bones’ axes. Wealso introduce motion-dependent skinning weights: the weight of apoint is not the same for twisting and bending rotations. The impactof bending on the base point set should be limited to an area looselyenclosing the rotating sphere joint, while the impact of twistingobviously extends to the length of the bones adjacent to the twistedarticulation. This is more coherent with the fact that underlyingmuscles are arranged along the bones and attached to the joints. Inour approach, a point p ∈ P k can only be influenced by the bonesadjacent to B k so that it has at least one and at most three weights. pp p (a) p (b)Fig. 8. Linear Blending artefacts: joint collapse (a); (b) trace of the evolutionof a point through iterated re-skinning with direct linear blend skinningafter splitting the rotation into smaller ones. Through Linear Blend Skinning,a point p located opposite to the bending is moved inside the joint alonga line p p where p and p are the initial positions of p with respect tobones B and B respectively. Bending rotation with anisotropic weights.
We decompose thebending rotation into a sequence of small bending rotations andupdate both the position through Linear Blend Skinning and thecorresponding weights of the points after each rotation (Figure 8b).At each step, the bone is slightly rotated by θ r around the jointbending axis and the points are moved following adapted LinearBlend Skinning weights. The weights of a point p during a bendingrotation follow a Gaussian profile of the distances from p to each ofthe bones that influence it. It is driven by a parameter ε r controllingthe size of the influence area. In our experiments, we set ε r largerthan the average distance between the point set and our model. Theweight ω j ( p ) , relative to one of p ’s influencing bones B j writes: ω j ( p ) = c exp − ∥ p ˜ p j ∥ ( ε r / ) (4)where c is a normalizing factor ensuring that the weights sum to 1,and ˜ p j is the regular orthogonal projection of p onto B j , Hence if p • Tong Fu, Raphaëlle Chaine, and Julie Digne is on bone B j (as is the case for base points), ˜ p j = p . Figure 9 showsthe weights profile along two bones. (a) (b)Fig. 9. Skinning weights for the bending rotation around a joint. (a) thered curve represents the influence weights of the left bone B k and the bluecurve represents the influence of the right bone B k + . The influence area ofeach bone is controlled by ε r . (b): values of anisotropic ε r on a bone. Pointssuch that ε r = are shown in white. From yellow to red, the value of ε r increases linearly. This discretization yields similar results as Dual Quaternion skin-ning. However, it is possible to further improve the method, bynoticing that if ε r is large, the convex part behaves well, it is pop-ulated by points sliding from the conic parts of both bones to thejoint sphere, while an artefact is created in the concave part. Onthe contrary, if ε r is close to 0, which corresponds to no skinning atall, a hole appears in the convex part, but a self-intersection is cre-ated in the concave part (Figure 10). Quite counter-intuitively, thisself-intersection is much more visually satisfying than the thinningartefact which can be observed otherwise. Indeed, only the outer en-velop is visible and, when allowing self-intersection, this envelop issimilar to the one obtained if a contact area was computed betweenthe two bones [32]. To get the most of the two possibilities, wepropose to adapt ε r so that it is close to 0 for points in the concavepart and it is larger for points on the convex parts. More precisely, ε r varies continuously with respect to an anisotropy angle . close-up (a) (b) (c) (d)Fig. 10. Influence of ε r on the skinning resulting from a bending rotation.In figure 10a, ε r = . The value ε r increases from (b) to (d). We define the anisotropy angle α b at p as the angle between theplane defined by B k and B k + axes and the plane defined by B k ’saxis and p . This angle serves to transition continuously over theskinned surface between points with no skinning, favoring localself-intersections ( α b ≤ π / α b ≥ π / ε r is deduced from the anisotropy angle as ε r = cos α b ε ′ r , with ε ′ r controlling the influence area size. Therefore, the weight ω j ( p ) associated to point p and one of its influencing bone B j is: ω j ( p ) = c exp − ∥ p ˜ p j ∥ ( cos α b ε ′ r ) if cos α b <
01 if cos α b ≥ p ∈ P j α b ≥ p (cid:60) P j (5) (a) Linear blend skin-ning (b) Dual Quaternionsskinning (c) Our anisotropic skin-ningFig. 11. Comparison of our anisotropic skinning method with Linear BlendSkinning and Dual Quaternions for a bending rotation. For a fair comparison,the two first methods use the Gaussian weight of Equation 4 which is madeanisotropic in our case. Dual Quaternions fix the volume collapse of LinarBlend Skinning near the convexity, but artefacts remain in the concave part,while our skinning method does not suffer from any of these flaws. Twisting rotation.
Let us consider a bone B k + twisted aroundits axis by an angle β , with its predecessor B k kept fixed. Such arotation impacts points attached to bones B k and B k + . To handlethis twist, we drop the skinning framework by linear combinationsof bone motions and replace it with rotations adapted to the points.More precisely, each point is rotated around B k ’s (resp. B k + ) axis byan angle that depends on its distance to B k (resp. B k + ). For p ∈ P k rotating around the axis of B k , this angle writes β ( p ) = ω ( p ) β with ω ( p ) = ∥ v k + − p ⋆ ∥∥ v k + − c k + ∥ + ∥ c k + − w k + ∥ (6) p ⋆ is the projection of p on B k ’s axis, and v k + (resp. w k + ) is apoint on B k ’s axis (resp. B k + ) delimiting the impacted areas (Figure13). By default v k + = c k and w k + = c k + , but different impactedareas can be designed by choosing different v k + and w k + . Here,the expression for ω ( p ) corresponds to a linear evolution of therotation angle along the bone, but other types of influences couldbe designed. On Figures 12 and 13, we compare our method withLinear Blend Skinning and Dual Quaternions for a twisting rotation,the trace of the points initially aligned is much smoother with ourapproach. (a) Linear blend skinning (b) Dual-quaternions skinningFig. 12. Linear blend skinning and dual-quaternions skinning to be com-pared with our approach in figure 13 that uses twist specific weights. Greendots show points that were aligned before the twisting rotation. AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 9
Fig. 13. The blue curve represents the value of ω β . Points v k + and w k + define the range of the twisting effect. p ⋆ is the projection of p on theskeleton line. Combination of twist and bend.
Twist and bend specific skinningare combined to get the skinning result for any rotation around ajoint. We perform first the twisting rotation and then the bendingrotation. The positions and the weights of the points relative to eachbone are updated between these two specific rotations. Figure 14shows a comparison of our approach with Linear Blend Skinning andDual Quaternions on synthetic data with a two-bones sphere-meshmodel. As was expected, Linear Blend Skinning yields the well-known collapse at the joint. To alleviate this effect, bending withDual Quaternions and our bending approach without anisotropyget a similar result, with a region of influence restricted to theneighborhood of the joint. However, the twist is handled differentlyin our approach and its effect is distributed over the length of thebone, yielding a more natural result.
Fig. 14. Comparison of skinning approaches on a sphere-mesh model withcombined twist and bend rotation movements. Left to right: Linear BlendSkinning, skinning with Dual Quaternions, Ours
In this section, we show the performance of FAKIR both on syn-thetic data and on point sets resulting from statue digitization. Wedeveloped our algorithm in C++, using OpenMP for distance updateparallelization. All experiments are run on an Intel Core i7-4790KCPU @ 4.00GHz.
We first test our algorithm on synthetic data to provide a quantitativeevaluation of the FAKIR performances. We consider a point set of5 k points sampled on a sphere-mesh of a 4-bone chain in a specificpose. We start from a generic 4-bone chain that we try to register tothe point set. Although the point set and the initial chain are quitedistant from each other, a user-given initial approximate positionof a single anchor point (one of the extremity) is enough to register accurately the chain. The accuracy of the registration is evaluatedas the average distance between the point set and the model: dist = N points (cid:213) p ∈ P ∥ p − ˜ p ∥ . (7)For 5 k points, without any additional noise, our algorithm takes5 . s to converge to dist = K = Fig. 15. Evolution of the registration distance with the iterations for differentnumber of points in the point set (left image) and different levels of noise(right image).
We selected some interesting statues from various sources.(1) Dancer with Crotales: Louvre Museum, Paris, France.(2) Dancing Faun: Pompei excavations, Italy.(3) Aphrodite, Museum of Thorvaldsens, Copenhagen, Denmark.(4) Old man walking: Nye Carlsberg Glyptotek, Copenhagen,Denmark
Fig. 16. Evaluation of FAKIR with respect to increasing Gaussian noise after iterations. The first row shows the initial point set and the bottom rowshows the registered bone chain. From left to right: without noise, σ = . , σ = , and σ = . The total groundtruth model length is 140.Fig. 17. Evaluation of FAKIR with respect to a bad initial anchor pointposition after iterations. The first row shows the initial point set and thebottom row shows the registered bone chain. The last column shows thatdue to a bad initialization, the points (plotted in red) that are affected tothe first bone do not bring enough information for the one-bone energyfunction to move the chain extremity. Then, not enough bones remain toapproximate the whole point set.Fig. 18. Evaluation of the FAKIR algorithm with respect to missing dataafter iterations. The first row shows the initial point set and the bottomrow shows the registered bone chain. (5) Esquiline Venus: found in 1874 at the Horti Lamiani in Rome.Capitoline Museums, Rome, Italy(6) Old Fisherman Vatican-Louvre (or Dying seneca): found inRome. Louvre Museum, Paris, France(7) Venus de Milo: Louvre Museum, Paris, France(8) Prince Paris: Ny Carlsberg Glyptotek, Copenhagen, Denmark.While the ’Dancer with crotales’ is a raw point set. The other 7models are point sets sampled on meshes extracted from the Sketch-fab website. Figure 21 shows obtained registrations on four stat-ues. It also compares resulting models after skeleton pose changeand skinning with our method or skinning with Dual Quaternions.Our skinning method clearly improves the quality of the modifiedmodel near bone joints. The registration algorithm performs wellfor statues depicting naked characters: in this case, the registration is not hindered by additional clothing or accessories, and the simplesphere-mesh model fits well the data. Even with moderate clothing(Dancer with Crotales) FAKIR recovers the pose of the statue, andthe skinning process yields a plausible result.As far as the complexity of FAKIR is concerned, the computa-tional bottleneck lies in the assignment and re-assignement of eachpoint several times during the optimization process. This reassign-ment takes place during the updates of the one-bone and two-bonesenergy updates. The number of these updates is related to the geom-etry of the surface and not to the number of points sampled on thatsurface. Therefore the overall complexity is linear with respect tothe number of points. From an experimental point of view, FAKIR isa reasonably light algorithm: for a point cloud of 10000 points andthe 22-bone model, the first forward pass of FAKIR takes 5 s and theaverage execution time of one pass of the FAKIR process takes lessthan 2 s .We compare FAKIR with Pinocchio [4] in Figure 19. The FAKIRalgorithm yields a better skeleton registration, in particular forthe shoulders and neck bones. As far as computation times areconcerned, the Pinocchio method takes about 35s for a mesh with138048 vertices, which is roughly the same time as the 10 iterationsof the FAKIR process optimizing not only for the joint positionsbut also for the bone radii (38s). Furthermore, a single iteration ofFAKIR takes 9s and already provides a better result with a much moreplausible shoulders location. However it is important to note thatthe Pinocchio method does not require an initial skeleton position,while our method requires one of the joint to be not far from itsoptimal position (in this experiment we chose the pelvis joint). Fig. 19. Skeletons for the Aphrodite statue using the Pinocchio riggingmethod and comparison with our FAKIR algorithm. From left to right: Pinoc-chio with the Pinocchio-provided initial skeleton (17 bones); Pinocchio withour initial skeleton (22 bones); FAKIR with our initial skeleton after a singleforward iteration; FAKIR with our initial skeleton in 10 iterations. Only theskeleton is displayed since the bone radii are not taken into account byPinocchio.
AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 11
A direct application of statue pose and anatomy modification isstatue restoration, by harmoniously combining parts belongingto different statues after bringing them to a common pose andanatomy. This type of statue restoration by part combinations wasquite common in the 18th and 19th centuries . Performing thisrestoration virtually has two advantages: first, several hypothesescan be tested, second the pose and anatomy of the statues canbe made similar before the combination, avoiding thus adding anoversized arm to an undersized body. Examples of statues withmissing parts are shown in Figures 1 and 22. To complete a statue,we first register our anatomical model to the point set using FAKIR.In case there is no clue for the size a limb, such as for the missingarm, forearm and hand of the Esquiline Venus (Figure 22, first row),we use default human proportions. Then we choose a statue which iscomplementary to the incomplete statue and change its elementaryanatomy and pose to match the ones of the incomplete statue. Thelast step is to integrate the selected parts into the first statue. Weassume that the selected parts and the statue to complete slightlyoverlap, which it necessary to blend harmoniously statue parts.The two point sets being brought to a common pose and anatomy,the combination area is defined as the area with two layers of points,one from each statue. Here we handle the case where there is only asingle layer in each fragment above the sphere-mesh in the combi-nation area. To prevent layers superposition artefacts, it is necessaryto merge the information in these areas. In a nutshell, the mergingconsists in removing the lower layer in the overlap area, creatinga sharp boundary between the point sets, and then blending thepoints near the boundary. Recall that our sphere-mesh model isused as a basis surface to express the residual heightfield informa-tion h after the registration. We propose to use it to combine thedata points. Let Q p be the subset of points on the two models thatproject on ˜ p , the projection of p on the sphere-mesh model, up toprecision δ . The first step is to keep only the upper layer in theoverlap area. To do so, we consider the subset H p of the points in Q p whose heightfield value is larger than h max − δ , where h max isthe maximum heightfield value for points in Q p . Then we replacethe heightfield value of p by a Gaussian-weighted average of theheightfield values in H p . The resulting heightfield value of p is: h ( p ) = S (cid:213) q ∈ H p e − ∥ ˜ p − ˜ q ∥ δ h ( q ) (8)with S a weight normalizing factor. This brings the points of thelower layer in the overlap area to the upper layer, creating a sharpboundary. Finally, the sharp boundary is smoothed by gaussian-weighted averaging of the heightfield values across the boundary.In figure 1, we show the process of completing a statue whichlacks arms and legs. Figure 22 shows the restoration of three otherstatues. The Esquiline Venus (first row) and the Venus de Milo (thirdrow) are both missing arms while The Old Fisherman is missinglegs (second row). The restoration method recovers plausible armsand legs in all three cases, leading to plausible restored statues. https://art.thewalters.org/detail/22879/torso-of-artemis-with-head-of-aphrodite/ Fig. 20. In the left figure, blue points and red points come from differentstatues. The right figure shows the result after merging the two parts bytaking the maximum followed by boundary smoothing.
Our FAKIR algorithm works well to estimate the anatomical posi-tion of statues with or without clothes as long as the anatomicalinformation remains visible. For example, in the Dancer with Cro-tales case (first row of Figure 21), the dimension and the pose ofthe legs is easy to infer although the legs are partially hidden bythe dress. However if the clothes hide a large part of the statue,such as in the Venus de Milo case (third row in figure 22), FAKIRfails at recovering the anatomy and pose of the legs, as they arecovered by the drape. For our point set model, a change in posemay reveal areas where there is no data. This occurs wherever twoparts are stuck together in the initial pose. For example in Figure21 (fourth row), there seems to be a veil between the right arm andthe body (see also Figure 22, second row). Finally, combining partsof different statues for restoration can sometimes look unnaturalbecause the materials and styles of the combined parts are different,style transfer would be necessary to alleviate this effect but this is awhole different research topic.
We introduced a sphere-mesh anatomical model and a combinedcalibration and registration algorithm to estimate the anatomy andthe pose of digitized archaeological statues. We also proposed apoint set skinning method to modify the point set when the poseof a statue is changed. We compared our registration and skinningapproaches with existing approaches to highlight their benefits. Wealso illustrated that this framework can be used to combine statueparts or add missing elements. While our method already gives goodresults, a further improvement would be to handle the case of aclothed statue which would involve modifying the FAKIR algorithmsince anatomy parts may be hidden. Sometimes, the point cloudfrom the scan of a statue is incomplete, or some areas, occludedin the original statue, are revealed by the pose change, creatingthus a hole in the point set. In that case an inpainting process isnecessary and will be the topic for our future work. Finally, ourcurrent approach to combine parts remains very rudimentary andis similar to a union despite a slight smoothing. As a future work,we would like to design a more respectful approach to the geometryand style of different fragments.
Acknowledgments.
This work was funded by the e-Roma projectfrom the Agence Nationale de la Recherche (ANR-16-CE38-0009).The
Dancer with crotales model is a point set of the Farman Dataset [10]. The other 7 statues data are sampled on meshes from theSketchfab website: the
Dancing Faun model is courtesy of MosheCaine, the
Venus de Milo model is courtesy of Sketchfab user "tux",and the other 5 statues models (
Aphrodite , Old Man Walking , Es-quiline Venus , Old Fisherman and
Prince Paris ) are courtesy of Geof-frey Marchal.
REFERENCES [1] Dicko Ali-Hamadi, Tiantian Liu, Benjamin Gilles, Ladislav Kavan, François Faure,Olivier Palombi, and Marie-Paule Cani. 2013. Anatomy Transfer.
ACM Trans.Graph.
32, 6, Article 188 (Nov. 2013), 8 pages. https://doi.org/10.1145/2508363.2508415[2] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, JimRodgers, and James Davis. 2005. SCAPE: shape completion and animation ofpeople.
ACM Trans. Graph
24 (2005), 408–416.[3] Andreas Aristidou and Joan Lasenby. 2011. FABRIK: A fast, iterative solverfor the Inverse Kinematics problem.
Graph. Models
73, 5 (Sept. 2011), 243–260.https://doi.org/10.1016/j.gmod.2011.05.003[4] Ilya Baran and Jovan Popović. 2007. Automatic Rigging and Animation of 3DCharacters.
ACM Trans. Graph.
Proceedingsof the 18th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’91) . ACM, New York, NY, USA, 251–256. https://doi.org/10.1145/122718.122757[7] Federica Bogo, Michael J. Black, Matthew Loper, and Javier Romero. 2015. DetailedFull-Body Reconstructions of Moving People from Monocular RGB-D Sequences.In
International Conference on Computer Vision (ICCV) . 2300–2308.[8] Siddhartha Chaudhuri, Evangelos Kalogerakis, Leonidas Guibas, and VladlenKoltun. 2011. Probabilistic Reasoning for Assembly-based 3D Modeling.
ACMTrans. Graph.
30, 4, Article 35 (July 2011), 10 pages. https://doi.org/10.1145/2010324.1964930[9] Siddhartha Chaudhuri and Vladlen Koltun. 2010. Data-driven Suggestions forCreativity Support in 3D Modeling.
ACM Trans. Graph.
29, 6, Article 183 (Dec.2010), 10 pages. https://doi.org/10.1145/1882261.1866205[10] Julie Digne, Nicolas Audfray, Claire Lartigue, Charyar Mehdi-Souzani, and Jean-Michel Morel. 2011. Farman Institute 3D Point Sets - High Precision 3D Data Sets.
Image Processing On Line
ACM Trans. Graph.
23, 3 (Aug. 2004), 652–663. https://doi.org/10.1145/1015706.1015775[12] Benjamin Gilles, Lionel Reveret, and Dinesh Pai. 2010. Creating and animatingsubject-specific anatomical models.
Computer Graphics Forum
29, 8 (Dec. 2010),2340–2351. https://doi.org/10.1111/j.1467-8659.2010.01718.x[13] Nils Hasler, Carsten Stoll, Bodo Rosenhahn, Thorsten Thormählen, and Hans-Peter Seidel. 2009. Estimating body shape of dressed humans.
Computers &Graphics
33, 3 (2009), 211 – 216. https://doi.org/10.1016/j.cag.2009.03.026 IEEEInternational Conference on Shape Modelling and Applications 2009.[14] Nils Hasler, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and Hans-Peter Seidel.2009. A Statistical Model of Human Pose and Body Shape.
Computer GraphicsForum (Proc. EUROGRAPHICS), Blackwell, 337-346 (2009)
28 (04 2009). https://doi.org/10.1111/j.1467-8659.2009.01373.x[15] David A. Hirshberg, Matthew Loper, Eric Rachlin, and Michael J. Black. 2012.Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape.In
Computer Vision – ECCV 2012 , Andrew Fitzgibbon, Svetlana Lazebnik, PietroPerona, Yoichi Sato, and Cordelia Schmid (Eds.). Springer Berlin Heidelberg, Berlin,Heidelberg, 242–255.[16] Alain Hus. 1955. Sculpture étrusque archaïque : le cavalier marin de la Villa Giulia.
Mélanges de l’école française de Rome
67, 1 (1955), 69–126. https://doi.org/10.3406/mefr.1955.7396[17] Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, and Vladlen Koltun.2012. A Probabilistic Model for Component-based Shape Synthesis.
ACM Trans.Graph.
31, 4, Article 55 (July 2012), 11 pages. https://doi.org/10.1145/2185520.2185551[18] Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O’Sullivan. 2007. Skinningwith Dual Quaternions. In
Proceedings of the 2007 Symposium on Interactive 3DGraphics and Games (I3D ’07) . ACM, New York, NY, USA, 39–46. https://doi.org/10.1145/1230100.1230107 [19] Ladislav Kavan and Jiří Žára. 2005. Spherical Blend Skinning: A Real-time De-formation of Articulated Models. In
Proceedings of the 2005 Symposium on In-teractive 3D Graphics and Games (I3D ’05) . ACM, New York, NY, USA, 9–16.https://doi.org/10.1145/1053427.1053429[20] Binh Huy Le and J P Lewis. 2019. Direct Delta Mush Skinning and Variants.
ACMTrans. Graph.
38, 4, Article 113 (July 2019), 13 pages. https://doi.org/10.1145/3306346.3322982[21] Dongwoon Lee, Michael Glueck, Azam Khan, Eugene Fiume, and Kenneth Jackson.2012. Modeling and Simulation of Skeletal Muscle for Computer Graphics: ASurvey.
Foundations and Trends® in Computer Graphics and Vision
Proceedings of the 27th Annual Conference on Computer Graphics and InteractiveTechniques (SIGGRAPH ’00) . ACM Press/Addison-Wesley Publishing Co., NewYork, NY, USA, 165–172. https://doi.org/10.1145/344779.344862[23] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, andMichael J. Black. 2015. SMPL: A Skinned Multi-Person Linear Model.
ACMTrans. Graphics (Proc. SIGGRAPH Asia)
34, 6 (Oct. 2015), 248:1–248:16.[24] Nadia Magnenat-Thalmann, Richard Laperrière, and Daniel Thalmann. 1988. Joint-dependent Local Deformations for Hand Animation and Object Grasping. In
Proceedings on Graphics Interface ’88 . Canadian Information Processing Society,Toronto, Ont., Canada, Canada, 26–33. http://dl.acm.org/citation.cfm?id=102313.102317[25] Alex Mohr, Luke Tokheim, and Michael Gleicher. 2003. Direct Manipulation ofInteractive Character Skins. In
Proceedings of the 2003 Symposium on Interactive3D Graphics (I3D ’03) . ACM, New York, NY, USA, 27–30. https://doi.org/10.1145/641480.641488[26] A. N. Newell. 1934. Gallo-Roman Religious Sculpture.
Greece and Rome
3, 8 (1934),74–84. https://doi.org/10.1017/S0017383500002424[27] Mark Pauly, Niloy J. Mitra, Joachim Giesen, Markus Gross, and Leonidas J. Guibas.2005. Example-based 3D Scan Completion. In
Proceedings of the Third EurographicsSymposium on Geometry Processing (SGP ’05) . Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, Article 23. http://dl.acm.org/citation.cfm?id=1281920.1281925[28] Edoardo Remelli, Anastasia Tkach, Andrea Tagliasacchi, and Mark Pauly. 2017.Low-Dimensionality Calibration Through Local Anisotropic Scaling for RobustHand Model Personalization. In
The IEEE International Conference on ComputerVision (ICCV) .[29] Andrea Tagliasacchi, Thomas Delame, Michela Spagnuolo, Nina Amenta, andAlexandru Telea. 2016. 3D Skeletons: A State-of-the-Art Report.
ComputerGraphics Forum
35 (05 2016). https://doi.org/10.1111/cgf.12865[30] Jean-Marc Thiery, Emilie Guy, and Tamy Boubekeur. 2013. Sphere-Meshes: ShapeApproximation using Spherical Quadric Error Metrics.
ACM Transaction onGraphics (Proc. SIGGRAPH Asia 2013)
32, 6 (2013), Art. No. 178.[31] Anastasia Tkach, Mark Pauly, and Andrea Tagliasacchi. 2016. Sphere-meshes forReal-time Hand Modeling and Tracking.
ACM Trans. Graph.
35, 6, Article 222(Nov. 2016), 11 pages. https://doi.org/10.1145/2980179.2980226[32] Rodolphe Vaillant, Loïc Barthe, Gaël Guennebaud, Marie-Paule Cani, DamienRohmer, Brian Wyvill, Olivier Gourmel, and Mathias Paulin. 2013. Implicit skin-ning: real-time skin deformation with contact modeling.
ACM Trans. Graph.
32, 4,Article 125 (July 2013), 12 pages. https://doi.org/10.1145/2461912.2461960[33] Li-Chun T. Wang and Chih-Cheng Chen. 1991. A combined optimization methodfor solving the inverse kinematics problems of mechanical manipulators.
IEEETrans. Robotics and Automation
Proceedings of the2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’02) .ACM, New York, NY, USA, 129–138. https://doi.org/10.1145/545261.545283[35] Chao Zhang, Sergi Pujades, Michael J. Black, and Gerard Pons-Moll. 2017. Detailed,Accurate, Human Shape Estimation From Clothed 3D Scan Sequences. In
TheIEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Appendix A ONE-BONE DISTANCE COMPUTATION
In this appendix, we detail the projection of a point p on a bone B .Instead of using the usual orthogonal projection on the bone, weconstrain the projection ˜ p to have a normal coherent with the oneof p . This constraint is helpful when the bone lies far away fromits corresponding point set: the point will then be projected on the“right side” of the bone. In the following, without loss of general-ity, let us assume r ≤ r . All the following computations dependon an angle α defined in Fig. 23 and which can be expressed as AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 13
Fig. 21. Registration and pose changes of 4 statues: the Dancer with Crotales (first row), the Dancing Faun (second row), Aphrodite (third row) and the OldMan Walking (fourth row). First column: initial point set, second column: overlay of the registered model and the point cloud, third column: registered model,fourth column: final point set in a modified pose by our skinning method, fifth column: skinning result with Dual Quaternion. As can be seen in particular inthe areas circled in red in the fifth column, our method suppresses or at least reduces the Dual Quaternion artefacts around the bone joints.
Fig. 22. Registration and restoration of 3 incomplete statues, Esquiline Venus (first row), Old Fisherman (second row) and Venus de Milo (third row). Firstcolumn: initial point set, second column: overlay of the registered model and the point set, third column: registered model alone, fourth column: finalrestoration. α = arctan | r − r | √ ∥ c c ∥ −( r − r ) . Let us first compute p ⋆ the projectionof p on line c c , and two translations of these points along thisline: p ⋆ α at the distance ∥ pp ⋆ ∥ tan α of p ⋆ and p ⋆ − α at the distance −∥ pp ⋆ ∥ tan α , as illustrated on Figure 23. Let τ α = p ⋆ α − c c − c , so that p ⋆ α can be expressed as τ α c + ( − τ α ) c . Different cases can occur: • < τ α <
1: the point projects on the cone part of the bone.Let ˜ p α be the intersection of segment [ p ⋆ α p ] with the cone. ˜ p α is the orthogonal projection of p on the bone. If the normal to ˜ p α has a positive scalar product with the normal of p , ˜ p = ˜ p α .Otherwise, normals are deemed inconsistent and ˜ p = ˜ p − α , i.e. the intersection of p − α p with the cone that is the farthestfrom p . • τ α < τ α > p is the projection of p on the spherecentered at c (resp. c ) with consistent normal direction,except if this normal-constrained projection falls within thebone and not on the envelop. AKIR : An algorithm for estimating the pose and elementary anatomy of archaeological statues • 15
In any case, the distance between p and its normal-constrainedprojection ˜ p vanishes when p is located near the surface of one bone,with a normal oriented consistently. It may happen that the returnedprojection does not provide a point belonging to the surface of thebone: on figure 23, ˜ q − α is the normal-constrained projection ofpoint q , but it is not on the surface of the bone. It corresponds to acase where the point is very far from the part of the bone which iscoherent with its normal. In this situation, we compute the distancesbetween point q and its projections on the two spheres and choosethe closest projection point for ˜ q . pc c r r q o h Fig. 23. Various projection cases. p has two possible projections ˜ p α and ˜ p − α depending on the orientation of the normal at p . Point p ⋆ is the projectionof p on line c c . If the normal at p is oriented upward ˜ p = ˜ p α . Otherwise, ˜ p = ˜ p − α . The same strategy is used to project points q and o . For completeness, let us express unsigned distance d ( p ) = ∥ p − ˜ p ∥ in the various cases since they will be required in the followingLevenberg-Marquardt optimization formulations. If τ α < τ α > d ( p ) = ∥ c p ∥ − r (resp. d ( p ) = ∥ c p ∥ − r ). If 0 ≤ τ ≤ d ( p ) = (cid:40) ∥ pp ⋆ α ∥ − r α ( p ) if n ˜ p · n p > ∥ pp ⋆ − α ∥ + r − α ( p ) if n ˜ p · n p ≤ c c : r α ( p ) = ∥ ˜ p α p ⋆ α ∥ = τ α ( p ) r + ( − τ α ( p )) r r − α ( p ) = ∥ ˜ p − α p ⋆ − α ∥ = τ − α ( p ) r + ( − τ − α ( p )) r (10)with: τ α ( p ) = c p ⋆ α · c c ∥ c c ∥ and τ − α ( p ) = c p ⋆ − α · c c ∥ c c ∥ . Furthermore ∥ pp ⋆ α ∥ = ∥ pp ⋆ − α ∥ = ∥ pp ⋆ ∥/ cos α . Hence, for each bone, we firstcompute the α angle, then, for each point p , we compute its pro-jection p ⋆ on c c and the corresponding τ α ( p ) yielding r ± α ( p ) and˜ p ± α . Appendix B OPTIMIZATION FOR ONE BONE
Let us assume that c (Fig. 23) is fixed and let us optimize for the poseand intrinsic parameters of bone B . In a local reference frame cen-tered at c with x-axis aligned with c c , c has coordinates ( , , ) and c has initial coordinates ( l , , ) . The rotation of the bone can beparameterized by a rotation of angle θ around the y-axis followedby a rotation of angle θ around the z-axis. The one-bone energy is invariant by rotation around the x-axis. After the double rota-tion, c has coordinates ( l cos θ cos θ , l sin θ , l cos θ sin θ ) . Letus call ( x , y , z ) the coordinates of point p in this local coordinatesystem and express d ( p ) with respect to parameters θ = ( θ , θ ) , l and r = ( r , r ) . We have:tan α = r − r (cid:112) l − ( r − r ) , cos α = (cid:112) l − ( r − r ) l ∥ c p ∥ = x + y + z ∥ c p ⋆ ∥ = x cos θ cos θ + y sin θ + z cos θ sin θ ∥ p ⋆ p ∥ = x + y + z − ( x cos θ cos θ + y sin θ + z cos θ sin θ ) ∥ p ⋆ p ⋆ α ∥ = ∥ p ⋆ p ∥ tan α ∥ p ⋆ α p ∥ = ∥ p ⋆ p ∥ cos ατ ± α ( p ) = ∥ c p ⋆ ∥ ± ∥ p ⋆ p ⋆ α ∥ l ∥ c p ∥ = ( x − l cos θ cos θ ) + ( y − l sin θ ) + ( z − l cos θ sin θ ) The one-bone energy function is (dropping the k subscript forsimplicity): E ( P , B ( l , r ) , θ ) = (cid:213) p ∈ P d ( p ) (11)The optimization is performed on three set of parameters in turn:angles θ , bone length l and bone radii r .the optimization for bone B with respect to θ writes:ˆ θ ≡ argmin θ E ( P , B ( l , r ) , θ ) = argmin θ (cid:213) p ∈ P d ( p , θ ) (12)Following the Levenberg-Marquardt algorithm, at each iteration,parameter θ is replaced by a new estimate θ + δ θ , computed as:argmin θ E ( P , B ( l , r ) , θ ) ≈ argmin δ θ E ( P , B ( l , r ) , θ + δ θ ) (13)which is computed by taking: ∂ E ( P , B ( l , r ) , θ + δ θ ) ∂ δ θ = We finally get δ θ : δ θ = −[ J T J + λdiaд ( J T J )] − J T g ( θ ) where J = [ J , J ] , J i = ∂ d ( p i ) ∂ θ and J i = ∂ d ( p i )) ∂ θ and g ( θ ) isa column vector whose entries are d ( p , θ ) for each point p . λ isa damping factor set to 0 .
01 initially and adapting it throughoutiterations.In the following, we assume 0 < τ α ( p ) < n ˜ p · n p >
0. Inthis case, p projects on ˜ p α and d ( p ) = ∥ pp ⋆ α ∥ − r α ( p ) with r α ( p ) = τ α r + ( − τ α ) r , and τ α = ∥ c p ⋆ α ∥ l . Hence: ∂ d ( p ) ∂ θ = α ∂ ∥ p ⋆ α p ∥ ∂ θ + ( r − r ) l ∂ ∥ c p ⋆ ∥ ∂ θ + tan α ∂ ∥ p ⋆ p ∥ ∂ θ (14) ∂ d ( p ) ∂ θ = α ∂ ∥ p ⋆ α p ∥ ∂ θ + ( r − r ) l ∂ ∥ c p ⋆ ∥ ∂ θ + tan α ∂ ∥ p ⋆ p ∥ ∂ θ (15)The full expression for the derivatives can be easily derived giventhe expressions for ∥ p ⋆ α p ∥ , ∥ c p ⋆ ∥ , ∥ p ⋆ p ∥ above. The cases τ α ( p ) < τ α ( p ) > n ˜ p · n p < Appendix C OPTIMIZATION FOR A JOINT BETWEENTWO CONSECUTIVE BONES
Let us consider the geometric optimization of the center of thejoint between two bones by optimizing the two-bones energy withrespect to the lengths l k and l k + . Each length is optimized in turn,with a side-effect on the value of the other length. The two-bonesenergy can be expressed as a function of l k : E ( k , k + ) ( l k ) = (cid:213) p ∈ P k ∥ ˜ p k − p ∥ + (cid:213) p ∈ P k + ∥ ˜ p k + − p ∥ (16)Following the Levenberg-Marquardt algorithm, at each iteration,each parameter l k is replaced by a new estimate l k + δl :argmin l k E ( k , k + ) ( l k ) ≈ argmin δl E ( k , k + ) ( l k + δl ) (17)By setting ∂ E ( k , k + ) ( l k + δl ) ∂ δl =
0, we get: δl = − (cid:205) p ∈ P k d k ∂ d k ∂ l k + (cid:205) p ∈ P k + d k + ∂ d k + ∂ l k (cid:205) p ∈ P k ( ∂ d k ∂ l k ) + (cid:205) p ∈ P k + ( ∂ d k + ∂ l k ) (18)where d k = ∥ p − ˜ p k ∥ and d k + = ∥ p − ˜ p k + ∥ are expressed asfunctions of l k .Let us detail the expression of d k with respect to l k : during thepairwise optimization c k and c k + remain fixed (Figure 6). Let c k be the origin of a local reference frame with the x-axis aligned with c k c k + . In this frame, the coordinates write c k ( , , ) , c k + ( l k , , ) and c k + ( x , y , z ) while a point P has coordinates ( x , y , z ) . Then c k + c k + = ( x − l k , y , z ) , c k + p = ( x − l k , y , z ) .Let us assume that p projects on ˜ p α (the case ˜ p − α can be deducedwith minor changes). Using the same notation as in figure 23 andappendix B, recall that d k = ∥ p − ˜ p k ∥ = ∥ pp ⋆ α ∥ − r α ( p ) . Since whenoptimizing l k the orthogonal projection on c k c k + does not change, ∥ pp ⋆ ∥ remains the same. However both α and r α ( p ) change. Since r α ( p ) = τ α ( p ) r k + ( − τ α ( p )) r k + with τ α ( p ) = ∥ c k p ⋆ α ∥ l k , we get: ∂ d k ∂ l k = − ∥ pp ⋆ ∥ cos α ∂ cos α ∂ l k − ( r k − r k + ) ∂ τ α ( p ) ∂ l k (19)Simple geometric considerations give cos α = (cid:114) − ( r k + − r k ) l k , τ α ( p ) = ∥ c k p ⋆ ∥ + ∥ pp ⋆ ∥ tan αl k and tan α = r k + − r k (cid:113) l k −( r k + − r k ) , whose dif-ferentiation with respect to l k is easy.One must also express distances d k + as functions of l k . In thatcase, the projection on bone B k + is slightly different, since the po-sition of point c k + changes with l k . The formulas are only slightlymodified by it, but this time ∥ pp ⋆ ∥ also depends on l k . We get: ∂ l k + ∂ l k = α ∂ ∥ pp ⋆ ∥ ∂ l k − ∥ pp ⋆ ∥ cos α ∂ cos α ∂ l k − ( r k + − r k + ) ∂ τ α ∂ l k (20)The full expression for the derivatives can be easily computedusing the following formulas:cos α = (cid:115) − ( r k + − r k + ) ( x − l k ) + y + z τ α ( p ) = (cid:115) ( x − l k ) + y + z ( x − l k ) + y + z ∥ pp ⋆ ∥ = c k + p · c k + c k + ∥ c k + c k + ∥ = ( x − l k )( x − l k ) + yy + zz ( x − l k ) + y + z Plugging all the derivatives in Equation 18 yields δl , and l k canbe updated as ˆ l k = l k + δl . This impacts the position of c k + , whosenew position is computed as ˆ c k + = c k + ˆ l k c k c k + c k c k + , and l k + isrecomputed as : l k + = ∥ ˆ c k + c k + ∥ .The two-bones energy E k , k + is then optimized with respect to l k + . This optimization is symmetric to the l k case above and can beeasily adapted. Finally, the optimization of the radius of the commonjoint and rotation angle around axis c k c k +2