Sub-Riemannian geometry on some step-two Carnot groups
SSub-Riemannian geometry on some step-two Carnotgroups
Hong-Quan Li, Ye Zhang
Abstract.
This paper is a continuation of the previous work of the first author. Wecharacterize a class of step-two groups introduced in [88], saying GM-groups, via somebasic sub-Riemannian geometric properties, including the squared Carnot-Carath´eodorydistance, the cut locus, the classical cut locus, the optimal synthesis, etc. Also, the short-est abnormal set can be exhibited easily in such situation. Some examples of such groupsare step-two groups of corank 2, of Kolmogorov type, or those associated to quadraticCR manifolds. As a byproduct, the main goal in [19] is achieved from the setting ofstep-two groups of corank 2 to all possible step-two groups, via a completely differentmethod. A partial answer to the open questions [20, (29)-(30)] is provided in this pa-per as well. Moreover, we provide a entirely different proof, based yet on [88], for theGaveau-Brockett optimal control problem on the free step-two Carnot group with threegenerators. As a byproduct, we provide a new and independent proof for the main resultsobtained in [103], namely, the exact expression of d ( g ) for g belonging to the classicalcut locus of the identity element o , as well as the determination of all shortest geodesicsjoining o to such g . Mathematics Subject Classification (2010): 22E25, 53C17, 53C22Key words and phrases:
Carnot-Carath´eodory distance, Gaveau-Brockett optimalcontrol problem, step-two Carnot group, cut locus, shortest geodesic, optimal synthesis
In the past several decades, step-two groups and their sub-Laplacians, as special Liegroups of polynomial volume growth or perfect sub-Riemannian manifolds, have attractedwide attention from experts in various fields, such as complex analysis, control theory,geometric measure theory, harmonic analysis, heat kernel, Lie group theory, probabilityanalysis, PDE, sub-Riemannian geometry, etc. We only mention some relevant workshere, [4–18, 20–28, 30, 32, 34–48, 50–53, 55–63, 65–72, 74–76, 78, 80–87, 90, 91, 96, 97, 103, 105,114, 118, 123, 124, 131, 132]. The list is far from exhaustive and in fact is rather limited.More related papers can be found in the references therein as well as their subsequentresearches.In this present paper, we will restrict our attention to the sub-Riemannian geometryon step-two Carnot groups. Many relevant works can be found in the literature as citedbefore. However, some most fundamental problems are far from being solved, or even1 a r X i v : . [ m a t h . DG ] F e b oorly known, in this very fine framework. Recently, in [88] (cf. also [89]), the firstauthor used Loewner’s theorem to study two basic problems of sub-Riemannian geometryon 2-step groups: one is to obtain the exact formula for the sub-Riemannian distance, thatis the Gaveau-Brockett optimal control problem; another is to characterize all (shortest)normal geodesics from the identity element o to any given g (cid:54) = o . In particular, there existsan enormous class of 2-step groups, saying GM-groups (see Subsection 2.3 below for thedefinition), which have some consummate sub-Riemannian geometric properties. Moreprecisely, the squared Carnot-Carath´eodory distance d ( g ) := d ( o, g ) and the cut locusof o , Cut o (namely the set of points where d is not smooth) can be characterized easilyin such situation. For example, all Heisenberg groups even generalized Heisenberg-typegroups (so step-two groups of corank 1), and star graphs are GM-groups. We emphasizethat in general, the expression of d ( g ) is extremely complicated. It is impossible toprovide an explicit expression, via a relatively simple inverse function, as in the mostspecial situation of generalized Heisenberg-type groups (cf. [63], [27] and [88]). We referthe reader to [88] for more details. The work is a continuation of [88], one of our main goalsis to provide various equivalent characterizations of GM-groups via basic sub-Riemanniangeometric properties.Moreover, the sub-Riemannian geometry in the setting of 2-step groups is not wellunderstood. Roughly speaking, the main reason for this is that the well-understoodexamples are merely the Heisenberg group (cf. [63]) and generalized Heisenberg groups (cf.[27]). Other known cases, such as generalized Heisenberg-type groups as well as the directproduct of a generalized Heisenberg group with a Euclidean space (in particular, step-twogroups of corank 1), are essentially the same. Hence, it is very meaningful and exigent tosupply some examples possessing richer sub-Riemannian geometric properties. Here, wewill provide more examples of GM-groups, such as groups of corank 2, of Kolmogorov type,or those associated to quadratic CR manifolds. In particular, the aforementioned groupsmay have complicated shortest abnormal set of o , Abn ∗ o , that is, the set of the endpointsof abnormal shortest geodesics starting from o . The existence of non-trivial abnormalshortest geodesics is closely related to the regularity of the Carnot-Carath´eodory distance.And its appearance makes an obstacle for us to deal with some topics, such as the heatkernel asymptotics and geometric inequalities, etc. See for example [2, 20, 29, 30, 88, 105]and the references therein for more details. Recall that (cf. [118, 119]) a sub-Riemannianmanifold is called ideal if it is complete and has no non-trivial abnormal shortest geodesics.In our setting, a step-two group G is ideal if and only if it is of M´etivier type (see Subsection2.2.1 for the definition).Optimal syntheses (namely the collection of all arclength parametrized geodesics withtheir cut times) are generally very difficult to obtain. In the setting of step-two groups, asfar as we know, a correct result about them can be found only on nonisotropic Heisenberggroups. See [2, §
13] and Remark 1 below for more details. However, we can now give theoptimal synthesis from the identity element o on GM-groups. As a result, the classicalcut locus of o , Cut CL o , that is the set of points where geodesics starting at o cease to beshortest, can be characterized on such groups as well.We say that d is semiconcave (resp. semiconvex ) in a neighborhood of g if there exist2 > δ > d ( g + g (cid:48) ) + d ( g − g (cid:48) ) − d ( g ) ≤ C | g (cid:48) | (resp. ≥ − C | g (cid:48) | ) , (1.1) DSSCC for all g ± g (cid:48) ∈ B ( g , δ ) = { g ∈ R q × R m ; | g − g | < δ } . Here we stress that | · | denotesthe usual Euclidean norm and g ± g (cid:48) the usual operation in the Euclidean space. Wealso remark that this definition is independent of the choice of local coordinates around g (here we use the canonical one) since d is locally Lipschitz w.r.t. the usual Euclideandistance (see for example [118, 119]). Set in the sequel SC − o := (cid:8) g ; d fails to be semiconcave in any neighborhood of g (cid:9) , (1.2) DoFSC1 SC + o := (cid:8) g ; d fails to be semiconvex in any neighborhood of g (cid:9) . (1.3) DoFSC2
Recall that SC − o = Abn ∗ o in the setting of M´etivier groups, all free Carnot groups ofstep 2, as well as some other sub-Riemannian structures. See [35], [54], [102], [20, § § − o = Abn ∗ o in the more general sub-Riemannian setting(where our Abn ∗ o is noted by Abn( o )). In the framework of GM-groups, Abn ∗ o can bedescribed easily; as a byproduct, we give a positive answer to this open problem. Also,other related results and step-two groups can be found in Subsection 2.5.In addition, the most challenging problem should be to study the sub-Riemanniangeometry in the setting of free step-two groups with k generators N k, ∼ = R k × R k ( k − ( k ≥ G with k generators, that is the first layer inthe stratification of Lie algebra has dimension k , there exists some relation between G and N k, by Rothschild-Stein lifting theorem (see [124] or [32]). Observe that N , isexactly the Heisenberg group, which is well-known (cf. [63] or [27]). Recall that (cf. [63]and [34]) the original Gaveau-Brockett optimal control problem is to determine the sub-Riemannian distance on N k, with k ≥
3. This is a long-standing open problem. Recently,it is completely solved on N , in [88, § N , , the classical cut locus of o , Cut CL o , has been determined in [113]and [103] by completely different techniques; furthermore, the expression of d ( g ) with g ∈ Cut CL o has been obtained in [103]. Strictly speaking, we have used the above knownresults in the proof of [88]. Also notice that Abn ∗ o and Cut CL o on N , are relatively verysimple. However, it is still an open problem to characterize the classical cut locus of o on N k, with k ≥
4, see [123] for more details. Motivated by this problem, we ask naturallyif we can determine first d ( g ) for any g then Cut CL o on N , . This is exactly another mainpurpose of this work.In the framework of step-two groups, first we recall that all shortest geodesics arenormal (cf. [3] or [119, § We would like to thank L. Rizzi for informing us of the addendum of [20] that the definition ofthe failure of semiconcavity/semiconvexity for the open questions should be the one stated here in oursituation (which is consist with the classical definition of local semiconcavity/semiconvexity), rather thanthe one given in [20]. For more details, we refer to the addendum on L. Rizzi’s homepage. o to any given g (cid:54) = o have been characterized by [88, Theorem 2.4]. Moreover,it follows from [88, Theorem 2.5] that the squared distance has been determined in asymmetric, scaling invariant subset with non-empty interior. In particular, for the specialcase of N , , we can simplify the Gaveau-Brockett problem via an orthogonal-invariantproperty, and some useful results can be found in [88, § N , first on some dense open subset, then onwhole space via a limiting argument. Then, from the regularity of the squared sub-Riemannian distance, we can further determine the cut locus Cut o . Finally, all shortestgeodesics joining o to any given point in Cut CL o are obtained by approximating themwith that joining o to some points in (Cut o ) c , which are relatively easy to describe. Asa consequence, we supply an independent and new proof for the main results obtainedin [103].Some applications will be given in a future work.This paper is organized as follows. In Section 2, we collect some preliminary materialsand give our main results, which will be proven in Section 3. In Section 4, we providea sufficient condition for a step-two group to be GM-group by using semi-algebraic the-ory. As a consequence, we find that all step-two groups of corank 2 are GM-groups.Furthermore, we also prove in this section that there exist M´etivier groups of corank 3and of sufficiently large dimension which are not of GM-type. In Section 5, we considerthe sub-Riemannian geometry in the setting of step-two K-type groups. Step-two groupsassociated to quadratic CR manifolds will be studied in Section 6. Finally, we give inSection 7 a completely different proof, based on [87], for the Gaveau-Brockett optimalcontrol problem on N , . As an application, we provide a new and independent proof forthe main results obtained in [103]. s2 s21 Recall that a connected and simply connected Lie group G is a step-two Carnot group ifits left-invariant Lie algebra g admits a stratification g = g ⊕ g , [ g , g ] = g , [ g , g ] = { } , where [ · , · ] denotes the Lie bracket on g . We identify G and g via the exponential map.As a result, G can be considered as R q × R m , q, m ∈ N ∗ = { , , , . . . } (in this paper weuse N to denote the set of natural numbers { , , , . . . } ), with the group law( x, t ) · ( x (cid:48) , t (cid:48) ) = (cid:18) x + x (cid:48) , t + t (cid:48) + 12 (cid:104) U x, x (cid:48) (cid:105) (cid:19) , g := ( x, t ) ∈ R q × R m , where (cid:104) U x, x (cid:48) (cid:105) := ( (cid:104) U (1) x, x (cid:48) (cid:105) , . . . , (cid:104) U ( m ) x, x (cid:48) (cid:105) ) ∈ R m . U = { U (1) , . . . , U ( m ) } is an m -tuple of linearly independent q × q skew-symmetricmatrices with real entries and (cid:104)· , ·(cid:105) (or · in the sequel when there is no ambiguity) denotesthe usual inner product on R q . Furthermore, in this article, we will not distinguish rowvectors from column vectors and we may write a column vector t with scalar coordinates t , . . . , t m , simply as ( t , . . . , t m ) unless otherwise stated in the context. Note that m ≤ q ( q − . We call such a group a step-two group of type ( q, m, U ), which is denoted by G ( q, m, U ) or G for simplicity. One can refer to [49] or [32] for more details.Let U ( j ) = ( U ( j ) l,k ) ≤ l,k ≤ q (1 ≤ j ≤ m ). The canonical basis of g is defined by theleft-invariant vector fields on G :X l ( g ) := ∂∂x l + 12 m (cid:88) j =1 (cid:16) q (cid:88) k =1 U ( j ) l,k x k (cid:17) ∂∂t j , ≤ l ≤ q. And the canonical sub-Laplacian is ∆ = q (cid:80) l =1 X l . G : some ele-mentary properties Let us first recall some basic facts about the sub-Riemannian geometry in the frameworkof 2-step groups. In our setting, we will sometimes use equivalent definitions for someconcepts in order to avoid recalling too many notations. We refer the reader to [2, 28–30,105, 119, 129] and references therein for further details. Also notice that partial but notall results below remain valid in some more general setting.The group G = G ( q, m, U ) is endowed with the sub-Riemannian structure, namelya scalar product on g , with respect to which { X l } ≤ l ≤ q are orthonormal (and the norminduced by this scalar product is denoted by (cid:107) · (cid:107) ). In the sequel, m is called the corank of G ( q, m, U ).A horizontal curve γ : [0 , → G is an absolutely continuous path such that˙ γ ( s ) = q (cid:88) j =1 u j ( s )X j ( γ ( s )) for a.e. s ∈ [0 , , and we define its length as follows (cid:96) ( γ ) := (cid:90) (cid:107) ˙ γ ( s ) (cid:107) ds = (cid:90) (cid:118)(cid:117)(cid:117)(cid:116) q (cid:88) j =1 | u j ( s ) | ds. The
Carnot-Carath´eodory (or sub-Riemannian ) distance between g, g (cid:48) ∈ G is then d ( g, g (cid:48) ) := inf { (cid:96) ( γ ); γ (0) = g, γ (1) = g (cid:48) , γ horizontal } . A geodesic is a horizontal curve γ satisfying: (cid:107) ˙ γ ( s ) (cid:107) is constant and for any s ∈ [0 , I of s in [0 ,
1] such that (cid:96) ( γ | I ) is equal to the distance5etween its endpoints. And a shortest geodesic is a geodesic γ which realizes the distancebetween its extremities, that is, (cid:96) ( γ ) = d ( γ (0) , γ (1)).By slightly abusing of notation in the sequel, 0 denotes the number 0 or the origin inthe Euclidean space. Let o = (0 ,
0) denote the identity element of G . It is well-knownthat d is a left-invariant distance on G . Hence we set in the following d ( g ) := d ( g, o ).Recall that d is locally Lipschitz on G with respect to the usual Euclidean distance. Thedilation on G is defined by δ r ( x, t ) := ( r x, r t ) , ∀ r > , ( x, t ) ∈ G . (2.1) nDS And the following scaling property is well-known: d ( r x, r t ) = r d ( x, t ) , ∀ r > , ( x, t ) ∈ G . (2.2) scap o ns221 In the setting of step-two Carnot groups, it is well-known that all shortest geodesics areprojections of normal Pontryagin extremals, that is integral curves of the sub-RiemannianHamiltonian in T ∗ G . See for example [3, § T ∗ G ∼ = ( R q × R m ) × ( R q × R m ) isdefined by H = H ( x, t, ξ, τ ) := 12 q (cid:88) j =1 ζ j , ζ j := ξ j + 12 m (cid:88) k =1 (cid:32) q (cid:88) l =1 U ( k ) j,l x l (cid:33) τ k , ≤ j ≤ q. And a normal Pontryagin extremal , (cid:16) γ ( s ) := ( x ( s ) , t ( s )) , ξ ( s ) , τ ( s ) (cid:17) : [0 , −→ T ∗ G , with γ (0) = o, is a solution of˙ x k = ∂H∂ξ k , ˙ t j = ∂H∂τ j , ˙ ξ k = − ∂H∂x k , ˙ τ j = − ∂H∂t j , ≤ k ≤ q, ≤ j ≤ m. (2.3) HJE
The covector ( ξ (0) , τ (0)) (resp. ( ξ (1) , τ (1))) is called the initial (resp. final) covectorof ( γ ( s ) , ξ ( s ) , τ ( s )). Its projection γ ( s ) := γ ( ξ (0) , τ (0); s ) = γ ( ξ (0) ,τ (0)) ( s ) = ( x ( s ) , t ( s )) : [0 , −→ G is said to be the normal geodesic starting from o with initial covector ( ξ (0) , τ (0)).Note that H is independent of t . Hence we have τ ( s ) ≡ τ (0) := 2 θ ∈ R m . (2.4)Set in the following (cid:101) U ( θ ) := m (cid:88) j =1 θ j U ( j ) and U ( θ ) := i (cid:101) U ( θ ) , for θ = ( θ , . . . , θ m ) ∈ R m . (2.5) Du1 G is a M´etivier group (or of M´etivier type ) if U ( θ ) is invertiblefor any θ (cid:54) = 0 (cf. [98]).Let ζ ( s ) := ξ ( s ) + (cid:101) U ( θ ) x ( s ). Remark that ξ (0) = ζ (0). A simple calculation impliesthat ζ ( s ) = e s (cid:101) U ( θ ) ζ (0) , x ( s ) = (cid:90) s ζ ( r ) dr, t ( s ) = 12 (cid:90) s (cid:104) U x ( r ) , ζ ( r ) (cid:105) dr. (2.6) GEn
In particular, we have x (1) = (cid:90) ζ ( r ) dr = sin U ( θ ) U ( θ ) e (cid:101) U ( θ ) ζ (0) , (2.7) endpointx and γ ( ξ (0) , τ (0); s ) = ( x ( s ) , t ( s )) is extendable and real analytic on [0 , + ∞ ). It is easyto check the following homogeneity property: γ ( α ζ , α θ ; s ) = γ ( ζ , θ ; α s ) , ∀ α > , s ≥ , ( ζ , θ ) ∈ R q × R m . From now on, the domain of the normal geodesic γ ( s ) = γ ( ζ , θ ; s ) is [0 , + ∞ ) andthat of γ ( s ) = γ ( ζ , θ ) ( s ) is [0 ,
1] by default. Also remark that γ = o if ζ = 0, which istrivial. And all normal geodesics are by convention starting from o in this work.Let ( x, t ) denote the endpoint of γ ( ζ,τ ) , then by (2.6), that of γ ( − ζ,τ ) is ( − x, t ). More-over, both γ ( ζ,τ ) and γ ( − ζ,τ ) have length | ζ | , where | · | denotes the usual Euclidean norm.Combining this with the fact that the Carnot-Carath´eodory distance is a left-invariantdistance on G , we have the following simple but useful observation: Lemma 1.
In the setting of step-two groups, it holds that d ( x, t ) = d ( − x, t ) = d ( x, − t ) = d ( − x, − t ) , ∀ ( x, t ) ∈ G . (2.8) te1 The following basic property is well-known:
BPG
Lemma 2.
Let ≤ s < s . Assume that γ ( ζ , θ ; s ) = γ ( ζ (cid:48) , θ (cid:48) ; s ) for all s ≤ s ≤ s .Then we have γ ( ζ , θ ; · ) ≡ γ ( ζ (cid:48) , θ (cid:48) ; · ) . Moreover, it holds that ζ = ζ (cid:48) . More information about such geodesics can be found in Proposition 1 below.
The sub-Riemannian exponential map based at o is the smooth map defined byexp : R q × R m −→ G ( ζ , θ ) (cid:55)−→ γ ( ζ , θ ; 1) . In our setting, it is surjective and has the following property: γ ( ζ , θ ; s ) = exp { s ( ζ , θ ) } , ∀ s ≥ , ( ζ , θ ) ∈ R q × R m . See for example [2, § emma 3. Suppose that exp( w, τ ) = ( x, t ) . Then we have exp( r w, τ ) = ( r x, r t ) , ∀ r (cid:54) = 0 , (2.9) symN2 exp( − e (cid:101) U ( τ ) w, − τ ) = ( − x, − t ) . (2.10) symN3 Indeed, using (2.6), (2.9) is trivial, and an elementary computation implies (2.10).Now assume that γ ( s ) = exp { s ( ζ , θ ) } is parametrized by arclength (or arclengthparametrized ), namely | ζ | = 1. Let g = γ ( s ). We say that g is conjugate to o along γ if s ( ζ , θ ) is a critical point of exp. The cut time along γ is defined as h cut := h cut ( γ ) = sup { s > γ | [0 , s ] is a shortest geodesic } . (2.11)When h cut < + ∞ , γ ( h cut ) is said to be the cut point of o along γ . And we say γ has nocut point if h cut = + ∞ . The optimal synthesis from o is the collection of all arclengthparametrized geodesics with their cut times. ns223 A normal geodesic is said to be abnormal (i.e. singular) if it has two (so infinitely many)different normal lifts (see [120, Remark 8] and [119, Remark 2.4]). However, we stressthat our definition of abnormal geodesic is not complete in general. In particular, on somesub-Riemannian manifolds, excluding our step-two groups, there are shortest geodesicswhich are not projections of normal Pontryagin extremals. For the original definition ofabnormal geodesic as well as counter-examples, we refer the reader to [2, 92, 104, 105, 119]and the references therein for more details.In this work, the (normal-) abnormal set of o , Abn o is defined byAbn o := { g ; there exists an abnormal (which is also normal) geodesic joining o to g } . And we define the shortest abnormal set of o as follows:Abn ∗ o := { g ; there exists an abnormal shortest geodesic joining o to g } , which is a subset of Abn o . The main difference between the two sets is that: in thedefinition of Abn o , we do not care about minimality of geodesics, while this is needed inthat of Abn ∗ o . Notice that o ∈ Abn ∗ o . Also remark that our Abn o is exactly Abn nor ( e )in [82, § § cAg Proposition 1.
Let θ (cid:54) = θ (cid:48) . Then γ ( w, θ ; · ) ≡ γ ( w, θ (cid:48) ; · ) if and only if for σ = θ − θ (cid:48) ∈ R m \ { } , we have U ( σ ) U ( θ ) k w = 0 , ∀ k ∈ N , (2.12) CharaAbn or equivalently, U ( σ ) e s (cid:101) U ( θ ) w = 0 , ∀ s ∈ R . (2.13) AGc hat is, γ ( w, θ ) (or γ ( w, θ ; · ) ) is abnormal if and only if there exists some σ (cid:54) = 0 suchthat (2.12) (or equivalently (2.13) ) satisfies. As a consequence, we get the following known fact:
Corollary 1. If γ ( w, θ ) is abnormal, then so does γ ( a w, b θ ) for any a, b ∈ R . In particular,for any < a < , the restriction of γ ( w, θ ) in [0 , a ] , γ ( w, θ ) | [0 , a ] = γ ( a w, a θ ) is alsoabnormal. A normal geodesic is called strictly normal if it is not abnormal. Let γ = γ ( w, θ ) (resp. γ ( w, θ ; · )) and 0 ≤ s < s ≤ ≤ s < s < + ∞ ). We consider the restrictionof γ in [ s , s ], γ | [ s , s ] as well as γ s ,s ( s ) := γ ( s ) − · γ ( s + s ( s − s )) , s ∈ [0 , . By (2.6), a simple calculation shows that γ s ,s = γ (( s − s ) e s (cid:101) U ( θ ) w, s − s ) θ ) , which is a normal geodesic starting from o . If γ s ,s is abnormal, then it follows fromProposition 1 that there exists a σ ∈ R m \ { } such that( s − s ) k +1 U ( σ ) U ( θ ) k e s (cid:101) U ( θ ) w = 0 , ∀ k ∈ N , which implies that γ = γ ( w, θ ) itself (so γ ( w, θ ; · )) is also abnormal by (2.13). We say anormal geodesic γ ( w, θ ) (resp. γ ( w, θ ; · )) does not contain abnormal segments if γ s ,s isnot abnormal for any 0 ≤ s < s ≤ ≤ s < s < + ∞ ). In conclusion, we getthe following: nLN Lemma 4.
In the framework of step-two groups, any strictly normal geodesic does notcontain abnormal segments.
It is worthwhile to point out that the above property is no longer valid in general.See [99] for more details.In this paper, the cut locus of o , Cut o , is defined asCut o := S c , with S := { g ; d is C ∞ in a neighborhood of g } . (2.14) DCUT
Recall that (see for example [2, § S = { g ; there exists a unique shortest geodesic γ from o to g , which isnot abnormal, and g is not conjugate to o along γ } , (2.15) CCL1 and it is open and dense in G . Hence Cut o is closed. Furthermore, it has measure zero(cf. [118, Proposition 15]). Remark also that o ∈ Abn ∗ o ⊆ Cut o .The classical cut locus of o , Cut CL o is defined as the set of points where geodesicsstarting at o cease to be shortest, that isCut CL o := { g ; g is the cut point of o along some arclength parametrized normal geodesic } . Now, we can give an affirmative answer to the open question [20, first part of (30)] inour framework, which follows from (2.15), Lemma 4 and [2, Theorem 8.72].9 Theorem 1.
In the setting of step-two Carnot groups, it holds that
Cut o = Cut CL o ∪ Abn ∗ o . s22 Let us begin by recalling the initial reference set and the reference function, introducedin [88], which are defined respectively byΩ ∗ := (cid:26) τ ∈ R m ; max | x | =1 (cid:104) U ( τ ) x, x (cid:105) < π (cid:27) = { τ ∈ R m ; (cid:107) U ( τ ) (cid:107) < π } , (2.16) oa φ ( g ; τ ) = (cid:104) U ( τ ) cot U ( τ ) x, x (cid:105) + 4 t · τ, τ ∈ Ω ∗ , g = ( x, t ) ∈ G . (2.17) RFn
Notice that the function φ ( g ; · ) is well-defined provided the spectrum of U ( τ ) doesnot contain any k π ( k ∈ Z \ { } ). Also, we will use its usual extension on Ω ∗ (which isdenoted by φ ( g ; · ) as well). And we have eP1 Proposition 2 ( [88], Proposition 2.1 and Remark 2.1) . For any g , φ ( g ; · ) is smooth andconcave in Ω ∗ . Moreover, for every g , there exists an θ g ∈ Ω ∗ such that φ ( g ; θ g ) = sup τ ∈ Ω ∗ φ ( g ; τ ) . Let ∇ θ = (cid:16) ∂∂θ , . . . , ∂∂θ m (cid:17) denote the usual gradient on R m . Recall that (cf. [88, § (cid:101) M := (cid:26)(cid:18) x, − ∇ θ (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) (cid:19) ; x ∈ R q , θ ∈ Ω ∗ (cid:27) , (2.18) nOM which is the union of disjoint and nonempty subsets M := { g ; ∃ θ ∈ Ω ∗ s.t. θ is a nondegenerate critical point of φ ( g ; · ) in Ω ∗ } = { g ; ∃ θ ∈ Ω ∗ s.t. the set of global maximizers of φ ( g ; · ) in Ω ∗ is { θ }} , (2.19) m and (cid:101) M := { g ; the set of global maximizers of φ ( g ; · ) in Ω ∗ has at least two points } . (2.20) dtM2 Also recall that M is an open set, M ⊆ S , o ∈ (cid:101) M ⊆ Abn ∗ o ⊆ Cut o and (cf. [88, § d ( g ) = max τ ∈ Ω ∗ φ ( g ; τ ) for g ∈ (cid:101) M , and d ( g ) = sup τ ∈ Ω ∗ φ ( g ; τ ) for g ∈ (cid:101) M . (2.21) Csrd
And we have the following
RLT
Theorem 2 ( [88], Theorems 2.4 and 2.5) . Assume that ζ ∈ R q \{ } and θ ∈ Ω ∗ . Then exp { ( ζ , θ ) } = g := ( x , t ) if and only if x = (cid:18) U ( θ )sin U ( θ ) e − (cid:101) U ( θ ) (cid:19) − ζ , t = − ∇ θ (cid:104) U ( θ ) cot U ( θ ) x , x (cid:105) . urthermore, in such case, we have d ( g ) = | ζ | = (cid:12)(cid:12)(cid:12)(cid:12) U ( θ )sin U ( θ ) x (cid:12)(cid:12)(cid:12)(cid:12) = φ ( g ; θ ) , and the unique shortest geodesic from o to g is exp { s ( ζ , θ ) } ( ≤ s ≤ ), which isstrictly normal if and only if g ∈ M . As a consequence, we yield immediately
Nc1
Corollary 2.
Let γ ( s ) := exp { s ( ζ , τ ) } be an arclength parametrized geodesic, that is | ζ | = 1 . Then its cut time h cut = + ∞ if τ = 0 , and in such case γ is a ray in the firstlayer. In addition, we have h cut ≥ π/ (cid:107) U ( τ ) (cid:107) when τ (cid:54) = 0 . Rkn1
Remark 1.
It follows from [88, Proposition 5.1 and/or Corollary 2.2] that if G is not ofM´etivier type, then there exist (cid:54) = θ ∈ R m and (cid:54) = x ∈ ker (cid:101) U ( θ ) such that | x | = 1 and exp { s ( x , θ ) } = exp { s ( x , } for all s > . Hence the cut time of exp { s ( x , θ ) } is equalto + ∞ and the statement of [19, Theorems 6 and 7] is misleadingly phrased. However, acorrect statement and their generalization can be found in Theorem 4, Corollaries 8 or 9below. Also notice that our method to determine the cut time is completely different fromtheirs. Another easy but very useful consequence is the following: nCcc
Corollary 3.
It holds that
Cut CL o ⊆ (cid:101) M c . Combining [88, Proposition 5.1 (b)] with Theorem 2 as well as Proposition 1, we havethe following characterization of (cid:101) M , Abn ∗ o and Abn o : NPA1
Proposition 3.
It holds that: (cid:101) M = (cid:110) γ ( s ) = γ ( ζ, τ ; s ); γ is abnormal, | ζ | = 1 and ≤ s < π (cid:107) U ( τ ) (cid:107) (cid:111) , Abn ∗ o = { γ ( s ) = γ ( ζ, τ ; s ); γ is abnormal, | ζ | = 1 and ≤ s ≤ h cut ( γ ) } , Abn o = { γ ( s ) = γ ( ζ, τ ; s ); γ is abnormal, | ζ | = 1 , s ≥ } . Remark 2.
Obviously, the arclength parametrized geodesic γ ( ζ , τ ; s ) is abnormal if andonly if there exists a s > such that (cid:107) U ( s τ ) (cid:107) < π and exp { s ( ζ , τ ) } ∈ (cid:101) M . In order to describe Abn ∗ o and Abn o , it suffices to determine (cid:101) M , which is much lessdifficult. Moreover, we have M2AbsAb
Corollary 4.
In the setting of step-two groups, if (cid:101) M ⊆ { ( x, x ∈ R q } , then Abn o =Abn ∗ o = (cid:101) M . Rna
Remark 3. (a) Recall that R q × { } ⊆ (cid:101) M . If ( x , ∈ Abn o for some x ∈ R q , then wehave also ( x , ∈ (cid:101) M ⊆ Abn ∗ o .(b) In general, Abn o = Abn ∗ o does not imply (cid:101) M ⊆ R q × { } . In fact, let H = R × R denote the Heisenberg group of real dimension 3 and consider G = H × H ∼ = R × R .We have Abn o = Abn ∗ o = (cid:0) { o H } × H (cid:1) ∪ (cid:0) H × { o H } (cid:1) , but { o } ∪ ( { o H } × M H ) ∪ ( M H × { o H } ) = (cid:101) M (cid:54)⊆ R × { } , where M H = { ( x, t ); x ∈ R \ { }} and o H denote the corresponding set M and identityelement in the setting of H respectively.(c) The example in (b) also provides a group on which (cid:101) M (cid:36) Abn ∗ o since ((0 , , , o H ) ∈ Abn ∗ o \ (cid:101) M ⊆ H × H = G . Furthermore, another example of (cid:101) M (cid:36) Abn ∗ o can be foundin the proof of Proposition 7 (see Subsection 6.2 below), and that of Abn ∗ o (cid:36) Abn o inSubsection 6.3 below.(d) Obviously, a step-two group is of M´etivier type if and only if Abn ∗ o = { o } . Recall that the nonempty open subset M ⊂ G is the set of points, g , where thereference function φ ( g ; · ) has a nondegenerate critical point in the initial reference setΩ ∗ . Observe that M is symmetric and scaling invariant; namely, if g ∈ M , then we have g − = − g ∈ M and δ r ( g ) ∈ M for all r >
0. A step-two group G is said to be a GM-group (or of type GM ) if it satisfies M = G . ( GM )Notice that if both G and G satisfy ( GM ), then so does the direct product G × G . SeeAppendix B for more details. Also remark that GM groups form a wild set. Indeed, for anygiven G ( q, m, U ), we can construct an uncountable number of GM-groups G ( q +2 n, m, (cid:101) U ).See [88, § R m defined by (cf. [88, § R := R , with R := (cid:26) θ = 14 ∇ t d ( g ) ; g = ( x, t ) / ∈ Cut o (cid:27) open . (2.22) GRs
It follows from [102, §
3] or [88, Proposition 5.2] that
R ∩ Ω ∗ is dense in Ω ∗ . Then R ⊇ Ω ∗ = { θ ; (cid:107) U ( θ ) (cid:107) ≤ π } . (2.23) grinc Set V := { ϑ ∈ R m ; det( kπ − U ( ϑ )) (cid:54) = 0 , ∀ k ∈ N ∗ } and W := exp( R q × (2 V c )) , (2.24) nW “bad” normal geodesic (resp. “good” normal geodesic ) means γ = γ ( w, θ ) with θ ∈ V c (resp. θ ∈ V ). It is clearlythat W is of measure zero.Finally some notations of special functions related to − s cot s are also recalled: f ( s ) := 1 − s cot s, µ ( s ) := f (cid:48) ( s ) = 2 s − sin (2 s )2 sin s , ψ ( s ) := f ( s ) s . (2.25) EFs
Our first result is the following: R The following theorem should be useful to determine all shortest geodesics in the settingof step-two groups. t1 Theorem 3.
Let o (cid:54) = g ∈ G , and γ g ( s ) ( ≤ s ≤ ) be a shortest geodesic joining o to g .Then there exist ζ ∈ R q with | ζ | = d ( g ) and θ ∈ R such that γ g ( s ) = exp { s ( ζ, θ ) } forall ≤ s ≤ . The following property is a direct consequence of (2.15) together with Lemma 4 and [2,Theorem 8.72]: bLn
Lemma 5.
Suppose that g = exp { ( ζ , τ ) } ∈ S and exp { s ( ζ , τ ) } ( ≤ s ≤ ) is theunique shortest geodesic between o and g . Then exp { s ( ζ , τ ) } ∈ S and − s τ ∈ R forall < s ≤ . nRnM Remark 4.
Recall that (cid:101) M = M ∪ (cid:101) M and (cid:101) M ⊆ Cut o = S c . By the fact that exp { ( ζ, τ ) } ∈ (cid:101) M whenever τ ∈ Ω ∗ , a direct consequence of Lemma 5 is that M (cid:54) = ∅ . See also [97, Propo-sition 6 and §
3] for another explanation.
It follows from Lemma 5 that R is star-shaped w.r.t. the origin 0, that is, if τ ∈ R ,then we have s τ ∈ R for all s ∈ [0 , §
11] to determinethe squared sub-Riemannian distance for general non-GM groups.
RKn2
Remark 5.
Theorem 3 could be considered as a somewhat converse statement of [120,Proposition 4] in our setting. In fact, from the proof of Theorem 3, we know that thereexist { g j } + ∞ j =1 ⊆ S with γ ( ζ ( j ) , θ ( j ) ) the shortest geodesic joining o to g j such that g j → g and ( ζ ( j ) , θ ( j ) ) → ( ζ, θ ) as j → + ∞ . As a result, every shortest geodesic can be inducedby some limiting sub-differential in our situation. Combining this with another basic property of R , namely [88, Corollary 2.4], this iswhy it is called the global reference set. 13 .4.2 Other sub-Riemannian geometric properties on step-two groups Let us begin with an upper bound about the cut time of an arclength parametrizedgeodesic: c1 Corollary 5.
Let C R := max τ ∈ R (cid:107) U ( τ ) (cid:107) , C τ := sup { s > r τ ∈ R , ∀ ≤ r ≤ s } ( τ ∈ R m ) . For any arclength parametrized geodesic γ ( s ) = exp { s ( ζ, τ ) } = γ ( ζ, τ ; s ) , its cut timesatisfies h cut ≤ sup { σ ; σ ∈ R m , γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ) }≤ sup (cid:26) R (cid:107) U ( σ ) (cid:107) ; σ ∈ R m , γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ) (cid:27) (2.26) nCUTUn with the understanding R = + ∞ . In particular, assume moreover that γ ( s ) = exp { s ( ζ, τ ) } is not abnormal, then h cut ≤ τ ≤ R (cid:107) U ( τ ) (cid:107) . nRKnSM Remark 6.
Fix ( ζ, τ ) and let Π ( ζ,τ ) := { σ − τ ∈ R m ; γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ) } . Proposition1 implies that Π ( ζ,τ ) is a linear subspace of R m . It is clear that the continuous function σ (cid:55)→ (cid:107) U ( σ ) (cid:107) defined on τ + Π ( ζ,τ ) attains its minimum. Hence, the last “ sup ” in (2.26) can be replaced by “ max ”. Moreover, we have the following:
NLA1
Lemma 6. In the framework of step-two Carnot groups,
Abn ∗ o is a closed set. On a step-two group G , notice that o (cid:54)∈ Cut CL o . Now assume that Cut CL o ∩ Abn ∗ o = ∅ andAbn ∗ o (cid:54) = { o } . Let γ ( s ) = exp { s ( ζ, τ ) } = γ ( ζ, τ ; s ) be an arclength parametrized abnormalgeodesic. Then it follows from Lemma 6 that its cut time is + ∞ . Using Corollary 5 andRemark 6, we obtain γ ( ζ, τ ; · ) = γ ( ζ, · ), which implies { γ ( s ); s ≥ } ⊆ R q × { } by(2.6). In conclusion, combining this with Corollaries 4 and 3, we get the following: NCorA1
Corollary 6.
In the setting of step-two Carnot groups,
Cut CL o ∩ Abn ∗ o = ∅ if and only if (cid:101) M ⊆ R q × { } . To finish this subsection, we provide the following:
CLMMW
Proposition 4.
In the context of step-two Carnot groups, it holds that (cid:101) M = M . For a general sub-Riemannian manifold M , we can define the shortest abnormal set of y ∈ M , Abn ∗ y ,as the set of the endpoints of abnormal (not necessarily normal) shortest geodesics starting from y andAbn ∗ y is a closed set as well. This is a result of the characterization of abnormal Pontryagin extremalsvia Lagrange multipliers rule and the compactness of minimal controls. We would like to thank L. Rizzifor informing us of this general result and providing a sketched proof. For the sake of completeness, wegive a proof in the setting of step-two groups without using the notion of the endpoint map in Section 3. .4.3 Characterizations of GM-groups GM-groups have very fine properties of sub-Riemannian geometry. More precisely,
NTh1
Theorem 4.
The following properties are equivalent: (i) G is of type GM;(ii) (cid:101) M is dense in G ;(iii) d ( g ) = sup θ ∈ Ω ∗ φ ( g ; θ ) for all g ∈ G ;(iv) The global reference set R is equal to Ω ∗ = { θ ; (cid:107) U ( θ ) (cid:107) ≤ π } ;(v) For any arclength parametrized, strictly normal geodesic γ ( s ) = exp { s ( ζ, τ ) } , itscut time is equal to h cut ( τ ) := 2 π/ (cid:107) U ( τ ) (cid:107) , with the understanding h cut (0) = + ∞ ;(vi) Cut co ∩ ∂ M = ∅ . Remark 7. (1) The condition (ii) should be the easiest to check among all those.(2) In Section 4, we can find that every step-two group of corank 1 or 2 is of typeGM. As a result, from the above Property (v), we obtain the cut time of any arclengthparametrized, strictly normal geodesic on G ( q, m, U ) with m = 1 or , which coincideswith that in [19, Theorems 6 and 7]. However, considering Remark 1, their results forthe cut time of abnormal geodesics need more explanations. See (ii) of Corollary 9 andRemark 9 below for more details. Furthermore, we emphasize that Property (v) is anequivalent characterization of GM-groups and thus we have found all possible step-twogroups satisfying this fine property. Recall that M is an open set. As a consequence, we obtain the following improvementof [88, Theorem 2.7]: Nc2
Corollary 7.
A step-two group G is of type GM if and only if Cut o = ∂ M . NcMc
Corollary 8.
Assume that G is of M´etivier type. Then G is a GM-group iff. for anyarclength parametrized geodesic γ ( s ) = exp { s ( ζ, τ ) } , its cut time is π/ (cid:107) U ( τ ) (cid:107) . If G is of GM-type, then our Theorem 3 can be improved. Indeed, the parameter θ can be further chosen as a maximum point of the reference function φ ( g ; · ) on Ω ∗ . Inother words, we have nThm1 Theorem 5.
Assume that G is of GM-type and o (cid:54) = g ∈ G . For any shortest geodesic γ g ( s ) ( ≤ s ≤ ) joining o to g , there exist ζ ∈ R q and θ ∈ Ω ∗ such that φ ( g ; θ ) = d ( g ) = sup τ ∈ Ω ∗ φ ( g ; τ ) = | ζ | , γ g ( s ) = exp { s ( ζ, θ ) } , ∀ ≤ s ≤ . Moreover, we have that θ ∈ ∂ Ω ∗ if g ∈ (cid:101) M c . Remark 8.
Suppose further that Ω ∗ is strictly convex, namely, for any τ (cid:54) = τ (cid:48) ∈ Ω ∗ and < s < , we have s τ + (1 − s ) τ (cid:48) ∈ Ω ∗ . For example, all M´etivier groups satisfy thisproperty, see [88, Lemma 9.1]. Then, for any g ∈ (cid:101) M c , the concave function φ ( g ; · ) hasa unique maximum point on ∂ Ω ∗ . This simple observation is very useful to determine allshortest geodesic(s) from o to g ∈ (cid:101) M c . o , aswell as the classical cut locus of o . More precisely, we have the following: NThA1
Corollary 9.
The following properties are equivalent: (i) G is of type GM;(ii) For any arclength parametrized geodesic γ ( s ) = exp { s ( ζ, τ ) } = γ ( ζ, τ ; s ), its cuttime is given by h cut = max (cid:26) π (cid:107) U ( σ ) (cid:107) ; σ ∈ R m , γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ) (cid:27) , with the understanding π = + ∞ ;(iii) Cut CL o = (cid:101) M c . nRKn9 Remark 9.
It follows from (ii) of Corollary 9 that the statement in [19, Theorems 6 and7] is correct if we choose the covector suitably. To be more precise, let G be a step-twogroup of corank 1 or 2, so it is a GM-group from Corollary 13 in Section 4 below. Let γ ( s ) = exp { s ( ζ, τ ) } = γ ( ζ, τ ; s ) be an arclength parametrized geodesic on G . Recall that Π ( ζ,τ ) := { σ − τ ∈ R m ; γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ) } is a linear subspace. Assume further that thecontinuous function σ (cid:55)→ (cid:107) U ( σ ) (cid:107) defined on τ + Π ( ζ,τ ) attains its minimum at τ . Thenthe cut time of γ is given by π/ (cid:107) U ( τ ) (cid:107) . d on (cid:101) M s23 Recall that o ∈ (cid:101) M ⊂ G is the set of points, g , where the reference function φ ( g ; · ) has adegenerate (so infinite) critical point in the initial reference set Ω ∗ . The following theoremis a kind of generalization of the first result in [102, Theorem 1.1], which will be provenby a completely different method. p1 Theorem 6.
Let g = ( x , t ) ∈ (cid:101) M . Then there exist a unit vector ν ∈ R m and aconstant c > such that d ( x , t + h ν ) + d ( x , t − h ν ) − d ( x , t ) ≥ c h, ∀ h > . (2.27) failconc In particular, together with (a) in Remark 3 and (2.8), it gives immediately
Corollary 10. If g = ( x , ∈ Abn ∗ o , there exist a unit vector ν ∗ ∈ R m and a constant c ∗ > such that d ( x , h ν ∗ ) − d ( x , ≥ c ∗ | h | , ∀ h ∈ R . (2.28) failconc0 A direct consequence of (2.27) is the following significantly weaker estimate:lim sup g (cid:48) −→ o d ( g + g (cid:48) ) + d ( g − g (cid:48) ) − d ( g ) | g (cid:48) | = + ∞ , (2.29) nMBn d for any g ∈ (cid:101) M .It follows from Lemma 6 that (cid:101) M ⊆ Abn ∗ o . A very interesting phenomenon is that(2.29) can be no longer valid for g ∈ (cid:101) M \ (cid:101) M even in the setting of GM-groups. Aconcrete example will be provided in Subsection 6.2 below.However, recall that our SC − o is defined by (1.2) instead of as the set of points where(2.29) satisfies. Obviously, SC − o is closed. When the underlying group is of type GM,combining (ii) of Corollary 9 with Theorem 6 obtained above, we can characterize Abn ∗ o via (cid:101) M ; as a byproduct, we answer the open problem [20, (29)] affirmatively: NThA2
Theorem 7.
In the setting of GM-groups, it holds that SC − o = Abn ∗ o = (cid:101) M . Furthermore, combining Theorem 7, Lemmas 4 and 6 with [20, Corollary 30] (withlittle modification in its proof), we answer the open question [20, second part of (30)],when the underlying group is GM-group:
Corollary 11.
In the framework of GM-groups, we have
Cut o = SC + o ∪ SC − o . Similarly, by Corollaries 4 and 6, we have the following result that provides an affir-mative, also partial, answer to the open questions [20, (29)-(30)]:
Corollary 12.
Let G be a step-two group such that (cid:101) M ⊆ { ( x, x ∈ R q } . Then it holdsthat SC − o = Abn ∗ o ( = (cid:101) M ) and Cut o = SC + o ∪ SC − o . Recall that (cid:101) M = { o } if and only if G is of M´etivier type. In the sequel, a step-twogroup G is said to be a SA-group (or of type SA ) if it satisfies { o } (cid:54) = (cid:101) M ⊆ { ( x, x ∈ R q } . ( SA )Notice that star graphs and N , , namely the free step-two Carnot group with threegenerators, are SA-groups. See [88] or § § N , is the simplest example of SA-groupthat is not of type GM. Of course, we can provide an uncountable number of SA but notGM groups.Furthermore, a trivial method to construct SA-groups can be found in Proposition 11of Appendix B. See Appendix C for another method which is much more meaningful. Inparticular, SA-groups of corank 1 are the direct product of a Euclidean space R k with ageneralized Heisenberg group. However, for any m ≥
2, SA-groups with corank m form avery complicated set.To finish this section, we point out the following facts:(1) The second result of [102, Theorem 1.1] says that on the free step-two Carnot groupwith k ( k ≥
4) generators, N k, , for any g = ( x , t ) ∈ Abn ∗ o , there exist a unit vector ν ∗ ∈ R k ( k − / and a constant c ∗ > d ( x , t + h ν ∗ ) − d ( x , t ) ≥ c ∗ | h | , − c ∗ ≤ h ≤ c ∗ . Hence, (2.29) is valid on N k, for any g ∈ Abn ∗ o .172) N k, ( k ≥
4) neither is a GM-group nor satisfies (cid:101) M ⊆ (cid:8) ( x, x ∈ R k (cid:9) . nsP Proof.
We first assume that the shortest geodesic joining o to g is unique. By [88, Corol-lary 2.4], there exist w ∈ R q with | w | = d ( g ) and θ ∈ R such that γ ( w, θ ) is a shortestgeodesic joining o to g . By uniqueness we must have γ g ( s ) = γ ( w, θ ) ( s ) = exp( s ( w, θ ))for all 0 ≤ s ≤
1, which ends the proof in this case.In general, from the characterization of the shortest geodesic in Subsection 2.2.1, thereexist w ∗ ∈ R q with | w ∗ | = d ( g ) and θ ∗ ∈ R m (not necessarily belonging to R ) such that γ g = γ ( w ∗ , θ ∗ ) . We now prove that for each s ∗ ∈ (0 , γ g on the interval[0 , s ∗ ], γ g | [0 , s ∗ ] = γ ( s ∗ w ∗ , s ∗ θ ∗ ) := ( γ g ) ,s ∗ is the unique shortest geodesic joining o to γ g ( s ∗ ) = exp( s ∗ w ∗ , s ∗ θ ∗ ). Notice that ( γ g ) ,s ∗ is a shortest geodesic joining o to γ g ( s ∗ )since γ g itself is shortest. To prove uniqueness, we argue by contradiction. Assume thatthere is another shortest geodesic γ s ∗ (cid:54) = ( γ g ) ,s ∗ joining o to γ g ( s ∗ ) with constant speed s ∗ | w ∗ | . Then we construct a horizontal curve with constant speed | w ∗ | = d ( g ) defined by (cid:101) γ s ∗ ( s ) := (cid:40) γ s ∗ (cid:16) ss ∗ (cid:17) , ≤ s ≤ s ∗ ,γ g ( s ) , s ∗ ≤ s ≤ . Obviously, (cid:101) γ s ∗ is a shortest geodesic joining o to g as well. Again from the characterizationof the shortest geodesic in Subsection 2.2.1, there exist w ∗∗ ∈ R q with | w ∗∗ | = d ( g ) and θ ∗∗ ∈ R m such that (cid:101) γ s ∗ = γ ( w ∗∗ , θ ∗∗ ) .By the fact that γ g ( s ) = (cid:101) γ s ∗ ( s ) when s ∈ [ s ∗ , γ g and (cid:101) γ s ∗ coincide on the whole interval [0 , γ g ( s ) = (cid:101) γ s ∗ ( s ) for all 0 ≤ s ≤ s ∗ ,that is ( γ g ) ,s ∗ = γ s ∗ , which contradicts with our assumption.For each s ∗ ∈ (0 , w ( s ∗ ) ∈ R q with | w ( s ∗ ) | = s ∗ d ( g ) and θ ( s ∗ ) ∈ R such thatexp( s ( w ( s ∗ ) , θ ( s ∗ ))) = ( γ g ) ,s ∗ ( s ) = exp( s ( s ∗ w ∗ , s ∗ θ ∗ )) , ∀ ≤ s ≤ . (3.1) equstar From compactness of R , we extract a sequence { s j } + ∞ j =1 ⊆ (0 ,
1) such that s j → w ( s j ) → w with | w | = d ( g ) and θ ( s j ) → θ ∈ R as j → + ∞ . With s ∗ replaced by s j in(3.1) and letting j → + ∞ , we obtain that exp( s ( w ∗ , θ ∗ )) = γ g ( s ) = exp( s ( w, θ )) forall 0 ≤ s ≤
1, which ends the proof of the theorem.
Proof.
For convenience, we set (cid:101) h = (cid:101) h ( ζ, τ ) := sup { σ ; σ ∈ R m , γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ) } . s ∗ ∈ (0 , h cut ), we know that γ ( s ∗ ζ, s ∗ τ ) is a shortest geodesic. It follows fromTheorem 3 that there exist ζ ( s ∗ ) ∈ R q and θ ( s ∗ ) ∈ R such that γ ( s ∗ ζ, s ∗ τ ) = γ ( ζ ( s ∗ ) , θ ( s ∗ ) ) .Then Lemma 2 implies that s ∗ ζ = ζ ( s ∗ ) and γ ( ζ, σ ( s ∗ ) ; · ) = γ ( ζ, τ ; · ) , with σ ( s ∗ ) := θ ( s ∗ ) s ∗ . Recalling that R is star-shaped w.r.t. 0, by the fact that θ ( s ∗ ) = s ∗ σ ( s ∗ ) ∈ R , we have s ∗ ≤ C σ ( s ∗ ) ≤ (cid:101) h , which implies s ∗ ≤ (cid:101) h . Since s ∗ ∈ (0 , h cut ) is arbitrary, we obtain h cut ≤ (cid:101) h . To prove thesecond inequality, it suffices to observe that for each σ ∈ R m , we have C σ ≤ C R (cid:107) U ( σ ) (cid:107) andthis finishes the proof of Corollary 5. Proof.
For any { g j } + ∞ j =1 ⊆ Abn ∗ o such that g j → g ∈ G as j → + ∞ , our aim is to prove g ∈ Abn ∗ o as well. For each j ∈ N ∗ , let γ j ( s ) (0 ≤ s ≤
1) be an abnormal shortest geodesicjoining o to g j . By Theorem 3, there exist ζ ( j ) ∈ R q with | ζ ( j ) | = d ( g j ) and θ ( j ) ∈ R such that γ j = γ ( ζ ( j ) , θ ( j ) ) . Since γ ( ζ ( j ) , θ ( j ) ) is abnormal, from Proposition 1 there exists a σ ( j ) ∈ S m − such that U ( σ ( j ) ) U ( θ ( j ) ) k ζ ( j ) = 0 , ∀ k ∈ N . (3.2) Abnj
Notice that | ζ ( j ) | = d ( g j ) → d ( g ) as j → + ∞ . From compactness, up to subsequences,we may assume that ζ ( j ) → ζ with | ζ | = d ( g ), θ ( j ) → θ ∈ R and σ ( j ) → σ ∈ S m − as j → + ∞ . Observe that γ ( ζ , θ ) (1) = exp( ζ , θ ) = lim j → + ∞ exp( ζ ( j ) , θ ( j ) ) = lim j → + ∞ g j = g. By the fact that | ζ | = d ( g ), we obtain that γ ( ζ , θ ) is a shortest geodesic joining o to g .It remains to prove that γ ( ζ , θ ) is also abnormal. In fact, letting j → + ∞ in (3.2), weget U ( σ ) U ( θ ) k ζ = 0 , ∀ k ∈ N , which implies γ ( ζ , θ ) is abnormal by Proposition 1.This ends the proof of Lemma 6. Proof.
Set Ξ := { ( x, θ ) ∈ R q × Ω ∗ ; det( − Hess θ (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) ) > } , Ξ := { ( x, θ ) ∈ R q × Ω ∗ ; det( − Hess θ (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) ) = 0 } , κ : R q × Ω ∗ −→ (cid:101) M ( x, θ ) (cid:55)−→ (cid:18) x, − ∇ θ (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) (cid:19) . It follows from Proposition 2 that Ξ ∪ Ξ = R q × Ω ∗ . Recall that (cf. (2.18)-(2.20)) κ (Ξ ) = M and κ (Ξ ) = (cid:101) M .Observe that the function ( x, θ ) (cid:55)→ det( − Hess θ (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) ) is real analyticin R q × Ω ∗ . We claim that Ξ has an empty interior. Otherwise Ξ should be R q × Ω ∗ (cf. [73, § = ∅ and thus M = ∅ . This leads to a contradictionsince M (cid:54) = ∅ from Remark 4. In conclusion, Ξ is dense in R q × Ω ∗ .Now, we shall show that (cid:101) M ⊆ M . Fix ( x, t ) ∈ (cid:101) M . There exists a θ ∈ Ω ∗ such that( x, θ ) ∈ Ξ and κ ( x, θ ) = ( x, t ). Since Ξ is dense in R q × Ω ∗ , there are { ( x ( j ) , θ ( j ) ) } + ∞ j =1 ⊆ Ξ such that ( x ( j ) , θ ( j ) ) → ( x, θ ) as j → + ∞ . Hence, M (cid:51) κ ( x ( j ) , θ ( j ) ) → κ ( x, θ ) = ( x, t )as j → + ∞ . As a result, we obtain that (cid:101) M ⊆ M and thus (cid:101) M = M , which ends the proofof the proposition. Proof. (i) ⇒ (ii): This is evident.(ii) ⇒ (iii): Just use (2.21).(iii) ⇒ (iv): For any given g = ( x, t ) ∈ S , under our assumption, it follows fromProposition 2 that there exists a θ ∈ Ω ∗ such that d ( x, t ) = φ (( x, t ); θ ) = (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) + 4 t · θ . (3.3) tmin Since S is open, there exists a r > { x } × B ( t, r ) = { x } × { τ ; | τ − t | < r } ⊆ S . Then it follows from (iii) that we have for s ∈ B ( t, r ), d ( x, s ) ≥ φ (( x, s ); θ ) = (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) + 4 s · θ . (3.4) sgmin So, the function s (cid:55)−→ d ( x, s ) − s · θ has a local minimum at the point s = t . As aresult, we have ∇ t d ( g ) = θ ∈ Ω ∗ and consequently R ⊆ Ω ∗ . The inverse inclusion isgiven by (2.23) and we obtain (iv).(iv) ⇒ (v): Just combine Corollary 2 with Corollary 5.(v) ⇒ (vi): We argue by contradiction. Assume that there exists a g ∈ Cut co ∩ ∂ M .Since (cid:101) M ⊆ Abn ∗ o ⊆ Cut o , we have g ∈ (cid:101) M \ (cid:101) M . Then from [88, (1) of Remark 2.6] thereexist ζ ∈ R q with | ζ | = d ( g ) and θ ∈ ∂ Ω ∗ such that γ ( ζ, θ ) is a shortest geodesic joining o to g . Since g ∈ Cut co = S , it follows from (2.15) that γ ( ζ, θ ) is strictly normal. As aresult, (v) implies g ∈ Cut CL o ⊆ Cut o and we obtain a contradiction.20vi) ⇒ (i): We argue by contradiction. Assume that M (cid:36) G . Since Cut co is dense in G by [2, Theorem 11.8], we can pick a g ∈ ( G \ M ) ∩ Cut co . From the characterizationof the smooth points (2.15), there exists a unique shortest geodesic γ = γ ( w, θ ) joining o to g , which is not abnormal. We first claim that θ / ∈ Ω ∗ , otherwise θ should be a criticalpoint of φ ( g ; · ) in Ω ∗ by Theorem 2 and thus g ∈ (cid:101) M = M ∪ (cid:101) M . Since g / ∈ M , then g ∈ (cid:101) M ⊆ Cut o , which gives a contradiction and proves this assertion.We further claim that θ / ∈ ∂ Ω ∗ . Otherwise, Lemma 5 and Theorem 2 should implythat for any s ∗ ∈ (0 , s ∗ ( w, θ )) ∈ M . Thus g = exp( w, θ ) ∈ M ,contradicting with our assumption that g ∈ ( G \ M ) ∩ Cut co .As a result, there exists a s ∈ (0 ,
1) such that s θ ∈ ∂ Ω ∗ . Set g = exp( s ( w, θ )) ∈ Cut co = S , where the “ ∈ ” is given by Lemma 5. Similarly, we get that exp( s ∗ ( w, θ )) ∈ M for all s ∗ ∈ (0 , s ) and g ∈ M .Now we are in a position to show that g / ∈ M . We argue by contradiction. Assumethat g ∈ M , then there exists ( w ∗ , θ ∗ ) such that θ ∗ ∈ Ω ∗ and γ ( w ∗ , θ ∗ ) is the uniqueshortest geodesic joining o and g . So γ ( w ∗ , θ ∗ ) coincides with the restriction of γ in [0 , s ],namely γ ,s = γ ( s w, s θ ) . Hence, γ ,s admits two different normal lifts. Consequently γ ,s is also abnormal by definition, which contradicts with, by Lemma 4, the fact that γ is strictly normal.After all, we have that g ∈ ∂ M ∩ Cut co , which leads to a contradiction. Therefore wefinishes the proof. Proof.
By the definition of (cid:101) M (see (2.18)), the second claim in Theorem 5 is a directconsequence of the first one, that needs to be proven. Indeed, from Corollary 7 wehave S = M . By Remark 5, there exist { g j = ( x ( j ) , t ( j ) ) } + ∞ j =1 ⊆ S = M with γ ( ζ ( j ) , θ ( j ) ) ( { θ ( j ) } + ∞ j =1 ⊆ Ω ∗ ) the unique shortest geodesic joining o to g j such that: g j −→ g, ( ζ ( j ) , θ ( j ) ) −→ ( ζ, θ ) as j → + ∞ , γ ( ζ, θ ) = γ g . It remains to prove that φ ( g ; θ ) = d ( g ) when θ ∈ ∂ Ω ∗ .Notice that U ( τ ) is semi-positive definite for every 0 (cid:54) = τ ∈ R m . Let 0 ≤ λ ( τ ) ≤ . . . ≤ λ q ( τ ) ( λ l ( τ ) ≥ , ≤ l ≤ q ) denote its eigenvalues and { P l ( τ ) } ql =1 the correspondingset of pairwise orthogonal projections (that is, ( P k ( τ ))( R q ) ⊥ ( P l ( τ ))( R q ) for k (cid:54) = l ). Thenwe have U ( τ ) = q (cid:88) l =1 λ l ( τ ) P l ( τ ) . (3.5) spec It follows from [77, Chapter two] that for every 1 ≤ l ≤ q , λ l ( τ ) is a continuous functionof τ (cid:54) = 0 and homogeneous of degree 1, namely λ l ( s τ ) = s λ l ( τ ) for s >
0. However, P l ( τ )21s not necessarily continuous, but it can be chosen to be symmetric and homogeneous ofdegree 0, namely P l ( r τ ) = P l ( τ ) ∀ r (cid:54) = 0 , ≤ l ≤ q. (3.6) homoP For the θ ∈ ∂ Ω ∗ obtained before, there exists an L ∈ { , . . . , q } such that λ L − ( θ ) < π when L >
1, and λ L ( θ ) = . . . = λ q ( θ ) = π . From the continuity of { λ l ( τ ) } ql =1 , there exist δ ∈ (cid:16) , π (cid:17) and r ∈ (cid:16) , | θ | (cid:17) such that for τ ∈ B ( θ, r ) = { τ ; | τ − θ | < r } , we have λ L − ( τ ) ≤ π − δ when L >
1, and π − δ ≤ λ L ( τ ) ≤ . . . ≤ λ q ( τ ) ≤ π + δ. For τ ∈ B ( θ, r ), let us set V ( τ ) := πi (cid:82) Γ z ( z − U ( τ ) ) − dz = L − (cid:80) l =1 λ l ( τ ) P l ( τ ) when L >
10 when L = 1 , (3.7)and the projection on (near π )-eigenspaces of U ( τ ) Q ( τ ) := 12 πi (cid:90) Γ ( z − U ( τ ) ) − dz = q (cid:88) l = L P l ( τ ) , (3.8)where the contours Γ , Γ ⊆ C are defined byΓ := { z ; dist( z, [0 , π − δ ]) = δ } and Γ := { z ; dist( z, [ π − δ, π + δ ]) = δ } respectively, with the counterclockwise orientation. From the integral representation, itis easy to see that the operator functions V ( τ ) and Q ( τ ) are continuous in B ( θ, r ). Itdeduces from Theorem 2 that q (cid:88) l =1 (cid:18) λ l ( θ ( j ) )sin λ l ( θ ( j ) ) (cid:19) | P l ( θ ( j ) ) x ( j ) | = (cid:12)(cid:12)(cid:12)(cid:12) U ( θ ( j ) )sin U ( θ ( j ) ) x ( j ) (cid:12)(cid:12)(cid:12)(cid:12) = | ζ ( j ) | → | ζ | , as j → + ∞ . As a result, since | Q ( θ ) x | = lim j → + ∞ | Q ( θ ( j ) ) x ( j ) | = lim j → + ∞ q (cid:88) l = L | P l ( θ ( j ) ) x ( j ) | = lim j → + ∞ q (cid:88) l = L (cid:18) λ l ( θ ( j ) )sin λ l ( θ ( j ) ) (cid:19) − (cid:18) λ l ( θ ( j ) )sin λ l ( θ ( j ) ) (cid:19) | P l ( θ ( j ) ) x ( j ) | , and sin s ∼ ( π − s ) for s near π , we get immediately | Q ( θ ) x | = 0 . (3.9) Px (cid:28)(cid:113) V ( θ ( j ) ) cot (cid:113) V ( θ ( j ) ) x ( j ) , x ( j ) (cid:29) = (cid:104) U ( θ ( j ) ) cot U ( θ ( j ) ) x ( j ) , x ( j ) (cid:105) + | Q ( θ ( j ) ) x ( j ) | − q (cid:88) l = L (cid:0) λ l ( θ ( j ) ) cot λ l ( θ ( j ) ) (cid:1) | P l ( θ ( j ) ) x ( j ) | , we yield that (cid:68)(cid:112) V ( θ ) cot (cid:112) V ( θ ) x, x (cid:69) = lim j → + ∞ (cid:28)(cid:113) V ( θ ( j ) ) cot (cid:113) V ( θ ( j ) ) x ( j ) , x ( j ) (cid:29) = lim j → + ∞ (cid:104) U ( θ ( j ) ) cot U ( θ ( j ) ) x ( j ) , x ( j ) (cid:105) . (3.10) vcotv Hence, it follows from Theorem 2 that d ( g ) = lim j → + ∞ d ( g j ) = lim j → + ∞ φ ( g j ; θ ( j ) )= lim j → + ∞ ( (cid:104) U ( θ ( j ) ) cot U ( θ ( j ) ) x ( j ) , x ( j ) (cid:105) + 4 t ( j ) · θ ( j ) )= (cid:68)(cid:112) V ( θ ) cot (cid:112) V ( θ ) x, x (cid:69) + 4 t · θ. Combining this with θ ∈ ∂ Ω ∗ and the fact that the orthogonal projection of x on π -eigenspace of U ( θ ) is zero (cf. (3.9)), it follows from [88, Remark 2.1] that d ( g ) = (cid:104) U ( θ ) cot U ( θ ) x, x (cid:105) + 4 t · θ = φ (( x, t ); θ ) . (3.11) expphibd This ends the proof of Theorem 5.
Proof. (i) ⇒ (ii): Just combine Corollary 2 with Corollary 5 (cf. also Remark 6).(ii) ⇒ (i): It is trivial because Theorem 4 (v) satisfies under our assumption.(iii) ⇒ (i): It follows from Theorem 1 and [118, Proposition 15] that (cid:101) M c = Cut CL o ⊆ Cut o is a set of measure zero, which means (cid:101) M is dense in G . This is exactly (ii) of Theorem4 and we obtain that G is of type GM.(i) + (ii) ⇒ (iii): From Corollary 3 it suffices to prove (cid:101) M c ⊆ Cut CL o . Fix g ∈ (cid:101) M c . It isclear that g (cid:54) = o . Theorem 5 guarantees that there exist ζ ∈ R q \ { } and θ ∈ ∂ Ω ∗ suchthat γ ∗ = γ ( ζ, θ ) is a shortest geodesic joining o to g . Consider the arclength parametrizedgeodesic (cid:101) γ := γ ( (cid:98) ζ, τ ; · ), where τ := θ | ζ | . Here and in the sequel, we adopt the convention (cid:98) u := (cid:40) u | u | , if u ∈ R (cid:96) \ { } , , if u = 0 . (3.12) defhat To prove that g ∈ Cut CL o , it remains to show that the cut time of (cid:101) γ , h cut ( (cid:101) γ ), equals | ζ | .First, notice that (cid:101) γ | [0 , | ζ | ] = γ ( ζ, θ ) = γ ∗ is a shortest geodesic. Hence we get h cut ( (cid:101) γ ) ≥ | ζ | .23n the other hand, for any σ ∈ R m such that γ ( (cid:98) ζ, σ ; · ) = γ ( (cid:98) ζ, τ ; · ), we have γ ∗ = γ ( ζ, | ζ | σ ) . Since g ∈ (cid:101) M c , it follows from Theorem 2 that (cid:13)(cid:13)(cid:13)(cid:13) U (cid:18) | ζ | σ (cid:19)(cid:13)(cid:13)(cid:13)(cid:13) ≥ π, or equivalently π (cid:107) U ( σ ) (cid:107) ≤ | ζ | . Then from (ii), we obtain that h cut ( (cid:101) γ ) ≤ | ζ | .Therefore, we finish the proof of Corollary 9. Proof.
Let g = ( x , t ) ∈ (cid:101) M . Then (2.21) and (2.20) imply that there exists a θ ∈ Ω ∗ such that d ( g ) = φ ( g ; θ ). Since Ω ∗ is open, there exists a r > B ( θ , r ) = { τ ∈ R m ; | τ − θ | < r } ⊆ Ω ∗ . From [88, Proposition 5.1 (c)] there exists a unit vector ν such that t · ν = 0 , φ ( g ; θ + s ν ) = φ ( g ; θ ) , ∀ s ∈ R with θ + s ν ∈ Ω ∗ . Consequently, for any h >
0, using [88, Theorem 2.1], we have that d ( x , t + h ν ) ≥ φ (( x , t + h ν ); θ + r ν ) = φ ( g ; θ ) + 4 h r + 4 h ν · θ ,d ( x , t − h ν ) ≥ φ (( x , t − h ν ); θ − r ν ) = φ ( g ; θ ) + 4 h r − h ν · θ . As a result, we obtain d ( x , t + h ν ) + d ( x , t − h ν ) − d ( g ) ≥ r h, ∀ h > , which finishes the proof of this theorem. Proof.
First, for any arclength parametrized abnormal geodesic γ ( s ) = exp { s ( ζ, τ ) } = γ ( ζ, τ ; s ) with cut time h cut and s ∗ < h cut , we claim that γ ( s ∗ ) = γ ( ζ, τ ; s ∗ ) ∈ (cid:101) M . Infact, from (ii) of Corollary 9, there exists a σ ∈ R m such that s ∗ < π (cid:107) U ( σ ) (cid:107) and γ ( ζ, σ ; · ) = γ ( ζ, τ ; · ). Then it follows from the first equation of Proposition 3 that γ ( s ∗ ) = γ ( ζ, τ ; s ∗ ) = γ ( ζ, σ ; s ∗ ) ∈ (cid:101) M . Thus, the second equation in Proposition 3 implies that Abn ∗ o ⊆ (cid:101) M .Next, from definition the set SC − o is closed. Moreover, Theorem 6 implies that (cid:101) M ⊆ SC − o . In conclusion, we have that Abn ∗ o ⊆ (cid:101) M ⊆ SC − o . Finally, we recall that Abn ∗ o is closed (cf. Lemma 6), so the inclusion SC − o ⊆ Abn ∗ o canbe deduced from [35, Theorem 1], which ends the proof of Theorem 7.24 Step-two groups of Corank are GM-groups s3 The purpose of this section is twofold. On one hand, we provide a sufficient condition for M = G by means of semi-algebraic theory. As a byproduct, we show that all G ( q, , U ) areof type GM. On the other hand, we prove that there exists a M´etivier group G (4 N, , U N ),which is not of type GM, for any N ∈ N ∗ .Let us begin by recalling (cf. [31, Chapter 2]): A set A ⊆ R q is semi-algebraic if it is the result of a finite number of unions and inter-sections of sets of the form { f = 0 } , { g > } , where f, g are polynomials on R q . If A is asemi-algebraic set, then its complement, boundary and any Cartesian projection of A aresemi-algebraic sets. If A and B are semi-algebraic sets, then so does A × B . Furthermore,any semi-algebraic set A ⊆ R q is the disjoint union of a finite number of semi-algebraic sets M i in R q where each M i is a smooth submanifold in R q and diffeomorphic to (0 , dim M i .If A ⊆ R q and B ⊆ R r are two semi-algebraic sets. A mapping h : A → B is semi-algebraic if its graph is a semi-algebraic set in R q + r . If S ⊆ A is a semi-algebraic set and h : A → B is a semi-algebraic mapping, then h ( S ) is a semi-algebraic set in R r .Let A ⊆ R q be a semi-algebraic set. Its dimension, dim A , can be defined in somealgebraic way. The basic properties that we will use later are: (1) If A is the finiteunion of semi-algebraic sets A , . . . , A p , then dim A = max ≤ i ≤ p dim A i . (2) If A and B aresemi-algebraic sets, then dim( A × B ) = dim A + dim B . (3) If h : A → B is a semi-algebraic mapping, then dim h ( A ) ≤ dim A . (4) Moreover, if A is a semi-algebraic setas well as a smooth submanifold in R q , its dimension as a semi-algebraic set coincideswith its dimension as a smooth manifold. As a result, if A ⊆ R q is a semi-algebraic setwith dim A < q , then it has measure 0 in R q by the usual Morse-Sard-Federer Theorem(cf. [79, p. 72]). M = G Recall that Ω ∗ is defined by (2.16) and U ( θ ) ( θ ∈ R m ) by (2.5). For θ (cid:54) = 0, let M ( θ )denote the multiplicity of the maximal eigenvalue of U ( θ ) , and M := min θ (cid:54) =0 M ( θ ) . (4.1)We have the following: t3 Theorem 8. If M ≥ m , then M = G .Proof. First observe that the open set Ω ∗ is a semi-algebraic set in R m by the fact thatΩ c ∗ = π (cid:0) { ( x, τ ); (cid:104) U ( τ ) x, x (cid:105) ≥ π } ∩ { ( x, τ ); | x | = 1 } (cid:1) , π denotes the projection from R q × R m to the second entry R m . Then, ∂ Ω ∗ is asemi-algebraic set and dim ( ∂ Ω ∗ )(= dim (Ω ∗ \ Ω ∗ )) ≤ m − { ( U ( θ ) − π ) y ; θ ∈ ∂ Ω ∗ , y ∈ R q } ⊆ R q , that is, the set of points x such that there exists a θ ∈ ∂ Ω ∗ satisfying that the orthogonalprojection of x on π -eigenspace of U ( θ ) is zero. From [88, Proposition 2.2], it remainsto prove that Σ has measure 0.Now, consider the map defined byΨ : R m × R q −→ R q ( θ, y ) (cid:55)−→ Ψ( θ, y ) := ( U ( θ ) − π ) y. Notice that it is a semi-algebraic mapping. Then Σ = Ψ( ∂ Ω ∗ × R q ) is a semi-algebraicset. It suffices to prove that dim Σ ≤ q − r ∈ N satisfying r ≤ q , setΠ q,r := { L ; there exist 1 ≤ j < . . . < j r ≤ q such that L = span { e j , . . . , e j r }} , where { e , . . . , e q } denotes the standard orthonormal basis of R q and we adopt the con-vention that span {∅} = { } . Then Π q,r is a finite set of C rq elements. Remark that for a q × q real matrix S with rank( S ) ≤ r , there exists an L ∈ Π q,r such that S ( R q ) = S ( L ).Under our assumption, we have rank( U ( θ ) − π ) ≤ q − m for any θ ∈ ∂ Ω ∗ . Hence, weget Σ = Ψ( ∂ Ω ∗ × R q ) = ∪ L ∈ Π q,q − m Ψ( ∂ Ω ∗ × L ) . It is clear that for each L ∈ Π q,q − m , we havedim( ∂ Ω ∗ × L ) = dim( ∂ Ω ∗ ) + dim L ≤ ( m −
1) + ( q − m ) = q − , and as a result,dim Σ = max L ∈ Π q,q − m dim Ψ( ∂ Ω ∗ × L ) ≤ max L ∈ Π q,q − m dim( ∂ Ω ∗ × L ) ≤ q − , which ends the proof of this theorem.By the fact that (cid:101) U ( θ ) is skew-symmetric for any θ , we have M ≥
2. Then c2 Corollary 13. If G = G ( q, , U ) or G ( q, , U ) , namely G is a step-two group of Corank or , then it is a GM-group. r1 Remark 10. (1) Theorem 8 is sharp in the sense that G may not be of type GM if M < m . The simplest example is the free Carnot group of step two and generatorsstudied in [88]. Notice that in such case, we have M = 2 < m . Other interestingexamples can be found in Subsection 4.3 below.(2) Remark also that M ≥ m is in general not necessary for M = G . See for examplethe star graphs studied in [88].(3) We do not know whether there exists a simple algebraic characterization for GM-groups similar to that of M´etivier groups. .3 Not all M´etivier groups are of type GM SS43
In the sequel, we will illustrate that when m = 3, the condition that q (instead of M ) issufficiently larger than m cannot guarantee M = G , even in the case that G is a M´etiviergroup. p4 Proposition 5.
For any N ∈ N , there exists a M´etivier group G = G (4 N + 4 , , U N ) which is not a GM-group.Proof. Let H (4 n,
3) = G (4 n, , U H (4 n, ) ( n ∈ N ∗ ) denote the (4 n + 3)-dimensional H-typegroup, that is U H (4 n, satisfies the following condition (cf. (2.5) for the related definition): U H (4 n, ( λ ) U H (4 n, ( λ (cid:48) ) + U H (4 n, ( λ (cid:48) ) U H (4 n, ( λ ) = 2 λ · λ (cid:48) I n , ∀ λ, λ (cid:48) ∈ R . We remark that the (4 n + 3)-dimensional quaternionic Heisenberg group provides a goodexample for it.For a row vector τ = ( τ , τ , τ ) ∈ R , set X ( τ ) := i − τ − τ − τ − − τ − τ τ − − τ τ − τ − − τ − τ τ ,U ( τ ) := X ( τ ) and for N ∈ N ∗ , U N ( τ ) := (cid:18) U H (4 N, (cid:0) τ (cid:1) O (4 N ) × O × (4 N ) X ( τ ) (cid:19) . Observe that U N ( τ ) = (cid:18) | τ | I N +1 | τ | I − τ T τ (cid:19) , (4.2) nEe1 whose eigenvalues are | τ | with the multiplicity 2 and | τ | with the multiplicity 4 N + 2.Hence, we get a M´etivier group, saying that G = G (4 N + 4 , , U N ).We only consider the case N ∈ N ∗ here and the proof is similar when N = 0.From now on, we fix N ∈ N ∗ , and write x = ( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) ∈ R N × R × R = R N +4 . In our situation, by (4.2), a direct calculation shows that the initial reference set,defined by (2.16), is given by Ω ∗ = { τ ; | τ | < π } , and the reference function (cf. (2.17)) is φ (( x, t ); τ ) = 4 t · τ + (cid:18) | τ | | τ | (cid:19) (cid:0) | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ · (cid:98) τ | (cid:1) + ( | τ | cot | τ | ) | (cid:101) x ∗ − ( (cid:101) x ∗ · (cid:98) τ ) (cid:98) τ | . φ ((( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) , t ); τ ) = φ ((( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) , − t ); − τ ) , (4.3) symM1 φ ((( (cid:101) x ∗ , (cid:101) x , O (cid:101) x ∗ ) , O t ); O τ ) = φ ((( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) , t ); τ ) , ∀ O ∈ O , (4.4) symM2 where O denotes the 3 × (cid:101) x ∗ = | (cid:101) x ∗ | e = | (cid:101) x ∗ | (1 , , , t = ( t , t ,
0) with t , t ≥
0. (4.5) nIc
Now, by recalling that (see (2.25)) ψ ( s ) := 1 − s cot ss , µ ( s ) := − ( s cot s ) (cid:48) = 2 s − sin (2 s )2 sin s , we can write φ (( x, t ); τ ) = 4 t · τ + | (cid:101) x ∗ | + (cid:18) | τ | | τ | (cid:19) ( | (cid:101) x ∗ | + (cid:101) x ) − ψ (cid:18) | τ | (cid:19) τ | (cid:101) x ∗ | − ψ ( | τ | )( τ + τ ) | (cid:101) x ∗ | . Suppose that θ ∈ Ω ∗ is a critical point of φ (( x, t ); · ) for some ( x, t ) satisfying (4.5).Then we have4 t = µ (cid:18) | θ | (cid:19) | (cid:101) x ∗ | + (cid:101) x | θ | θ + ψ (cid:48) (cid:18) | θ | (cid:19) θ | (cid:101) x ∗ | | θ | θ + ψ (cid:18) | θ | (cid:19) | (cid:101) x ∗ | θ e + ψ (cid:48) ( | θ | ) ( θ + θ ) | (cid:101) x ∗ | | θ | θ + 2 ψ ( | θ | ) | (cid:101) x ∗ | ( θ e + θ e ) . We further assume that | (cid:101) x ∗ | + (cid:101) x (cid:54) = 0 and (cid:101) x ∗ (cid:54) = 0. Using [88, Lemmas 3.1 and 3.2],the fact that t = 0 implies θ = 0. Thus4 (cid:18) t t (cid:19) = µ (cid:18) | θ | (cid:19) | (cid:101) x ∗ | + (cid:101) x | θ | (cid:18) θ θ (cid:19) + ψ (cid:48) (cid:18) | θ | (cid:19) θ | (cid:101) x ∗ | | θ | (cid:18) θ θ (cid:19) + ψ (cid:18) | θ | (cid:19) | (cid:101) x ∗ | (cid:18) θ (cid:19) + ψ (cid:48) ( | θ | ) θ | (cid:101) x ∗ | | θ | (cid:18) θ θ (cid:19) + 2 ψ ( | θ | ) | (cid:101) x ∗ | (cid:18) θ (cid:19) := Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); ( θ , θ )) . Next, following the proof of [88, Proposition 10.3], we can establish the followinglemma. For completeness, we include its proof in “Appendix A”.
LMng
Lemma 7.
Suppose that | (cid:101) x ∗ | + (cid:101) x (cid:54) = 0 and (cid:101) x ∗ (cid:54) = 0 . Let B R (0 , π ) := (cid:26) ( v , v ) ∈ R ; (cid:113) v + v < π (cid:27) , R r ( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) := (cid:26) ( u , u ) ∈ R ; | u | < π (cid:18) u | (cid:101) x ∗ | + | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ | (cid:19)(cid:27) . Then
Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); · ) is a C ∞ -diffeomorphism from B R (0 , π ) onto R r ( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) . G \ (cid:101) M contains the subset (cid:40) ( x, t ); (cid:101) x ∗ (cid:54) = 0 , | (cid:101) x ∗ | + (cid:101) x (cid:54) = 0 , | t · (cid:101) x ∗ | π | (cid:101) x ∗ | > | (cid:101) x ∗ | (cid:12)(cid:12)(cid:12)(cid:12) t − t · (cid:101) x ∗ | (cid:101) x ∗ | (cid:101) x ∗ | (cid:101) x ∗ | (cid:12)(cid:12)(cid:12)(cid:12) + | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ | (cid:41) , which implies immediately M ⊆ (cid:101) M (cid:36) G . s6 Let p , p ∈ { , , , . . . } with p ≥ p . Consider a p × p full-rank real matrix B = (cid:101) b T1 ... (cid:101) b T p with column vectors (cid:101) b j ∈ R p \ { } , ≤ j ≤ p . A step-two Kolmogorov type group (or K-type group, in short) of type B , G (K) B , is definedby G (1 + p , p , U (K) B ) with (see [32, § U (K) , ( j ) B = (cid:32) (cid:101) b T j − (cid:101) b j O p × p (cid:33) , ≤ j ≤ p . Notice that we have for τ ∈ R p U (K) B ( τ ) = i (cid:18) τ T B − B T τ O p × p (cid:19) , U (K) B ( τ ) = (cid:18) | B T τ | O × p O p × B T τ τ T B (cid:19) . (5.1) Ku2
Hence the initial reference set, defined by (2.16), isΩ ∗ := Ω (K) B = { τ ∈ R p ; | B T τ | < π } . In the rest of this section, we write x = ( x , x ∗ ) ∈ R × R p = R p . A simple calculation shows that the reference function in this setting is given by φ (( x, t ); τ ) := φ (K) B (( x, t ); τ ) = | x | + 4 t · τ − (cid:2) x f ( | B T τ | ) + | τ · B x ∗ | ψ ( | B T τ | ) (cid:3) . Remark that the case B = I n corresponds to the star graph K ,n , on which the squaredsub-Riemannian distance and the cut locus have been characterized by [88, Theorem 10.1].We will use this known result to deduce the counterpart for general G (K) B .29otice that BB T is positive definite, then we can define an isomorphism:T B : R p −→ R p , T B ( τ ) := (cid:0) BB T (cid:1) τ. (5.2) TBn
Let T − B denote its inverse. Set T := T ( t ) = T − B ( t ) , t ∈ R p ; X ∗ := X ∗ ( x ∗ ) = T − B ( B x ∗ ) , x ∗ ∈ R p , (5.3) TX* and X ∗∗ := X ∗∗ ( x ∗ ) = x ∗ − B T (cid:0) T − B ( X ∗ ) (cid:1) = x ∗ − B T ( BB T ) − B x ∗ , x ∗ ∈ R p . (5.4)Observe that B T T − B is an isometry on R p , then we have τ · B x ∗ = (cid:104) B T T − B τ, B T T − B B x ∗ (cid:105) = ( BB T T − B τ ) · (T − B B x ∗ ) = (T B τ ) · X ∗ , (5.5) nIso and similarly t · τ = T · (T B τ ). Moreover, by the fact that B T ( BB T ) − B is a projectionon R p and | T B ( τ ) | = | B T τ | for any τ ∈ R p , we find that φ (K) B ((( x , x ∗ ) , t ); τ ) = φ (K) I p ((( x , X ∗ ) , T ); T B ( τ )) + | X ∗∗ | . Thus, the following result is a direct consequence of [88, Theorem 10.1]: t5 Theorem 9. (1) We have d ( g ) = sup τ ∈ Ω (K) B φ (K) B ( g ; τ ) , ∀ g ∈ G (K) B . (2) The cut locus of o , Cut o , is exactly M c = (cid:40) ((0 , x ∗ ) , t ); | X ∗ · T | ≤ | X ∗ | √ π (cid:114)(cid:12)(cid:12)(cid:12) T − ( (cid:99) X ∗ · T ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12)(cid:41) , where we have used the convention (3.12) . And for ((0 , x ∗ ) , t ) ∈ M c , it holds that d ((0 , x ∗ ) , t ) = | x ∗ | + 4 π (cid:12)(cid:12)(cid:12) T − ( (cid:99) X ∗ · T ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12) . (3) If ( x, t ) = (( x , x ∗ ) , t ) ∈ M , then there exists a unique θ = θ ( x, t ) ∈ Ω (K) B such that t = 14 ∇ θ (cid:2) x f ( | B T θ | ) + | θ · B x ∗ | ψ ( | B T θ | ) (cid:3) . Furthermore, we have d ( x, t ) = φ (K) B (( x, t ); θ ) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) U (K) B ( θ )sin U (K) B ( θ ) x (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:18) | B T θ | sin | B T θ | (cid:19) x + | x ∗ | + (cid:34)(cid:18) | B T θ | sin | B T θ | (cid:19) − (cid:35) | θ · B x ∗ | | B T θ | . Remark 11.
1. Using [88, Corollary 10.1] and (iii) of Corollary 9, we find that (cid:101) M = { ((0 , x ∗ ) , x ∗ ∈ R p } = Abn ∗ o = Abn o , Cut CL o = M c \ (cid:101) M . G (K) B is of type GM, then other properties in Theorem 4, Corollaries 7 and 9 arealso valid. To end this section, we describe all o to any given g (cid:54) = o , as well as“bad” normal geodesics Let us begin with the γ ( w, θ ; s ) on K-type groups Let x = ( x , x ∗ ), (cid:101) x = ( (cid:101) x , (cid:101) x ∗ ) ∈ R × R p = R p and τ ∈ R p . A simple calculation givesthat (cid:104) U (K) B x, (cid:101) x (cid:105) · τ = (cid:104) (cid:101) U (K) B ( τ ) x, (cid:101) x (cid:105) = ( (cid:101) x B x ∗ − x B (cid:101) x ∗ ) · τ, ∀ τ ∈ R p , (5.6) skew which implies (cid:104) U (K) B x, (cid:101) x (cid:105) = (cid:101) x B x ∗ − x B (cid:101) x ∗ := x (cid:63) (cid:101) x. (5.7) skew2 Next we consider exp { (cid:101) U (K) B ( τ ) } ζ (0) with ζ (0) ∈ R p and τ ∈ R p . It can be treatedvia the Spectral Theorem or directly by the fact thatexp { (cid:101) U (K) B ( τ ) } = exp {− i U (K) B ( τ ) } = cos U (K) B ( τ ) + (cid:101) U (K) B ( τ ) sin U (K) B ( τ ) U (K) B ( τ ) , which is, by (5.1), equal to (cid:32) cos ( | B T τ | ) I p + cos ( | B T τ | ) − | B T τ | B T τ τ T B (cid:33) + sin ( | B T τ | ) | B T τ | (cid:18) τ T B − B T τ O p × p (cid:19) . (5.8)Now, we describe γ ( w, θ ; s ) := ( x ( s ) , t ( s )) for given w := ( w , w ∗ ) ∈ R p and θ ∈ R p . Let us introduce the column vectorsv := (cid:32) θ · B w ∗ − w B T θ (cid:33) , v := (cid:32) w θ · B w ∗ | B T θ | B T θ (cid:33) , v := (cid:32) w ∗ − θ · B w ∗ | B T θ | B T θ (cid:33) , (5.9) n1v w := v (cid:63) v = − (cid:18) w + ( θ · B w ∗ ) | B T θ | (cid:19) BB T θ, (5.10) w1 w := v (cid:63) v = − ( θ · B w ∗ ) B w ∗ + ( θ · B w ∗ ) | B T θ | BB T θ, (5.11)w := v (cid:63) v = − w B (cid:18) w ∗ − θ · B w ∗ | B T θ | B T θ (cid:19) . (5.12) w3
31t follows from (2.6) that ζ ( s ) = sin(2 s | B T θ | ) | B T θ | v + cos(2 s | B T θ | ) v + v , (5.13) expzeta x ( s ) = 1 − cos(2 s | B T θ | )2 | B T θ | v + sin(2 s | B T θ | )2 | B T θ | v + s v . (5.14) expx The calculation of t ( s ) is cumbersome. However, (5.7)-(5.12), (2.6) together with (5.13)and (5.14) imply that˙ t ( s ) = 12 x ( s ) (cid:63) ζ ( s )= cos(2 s | B T θ | ) − | B T θ | w + 12 (cid:18) − cos(2 s | B T θ | )2 | B T θ | − s sin(2 s | B T θ | ) | B T θ | (cid:19) w + 12 (cid:18) sin(2 s | B T θ | )2 | B T θ | − s cos(2 s | B T θ | ) (cid:19) w . (5.15) expt Noticing that (cid:90) s r sin(2 r | B T θ | ) dr = − s | B T θ | cos(2 s | B T θ | ) + sin(2 s | B T θ | )4 | B T θ | , (5.16) n*n1 (cid:90) s r cos(2 r | B T θ | ) dr = 2 s | B T θ | sin(2 s | B T θ | ) − s | B T θ | )4 | B T θ | , (5.17) n*n2 from (5.15), we get that t ( s ) = sin(2 s | B T θ | ) − s | B T θ | | B T θ | w + s | B T θ | + s | B T θ | cos(2 s | B T θ | ) − sin(2 s | B T θ | )4 | B T θ | w + − s | B T θ | sin(2 s | B T θ | ) + 1 − cos(2 s | B T θ | )4 | B T θ | w . (5.18) exptp In particular, we can get some information about the set of the endpoints of nontrivial“bad” normal geodesics, as well as ss512
We suppose in this subsection that w = ( w , w ∗ ) (cid:54) = 0 , and | B T θ | = kπ with k ∈ N ∗ . Recall that the isomorphism T B is defined by (5.2) and | T B ( τ ) | = | B T τ | for any τ ∈ R p .Set W ∗ := T − B ( B w ∗ ) , (cid:98) η := T B ( θ ) | T B ( θ ) | = T B ( θ ) | B T θ | = T B ( θ ) k π . (5.19) nWn θ · B w ∗ | B T θ | = T B ( θ ) · W ∗ | B T θ | = (cid:98) η · W ∗ := (cid:101) w . (5.20) nX Let ( x, t ) = (( x , x ∗ ) , t ) := exp( w, θ ) = γ ( w, θ ) (1) , ( x ( s ) , t ( s )) := γ ( w, θ ) ( s ) . Taking s = 1 and | B T θ | = k π in (5.14), we deduce that x = (cid:18) , w ∗ − θ · B w ∗ | B T θ | B T θ (cid:19) , (5.21) nxz namely x = 0 and x ∗ = w ∗ − ( (cid:98) η · W ∗ ) B T θ | B T θ | , (5.22) xstar where we have used (5.20). Similarly, (5.18) implies that t = − | B T θ | w + 12 | B T θ | w (5.23) ntz = − θ · B w ∗ | B T θ | B w ∗ + | B T θ | w + 3 ( θ · B w ∗ ) | B T θ | BB T θ. In other words, by (5.20), we have t = − (cid:98) η · W ∗ | B T θ | B w ∗ + w + 3 ( (cid:98) η · W ∗ ) | B T θ | BB T θ. (5.24) n1t Recall that T B := ( BB T ) , X ∗ := T − B ( B x ∗ ) and T := T − B ( t ). Applying T − B B toboth sides of (5.22), it follows from (5.19) and (5.20) that X ∗ = W ∗ − ( (cid:98) η · W ∗ ) (cid:98) η = W ∗ − (cid:101) w (cid:98) η. (5.25) Xstar
And similarly, applying T − B to both sides of (5.24), we obtain T = − (cid:101) w k π W ∗ + w + 3 (cid:101) w k π (cid:98) η. (5.26) TWstar
We split it into cases.
Case 1. t = 0 so T = 0 . In such case, taking inner product with (cid:98) η on both sides of(5.26) and using (5.20), we yield w = (cid:101) w = 0. So θ · B w ∗ = 0, and w ∗ = x ∗ because of(5.21). Hence, the vectors defined by (5.9)-(5.12) in this situation are w = w = w = 0,v = v = 0 and v = x respectively. In conclusion, by (5.14) and (5.18), a simplecalculation shows that the “bad” normal geodesic from o to ((0 , x ∗ ) ,
0) is the the straightsegment γ ((0 ,x ∗ ) , ( s ).From now on, we further assume that: 33 ase 2. t (cid:54) = 0 so T (cid:54) = 0 . In such case, we have w + (cid:101) w >
0. Taking inner productwith (cid:98) η on both sides of (5.25), by (5.20), we get X ∗ · (cid:98) η = 0. Inserting (5.25) into (5.26),we obtain that T = − (cid:101) w k π X ∗ + w + (cid:101) w k π (cid:98) η. (5.27) TXstar
And we will consider the cases X ∗ (cid:54) = 0 and X ∗ = 0. (I) Assume that X ∗ (cid:54) = 0 . In such case, X ∗ and the unit vector (cid:98) η = T B θk π are orthogonal.It is easy to solve out ( w , (cid:101) w ) as well as θ from (5.27). Using (5.19), a direct calculationshows that (cid:101) w = − k π T · (cid:99) X ∗ | X ∗ | , θ = k π T − B (cid:32) T − ( T · (cid:99) X ∗ ) (cid:99) X ∗ | T − ( T · (cid:99) X ∗ ) (cid:99) X ∗ | (cid:33) , (5.28) nP1n w = ± (cid:118)(cid:117)(cid:117)(cid:116) k π (cid:12)(cid:12)(cid:12) T − ( T · (cid:99) X ∗ ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12) − k π (cid:32) T · (cid:99) X ∗ | X ∗ | (cid:33) . (II) Assume that X ∗ = 0 , that is B x ∗ = 0 . In such case, by (5.19), (5.27) implies that θ = k π T − B (cid:16) (cid:98) T (cid:17) , ( w , (cid:101) w ) = (cid:112) k π | T | (cos σ, sin σ ) ( σ ∈ R ) . (5.29) nP2n In conclusion, we have always X ∗ · (cid:98) η = 0 . (5.30) o1n1 And ( x, t ) = ((0 , x ∗ ) , t ) ( t (cid:54) = 0) is the endpoint of some “bad” normal geodesic γ (( w ,w ∗ ) , θ ) satisfying | B T θ | = k π ( k ∈ N ∗ ) if and only if | X ∗ · T | ≤ | X ∗ | √ kπ (cid:114)(cid:12)(cid:12)(cid:12) T − ( (cid:99) X ∗ · T ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12) . (5.31)In such case, taking ( w , (cid:101) w , θ ) as in (5.28) for X ∗ (cid:54) = 0, or in (5.29) for X ∗ = 0, itfollows from (5.22) and (5.20) that w ∗ = x ∗ − T · (cid:99) X ∗ | X ∗ | B T θ = x ∗ + (cid:101) w k π B T θ. (5.32) w*n Now recall that (see (5.21)) x ∗ = w ∗ − θ · B w ∗ | B T θ | B T θ . Substituting this as well as (5.20) and | B T θ | = kπ in (5.9)-(5.12), we get thatv = (cid:18) x ∗ (cid:19) , v = (cid:18) w (cid:101) w k π B T θ (cid:19) , v = k π (cid:18) (cid:101) w − w k π B T θ (cid:19) , w = − ( θ · B w ∗ ) B x ∗ = − k π (cid:101) w B x ∗ , w = − w B x ∗ , = 2 w − | B T θ | t = − k π ( (cid:101) w B x ∗ + 2 k π t ) . Substituting them into (5.14) and (5.18), and replacing | B T θ | by k π , we obtain finallythe expression of γ (( w ,w ∗ ) , θ ) ( s ) as follows: x ( s ) = s (cid:32) x ∗ (cid:33) + sin(2 s k π )2 k π (cid:32) w (cid:101) w k π B T θ (cid:33) + − cos(2 s k π )2 k π (cid:32) (cid:101) w − w k π B T θ (cid:33) t ( s ) = (cid:16) s − sin(2 s k π )2 k π (cid:17) t + − cos(2 s k π )4 k π s (cid:101) w B x ∗ + s k π sin(2 s kπ ) − s k π )4 k π w B x ∗ . (5.33) bngE Moreover, by (5.22) and (5.20), we get that | w ∗ | = (cid:12)(cid:12)(cid:12)(cid:12) x ∗ + (cid:101) w B T θ | B T θ | (cid:12)(cid:12)(cid:12)(cid:12) = | x ∗ | + (cid:101) w , since (5.5) says that x ∗ · B T θ | B T θ | = X ∗ · T B θ | B T θ | = X ∗ · (cid:98) η = 0 , where we have used (5.30) in the last equality. In conclusion, (cid:96) ( γ (( w ,w ∗ ) , θ ) ) = w + | w ∗ | = | x ∗ | + w + (cid:101) w = | x ∗ | + 4 k π (cid:12)(cid:12)(cid:12) T − ( T · (cid:99) X ∗ ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12) . (5.34) Kgl o to any given g (cid:54) = o Recall that the “good” normal geodesics from o to any given g (cid:54) = o are characterizedby [88, Theorem 2.4] in the general setting of step-two groups. Also in our special settingof K-type groups, the “bad” normal geodesics from o to any given g ∈ W \ { o } arecharacterized in the last subsection. Hence, we only study in the sequel the shortestgeodesic(s) from o to any given g (cid:54) = o .Consider the cases g ∈ M , g ∈ Abn ∗ o \ { o } and g ∈ Cut o \ Abn ∗ o = Cut CL o .1. If g ∈ M , there exists a unique shortest geodesic steering o to g , and its equationis well-known by Theorem 2 together with (5.14) and (5.18).2. For g ∈ Abn ∗ o \ { o } , the unique shortest geodesic is a straight segment and it isabnormal.3. From now on, assume that g ∈ Cut o \ Abn ∗ o . Using Theorem 5, any shortestgeodesic joining o to g is given by γ ( w, θ ) , where( w, θ ) := (( w , w ∗ ) , θ ) ∈ R p × R p with | w | = d ( g ) , | B T θ | = π, and exp( w, θ ) = g . More precisely, by the results known in Subsection 5.1.2, we yield:35 roposition 6.
Let g ∈ Cut o \ Abn ∗ o = Cut CL o , namely g = ((0 , x ∗ ) , t ) with t (cid:54) = 0 and | X ∗ · T | ≤ | X ∗ | √ π (cid:114)(cid:12)(cid:12)(cid:12) T − ( (cid:99) X ∗ · T ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12) , where T and X ∗ are defined by (5.3) . Then any shortest geodesic from o to g , γ (( w ,w ∗ ) , θ ) ( s ) = ( x ( s ) , t ( s )) , can be written as in (5.33) with k = 1 , w ∗ defined by (5.32) , and ( w , (cid:101) w , θ ) by (5.28) for X ∗ (cid:54) = 0 , or by (5.29) for X ∗ = 0 . Remark 12.
If we suppose further that X ∗ (cid:54) = 0 and | X ∗ · T | < | X ∗ | √ π (cid:114)(cid:12)(cid:12)(cid:12) T − ( (cid:99) X ∗ · T ) (cid:99) X ∗ (cid:12)(cid:12)(cid:12) , (5.28) implies that there are exactly two distinct shortest geodesics joining o to g . To end this part, we determine on K-type groups
In the special case of K-type groups, combining (ii) of Corollary 9 with Proposition 1 andthe result in Case 1 of Subsection 5.1.2, we obtain the following:
Corollary 14.
Let exp { s ( w, θ ) } be an arclength parametrized geodesic. Then its cuttime h cut equals + ∞ when w = (0 , w ∗ ) with θ · B w ∗ = 0 , and π/ (cid:107) U ( θ ) (cid:107) otherwise. s7 Let m, n ∈ N ∗ with n ≥ m . Consider a full-rank m × n real matrix A = ( (cid:101) a , . . . , (cid:101) a n ) with column vectors (cid:101) a j := ( a ,j , . . . , a m,j ) T ∈ R m . A step-two group associated to quadratic CR manifolds of type A , G (CR) A , is defined by G (2 n, m, U (CR) A ) with (cf. for example [114] for more details) U (CR) , ( j ) A = (cid:18) a j, − a j, (cid:19) . . . (cid:18) a j,n − a j,n (cid:19) , ≤ j ≤ m. m = 1 and A = (1 , . . . , G (CR) A is the Heisenberg group of realdimension 2 n + 1, H n +1 . Moreover, for m = n and A = I n , G (CR) I n is the direct productof n copies of Heisenberg group H , namely G (CR) I n = H × · · · × H . Observe that we have for τ ∈ R m , U (CR) A ( τ ) = i (cid:18) (cid:101) a · τ − (cid:101) a · τ (cid:19) . . . (cid:18) (cid:101) a n · τ − (cid:101) a n · τ (cid:19) . (6.1) ucr Then the initial reference set in this situation is given byΩ ∗ := Ω (CR) A = n (cid:92) j =1 { τ ∈ R m ; | (cid:101) a j · τ | < π } . (6.2) rsCR We identify R with C in the usual way, so R n with C n . Write in the sequel z = ( z , . . . , z n ) ∈ C n . A simple calculation shows that the reference function is φ (( z, t ); τ ) := φ (CR) A (( z, t ); τ ) = n (cid:88) j =1 | z j | ( (cid:101) a j · τ ) cot( (cid:101) a j · τ ) + 4 t · τ. (6.3) nRFe Recall that f ( s ) = 1 − s cot s and µ ( s ) = f (cid:48) ( s ) (see (2.25)). The gradient and theHessian matrix of φ (( z, t ); · ) at τ ∈ Ω ∗ are clearly ∇ τ φ (( z, t ); τ ) = − n (cid:88) j =1 µ ( (cid:101) a j · τ ) | z j | (cid:101) a j + 4 t, (6.4) HMnn0
Hess τ φ (( z, t ); τ ) = − A Λ( z ; τ ) A T = − A (cid:112) Λ( z ; τ ) (cid:16) A (cid:112) Λ( z ; τ ) (cid:17) T , (6.5) HMnn respectively, whereΛ( z ; τ ) = µ (cid:48) ( (cid:101) a · τ ) | z | . . . µ (cid:48) ( (cid:101) a n · τ ) | z n | ≥ , ∀ τ ∈ Ω ∗ , since µ (cid:48) ( s ) > − π < s < π (see for example Lemma 8 in Subsection 7.2 below).Recall that M is defined in Subsection 2.3. We can characterize the squared sub-Riemannian distance as well as the cut locus in the following theorem.37 Theorem 10. (1) We have d ( g ) = sup τ ∈ Ω (CR) A φ (CR) A ( g ; τ ) , ∀ g ∈ G (CR) A . (2) The cut locus of o , Cut o , is M c , where M = (cid:40) ( z, t ); span { (cid:101) a j ; | z j | (cid:54) = 0 } = R m and ∃ θ ∈ Ω ∗ s.t. t = 14 n (cid:88) j =1 µ ( (cid:101) a j · θ ) | z j | (cid:101) a j (cid:41) . (3) If ( z, t ) ∈ M , then there exists a unique θ = θ ( z, t ) ∈ Ω (CR) A such that t = 14 n (cid:88) j =1 µ ( (cid:101) a j · θ ) | z j | (cid:101) a j . (6.6) Moreover, we have d ( z, t ) = φ (( z, t ); θ ) = n (cid:88) j =1 (cid:18) (cid:101) a j · θ sin( (cid:101) a j · θ ) (cid:19) | z j | = n (cid:88) j =1 ( (cid:101) a j · θ ) cot( (cid:101) a j · θ ) | z j | + 4 t · θ. (6.7) Proof.
Let r ( A ) denote the rank of a real matrix A . Using the basic property r ( AA T ) = r ( A ), it deduces from (6.5) that r (Hess τ φ (( z, t ); τ )) = r (cid:16) A (cid:112) Λ( z ; τ ) (cid:17) = dim span { (cid:101) a j ; | z j | (cid:54) = 0 } , since µ (cid:48) ( s ) > s ∈ ( − π, π ) (see Lemma 8 in Subsection 7.2 below). Combining thiswith (6.4), we get immediately the characterization of M . And the third assertion of thistheorem follows directly from Theorem 2.Moreover, it follows from [88, Proposition 2.2] that M ⊇ { ( z, t ); | z j | (cid:54) = 0 , ∀ ≤ j ≤ n } , (6.8) charM which is dense in G (CR) A . Then G (CR) A is of type GM. By Theorem 4 and Corollary 7, wededuce the first assertion, Cut o = M c = ∂ M , as well as other sub-Riemannian geometricproperties.Now we describe o to any g (cid:54) = o nSs61 Let us begin with the 38 .1.1 Sub-Riemannian exponential map
In the setting of step-two groups associated to quadratic CR manifolds, the equation of thenormal geodesic (cf. (2.6)) as well as the sub-Riemannian exponential map becomes veryconcise via their special group structure, namely (6.1). More precisely, if p = ( p , . . . , p n ) ∈ C n ∼ = R n and θ ∈ R m , letexp { s ( p, θ ) } := ( z ( p, θ ; s ) , t ( p, θ ; s )) := ( z ( s ) , t ( s )) , (6.9)and exp( p, θ ) := ( z ( p, θ ) , t ( p, θ )) := ( z, t ) . (6.10)It follows from (2.6) that ζ j ( s ) = e − i s ( (cid:101) a j · θ ) p j , z j ( s ) = 1 − e − i s ( (cid:101) a j · θ ) i ( (cid:101) a j · θ ) p j , ≤ j ≤ n, and for 1 ≤ k ≤ m , t k ( s ) = 2 − n (cid:88) j =1 a k,j (cid:90) s (cid:28)(cid:18) − (cid:19) z j ( r ) , ζ j ( r ) (cid:29) dr = − − n (cid:88) j =1 a k,j (cid:90) s (cid:60) (cid:16) i z j ( r ) · ζ j ( r ) (cid:17) dr = n (cid:88) j =1 s ( (cid:101) a j · θ ) − sin (2 s ( (cid:101) a j · θ ))8 ( (cid:101) a j · θ ) | p j | a k,j . In conclusion, the normal geodesic with the initial covector ( p, θ ), γ ( p, θ ; s ), is givenby z j ( s ) = − e − i s ( (cid:101) aj · θ ) i ( (cid:101) a j · θ ) p j , ≤ j ≤ n,t ( s ) = n (cid:80) j =1 2 s ( (cid:101) a j · θ ) − sin (2 s ( (cid:101) a j · θ ))8 ( (cid:101) a j · θ ) | p j | (cid:101) a j . (6.11) ngE In particular, we yield the expression of exp { ( p, θ ) } , z j ( p, θ ) = − e − i ( (cid:101) aj · θ ) i ( (cid:101) a j · θ ) p j , ≤ j ≤ n,t ( p, θ ) = n (cid:80) j =1 2 ( (cid:101) a j · θ ) − sin(2 ( (cid:101) a j · θ ))8 ( (cid:101) a j · θ ) | p j | (cid:101) a j . (6.12) expmap3 o to any given g (cid:54) = o ss612 We are in a position to determine all shortest geodesics from o to any given o (cid:54) = g :=( z , t ) with z := ( z (0)1 , . . . , z (0) n ). First, notice that in our framework, the set defined by392.18) is (cid:101) M = (cid:40) ( z, t ); ∃ θ ∈ Ω ∗ s.t. t = 14 n (cid:88) j =1 µ ( (cid:101) a j · θ ) | z j | (cid:101) a j (cid:41) . And we consider the following two cases g ∈ (cid:101) M \ { o } and g ∈ (cid:101) M c . Case 1. g ∈ (cid:101) M \ { o } , namely φ ( g ; · ) attains its maximum at some point in Ω ∗ , saying θ . It follows from Theorem 2 that there exists a unique shortest geodesic joining o to g ,that is, exp { s ( p ( g ) , θ ) } (0 ≤ s ≤
1) with p ( g ) = U ( θ )sin U ( θ ) e − (cid:101) U ( θ ) z , i.e. p j ( g ) = (cid:101) a j · θ sin ( (cid:101) a j · θ ) e i ( (cid:101) a j · θ ) z (0) j , ∀ ≤ j ≤ n. (6.13)Substituting this in (6.11), we yield its concise expression. Moreover, it is strictly normalif g ∈ M and also abnormal for g ∈ (cid:101) M \ ( M ∪ { o } ). Case 2. g ∈ (cid:101) M c , that is φ ( g ; · ) only attains its supremum in Ω ∗ at some θ ∈ ∂ Ω ∗ . Inorder to characterize all shortest geodesics from o to g , we set E ( g ) := (cid:26) θ ∈ ∂ Ω ∗ ; φ ( g ; θ ) = sup τ ∈ Ω ∗ φ ( g ; τ ) = d ( g ) (cid:27) . From Theorem 5, it remains to determine all ( p ( g ) , θ ( g )) such that θ ( g ) ∈ E ( g ) , | p ( g ) | = d ( g ) and exp { ( p ( g ) , θ ( g )) } = g . (6.14) crIC Up to rearrangements, we may assume that there exists an L ∈ N , L < n such that | (cid:101) a j · θ ( g ) | < π for 1 ≤ j ≤ L, and | (cid:101) a j · θ ( g ) | = π if L + 1 ≤ j ≤ n. (6.15) CRc1
By (6.12), we obtain that p j ( g ) = 2 i ( (cid:101) a j · θ ( g ))1 − e − i ( (cid:101) a j · θ ( g )) z (0) j , ∀ ≤ j ≤ L, z (0) j = 0 for L + 1 ≤ j ≤ n, (6.16) equgeocr and (6.14) holds if and only if ( p L +1 ( g ) , . . . , p n ( g )) is a solution of the following equation: n (cid:88) j = L +1 (cid:101) a j · θ ( g ) | p j ( g ) | (cid:101) a j = 4 t − L (cid:88) j =1 µ ( (cid:101) a j · θ ( g )) | z (0) j | (cid:101) a j . (6.17) equgeoCR To prove this result, it suffices to show that | p ( g ) | = d ( g ) under our assumptions(6.15)-(6.17), and other claims are clear.Indeed, from (3.11) and (6.16), we have d ( g ) = φ ( g ; θ ( g )) = L (cid:88) j =1 ( (cid:101) a j · θ ( g )) cot( (cid:101) a j · θ ( g )) | z (0) j | + 4 t · θ ( g ) . (6.18) expphibdCR θ ( g ) on both sides of (6.17), we obtainthat n (cid:88) j = L +1 | p j ( g ) | = 4 t · θ ( g ) − L (cid:88) j =1 ( (cid:101) a j · θ ( g )) µ ( (cid:101) a j · θ ( g )) | z (0) j | . Summing with L (cid:80) j =1 | p j ( g ) | on both sides of last equality, it follows from (6.16) that | p ( g ) | = 4 t · θ ( g ) + L (cid:88) j =1 (cid:32)(cid:18) (cid:101) a j · θ ( g )sin( (cid:101) a j · θ ( g )) (cid:19) − ( (cid:101) a j · θ ( g )) µ ( (cid:101) a j · θ ( g )) (cid:33) | z (0) j | . By (6.18) and the elementary identity (cid:16) s sin s (cid:17) − s µ ( s ) = s cot s, we get | p ( g ) | = d ( g ) .In particular, if (( p ( g ) , . . . , p L +1 ( g ) , . . . p n ( g )) , θ ( g )) satisfies the condition (6.14),then so does (( p ( g ) , . . . , w L +1 p L +1 ( g ) , . . . , w n p n ( g )) , θ ( g )) for any complex numbers( w L +1 , . . . , w n ) satifying | w L +1 | = . . . = | w n | = 1 . Thus, there exist infinitely many shortest geodesics from o to g ∈ (cid:101) M c if4 t − L (cid:88) j =1 µ ( (cid:101) a j · θ ( g )) | z (0) j | (cid:101) a j (cid:54) = 0 . In such case, it follows from [123, Lemma 9] that g belongs to the classical cut locus of o , Cut CL o . This provides another explanation for such g ∈ (cid:101) M c belonging to Cut CL o besides(iii) of Corollary 9. Finally we remark that the meaning for the case L = 0 is clear in theabove discussion.Step-two groups associated to quadratic CR manifolds have very rich sub-Riemanniangeometric properties. First, we provide an example of such groups on which (2.27) is nolonger valid for some g ∈ Abn ∗ o \ (cid:101) M . (2.29) can be false at a point belonging to the shortest ab-normal set on GM-groups s71p5 Proposition 7.
There exist G (CR) A , g ∈ Abn ∗ o \ (cid:101) M and c > such that d ( g + hg ) + d ( g − hg ) − d ( g ) ≤ , (6.19) semiconc for all g = ( z, t ) ∈ C n × R m with | g | = | z | + | t | = 1 and < h < c . roof. Setting (cid:101) a := (cid:18) (cid:19) , (cid:101) a := (cid:18) (cid:19) , (cid:101) a := (cid:18) − (cid:19) and A := ( (cid:101) a , (cid:101) a , (cid:101) a ), we obtain a step-two group associated to quadratic CR manifolds G (CR) A . In our case, we haveΩ ∗ = Ω (CR) A = { τ ; − π < τ ± τ < π } . Consider g := (0 , e ) ∈ C × R with e = (1 , g (cid:54)∈ (cid:101) M (see (2.18) for its definition) since the unique θ ∈ Ω ∗ satisfying d (0 , e ) = sup τ ∈ Ω ∗ φ ((0 , e ); τ ) = sup τ ∈ Ω ∗ e · τ ) = 4 π = 4 ( e · θ ) , (6.20) element2 is θ = ( π, ∈ ∂ Ω ∗ .Next, we will show that g ∈ Abn ∗ o . Let p = ( p , p , p ) := (2 √ π, , ∈ C . Observe that, by (6.12), we haveexp( p, θ ) = (0 , e ) . (6.21) element1 Combining this with (6.20), we find that γ ( p, θ ) is a shortest geodesic from o to g . Itremains to prove that it is also abnormal.Indeed, set e = (0 , p as well as U (CR) A ( θ ) k p ( k ∈ N ∗ ) belongs to thekernel of U (CR) A ( e ), namely U (CR) A ( e ) U (CR) A ( θ ) k p = 0 , ∀ k ∈ N . (6.22) element3 Hence, Proposition 1 implies that γ ( p, θ ) is also abnormal. So (0 , e ) ∈ Abn ∗ o .Now, we are in a position to prove (6.19) under our assumptions. Set in the sequelΩ ♦ := (cid:26) τ ; 3 π < τ ± τ < π (cid:27) ⊆ Ω ∗ . First, we suppose that g = ( z, t ) with z j (cid:54) = 0 for all j = 1 , ,
3. Then (6.8) impliesthat g ( ± h ) := g ± h g ∈ M for any h (cid:54) = 0. Let us begin with g ( h ). By Theorem 2, thereexists a unique ( ζ ( h ) , θ ( h )) ∈ ( C \ { } ) × Ω ∗ such that exp { ( ζ ( h ) , θ ( h )) } = g ( h ).Next, we claim that θ ( h ) ∈ Ω ♦ for any | g | = 1 and any 0 < h ≤ c with c < / ζ ∗ , θ ∗ ) ∈ ( C \ { } ) × Ω ∗ \ Ω ♦ suchthat | ζ ∗ | = d ( g ) and exp( ζ ∗ , θ ∗ ) = g , which implies γ ( ζ ∗ , θ ∗ ) is also a shortest geodesic42rom o to g . From Case 2 in Subsection 6.1.2, we have that γ ( ζ ∗ , θ ∗ ) = γ ( p ∗ , θ ) since E ( g ) = { θ } . Furthermore, p ∗ := ( p ∗ , p ∗ , p ∗ ) ∈ C , satisfying | p ∗ | = 4 π , is a solution of (cid:88) j =1 | p ∗ j | (cid:101) a j = 4 π e . (6.23) element4 From Lemma 2, we get p ∗ = ζ ∗ , and Proposition 1 implies that U (CR) A ( θ − θ ∗ ) p ∗ = 0 . (6.24) element5 If p ∗ (cid:54) = 0, using (6.1), we obtain that (cid:101) a · θ ∗ = (cid:101) a · θ = π , so θ ∗ = ( π,
0) since θ ∗ ∈ Ω ∗ ,which contradicts with the fact θ ∗ ∈ Ω ∗ \ Ω ♦ . In the opposite case p ∗ = 0, it follows from(6.23) that | p ∗ | = | p ∗ | = √ π . Using (6.1), we get that (cid:101) a j · ( θ − θ ∗ ) = 0 for j = 2 , p ∗ , p ∗ (cid:54) = 0. Observe that (cid:101) a and (cid:101) a are linearly independent, and thus we have θ = θ ∗ ,which leads to a contradiction as well.Consequently, for such g ( h ), by (3) of Theorem 10, we get that d ( g ( h )) = φ ( g ( h ); θ ( h )) = (cid:88) j =1 ( (cid:101) a j · θ ( h )) cot( (cid:101) a j · θ ( h )) | z j | h + 4 ( e + h t ) · θ ( h ) ≤ e + h t ) · θ ( h ) , where we have used in the inequality that θ ( h ) ∈ Ω ♦ . Let ( a, b ) := 4 ( e + h t ). Observethat a ≥ ≥ | b | since 0 < h ≤ c < and | t | ≤ ∗ , κ ( τ , τ ) := a τ + b τ attains its maximum at θ . In conclusion, d ( g ( h )) ≤ e + h t ) · θ , ∀ | g | = 1 , < h < c (cid:28) . (6.25) estplus And similarly, we have d ( g ( − h )) ≤ e − h t ) · θ , ∀ | g | = 1 , < h < c (cid:28) . (6.26) estminus We use (6.20) together with (6.25) and (6.26), and obtain (6.19) under the additionalcondition that ( z, t ) satisfying z j (cid:54) = 0 for all j = 1 , , G (CR) A on which Abn o (cid:54) = Abn ∗ o . s72p6 Proposition 8.
There exists a step-two group associated to quadratic CR manifolds G (CR) A such that Abn ∗ o (cid:36) Abn o . roof. Setting (cid:101) a = (cid:18) − (cid:19) , (cid:101) a = (cid:18) (cid:19) , (cid:101) a = (cid:18) − (cid:19) and A = ( (cid:101) a , (cid:101) a , (cid:101) a ) , we obtain a step-two group associated to quadratic CR manifolds G (CR) A . And we willshow that the point g := (0 , e ) ∈ C × R with e = (1 ,
0) satisfies g ∈ Abn o \ Abn ∗ o .In our situation, (6.2) implies that the initial reference set is given byΩ ∗ = Ω (CR) A = { τ = ( τ , τ ); − π < τ ± τ < π } . We argue as in the proof of Proposition 7, and get that d ( g ) = sup τ ∈ Ω ∗ φ ( g ; τ ) = sup τ ∈ Ω ∗ e · τ ) = 4 π = 4 ( e · θ ) , where θ = ( π, ∈ ∂ Ω ∗ is the unique maximum point of φ ( g ; · ) in Ω ∗ . So g ∈ (cid:101) M c . Viaa simple calculation, Case 2 in Subsection 6.1.2 implies that any shortest geodesic joining o to g can be written as γ p ( s ) := exp { s ( p, θ ) } (0 ≤ s ≤
1) where p := (0 , p , p ) ∈ C with | p | = | p | = √ π .We claim that all γ p are strictly normal, so g / ∈ Abn ∗ o . We argue by contradiction:assume that there exists p ∗ = (0 , p ∗ , p ∗ ) with | p ∗ | = | p ∗ | = √ π such that γ p ∗ is abnormal.It follows from Proposition 1 that there exists a σ ∈ R \ { } such that U (CR) A ( σ ) U (CR) A ( θ ) k p ∗ = 0 , ∀ k ∈ N . (6.27)In particular, we yield U (CR) A ( σ ) p ∗ = 0. Using (6.1), we obtain that (cid:101) a j · σ = 0 for j = 2 , p ∗ , p ∗ (cid:54) = 0. Notice that (cid:101) a and (cid:101) a are linearly independent. Hence we have σ = 0,which leads to a contradiction.On the other hand, we set p ∗ = (2 √ π, , ∈ C , θ ∗ = (2 π, , and consider the normal geodesic γ ∗ ( s ) := exp( s ( p ∗ , θ ∗ )) (0 ≤ s ≤ { ( p ∗ , θ ∗ ) } = g , that is, γ ∗ is a normal geodesic joining o to g .Furthermore, let e = (0 , U (CR) A ( e ) U (CR) A ( θ ∗ ) k p ∗ = 0 , ∀ k ∈ N . (6.28)Combining this with Proposition 1, γ ∗ is also abnormal. As a result, we have g ∈ Abn o ,which ends the proof of this proposition. N , s5 The purpose of this section is to provide a new and independent proof, based on [88], forthe Gaveau-Brockett optimal control problem on the free Carnot group of step two and 3generators N , . More precisely, we will give a different proof for Theorem 12 below. Forthis purpose, we start by 44 .1 Preliminaries and known results obtained in [88] Recall that N , = R × R with U ( τ ) := i − τ τ τ − τ − τ τ , τ = ( τ , τ , τ ) ∈ R , that is, (cid:104) U x, x (cid:48) (cid:105) = x × x (cid:48) , where “ × ” denotes the cross product on R and hence (cid:101) U ( τ ) x = τ × x .In our situation, by considering τ as a column vector, U ( τ )sin U ( τ ) = | τ | sin | τ | I − (cid:18) | τ | sin | τ | − (cid:19) τ τ T | τ | and the initial reference set and the reference function are given respectively by Ω ∗ = { τ ; | τ | < π } and φ (( x, t ); τ ) = ( | τ | cot | τ | ) | x | + 1 − | τ | cot | τ || τ | ( τ · x ) + 4 t · τ. (7.1)See [88, §
11] for more details.We will solve the Gaveau-Brockett optimal control problem on N , , namely to find theexact expression of d ( x, t ) . Using a limiting argument, the scaling property (see (2.2)),and an orthogonal invariance, namely (cf. [88, Lemma 11.1]) d ( x, t ) = d ( O x, O t ) , ∀ ( x, t ) ∈ N , , ∀ O ∈ O , (7.2) nOIP where O denotes the 3 × d ( e , t e + t e ) with t > + := { ( v , v ) ∈ R ; v > , v + v < π } , (7.3) O+ R > := (cid:26) ( u , u ) ∈ R ; u > √ π (cid:112) | u | ≥ (cid:27) , (7.4) R <, + := (cid:26) ( u , u ); u > , < u < √ π √ u (cid:27) . (7.5)Set in the sequel √ π < ϑ < π such that tan ϑ = ϑ , (7.6)K ( v , v ) := 2 ψ ( r ) + ψ (cid:48) ( r ) r v with r := (cid:113) v + v , (7.7) nK3N and Ω − , := (cid:26) ( v , v ); v < , π < v < r = (cid:113) v + v < ϑ , K ( v , v ) < (cid:27) = { ( v , v ); v < < v , K ( v , v ) < , π (cid:54) = r < ϑ } . (7.8) Omega-4 π < r < ϑ and 0 > K ( v , v ) = 4 (cid:34) + ∞ (cid:88) j =1 v (cid:0) ( j π ) − r (cid:1) − + + ∞ (cid:88) j =2 (cid:0) ( j π ) − r (cid:1) − − r − π (cid:35) > π − v ( r − π ) , which implies v > π since v > E ⊆ R , we define the smooth function Λ,Λ( v , v ) := v (cid:20) ψ (cid:48) ( r ) r v v + 2 ψ ( r ) e (cid:21) , v = ( v , v ) ∈ E, r = | v | . (7.9) The following results can be found or deduced directly from [88, § RLT1
Theorem 11 ( [88]) . It holds that: (i) d ( e , t e + t e ) = d ( e , | t | e + t e ).(ii) Λ is a C ∞ -diffeomorphism from Ω + onto R > .(iii) For suitable ( θ , θ ) ∈ R , set θ := ( θ , θ , , t θ := 14 (Λ( θ , θ ) , , g θ := ( e , t θ ) . (7.10) n78n Then d ( g θ ) = φ ( g θ ; θ ) = θ | θ | + (cid:18) θ sin | θ | (cid:19) = (cid:12)(cid:12)(cid:12)(cid:12) U ( θ )sin U ( θ ) e (cid:12)(cid:12)(cid:12)(cid:12) , ∀ ( θ , θ ) ∈ Ω + . (7.11) dEn (iv) For any α ≥ (cid:101) t ( α ) := 4 − ( α π , π α ) ∈ ∂ R > and d ( e , ( (cid:101) t ( α ) , = 1 + α .(v) Abn ∗ o = Abn o = { ( x, x ∈ R } = (cid:101) M and (cid:26) ( x, t ); x (cid:54) = 0 , t (cid:54) = 0 , (cid:12)(cid:12)(cid:12)(cid:12) t − (cid:104) t, x | x | (cid:105) x | x | (cid:12)(cid:12)(cid:12)(cid:12) > √ π (cid:112) | x | | t · x | (cid:27) ⊆ S . (vi) Λ is a C ∞ -diffeomorphism from Ω − , onto R <, + .(vii) We have d (0 , t ) = 4 π | t | for all t ∈ R .In conclusion, via a limiting argument, it remains to determine d ( g θ ) with ( θ , θ ) ∈ Ω − , . Indeed, we have the following: t4 Theorem 12 ( [88], Theorem 11.3) . (7.11) remains valid for any ( θ , θ ) ∈ Ω − , . .2 Properties of some functions related to − s cot s s4 Recall that the functions f , µ and ψ are defined by (2.25). The following lemma can befound in [63, Lemme 3, p. 112] or [27, Lemma 1.33]: NL31
Lemma 8.
The function µ is an odd function, and a monotonely increasing diffeomor-phism between ( − π, π ) and R . In the sequel, let us define ϕ ( s ) := (cid:16) s sin s (cid:17) − , s ∈ R , (7.12) nvp0 and for s > h ( s ) := ψ (cid:48) ( s ) s sin s = s + s sin s cos s − s, (7.13) ndH ϕ ( s ) := s − sin ss − sin s cos s (cid:18) = ϕ ( s ) µ ( s ) = ϕ ( s ) s ψ (cid:48) ( s ) + 2 s ψ ( s ) (cid:19) , (7.14) nvp1 ϕ ( s ) := s ( s − sin s ) s + s sin s cos s − s (cid:18) = ϕ ( s ) µ ( s ) − s ψ ( s ) = ϕ ( s ) s ψ (cid:48) ( s ) (cid:19) , (7.15) nvp2 and ϕ ( s ) := (cid:112) ϕ ( s ) ϕ ( s ) . (7.16) nvp3 For k ∈ N ∗ , let ϑ k denote the unique solution of s = tan s on (cid:0) kπ, ( k + ) π (cid:1) .We will need the following lemma in order to prove Theorem 12: l1 Lemma 9.
We have (1) h ( r ) > r >
0. So, ψ (cid:48) ( r ) > < r (cid:54)∈ { kπ ; k ∈ N ∗ } .(2) ϕ is strictly increasing on (0 , + ∞ ).(3) ϕ is strictly increasing on ∪ + ∞ k =1 ( kπ, ϑ k ).(4) ϕ is strictly increasing on ( π, + ∞ ). Proof.
Notice that (1) can be found in [103, Lemma 3.1], and (2) as well as the strictmonotonicity of ϕ on ∪ + ∞ k =1 ( kπ, ϑ k ) can be found in the proof of [103, Lemma 3.4]. Forthe sake of clarity, we will provide a complete proof which is not complicated.We begin with the proof of (1). Obviously, it suffices to prove the first claim. Indeed,when r ≥ π , we have h (cid:48) ( r ) = r [2 + cos(2 r )] −
32 sin(2 r ) ≥ π − > . (7.17) fprime In the opposite case r ∈ (cid:0) , π (cid:1) , we have the elementary inequality sin r > r cos r and itis clear that h (cid:48)(cid:48) ( r ) = 2 − r ) − r sin(2 r ) = 4 sin r (sin r − r cos r ) > , h (cid:48) ( r ) > lim r → + h (cid:48) ( r ) = 0 for r ∈ (cid:0) , π (cid:1) . Combining this with (7.17), weget that h ( r ) > lim r → + h ( r ) = 0 for r >
0, which ends the proof of the first assertion.To prove (2), let us set F ( r ) := r − sin r cos rr − sin r and G ( r ) := − sin r + r sin r cos rr ( r − sin r ) . (7.18) dFG Remark that F ( r ) = 1 ϕ ( r ) , F ( r ) + 2 G ( r ) = 1 ϕ ( r ) , F ( r ) + G ( r ) = 1 r . (7.19) Fp2G
A simple computation gets that F (cid:48) ( r ) = − r cos r − sin r ) ( r − sin r ) < , ∀ r ∈ (0 , + ∞ ) \ { ϑ k ; k ∈ N ∗ } , (7.20) doF which implies the strict monotonicity of ϕ .We return to the proof of (3). Using (7.19), we have that (cid:18) ϕ ( r ) (cid:19) (cid:48) = 2( F ( r ) + G ( r )) (cid:48) − F (cid:48) ( r ) = − r + 2 ( r cos r − sin r ) ( r − sin r ) = 2 ( r cos r − r sin r − r + sin r )( r cos r − r sin r + r − sin r ) r ( r − sin r ) . Observe that for r ∈ ( kπ, ϑ k ) with k odd, we have that r cos r < sin r <
0, so (cid:40) r cos r − r sin r − r + sin r = r ( r cos r − sin r ) − ( r − sin r ) < r cos r − r sin r + r − sin r = r (cos r + 1) − sin r ( r + sin r ) > . Similarly, if r ∈ ( kπ, ϑ k ) with k even, we have that 0 < sin r < r cos r and (cid:40) r cos r − r sin r − r + sin r = r (cos r −
1) + sin r ( − r + sin r ) < r cos r − r sin r + r − sin r = r ( r cos r − sin r ) + ( r − sin r ) > . Hence we have that (cid:16) ϕ (cid:17) (cid:48) < ∪ + ∞ k =1 ( kπ, ϑ k ). Finally, a direct computation gives ϕ ( kπ ) = kπ and ϕ ( ϑ k ) = ϑ k ∀ k ≥ , which finishes the proof of the strict monotonicity of ϕ on ∪ + ∞ k =1 ( kπ, ϑ k ).We are in a position to prove the strict monotonicity of ϕ . By the fact that ϕ = √ ϕ ϕ , it following from (2) and (3) that ϕ is strictly increasing on ∪ + ∞ k =1 ( kπ, ϑ k ). Then48t remains to prove that it is also strictly increasing on ∪ + ∞ k =1 ( ϑ k , ( k + 1) π ). Indeed, byusing (7.19) again, we have that (cid:18) ϕ ( r ) (cid:19) (cid:48) = [( F ( r ) + 2 G ( r )) F ( r )] (cid:48) = (cid:20)(cid:18) r − F ( r ) (cid:19) F ( r ) (cid:21) (cid:48) = (cid:20)(cid:18) r − F ( r ) (cid:19)(cid:21) (cid:48) F ( r ) + ( F ( r ) + 2 G ( r )) F (cid:48) ( r )= − r F ( r ) + 2 G ( r ) F (cid:48) ( r ) . From (7.18) and (7.20), the last term equals − r − sin r cos r )( r − sin r ) + 2 r ( r cos r − sin r ) ( − sin r + r sin r cos r ) r ( r − sin r ) . Note that we have for r > ϑ > r − sin r cos r )( r − sin r ) + 2 r ( r cos r − sin r ) ( − sin r + r sin r cos r ) ≥ (cid:18) r − (cid:19) ( r − − r ( r + 1) (cid:16) r (cid:17) = ( r + 1) (cid:20) ( r −
12 ) ( r − − r − r (cid:21) ≥ ( r + 1) (cid:2) r − − r − r (cid:3) > r ( r −
4) + 3 > , which proves our lemma. W in our situation In order to prove Theorem 12, we will use [88, Corollary 2.1] in which we have assumedthat g does not belong to W (cf. (2.24)).Let us begin with the γ ( w, θ ; s ) = ( x ( s ) , t ( s )) on N , We first recall the convention (3.12). As in the setting of K-type groups, an elementarycomputation gives thatcos(2 s U ( θ )) w = cos (2 s | θ | ) ( w − ( w · (cid:98) θ ) (cid:98) θ ) + ( w · (cid:98) θ ) (cid:98) θ, sin(2 s U ( θ )) U ( θ ) w = sin(2 s | θ | ) | θ | ( w − ( w · (cid:98) θ ) (cid:98) θ ) + 2 s ( w · (cid:98) θ ) (cid:98) θ, (cid:101) U ( θ ) sin(2 s U ( θ )) U ( θ ) w = sin(2 s | θ | )( (cid:98) θ × w ) , (cid:101) U ( τ ) x = τ × x . Consequently, using (2.6), we obtainthat ζ ( s ) = ˙ x ( s ) = cos(2 s U ( θ )) w + (cid:101) U ( θ ) sin(2 s U ( θ )) U ( θ ) w = cos(2 s | θ | )( w − ( w · (cid:98) θ ) (cid:98) θ ) + ( w · (cid:98) θ ) (cid:98) θ + sin(2 s | θ | )( (cid:98) θ × w ) ,x ( s ) = sin(2 s | θ | )2 | θ | ( w − ( w · (cid:98) θ ) (cid:98) θ ) + s ( w · (cid:98) θ ) (cid:98) θ + 1 − cos(2 s | θ | )2 | θ | ( (cid:98) θ × w ) . (7.21) expxN Then using (2.6) again we have˙ t ( s ) = 12 x ( s ) × ζ ( s ) = 12 (cid:18) sin(2 s | θ | )2 | θ | − s cos(2 s | θ | ) (cid:19) u + 1 − cos(2 s | θ | )4 | θ | u + 12 (cid:18) s sin(2 s | θ | ) − − cos(2 s | θ | )2 | θ | (cid:19) u , (7.22) exptN with u = ( w − ( w · (cid:98) θ ) (cid:98) θ ) × [( w · (cid:98) θ ) (cid:98) θ ] = − ( w · (cid:98) θ ) ( (cid:98) θ × w ) , (7.23)u = ( w − ( w · (cid:98) θ ) (cid:98) θ ) × ( (cid:98) θ × w ) = ( | w | − ( w · (cid:98) θ ) ) (cid:98) θ, (7.24)u = ( w · (cid:98) θ ) (cid:98) θ × ( (cid:98) θ × w ) = − ( w · (cid:98) θ ) [ w − ( w · (cid:98) θ ) (cid:98) θ ] , (7.25)where we have used the well-known vector triple product expansion: a × ( b × c ) = ( a · c ) b − ( a · b ) c, ∀ a, b, c ∈ R . (7.26) triex As a result, by using (5.16) and (5.17), we can write t ( s ) = 1 − cos(2 s | θ | ) − s | θ | sin(2 s | θ | )4 | θ | u + 2 s | θ | − sin(2 s | θ | )8 | θ | u + sin(2 s | θ | ) − s | θ | − s | θ | cos(2 s | θ | )4 | θ | u . (7.27) exptpN Now, we provide the
W ∩ (cid:8) ( e , ( u , u , u , u ∈ R (cid:9) In this subsection, we suppose that | θ | = kπ with k ∈ N ∗ . Substituting this with s = 1 in(7.21) and (7.27), ( x ( w, θ ) , t ( w, θ )) := exp( w, θ ) is given by (cid:40) x ( w, θ ) = ( w · (cid:98) θ ) (cid:98) θt ( w, θ ) = | θ | u − | θ | u = w · (cid:98) θ | θ | w + | w | − w · (cid:98) θ ) | θ | (cid:98) θ . (7.28) expmap Then we have the following lemma, which implies that the set W is negligible in the“subspace” as well. 50 Lemma 10.
Let t = ( u,
0) = ( u , u , such that ( e , t ) ∈ W . Then we have u = k π u for some k ∈ N ∗ .Proof. Let ( w, θ ) ∈ R × R with w := ( w , w , w ) such that exp( w, θ ) = ( e , t ) and | θ | = kπ for some k ∈ N ∗ . It follows from (7.28) that e = ( w · (cid:98) θ ) (cid:98) θ, t = w · (cid:98) θ | θ | w + | w | − w · (cid:98) θ ) | θ | (cid:98) θ. (7.29) n723n The first equality implies that θ = ± ( kπ, ,
0) := ( θ , , w · (cid:98) θ = ±
1. Furthermore, w · (cid:98) θ and θ have the same sign. Taking inner product on both sides of the second identity in(7.29) with (cid:98) θ , we obtain t · (cid:98) θ = | w | − ( w · (cid:98) θ ) | θ | , (7.30) n724n and θ u ≥ − (cid:98) θ , and summing with both sides of the secondequation in (7.29) respectively, we have t − ( t · (cid:98) θ ) (cid:98) θ = w · (cid:98) θ | θ | ( w − ( w · (cid:98) θ ) (cid:98) θ ) . (7.31) tminus In particular, we get that w = 0 and u = 2 w · (cid:98) θ | θ | w .By Pythagoras Theorem, we can write | t | = | ( t · (cid:98) θ ) (cid:98) θ | + | t − ( t · (cid:98) θ ) (cid:98) θ | = ( t · (cid:98) θ ) + ( w · (cid:98) θ ) | θ | ( | w | − ( w · (cid:98) θ ) )= ( t · (cid:98) θ ) + 14 | θ | (4 t · θ )= 1 | θ | (cid:0) ( t · θ ) + ( t · θ ) (cid:1) , where we have used (7.31) and Pythagoras Theorem in the second “=”, (7.30) and | w · (cid:98) θ | =1 in the third “=”. Inserting t = ( u , u ,
0) and θ = ± ( kπ, ,
0) in the last equation givesthe desired result. r2 Remark 13.
Assume t = ( u , √ k π √ u , with u > and k ∈ N ∗ . It follows from theproof above that the “bad” normal geodesic joining o to ( e , t ) ∈ W is γ ( w, θ ) ( s ) (0 ≤ s ≤ with w = (1 , (cid:112) kπ u , and θ = ( kπ, , . ore precisely, from (7.21) and (7.27) , γ ( w, θ ) ( s ) := ( x ( s ) , t ( s )) (0 ≤ s ≤ is given by: x ( s ) = s + sin(2 s kπ )2 kπ √ kπ u + 1 − cos(2 s kπ )2 kπ √ kπ u t ( s ) = 14 − sin(2 s kπ ) − s kπ − s kπ cos(2 s kπ ) k π √ kπ u + s kπ sin(2 s kπ ) − s kπ ) k π √ kπ u + 2 s kπ − sin(2 s kπ )2 k π kπ u . We are in a position to provide the
Recall that ψ ( s ) = − s cot ss and ϕ ( s ) is defined by (7.12). Set in the following:Φ( w ) := ϕ ( | w | ) w | w | , w = ( w , w , w ) ∈ R . Let ( θ , θ ) ∈ Ω − , . Recall that (cf. (7.10)) θ := ( θ , θ , , ( u , u ) := Λ( θ , θ ) ∈ R <, + , t θ := 14 ( u , u , , g θ := ( e , t θ ) . By Lemma 10, via a limiting argument, we may suppose in the sequel that g θ / ∈ W .Then all the normal geodesics joining o to g θ are “good” ones, so [88, Corollary 2.1] givesthat d ( e , t θ ) = inf τ ∈ Υ θ (cid:34) τ | τ | + (cid:18) τ sin | τ | (cid:19) (cid:35) = inf τ ∈ Υ θ { Φ( τ ) + 1 } , (7.32)where Υ θ denotes the set of τ = ( τ , τ ,
0) such that Λ( τ , τ ) = ( u , u ) (see (7.9)), namely, (0 < ) u = ψ (cid:48) ( | τ | ) | τ | τ τ (0 < ) u = τ K ( τ , τ ) = τ (cid:16) ψ (cid:48) ( | τ | ) | τ | τ + 2 ψ ( | τ | ) (cid:17) . (7.33) nELn Here are some direct observations. 52 bservations:
Under the above assumptions, for any τ ∈ Υ θ , we have1. | τ | / ∈ { k π ; k ∈ N ∗ } since ( e , t θ ) / ∈ W .2. Moreover τ (cid:54) = 0 and τ > u > ψ (cid:48) > | τ | > π . Otherwise | τ | < π , u > τ >
0. So τ ∈ Ω + . Hence it follows from (ii) of Theorem 11 that ( u , u ) ∈ R > .This leads to a contradiction.4. The following equalities hold, in particular for θ ,Φ( τ ) = ϕ ( | τ | ) | τ | τ u = ϕ ( | τ | ) (cid:18) τ | τ | u + τ | τ | u (cid:19) = ϕ ( | τ | ) (cid:115) u (cid:18) u + τ τ u (cid:19) . (7.34) ndEn Indeed, the first “=” follows from the definition of ϕ (see (7.15)) and the first equationin (7.33), the second (resp. third) one from (7.33) and (7.14) (resp. (7.16) and the firsttwo equalities).It remains to show that Φ( θ ) < Φ( τ ) , ∀ τ ∈ Υ θ \ { θ } . (7.35) nGn And we split the proof into three cases.
Case 1: τ ∈ Υ θ with | τ | < | θ | . In such case, we get | τ | ∈ ( π, | θ | ). Moreover, wehave τ >
0. Otherwise τ <
0, and combining with the fact that u > ( τ , τ ) = ψ (cid:48) ( | τ | ) | τ | τ + 2 ψ ( | τ | ) <
0. So ( τ , τ ) ∈ Ω − , . Thusit follows from (vi) of Theorem 11 that ( τ , τ ) = ( θ , θ ), which is a contradiction.Next, remark that s ψ ( s ) = s − cot s ( <
0) is strictly increasing on ( π, ϑ ), then wehave 2 | τ | ψ ( | τ | ) < | θ | ψ ( | θ | ). By (7.33), we can write u | τ | τ − u | τ | τ = 2 | τ | ψ ( | τ | ) < | θ | ψ ( | θ | ) = u | θ | θ − u | θ | θ . By the fact that u , u , τ , τ , θ > θ <
0, the last inequality implies that0 < τ | τ | < θ | θ | and so | τ || τ | = (cid:115) − (cid:18) τ | τ | (cid:19) > (cid:115) − (cid:18) θ | θ | (cid:19) = | θ || θ | > . Then we haveΦ( θ ) = (cid:34)(cid:18) | θ | sin | θ | (cid:19) − (cid:35) (cid:18) θ | θ | (cid:19) < (cid:34)(cid:18) | τ | sin | τ | (cid:19) − (cid:35) (cid:18) τ | τ | (cid:19) = Φ( τ ) , since the function (cid:0) s sin s (cid:1) ( >
1) is strictly decreasing on ( π, ϑ ) (cf. [27, (1.45)]), whichends the proof in this case. 53 ase 2: τ ∈ Υ θ with | τ | ≥ | θ | , τ (cid:54) = θ and τ < . We argue as in the beginning ofCase 1, we have that K ( τ , τ ) = ψ (cid:48) ( | τ | ) | τ | τ + 2 ψ ( | τ | ) < | τ | / ∈ ( π, ϑ ). Moreover,since ψ (cid:48) is always positive (see (1) of Lemma 9) and ψ is negative only on ∪ + ∞ k =1 ( kπ, ϑ k ),then we get that | τ | ∈ ∪ + ∞ k =2 ( kπ, ϑ k ).We begin with the case where | τ || τ | ≥ | θ || θ | . Then we yield that0 < τ | τ | ≤ θ | θ | , so | τ | τ ≥ | θ | θ > . Combining this with the first equality in (7.34), we getΦ( θ ) = ϕ ( | θ | ) u | θ | θ < ϕ ( | τ | ) u | τ | τ = Φ( τ ) , where we have used, in the inequality, the fact that u > ϕ ( > π ) is strictlyincreasing on ∪ + ∞ k =1 ( k π, ϑ k ) from Lemma 9.We continue with the opposite case | τ || τ | < | θ || θ | , which is equivalent to θ | θ | < τ | τ | < τ | τ | > θ | θ | >
0. Hence, via the second equality in (7.34),0 < Φ( θ ) = ϕ ( | θ | ) (cid:18) u θ | θ | + u θ | θ | (cid:19) < ϕ ( | τ | ) (cid:18) u τ | τ | + u τ | τ | (cid:19) = Φ( τ ) , where we have used the fact that u , u > ϕ ( >
0) is strictly increasing on (0 , + ∞ )from Lemma 9. Case 3: τ ∈ Υ θ with | τ | ≥ | θ | and τ > . By the third equality in (7.34), we getΦ( θ ) = ϕ ( | θ | ) (cid:115) u (cid:18) u + u θ θ (cid:19) < ϕ ( | τ | ) (cid:115) u (cid:18) u + u τ τ (cid:19) = Φ( τ ) , where we have used, in the inequality, the fact that u , u > θ θ < < τ τ , and ϕ ( > π, + ∞ ) from Lemma 9.This finishes the proof of Theorem 12. In this sub-section, we provide some applications of Theorems 11 and 12. More precisely,we determine the exact formulas of d ( g ) on the whole space via a limiting argument, thecut locus Cut o as well as all shortest geodesics from o to any given g (cid:54) = o . For the sake ofclarity, we will first reformulate Theorem 12.Recall that ϑ is the unique solution of tan s = s on ( π, π ). Also for π < s < ϑ , f ( s ) = 1 − s cot s, µ ( s ) = f (cid:48) ( s ) , ψ ( s ) = f ( s ) s , ϕ ( s ) = (cid:16) s sin s (cid:17) − ,ϕ ( s ) = ϕ ( s ) µ ( s ) , ϕ ( s ) = ϕ ( s ) s ψ (cid:48) ( s ) , ϕ ( s ) = (cid:112) ϕ ( s ) ϕ ( s ) . T4n
Theorem 13.
Let u , u > such that u < √ π √ u . Suppose that (cid:101) θ := (cid:101) θ ( u , u ) = ( θ , θ ) with θ < < θ < | (cid:101) θ | ( (cid:54) = π ) < ϑ is the unique solution of: u = ψ (cid:48) ( | (cid:101) θ | ) | (cid:101) θ | θ θ u = θ (cid:32) ψ (cid:48) ( | (cid:101) θ | ) | (cid:101) θ | θ + 2 ψ ( | (cid:101) θ | ) (cid:33) . Then we have θ > π and d (cid:18) e ,
14 ( u , u , (cid:19) = (cid:32) θ | (cid:101) θ | (cid:33) + (cid:32) θ sin | (cid:101) θ | (cid:33) = ϕ ( | (cid:101) θ | ) (cid:32) u θ | (cid:101) θ | + u θ | (cid:101) θ | (cid:33) + 1= ϕ ( | (cid:101) θ | ) u | (cid:101) θ | θ + 1 = ϕ ( | (cid:101) θ | ) (cid:115) u (cid:18) u + u θ θ (cid:19) + 1 . (7.36) EDnD
Combining this with Theorem 11, it only remains to find the d ( e , β e ) with β > c3 Corollary 15.
Let t ( β ) = ( β, , with β > . Then it holds that d ( e , t ( β )) = ϕ ( r ) β + 1 , where r is the unique solution of the following equation in ( π, ϑ ) : − ψ ( r ) (cid:115) r + 2 r ψ ( r ) ψ (cid:48) ( r ) = β. (7.37) DCUTP
Proof.
Let 0 < (cid:15) < √ π √ β and t ( β, (cid:15) ) = ( β, (cid:15), (cid:101) θ (cid:15) := ( θ ( (cid:15) ) , θ ( (cid:15) )) is theunique solution of θ ( (cid:15) ) < < π < θ ( (cid:15) ) < | (cid:101) θ (cid:15) | < ϑ ,β = ψ (cid:48) ( | (cid:101) θ (cid:15) | ) | (cid:101) θ (cid:15) | θ ( (cid:15) ) θ ( (cid:15) ) , (cid:15) = θ ( (cid:15) ) (cid:32) ψ (cid:48) ( | (cid:101) θ (cid:15) | ) | (cid:101) θ (cid:15) | θ ( (cid:15) ) + 2 ψ ( | (cid:101) θ (cid:15) | ) (cid:33) . (7.38) ncin By the compactness of B R (0 , ϑ ), up to subsequences, we may take (cid:15) j −→ + as j −→ + ∞ such that the corresponding (cid:101) θ (cid:15) j −→ (cid:101) θ := ( θ (0)1 , θ (0)2 ). Obviously π ≤ θ (0)1 ≤ ϑ and θ (0)2 ≤
0. 55e claim that θ (0)2 (cid:54) = 0 so (cid:101) θ / ∈ { ( π, , ( ϑ , } and π < r := | (cid:101) θ | < ϑ by (7.38).Indeed, this is ensured by the choice of Ω − , in [88, § θ ( (cid:15) j ) −→ − . Then the first equation in (7.38) implies thatlim j −→ + ∞ ψ (cid:48) ( | (cid:101) θ (cid:15) j | ) | (cid:101) θ (cid:15) j | = + ∞ , so | (cid:101) θ (cid:15) j | −→ π + and θ ( (cid:15) j ) −→ π + , since π < θ ( (cid:15) j ) < | (cid:101) θ (cid:15) j | < ϑ and ψ (cid:48) ( s ) −→ + ∞ ( π < s < ϑ ) only if s −→ π + . Moreover,a direct calculation shows that (see also [88, Lemma 3.4])lim s −→ π + ( s − π ) ψ ( s ) = − π , lim s −→ π + ( s − π ) ψ (cid:48) ( s ) = 1 π . Combining this with (7.38), we get thatlim j −→ + ∞ π (cid:32) θ ( (cid:15) j ) | (cid:101) θ (cid:15) j | − π (cid:33) = β, − π lim j −→ + ∞ θ ( (cid:15) j ) | (cid:101) θ (cid:15) j | − π = 2 √ π (cid:112) β > . This leads to a contradiction.In conclusion, by the continuity of d and the last equality in (7.36), we obtain that d ( e , t ( β )) = ϕ ( r ) β + 1, where π < r < ϑ satisfies β = ψ (cid:48) ( r ) r θ (0)1 ( θ (0)2 ) , ψ (cid:48) ( r ) r ( θ (0)2 ) + 2 ψ ( r ) = 0 . (7.39) NICN
That is β = − ψ ( r ) (cid:113) r − ( θ (0)2 ) = − ψ ( r ) (cid:115) r + 2 r ψ ( r ) ψ (cid:48) ( r ) . Note that the RHS of the last equality is exactly P ( s ) with the function P definedin [103, (3.3)]. Then from [103, Lemma 3.5], we know that P is a strictly increasing dif-feomorphism between ( π, ϑ ) and (0 , + ∞ ), which justifies the uniqueness of the solution r in ( π, ϑ ). N , We can characterize the cut locus of o in N , from the exact formula for d as well. Recallthat S denotes the set of points g such that d is C ∞ in a neighborhood of g . We havethe following result: nP6n Proposition 9.
It holds that
S ⊇ { ( x, t ); x and t are linearly independent } on N , .Proof. Using the scaling property (cf. (2.2)), the orthogonal invariance (see (7.2)) as wellas (i) of Theorem 11, it suffices to show that d is smooth at ( e , ( u , u , u > u ≥
0. By recalling the notations defined by (7.3)-(7.9), we divide it into cases.56 ase (1): ( u , u ) ∈ R <, + . Consider the smooth mapΠ : O × (0 , + ∞ ) × Ω − , −→ R × R = N , ( O, r, ( θ , θ )) (cid:55)−→ ( r O e , r O (Λ( θ , θ ) , , where O denotes the 3 × I , , Λ − ( u , u )) is invertible since Λ is a C ∞ -diffeomorphismfrom Ω − , onto R <, + by (vi) of Theorem 11. As a result, from the inverse function theorem,Theorem 13 and the fact that the function r θ θ + θ + r (cid:32) θ sin (cid:112) θ + θ (cid:33) is smooth on (0 , + ∞ ) × Ω − , , we have ( e , ( u , u , ∈ S . Case (2): ( u , u ) ∈ R > . Similarly, we have ( e , ( u , u , ∈ S . Case (3): ( u , √ π √ u ) with u >
0. In such case, it can be proven by using [20,Theorem 26] with the function defined in [88, § ∗ o obtained in (v) of Theorem 11. However, we will provide here adirect proof, which is of independent interest.Set Ω + , := Ω + ∩ { ( v , v ); v > } , R >, + := R > ∩ { ( u , u ); u > } . Using the polar coordinate in R \ { ( v , v ≤ } , ( r, η ) where v = r cos η and v = r sin η with − π < η < π , we introduce another map Θ : ( r, η ) (cid:55)→ ( r, ρ := sin η sin r ) withsuitable domain.Notice that we have r > η ∈ (cid:0) , π (cid:1) (resp. (cid:0) − π , (cid:1) ) on Ω + , (resp. Ω − , ).Moreover, it is not hard to show that Θ is injective on Ω + , (resp. Ω − , ) and its Jacobiandeterminant is cos η sin r . It follows from the global inverse function Theorem that Θ is a C ∞ -diffeomorphism from Ω + , (resp. Ω − , ) onto (cid:101) Ω + := Θ(Ω + , ) (resp. (cid:101) Ω − := Θ(Ω − , )). From(7.3), (7.8), (7.7) and (7.13), a direct calculation yields that (cid:101) Ω + = { ( r, ρ ); 0 < ρ sin r < , < r < π } , (cid:101) Ω − = { ( r, ρ ); π < r < ϑ , ρ > , ρ h ( r ) + 2 r ψ ( r ) < } . By (ii) and (vi) of Theorem 11, Ξ := Λ ◦ Θ − : ( r, ρ ) (cid:55)→ ( u , u ) is a C ∞ -diffeomorphismfrom (cid:101) Ω + (resp. (cid:101) Ω − ) onto R >, + (resp. R <, + ). Using (7.9), (7.13) and the definition of ψ (cf. (2.25)), it can be written explicitly: u = (cid:16) r + sin r cos r − sin rr (cid:17) (cid:112) − ρ sin r ρ u = sin r (cid:16) r + sin r cos r − sin rr (cid:17) ρ + 2 (cid:0) sin rr − cos r (cid:1) ρ . (7.40) Xi π π π π π (cid:101) Ω + (cid:101) Ω − { π } × (0 , + ∞ ) −→ (cid:46) ( ϑ , ρ = r (cid:38) (cid:46) ρ = (cid:113) − r ψ ( r ) h ( r ) rρ Figure 1: Plot of (cid:101)
Ω = (cid:101) Ω + ∪ (cid:101) Ω − ∪ ( { π } × (0 , + ∞ )) fig1 A key observation is that Ξ is also meaningful on ( π, ρ ) for ρ > π, ρ ) =( π ρ , ρ ) is a bijection from { π } × (0 , + ∞ ) to (cid:110) ( u , u ) : u = √ π √ u > (cid:111) . Set (cid:101) Ω := (cid:101) Ω + ∪ (cid:101) Ω − ∪ ( { π } × (0 , + ∞ )) . See the plot in Figure 1.Moreover, notice that Ξ is C ∞ on (cid:101) Ω. A direct computation shows that the Jacobiandeterminant of Ξ at ( π, ρ ) ( ρ >
0) equals J (Ξ)( π, ρ ) = 2 (cid:0) π ρ + 4 ρ (cid:1) > . As a consequence, Ξ is a C ∞ -diffeomorphism from (cid:101) Ω onto (0 , + ∞ ) × (0 , + ∞ ).In conclusion, by (iii), (iv) of Theorem 11 as well as Theorem 12, we have d ( e ,
14 ( u , u , = ( r − sin r ) ρ + 1 , ∀ u , u > , (7.41)where ( r, ρ ) = Ξ − ( u , u ). Finally, by using the smooth map (cid:101) Π : O × (0 , + ∞ ) × (cid:101) Ω −→ R × R = N , ( O, R, ( r, ρ )) (cid:55)−→ ( R O e , R O (Ξ( r, ρ ) , , we can get that ( e , ( u , √ π √ u , ∈ S .This completes the proof of this proposition.58ndeed, the relation “ ⊇ ” in Proposition 9 can be improved to “=”. In other words,we have the following: c4 Corollary 16. On N , , we have Cut o = { ( x, t ); x and t are linearly dependent } .Proof. It follows from Proposition 9 thatCut o ⊆ { ( x, t ); x and t are linearly dependent } . (7.42) cutA Moreover, we have that (see (v) of Theorem 11):Abn ∗ o = { ( x, x ∈ R } = (cid:101) M . (7.43) abnN By Theorem 1, it remains to show that the classical cut locus of o isCut CL o = { ( x, t ); t (cid:54) = 0 and x = λt for some λ ∈ R } := E , (7.44) n32CC which has been already proven in [113] and [103] by completely different technique. In-deed, once we get (7.42) and (7.43), via [123, Lemma 9], (7.44) is a direct consequence ofthe simple fact that there exist two distinct shortest geodesics from o to any g ∈ E . SeeSubsection 7.5.3 below for more details. o to any given g (cid:54) = o SS76
Let us begin by the following observation: n32nLn
Lemma 11.
Let SO denote the × special orthogonal group, and ( x, t ) = exp( w, τ ) .Then we have exp( O w, O τ ) = (
O x, O t ) , ∀ O ∈ SO , (7.45) symN1 exp( O e (cid:101) U ( τ ) w, O τ ) = ( O x, O t ) , ∀ O ∈ O \ SO . (7.46) symN4 Proof.
Using the well-known basic property of the cross product: O ( τ × η ) = ( O τ ) × ( O η ) , ∀ O ∈ SO , (7.47) orthocro (7.45) can be checked directly by (7.21) and (7.27), or explained by [101, § (cid:101) U ( τ ) η = τ × η as well as (7.47), and get that (cid:101) U ( O τ ) η = ( O τ ) × η = O ( τ × ( O T η )) = O (cid:101) U ( τ ) O T η, ∀ O ∈ SO , (7.48) orthoU which implies that (cid:101) U ( O τ ) = O (cid:101) U ( τ ) O T , ∀ O ∈ SO . (7.49) orthoU2 Then for any O ∈ O \ SO , we have − O ∈ SO , and (7.45) implies thatexp( − O w, − O τ ) = ( − O x, − O t ) . Consequently, applying (2.10) to the last equation and using (7.49), we obtain (7.46).59ombining (7.45) and (7.46) with (2.9), it suffices to determine all shortest geodesic(s)from o to g , where: (1) g = ( e , ( u , u , u ≥ u >
0; (2) g = ( e , g = (0 , e ) or g = ( e , β e ) with β > Case 1. g = ( e , ( u , u , u ≥ u >
0. In such case, we have g ∈ S . Sothere exists a unique shortest geodesic from o to g , which is strictly normal. If ( u , u ) ∈ R > ∪ R <, + , it follows from [88, Theorem 2.4 and Theorem 2.5] that the shortest geodesic isgiven by γ ( w, θ ) with w = U ( θ )sin U ( θ ) e − (cid:101) U ( θ ) e and θ = (Λ − ( u , u ) , u = √ π √ u > γ ( w, θ ) with w = (1 , √ π u ,
0) and θ = ( π, , Case 2. g = ( e , ∈ Abn ∗ o \ { o } . The unique shortest geodesic joining o to g is astraight segment and it is abnormal. Case 3. g = (0 , e ) or g = ( e , β e ) with β >
0. In such case, g ∈ E . By (7.45),a trivial observation is that there exist at least two distinct shortest geodesics from o to g . Indeed, a complete description can be found in [103, § w, θ ) of any shortest geodesic from o to g , γ ( w, θ ) . Case 3 (a).
Let us begin with the case g ( β ) = ( e , ( β, , β >
0. By Remark5, there exist { g n := ( x ( n ) , t ( n ) ) } + ∞ n =1 ⊆ Cut co , with the corresponding unique shortestgeodesic γ ( w ( n ) , θ ( n ) ) , such that ( g n , w ( n ) , θ ( n ) ) → ( g ( β ) , w, θ ) as n → + ∞ . Without loss ofgenerality, we may assume that | x ( n ) | (cid:54) = 0 and x ( n ) · t ( n ) > n ≥
1. For each n ≥ O ( n ) ∈ O such that O ( n ) x ( n ) = | x ( n ) | e , O ( n ) t ( n ) = | x ( n ) | u ( n )1 , u ( n )2 ,
0) with u ( n )1 , u ( n )2 >
0. (7.50) icNN
Combining this with the fact that ( x ( n ) , t ( n ) ) → ( e , β e ) as n → + ∞ , we get that( u ( n )1 , u ( n )2 ) → ( β,
0) as n → + ∞ . By arguing as in the proof of (7.39) and using (7.45),(7.46) as well as (2.9), we get that O ( n ) θ ( n ) → (Θ , Θ ,
0) with Θ = (cid:115) r + 2 r ψ ( r ) ψ (cid:48) ( r ) , Θ = − (cid:115) − r ψ ( r ) ψ (cid:48) ( r ) , where r is the unique solution of (7.37) in ( π, ϑ ). Since O is compact, up to sub-sequences, we may further assume that O ( n ) → O (cid:48) as n → + ∞ . Moreover, it followsfrom the first equation in (7.50) that the orthogonal matrix O (cid:48) satisfies O (cid:48) e = e . Since O (cid:48) θ = (Θ , Θ , θ = (Θ , | Θ | cos σ, | Θ | sin σ ) for some σ ∈ R ,w = lim n → + ∞ w ( n ) = lim n → + ∞ U ( θ ( n ) )sin U ( θ ( n ) ) e − (cid:101) U ( θ ( n ) ) x ( n ) = U ( θ )sin U ( θ ) e − (cid:101) U ( θ ) e , where we have used (2.7) in the second “=” of the last formula.60n other words, we have proven that every shortest geodesic from o to g ( β ) can beexpressed as γ ( w, θ ) , where the parameter ( w, θ ) have the form θ ( σ ) := (Θ , | Θ | cos σ, | Θ | sin σ ) , w ( σ ) := U ( θ ( σ ))sin U ( θ ( σ )) e − (cid:101) U ( θ ( σ )) e . (7.51) thetaw Furthermore, we will prove that the converse is also valid, namely every such parameterprovides a shortest geodesic steering o to g ( β ) .Let us fix a such shortest geodesic γ ( w ( σ ) , θ ( σ )) , with σ ∈ R . By (7.45), for any α ∈ R , γ ( O ( α ) w ( σ ) , O ( α ) θ ( σ )) is also a shortest geodesic from o to g ( β ) , where O ( α ) = α − sin α α cos α ∈ SO . It remains to show that( O ( α ) w ( σ ) , O ( α ) θ ( σ )) = ( w ( σ + α ) , θ ( σ + α )) , ∀ α ∈ R , (7.52) nCvn since ( w ( σ + α ) , θ ( σ + α )) runs over all possible ( w ( σ ) , θ ( σ )) as the parameter α runsover R .In fact, it follows from (7.49) that O ( α ) w ( σ ) = O ( α ) U ( θ ( σ ))sin U ( θ ( σ )) e − (cid:101) U ( θ ( σ )) O ( α ) T O ( α ) e = U ( O ( α ) θ ( σ ))sin U ( O ( α ) θ ( σ )) e − (cid:101) U ( O ( α ) θ ( σ )) e . (7.53) invarw To finish the proof of (7.52), it suffices to notice that we have obviously O ( α ) θ ( σ ) = θ ( σ + α ) . Via some elementary but tedious calculations, our result can be identified with thatof [103, Theorem 3.2] with ( y, y ⊥ , θ ) = (cid:16) √ β e , √ β e , r (cid:17) therein, where we identify ∧ R with R via the map T defined by T : ∧ R −→ R a ∧ b (cid:55)−→ a × b. Case 3 (b).
In the opposite case where g = (0 , e ), similarly, it follows from Remark5 and (7.28) that all shortest geodesics joining o to g are given by γ ( w, θ ) , where θ = ( π, , , w = 2 √ π (0 , − cos σ, − sin σ ) with σ ∈ R . Via some elementary but tedious calculations, our result can be identified with thatof [103, Theorem 3.2] with ( y, y ⊥ , θ ) = ( e , e , π ) therein (via the map T as well).61 Appendix A: Proof of Lemma 7
Axa
The proof essentially follows that of [88, Proposition 10.3] and we include it for the sakeof completeness.Set v := ( v , v ) ∈ R . Then from the definition of Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); · ), for | (cid:101) x ∗ | + (cid:101) x (cid:54) = 0and (cid:101) x ∗ (cid:54) = 0, we have( u , u ) := Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); ( v , v ))= ∇ v (cid:20)(cid:18) v ψ (cid:18) | v | (cid:19) + v ψ ( | v | ) (cid:19) | (cid:101) x ∗ | + f (cid:18) | v | (cid:19) ( | (cid:101) x ∗ | + (cid:101) x ) (cid:21) . (8.1) graexpUps More precisely, u = v (cid:104) µ (cid:16) | v | (cid:17) | (cid:101) x ∗ | + (cid:101) x | v | + ψ (cid:48) (cid:16) | v | (cid:17) v | (cid:101) x ∗ | | v | + ψ (cid:16) | v | (cid:17) | (cid:101) x ∗ | + ψ (cid:48) ( | v | ) v | (cid:101) x ∗ | | v | (cid:105) u = v (cid:104) µ (cid:16) | v | (cid:17) | (cid:101) x ∗ | + (cid:101) x | v | + ψ (cid:48) (cid:16) | v | (cid:17) v | (cid:101) x ∗ | | v | + ψ (cid:48) ( | v | ) v | (cid:101) x ∗ | | v | + 2 ψ ( | v | ) | (cid:101) x ∗ | (cid:105) . (8.2) expUps It follows from [88, Lemma 3.3] that0 ≤ ψ (cid:48) ( | v | ) | v | | v | v ≤ π v (cid:18) ψ ( | v | ) + ψ (cid:48) ( | v | ) | v | v (cid:19) , ∀ | v | < π, which implies that | u | ≤ µ (cid:18) | v | (cid:19) | (cid:101) x ∗ | + (cid:101) x (cid:18) ψ (cid:48) (cid:18) | v | (cid:19) | v | ψ (cid:18) | v | (cid:19) | v | (cid:19) | (cid:101) x ∗ | + π v (cid:18) ψ ( | v | ) + ψ (cid:48) ( | v | ) | v | v (cid:19) | (cid:101) x ∗ | . Using the identity s ψ (cid:48) ( s ) + 2 s ψ ( s ) = µ ( s ), the second equality in (8.2) implies that | u | ≤ µ (cid:18) | v | (cid:19) | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ | π u | (cid:101) x ∗ | < π (cid:18) u | (cid:101) x ∗ | + | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ | (cid:19) , where we have used in “ < ” the fact that µ ( π/
2) = π/ µ is strictly increasing on( − π, π ) (see Lemma 8), so Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); · ) is from B R (0 , π ) to R r ( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ).Next, it follows from (8.1) that the Jacobian of Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); · ) is given byHess v (cid:20)(cid:18) v ψ (cid:18) | v | (cid:19) + v ψ ( | v | ) (cid:19) | (cid:101) x ∗ | + f (cid:18) | v | (cid:19) ( | (cid:101) x ∗ | + (cid:101) x ) (cid:21) > , since | (cid:101) x ∗ | + (cid:101) x (cid:54) = 0, (cid:101) x ∗ (cid:54) = 0, Hess v ( f ( | v | )) > v ( v ψ ( | v | )) ≥ v ( v ψ ( | v | )) ≥
0) for | v | < π , which can be found in the proof of [88, Proposition 10.1].Finally, using Hadamard’s theorem, it remains to show that Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); · ) is proper.To this end, we consider the behaviour of { v ( j ) } + ∞ j =1 ⊆ B R (0 , π ) satisfying v ( j ) = ( v ( j )1 , v ( j )2 ) −→ B R (0 , π ) and we split it into cases.(1) If | v ( j ) | −→ π − and | v ( j )2 | ≥ ε >
0, then Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); v ( j ) ) −→ ∞ .(2) If | v ( j ) | −→ π − and | v ( j )2 | −→ + , set Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); v ( j ) ) = ( u ( j )1 , u ( j )2 ).In the case | u ( j )1 | −→ + ∞ , it is easy to see Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); v ( j ) ) −→ ∞ .In the opposite case, assume | u ( j )1 | −→ a ( ≥ s ψ (cid:48) ( s ) + 2 s ψ ( s ) = µ ( s ) as well as µ ( π/
2) = π/
2, we have that a = π (cid:0) | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ | (cid:1) + lim j −→ + ∞ π (cid:32) v ( j )2 π − | v ( j ) | (cid:33) | (cid:101) x ∗ | , where we have used the fact that (see also [88, Lemma 3.4])lim s −→ π − ( π − s ) ψ (cid:48) ( s ) = 1 π . Since lim s −→ π − ( π − s ) ψ ( s ) = 1 π , the second equation of (8.2) gives thatlim j −→ + ∞ | u ( j )2 | = lim j −→ + ∞ π | v ( j )2 | π − | v ( j ) | | (cid:101) x ∗ | = 2 | (cid:101) x ∗ |√ π (cid:114) a − π | (cid:101) x ∗ | + (cid:101) x + | (cid:101) x ∗ | ) . In conclusion, we have Υ(( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ); v ( j ) ) −→ ∂ R r ( (cid:101) x ∗ , (cid:101) x , (cid:101) x ∗ ) when v ( j ) −→ ∂B R (0 , π )in B R (0 , π ), which means it is proper.This finishes the proof of this lemma. Axb
Assume that G j = G ( q j , m j , U j ) ( j = 1 , U j = { U (1) j , . . . , U ( m j ) j } . Consider the direct product G × G := G ( q + q , m + m , U ) with U defined by (cid:26)(cid:18) U (1)1 O q × q (cid:19) , . . . , (cid:18) U ( m )1 O q × q (cid:19) , (cid:18) O q × q U (1)2 (cid:19) , . . . , (cid:18) O q × q U ( m )2 (cid:19)(cid:27) , where O k × k denotes the k × k null matrix. Let g j := (x j , t j ) ∈ G j , j = 1 ,
2. We identify(g , g ) with ((x , x ) , (t , t )). Then, it is clear thatΩ ( G × G ) ∗ = Ω ( G ) ∗ × Ω ( G ) ∗ ,φ ( G × G ) ((g , g ); ( θ (1) , θ (2) )) = φ ( G ) (g ; θ (1) ) + φ ( G ) (g ; θ (2) ) , M ( G × G ) = M ( G ) × M ( G ) , d G × G ((g , g )) = d G (g ) + d G (g ) , γ G ( s ) , γ G ( s )) isa normal geodesic if and only if both γ G and γ G are normal geodesics. Moreover, thenormal geodesic ( γ G ( s ) , γ G ( s )) is abnormal if and only if γ G or γ G is abnormal. Also,it is shortest if and only if both γ G and γ G are shortest. Hence, we have the followingwell-known properties: on G × G , it holds that:Cut ( G × G ) o = (cid:16) Cut ( G ) o × G (cid:17) ∪ (cid:16) G × Cut ( G ) o (cid:17) , (9.1)Cut CL ( G × G ) o = (cid:16) Cut
CL ( G ) o × G (cid:17) ∪ (cid:16) G × Cut
CL ( G ) o (cid:17) , (9.2)Abn ∗ ( G × G ) o = (cid:16) Abn ∗ ( G ) o × G (cid:17) ∪ (cid:16) G × Abn ∗ ( G ) o (cid:17) , (9.3)and the meaning of the notations herein is obvious.By the fact that M ( G × G ) = M ( G ) × M ( G ) , we have the following: Proposition 10. G × G is of type GM if and only if both G and G are GM-groups. Similarly, on the direct product of the Euclidean space R k with a step-two group G ,we have Cut ( R k × G ) o = R k × Cut ( G ) o , Cut
CL ( R k × G ) o = R k × Cut
CL ( G ) o , (9.4)Abn ∗ ( R k × G ) o = R k × Abn ∗ ( G ) o , (9.5)and the meaning of the notations herein is obvious. In particular, NSAa
Proposition 11.
Let G be a step-two group. Consider the direct product of the Euclideanspace R k with G . Then: (1) R k × G is of type GM if and only if G is a GM-group; (2) R k × G is of type SA if and only if G is a M´etivier or SA-group.
10 Appendix C: Construction of SA-groups
Axc
In [88, § m ≥ G j = G ( q j , m, U j ) ( j = 1 , U j = { U (1) j , . . . , U ( m ) j } . We consider G := G ( q + q , m, U ) with U defined by (cid:40)(cid:32) U (1)1 U (1)2 (cid:33) , . . . , (cid:32) U ( m )1 U ( m )2 (cid:33)(cid:41) . g = ( x , x , t ) ∈ R q × R q × R m . Obviously, we have:Ω ( G ) ∗ = Ω ( G ) ∗ ∩ Ω ( G ) ∗ ,φ ( G ) ( g ; τ ) = (cid:104) U ( G ) ( τ ) cot U ( G ) ( τ ) x , x (cid:105) + (cid:104) U ( G ) ( τ ) cot U ( G ) ( τ ) x , x (cid:105) + 4 t · τ, and the meaning of the notations herein is clear. Moreover, g ∈ (cid:101) M ( G )2 if and only if thereare θ ∈ Ω ( G ) ∗ ∩ Ω ( G ) ∗ and t (cid:48) , t (cid:48)(cid:48) ∈ R m such that: t = t (cid:48) + t (cid:48)(cid:48) , t (cid:48) = − ∇ θ (cid:104) U ( G ) ( θ ) cot U ( G ) ( θ ) x , x (cid:105) ,t (cid:48)(cid:48) = − ∇ θ (cid:104) U ( G ) ( θ ) cot U ( G ) ( θ ) x , x (cid:105) , and the sum of two positive semidefinite matrices (cid:0) − Hess θ (cid:104) U ( G ) ( θ ) cot U ( G ) ( θ ) x , x (cid:105) (cid:1) + (cid:0) − Hess θ (cid:104) U ( G ) ( θ ) cot U ( G ) ( θ ) x , x (cid:105) (cid:1) is singular. In such case, we have ( x , t (cid:48) ) ∈ (cid:101) M ( G )2 and ( x , t (cid:48)(cid:48) ) ∈ (cid:101) M ( G )2 . Thus, we getimmediately the following: Proposition 12.
With the above notations, G is a SA-group when G is of type SA and G is of type M´etivier or SA. Acknowledgement
This work is partially supported by NSF of China (Grants No. 11625102 and No.11831004) and “The Program of Shanghai Academic Research Leader” (18XD1400700).The authors would like to thank L. Rizzi for many useful suggestions.
References
ABB12 [1] A. Agrachev, D. Barilari, and U. Boscain. On the Hausdorff volume in sub-Riemannian geometry.
Calc. Var. Partial Differential Equations , 43(3-4):355–388,2012.
ABB20 [2] A. Agrachev, D. Barilari, and U. Boscain.
A comprehensive introduction to sub-Riemannian geometry , volume 181 of
Cambridge Studies in Advanced Mathematics .Cambridge University Press, Cambridge, 2020. From the Hamiltonian viewpoint,With an appendix by Igor Zelenko.
AS04 [3] A. A. Agrachev and Y. L. Sachkov.
Control theory from the geometric viewpoint ,volume 87 of
Encyclopaedia of Mathematical Sciences . Springer-Verlag, Berlin, 2004.Control Theory and Optimization, II.65 [4] G. Alexopoulos. An application of homogenization theory to harmonic analysis:Harnack inequalities and Riesz transforms on Lie groups of polynomial growth. Canad. J. Math. , 44(4):691–727, 1992.
A94 [5] G. Alexopoulos. Spectral multipliers on Lie groups of polynomial growth.
Proc.Amer. Math. Soc. , 120(3):973–979, 1994.
A02 [6] G. K. Alexopoulos. Sub-Laplacians with drift on Lie groups of polynomial volumegrowth.
Mem. Amer. Math. Soc. , 155(739):x+101, 2002.
AR04 [7] L. Ambrosio and S. Rigot. Optimal mass transportation in the Heisenberg group.
J. Funct. Anal. , 208(2):261–301, 2004.
BR18 [8] Z. Badreddine and L. Rifford. Measure contraction properties for two-step analyticsub-Riemannian structures and Lipschitz Carnot groups.
To appear in A.I.F. arXive-prints , page arXiv:1712.09900, Dec. 2017.
BFKG16 [9] H. Bahouri, C. Fermanian-Kammerer, and I. Gallagher. Dispersive estimates forthe Schr¨odinger operator on step-2 stratified Lie groups.
Anal. PDE , 9(3):545–574,2016.
BGX00 [10] H. Bahouri, P. G´erard, and C.-J. Xu. Espaces de Besov et estimations de Strichartzg´en´eralis´ees sur le groupe de Heisenberg.
J. Anal. Math. , 82:93–118, 2000.
BBBC08 [11] D. Bakry, F. Baudoin, M. Bonnefont, and D. Chafa¨ı. On gradient bounds for theheat kernel on the Heisenberg group.
J. Funct. Anal. , 255(8):1905–1938, 2008.
BGL14 [12] D. Bakry, I. Gentil, and M. Ledoux.
Analysis and geometry of Markov diffusionoperators , volume 348 of
Grundlehren der Mathematischen Wissenschaften [Funda-mental Principles of Mathematical Sciences] . Springer, Cham, 2014.
BCC19 [13] A. Baldi, G. Citti, and G. Cupini. Schauder estimates at the boundary for sub-laplacians in Carnot groups.
Calc. Var. Partial Differential Equations , 58(6):PaperNo. 204, 43, 2019.
BFMT12 [14] Z. M. Balogh, K. F¨assler, P. Mattila, and J. T. Tyson. Projection and slicingtheorems in Heisenberg groups.
Adv. Math. , 231(2):569–604, 2012.
BKS18 [15] Z. M. Balogh, A. Krist´aly, and K. Sipos. Geometric inequalities on Heisenberggroups.
Calc. Var. Partial Differential Equations , 57(2):Paper No. 61, 41, 2018.
BKS19 [16] Z. M. Balogh, A. Krist´aly, and K. Sipos. Jacobian determinant inequality on corank1 Carnot groups with applications.
J. Funct. Anal. , 277(12):108293, 36, 2019.
BTW09 [17] Z. M. Balogh, J. T. Tyson, and B. Warhurst. Sub-Riemannian vs. Euclidean dimen-sion comparison and fractal geometry on Carnot groups.
Adv. Math. , 220(2):560–619, 2009.
BGM18 [18] S. Banerjee, M. Gordina, and P. Mariano. Coupling in the Heisenberg group andits applications to gradient estimates.
Ann. Probab. , 46(6):3275–3312, 2018.66
BG12 [19] D. Barilari, U. Boscain, and J.-P. Gauthier. On 2-step, corank 2, nilpotent sub-Riemannian metrics.
SIAM J. Control Optim. , 50(1):559–582, 2012.
BR19 [20] D. Barilari and L. Rizzi. Sub-Riemannian interpolation inequalities.
Invent. Math. ,215(3):977–1038, 2019.
BB16 [21] F. Baudoin and M. Bonnefont. Reverse Poincar´e inequalities, isoperimetry, andRiesz transforms in Carnot groups.
Nonlinear Anal. , 131:48–59, 2016.
BG17 [22] F. Baudoin and N. Garofalo. Curvature-dimension inequalities and Ricci lowerbounds for sub-Riemannian manifolds with transverse symmetries.
J. Eur. Math.Soc. (JEMS) , 19(1):151–219, 2017.
BGM19 [23] F. Baudoin, M. Gordina, and P. Mariano. On the Cheng-Yau gradient esti-mate for Carnot groups and sub-Riemannian manifolds.
Proc. Amer. Math. Soc. ,147(7):3181–3189, 2019.
BHT08 [24] F. Baudoin, M. Hairer, and J. Teichmann. Ornstein-Uhlenbeck processes on Liegroups.
J. Funct. Anal. , 255(4):877–890, 2008.
BK14 [25] F. Baudoin and B. Kim. Sobolev, Poincar´e, and isoperimetric inequalities for subel-liptic diffusion operators satisfying a generalized curvature dimension inequality.
Rev. Mat. Iberoam. , 30(1):109–131, 2014.
BGG96 [26] R. Beals, B. Gaveau, and P. Greiner. The Green function of model step two hypoel-liptic operators and the analysis of certain tangential Cauchy Riemann complexes.
Adv. Math. , 121(2):288–345, 1996.
BGG00 [27] R. Beals, B. Gaveau, and P. C. Greiner. Hamilton-Jacobi theory and the heat kernelon Heisenberg groups.
J. Math. Pures Appl. (9) , 79(7):633–689, 2000.
BR96 [28] A. Bella¨ıche and J.-J. Risler, editors.
Sub-Riemannian geometry , volume 144 of
Progress in Mathematics . Birkh¨auser Verlag, Basel, 1996.
B88 [29] G. Ben Arous. D´eveloppement asymptotique du noyau de la chaleur hypoelliptiquehors du cut-locus.
Ann. Sci. ´Ecole Norm. Sup. (4) , 21(3):307–331, 1988.
B84 [30] J.-M. Bismut.
Large deviations and the Malliavin calculus , volume 45 of
Progressin Mathematics . Birkh¨auser Boston, Inc., Boston, MA, 1984.
BCR98 [31] J. Bochnak, M. Coste, and M.-F. Roy.
Real algebraic geometry , volume 36 of
Ergeb-nisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Re-lated Areas (3)] . Springer-Verlag, Berlin, 1998. Translated from the 1987 Frenchoriginal, Revised by the authors.
BLU07 [32] A. Bonfiglioli, E. Lanconelli, and F. Uguzzoni.
Stratified Lie groups and potentialtheory for their sub-Laplacians . Springer Monographs in Mathematics. Springer,Berlin, 2007. 67
CH20 [33] M. Bonnefont, D. Chafa¨ı, and R. Herry. On logarithmic Sobolev inequalities for theheat kernel on the Heisenberg group.
Annales de la Facult´e des sciences de Toulouse: Math´ematiques , 29(2):335–355, 2020. identifier: 10.5802/afst.1633.
B82 [34] R. W. Brockett. Control theory and singular Riemannian geometry. In
New direc-tions in applied mathematics (Cleveland, Ohio, 1980) , pages 11–27. Springer, NewYork-Berlin, 1982.
CL08 [35] P. Cannarsa and L. Rifford. Semiconcavity results for optimal control problems ad-mitting no singular minimizing controls.
Ann. Inst. H. Poincar´e Anal. Non Lin´eaire ,25(4):773–802, 2008.
CWW95 [36] A. Carbery, S. Wainger, and J. Wright. Hilbert transforms and maximal functionsassociated to flat curves on the Heisenberg group.
J. Amer. Math. Soc. , 8(1):141–179, 1995.
CC15 [37] V. Casarino and P. Ciatti. A restriction theorem for M´etivier groups.
Adv. Math. ,245:52–77, 2013.
CFZ20 [38] M. Chatzakou, S. Federico, and B. Zegarli´nski. q -Poincar´e inequalities on CarnotGroups. arXiv e-prints , page arXiv:2007.04689, July 2020. C85 [39] M. Christ. Hilbert transforms along curves. I. Nilpotent groups.
Ann. of Math. (2) ,122(3):575–596, 1985.
C91 [40] M. Christ. L p bounds for spectral multipliers on nilpotent groups. Trans. Amer.Math. Soc. , 328(1):73–81, 1991.
CG84 [41] M. Christ and D. Geller. Singular integral characterizations of Hardy spaces onhomogeneous groups.
Duke Math. J. , 51(3):547–598, 1984.
C93 [42] T. Coulhon. It´eration de Moser et estimation gaussienne du noyau de la chaleur.
J.Operator Theory , 29(1):157–165, 1993.
CMZ96 [43] T. Coulhon, D. M¨uller, and J. Zienkiewicz. About Riesz transforms on the Heisen-berg groups.
Math. Ann. , 305(2):369–379, 1996.
CS08 [44] T. Coulhon and A. Sikora. Gaussian heat kernel upper bounds via the Phragm´en-Lindel¨of theorem.
Proc. Lond. Math. Soc. (3) , 96(2):507–544, 2008.
C79 [45] J. Cygan. Heat kernels for class 2 nilpotent groups.
Studia Math. , 64(3):227–238,1979.
DP89 [46] E. B. Davies and M. M. H. Pang. Sharp heat kernel bounds for some Laplaceoperators.
Quart. J. Math. Oxford Ser. (2) , 40(159):281–290, 1989.
DH05 [47] M. Del Hierro. Dispersive and Strichartz estimates on H-type groups.
Studia Math. ,169(1):1–20, 2005. 68
LLW19 [48] X. T. Duong, H.-Q. Li, J. Li, and B. D. Wick. Lower bound of Riesz transformkernels and commutator theorems on stratified nilpotent Lie groups.
J. Math. PuresAppl. (9) , 124:273–299, 2019.
E03 [49] P. Eberlein. The moduli space of 2-step nilpotent Lie algebras of type ( p, q ). In
Explorations in complex and Riemannian geometry , volume 332 of
Contemp. Math. ,pages 37–72. Amer. Math. Soc., Providence, RI, 2003.
E09 [50] N. Eldredge. Precise estimates for the subelliptic heat kernel on H -type groups. J.Math. Pures Appl. (9) , 92(1):52–85, 2009.
E10 [51] N. Eldredge. Gradient estimates for the subelliptic heat kernel on H -type groups. J. Funct. Anal. , 258(2):504–533, 2010.
FF15 [52] F. Ferrari and B. Franchi. Harnack inequality for fractional sub-Laplacians in Carnotgroups.
Math. Z. , 279(1-2):435–458, 2015.
FJ08 [53] A. Figalli and N. Juillet. Absolute continuity of Wasserstein geodesics in the Heisen-berg group.
J. Funct. Anal. , 255(1):133–141, 2008.
FR10 [54] A. Figalli and L. Rifford. Mass transportation on sub-Riemannian manifolds.
Geom.Funct. Anal. , 20(1):124–159, 2010.
F73 [55] G. B. Folland. A fundamental solution for a subelliptic operator.
Bull. Amer. Math.Soc. , 79:373–376, 1973.
F75 [56] G. B. Folland. Subelliptic estimates and function spaces on nilpotent Lie groups.
Ark. Mat. , 13(2):161–207, 1975.
FS74 [57] G. B. Folland and E. M. Stein. Estimates for the ¯ ∂ b complex and analysis on theHeisenberg group. Comm. Pure Appl. Math. , 27:429–522, 1974.
FS82 [58] G. B. Folland and E. M. Stein.
Hardy spaces on homogeneous groups , volume 28of
Mathematical Notes . Princeton University Press, Princeton, N.J.; University ofTokyo Press, Tokyo, 1982.
FLW95 [59] B. Franchi, G. Lu, and R. L. Wheeden. Representation formulas and weightedPoincar´e inequalities for H¨ormander vector fields.
Ann. Inst. Fourier (Grenoble) ,45(2):577–604, 1995.
FSS01 [60] B. Franchi, R. Serapioni, and F. Serra Cassano. Rectifiability and perimeter in theHeisenberg group.
Math. Ann. , 321(3):479–531, 2001.
FSS03 [61] B. Franchi, R. Serapioni, and F. Serra Cassano. On the structure of finite perimetersets in step 2 Carnot groups.
J. Geom. Anal. , 13(3):421–466, 2003.
FL12 [62] R. L. Frank and E. H. Lieb. Sharp constants in several inequalities on the Heisenberggroup.
Ann. of Math. (2) , 176(1):349–381, 2012.69 [63] B. Gaveau. Principe de moindre action, propagation de la chaleur et estim´ees souselliptiques sur certains groupes nilpotents. Acta Math. , 139(1-2):95–153, 1977.
GR07 [64] I. S. Gradshteyn and I. M. Ryzhik.
Table of integrals, series, and products . El-sevier/Academic Press, Amsterdam, seventh edition, 2007. Translated from theRussian, Translation edited and with a preface by Alan Jeffrey and Daniel Zwill-inger, With one CD-ROM (Windows, Macintosh and UNIX).
HK00 [65] P. Haj(cid:32)lasz and P. Koskela. Sobolev met Poincar´e.
Mem. Amer. Math. Soc. ,145(688):x+101, 2000.
HZ10 [66] W. Hebisch and B. Zegarli´nski. Coercive inequalities on metric measure spaces.
J.Funct. Anal. , 258(3):814–851, 2010.
H67 [67] L. H¨ormander. Hypoelliptic second order differential equations.
Acta Math. ,119:147–171, 1967.
HL10 [68] J.-Q. Hu and H.-Q. Li. Gradient estimates for the heat semigroup on H-type groups.
Potential Anal. , 33(4):355–386, 2010.
HM89 [69] H. Hueber and D. M¨uller. Asymptotics for some Green kernels on the Heisenberggroup and the Martin boundary.
Math. Ann. , 283(1):97–119, 1989.
H76 [70] A. Hulanicki. The distribution of energy in the Brownian motion in the Gaussianfield and analytic-hypoellipticity of certain subelliptic operators on the Heisenberggroup.
Studia Math. , 56(2):165–173, 1976.
J86 [71] D. Jerison. The Poincar´e inequality for vector fields satisfying H¨ormander’s condi-tion.
Duke Math. J. , 53(2):503–523, 1986.
JL88 [72] D. Jerison and J. M. Lee. Extremals for the Sobolev inequality on the Heisenberggroup and the CR Yamabe problem.
J. Amer. Math. Soc. , 1(1):1–13, 1988.
J82 [73] F. John.
Partial differential equations , volume 1 of
Applied Mathematical Sciences .Springer-Verlag, New York, fourth edition, 1982.
J09 [74] N. Juillet. Geometric inequalities and generalized Ricci bounds in the Heisenberggroup.
Int. Math. Res. Not. IMRN , (13):2347–2373, 2009.
J14 [75] N. Juillet. Diffusion by optimal transport in Heisenberg groups.
Calc. Var. PartialDifferential Equations , 50(3-4):693–721, 2014.
K80 [76] A. Kaplan. Fundamental solutions for a class of hypoelliptic PDE generated bycomposition of quadratic forms.
Trans. Amer. Math. Soc. , 258(1):147–153, 1980.
K95 [77] T. Kato.
Perturbation theory for linear operators . Classics in Mathematics. Springer-Verlag, Berlin, 1995. Reprint of the 1980 edition.
KR95 [78] A. Kor´anyi and H. M. Reimann. Foundations for the theory of quasiconformalmappings on the Heisenberg group.
Adv. Math. , 111(1):1–87, 1995.70
P13 [79] S. G. Krantz and H. R. Parks.
The implicit function theorem . Modern Birkh¨auserClassics. Birkh¨auser/Springer, New York, 2013. History, theory, and applications,Reprint of the 2003 edition.
LO14 [80] J. Lauret and D. Oscari. On non-singular 2-step nilpotent Lie algebras.
Math. Res.Lett. , 21(3):553–583, 2014.
LLMV13 [81] E. Le Donne, G. P. Leonardi, R. Monti, and D. Vittone. Extremal curves in nilpotentLie groups.
Geom. Funct. Anal. , 23(4):1371–1401, 2013.
LDMOPV16 [82] E. Le Donne, R. Montgomery, A. Ottazzi, P. Pansu, and D. Vittone. Sard propertyfor the endpoint map on some Carnot groups.
Ann. Inst. H. Poincar´e Anal. NonLin´eaire , 33(6):1639–1666, 2016.
LDR17 [83] E. Le Donne and S. Rigot. Besicovitch covering property for homogeneous distanceson the Heisenberg groups.
J. Eur. Math. Soc. (JEMS) , 19(5):1589–1617, 2017.
Li06 [84] H.-Q. Li. Estimation optimale du gradient du semi-groupe de la chaleur sur legroupe de Heisenberg.
J. Funct. Anal. , 236(2):369–394, 2006.
Li07 [85] H.-Q. Li. Estimations asymptotiques du noyau de la chaleur sur les groupes deHeisenberg.
C. R. Math. Acad. Sci. Paris , 344(8):497–502, 2007.
Li09 [86] H.-Q. Li. Fonctions maximales centr´ees de Hardy-Littlewood sur les groupes deHeisenberg.
Studia Math. , 191(1):89–100, 2009.
Li10 [87] H.-Q. Li. Estimations optimales du noyau de la chaleur sur les groupes de typeHeisenberg.
J. Reine Angew. Math. , 646:195–233, 2010.
Li19 [88] H.-Q. Li. The Carnot-Carath´eodory distance on 2-step groups. 2019.
Li19O [89] H.-Q. Li. The Carnot-Carath´eodory distance on 2-step groups.
To appear in theproceedings of the th ICCM , 2019. LQ14 [90] H.-Q. Li and B. Qian. Centered Hardy-Littlewood maximal functions on Heisenbergtype groups.
Trans. Amer. Math. Soc. , 366(3):1497–1524, 2014.
LZ19 [91] H.-Q. Li and Y. Zhang. Revisiting the heat kernel on isotropic and nonisotropicHeisenberg groups.
Comm. Partial Differential Equations , 44(6):467–503, 2019.
LS95 [92] W. Liu and H. J. Sussman. Shortest paths for sub-Riemannian metrics on rank-twodistributions.
Mem. Amer. Math. Soc. , 118(564):x+104, 1995.
LP03 [93] F. Lust-Piquard. A simple-minded computation of heat kernels on Heisenberggroups.
Colloq. Math. , 97(2):233–249, 2003.
LP04 [94] F. Lust-Piquard. Riesz transforms on generalized Heisenberg groups and Riesztransforms associated to the CCR heat flow.
Publ. Mat. , 48(2):309–333, 2004.71
P10 [95] F. Lust-Piquard. Ornstein-Uhlenbeck semi-groups on stratified groups.
J. Funct.Anal. , 258(6):1883–1908, 2010.
Ma02 [96] V. Magnani.
Elements of geometric measure theory on sub-Riemannian groups .Scuola Normale Superiore, Pisa, 2002.
MaM16 [97] A. Martini and D. M¨uller. Spectral multipliers on 2-step groups: topological versushomogeneous dimension.
Geom. Funct. Anal. , 26(2):680–702, 2016.
M80 [98] G. M´etivier. Hypoellipticit´e analytique sur des groupes nilpotents de rang 2.
DukeMath. J. , 47(1):195–221, 1980.
MR20 [99] T. Mietton and L. Rizzi. Branching geodesics in sub-riemannian geometry.
Geom.Funct. Anal. , 30(4):1139–1151, 2020.
M85 [100] J. Mitchell. On Carnot-Carath´eodory metrics.
J. Differential Geom. , 21(1):35–45,1985.
MPAM06 [101] F. Monroy-P´erez and A. Anzaldo-Meneses. The step-2 nilpotent ( n, n ( n + 1) / J. Dyn. Control Syst. , 12(2):185–216, 2006.
MM16 [102] A. Montanari and D. Morbidelli. On the lack of semiconcavity of the subRiemanniandistance in a class of Carnot groups.
J. Math. Anal. Appl. , 444(2):1652–1674, 2016.
MM17 [103] A. Montanari and D. Morbidelli. On the subRiemannian cut locus in a model offree two-step Carnot group.
Calc. Var. Partial Differential Equations , 56(2):PaperNo. 36, 26, 2017.
Mont94 [104] R. Montgomery. Abnormal minimizers.
SIAM J. Control Optim. , 32(6):1605–1620,1994.
M02 [105] R. Montgomery.
A tour of subriemannian geometries, their geodesics and applica-tions , volume 91 of
Mathematical Surveys and Monographs . American MathematicalSociety, Providence, RI, 2002.
M90 [106] D. M¨uller. A restriction theorem for the Heisenberg group.
Ann. of Math. (2) ,131(3):567–587, 1990.
MR96 [107] D. M¨uller and F. Ricci. Solvability for a class of doubly characteristic differentialoperators on 2-step nilpotent groups.
Ann. of Math. (2) , 143(1):1–49, 1996.
MRS95 [108] D. M¨uller, F. Ricci, and E. M. Stein. Marcinkiewicz multipliers and multi-parameterstructure on Heisenberg (-type) groups. I.
Invent. Math. , 119(2):199–233, 1995.
MS04 [109] D. M¨uller and A. Seeger. Singular spherical maximal operators on a class of twostep nilpotent Lie groups.
Israel J. Math. , 141:315–340, 2004.
MS15 [110] D. M¨uller and A. Seeger. Sharp L p bounds for the wave equation on groups ofHeisenberg type. Anal. PDE , 8(5):1051–1100, 2015.72
S94 [111] D. M¨uller and E. M. Stein. On spectral multipliers for Heisenberg and relatedgroups.
J. Math. Pures Appl. (9) , 73(4):413–440, 1994.
MS99 [112] D. M¨uller and E. M. Stein. L p -estimates for the wave equation on the Heisenberggroup. Rev. Mat. Iberoamericana , 15(2):297–334, 1999.
My02 [113] O. Myasnichenko. Nilpotent (3 ,
6) sub-Riemannian problem.
J. Dynam. ControlSystems , 8(4):573–597, 2002.
NRS01 [114] A. Nagel, F. Ricci, and E. M. Stein. Singular integrals with flag kernels and analysison quadratic CR manifolds.
J. Funct. Anal. , 181(1):29–118, 2001.
NSW85 [115] A. Nagel, E. M. Stein, and S. Wainger. Balls and metrics defined by vector fields.I. Basic properties.
Acta Math. , 155(1-2):103–147, 1985.
Q11 [116] B. Qian. Positive curvature property for some hypoelliptic heat kernels.
Bull. Sci.Math. , 135(3):262–278, 2011.
RS87 [117] F. Ricci and E. M. Stein. Harmonic analysis on nilpotent groups and singularintegrals. I. Oscillatory integrals.
J. Funct. Anal. , 73(1):179–194, 1987.
R13 [118] L. Rifford. Ricci curvatures in Carnot groups.
Math. Control Relat. Fields , 3(4):467–487, 2013.
R14 [119] L. Rifford.
Sub-Riemannian geometry and optimal transport . SpringerBriefs inMathematics. Springer, Cham, 2014.
RT05 [120] L. Rifford and E. Tr´elat. Morse-Sard type results in sub-Riemannian geometry.
Math. Ann. , 332(1):145–159, 2005.
R04 [121] S. Rigot. Counter example to the Besicovitch covering property for some Carnotgroups equipped with their Carnot-Carath´eodory metric.
Math. Z. , 248(4):827–848,2004.
R05 [122] S. Rigot. Mass transportation in groups of type H . Commun. Contemp. Math. ,7(4):509–537, 2005.
RS17 [123] L. Rizzi and U. Serres. On the cut locus of free, step two Carnot groups.
Proc.Amer. Math. Soc. , 145(12):5341–5357, 2017.
RS76 [124] L. P. Rothschild and E. M. Stein. Hypoelliptic differential operators and nilpotentgroups.
Acta Math. , 137(3-4):247–320, 1976.
S90 [125] L. Saloff-Coste. Analyse sur les groupes de Lie `a croissance polynˆomiale.
Ark. Mat. ,28(2):315–331, 1990.
S96 [126] A. Sikora. Sharp pointwise estimates on heat kernels.
Quart. J. Math. Oxford Ser.(2) , 47(187):371–382, 1996.
S04 [127] A. Sikora. Riesz transform, Gaussian bounds and the method of wave equation.
Math. Z. , 247(3):643–662, 2004. 73 [128] G. Stefani. Generalized Bakry- ´Emery curvature condition and equivalent entropicinequalities in groups. arXiv e-prints , page arXiv:2008.13731, Aug. 2020. S86 [129] R. S. Strichartz. Sub-Riemannian geometry.
J. Differential Geom. , 24(2):221–263,1986.
S89 [130] R. S. Strichartz. Corrections to: “Sub-Riemannian geometry” [J. DifferentialGeom. (1986), no. 2, 221–263; MR0862049 (88b:53055)]. J. Differential Geom. ,30(2):595–596, 1989.
ERS99 [131] A. F. M. ter Elst, D. W. Robinson, and A. Sikora. Riesz transforms and Lie groupsof polynomial growth.
J. Funct. Anal. , 162(1):14–51, 1999.
VSC92 [132] N. T. Varopoulos, L. Saloff-Coste, and T. Coulhon.
Analysis and geometry ongroups , volume 100 of