Singular conformally invariant trilinear forms, I Multiplicity one results
aa r X i v : . [ m a t h . R T ] F e b Singular conformally invariant trilinear forms, Ithe multiplicity one theorem
Jean-Louis ClercFebruary 15, 2015
Abstract A normalized holomorphic family (depending on λ ∈ C ) of con-formally invariant trilinear forms on the sphere is studied. Its zero set Z is described. For λ / ∈ Z , the multiplicity of the space of conformallyinvariant trilinear forms is shown to be 1. Introduction
The present paper is a continuation of the study of conformally invariant tri-linear forms for three representations belonging to the scalar principal seriesof the conformal group G = SO (1 , n ). More precisely, the representationsare realized on C ∞ ( S ), where S ≃ S n − is the unit sphere in a n -dimensionalEuclidean space E , and trilinear forms are required to be continuous for thenatural topology on C ∞ ( S ) × C ∞ ( S ) × C ∞ ( S ).In the reference [6], co-authored with B. Ørsted, the generic case wasstudied. The trilinear forms K λ are formally defined by an integral formula,depending on a parameter λ = ( λ , λ , λ ) in C , each λ j indexing a rep-resentation of the scalar principal series of G . The domain of convergenceis described, and the meromorphic extension is obtained, showing simplepoles along four families of parallel equally spaced planes in C . A genericmultiplicity one theorem (valid for λ not a pole) is proved for the space oftrilinear conformally invariant forms.In a second article [1], coauthored with R. Beckmann, the residues alongthe planes of poles were computed, at least generically. The planes of poles1re partitioned in two types, those of type I and those of type
II. Viewedas a distribution on S × S × S , the residue of K λ at a pole of type I is asingular distribution, supported on a submanifold of codimension ( n − covariant differential operators on S , while the residue ata pole of type II is supported on a submanifold of codimension 2( n −
1) (thediagonal in S × S × S ) and its expression requires covariant bi-differentialoperators (mapping functions on S × S to functions on S ).In the present paper the normalized holomorphic family e K λ (obtainedfrom K λ by multiplication by appropriate inverse Γ factors) is introducedand studied. The first result is the determination of the zero set Z of theholomorphic function λ e K λ . The set Z is contained in the set of polesand is of codimension 2 ( Z is a denumerable union of lines in C ). Thedetermination of Z uses K -harmonic analysis on S × S × S , where K ≃ SO ( n )is the maximal compact subgroup of G . The explicit computation of someintegrals (called Bernstein-Reznikov integrals) (see [2, 8, 6, 7]) plays a majorrole in this approach.The second main result is an extension of the generic multiplicity onetheorem. For λ ∈ C , let T ri ( λ ) be the space of conformally invarianttrilinear forms with respect to the three representations indexed by λ . Theorem 0.1. λ ∈ C \ Z = ⇒ dim T ri ( λ ) = 1 . An equivalent formulation is that
T ri ( λ ) = C e K λ for λ / ∈ Z .To obtain this result, we first reformulate Bruhat’s theory ([3]). Theconcept of a distribution smoothly supported on a submanifold is introduced,which allows, for such a distribution, the definition of its transverse symbol (see [15] for similar ideas). This gives more flexibility for studying ”nearlyinvariant” distributions supported on a submanifold, and this is crucial insection 6. The strategy for the proof of Theorem 0.1 is presented with moredetails at the beginning of section 6.T. Oshima (personal communication) observed that dim T ri ( λ ) ≥ λ ∈ Z . This (Theorem 7.4)is obtained by using a general result on theclosure of a meromorphic family of distributions (Lemma 6.3 in [18]). Hence λ ∈ C \ Z ⇐⇒ dim T ri ( λ ) = 1 . Throughout this paper, we assume that n ≥
4. The methods of thepresent paper could be (to the price of slight modifications) developed alsofor n = 2 or n = 3, but these two cases are special. This can be observed al-ready in Liouville’s theorem on local conformal diffeomorphisms, for which2 and S stand apart. Perhaps a deeper insight into the difference canbe obtained from the geometric structure of spheres when viewed as sym-metric R-spaces . Recall that a symmetric R-space is a homogeneous space S = G/P where G is a semi-simple Lie group, and P a parabolic subgroup,such that, denoting by K a maximal compact subgroup of G , the space S ,viewed as S ≃ K/ ( K ∩ P ) is a compact Riemannian symmetric space. Asa major result, a symmetric R-space is a real form of a compact Hermitianspace . Now observe that S is a real form of P ( C ), which is a Hermitiancompact symmetric space of rank 1, S ≃ P ( C ) is a compact Hermitiansymmetric space (hence a real form of P ( C ) × P ( C ), a product of twoHermitian symmetric spaces of rank 1), whereas, for d ≥ S d is a real formof the complex projective quadric Q d ( C ), which is an irreducible compactHermitian symmetric space of rank 2 (see [4] for more details on the geom-etry and analysis of symmetric R-spaces). For n = 3, which corresponds tothe Riemann sphere S ≃ P ( C ) as a homogenous space for SL ( C ), theconformally invariant trilinear forms where studied by Oksak in [16] (evenmore generally for representations of the principal series which are not nec-essarily scalar). He obtained fairly complete results, making heavy use ofthe complex nature of S , and his techniques do not seem to be transposableto higher dimensional spheres.Let us mention connections to two other problems. First, the space T ri ( λ ) is isomorphic to the space Hom G ( π λ ⊗ π λ , π − λ ) (consequence ofa lemma due to Poulsen, see [20]). From this point of view, the presentstudy is a special case of the restriction program designed by T. Kobayashiand collaborators, where the ”big group” is G × G and the ”small group”is diag ( G ) ≃ G , the representations are π λ ⊗ π λ as a representation of G × G , and π − λ as a representation of G . In this respect, the reference [14]was a source of inspiration for the present paper. A second problem con-cerns those invariant trilinear forms, which, when viewed as distributions on S × S × S , are supported in the diagonal. They are expressed by covariantbi-differential operators . Those have been studied (see [19, 21]), also in con-nection with problems about tensor product of generalized Verma modules(see [21, 13],) and they deserve further study. Summary0. Introduction1. Conformal geometry and analysis on the sphere S × S × S . The holomorphic family e K λ K -analysis of e K λ K -invariant polynomial functions on S × S × S K -analysis of e K λ e K λ
4. The zero set Z of e K λ e K λ e K λ e K λ
5. Distributions smoothly supported on a submanifold6. The theorem of the support
7. Multiplicity 1 results
References
Let S ≃ S n − be the unit sphere in a Euclidean space E of dimension n .The Euclidean distance is denoted by | x − y | . Let G = SO ( n,
1) be theconnected component of the neutral element in the Lorentz group. Theaction of G on S is by conformal transformations. For g ∈ G and x ∈ S , the conformal factor κ ( g, x ) of g at x is defined by the relation | Dg ( x ) ξ | = κ ( g, x ) | ξ | , (1)for any vector ξ in the tangent space T x ( S ) of S at x .The restriction to the sphere of the Euclidean distance satisfies an im-portant covariance relation under the action of G , namely | g ( x ) − g ( y ) | = κ ( g, x ) | x − y | κ ( g, y ) (2)4or x, y ∈ S, g ∈ G .The subgroup K ≃ SO ( n ) can be identified with a maximal compactsubgroup of G . It acts transitively on S . Let dx be the measure on S induced by the Euclidean structure. It transforms under the action of G bythe rule d ( g ( x )) = κ ( g, x ) n − dx , for g ∈ G .Let E ,n = R ⊕ E equipped with the Lorentzian form [( x , x )] = x −| x | .The Lie algebra g ≃ o (1 , n ) of the Lie group G is realized in matrix form as= ( u t u X , u ∈ E, X ∈ o ( E ) ) . The Cartan decomposition of g w.r.t. the standard Cartan involution is g = k ⊕ p , where k = ( (cid:18) X (cid:19) , X ∈ o ( E ) ) ≃ o ( n ) , p = ( X u = (cid:18) u t u (cid:19) , u ∈ E ) ≃ E Fix an orthonormal basis { e , e , . . . , e n } of E , and choose = (1 , , . . . , S . The stabilizer of in G is a parabolic subgroup P whichcontains in particular the Abelian subgroup ( a t ) t ∈ R , where a t = cosh t sinh t . . . . . . sinh t cosh t . . . . . . . . . . Notice that a t = exp( tX e ). These elements act on S by the formula x = x x ... x n a t ( x ) = sinh t + x cosh t cosh t + x sinh tx cosh t + x sinh t ... x n cosh t + x sinh t , and κ ( a t , x ) = (cosh t + x sinh t ) − . To any X ∈ p , we denote by e X thevector field on S given by e Xf ( x ) = ddt f (exp tX ( x )) | t =0 . emma 1.1. Let X ∈ p ≃ E . Then ddt κ (exp tX, x ) | t =0 = −h X, x i . (3) Proof.
Let X = X e . Then exp tX = a t , and κ ( a t , x ) = (cosh t + x sinh t ) − =1 − x t + O ( t ). Hence ddt κ (exp tX, x ) | t =0 = − x = −h X, x i . The generalcase follows, as it is always possible to choose the orthonormal basis of E such that X is (a multiple of) e . For λ ∈ C , let π λ be the representation ( spherical principal series ) realizedon C ∞ ( S ) by π λ ( g ) f ( x ) = κ ( g − , x ) ρ + λ f (cid:0) g − ( x ) (cid:1) , where ρ = n − . Observe that for k ∈ K , π λ ( k ) is independant of λ andcoincides with the regular action of K on C ∞ ( S ). Lemma 1.2.
Let X ∈ p . Then dπ λ ( X ) f ( x ) = − e Xf ( x ) + ( λ + ρ ) h X, x i f ( x ) . (4) Proof. dπ λ ( X ) f ( x ) = ddt (cid:0) κ (exp − tX ) , x ) λ + ρ f (exp − tX ( x )) (cid:1) | t =0 = − e Xf ( x ) + ( λ + ρ ) h X, x i f ( x )by using (3).Recall the duality relation for π λ and π − λ given by Z S (cid:0) π − λ ( g ) ϕ (cid:1) ( x ) ψ ( x ) dx = Z S ϕ ( x ) (cid:0) π λ ( g − ) ψ (cid:1) ( x ) dx , (5)where ϕ, ψ ∈ C ∞ ( S ) , g ∈ G .Finally, recall that the representation π λ is irreducible, unless λ ∈ (cid:0) − ρ − N (cid:1) ∪ (cid:0) ρ + N (cid:1) . .3 Geometry of orbits in S × S × S The action of G can be extended ”diagonally” to S × S × S by g ( x, y, z ) = (cid:0) g ( x ) , g ( y ) , g ( z ) (cid:1) . The next proposition recalls the structure of G -orbits forthis action. Proposition 1.1.
There are five orbits in S × S × S under the action of G .Denoting by ( x, y, z ) a generic element of S × S × S , they are given by O = { x = y, y = z, z = x }O = { y = z = x } , O = { z = x = y } , O = { x = y = z } , O = { x = y = z } . The orbit O is open and dense in S × S × S , O is closed, and for j = 1 , , , O j = O j ∪ O . See e.g. [6] Proposition 1.2. e K λ Let λ , λ , λ ∈ C . A continuous trilinear form T on C ∞ ( S ) ×C ∞ ( S ) ×C ∞ ( S )is invariant w.r.t. ( π λ , π λ , π λ ) if T (cid:0) π λ ( g ) f , π λ ( g ) f , π λ ( g ) f (cid:1) = T ( f , f , f )for all f , f , f ∈ C ∞ ( S ) and g ∈ G .By Schwartz’s kernel theorem, such a trilinear form can also be viewedas a distribution on S × S × S , still denoted by T . Let λ = ( λ , λ , λ ) andlet π λ be the representation on C ∞ ( S × S × S ) (= the canonical completionof the tensor product π λ ⊗ π λ ⊗ π λ ) given by π λ ( g ) f ( x, y, z ) = κ ( g − , x ) λ + ρ κ ( g − , y ) λ + ρ κ ( g − , z ) λ + ρ f ( g − ( x ) , g − ( y ) , g − ( z )) . Then the invariance condition simply reads T ( π λ ( g ) f ) = T ( f )for all f ∈ C ∞ ( S × S × S ) and g ∈ G . When this is verified, we also saysimply that T is λ -invariant .For λ ∈ C , let T ri ( λ ) be the space of λ -invariant distributions on S × S × S . When n = 2, there are six orbits, two of them being open. .1 The construction of the generic family In [6], B. Ørsted and the present author constructed trilinear invariant formsfor the family of representations ( π λ ) λ ∈ C . Given λ = ( λ , λ , λ ) ∈ C , let α = ( α , α , α ) be given by α = − ρ − λ + λ + λ α = − ρ + λ − λ + λ α = − ρ + λ + λ − λ (6)Relations (6) can be inverted to yield λ = ρ + α + α λ = ρ + α + α λ = ρ + α + α . (7)In the sequel, α = ( α , α , α ) (called the geometric parameter describ-ing the singularities of K α ) and λ = ( λ , λ , λ ) (called the spectral param-eter describing the invariance of K λ ) will be considered as two associatedparameters on the same space C , related by the relations (6) or (7).Let λ ∈ C and let α be its associated geometric parameter. For f ∈C ∞ ( S × S × S ), let K λ ( f ) = K α ( f ) = Z Z Z S × S × S | x − y | α | y − z | α | z − x | α f ( x, y, z ) dx dy dz . (8)The formula defines (formally) a map on C ∞ ( S × S × S ), which is invariant under π λ . Less formally, assume that Supp ( f ) ⊂ O . Then the integralmakes sense and thus defines a distribution K α , O on the open orbit O .The integral (8) is absolutely convergent for all f ∈ C ∞ ( S × S × S ) ifand only if ℜ α j > − ( n − , j = 1 , , , ℜ ( α + α + α ) > − n − . The integral can be meromorphically continued to C , with simple polesalong four families of planes in C , α j = − ( n − − k j , k j ∈ N for j = 1 , α + α + α = − n − − k, k ∈ N . α is said to be • of type I j if α j ∈ − ( n − − N (and in general of type I if it is oftype I j for some j ∈ { , , } ) • of type II if α + α + α ∈ − n − − N • of type I+II if α is at the same time a pole of type I and a pole oftype II. Definition 2.1.
A pole α is said to be generic if α belongs to a unique plane of poles. Introduce the normalized trilinear invariant functional e K α defined by e K α = K α Γ( ρ + α )Γ( ρ + α )Γ( ρ + α )Γ( α + α + α + 2 ρ ) , or similarly, e K λ = K λ Γ( λ + λ + λ + ρ )Γ( − λ + λ + λ + ρ )Γ( λ − λ + λ + ρ )Γ( λ + λ − λ + ρ ) . Both definitions define entire (distribution-valued) functions in C , as thegeneric poles of K α are simple and by using Hartog’s prolongation principle.Notice that a pole α is generic if and only if among the four Γ factors inthe normalization process, exactly one is singular. The following calculation was achieved in [6] (see also [7, 8]), extending anearlier result in the case n = 2 (see [2]). Proposition 2.1.
Let α = ( α , α , α ) and assume it is not a pole. Then Z S × S × S | x − y | α | y − z | α | z − x | α dx dy dz = (cid:0) π (cid:1) ( n − α + α + α Γ (cid:0) α + α + α + 2 ρ (cid:1) Γ( α + ρ ) Γ( α + ρ ) Γ( α + ρ )Γ( α + α + 2 ρ ) Γ( α + α + 2 ρ ) Γ( α + α + 2 ρ )This formula can be equivalently written as e K α (1 , ,
1) = (cid:0) π (cid:1) ( n − α + α + α Γ( α + α + 2 ρ ) Γ( α + α + 2 ρ ) Γ( α + α + 2 ρ ) (9)9r, in terms of the spectral parameter λ = ( λ , λ , λ ) e K λ (1 , ,
1) = (cid:18) √ π (cid:19) n − λ + λ + λ Γ( ρ + λ ) Γ( ρ + λ ) Γ( ρ + λ ) . (10) K -analysis of e K λ K -invariant polynomial functions on S × S × S The group K ≃ SO ( n ) acts on S by rotations. A K -invariant function on S × S × S is a function on S × S × S which is invariant under the diagonalaction of K on S × S × S . A K -invariant distribution on S × S × S is definedby duality.A polynomial function function on S (resp. S × S × S ) is by definition therestriction to S (resp. to S × S × S ) of a polynomial on E (resp. E × E × E ). Lemma 3.1. A K -invariant polynomial function on S × S × S is the re-striction to S × S × S of a K -invariant polynomial on E × E × E .Proof. Let P be a polynomial on E × E × E , and assume that its restrictionto S × S × S , say p = P | S × S × S is K -invariant. Then let for x, y, z ∈ EQ ( x, y, z ) = Z K P ( kx, ky, kz ) dk , where dk is the normalized Haar measure on K . Then Q is a K -invariantpolynomial. When x, y, z belong to S , Q ( x, y, z ) = R K p ( kx, ky, kz ) dk = p ( x, y, z ) by the K -invariance of p , hence p = Q | S × S × S .With some abuse, we will often use the same notation for a polynomialon E × E × E and its restriction to S × S × S . Lemma 3.2.
The space of K -invariant polynomial functions is dense in thespace of K -invariant functions in C ∞ ( S × S × S ) .Proof. By K × K × K -Fourier analysis, the space of polynomial functionson S × S × S is dense in C ∞ ( S × S × S ). As the topology on C ∞ ( S × S × S )can be defined by K -invariant semi-norms (e.g. the Sobolev semi norms k (∆ x + ∆ y + ∆ z ) N f k ), K -invariant polynomial functions are dense in thespace of K -invariant functions in C ∞ ( S × S × S ).10 emma 3.3. The algebra of K -invariant polynomial functions on S × S × S is generated (as an algebra) by the polynomial functions | x − y | , | y − z | , | z − x | . Proof.
By the first fundamental theorem (see e.g. [10]), the algebra of K -invariant polynomials on E × E × E is generated by the polynomials | x | , | y | , | z | , < x, y >, < y, z >, < z, x > . Hence, by Lemma 3.1, the algebra of K -invariant polynomial functions on S × S × S is generated by the restrictions to S × S × S of the previouspolynomials, that is to say by1 , < x, y >, < y, z >, < z, x > . But for x, y ∈ S , < x, y > = 1 − | x − y | , so that {| x − y | , | y − z | , | z − x | } is also a generating family. Remark.
The assumption n ≥ n = 3,the algebra of polynomials on E × E × E which are invariant under SO (3)is generated by | x | , | y | , | z | , < x, y >, < y, z >, < z, x > and det( x, y, z ).For n ≥
4, the algebra of SO ( n )-invariant polynomials is the same as thealgebra of O ( n )-invariant polynomials. Lemma 3.4.
Let T be a K -invariant distribution on S × S × S . Then T ≡ if and only if ( T , p ) = 0 for any K -invariant polynomial function p on S × S × S .Proof. The only if part being trivial, assume that T satisfies ( T , p ) = 0 forany K -invariant polynomial function p on S × S × S . Lemma 3.2 impliesthat ( T , ϕ ) = 0 for all K -invariant C ∞ functions ϕ on S × S × S . Let f ∈ C ∞ ( S × S × S ). For k ∈ K , let f k be the function defined by f k ( x, y, z )) = f ( kx, ky, kz ). By K -invariance of T , ( T , f k ) = ( T , f ). Let ϕ be defined by ϕ ( x, y, z )) = R K f ( kx, ky, kz )) dk . Then ϕ is a K -invariant C ∞ function on( S × S × S ). Now ( T , f ) = ( T , Z K f k dk ) = ( T , ϕ ) = 0 , hence T = 0. 11 .2 K -analysis of e K λ For a , a , a ∈ N , let p a ,a ,a be the polynomial function on S × S × S defined by p a , a , a ( x, y, z ) = | x − y | a | y − z | a | z − x | a . Proposition 3.1.
Let T be a K -invariant distribution on S × S × S . Then T ≡ if and only if T ( p a , a ,a ) = 0 for any a , a , a ∈ N .Proof. The proposition is a consequence of Lemma 3.3 and Lemma 3.4.Recall the
Pochhammer symbol ( x ) n which is defined for x ∈ C and n ∈ N by ( x ) = 1 , ( x ) = x, ( x ) n = x ( x + 1) . . . ( x + n − . Observe that ( x ) n = 0 if and only if x ∈ − N and n > − x . Proposition 3.2.
Let α ∈ C , and let a , a , a ∈ N . I α ( a , a , a ) := e K α ( p a , a , a ) = (cid:0) π (cid:1) ( n − α + α + α a + a + a ) . . . (11) (cid:0) α + α + α + 2 ρ (cid:1) a + a + a (cid:0) α + ρ (cid:1) a (cid:0) α + ρ (cid:1) a (cid:0) α + ρ (cid:1) a Γ( α + α + 2 ρ + a + a ) Γ( α + α + 2 ρ + a + a ) Γ( α + α + 2 ρ + a + a ) . Proof.
Suppose first that α is not a pole of K α . Then K α ( p a ,a ,a ) = K α +2 a , α +2 a , α +2 a (1 ⊗ ⊗ , whose value is known by (9). Then take into account the normalizing factor,to get (3.1) for α not a pole. The two handsides of (11) being holomorphicon C are equal for all α in C .For further use, we also give the same result formulated in terms of thespectral parameter. I λ ( a , a , a ) = e K λ ( p a ,a ,a )= ( √ π n − λ + λ + λ a + a + a ) (cid:0) λ + λ + λ + ρ (cid:1) a + a + a ( − λ + λ + λ + ρ ) a ( λ − λ + λ + ρ ) a ( λ + λ − λ + ρ ) a Γ( λ + ρ + a + a )Γ( λ + ρ + a + a )Γ( λ + ρ + a + a ) . (12)12 .3 The support of e K λ Proposition 3.3.
Let λ ∈ C . i ) if λ is not a pole, Supp ( e K λ ) = S × S × S . ii ) if λ is a pole of type I j , j ∈ { , , } , then Supp ( e K λ ) ⊂ O j . iii ) if λ is a generic pole of type I j , j ∈ { , , } , then Supp ( e K λ ) = O j . iv ) if λ is a pole of type II, then Supp ( e K λ ) ⊂ O .Proof. The support of any λ -invariant distribution is a closed G -invariantsubset of S × S × S . If λ is not a of pole, then e K λ is a non zero multipleof K λ and hence has the same support. The restriction of K λ to O iscertainly not 0, and hence Supp ( K λ ) = S × S × S . Hence i ) is verified.Let λ be a generic pole (recall Definition 2.1), either of type I or oftype II . The normalizing factor in the definition of e K λ is a product ofthree Γ factors which are non singular at λ and only one Γ factor whichhas a simple pole at λ . Hence, e K λ is equal (up to a non zero scalar) to theresidue of K λ at λ . The residues are computed in [1], at least generically.More precisely, in each plane of poles, the residues are computed for λ in adense open subset. From the formulæ for the residues, it is easy to deducethe inclusion of their support as indicated in the proposition. The generalresult follows by analytic continuation in each plane of poles. Hence ii ) and iv ) hold true.Assume now that λ is a generic pole of type I, e.g. of type I . As Supp ( e K λ ) ⊂ O and Supp ( e K λ ) is invariant under G , either Supp ( e K λ ) = O or Supp ( e K λ ) ⊂ O . So, assume that Supp ( e K λ ) ⊂ O . By Schwartz’stheorem on the local structure of distributions supported by a submanifoldand the compactness of S , if a test function ϕ ∈ C ∞ ( S × S × S ) vanisheson O at a sufficiently large order, then e K λ ( ϕ ) = 0. For a , a ∈ N largeenough, the function p a , a , vanishes on O at an arbitrary large order. Acareful inspection of (3.1) shows that, for a , a large, e K λ ( p a , a , ) = 0, thusgetting a contradiction. Hence Supp ( e K λ ) = O . Z of e K λ Proposition 4.1.
For α ∈ C , the trilinear form e K α is identically if andonly if I α ( a , a , a ) = 0 for any a , a , a ∈ N .Proof. Viewed as a distribution on S × S × S , e K α is K -invariant. Hence wemay apply Proposition 3.1. 13he rest of this section is devoted to the (rather long) proof of thefollowing theorem. Theorem 4.1.
Let α = ( α , α , α ) ∈ C . The trilinear form e K α vanishesidentically if and only if (at least) one of the following properties (up to apermutation of the indices) is satisfied α = − ( n − − k , α = − ( n − − k (13) for some k , k ∈ N , or α + α + α = − n − − k, α = 2 l (14) for some k, l ∈ N . This theorem can be translated in terms of the spectral parameter λ . Theorem 4.2.
Let λ = ( λ , λ , λ ) . Then e K λ vanishes identically if andonly if (at least) one of the following properties (up to permutation of theindices) is satisfied λ = − ρ − l, λ − λ = m, l ∈ N , m ∈ Z , | m | ≤ l, l ≡ m (2) (15) λ = − ρ − l, λ + λ = m, l ∈ N , m ∈ Z , | m | ≤ l, l ≡ m (2) (16) Proof of the equivalence of the two formulations. • First case. Let α = − ( n − − k and α = − ( n − − k where k , k ∈ N . Then, by (7) λ = − k + α , λ = − k + α , λ = − ρ − k − k . Set l = k + k , and m = k − k , so that λ = − ρ − l, λ − λ = m . Then l ∈ N , m ∈ Z , l ≡ m (2), and | m | ≤ l .Conversely, if λ satisfies (15), set k = l − m , k = l + m . Then k , k ∈ N and α = − ( n − − k , α = − ( n − − k . • Second case. Let α + α + α = − n − − k and α = 2 l , forsome k, l ∈ N . Then, by (7) λ = ρ + l + α , λ = − ρ − k − α , λ = − ρ − k − l . Set l = k + l , m = l − k . Then l ∈ N , m ∈ Z , m ≡ l (2) and | m | ≤ l .Moreover, λ + λ = l − k = m, λ = − ρ − l .Conversely, if λ satisfies (16), set k = l − m , l = l + m . Then k, l ∈ N andthe statement follows. 14 .1 The first vanishing situation Proposition 4.2.
Let α be such that α = − ( n − − k , α = − ( n − − k , k , k ∈ N . Then e K α ≡ .Proof. The assumptions on α imply that α ρ = − k , α ρ = − k , α + α ρ = − k − k so that, for a , a , a ∈ N , (11) can be rewritten as I α ( a , a , a ) = ( − k ) a ( − k ) a Γ( − k − k + a + a ) × . . . If a + a ≤ k + k , the denominator is singular, so I α ( a , a , a ) = 0. If a + a > k + k , then either a > k or a > k . If for instance a > k ,then ( − k ) a = 0, so that I α ( a , a , a ) = 0. Proposition 4.3.
Let α be such that α + α + α = − n − − k, α = 2 l , k, l ∈ N . Then e K α ≡ .Proof. The assumptions on α imply that α + α + α n −
1) = − k, α + α n −
1) = − k − l so that for a , a , a ∈ N , (11) can be rewritten as I α ( a , a , a ) = ( − k ) a + a + a Γ( − k − l + a + a ) × . . . If a + a ≤ k + l , ( − k − l + a + a ) is a pole of Γ, so that I α ( a , a , a ) = 0.If a + a > k + l , then a + a + a > k + l ≥ k , so that ( − k ) a + a + a = 0.Hence, in any case, I α ( a , a , a ) = 0.15 .3 Necessary conditions for the vanishing We now prove that the conditions of Theorem 4.1 are necessary to have e K α ≡ Step 1 . Reduction of the problem
Proposition 4.4.
Let α be such that e K α ≡ . Then either α belongs to Z or, up to a permutation of the indices, α + α + α = − n − − k, α + α = − n − − l for some k, l ∈ N , k > l .Proof. If α is not a pole, then e K α is a (non zero) multiple of K α , and therestriction of K α to the open orbit O is non identically 0. Hence e K α ≡ α is a pole.Next, e K α ≡ I α (0 , ,
0) = 0. Hence some Γ factor in (9) has apole, or explicitly α i + α j + 2( n − ∈ − N (17)for some 1 ≤ i = j ≤ α a pole of type II, up to a permutation of the indices, thereexist k, l ∈ N , such that α + α + α = − n − − k, α + α = − n − − l . Let α be a pole of type I , say of type I , i.e. α = − ( n − − k , forsome k ∈ N . For a and a sufficiently large, (11) implies I α (0 , a , a ) = (cid:0) α + α + α ρ (cid:1) a + a + a (cid:0) α ρ (cid:1) a (cid:0) α ρ (cid:1) a × N V T (NVT = non vanishing term). Hence e K α ≡ • α + ρ ∈ − N (or α + ρ ∈ − N )or • α + α + α + 2 ρ ∈ − N .In the first case, α is also a pole of type I (or I ), hence α belongs to Z by (13). In the second case, we are back to the case of a pole of type II.Summing up, and up to a permutation of the indices, it remains toconsider the situation where α + α + α = − n − − k, α + α = − n − − l . for some k, l ∈ N . The two conditions imply α = − k + 2 l . If k ≤ l , then α is in Z . So, only the case where k > l remains open, and this is the contentof the proposition. 16 roposition 4.5. Let α satisfy α + α + α = − n − − k, α + α = − n − − l for some k, l ∈ N , k > l . Then either α belongs to Z or there exists a , a , a such that e K α ( p a ,a ,a ) = 0 .Proof. By (11) I α ( a , a , a ) can be written as( − k ) a + a + a ( α + ρ ) a ( α + ρ ) a ( α + ρ ) a Γ( − l + a + a ) Γ( α + α + ( n −
1) + a + a ) Γ( α + α + ( n −
1) + a + a )Choose a = 0 and a , a such that l < a + a ≤ k . Then ( − k ) a + a + a = 0and Γ( − l + a + a ) is finite, so that I α ( a , a , a ) is = 0 unless (perhaps)one of the following is true : • α ∈ − ( n − − N or α ∈ − ( n − − N • α + α ∈ − n − − N or α + α ∈ − ( n − − N .Up to a permutation of the indices 1 and 2, we are reduced to examinethe following cases: • case (A) α + α + α = − n − − k, α + α = − n − − l, α = − ( n − − m for some k, l, m ∈ N , k > l . • case (B) α + α + α = − n − − k, α + α = − n − − l, α + α = − n − − m for some k, l, m ∈ N , k > l, k > m .In each case, we have to prove that either α is in Z or there exists a , a , a such that I α ( a , a , a ) = 0. Step 2.
Case (A)From the assumptions, we get α = − ( n − − l − m ) . If l ≥ m , then α is in two planes of poles of type I, hence is in Z . So assumethat l < m . Conditions ( A ) imply α + α = − ( n − − k − l − m, α + α = − ( n −
1) + 2( m − k ) . Suppose that n − α = 2 l − k / ∈ − ( n − − N . So I α ( a , a , a ) is equal to( − k ) a + a + a ( − m ) a Γ( − l + a + a ) × N V T (NVT = non vanishing term). As l < k and l < m , it is possible tochoose a ∈ N such that a > l and a ≤ k, a ≤ m . Let a = a = 0.Then ( − k ) a + a + a = 0 , ( − m ) a = 0, and Γ( − l + a + a ) is finite, thus I α ( a , a , a ) = 0. • Suppose that n − α = − ( n −
1) + 2( n − − k + l ). Let p = n − − k + l ∈ Z . If p ≤
0, then α lies in two planes of poles of type Iand hence belongs to Z . Assume that p >
0. Now, α + α n − n − − m − k + l = − m + pα + α n −
1) = n −
12 + m − k . But n − + m − k > n − + l − k >
0. Hence I α ( a , a , a ) is equal to( − k ) a + a + a ( − m ) a Γ( − l + a + a )Γ( − m + p + a + a ) × N V T
First choose a such that m − p < a ≤ m and l ≤ a . Now choose a suchthat l < a + a ≤ k , and let a = 0. It follows that I α ( a , a , a ) = 0. Step 3 . Case (B)We now assume that α satisfy the conditions α + α + α = − n − − k, α + α = − n − − l, α + α = − n − − m , where l < k and m < k .We may further assume that α j / ∈ − ( n − − N , j = 1 , ,
3, otherwisewe are in case A .Now α = − n −
1) + 2 k − m − l . If k ≥ ( n −
1) + l + m , then α ∈ N , so that α ∈ Z by (14). So assume that k < ( n −
1) + l + m . Then α + α = − k − m − l ). • Assume first that 2 k − m − l < ( n − I α ( a , a , a ) = ( − k ) a + a + a Γ( − l + a + a )Γ( − m + a + a ) × N V T . a = sup( l, m ) + 1 , a = a = 0, the Γ factors are finite and the factorin the numerator is not 0 because l, m < k . • Assume now that 2 k − m − l ≥ ( n − k < l + m + ( n − ≤ k . Then I α ( a , a , a )= ( − k ) a + a + a Γ( − l + a + a )Γ( − m + a + a )Γ( − (2 k − l − m − ( n − a + a ) × N V T .
Let us exhibit three nonnegative integers a , a , a such that a + a ≥ l + 1 a + a ≥ m + 1 a + a > k − l − m − ( n − a + a + a ≤ k (18)because for such a choice, I α ( a , a , a ) = 0. Recall that k > l and k > m .Now the condition l + m +( n − ≤ k can be rewritten as ( k − l )+( k − m ) ≥ n −
1. Hence there exist integers p, q ≥ k − l − p ≥ , k − m − q ≥ , p + q = ( n − . As n ≥
4, either p or q is greater than or equal to 2. Up to a permutationof the indices, we may assume that p ≥
2. Now let a = k − l − p + 1 , a = k − m − q, a = m + l − k + ( n − − ≥
0. Moreover a + a = l + ( n − − q − a + a = m + ( n − − pa + a = 2 k − l − m − ( n −
1) + 1 a + a + a = k and conditions (18) are satisfied as ( n − − q − p − ≥ n − − p = q ≥ heorem 4.3 (generic multiplicity 1 theorem) . Assume that λ ∈ C is nota pole. Then K λ is, up to a scalar, the unique λ -invariant distribution on S × S × S . In other words, when λ is not a pole, dim T ri ( λ ) = 1 or, more precisely, T ri ( λ ) = C e K λ .The proof of this theorem used two results, which we now recall , becausethey will be needed in the sequel. Lemma 4.1.
Let α ∈ C and λ the associated spectral parameter. Let O be a G -invariant open set of S × S × S which contains O as a closedsubmanifold. Let T = 0 be a distribution in O , λ -invariant and such that Supp ( T ) ⊂ O . Then α = − ( n − − k for some k ∈ N . Moreover, if S is a λ -invariant distribution in O such that Supp ( S ) ⊂ O , then S isproportional to T . Lemma 4.2.
Let α ∈ C and λ the associated spectral parameter. Let T = 0 be a distribution on S × S × S which is λ -invariant and such that Supp ( T ) ⊂ O . Then α + α + α = − n − − k for some k ∈ N . The proof of both statements is obtained by standard application ofBruhat’s theory, and is essentially contained in [6]. The only exception isthe second statement of the first lemma, the multiplicity one result, butit is only a refinement and its proof can be considered as routine. Noticethat there is no multiplicity 1 assertion in the second lemma, a point whichreflects a basic difference between poles of type I and poles of type II.
Let E be a real vector space of dimension n , and F a subspace of codimension p . Choose coordinates ( x , x , . . . , x p , y p +1 . . . , y n ) such that F = { ( x, y ) ∈ R p × R q , x = x = x p = 0 } . For J = ( j , j , . . . , j p ) a multi-index, let x J = x j . . . x j p p , ∂ J = (cid:16) ∂∂x (cid:17) j . . . (cid:16) ∂∂x p (cid:17) j p . Let U be an open set in E . A (smooth) transverse differential operator on U is an operator from C ∞ ( U ) into C ∞ ( F ∩ U ) given by Df ( y ) = X J a J ( y ) ∂ J f ( y ) , f ∈ C ∞ ( U ) , y ∈ F ∩ U J runs through all multi-indices and ( a J ) is a locally finite family ofsmooth functions on F ∩ U . Notice that the functions a J are well determined,as can be observed by testing the operator D against the functions x J . Atransverse differential operator D is said to be of transverse order ≤ m if a J ≡ J such that | J | = j + j + · · · + j p > m .Let E ∗ be the dual space of E , and let F ⊥ = { ψ ∈ E ∗ , ψ | E = 0 } .The space F ⊥ is canonically isomorphic to ( E/F ) ∗ . The coordinate forms ξ j : ( x, y ) x j , ≤ j ≤ p form a basis of F ⊥ .Let D = P | J |≤ m a J ∂ J be a transverse differential operator on U oftransverse order ≤ m . Then its symbol σ m ( D ) is defined by σ m ( D )( y, ξ ) = X | J | = m a J ( y ) ξ J , where y ∈ F ∩ U and ξ ∈ F ⊥ . To see the intrinsic character of the symbol,there is a useful formula. For y ∈ F ∩ U and ξ ∈ F ⊥ , choose smoothfunctions f, ϕ on U such that f ( y ) = 1 and dϕ ( y ) = ξ . Then σ ( D )( y, ξ ) = lim t →∞ t − m e − tϕ D ( e tϕ f )( y ) . (19)Fix a Lebesgue measure dy on F . To any transverse differential operator D is associated the distribution T D given by T D ( f ) = Z U ∩ F Df ( y ) dy for any test function f ∈ C ∞ c ( U ).It is possible to characterize the distributions which are of this form. For T a distribution on U , denote by W F ( T ) the wave front set of T (see [11]ch. VIII). Lemma 5.1.
Let T be a distribution on U , with Supp ( T ) ⊂ F ∩ U . Thenthe two following properties are equivalent : i ) W F ( T ) ⊂ ( F ∩ U ) × F ⊥ ii ) There exists a unique transverse differential operator D on F ∩ U such that ( T, ϕ ) = Z F Dϕ ( y ) dy . For a proof see [1]. A distribution T which satisfy the conditions of theLemma is said to be smoothly supported on F .An important source of smoothly supported distribution is obtained bythe following result, which is a version in the present context of the principle”invariance implies smoothness”. 21 emma 5.2. Let X j , ≤ j ≤ m be a family of smooth vector fields on U such that ∀ y ∈ F ∩ U, ∀ j, ≤ j ≤ m, X j ( y ) ∈ F and { X j ( y ) , ≤ j ≤ m } generate F .
Let ( a j ) ≤ j ≤ m be a family of smooth functions on E and ( U j ) ≤ j ≤ m a familyof distributions on U smoothly supported in F ∩ U . Let T be a distributionon U which satisfies i ) Supp ( T ) ⊂ F ∩ Uii ) ∀ j, ≤ j ≤ m , ( X j + a j ) T = U j .Then T is smoothly supported on F ∩ U .Proof. Using Lemma 5.1, it suffices to prove that
W F ( T ) ⊂ ( F ∩ U ) × F ⊥ .Now for D any differential operator on U , W F ( T ) ⊂ Char ( D ) ∪ W F ( DT ) , where Char ( D ) ⊂ U × E ∗ is the characteristic set of D (see [11] Theorem8.3.1). Apply to D = X j + a j . Observe that W F ( U j ) ⊂ ( F ∩ U ) × F ⊥ byassumption. Next Supp ( T ) ⊂ F ∩ U , and for x ∈ F ∩ U, ξ ∈ E ∗ ,( x, ξ ) ∈ Char ( X j + a j ) ⇐⇒ ξ ( X j ( x )) = 0 . As the X j ( x ) generate F , it follows that ( x, ξ ) ∈ W F ( T ) implies ξ ∈ F ⊥ and the statement follows.Let E ′ be another vector space, also of dimension n and F ′ a subspaceof E ′ also of codimension p . Let Φ be a diffeomorphism on an open subset U ′ of E ′ which satisfies Φ( U ∩ F ) ⊂ U ′ ∩ F ′ . Let Φ ∗ f = f ◦ Φ. This formuladefines isomorphisms between C ∞ ( U ′ ) (resp. C ∞ ( U ′ ∩ F ′ )) and C ∞ ( U ) (resp C ∞ ( U ∩ F )). If D is a transverse differential operator on U of order ≤ m ,let Φ ∗ D = Φ ∗− ◦ D ◦ Φ ∗ . Then Φ ∗ D is a transverse differential operator on U ′ (relative to F ′ ) oftransverse order ≤ m . The transverse symbols of D and Φ ∗ D are related bythe formula σ m (Φ ∗ D )( y, ξ ) = Φ ∗ σ m ( D )( y, ξ ) = σ m ( D ) (cid:0) Φ − ( y ) , ξ ◦ d Φ − ( y ) − (cid:1) . (20)The formula is easily obtained from (19).22et T be a distribution on U smoothly supported on F , and let D T bethe associated transverse differential operator. Let Φ ∗ T be the distributionon U ′ defined by (Φ ∗ T, ϕ ) = (
T, ϕ ◦ Φ) , ϕ ∈ C ∞ c ( U ′ )Then Φ ∗ T is smoothly supported on U ′ ∩ F ′ . Fix a Lebesgue measure dx ′ on F ′ . Then there exists an associated transverse differential operator D Φ ∗ T on U ′ ∩ F . Lemma 5.3.
The transverse differential operator D Φ ∗ T is given by D Φ ∗ T = | det (cid:0) ( D Φ − ) | F ′ → F (cid:1) | Φ ∗ D T (21) Proof.
By definition,(Φ ∗ T, f ) = Z F ∩ U D ( f ◦ Φ)( y ) dy = Z F ∩ U ′ D ( f ◦ Φ) ◦ Φ − ( y ′ ) | det (cid:0) ( D Φ − ( y ′ )) | F (cid:1) | dy ′ by the change of variables y ′ = Φ( y ). The result follows.For T a distribution smoothly supported on U ∩ F , which is of transverseorder less than m , define its transverse symbol σ m ( T ) by σ m ( T )( x, ξ ) = σ m ( D T )( x, ξ ) dx . These notions can be generalized to the setting of a regular submanifold N ofa manifold M to get definitions of a transverse differential operator on N , ofa distribution on M smoothly supported on N and of its transverse symbol(after the choice of a smooth measure is chosen on N ). In particular, thetransverse symbol of a differential operator of order ≤ m is a section of thebundle S m N ( m -symmetric tensor product of the conormal bundle on N ),whereas the transverse symbol of a distribution on M smoothly supportedin N and of transverse order ≤ m is a section of the bundle S m N ⊗ | Λ | ,where | Λ | is the density bundle of N (this is the reason to add ” dx ” in thedefinition of the transverse symbol of a distribution). We omit details (see[15] for related ideas).The main properties of the symbol map are summarized in the nextproposition. Proposition 5.1.
Let N be a regular submanifold of a manifold M . Let T be a distribution smoothly supported in N , of transverse order ≤ m . Let σ m ( D ) be the transverse symbol of T . ) let Φ be a diffeormorphism of M , such that Φ( N ) = N . Then Φ ∗ T issmoothly supported on N , of transverse order less than or equal to m andits transverse symbol is given by σ m (Φ ∗ T )( x, ξ ) = (cid:12)(cid:12) det (cid:0) D Φ − | T x N ( x ) (cid:1)(cid:12)(cid:12) σ m (Φ − ( x ) , ξ ◦ D Φ − ( x ) − (cid:1) .ii ) Let a be a smooth function on M . Then aT is smoothly supported on N and is of transverse order ≤ m . Its transverse symbol satisfies σ m ( aT ) = a σ m ( T ) . iii ) if σ T ≡ , then T is of transverse order ≤ m − . Let λ ∈ C be a pole and assume that λ / ∈ Z . The distribution e K λ is not0 (by the very definition of Z ), and is singular as stated in Proposition 3.3.The following result is a major step towards proving Theorem 0.1. Theorem 6.1.
Let λ be a pole and assume λ / ∈ Z . Let T = 0 be a λ -invariant distribution on S × S × S . Then Supp ( T ) = Supp ( e K λ ) . Recall that K α , O is the distribution on O defined by integration against | x − y | α | y − z | α | z − x | α . The distribution K α , O is λ -invariant. The mostdifficult step towards proving Theorem 6.1 is the following result. Theorem 6.2.
Let α be a pole and assume that α / ∈ Z . Let λ be the asso-ciated spectral parameter. Then the distribution K α , O cannot be extendedto S × S × S as a λ -invariant distribution. As the proof is long, let us sketch the main steps. • The distribution K α , O has a ”natural” extension (say F ) to S × S × S , given by the degree 0 coefficient in the Laurent expansion of themeromorphic function α α restricted to a (well chosen) complex linein C . • The distribution F is not λ -invariant, but ”almost” invariant, in thesense that, for each g ∈ G , F ◦ π λ ( g ) − F = E g where E g is a distribution (depending on g ) which is identically 0 on O .24 If T is a λ -invariant distribution extending K α , O , then S = F − T vanishes identically on O and satisfies S ◦ π λ ( g ) − S = E g . (22) • Both S and E g are supported in O ∪ O ∪ O ∪ O . Successively foreach orbit O j , it is possible to define and compute the transverse symbolsof both sides of (22). In O j , there is a point which is fixed by all a t , t ∈ R .Evaluate both symbols at this point. Studying their behavior as t varieseventually leads to a contradiction.Three separate cases have to be considered : the case of a generic pole oftype I (which automatically is not in Z ), the case of a generic pole of typeII not in Z , and the case of a pole of type I+II not in Z . The proofs relyon the same ideas, but are formally different. Details are given for the firstcase, we ask the the friendly reader to accept the more sketchy treatment ofthe last two cases. Assume α = ( α , α , α ) is a generic pole (recall Definition 2.1) of type I ,that is α = − ( n − − k for some k ∈ N . The assumption of genericityimplies that α / ∈ Z . It also implies that among the four Γ factors in theholomorphic normalization of K α only one (namely Γ( α + ρ )) becomessingular. Let s be a complex parameter, s = 0, | s | small. Let α ( s ) = ( α , α , α + 2 s ) , λ ( s ) = ( λ + s, λ + s, λ )and let F ( s ) = ( − k k ! Γ( α ρ ) Γ( α ρ ) Γ( α + α + α ρ + s ) e K α ( s ) . This is well defined distribution-valued function, at least for | s | small.Consider its Taylor expansion at 0 F ( s ) = F + sF + O ( s ) , (23)where F , F are distributions on S × S × S . The observation on the nor-malization factor implies that the distribution F is a non zero multiple of e K α , which, as α / ∈ Z , is not equal to 0. Hence F = 0. Moreover F is λ -invariant and Supp ( F ) ⊂ O . 25 emma 6.1. i ) The restriction of F to O is equal to K α , O . ii ) for any g ∈ G , F ◦ π λ ( g ) − F = − (ln κ ( g − , x ) κ ( g − , y )) F (24) iii ) for any X ∈ p ≃ EF ◦ d π λ ( X ) = −h X, x + y i F . (25) Proof.
Let ϕ be a test function supported in O . For s = 0, replace e K α ( s ) by its expression in term of K α ( s ) to get F ( s )( ϕ ) = ( − k k ! 1Γ( − k + s ) K α ( s ) , O ( ϕ ) . Let s → F ( ϕ ) = K α , O ( ϕ ) and i ) follows.Next, for f ∈ C ∞ ( S ) and arbitrary λ ∈ C , (cid:0) π λ + s ( g ) f (cid:1) ( x ) = κ ( g − , x ) s (cid:0) π λ ( g ) f (cid:1) ( x )and hence for f ∈ C ∞ ( S × S × S ) (cid:0) π λ ( s ) ( g ) f (cid:1) ( x, y, z ) = κ ( g − , x ) s κ ( g − , y ) s (cid:0) π λ ( g ) f (cid:1) ( x, y, z )= (cid:0) s ln (cid:0) κ ( g − , x ) κ ( g − , y ) (cid:1) + O ( s ) (cid:1) (cid:0) π λ ( g ) f (cid:1) ( x, y, z ) . Recall that F ( s ) ◦ π λ ( s ) ( g ) = F ( s ). Compare the Taylor expansion of bothsides to obtain (cid:0) F + ln (cid:0) κ ( g − , x ) κ ( g − , y ) (cid:1) F (cid:1) ◦ π λ ( g ) = F , (26)and use the λ -invariance of F to get ii ). Notice that, as the conformalfactor of an element in K (a rotation) is identically equal to 1, ii ) impliesthat F is K -invariant.For iii ), let X ∈ p . Then d π λ ( x ) F = 0 by the λ -invariance of F , andusing (3)ln (cid:0) κ (exp − tX, x ) κ (exp − tX, y ) (cid:1) = ln (cid:0) t ( h X, x i + h X, y i ) + O ( t ) (cid:1) = t h X, x + y i + O ( t ) . Let g = a t in (24) and take the derivative of both sides at t = 0 to get F ◦ d π λ ( X ) = −h X, x + y i F . α , α / ∈ − ( n − − N . Hence, on O ′ = S × S × S \ O = { ( x, y, z ) ∈ S × S × S, x = y } it is possible to extend the distribution K α , O by meromorphic continua-tion, to get a distribution on O ′ (denoted by K α , O ′ ) which is λ -invariant.Moreover, it is the only extension to O ′ which is λ -invariant. In fact, ifthere is one such, say T ′ , then the difference T ′ − K α , O ′ is supported in O ′ \ O = O ∪ O and λ -invariant. Now use twice Lemma 4.1 to concludethat this distribution has to be 0 : once for O = O ∪ O containing O , thesecond for O = O ∪ O containing O . On the other hand, the restriction to O ′ of F is 0, and hence, by (24), the restriction of F to O ′ is λ -invariant.Hence F coincides on O ′ with K α , O ′ . We now are in position to provethe following result which implies a fortiori Theorem 6.1 for the case of ageneric pole of type I.
Proposition 6.1.
There is no λ -invariant distribution on O ′ = S × S × S \ O which extends K α , O ′ .Proof. For convenience, when F is a distribution on S × S × S , denote by F ′ its restriction to O ′ . Assume T is a distribution on O ′ which extends K α , O ′ and is λ -invariant. Let S = T − F ′ . Then S is supported in O ,which is a closed regular submanifold of O ′ . Let g be the Lie algebra of G ,and for X ∈ g , let e X be the vector field on O ′ induced by the one-parametersubgroup exp tX, t ∈ R . Recall that G acts transitively on O , so that forany m ∈ O the vector space generated by { e X ( m ) , X ∈ g } is equal to T m O ,the tangent space to O at m .As both T and F ′ are K -invariant, ∀ X ∈ k , S ◦ d π λ ( X ) = 0 . (27)Now for X ∈ p , d π λ ( X )( x, y, z ) = − e X ( x, y, z )+( λ + ρ ) h X, x i +( λ + ρ ) h X, y i +( λ + ρ ) h X, z i is a first order differential operator of the form − e X + a X with a X a smoothfunction on S × S × S . Moreover T ◦ d π λ ( X ) = 0 as T is assumed to be λ -invariant. Restrict (25) to O ′ to get F ′ ◦ d π λ ( X ) = −h X, x + y i F ′ . Hence ∀ X ∈ p , S ◦ d π λ ( X ) = h X, x + y i F ′ . (28)As F ′ is invariant under π λ , F ′ is smoothly supported in O . Take intoaccount (27) and (28), use Lemma 5.2 to conclude that S is smoothly sup-ported in O . 27or any g ∈ G , (24) and the λ -invariance of T imply S ◦ π λ ( g ) − S = − ln (cid:0) κ ( g − , x ) κ ( g − , y ) (cid:1) F ′ . (29)The next elementary result is needed in order to make connection with thenotation and results of section 5. Lemma 6.2.
Let F be a distribution on a G -invariant open set in S × S × S .For any g ∈ G , F ◦ π λ ( g ) = a g (cid:0) g − (cid:1) ∗ F (30) where a g is given by a g ( x, y, z ) = κ ( g, x ) − ( λ + ρ ) κ ( g, y ) − ( λ + ρ ) κ ( g, z ) − ( λ + ρ ) . Proof.
Let ϕ ∈ C ∞ ( S ). As κ ( g − , x ) = κ ( g, g − x ) − , π λ ( g ) ϕ ( x ) = κ (cid:0) g − , x ) λ + ρ ϕ ( g − ( x ) (cid:1) = (cid:0) κ ( g, . ) − ϕ (cid:1) ( g − ( x )) . Hence for f ∈ C ∞ ( S × S × S ), π λ ( g ) f ( x ) = ( a g f )( g − ( x ))which amounts to π λ ( g ) f = ( g − ) ∗ ( a g f ). Hence (cid:0) F ◦ π λ ( g ) (cid:1) f = F ( π λ ( g ) f ) = F (( g − ) ∗ ( a g f )) = ( a g ( g − ) ∗ F )( f ) . From the computation of the residue of K λ at a generic pole of type I(see [1] Theorem 2.2), it is known that F has a (global) transverse orderalong O equal to 2 k . From (29) and (30) follows that S also has a globaltransversal order along O , say m , and m ≥ k .At a point p = ( x, x, z ) ∈ O with x = z , the tangent space to O is T p O = { ( u, u, v ) , u ∈ T x S, v ∈ T z S, } , the conormal space (as a subspace of the cotangent space) is given by N p = { n ξ = ( ξ, − ξ, , ξ ∈ T ∗ x S } Let g ∈ G such that g ( p ) = p . The Jacobian of the differential Dg ( p ) on T p O is equal to κ ( g, x ) n − κ ( g, z ) n − . So, using the transformation rules(20) and (21) and Lemma 5.1, (30) leads to28 m ( S ◦ π λ ( g )) (cid:0) ( x, x, z ) , n ξ (cid:1) = κ ( g, x ) − ( λ + λ ) κ ( g, z ) − λ + ρ σ m ( S ) (cid:0) ( x, x, z ) , n ξ ◦ Dg ( x ) − (cid:1) . Apply this to x = , z = − , and g = a t . Then a t ( ) = , a t ( − ) = − , Da t ( ) = e − t Id , Da t ( − ) = e t Id. Recall that λ + λ − λ + ρ = − k . So,after some computation, (29) implies (cid:0) e t (2 k − m ) − (cid:1) σ m ( S ) (cid:0) ( , , − ) , n ξ (cid:1) = − t σ m ( F ) (cid:0) ( , , − ) , n ξ (cid:1) (31)If m > k , the right hand side is 0, which forces σ m ( S ) (cid:0) , , − ) = 0 byand hence σ m ( S ) = 0 by (29), and hence S is of transverse order ≤ m − m = 2 k . But then, the left handside is 0. So σ k ( F , ( , , − )) = 0 and be the invariance of F , σ k ( F ) = 0on all of O . Hence (31) leads to a contradiction, thus proving Proposition6.1.The proof of Theorem 6.1 in the case of a generic pole of type I is noweasy. Let T be a distribution on S × S × S which is λ -invariant. Considerits restriction T O to O . As O is a unique orbit under G , T O has tobe a multiple of K α , O . By Proposition 6.1, T O has to be 0. Hence T issupported in O ∪ O ∪ O ∪ O . As α , α / ∈ − ( n − − N by assumption,two applications of Lemma 4.1 show that T is supported in O , which is thestatement to be proved. Proposition 6.2.
Let α be a generic pole of II, and assume that α is not in Z . Let λ be the associated spectral parameter. Then the distribution K α , O cannot be extended to a λ -invariant distribution on S × S × S .Proof. Let α = ( α , α , α ) be a generic pole of type II, i.e. α + α + α = − n − − k for some k ∈ N , which moreover is not in Z .The distribution K α , O can be extended in a unique way to a λ -invariantdistribution on O ′ . The proof is similar to the argument given in the proofof Proposition 6.1, as by assumption, α j / ∈ − ( n − − N for j = 1 , K α , O ′ the extension.Now, again by assumption, among the Γ-factors involved in the normal-ization of K α only one is singular at α , namely Γ( α + α + α + 2 ρ ). For s acomplex parameter, s = 0 , | s | small, define α ( s ) = ( α + 23 s, α + 23 s, α + 23 s ) , λ ( s ) = ( λ + 23 s, λ + 23 s, λ + 23 s ) . α ( s ) + α ( s ) + α ( s ) = α + α + α + 2 s . For s ∈ C , s = 0 and | s | small, let F ( s ) = ( − k k ! Γ (cid:16) α ( s )2 + ρ (cid:17) Γ (cid:16) α ( s )2 + ρ (cid:17) Γ (cid:16) α ( s )2 + ρ (cid:17) K α ( s ) The Taylor expansion of F ( s ) at s = 0 reads F ( s ) = F + sF + O ( s )where F , F are distributions on S × S × S .The distribution F is a (non zero) multiple of e K α , which is not 0 as α / ∈ Z . Hence F is a (non zero) λ -invariant distribution supported in O . Proposition 6.3.
The distribution F satisfies the following properties : i ) on O , F coincides with K α , O . ii ) for any g ∈ G , F ◦ π λ ( g ) − F = −
23 ln (cid:0) κ ( g − , x ) κ ( g − , y ) κ ( g − , z ) (cid:1) F (32) Proof.
For s = 0, and ϕ ∈ C ∞ ( S × S × S ) supported in O , F ( s )( ϕ ) = ( − k k ! 1Γ( − k + s ) K α ( s ) , O ( ϕ )Let s → i ). The proof of ii ) is similar to the proof of (24).Assume now there exists a distribution T on S × S × S , which extends K α , O and is λ -invariant. As Supp ( F ) ⊂ O , the restriction to O ′ of F and K α , O ′ coincide. Let S = T − F . Then S is smoothly supported in O and satisfies S ◦ π λ ( g ) − S = −
12 (ln κ ( g − , x ) κ ( g − , y ) κ ( g − , z ) F (33)Let ( x, x, x ) ∈ O . Then T ( x,x,x ) O = { ( u, u, u ) , u ∈ T x S } so that N ( x,x,x ) = { ξ = ( ξ , ξ , ξ ) , ξ + ξ + ξ = 0 , ξ j ∈ T ∗ x S, j = 1 , , } . From the computation of the residue of K α at a pole of type II, the distri-bution F has global transverse order equal to 2 k (see [1]). From (33), thedistribution S has also a global order, say m , and m ≥ k . Let g ∈ G suchthat g ( x ) = x . Then σ m (cid:0) S ◦ π λ ( g ) (cid:1)(cid:0) ( x, x, x ) , ξ (cid:1) = κ ( g, x ) − ( λ + λ + λ + ρ ) σ m ( S ) (cid:0) ( x, x, x ) , ξ ◦ Dg ( x ) − (cid:1) . (34)30et x = , and let g = a t for t ∈ R . Then a t ( ) = and Da t ( ) = e − t Id,so that (29) implies( e t (2 k − m ) − σ m ( S ) (cid:0) ( , , ) , ξ (cid:1) = − t σ m ( F ) (cid:0) ( , , ) , ξ (cid:1) (35)An argument similar to the one used in the proof of Proposition 6.1 yieldsa contradiction.The proof of Theorem 6.1 in the case of a pole of type II (generic andnot in Z ) is similar to the proof given for the case of a pole of type I, andwe omit it. Proposition 6.4.
Let α be a pole of type I+II, and assume moreover that α / ∈ Z . Let λ be its associated spectral parameter. Then the distribution K α , O can not be extended to a λ -invariant distribution on S × S × S .Proof. Let α = ( α , α , α ) and assume that α + α + α = − n − − k, α = − ( n − − l for some k, l ∈ N . Assume moreover that α / ∈ Z . This time, two of the fourΓ-factors (namely Γ( α + ρ ) and Γ( α + α + α + 2 ρ )) in the normalization of K α are singular at α .For s ∈ C , let α ( s ) = ( α , α , α + 2 s ) , λ ( s ) = ( λ + s, λ + s, λ ) , and consider the distribution-valued function F ( s ) = ( − k + l k ! l ! Γ (cid:16) α ρ (cid:17) Γ (cid:16) α ρ (cid:17) e K α ( s ) . (36)The Taylor expansion of F at s = 0 reads F ( s ) = F + s F + s F + O ( s ) , (37)where F , F , F are distributions on S × S × S . Lemma 6.3. i ) Supp ( F ) ⊂ O ii ) Supp ( F ) ⊂ O iii ) on O , F coincides with K α , O . roof. i ) was proved in Proposition 3.3.For s = 0 and ϕ ∈ C ∞ ( S × S × S ) with Supp ( ϕ ) ⊂ O , F ( s )( ϕ ) = ( − k + l k ! l ! 1Γ( − k + s ) 1Γ( − l + s ) K α ( s ) , O ( ϕ ) . (38)Let s → iii ) follows. Now let ϕ ∈ C ∞ ( S × S × S ) such that Supp ( ϕ ) ∩ ( O ∪ O ) = ∅ . As α , α / ∈ − ( n − − N , the expression K α ( s ) ( ϕ ) can bedefined by analytic continuation in s , and (38) is still valid. When s → F ( ϕ ) = F ( ϕ ) = 0, thusproving ii ). Lemma 6.4.
For any g ∈ G , F ◦ π λ ( g ) − F = 0 (39) F ◦ π λ ( g ) − F = − A g F (40) F ◦ π λ ( g ) − F = − A g F + 12 A g F (41) where A g is the function on S × S × S given by A g = ln κ ( g − , x ) + ln κ ( g − , y ) . (42) Proof.
Recall that F ( s ) ◦ π λ ( s ) ( g ) = F ( s ). Now π λ ( s ) ( g ) = κ ( g − , x ) s κ ( g − , y ) s π λ ( g ) = (cid:0) sA g + s A g O ( | s | ) (cid:1) π λ ( g )(43)Now use (37) and (43) to get the conclusion.In order to prove Proposition 6.4, a lemma is needed. Lemma 6.5.
Supp ( F ) = O . Proof.
By Lemma 6.4, the restriction of the distribution F to O ′ is λ -invariant and is supported in O . From this follows that either Supp ( F ) = O or Supp ( F ) ⊂ O . Let us assume that Supp ( F ) ⊂ O and get acontradiction. If this was the case, then the distribution F would be offinite (global) transverse order along the submanifold O . The function p a ,a ,a vanishes on O to an arbitrary given order provided a + a + a islarge enough. Hence F ( p a ,a ,a ) = 0 as a + a + a is large enough.So to get a contradiction , it is enough to prove that F ( p a ,a ,a ) = 0for some triples ( a , a , a ) with a + a + a arbitrary large.32 emma 6.6. Let a , a , a be chosen so that ≤ a ≤ l and a + a + a > k .Then e K α ( s ) ( p a ,a ,a ) = C a ,a ,a s + O ( s ) as s → , where the constant C a ,a ,a = 0 for a , a large enough.Proof. A careful look at (11) shows that e K α ,α ,α + s ( p a ,a ,a ) = Φ( s )( − k + s ) a + a + a ( − l + s ) a where Φ( s ) → Φ(0) and Φ(0) = 0 provided a , a are large enough. If0 ≤ a ≤ l then ( − l ) a = 0 and if a + a + a > k , then ( − k + s ) a + a + a has a simple s = 0. The statement follows.From (36) and (37) follows F ( p a ,a ,a ) = 0 , F ( p a ,a ,a ) = 0 under thesame constraints on a , a , a as in Lemma 6.6.We are now ready for the proof of Proposition 6.4. As α , α / ∈ − ( n − − N , the distribution K α , O can be extended (by analytic continuation)to a λ -invariant distribution K α , O ′ on O ′ . Moreover it is the unique λ -invariant extension to O ′ of K α , O , by the same argument as in the proofof Proposition 6.1.Suppose there exists a λ -invariant distribution T on S × S × S whichextends K α , O to S × S × S . From the last remark follows that T alsoextends K α , O ′ . Form the distribution S = T − F . Then the distribution S satisfies S ◦ π λ ( g ) − S = gF − A g F (44)and Supp ( S ) ⊂ O (45)Restrict (44) to O ′ to get S ′ − S ′ ◦ π λ ( g ) = A g F ′ , (46)where ′ means ”restriction to O ′ ”. From (40), F ′ is λ -invariant on O ′ .Arguing as in the previous situations, S ′ is smoothly supported in O , aswell as F ′ . We now reproduce the argument of Proposition 6.1, getting acontradiction as, by Lemma 6.5, F ′ = 0.The proof of Theorem 6.1 in the case of a pole of type I+II (and not in Z ) requires some more work. 33 emma 6.7. Let α be a pole of typeI+II, and assume that α / ∈ Z . Let λ bethe associated spectral parameter. Let T be a λ -invariant distribution. Then Supp ( T ) ⊂ O .Proof. Let α = ( α , α , α ) satisfying α + α + α = − n − − k, α = − ( n − − l where k, l ∈ N and assume moreover that α / ∈ Z . Let λ be its associatedspectral parameter, and let T be a λ -invariant distribution on S × S × S .By the same argument as in the case of a generic pole of type I, we get that T is supported in O = O ∪ O . Let T ′ be its restriction to O ′ . Then F ′ and T ′ are two λ -invariant distributions on O ′ which are supported in O .Hence by Lemma 4.1, T ′ = cF ′ for some constant c . Assume c = 0, andsubstituting c T to T , we assume that c = 1. Consider then the distribution S = T − cF . Then Supp ( S ) ⊂ O , and S satisfies S ◦ π λ ( g ) − S = AF . As argued previously, this implies that S is smoothly supported on O . Let p be its transverse order along O . As S is of transverse order 2 k (see [1]), p ≥ k . Now the transverse symbols at ( , , ) satisfy (cid:0) e t (2 k − p ) − (cid:1) σ p ( S ) (cid:0) ( , , ) , ξ (cid:1) = 2 tσ p ( F ) (cid:0) ( , , ) , ξ (cid:1) for any t ∈ R . As F = 0 (and hence σ p ( F ) (cid:0) ( , , ) ξ (cid:1) = 0), letting t go toinfinity yields a contradiction. Hence T is supported in O . Theorem 7.1.
Let α be a generic pole of type I , and let λ be its associatedspectral parameter. Let T = 0 be a λ -invariant distribution. There exists aconstant C such that T = C e K λ .Proof. From Theorem 6.1, T is supported in O = O ∪ O . Let O ′ = S × S × S \ O , and recall that O is a regular submanifold in O ′ . Let T ′ be the restriction of T to O ′ . It is a λ -invariant distribution supported in O . On the other hand, consider the restriction of e K α to O ′ . Then it isa λ -invariant distribution supported in O , and it is not 0, otherwise, e K α would be supported in O , a fact that would imply, by Lemma 4.2 that α is34 pole of type II, which is not the case. Now Lemma 4.1 implies that thereexists a constant C such T ′ = C e K α , O ′ .Now T − C e K λ is a λ -invariant distribution supported in O . As λ is nota pole of type II, Lemma 4.2 forces T − C e K λ = 0 and the conclusion follows. Theorem 7.2.
Let λ be a pole of type II and assume that λ / ∈ Z . Let T = 0 be a λ -invariant distribution. There exists a constant C such that T = C e K λ . The proof is more difficult, and we need first to recall the relation be-tween λ -invariant distributions supported in O and covariant bi-differentialoperators , for which we refer to [1]. Lemma 7.1.
Let λ ∈ C . Let T = 0 be a distribution on S × S × S whichis λ -invariant and such that Supp ( T ) ⊂ O . There exists a bi-differentialoperator B : C ∞ ( S × S ) −→ C ∞ ( S ) which intertwines ( π λ ⊗ π λ ) and π − λ such that, for f ∈ C ∞ ( S × S ) , g ∈ C ∞ ( S ) T ( f ⊗ g ) = Z S Bf ( x ) g ( x ) dx . (47) Conversely, given a bi-differential operator B : C ∞ ( S × S ) −→ C ∞ ( S ) whichintertwines ( π λ ⊗ π λ ) and π − λ , then (47) defines a λ -invariant distribu-tion. Given λ, µ ∈ C , denote by BD G ( λ, µ ; k ) the space of bi-differential oper-ators from C ∞ ( S × S ) into C ∞ ( S ) which are covariant w.r.t. π λ ⊗ π µ and π λ + µ + ρ +2 k . Proposition 7.1.
Let λ be a pole of type II (including type I+II), andassume λ / ∈ Z . Let k be the integer such that λ + λ + λ = − ρ − k . Then dim T ri ( λ ) = dim (cid:0) BD G ( λ , λ ; k ) (cid:1) . Proof.
Let T ∈ T ri ( λ ). By Theorem 6.1, in both cases Supp ( T ) ⊂ O .Hence the result follows from Lemma 7.1. Lemma 7.2.
Let k be a given integer. Let H k be the plane in C given bythe equation λ + λ + λ = − ρ − k . hen λ ∈ Z if and only if (at least) one of the following properties i ) to vi ) is verified i ) λ + λ ∈ − k + N ii ) λ ∈ − ρ − k − N iii ) λ ∈ − ρ − k − N iv ) λ = − ρ − l, l ∈ N and λ ∈ {− k, − k + 1 , . . . , − k + l } . v ) λ = − ρ − l, l ∈ N and λ ∈ {− k, − k + 1 , . . . , − k + l } . vi ) λ = − k + p, λ = − k + q , p, q ∈ N , p + q ≤ k . The proof is elementary and we omit it. Let Z k = { ( λ , λ ) ∈ C , ( λ , λ , − ρ − k − λ − λ ) ∈ Z } . The elements of Z k have to satisfy (at least) one of the properties i ) through vi ).To finish the proof of Theorem 7.2, it remains to prove the followingresult, which will be done in the next subsection. Theorem 7.3.
Let ( λ , λ ) ∈ C , ( λ , λ ) / ∈ Z k , where k ∈ N . Then dim B D G ( λ , λ ; k ) = 1 . For the first time in this article, we will use the noncompact picture . Let F ≃ R n − be the tangent space of S at the point . The stereographicprojection is a diffeomorphism from S \ {− } onto F . The action of G istransferred to a (rational) action of G on F . The representations π λ canalso be transferred, and in fact for the problem at hand, it is enough toconsider the derivate of the representation, viewed as a representation of g .For X ∈ g , dπ λ ( X ) acts by a first order differential operator with polynomialcoefficients. A covariant bi-differential operator on S × S is transferred to acovariant bi-differential operator on F × F and vice versa. In our context,these operators have been studied in [19], to which we refer for more details.Choose coordinates on F with respect to some orthogonal basis of F , andconsider the bi-differential operators on F × F given by∆ x x = n − X j =1 ∂ ∂x j , ∆ x y = n − X j =1 ∂ ∂x j ∂y j , ∆ y y = n − X j =1 ∂ ∂y j . Let D be a bi-differential operator on F × F which is covariant w.r.t.( π λ ⊗ π λ , π λ + λ + ρ +2 k ) for some k ∈ N . As a first (and elementary) reduc-tion of the problem, the operator D has to be of the form36 = X r + t ≤ k c r,t ∆ rx x ∆ sx y ∆ ty y (48)where s = k − r − t and the c r,t are constants.The next propositions are two main results obtained in [19]. Noticethat in their notation dim F = n , and their index λ for the representationcorresponds to λn − − in our notation. Proposition 7.2.
The bi-differential operator D given by (48) is covariantw.r.t. ( π λ ⊗ π λ , π λ + λ + ρ +2 k ) if and only if the ( c r,t ) satisfy the system ( S ) of homogeneous linear equations : r + 1)( r + 1 + λ ) c r +1 , t +2( k − r − t )( k − r + t − ρ + λ ) c r, t − ( k − r − t + 1)( k − r − t ) c r, t − = 0 E (1) r,t t + 1)( t + 1 + λ ) c r, t +1 +2( k − r − t )( k + r − t − ρ + λ ) c r, t − ( k − r − t + 1)( k − r − t ) c r − , t = 0 E (2) r,t for r + t ≤ k . Proposition 7.3.
Let λ , λ such that λ , λ / ∈ {− ρ, − ρ − , . . . , − ρ − ( k − } ∪ {− , − , . . . , − k } . Then the system ( S ) has, up to a constant, a unique non trivial solution. Hence, the proof of Theorem 7.3 is reduced to the following statement.
Proposition 7.4.
Let λ , λ ∈ C such that λ or λ belongs to {− ρ, − ρ − , . . . , − ρ − ( k − } ∪ {− , − , . . . , − k } , but ( λ , λ ) / ∈ Z k . Then thesystem ( S ) has, up to a constant a unique solution. The proof of Proposition 7.4 is a case-by-case proof. As the argumentsare of the same type in each case, we have chosen to present the worst casewith full details, giving only a sketch of proof in the others.
Proposition 7.5.
Let λ = − p, λ = − q , where ≤ p, q ≤ k , and assumethat ( λ , λ ) / ∈ Z k . Then the system ( S ) has a unique solution, up to aconstant. roof. As vi ) in Lemma 7.2 is not satisfied, p + q < k . This fact will revealcrucial for the non vanishing of some of the coefficients of the equations E (1) r,t or E (2) r,t . Lemma 7.3.
Assume previous conditions are satisfied. Then c r,t = 0 if ≤ r ≤ p − or ≤ t ≤ q − . Proof.
Step 1.
Equation E (1) p − , reads2( k − p + 1)( k − p + ρ − q ) c p − , + 0 = 0which implies c p − , = 0. Step 2.
Use equations E (1) r, successively for r = p − , . . . , c r, .Notice that the coefficient in the equation corresponding to the unknown c r, is equal to 2( k − r )( k − r − q + ρ − = 0 . Hence c r, = 0 for 0 ≤ r ≤ p − Step 3.
For 1 ≤ t ≤ q −
1, and 0 ≤ r ≤ p −
1, use equation E (2) r,t − to compute c r,t . Notice that the coefficient of the unknown in the equation is equal to4 t ( t − q ) = 0. Let first t = 1 to obtain c r, = 0 for 0 ≤ r ≤ p −
1, andthen increment t by 1 up to q − c r, = 0 for 0 ≤ r ≤ p −
1, . . . , c r,q − = 0 for 0 ≤ r ≤ p − Step 4.
For r = p, p + 1 , . . . , k − q + 1, consider the equations E (2) r,q − which reduceto2( k − r − q + 1)( k + r − p − q + ρ ) c r,q − + ( k − r − q )( k − r − q + 1) c r − ,q − = 0 , and hence c r,q − = 0 for p ≤ r ≤ k − q + 1. Step 5.
For p ≤ r ≤ k − q + 1 and 0 ≤ t ≤ q −
1, use equation E (2) r,t to compute c r,t . Notice that the coefficient of the unknown in the equation is equal to2( k − r − t )( k − t − p + r + ρ − = 0 . t = q − r = p to conclude that c p,q − = 0, then increment r by 1 up to k − q + 1 to get c r,q − = 0. Then increment t by −
1, . . . , toconclude that c r,t = 0 if p ≤ r ≤ k − q + 1 and 0 ≤ t ≤ q − Step 6.
For r > k − q + 1 and 0 ≤ t ≤ q −
1, use equation E (1) r − ,t to compute c r,t . Notice that the coefficient of the unknown in the equation is equal to4 r ( r − p ) and conclude that c r,t = 0 for r > k − q + 1 and 0 ≤ t ≤ q − c r,t = 0 for t ≤ q − r . By exchanging the role of r and t (and of p and q ) follows c r,t = 0 for0 ≤ r ≤ p − t arbitrary.Now, choose c p,q as principal unknown and show that all the remainingunknowns (i.e. c r,t for r ≥ p, t ≥ q ) can be expressed in terms of the principalunknown.Let r ≥ p . To determine c r +1 ,q , use E (1) r,q which, as c r,q − = 0 reads4( r + 1)( r + 1 − p ) c r +1 ,q + 2( k − r − q )( k − r − ρ ) c r,q = 0 . Starting with r = p and incrementing r by 1 up to k − q − c r,q with r ≥ p + 1 in term of c p,q .Symmetrically, compute the unknowns c p,t , t ≤ q + 1.For r > p and t > q , use equation E (2) r,t − to compute c r,t . Notice that thecoefficient of the unknown in the equation is equal to 4 t ( t − q ) = 0. Startwith r = p, t = q + 1 and increment r by 1. Then increment t by 1, andrepeat the process over r . . . , to effectively compute c r,t , r > p, t > q . Thisachieves the proof of the proposition. Proposition 7.6.
Let λ = − ρ − k , λ = − ρ − k , where ≤ k , k ≤ k − and assume that ( λ , λ ) / ∈ Z . Then the system ( S ) has a unique non trivialsolution, up to a constant.Proof. As property iv ) of Lemma 7.2 is not true, λ / ∈ {− k, − k + 1 , . . . , − k + k } and similarly λ / ∈ {− k, − k + 1 , . . . , − k + k } . As in the previous case,this is crucial for the non vanishing of some of the coefficients of the equa-tions.First, for r ≥ k − k or t ≥ k − k , c r,t = 0, which is proved witharguments similar to those used in the previous case. Then choose c , asprincipal unknown, and compute the c r,t for 0 ≤ r ≤ k − k − , ≤ t ≤ k − k − c , .Now, assume λ = − ρ − k , λ = − k , with 0 ≤ k ≤ k − ≤ k ≤ k , and moreover ( λ , λ ) / ∈ Z . As λ + λ + λ = − ρ − k ,39 = − k + k + k . By Zk iv ), − k + k < − k , and hence λ = − k − ( k − k − l ) / ∈ {− k, − k + 1 , . . . , − } . Because of the symmetry of our problemin ( λ , λ , λ ), ( λ , λ ) or ( λ , λ ) do not belong to Z k , and proving theuniqueness statement for ( λ , λ ) or for ( λ , λ ) or for ( λ , λ ) is equivalent.So if λ ∈ {− ρ, − ρ − , . . . , − ρ − ( k − } , the uniqueness statement for ( λ , λ )is obtained by Proposition 7.6. Otherwise λ / ∈ {− ρ, − ρ − , . . . , − ρ − ( k − } ∪ {− , − , . . . , − k } , the uniqueness for ( λ , λ ) is part of the situationsto be analyzed next.Consider now the case where λ = − p with 1 ≤ p ≤ k , but λ / ∈{− ρ, − ρ − , . . . , − ρ − ( k − } ∪ {− , − , . . . , − k } . By the same argumentused in first part c r,t = 0 if r ≤ p −
1. Then choose c p, as principal unknown,and then compute the other c r,t for r ≤ p in terms of c p, as done in theproof of Proposition 7.4.In the case where λ = − ρ − p for 0 ≤ p ≤ k − λ / ∈ {− ρ, − ρ − , . . . , − ρ − ( k − }∪{− , − , . . . , − k } , first prove that c ,t = 0 for t ≥ k − k .Choose c , as principal unknown and then solve the system as done inProposition 7.6. This achieves the proof Theorem 7.3. Corollary 7.1.
Let λ ∈ C and assume that the three representations π λ j are irreducible. Then dim T ri ( λ ) = 1 .Proof. The assumption of irreducibility amounts to λ j / ∈ ( − ρ − N ) ∪ ( ρ + N )for j = 1 , ,
3. These conditions guarantee that λ is not in Z . Hence theresult follows from Theorem 0.1.The next result is based on a remark due to T. Oshima (personal com-munication). Theorem 7.4.
Let µ be in Z . Then dim T ri ( µ ) ≥ . Proof. (Sketch) It is a consequence of Lemma 6.3 in [18]. We follow thenotation of this paper. Let U = C and let r ( λ ) = 1Γ( λ + λ + λ + ρ )Γ( − λ + λ + λ + ρ )Γ( λ − λ + λ + ρ )Γ( λ + λ − λ + ρ )40this is nothing but the normalizing factor for K λ ). Let U r = { λ , r ( λ ) = 0 } .If λ ∈ U r , then λ is not a pole and hence V λ = T ri ( λ ) is of dimen-sion 1. For µ ∈ U \ U r , let V µ be the closure of the holomorphic fam-ily V λ (see precise definition in [18]). Then V µ ⊂ T ri ( µ ). Now, by [18],dim V µ ≥ dim λ ∈ U r V λ = 1, and there is a necessary and sufficient conditionfor having equality (in which case µ is said to be a removable point). Now Z is contained in U \ U r . Let µ be in Z . The fact that e K µ = 0 implies that µ does not satisfy the criterion for removability, so that dim V µ > dim V λ .All together, dim T ri ( µ ) > Final remark.
When λ is a pole of type II, and λ is in Z , a λ -invariantdistribution is not necessarily supported on the diagonal O (such cases willappear in [5]). So it is conceivable that for some λ in Z dim BD G ( λ , λ ; k ) =1 although dim T ri ( λ ) > B D ( λ , λ ; k ) = 1 is still an open prob-lem. This problem is to be studied in relation with a problem about tensorproducts of generalized Verma modules (see [21] for some work in this di-rection). References [1] Beckmann R. and Clerc J-L.,
Singular invariant trilinear forms andcovariant (bi-)differential operators under the conformal group , J. ofFunct. Anal. (2012) 4341–4376[2] Bernstein J. and Reznikov A.,
Estimates of automorphic functions ,Mosc. Math. J. (2004), 19–37[3] Bruhat F. Sur les repr´esentations induites des groupes de Lie , Bull. Soc.Math. France (1956), 97–205[4] Clerc J-L., Intertwining operators for the generalized principal series ona symmetric R-space , to appear in Trans. Amer. Math. Soc.[5] Clerc J-L.,
Singular conformally invariant trilinear forms II, Construc-tion of holomorphic families , in preparation[6] Clerc J-L. and Ørsted B.,
Conformally invariant trilinear forms on thesphere , Ann. Instit. Fourier (2011), 1807–1838[7] Clerc J-L., Kobayashi T., Ørsted B. and Pevzner M., GeneralizedBernstein-Reznikov integrals , Math. Annalen (2011), 395–431[8] Deitmar A.,
Invariant triple products , Int. J. Math. Sci. ID 48274 (2006)[9] Guelfand I. et Chilov G.,
Les distributions , Dunod (1962)4110] Goodman R. and Wallach N.,
Representations and Invariants of theClassical Groups , Cambridge University Press (1998)[11] H¨ormander L.,
The analysis of linear partial differential operators I ,Grundlehren der mathematischen Wissenschaften 256, Springer-Verlag(1983).[12] Knapp A.,
Representation theory of semi simple groups, An overviewbased on examples , Princeton University Press (1986)[13] Kobayashi T., Ørsted B., Somberg P. and Souˇcek V., Branching lawsfor Verma modules and applications in parabolic geometry. II, preprint(2013)[14] Kobayashi T. and Speh B.,
Symmetry breaking for representations ofrank one orthogonal groups , arXiv:1310.3213v11 (2013)[15] Kolk J. and Varadarajan V.,
On the transverse symbol of vectorial dis-tributions and some applications to harmonic analysis , Indag. Mathem. (1996), 67–96[16] Oksak A., Trilinear Lorentz invariant forms , Comm. math. Phys. (1973), 189–217[17] Oksak A., Analyticity of trilinear SL (2 , C ) invariant forms in elemen-tary representation parameters , Reports on Mathematical Physics (1975) 9–36[18] Oshima T. Annihilators of generalized Verma modules of the scalar typefor classical Lie algebras
Harmonic Analysis, Group Representations,Automorphic forms and Invariant Theory, in honor of Roger Howe,Lecture Notes Series, Vol. 12, 277–319 (2007), National University ofSingapore[19] Ovsienko V. and Redou P.,
Generalized transvectants Rankin-Cohenbrackets , Lett. Math. Phys. (2003), 19–28[20] Poulsen N., On C ∞ -vectors and intertwining bilinear forms for repre-sentations of Lie groups , J. Funct. Anal. (1972), 87–120[21] Somberg P. Rankin-Cohen brackets for orthogonal Lie algebras and bi-linear conformally invariant differential operators , arXiv:1301.2687v1(2013)
AddressJean-Louis ClercInstitut Elie Cartan, Universit´e de Lorraine, 54506 Vandœuvre-l`es-Nancy, France [email protected]@univ-lorraine.fr