A hidden symmetry of a branching law
aa r X i v : . [ m a t h . R T ] J un A hidden symmetry of a branching law
Toshiyuki Kobayashi ∗ and Birgit Speh † Abstract
We consider branching laws for the restriction of some irreducible unitaryrepresentations Π of G = O ( p, q ) to its subgroup H = O ( p − , q ). In Kobayashi(arXiv:1907.07994, [14]), the irreducible subrepresentations of O ( p − , q ) inthe restriction of the unitary Π | O ( p − ,q ) are determined. By considering therestriction of packets of irreducible representations we obtain another very sim-ple branching law, which was conjectured in Ørsted–Speh (arXiv:1907.07544,[17]). Mathematic Subject Classification (2020): Primary 22E46; Secondary 22E30, 22E45,22E50
I Introduction
The restriction of a finite-dimensional irreducible representation Π G of a connectedcompact Lie group G to a connected Lie subgroup H is a classical problem. For exam-ple, the restriction of irreducible representations of SO ( n + 1) to the subgroup SO ( n )can be expressed as a combinatorial pattern satisfied by the highest weights of the ir-reducible representation Π G of the large group and of the irreducible representations ∗ Graduate School of Mathematical Sciences, The University of Tokyo and Kavli IPMU (WPI),3-8-1 Komaba, Tokyo, 153-8914 Japan
E-mail address : [email protected] † Department of Mathematics, Cornell University, Ithaca, NY 14853-4201, USA
E-mail address : [email protected] π H [20]. For the pair ( G, H ) = ( SO ( n + 1) , SO ( n )),the branching law is always multiplicity-free, i.e., dim Hom H ( π H , Π G | H ) ≤ . In this article we consider a family of infinite-dimensional irreducible representationsΠ p,qδ,λ with parameters λ ∈ Z + ( p + q ), and δ ∈ { + , −} of noncompact orthogonalgroups G = O ( p, q ) with p ≥ q ≥
2, which have the same infinitesimalcharacter as a finite-dimensional representation and which are subrepresentations of L ( O ( p, q ) /O ( p − , q )) for δ = +, respectively of L ( O ( p, q ) /O ( p, q − δ = − .We shall assume a regularity condition of the parameter λ (Definition III.7). Similarlywe consider a family of infinite-dimensional irreducible unitary representations π p − ,qε,µ , ε ∈ { + , −} of noncompact orthogonal groups H = O ( p − , q ).Reviewing the results of [14] we see in Section IV that the restriction of these rep-resentations to the subgroup H = O ( p − , q ) is either of “finite type” (ConventionIV.13) if δ = + or of “discretely decomposable type” (Convention IV.6) if δ = − . Ifthe infinitesimal characters of Π p,qδ,λ and of a direct summand of (Π p.qδ,λ ) | H satisfy aninterlacing condition (4.12) similar to that of the finite-dimensional representationsof ( SO ( n + 1) , SO ( n )), then δ = + and the restriction of a representations Π p,qδ,λ isof finite type. On the other hand, if the infinitesimal characters Π p,qδ,λ and of a directsummand of (Π p.qδ,λ ) | H satisfy another interlacing condition (4.9) similar to those of theholomorphic discrete series representations of ( SO ( p, , SO ( p − , δ = − and the restriction of a representations Π p,qδ,λ is of discretely decomposable type.For each λ we define a packet { Π p,q + ,λ , Π p.q − ,λ } of representations with the same infinites-imal character. For simplicity, we assume p ≥ q ≥
2. Using the branchinglaws for the individual representations we show in Section V:
Theorem I.1.
Let ( G, H ) = ( O ( p, q ) , O ( p − , q )) . Suppose that λ and µ are regularparameters. (1) Let Π λ be a representation in the packet { Π + ,λ , Π − ,λ } . There exists exactly onerepresentations π µ in the packet { π + ,µ , π − ,µ } so that dim Hom H (Π λ | H , π µ ) = 1 . (2) Let π µ be in the packet { π + ,µ , π − ,µ } . There exists exactly one representation Π λ in the packet { Π + ,λ , Π − ,λ } so that dim Hom H (Π λ | H , π µ ) = 1 . Theorem I.2 (Version 2) . Suppose that λ and µ are regular parameters. Then dim Hom H ((Π + ,λ ⊕ Π − ,λ ) | H , ( π + ,µ ⊕ π − ,µ )) = 1 . Another version of this theorem using interlacing properties of infinitesimal charac-ters is stated in Section V.
Acknowledgements
The authors would like to acknowledge support by the MFOduring research in pairs stay during which part of this work was accomplished.The first author was partially supported by Grant-in-Aid for Scientific Research (A)(18H03669), Japan Society for the Promotion of Science.The second author was partially supported by Simons Foundation collaborationgrant, 633703.
Notation: : N = { , , , . . . , } and N + = { , , . . . , } . II Generalities
We will use in this article the notation and conventions of [14] which we recall now.These conventions differ from those used in [17].Consider the standard quadratic form on R p + q Q ( X, X ) = x + · · · + x p − x p +1 − · · · − x p + q (2.1)of signature ( p, q ) in a basis e , . . . , e p , e p +1 , . . . e p + q . We define G = O ( p, q ) to be theindefinite special orthogonal group that preserves the quadratic form Q . Let H bethe stabilizer of the vector e . Then H is isomorphic to O ( p − , q ).Consider another quadratic form on R p + q Q − ( X, X ) = x + · · · + x q − x q +1 − · · · − x p + q (2.2)of signature ( q, p ) with respect to a basis e − , , . . . , e − ,p , e − ,p +1 , . . . e − ,p + q . The or-thogonal group G − = O ( q, p ) that preserves the quadratic form Q − is conjugate to O ( p, q ) in GL ( p + q, R ). Thus we may consider representations of G − = O ( q, p ) asrepresentations of G = O ( p, q ) . G and G − are conjugate, the subgroup H of G is also conjugate to a subgroup H − of G − which is isomorphic to O ( q, p − G/H = O ( p, q ) /O ( p − , q ) and G − /H − = O ( q, p ) /O ( q, p − O ( p, q ) /O ( p − , q ) and O ( q, p ) /O ( q − , p )are not even homeomorphic to each other if p = q . In the rest of the article we willassume that the subgroup H − preserves the vector e − ,p + q .The maximal compact subgroups of G , G − and H, H − are denoted by K, K − respec-tively K H , K H − . The Lie algebras of the groups are denoted by the correspondinglowercase Gothic letters. To avoid considering special cases we make in this article the following:
Assumption O : p ≥ and q ≥ . III Representations
We consider in this article a family of irreducible unitary representations introducedin [14]. Using the notation in [14] we recall their parametrization and some importantproperties in this section. The main reference is [14, Sect. 2].The irreducible unitary subrepresentations of L ( O ( p, q ) /O ( p − , q )) were consideredby many authors after the pioneering work by I. M. Gelfand et. al. [6], T. Shintani,V. Molchanov, J. Faraut [4], and R. Strichartz [18]. For p ≥ q ≥
1, they areparametrized by λ ∈ Z + ( p + q ) with λ >
0. Following the notation of [14] wedenote them by Π p,q + ,λ . They have infinitesimal character( λ, p + q − , p + q − , . . . , p + q − [ p + q , in the Harish-Chandra parametrization (see (4.8) below), and the minimal K -type ( H b ( λ ) ( R p ) ⊠ if b ( λ ) ≥ ⊠ if b ( λ ) ≤
0, (3.3)4here b ( λ ) := λ − ( p − q −
2) ( ∈ Z ) and H b ( R p ) stands for the space of sphericalharmonics of degree b . We note that Π p,q + ,λ are so called Flensted-Jensen representa-tions discussed in [5] if b ( λ ) ≥
0, namely, if λ ≥ ( p − q − λ is regular (Definition III.7). The underlying ( g , K )-module of Π p,q + ,λ is given by aZuckerman derived functor module. See [9, Thm. 3] or [14, Sect. 2.2]. Remark
III.1 . When p = 1 and q ≥
1, there are no irreducible subrepresentations in L ( O ( p, q ) /O ( p − , q )), and we regard π p,q + ,λ as zero in this case. Remark
III.2 . (1) For any p ≥ q ≥ Z + ( p + q ) ∋ λ >
0, the representationΠ p,q + ,λ of G = O ( p, q ) stays irreducible when restricted to SO ( p, q ), see also RemarkIII.6.(2) If p = 2 and λ ≥ ( p + q − p,q + ,λ is a direct sum ofa holomorphic discrete series representation and an anti-holomorphic discrete seriesrepresentation when restricted to the identity component G = SO ( p, q ) of G .Similarly there exist a family of irreducible unitary subrepresentationsΠ q,p + ,λ ( λ ∈ Z + 12 ( p + q ) , λ > G − = O ( q, p ) in L ( G − /H − ) = L ( O ( q, p ) /O ( q − , p )) when p ≥ q ≥
2, withthe same infinitesimal character and the same properties. Via the isomorphism be-tween ( G − , H − ) and ( G, H ), we may consider them as representations of G = O ( p, q )and irreducible subrepresentations of L ( G/H ) = L ( O ( p, q ) /O ( p, q − + ,λ = Π p,q + ,λ and Π − ,λ ≃ Π q,p + ,λ (via G − ≃ G ),to denote representations of G = O ( p, q ). Remark
III.3 . The irreducible representation Π + ,λ are nontempered if p ≥
3, andΠ − ,λ are nontempered if q ≥ Lemma III.4.
Assume that λ ≥ ( p + q − . The representations Π + ,λ , Π − ,λ areinequivalent, but have the same infinitesimal character. roof. The representation Π + ,λ and Π − ,λ are irreducible representations of G = O ( p, q ) with respective minimal K -types H b ( R p ) ⊠ , b := λ −
12 ( p − q − , ⊠ H b ′ ( R q ) , b ′ := λ −
12 ( q − p − , because the assumption λ ≥ ( p + q −
2) implies both b ≥ b ′ ≥ Remark
III.5 . Lemma III.4 holds in the more general setting where λ ≥
0, see [9,Thm. 3 (4)] for the proof.
Remark
III.6 . For p and q positive and even, the restriction of the representationsΠ + ,λ , Π − ,λ to SO ( p, q ) are in an Arthur packet as discussed in [3, 16]. Global ver-sions of Arthur packets were introduced by J. Arthur in the theory of automorphicrepresentations and are inspired by the trace formula [1, 2]. Our considerations ofArthur packets of representations of the orthogonal groups which are discrete seriesrepresentations for symmetric spaces are inspired by Arthur’s considerations as wellas by the conjectures of B. Gross and D. Prasad. In this article we will refer to { Π + ,λ , Π − ,λ } as a packet of irreducible representations.Similarly we have µ ∈ Z + ( p + q −
1) satisfying µ ≥ ( p + q −
3) a packet { π + ,µ , π − ,µ } of unitary irreducible representations of G ′ = O ( p − , q ). Definition III.7.
We say λ ∈ Z + ( p + q ) respectively µ ∈ Z + ( p + q − are regular if λ ≥ ( p + q − respectively µ ≥ ( p + q − .Remark III.8 . The irreducible representation Π + ,λ (or Π − ,λ ) has the same infinitesi-mal character as a finite-dimensional irreducible representation of G = O ( p, q ) if andonly if λ ≥ ( p + q − λ is regular. Similarly, π + ,µ (or π − ,µ ) has the sameinfinitesimal character with a finite-dimensional representation of G ′ = O ( p − , q ) ifand only if µ ≥ ( p + q − µ is regular.For later use we define for regular λ and µ the reducible representations U ( λ ) = Π + ,λ ⊕ Π − ,λ (3.4)and V ( µ ) = π + ,µ ⊕ π − ,µ . (3.5)of G = O ( p, q ) respective of H = O ( p − , q ).6 V Branching laws
In this section we summarize the results of [14]. For simplicity, we suppose that theassumption O is satisfied, namely, we assume p ≥ q ≥
2. We note that theresults in Section IV.2 hold in the same form for p ≥ q ≥
2, and those inSection IV.3 hold for p ≥ q ≥ IV.1 Quick introduction to branching laws
Consider the restriction of a unitary representation Π of G to a subgroup G ′ . Wesay that an irreducible unitary representation π of H is in the discrete spectrumof the restriction Π | H if there exists an isometric H -homomorphism π → Π | H , orequivalently, if Hom H ( π, Π | H ) = { } where Hom H ( , ) denotes the space of continuous H -homomorphisms. We define themultiplicity for the unitary representations by m (Π , π ) := dim Hom H ( π, Π | H ) = dim Hom H (Π | H , π ) . Remark
IV.1 . As in [7, 15], we also may consider the multiplicity m (Π ∞ , π ∞ ) forsmooth admissible representations Π ∞ of G and π ∞ of G ′ by m (Π ∞ , π ∞ ) := dim Hom H (Π ∞ , π ∞ ) . In general, one has m (Π ∞ , π ∞ ) ≥ m (Π , π ) . Besides the discrete spectrum there may be also continuous spectrum. Here are twointeresting cases:1. There is no continuous spectrum and the representation Π is a direct sum ofirreducible representations of H , i.e. , the underlying Harish-Chandra moduleis a direct sum of countably many Harish-Chandra modules of ( h , K H ). Wesay that the restriction Π | H is discretely decomposable .2. There is continuous spectrum and there are only finitely many representationsin the discrete spectrum in the irreducible decomposition of the restriction Π | H .7e refer to the necessary and sufficient conditions of the parameters of the irreduciblerepresentations Π , π so that m (Π , π ) = 0 (or m (Π ∞ , π ∞ ) = 0) as a branching law . Inthe examples below, m (Π ∞ , π ∞ ), m (Π , π ) ∈ { , } for all Π and π . Examples of branching laws:
1. Finite-dimensional representations of semisimple Lie groups are parametrizedby highest weights. The classical branching law of the restriction of finite-dimensional representations of SO ( n ) to SO ( n −
1) is phrased as an interlacingpattern of highest weights, see Weyl [20].2. The Gross–Prasad conjectures for the restriction of discrete series representa-tions of SO (2 m, n ) to SO (2 m − , n ) are expressed as interlacing propertiesof their parameters, see [7].3. The branching laws for the restriction of irreducible self-dual representationsΠ ∞ of SO ( n + 1 ,
1) to SO ( n,
1) are expressed by using signatures , heights andinterlacing properties of weights, see [15].If Π ∈ { Π + ,λ , Π − ,λ } , and Hom H ( π H , Π | H ) = { } then for a character χ of O(1)Hom H × O (1) ( π H ⊠ χ, Π | H × O (1) ) = { } . Moreover, by [14, Thm. 1.1] there exists a regular µ so that π H ∈ { π + ,µ , π − ,µ } .If Π is in the packet { Π + ,λ , Π − ,λ } and π in the packet { π + ,µ , π − ,µ } the branching lawsdiscussed in the next part will involve the parameters λ, µ, ε, δ . IV.2 Branching laws for the restriction of Π − ,λ to H = O ( p − , q ) — discretely decomposable type This section treats the restriction Π − ,λ | H , which is discretely decomposable. Weuse the explicit branching law given in [14, Example 1.2 (1)]. The results were alsoobtained in [10] by using different techniques, see [12, 13] for details.We begin with the pair ( G − , H − ) = ( O ( q, p ) , O ( q, p − q,p + ,λ of G − to the subgroup H − × O (1) = O ( q, p − × O (0 ,
1) is a8irect sum of irreducible representations, and is isomorphic to the Hilbert direct sumof countably many Hilbert spaces: M n ∈ N π q,p − ,λ + n + ⊠ (sgn) n where sgn stands for the nontrivial character of O (1) = O (0 , G − , H − ) ≃ ( G, H ) = ( O ( p, q ) , O ( p − , q )) and Π q,p + ,λ ≃ Π − ,λ as arepresentation of G − ≃ G , we see the restriction of Π − ,λ to H × O (1) = O ( p − , q ) × O (1 ,
0) is discretely decomposable, and we have an isomorphismΠ − ,λ | H ≃ M n ∈ N π − ,λ + n + ⊠ (sgn) n . Hence
Proposition IV.2 (Version 1) . The restriction of Π − ,λ to H = O ( p − , q ) is aHilbert direct sum M n ∈ N π − ,λ + n + and each representation has multiplicity one.Remark IV.3 . If λ is regular, then µ is regular whenever Hom H ( π − ,µ , Π − ,λ | H ) = { } .In contrast, an analogous statement fails for the restriction Π + ,λ | H , see Remark IV.10below. Remark
IV.4 . If G = SO ( p,
2) the representation Π − ,λ with λ regular is a holomor-phic discrete series representation. In this case, this result follows from the work ofH. Plesner-Jacobson and M. Vergne [8, Cor. 3.1] or as a special case of the generalformula proved in [11, Thm. 8.3].We define κ : N → { , } by κ ( n ) = 0 for n even; = 12 for n odd.Then the infinitesimal character of the representation Π − ,λ of G is( λ, p + q − , . . . , κ ( p + q )) , (4.6)and the infinitesimal character of the representations in π − + ,µ of H is( µ, p + q − , . . . , κ ( p + q − . (4.7)9ere we note that the groups G and H are not of Harish-Chandra class, but theinfinitesimal characters of the centers Z G ( g ) := U ( g ) G and Z H ( h ) := U ( h ) H of theenveloping algebras can be still described by elements of C M with M := [ ( p + q )]and C N with N := [ ( p + q − C -alg ( Z G ( g ) , C ) ≃ C M / S M ⋉ ( Z / Z ) M , (4.8)Hom C -alg ( Z H ( h ) , C ) ≃ C N / S N ⋉ ( Z / Z ) N . In our normalization, the infinitesimal character of the trivial one-dimensional rep-resentation of G = O ( p, q ) is given by( p + q − , p + q − , · · · , κ ( p + q )) . Hence we may also reformulate the branching laws in Proposition IV.2 as follows.
Proposition IV.5 (Version 2) . Suppose λ is a regular parameter (Definition III.7).Then an irreducible representation π of H = O ( p − , q ) in the discrete spectrum ofthe restriction of Π p,q − ,λ must be isomorphic to π − ,µ for some regular parameter µ , andthe infinitesimal characters have the interlacing property µ > λ > p + q − > · · · > > . (4.9) Conversely, π = π − ,µ occurs in the discrete spectrum of the restriction Π p,q − ,λ | H if theinterlacing property (4.9) is satisfied. Convention IV.6.
We say that the restriction of the representation Π − ,λ of G to H = O ( p − , q ) is of discretely decomposable type. IV.3 Branching laws for the restriction of Π + ,λ to H = O ( p − , q ) — finite type This section treats the restriction Π + ,λ | H which is not discretely decomposable. Weuse [14, Example 1.2 (2)] which determines the whole discrete spectrum in the re-striction Π + ,λ | H . A large part of discrete summands are also obtained in [17] usingdifferent techniques.The restriction Π + ,λ | H contains at most finitely many irreducible summands. Werecall from [14, Thm. 1.1] (or [14, Ex. 1.2 (2)]), an irreducible representation π of10 × O (1 ,
0) = O ( p − , q ) × O (1) occurs in the discrete spectrum of the restrictionof Π + ,λ if and only it is of the form π p − ,q + ,λ − n − ⊠ (sgn) n for some 0 ≤ n < λ − , where sgn stands for the nontrivial character of O (1). Proposition IV.7 (Version 1) . An irreducible representation π of H = O ( p − , q ) occurs in the discrete spectrum of the restriction of Π + ,λ of G = O ( p, q ) when re-stricted to H if and only if it is of the form π p − ,q + ,λ − − n where λ − − n for ≤ n < λ − . Remark
IV.8 . There does not exist discrete spectrum in the restriction Π + ,λ | H if p = 2. In fact π ,q + ,µ is zero for all µ if q ≥
1, see Remark III.1.
Remark
IV.9 . The representation π p − ,q + ,λ − − n has a regular parameter, or equivalently,has the same infinitesimal character as a finite-dimensional representation iff λ − − n > p + q − . Remark
IV.10 . In contrast to the discretely decomposable case (Remark IV.3),Proposition IV.7 tells that the implication λ regular ⇒ µ regulardoes not necessarily hold when Hom H ( π + ,µ , Π + ,λ | H ) = { } , see Remark IV.9 above.We observe that for these representations the condition in the proposition dependsonly on p + q and thus the proposition for these representations does not depends onthe inner form SO ( r, s ) of SO ( p + q, C ) when r + s = p + q with r ≥ + ,λ is( λ, p + q − , . . . , κ ( p + q )) (4.10)and the infinitesimal character of the representations in π + ,µ ( µ, p + q − , . . . , κ ( p + q − roposition IV.11 (Version 2) . Suppose π is an irreducible unitary representationof H = O ( p − , q ) . If π occurs in the discrete spectrum of the restriction of Π + ,λ to H , then π must be isomorphic to π + ,µ for some µ > with µ ∈ Z + ( p + q − .Assume further that λ and µ are regular. Then π + ,µ occurs in the discrete spectrumof the restriction Π + ,λ | H if and only if the two infinitesimal characters (4.10) and (4.11) have the interlacing property λ > µ > p + q − > · · · > > . (4.12) Remark
IV.12 . Consider the example: q = 0 and so G is compact. The representationΠ − ,λ is finite-dimensional and has highest weight( λ − p , , . . . , λ . A representation π − ,µ is a summand of the restriction to H = SO ( p −
1) if it has highest weight( µ − p − , , . . . , µ ∈ N + with µ ≥ p − and λ − p ≥ µ − p − ≥ i.e., if there exists and integer n ∈ N so that µ = λ − − n ≥ ( p − Convention IV.13.
We say that the restriction of the representation Π − ,λ to H = SO ( p − , q ) is of finite type. V The main theorems
We retain Assumption O , namely, p ≥ q ≥
2. Combing the branching lawsin the previous section proves the conjectures in [17, Sect. V] and suggests a gener-alization of a conjecture by B. Gross and D. Prasad [7], which was formulated fortempered representations.
V.1 Results for pairs ( O ( p, q ) , O ( p − , q )) Theorem V.1 (Version 1) . Suppose that λ and µ are regular parameters (DefinitionIII.7). . Let Π λ be a representations in the packet { Π + ,λ , Π − ,λ } . There exists exactlyone representations π µ in the packet { π + ,µ , π − ,µ } so that dim Hom H (Π λ | H , π µ ) = 1 .
2. Let π µ be in the packet { π + ,µ , π − ,µ } . There exists exactly one representation Π λ in the packet { Π + ,λ , Π − ,λ } so that dim Hom H (Π λ | H , π µ ) = 1 . Equivalently we may formulate the results in terms of reducible representations U ( λ )and V ( µ ) defined in (3.4) and (3.5) as follows: Theorem V.2 (Version 2) . Suppose that λ and µ are regular parameters. Then dim Hom H ( U ( λ ) | H , V ( µ )) = 1 . We may formulate the results in interlacing properties of parameter the infinitesimalcharacters similar to the results in [7].Recall that the infinitesimal character of the representations of G in the packet { Π + ,λ , Π − ,λ } is ( λ, p + q − , . . . , κ ( p + q ))and the infinitesimal character of the representations of the subgroup H in the packet { π + ,µ , π − ,µ } is ( µ, p + q − , . . . , κ ( p + q − , where we recall ( κ ( p + q ) , κ ( p + q − , ) if p + q is even, = ( ,
0) if p + q isodd. Theorem V.3 (Version 3) . Suppose that λ and µ are regular parameters.1. If the two infinitesimal characters satisfy the following interlacing property: µ > λ > p + q − > · · · > > then dim Hom H (Π − ,λ | H , π − ,µ ) = 1 . . If the two infinitesimal characters satisfy the following interlacing property: λ > µ > p + q − > · · · > > then dim Hom H (Π + ,λ | H , π + ,µ ) = 1 . Remark