aa r X i v : . [ m a t h . R T ] F e b A GENERALIZATION OF DUFLO’S CONJECTURE
HONGFENG ZHANG
Abstract.
In this article, we generalize Duflo’s conjecture to un-derstand the branching laws of non-discrete series. We give a uni-fied description on the geometric side about the restriction of anirreducible unitary representation π of GL n ( k ), k = R or C , tothe mirabolic subgroup, where π is attached to a certain kind ofcoadjoint orbit. Mathematics Subject Classification (2010).
Keywords.
Kirillov’s conjecture, Duflo’s conjecture, orbit method,moment map.
Contents
1. Introduction 12. P n ( k )-coadjoint orbit 42.1. Coadjoint action, the dual map and the moment map 42.2. Classification of P n ( k )-coadjoint orbit 53. Geometry of the moment map p : O f → p ∗ n ( C ) case 73.2. Moment map in the GL n ( R ) case 234. Results of Kirillov’s conjecture 275. Orbit method 305.1. Representations attached to coadjoint orbits of GL n ( k ) 315.2. Representations attached to coadjoint orbits of P n ( k ) 366. A generalization of Duflo’s conjecture 41Acknowledgements 42References 421. Introduction
One goal of the orbit method is to establish a correspondence be-tween coadjoint orbits and irreducible unitary representations of Liegroups. One can benefit a lot from such a correspondence, for exam-ple, finding out irreducible unitary representations through coadjointorbits; or predicting the decomposition of π | H by looking at the de-composition of p( O π ) as H -orbit, where π is a unitary representationof the Lie group G and is attached to the coadjoint orbit O π , H is a closed subgroup of G , and p is the moment map from O π to h ∗ with h ∗ the dual of the Lie algebra of H ; etc.(see Kirillov [12], Introduction).The orbit method is brought up by Kirillov, who attached an irre-ducible unitary representation to a coadjoint orbit for nilpotent groupsin a perfect way. Later, Kostant (see Auslander-Kostant [3]) estab-lished the theory of quantization, and built up the orbit method forsolvable groups. Duflo [6] constructed all irreducible unitary repre-sentations of almost algebraic groups from certain coadjoint orbits,assuming one knew unitary dual of reductive groups. It suggested thatto built up the orbit method for general Lie groups, one should con-centrate on reductive groups. The quantization problem of coadjointorbits of reductive groups can be reduced to quantizing nilpotent coad-joint orbits. It has not been solved completely, but there are somewonderful results (see Vogan [25], Chapter 10).In spirit by some work on how the branching laws behave in theorbit method, including Heckman [11] and Guillemin-Sternberg [10]for compact groups, Fujiwara [9] for exponential solvable groups, andthe work of Kobayashi [14] on the branching laws of reductive groups,Duflo [8] formulated a conjecture for the branching problem of discreteseries of almost algebraic group (see also Liu [15]).Let G be an (almost) algebraic group, let H be a closed algebraicsubgroup of G . Let g (resp. h ) be the Lie algebra of G (resp. of H ).Let g ∗ (resp. h ∗ ) be the dual of g (resp. of h ). Let π be a discrete seriesrepresentation of G . By Duflo [7], π is attached to a strongly regular G -coadjoint orbit O π . Consider the restriction of π to H , denoted by π | H , Duflo’s conjecture says that(i) π | H is H -admissible if and only if the moment map p : O π → h ∗ is weakly proper.(ii) If π | H is H -admissible, then each irreducible H -representation σ which appears in π | H is attached to a strongly regular H -coadjoint orbit Ω which is contained in p( O π ).(iii) If π | H is H -admissible, then the multiplicity of each such σ canbe expressed geometrically in terms of the reduced space of Ωwith respect to the moment map p.Let’s explain in details. The notion “almost algebraic group” isexplained in [7]. An element f ∈ g ∗ is called strongly regular if f isregular (i.e. the coadjoint orbit containing f is of maximal dimension)and its “reductive factor” s ( f ) := { X ∈ g ( f ) : ad X is semisimple } isof maximal dimension among the reductive factors of all the regularelements in g ∗ , where g ( f ) denotes the centralizer of f in g .Let O be a G -coadjoint orbit in g ∗ . Then O is equipped withthe Kirillov-Kostant-Souriau symplectic form ω and becomes an H -Hamiltonian space. The corresponding moment map is the naturalprojection p : O → h ∗ . GENERALIZATION OF DUFLO’S CONJECTURE 3
In (i), the notion “ H -admissible” is due to Kobayashi, which meansthat π | H is discretely decomposable (i.e. π | H is a direct sum of ir-reducible unitary representations of H ) and all irreducible represen-tations of H has only finite multiplicities in π | H . The notion “weakproperness” means that the preimage (for p) of each compact subsetwhich is contained in p( O π ) ∩ Υ sr is compact in O π , where Υ sr denotesthe set of strongly regular elements in h ∗ .In (iii), the reduce space of Ω is equal to p − (Ω) /H .Duflo’s conjecture has been proved in some cases (see Liu [15]), andis also generalized by Liu-Yu [17], who verified Duflo’s conjecture forthe restriction of tempered representations of GL n ( k ) (for k = R or C ) to its mirabolic subgroup and gave a geometric interpretation ofthe Kirillov’s conjecture. Recently, Liu-Oshima-Yu [16] verifies Duflo’sconjecture for the restriction of irreducible unitary representations ofSpin( N,
1) to its parabolic subgroups.This article generalizes Duflo’s conjecture in the framework of Kir-illov’s conjecture in spirit by Liu-Yu [17]. We obtain a generalizationfor the irreducible unitary representations of GL n ( k ) (for k = R or C )which are attached to some coadjoint orbits in section 5.Let’s say a few words about the Kirillov’s conjecture. The conjec-ture says that the restriction of any irreducible unitary representationof GL n ( k ) (for k = R , C or Q p ) to the mirabolic subgroup is also irre-ducible. It was proved by Bernstein for p -adic fields [5], Sahi [19] fortempered representations of GL n ( R ) or GL n ( C ), and Sahi-Stein [21] forSpeh representations of GL n ( R ), and Baruch [4] for the archimedeanfields.Here are the main results of this article. Let k = R or C , let G = GL n ( k ), and let P = P n ( k ) be the mirabolic subgroup of G ,consisting of the elements whose last row is (0 , · · · , , p de-note the Lie algebra of P and p ∗ denote the dual of p . Assume that π is an irreducible unitary representation of G and π is attached tothe G -coadjoint orbit O π in section 5. It turns out that the image ofthe moment map p : O π → p ∗ contains finite P -coadjoint orbits, andthere is a unique dense P -coadjoint orbit in p( O π ), denoted by Ω. Themoment map p is proper over Ω. Moreover, π | P is attached to Ω in thesense of Duflo. It explores more about the geometry of the Kirillov’sconjecture.The article is organized as follows.In section 2, we present the classification of the P n ( k )-coadjoint or-bits and its proof, which comes from Liu-Yu [17].In section 3, we compute the moment map and obtain similar resultsas Liu-Yu [17] on the geometric side.In section 4, we summarize some results of Kirillov’s conjecture, fromwhich we obtain the restrictions of all irreducible unitary representa-tions of GL n ( k ) to P n ( k ). HONGFENG ZHANG
In section 5, firstly, we summarize the orbit method for reductivegroups, and obtain the correspondence between coadjoint orbits andirreducible unitary representations of GL n ( k ). Secondly, we show howto construct irreducible unitary representations from coadjoint orbitsof P n ( k ) in the sense of Duflo.In section 6, our generalization is presented and proved by compar-ing the results of the moment maps and the restrictions of irreducibleunitary representations of GL n ( k ) to P n ( k ).2. P n ( k ) -coadjoint orbit This section is mainly adapted from Liu-Yu [17], which will be usedin the rest of the article.2.1.
Coadjoint action, the dual map and the moment map.
Let k be the field R or C , and let n ∈ Z + . Set G n ( k ) = GL n ( k ) and P n ( k ) = { (cid:18) A α (cid:19) : A ∈ GL n − ( k ) , α ∈ k n − } be the mirabolic subgroup of G n ( k ). g n ( k ) denotes the Lie algebra of G n ( k ) and p n ( k ) = { (cid:18) A α (cid:19) : A ∈ gl n − ( k ) , α ∈ k n − } denotes the Lie algebra of P n ( k ). g n ( k ) ∗ and p n ( k ) ∗ denote the dualspaces of g n ( k ) and p n ( k ), respectively. G n ( k ) acts on g n ( k ) ∗ through( g · f ) ξ = f ( g − · ξ ) , ∀ g ∈ G n ( k ) , ∀ f ∈ g n ( k ) ∗ , ∀ ξ ∈ g n ( k ) . This is called the coadjoint action, and a G n ( k )-orbit in g n ( k ) ∗ is calleda coadjoint orbit. Similarly, one can define the coadjoint action of P n ( k )on p n ( k ) ∗ and the corresponding coadjoint orbit.By taking trace, we can define a G n ( k )-invariant nondegenerate bi-linear form on g n ( k ) × g n ( k ),( X, Y ) tr( XY ) , X, Y ∈ g n ( k ) . We have a G n ( k )-module isomorphismpr : g n ( k ) → g n ( k ) ∗ , X ( Y tr( XY ) , ∀ Y ∈ g n ( k )) , ∀ X ∈ g n ( k ) . Let pr ′ : g n ( k ) → p n ( k ) ∗ , X ( Y tr( XY ) , ∀ Y ∈ p n ( k )) , ∀ X ∈ g n ( k ), then ker pr ′ = { (cid:18) α t (cid:19) : α ∈ k n − , t ∈ k } . Let ¯ p n ( k ) = { (cid:18) A α (cid:19) | A ∈ g n − ( k ) , α t ∈ k n − } , then g n ( k ) =ker pr ′ ⊕ ¯ p n ( k ), and pr ′ | ¯ p n ( k ) is an isomorphism.We use pr n (resp. pr ′ n ) to denote pr (resp. pr ′ ) if there is ambiguity. GENERALIZATION OF DUFLO’S CONJECTURE 5
Classification of P n ( k ) -coadjoint orbit. Let L n ( k ) = { (cid:18) A
00 1 (cid:19) | A ∈ G n − ( k ) } and N n ( k ) = { (cid:18) I n − α (cid:19) | α ∈ k n − } . Let l n ( k ) and n n ( k ) denote the Lie algebra of L n ( k ) and N n ( k ), respec-tively.Then L n ( k ) is the Levi subgroup of P n ( k ), N n ( k ) is the unipotentradical of P n ( k ), and P n ( k ) = L n ( k ) ⋉ N n ( k ). Moreover, we have thefollowing exact sequence as P n ( k )-module0 → n n ( k ) → p n ( k ) → l n ( k ) → , so is 0 → l n ( k ) ∗ → p n ( k ) ∗ → n n ( k ) ∗ → . We will regard l n ( k ) ∗ (resp. n n ( k ) ∗ ) as a P n ( k )-submodule (resp. quo-tient module) of p n ( k ) ∗ in the following.Let h ∈ p n ( k ) ∗ such that h | l n ( k ) = 0, h | n n ( k ) = 0. We have followingkey proposition. Proposition 2.1. (1)
Every element in p n ( k ) ∗ is P n ( k ) -conjugatedto an element in l n ( k ) ∗ or h + l n ( k ) ∗ . (2) Stab P ( h | n n ( k ) ) = L n ( k ) h ⋉ N n ( k ) and ( h + l n ( k ) ∗ ) /N n ( k ) is h +( l n ( k ) h ) ∗ . (3) The stabilizer L n ( k ) h ∼ = P n − ( k ) .As a result, p n ( k ) ∗ /P n ( k ) ∼ = l n ( k ) ∗ /L n ( k ) G p n − ( k ) ∗ /P n − ( k ) . Proof. (1) Take any x ∈ p n ( k ) ∗ , if x | n n ( k ) = 0, then x ∈ l n ( k ) ∗ . Assumethat x | n n ( k ) = 0. As L n ( k ) acts on n n ( k ) ∗ transitively, x is conjugatedto an element in h + l n ( k ) ∗ .(2) As N n ( k ) is abelian, N n ( k ) acts trivially on n n ( k ) ∗ , so N n ( k ) ⊂ Stab P ( h | n n ( k ) ) . Using P n ( k ) = L n ( k ) ⋉ N n ( k ), we only need to compute Stab L n ( k ) ( h | n n ( k ) ).As h | l n ( k ) = 0, Stab L n ( k ) ( h | n n ( k ) ) = Stab L n ( k ) ( h ) = L n ( k ) h .Let X ∈ n n ( k ), exp( X ) ∈ N n ( k ) = exp n n ( k ), exp( X ) acts on h by h ( Y h (Ad(exp( − X ))( Y )) , ∀ Y ∈ p n ( k )) . Since Ad(exp( − X ))( Y ) = Y − [ X, Y ] , ∀ Y ∈ p n ( k ), we get that h (Ad(exp( − X ))( Y )) = h ( Y )+ h ( − [ X, Y ]) = ( h +ad( X ) h )( Y ) , ∀ Y ∈ p n ( k ) . HONGFENG ZHANG
As the pairing between l n ( k ) and l n ( k ) ∗ is nondegenerate, we only needto check that l n ( k ) h is the zero set of { ad( X )( h ) | X ∈ n } . Take any l ∈ l n ( k ) such that l vanishes at { ad( X )( h ) | X ∈ n } , that is, l (ad( X )( h )) = h ( − [ X, l ]) = − X (ad( l )( h )) = 0 , so l ∈ l n ( k ) h .(3) It is easy to check directly. (cid:3) We go to calculate Stab P n ( k ) ( x ) for any x ∈ p n ( k ) ∗ . Proposition 2.2.
Let x be an element in h + l n ( k ) ∗ , and let [ x ] be theimage of x in h + ( l n ( k ) h ) ∗ as Proposition 2.1.(2) and x ′ = [ x ] − h .Then Stab P n ( k ) ( x ) = Stab L n ( k ) h ( x ′ ) .Proof. Take any g ∈ Stab P n ( k ) ( x ), then g = ln , l ∈ L n ( k ), n ∈ N n ( k ).As n stabilizes h | n n ( k ) and l n ( k ) ∗ , l stabilizes h | n n ( k ) , hence also stabilizes h . As a result, we get a group morphism φ : Stab P n ( k ) ( x ) → Stab L n ( k ) h ( x ′ ) , g = ln l. We claim that φ is an isomorphism. If l ∈ Stab L n ( k ) h ( x ′ ), Proposition2.1 shows l n ( k ) ∗ / ad( n n ( k )) h = ( L n ( k ) h ) ∗ , and one can choose an n l ∈ N n ( k ), such that ln l ∈ Stab P n ( k ) ( x ). So weget a group morphism ψ : Stab L n ( k ) h ( x ′ ) → Stab P n ( k ) ( x ) , l ln l . It is obvious that φ ◦ ψ = 1, and it remains to check that φ is injective.Let ln ∈ ker( φ ), then n stabilizes h . It can be checked directly that n = 1. (cid:3) Proposition 2.3.
Let x be an element in ( l n ( k )) ∗ , then Stab P n ( k ) ( x ) =Stab L n ( k ) ( x ) ⋉ N n ( k ) .Proof. It is easy to check directly. (cid:3)
In conclusion, we get the following result.
Theorem 2.4.
Every element of p n ( k ) ∗ is P n ( k ) -conjugated to an el-ement of the form α = pr ′ ( ξ ) , where ξ = (cid:18) A J j (cid:19) , where A ∈ g n − j ( k ) , and J j = . . .. . . . . . ∈ g j ( k ) . The stabilizer
Stab P n ( k ) ( α ) is isomorphic to Stab G n − j ( k ) ( A ) ⋉ N n − j +1 ( k ) . GENERALIZATION OF DUFLO’S CONJECTURE 7
Proof.
Choose h = pr ′ ( (cid:18) ( n − × ( n − ( n − × ( n − × × ( n − (cid:19) ), applyingProposition 2.1, we get that every element of p n ( k ) ∗ is P n ( k )-conjugatedto pr ′ ( (cid:18) A (cid:19) ) for some A ∈ g n − ( k ) or pr ′ ( (cid:18) A (cid:19) ) + h , where A ∈ ¯ p n − ( k ). Using the result of Proposition 2.1 and by induction, onecan prove the first statement. Applying Proposition 2.2 and 2.3, wecan get the stabilizer as the statement. (cid:3) Definition 2.1.
Define the depth of such element in p n ( k ) ∗ as j . It iswell defined by the proof of Theorem 2.4. Corollary 2.5.
The elements with depth n form a unique dense open P n ( k ) -orbit in p n ( k ) ∗ and they are all strongly regular elements in p n ( k ) ∗ .Proof. Applying Theorem 2.4, we have α ∈ p n ( k ) ∗ has depth n if andonly if Stab P n ( k ) ( α ) has minimal dimension, if and only if Stab P n ( k ) ( α ) =1. So we get the statement immediately. (cid:3) Example 2.1.
Set B = a . . . a n − b · · · b n − ∈ g n ( k ) ,a i = a j when ≤ i = j ≤ n − , b i = 0 , when ≤ i ≤ n − , and f = pr ′ ( B ) ∈ p n ( k ) ∗ . Then f has depth n .Proof. It is sufficient to check that Stab P n ( k ) ( f ) = { } . See Liu-Yu [17]for a proof. An alternative proof is given in Lemma 3.2 and Lemma3.3. (cid:3) Geometry of the moment map p : O f → p ∗ Moment map in the GL n ( C ) case. Let n ∈ Z + , let G = G n ( C )and let P = P n ( C ) be the mirabolic subgroup of G . g (resp. p ) denotesthe Lie algebra of G (resp. of P ).Let g ∗ (resp. p ∗ ) denote the dual space of g (resp. of p ). By the Lietheory, we have n = min x ∈ g ∗ dim(Stab G ( x )) . For any element f ∈ g ∗ , we call f a regular element in g ∗ , or say that f is regular, if dim(Stab G ( f )) = n . And we call a G -coadjoint orbit O f a regular orbit if f is regular.We will calculate the moment map p : O f → p ∗ for any f ∈ g ∗ . Wefind that the result about any regular element f ∈ g ∗ is similar to theresults about semisimple regular elements, for example, p( O f ) containsthe unique strongly regular orbit of p ∗ (see Liu-Yu [17]). Using the HONGFENG ZHANG results about the regular elements, we can settle the cases of singularelements in g ∗ .Furthermore, using the results of the case of g n ( C ), one can settlethe case of g n ( R ) immediately.3.1.1. Regular orbits.
Write J s ( a ) = a a . . . . . .1 a ∈ g s ( C ) , s ∈ Z and s ≥ , a ∈ C , and J ( a ) = a × , a ∈ C . Set J s = J s (0), ∀ s ∈ Z + .Let a i ∈ C , s i ∈ Z + , ∀ ≤ i ≤ m , such that a i = a j when 1 ≤ i = j ≤ m , and P mi =1 s i = n . Set ξ = diag( J s ( a ) , · · · , J s m ( a m )) ∈ g , and set f = pr( ξ ) ∈ g ∗ . It is clear that every regular G -coadjoint orbitin g ∗ is of the form O f .Firstly, we describe all P -orbits in O f .For any s ∈ Z + , let L s = Stab G s ( C ) ( J s ) = { x x x ... . . . . . . x s · · · x x | x i ∈ C , ≤ i ≤ s, x = 0 } , Set L = Stab G ( f ) = { diag( X s , · · · , X s m ) | X s i ∈ L s i , ≤ i ≤ m } . We have O f ∼ = G/L and P \ O f ∼ = P \ G/L .Define the right action of G on C n by v · g = ( x , · · · , x n ) A, where v = ( x , · · · , x n ) t ∈ C n and g = A ∈ G n ( C ). Set v =(0 , · · · , , t , we get Stab G ( v ) = P , and C n − { } ∼ = P \ G .To find the representatives of P \ O f , we need to find the represen-tatives of ( C n − { } ) /L . Proposition 3.1.
Set I = { , , · · · , n } , K = { , · · · , m } and s = 0 .For any ∅ 6 = K ⊆ K and r k ∈ { , , · · · , s k } for k ∈ K , define I = { k − X j =0 s j + r k | k ∈ K } ⊆ I attached to K and { r k , k ∈ K } . Define v I = ( x , · · · , x n ) by x i = 1 when i ∈ I and x i = 0 when i / ∈ I . Let I a be the set of all I ’s constructedas above.Then { v I | I ∈ I a } form all different representatives of ( C n − { } ) /L . GENERALIZATION OF DUFLO’S CONJECTURE 9
Proof.
The L -orbit of v I is the set { ( y , · · · , y n ) t | y i ∈ C , ≤ i ≤ n ; y i = 0 if i ∈ I ; y i = 0 if i / ∈ J,J = G k ∈ K { k − X j =0 s j + 1 , · · · , k − X j =0 s j + r k }} . It follows that { v I | I ∈ I a } are all different representatives of ( C n −{ } ) /L . (cid:3) Let g I ∈ G be the element corresponding to the representative v I under the isomorphism C n − { } ∼ = P \ G , then G = G I ∈ I a P g I L. So there are I a = Q ≤ i ≤ m ( s i + 1) − P -orbits in O f , { P · ( g I · f ) | I ∈ I a } . Set S = { s , s + s , · · · , n } , then P g S L is the unique dense opensubset of G among { P g I L, I ∈ I a } , since v S · L is the unique dense L -orbit in C n − { } . As a consequence, P · ( g S · f ) is the unique denseopen P -orbit in O f .By definition, we can choose { g I | I ∈ I a } as follows.When n ∈ I , g I = (cid:18) I n − β (cid:19) , β = ( x , · · · , x n − ) is defined by x i = 1 when i ∈ I − { n } and x i = 0 when i / ∈ I − { n } .When n / ∈ I , let k = max { i : i ∈ I } , g I = I k − I n − k β , β = ( x , · · · , x k − ) is defined by x i = 1 when i ∈ I − { k } and x i = 0when i / ∈ I − { k } .We go to calculate the image of each P -orbit of g I · f under themoment map p : O f → p ∗ and find that p( O f ) contains the uniquedense open P -orbit in p ∗ , more precisely, p( g S · f ) is a strongly regularelement in p ∗ . Lemma 3.2.
We have
Stab P ( g S · f ) = 1 , so the dimension of P -orbitof g S · f equals to dim P = n − n .Proof. It is obvious thatStab P ( g S · f ) = P ∩ g S Stab G ( f ) g − S . And P ∩ g S Stab G ( f ) g − S = P ∩ g S Lg − S = g S ( g − S P g S ∩ L ) g − S . Since g − S P g S is the stabilizer of v · g S = v S , we have g − S P g S ∩ L =Stab L ( v S ).By direct calculation, Stab L ( v S ) = 1, so Stab P ( g S · f ) = 1. (cid:3) For convenience, let [ U , · · · , U k ] denote an n × n matrix with k blocks, where the i -th block is an n × n i (1 ≤ n i ≤ n ) submatrix U i for1 ≤ i ≤ k , and P ki =1 n i = n . Lemma 3.3. If x ∈ p ∗ is not strongly regular, then for any element z ∈ p ′− ( x ) , the P -orbit of z has dimension < n − n . Here p ′ : g ∗ → p ∗ is the natural projection.Proof. Take any element x = pr ′ ([ X, n × ]) ∈ p ∗ which is not stronglyregular, by the definition, p ′− ( x ) = { pr([ X, ˜ Y ]) , ˜ Y ∈ C n } . Fix anelement z ∈ p ′− ( x ), z = pr([ X, Y ]) for some Y = ( y , · · · , y n ) t ∈ C n .Since the P -orbit of z has dimensiondim P − dim(Stab P ( z )) = n − n − dim(Stab P ( z )) , we only need to prove dim(Stab P ( z )) ≥ { pr([0 , ˜ Y ]) , ˜ Y ∈ C n } is a P -stable subspace, Stab P ( z ) ⊆ Stab P ( x ).Using the result of the classification of P -coadjoint orbits in p ∗ , weknow that x is P -conjugated to an element of the form α = pr ′ ( (cid:18) A J k (cid:19) ),where A ∈ g n − k ( C ),Stab P ( α ) ∼ = Stab G n − k ( C ) ( A ) ⋉ N n − k +1 ( C )with dimension ≥ n − k ) and k ≤ n − x is not strongly regular.It doesn’t matter to replace x by the elements in its P -orbit and weassume that x = α .By direct calculation, we getStab P ( x ) = { (cid:18) C I k (cid:19) | C ∈ G n − k ( C ) , CAC − = A } ⋉ { (cid:18) I n − k NI k (cid:19) | N = [ N , · · · , N k ] , N i ∈ C n − k , ≤ i ≤ k, N i = A i − N , ≤ i ≤ k } . Let t = (cid:18) C CNI k (cid:19) ∈ Stab P ( x ), then t ∈ Stab P ( z ) if and only if (cid:18) C CNI k (cid:19) ( (cid:18) A J k (cid:19) +[0 , Y ]) (cid:18) C − − NI k (cid:19) = (cid:18) A J k (cid:19) +[0 , Y ] , So t ∈ Stab P ( z ) ⇔ − CA k N + C y ... y n − k + CN y n − k +1 ... y n = y ... y n − k . Since 1 ∈ Stab P ( z ), we obtain a nonempty subvariety Stab P ( z ) ofStab P ( x ) given by n − k algebraic equations. Thereforedim(Stab P ( z )) ≥ dim(Stab P ( x )) − ( n − k ) . Since dim(Stab P ( x )) ≥ n − k ) and n − k ≥
1, we get dim(Stab P ( z )) ≥ (cid:3) GENERALIZATION OF DUFLO’S CONJECTURE 11
Proposition 3.4. p( g S · f ) is a strongly regular element in p ∗ .Proof. By Lemma 3.2, the P -orbit of g S · f has dimension n − n .Applying Lemma 3.3, we see that p( g S · f ) is strongly regular. (cid:3) For convenience, let A ( I , I ) denote the submatrix of the matrix A with rows indexed by I and columns indexed by I , and let a ( I )denote the subvector of the vector a with index I .With the result above, we can calculate the P -orbit of p( g I · f ) forany I ∈ I a . We find out that { p( g I · f ) | I ∈ I a } represent all different P -orbits in p( O f ). More precisely, we have the following result. Proposition 3.5.
For any ∅ 6 = K ⊆ K and r k ∈ { , , · · · , s k } for k ∈ K , define I = { P k − j =0 s j + r k | k ∈ K } and g I as above. Then p( g I · f ) has depth d = P k ∈ K r k . Moreover, if we set t k = (cid:26) r k , k ∈ K, , k ∈ K − K, then p( g I · f ) is P -conjugated to pr ′ ( η ) , where η = diag( J s − t ( a ) , · · · , J s m − t m ( a m ) , J d ) . Proof.
Firstly, we change p( g I · f ) = pr ′ ( g I · ξ ) to a good form by aseries of explicit P -conjugations, then we use the result of Proposition3.4 to get the statement. Set A I = g I · ξ .In the case of n ∈ I , by direct calculation, we have A I = diag( J s ( a ) , · · · , J s m ( a m )) + (cid:18) ( n − × n α (cid:19) , where α = ( α , · · · , α m ), α k = ( α k, , · · · , α k,s k ) is defined by α k = (0 , · · · , | {z } s k ) , k / ∈ K − { n } , ( a k − a m , , · · · , | {z } s k − ) , k ∈ K − { n } , r k = 1(0 , · · · , | {z } r k − , , a k − a m , , · · · , | {z } s k − r k ) , k ∈ K − { n } , r k > . We claim that pr ′ ( A I ) is P -conjugated to pr ′ ( ζ ), where ζ = J t ( a ) J s − t ( a ). . . J t m ( a m ) J s m − t m ( a m ) + (cid:18) ( n − × n α (cid:19) . It is sufficient to prove that pr ′ ( A I ) is P -conjugated to pr ′ ( ζ k ) for0 ≤ k ≤ m , where ζ k = J t ( a ) J s − t ( a ). . . J t k ( a k ) J s k − t k ( a k ) J s k +1 ( a k +1 ). . . J s m ( a m ) + (cid:18) ( n − × n α (cid:19) , since ζ m = ζ .Let’s prove it by induction on k . When k = 0, it is trivially true.Assume that k ≥ k − k / ∈ K , or if k ∈ K and r k = s k , then ζ k = ζ k − and so it istrue for k . Assume that k ∈ K, r k < s k , we go to prove that it is truefor k . Since we assume that pr ′ ( A I ) is P -conjugated to pr ′ ( ζ k − ), it issufficient to prove that pr ′ ( ζ k − ) is P -conjugated to pr ′ ( ζ k ).Step 1. We have ζ k − = B C D C B v w v a m , where B = diag( J t ( a ) , J s − t ( a ) , · · · , J t k − ( a k − ) , J s k − − t k − ( a k − )) ,B = diag( J s k +1 ( a k +1 ) , · · · , J s m − ( a m − ) , J s m − ( a m )) ,C = J r k ( a k ) , C = J s k − r k ( a k ) , v = ( α , · · · , α k − ) , w = ( α k, , · · · , α k,r k ) ,v = ( α k +1 , · · · , α m − ) , s m = 1 , ( α k +1 , · · · , α m − , , · · · , | {z } s m − , , s m > ,D is the ( s k − r k ) × r k matrix with 1 in the position (1 , r k ) and zero elsewhere . Let P = I s + ··· + s k − I r k N I s k − r k N I s k +1 + ··· + s m −
00 0 0 0 1 with N = [0 ( s k − r k ) × ( r k − , n ], N = − n and n = ( 1 a k − a m , − a k − a m ) , · · · , ( − s k − r k +1 ( a k − a m ) s k − r k ) t ∈ C s k − r k . GENERALIZATION OF DUFLO’S CONJECTURE 13
By direct calculation, we get P ζ k − P − = B C N v C N v a m N − A N B v w v a m . Step 2. Let P = I s + ··· + s k − I r k M I s k − r k M
00 0 0 I s k +1 + ··· + s m −
00 0 0 0 1 ,M is an ( s k − r k ) × ( s + · · · + s k − ) matrix, and M is an ( s k − r k ) × ( s k +1 + · · · + s m −
1) matrix, then P ( P ζ k − P − ) P − = B C N v + ( M B − C M ) 0 C N v + ( M B − C M ) a m N − A N B v w v a m . By solving the linear equations, one see that there exists M and M ,such that N v + ( M B − C M ) = 0 , and N v + ( M B − C M ) = 0 . Actually, we can write B = D ′ + N ′ and C = D ′′ + N ′′ as Jordandecomposition, where D ′ , D ′′ are semisimple matrices and N ′ , N ′′ arenilpotent matrices. It is easy to see that the linear map M M N ′ − N ′′ M is nilpotent, the linear map M M D ′ − D ′′ M is semisimple, and they commute with each other. Moreover, the linearmap M M D ′ − D ′′ M is nondegenerate since B and C have nocommon eigenvalues. Therefore, we can easily see that the equation N v + ( M B − C M ) = 0 has a (unique) solution M . Similarly wecan get M .Using such P defined as above, we have pr ′ ( P ( P ζ k − P − ) P − ) =pr ′ ( ζ k ). By induction, one can finish the proof of the claim.Let ζ ′ be the submatrix of ζ with the rows and columns indexed by G h ∈ K { h − X j =0 s j + 1 , · · · , h − X j =0 s j + r h } . Applying Proposition 3.4, we see that pr ′ d ( ζ ′ ) is P d ( C )-conjugated topr ′ d ( J d ), where pr ′ d : g d ( C ) → p d ( C ) ∗ . So pr ′ ( ζ ) is P -conjugated to theelement pr ′ ( η ) in the statement, and so is p( g I · ξ ) .In the case of n / ∈ I, m ∈ K , one can prove the statement similarly.We have A I = B C B v v a m , where B = diag( J s ( a ) , · · · , J s m − ( a m − ) , J r m − ( a m )) , B = J s m − r m ( a m ) ,C = (cid:18) − v I (1 , · · · , n − ( s m − r m ) − ( s m − r m ) × ( n − ( s m − r m ) − (cid:19) ,v = (1 , , · · · , | {z } s m − r m − ) t , v = ( α (1 , · · · , n − s m ) , , · · · , | {z } r m − , , r m ≥ α (1 , · · · , n − s m ) , r m = 1 . We can use the method in the step 2 to show that p( A I ) is P -conjugatedto pr( ζ ′ ), where ζ ′ = B B v v a m . Then the submatrix of ζ ′ with rows and columns indexed by { , · · · , n }−{ n − ( s k − r k ) , · · · , n − } is (cid:18) B v a m (cid:19) , which is of the form in the case of n ∈ I . Applying the result of thecase of n ∈ I , we can get the statement in the case of n / ∈ I, m ∈ K .In the case of n / ∈ I, m / ∈ K , assume m = max { k ∈ K } . By directlycalculation, we get that the submatrix of A I with rows and columnsindexed by { , , · · · , s + · · · + s m − } ∪ { n } is of the form in theabove two cases. Applying the results of the above two cases, we canget the statement in the case of n / ∈ I, m / ∈ K .This completes the proof of the statement. (cid:3) General orbits.
We go to calculate the moment map p : O f → p ∗ for any f ∈ g ∗ . Write J lk ( a ) = diag( J k ( a ) , · · · , J k ( a ) | {z } l blocks ) , for any a ∈ C , k, l ∈ Z + .Let m ∈ Z + , m ≤ n . Let a , · · · , a m ∈ C , a i = a j when 1 ≤ i = j ≤ m . Set J lk,a i = J lk ( a i ) GENERALIZATION OF DUFLO’S CONJECTURE 15 for 1 ≤ i ≤ m , k, l ∈ Z + . Let ξ = diag( J l k ,a , J l k ,a , · · · , J l r k r ,a , · · · , J l m k m ,a m , J l m k m ,a m , · · · , J l mrm k mrm ,a m ) , where r j ∈ Z + , ≤ j ≤ m ; and l ji , k ji ∈ Z + , ∀ ≤ j ≤ m, ≤ i ≤ r j ; k ji < k ji when 1 ≤ j ≤ m , 1 ≤ i < i ≤ r j ; and X ≤ i ≤ m, ≤ i ≤ r j k ji · l ji = n. So ξ has m different eigenvalues, denoted by a j , ≤ j ≤ m , and has r j different kinds of Jordan blocks for the eigenvalue a j , with size k ji and l ji pieces of such size for 1 ≤ i ≤ r j .Set f = pr( ξ ). It is clear that every orbit in g ∗ is of the form O f .Now, we fix a ξ , and set ξ j = diag( J l j k j ,a j , J l j k j ,a j , · · · , J l jrj k jrj ,a j ), and n j = P r j r =1 k jr · l jr for 1 ≤ j ≤ m . For convenience, set n = 0, k j = 0 , l j = 0 for 1 ≤ j ≤ m .Let M s × s ( C ) be the set of all s × s matrices over C . Write K ( x , · · · , x s ) = x x x ... . . . . . . x s · · · x x , x i ∈ C , ≤ i ≤ s, then { X ∈ M s × s ( C ) | XJ s = J s X } = { K ( x , · · · , x s ) | x i ∈ C , ≤ i ≤ s } , and we have the following Proposition. Proposition 3.6.
For any a ∈ C , k, l ∈ Z + , L k,l = Stab GL k · l ( C ) ( J lk ( a )) = { K ( x , , · · · , x ,k ) · · · K ( x l, , · · · , x l,k ) ... . . . ... K ( x l , , · · · , x l ,k ) · · · K ( x ll, , · · · , x ll,k ) | x ij,h ∈ C , ∀ ≤ i, j ≤ l, ≤ h ≤ k, det( x ij, ) ≤ i,j ≤ l = 0 } . Proof.
By direct calculation, we see that the determinate of the ele-ment on the right hand side equals to (det( x ij, ) ≤ i,j ≤ l ) k and get thestatement immediately. (cid:3) For any k , k ∈ Z + such that k ≤ k , and x , · · · , x k ∈ C , define H ( x , · · · , x k ) as the k × k matrix[ K ( x , · · · , x k ) , k × ( k − k ) ] , and define H ( x , · · · , x k ) is the k × k matrix (cid:18) ( k − k ) × k K ( x , · · · , x k ) (cid:19) . Proposition 3.7.
For any a ∈ C , k , k , l , l ∈ Z + , k ≤ k , M l ,l k ,k = { X ∈ M k l × k l ( C ) | X ( J l k ( a )) = ( J l k ( a )) X } = { H ( x , , · · · , x ,k ) · · · H ( x l , , · · · , x l ,k ) ... . . . ... H ( x l , , · · · , x l ,k ) · · · H ( x l l , , · · · , x l l ,k ) | x ij,h ∈ C , ∀ ≤ i ≤ l , ≤ j ≤ l , ≤ h ≤ k } ,N l ,l k ,k = { X ∈ M k l × k l ( C ) | X ( J l k ( a )) = ( J l k ( a )) X } = { H ( x , , · · · , x ,k ) · · · H ( x l , , · · · , x l ,k ) ... . . . ... H ( x l , , · · · , x l ,k ) · · · H ( x l l , , · · · , x l l ,k ) | x ij,h ∈ C , ∀ ≤ i ≤ l , ≤ j ≤ l , ≤ h ≤ k } . Proof.
It can be checked easily. (cid:3)
For convenience, we fix j and set r = r j , k u = k ju , l u = l ju , for1 ≤ u ≤ r j , in the Proposition 3.8 and its proof. Proposition 3.8.
The stabilizer L j = Stab GL nj ( C ) ( ξ j ) = { X X · · · X r X X · · · X r ... ... . . . ... X r X r · · · X rr | X uu ∈ L k u ,l u , ∀ ≤ u ≤ r,X st ∈ M l s ,l t k s ,k t , ∀ ≤ s < t ≤ r, X st ∈ N l s ,l t k s ,k t , ∀ ≤ t < s ≤ r } . Proof.
It is clear thatStab GL nj ( C ) ( ξ j ) ⊆ { X X · · · X r X X · · · X r ... ... . . . ... X r X r · · · X rr | X st ∈ M l s ,l t k s ,k t , ∀ ≤ s ≤ t ≤ r,X st ∈ N l s ,l t k s ,k t , ∀ ≤ t < s ≤ r } , so we only need to prove the element on the right hand side is invertibleif and only if X uu is invertible for all 1 ≤ u ≤ r . This follows fromthe fact that the determinant of the element on the right hand side isequal to Q ≤ u ≤ r det X uu . (cid:3) It is clear thatStab G ( f ) = Stab G ( ξ ) = { diag( X , · · · , X m ) | X i ∈ L i , ≤ i ≤ m } . And we have O f ∼ = G/L and P \ O f ∼ = P \ G/L .Define the right action of G on C n by v · g = ( x , · · · , x n ) A, GENERALIZATION OF DUFLO’S CONJECTURE 17 where v = ( x , · · · , x n ) t ∈ C n and g = A ∈ G n ( C ). Set v =(0 , · · · , , t , we get Stab G ( v ) = P , and C n − { } ∼ = P \ G .To calculate P \ G/L , we need to calculate ( C n − { } ) /L . Actually,we only need to calculate ( C n j − { } ) /L j . Once we obtain all thedifferent representatives of ( C n j − { } ) /L j , denoted by V j , we can getall the different representatives of ( C n − { } ) /L as follows, { v = ( v , · · · , v m ) = 0 | v i ∈ C n i , v i ∈ V i or v i = 0 , ≤ i ≤ m } . We fix j and adopt the same notation as Proposition 3.8. We havefollowing proposition about ( C n j − { } ) /L j . Proposition 3.9.
Write R = { , · · · , r } . For any ∅ 6 = R ⊂ R , and ≤ x i ≤ k i for i ∈ R , such that (1) x i < x i ′ for any i, i ′ ∈ R such that i < i ′ , (2) k i − x i < k i ′ − x i ′ for any i, i ′ ∈ R such that i < i ′ ,we define I = { i − X u =0 k u l u + k i ( l i −
1) + x i , i ∈ R } ⊆ { , , · · · , n j } , and v I = ( y , y , · · · , y n j ) , y i = 1 when i ∈ I and y i = 0 when i / ∈ I .Let I a be the set of all I ’s constructed as above. Then { v I | I ∈ I a } represent all different L j -orbits in C n j − { } .Proof. We only need to prove that any L j -orbit in ( C n j − { } ) is of theform v I · L j for some I given as above, and such orbits are differentfrom each other.For any k, l ∈ Z + , set e ( k, l ) = (0 , · · · , | {z } k − , , , · · · , | {z } l − k ) ∈ C l . For1 ≤ i ≤ r , set t i = { i − X u =0 k u l u + 1 , i − X u =0 k u l u + 2 , · · · , i X u =0 k u l u } . Let z = ( z , · · · , z n j ) ∈ C n j − { } . We use z ( t i ) to denote the subvectorof z with the positions indexed by t i .For any 1 ≤ i ≤ r , we show that one can use the right action of L j to change z to z ′ , such that z ′ ( t i ) equals 0 or e ( k i ( l i −
1) + x i , k i l i ), forsome 1 ≤ x i ≤ k i , without changing other elements of z .If z ( t i ) = 0, we don’t need to change z any more. Otherwise, take v i to be the maximal integer such that 1 ≤ v i ≤ k i and z ( i − X u =0 k u l u + k i · w + v i ) = 0for some integer 0 ≤ w ≤ l i −
1. Applying Proposition 3.6, we canrealize z ( t i ) as the k i ( l i −
1) + v i -th row of some element in L k i ,l i , denoted by ˜ L . By the right action of the elementdiag( I k l , · · · , I k i − l i − , ˜ L − , I k i +1 l i +1 , · · · , I k r l r ) ∈ L j , one can change z to z ′ , such that z ′ ( t i ) = e ( k i ( l i −
1) + v i , k i l i ), withoutchanging other elements of z .Now we can get all representatives of ( C n j − { } ) /L j . For any ∅ 6 = R ⊆ R , and 1 ≤ x i ≤ k i , i ∈ R , define J = { i − X u =0 k u l u + k i ( l i −
1) + x i , i ∈ R } ⊆ { , , · · · , n j } and v J = ( y , y , · · · , y n j ), y i = 1 when i ∈ J and y i = 0 when i / ∈ J .Then such v J ’s include all representatives of ( C n j − { } ) /L j . However,some of them may be the same representatives.We go to prove that if x i ≥ x i ′ for some i < i ′ , then v I ′ · L j = v I · L j with I ′ = I − { P i ′ − u =0 k u l u + k i ′ ( l i ′ −
1) + x i ′ } . For convenience, set a = P i − u =0 k u l u + k i ( l i −
1) and b = P i ′ − u =0 k u l u + k i ′ ( l i ′ − L ′ = I n j + X ≤ k ≤ k i − x i + x i ′ E a + x i − x i ′ + k,b + k ∈ L j . Moreover, v I ′ · ˜ L ′ = v I , so v I ′ · L j = v I · L j .If k i − x i ≥ k i ′ − x i ′ for some i < i ′ , then v I ′′ · L j = v I · L j , I ′′ = I − { P i − u =0 k u l u + k i ( l i −
1) + x i } . For convenience, set a = P i − u =0 k u l u + k i ( l i −
1) and b = P i ′ − u =0 k u l u + k i ′ ( l i ′ − L ′′ = I n i ′ + X ≤ k ≤ k i ′ − x i ′ + x i E b + x i ′ − x i + k,a + k ∈ L j . Moreover, v I ′′ · ˜ L ′′ = v I , so v I ′′ · L j = v I · L j .Now, we have proved that the set of { v I | I ∈ I a } contains all rep-resentatives of ( C n j − { } ) /L j . It remains to prove that the orbits of { v I | I ∈ I a } are different from each other. Assume v I · L j = v ˜ I · L j , ˜ I is corresponding to ˜ R and ˜ x i . Then by direct calculation of the orbit v I · L j and v ˜ I · L j , one see that r ∈ R ⇒ r ∈ ˜ R, and then ˜ x r = x r . And get that i ∈ R ⇒ i ∈ ˜ R , and ˜ x i = x i for i = r − , · · · , (cid:3) Remark 3.1.
Let R = { r } and x r = k r , define I o = { n j } , and thecorresponding v I o . From the structure of L j , we can see that the L j -orbit of v I o is the unique dense open orbit in C n j − { } . Remark 3.2.
If we just assume that “ k ji = k ji ′ for any ≤ i = i ′ ≤ r j ”rather than “ k ji < k ji ′ for any ≤ i < i ′ ≤ r j ”, the conditions (1) and(2) in the Proposition 3.9 should be replaced by (1)’ x i < x i ′ for any GENERALIZATION OF DUFLO’S CONJECTURE 19 i, i ′ ∈ R such that k ji < k ji ′ , and (2)’ k i − x i < k i ′ − x i ′ for any i, i ′ ∈ R such that k ji < k ji ′ . With the result above, we can get the representatives of ( C n −{ } ) /L immediately. Proposition 3.10.
Set M = { , , · · · , m } and R j = { , , · · · , r j } , ≤ j ≤ m . For any ∅ 6 = M ⊆ M , and ∅ 6 = R j ⊆ R j for each j ∈ M , and ≤ x ji ≤ k ji for i ∈ R j , j ∈ M satisfying the followingtwo conditions, (1) x ji < x ji for any i , i ∈ R j such that i < i , (2) k ji − x ji < k ji − x ji for any i , i ∈ R j such that i < i ,set I = G j ∈ M { j − X v =0 n v + i − X u =0 k ju l ju + k ji ( l ji −
1) + x ji , i ∈ R j } ⊆ { , , · · · , n } , and v I = ( y , y , · · · , y n ) , y i = 1 when i ∈ I and y i = 0 when i / ∈ I .Let I a be the set of all such I ’s. Then { v I | I ∈ I a } form all differentrepresentatives of ( C n − { } ) /L .Proof. It follows from the Proposition 3.9. (cid:3)
Remark 3.3.
Set M = M , R j = { r j } and x jr j = k jr j for ≤ j ≤ m .Let I o be the set corresponding to such M , R j and x jr j , i.e. I o = { n , n + n , · · · , n } . Then v I o is the unique dense open L -orbit in C n − { } . We can define g I in the same way as the contents about regular orbitsand such g I · f ’s form all different representatives of P \ O f and theorbit of g I o · f is the unique dense open orbit among them.We go to calculate the moment map p : O f → p ∗ . Let’s fix some I corresponding to the given M , R j ( j ∈ M ) and 1 ≤ x ji ≤ k ji ( i ∈ R j , j ∈ M ). For any j , let q j = R j , and R j = { c j , · · · , c jq j } with c j < c j < · · · < c jq j . For j ∈ M , set x jp = x jc jp , k jp = k jc jp for1 ≤ p ≤ q j , x j = 0, and d j = x jq j . Proposition 3.11.
Given I as above, then p( g I · f ) has depth d = P j ∈ M d j , and is P -conjugated to the element pr ′ ( η ) , where η = diag( A , · · · , A m , J d ) , where A j = ξ j if j / ∈ M ; and A j = diag( A j , A j , · · · , A jr j ) , if j ∈ M ,where A ji ’s are defined as follows, A ji = J l ji k ji ,a j if i / ∈ R j , and A ji = diag( J l ji − k ji ,a j , J t,a j ) with t = k jh − x jh + x j ( h − , if i = c jh ∈ R j for ≤ h ≤ q j . Proof.
As in the proof of Proposition 3.5, by block calculation, we canassume l ji = 1, ∀ ≤ i ≤ r j and R j = { , · · · , r j } for all 1 ≤ j ≤ m atthe beginning. Hence, d j = x jr j for 1 ≤ j ≤ m . For convenience, set A I = g I · ξ .In the case of n ∈ I , we have r m = 1 since { x mi , i ∈ R m } satisfiescondition (1) and (2).We claim that pr ′ ( A I ) is P -conjugated pr ′ ( ζ j ) for 0 ≤ j ≤ m − ζ j = A J d ,a . . . A j J d j ,a j ξ j +1 . . . ξ m + (cid:18) ( n − × n α j (cid:19) , and α j = ( β ′ , · · · , β ′ j , β j +1 , · · · , β m ) . When 1 ≤ i ≤ m − β ′ i and β i are defined as follows. β ′ i = (0 , · · · , | {z } n i − , , a i − a m ) , d i > , (0 , · · · , | {z } n i − , a i − a m ) , d i = 1 , and β i = ( β i , · · · , β ir i ), for 1 ≤ h ≤ r i , β ih = (0 , · · · , | {z } x ih − , , a i − a m , , · · · , | {z } k ih − x ih ) , x ih > a i − a m , , · · · , | {z } k ih − ) , x ih = 1 . And β m = (0 , · · · , | {z } n m ).It is trivially true that pr ′ ( A I ) is P -conjugated pr ′ ( ζ j ) for j = 0.Assume that it is true for j −
1, let’s prove that pr ′ ( ζ j − ) is P -conjugatedpr ′ ( ζ j ), so we get that pr ′ ( A I ) is P -conjugated pr ′ ( ζ j ).Recall that n j ′ = P ≤ i ≤ r j ′ k j ′ i for 1 ≤ j ′ ≤ m , n = 0. For conve-nience, set r = r j , k u = k ju for 1 ≤ u ≤ r j , x h = x jh for 1 ≤ h ≤ r j , x = 0. Set V = { j − X j ′ =0 n j ′ + 1 , j − X j ′ =0 n j ′ + 2 , · · · , j − X j ′ =0 n j ′ + r X u =1 k u } and U = V ∪ { n } . GENERALIZATION OF DUFLO’S CONJECTURE 21
We claim that one can use P -conjugations, which only change theelements in the positions ( U, V ) or in the n -th column, to change ζ j − to B such that B ( U, V ) = (cid:18) diag( J e ( a j ) , · · · , J e r ( a j ) , J x r ( a j )) β ′ j (cid:19) , where e h = k h − x h + x h − , ≤ h ≤ r .Step 1. Let P = I n + ··· + n j − I k E I k E . . .. . . . . . E r − I k r I n j +1 + ··· + n m , where E i is the k i +1 × k i matrix, with − x i +1 , x i ) , ( x i +1 − , x i − , · · · , ( x i +1 − x i + 1 , x i +1 > x i ) and 0 in otherpositions, for 1 ≤ i ≤ r − B = P − ζ j − P , then B ( U, V ) = J k ,a j E ′ J k ,a j E ′ . . .... . . . . . . . . .0 0 · · · E ′ r − J k r ,a j · · · β jr , where E ′ i is k i +1 × k i matrix with − x i +1 + 1 , x i ) and0 in other positions, for 1 ≤ i ≤ r − P -conjugation, which just change the elements in the positions ( U, V ) orin the n -th column, to change B to B such that B ( U, V ) = J k ,a j E ′ J k ,a j E ′ . . .... . . . . . . . . .0 0 . . . E ′ r − J k r − ,a j · · · J x r ,a j · · · E ′′ r J k r − x r ,a j · · · β ′ j ,E ′′ r is the ( k r − x r ) × k r − matrix with 1 in the position (1 , x r − ) and 0in other positions. Step 3. Let P = I n + ··· + n j − I k F . . . . . .. . . F r − I k r − F r − I k r I n j +1 + ··· + n m ,F i is the k i × k i +1 matrix with − x i + 1 , x i +1 +1) , · · · , ( k i , x i +1 + k i − x i )(here we use k i +1 − x i +1 > k i − x i ), and 0 inother positions, for 1 ≤ i ≤ r .Let B = P − B P , then B ( U, V ) = J x ,a j J k − x ,a j J x ,a j E ′′ J k − x ,a j J x ,a j E ′′ J k − x ,a j ... ... ... . . . . . . . . . . . .... ... ... . . . . . . . . . . . . J x r − ,a j ... ... ... . . . . . . . . . . . . 0 J k r − − x r − ,a j J x r ,a j · · · E ′′ r − J k r − x r ,a j · · · β ′ j , where E ′′ i is the ( k i +1 − x i +1 ) × x i matrix with 1 in the position (1 , x i )and 0 in other positions, for 1 ≤ i ≤ r −
1. Now we can easily see B is P -conjugated to ζ j . This finishes the proof of the claim.Applying Proposition 3.5, by block calculation, we can easily showthat pr ′ ( ζ m − ) is P -conjugated to pr ′ ( η ). This finish the proof of thiscase.In the case of n / ∈ I , using similar method we can also get thestatement.This completes the proof of the statement. (cid:3) Theorem 3.12.
We have that p : O f → p ∗ sends different P -orbitsof O f to different P -orbits of p ∗ , the image p( O f ) contains a uniquedense orbit P · ( g I o · f ) , and p is proper over P · ( g I o · f ) . The reducespace of the unique open dense orbit is singleton.Proof. The first statement and the last statement is from the Proposi-tion 3.11. Let’s check the properness.In the case of m = 1, set r = r , k i = k i , l i = l i for 1 ≤ i ≤ r . GENERALIZATION OF DUFLO’S CONJECTURE 23
As Lemma 3.2, Stab P ( g I o · f ) ∼ = Stab L ( v I o ). For convenience, wedenote G k ( C ) by G k for any k ∈ Z + . We know that L ∼ = ( G l × G l × · · · G l r ) ⋉ N ( r − X i =1 k i l i · (2( l i +1 + · · · + l r )) + r X i =1 l i ( k i − , where N ( a ) means the unipotent group which is homeomorphic to C a as manifold.AndStab L ( v I o ) ∼ =( G l × G l × · · · G l r − ) ⋉ N ( r − X i =1 k i l i · (2( l i +1 + · · · + l r ))+ r X i =1 l i ( k i −
1) + ( l r − − ( r − X i =1 k i l i + ( k r − l r )) . Similarly, we haveStab P (p( g I o · f )) ∼ =( G l × G l × · · · G l r − ) ⋉ N ( r − X i =1 k i l i · (2( l i +1 + · · · + l r − r − X i =1 l i ( k i −
1) + ( l r − ( k r −
1) + ( r − X i =1 k i l i + ( l r − k r )) . By direct calculation, we have that the quotientStab P (p( g I o · f )) / Stab P ( g I o · f )is a single element. So the restriction of p on P · ( g I o · f ) is proper.In the case of m >
1, one can get the statement immediately. (cid:3)
Moment map in the GL n ( R ) case. Now, we can calculate themoment map in the G n ( R ) case and get the similar results as the caseof G n ( C ).Let n ∈ Z + , let G = G n ( R ) and let P = P n ( R ) be the mirabolicsubgroup of G . g (resp. p ) denotes the Lie algebra of G (resp. of P ).Write J s ( a ) = a a . . . . . .1 a ∈ g s ( R ) , s ∈ Z and s ≥ , a ∈ R , and J ( a ) = a × , a ∈ R . Set J s = J s (0), ∀ s ∈ Z + . Write J lk ( a ) = diag( J k ( a ) , · · · , J k ( a ) | {z } l blocks ) , ∀ l ∈ Z + . Write J k ( a, b ) = (cid:18) J k ( a ) bI k − bI k J k ( a ) (cid:19) , ∀ a, b ∈ R , k ∈ Z + , and J lk ( a, b ) = diag( J k ( a, b ) , · · · , J k ( a, b ) | {z } l blocks ) , Let u, v ∈ N such that 2 u + v ≤ n . Let a , · · · , a u + v , b , · · · , b u ∈ R such that z k − = a k + ib k , z k = a k − ib k , for k = 1 , · · · , u , and z u + k = a u + k for 1 ≤ k ≤ v , are 2 u + v distinct complex number, Q ≤ i ≤ u b i = 0.Let ξ = diag( ξ , · · · , ξ u + v ) , with ξ j = diag( J l j k j ( a j , b j ) , J l j k j ( a j , b j ) , · · · , J l jrj k jrj ( a j , b j )) , ≤ j ≤ u, diag( J l j k j ( a j ) , J l j k j ( a j ) , · · · , J l jrj k jrj ( a j )) , u + 1 ≤ j ≤ u + v, where r j ∈ Z + , ≤ j ≤ u + v , l ji , k ji ∈ Z + , ∀ ≤ j ≤ u + v, ≤ i ≤ r j ,and k ji < k ji for 1 ≤ j ≤ u + v , 1 ≤ i < i ≤ r i , and X ≤ j ≤ u, ≤ i ≤ r j k ji l ji + X u +1 ≤ j ≤ u + v, ≤ i ≤ r j k ji l ji = n. Set f = pr( ξ ). It is clear that every orbit in g ∗ is of the form O f .Now, we fix a ξ . Set n j = (cid:26) P r j i =1 k ji · l ji , ≤ j ≤ u, P r j i =1 k ji · l ji , u + 1 ≤ j ≤ u + v, and n = 0. For convenience, set k j = 0 , l j = 0 for 1 ≤ j ≤ u + v .Set L = Stab G ( f ), then P \ O f ∼ = P \ G/L ∼ = ( R n − { } ) /L. By calculating the representatives of ( R n − { } ) /L , one can get therepresentatives of the P -orbits in O f .Since ξ is block diagonal, L = diag( X , · · · , X u + v ) | X i ∈ Stab GL nj ( R ) ( ξ j ) } . For u + 1 ≤ j ≤ u + v , one can obtain L j = Stab GL nj ( R ) ( ξ j ) from theresult in the case of G n ( C ) by changing the field C to R (see Proposition3.8).For 1 ≤ j ≤ u , let’s calculate L j . Let M s × s ( R ) be the set of all s × s matrices over R . Write K ( x , · · · , x s ) = x x x ... . . . . . . x s · · · x x , x i ∈ R , ≤ i ≤ s, then { X ∈ M s × s ( R ) | XJ s = J s X } = { K ( x , · · · , x s ) | x i ∈ R , ≤ i ≤ s } . GENERALIZATION OF DUFLO’S CONJECTURE 25
For any ~x = ( x , · · · , x s ) , ~y = ( y , · · · , y s ) ∈ R s , define H ( ~x, ~y ) asthe 2 s × s matrix (cid:18) K ( x , · · · , x s ) K ( y , · · · , y s ) − K ( y , · · · , y s ) K ( x , · · · , x s ) (cid:19) . We have the following proposition.
Proposition 3.13.
For any a, b ∈ R , b = 0 , k, l ∈ Z + , H k,l = Stab GL k · l ( R ) ( J lk ( a, b ))= { H ( ~x , ~y ) · · · H ( ~x l , ~y l ) ... . . . ... H ( ~x l , ~y l ) · · · H ( ~x ll , ~y ll ) | ~x ij , ~y ij ∈ R k , ∀ ≤ i, j ≤ l, det (cid:18) A B − B A (cid:19) = 0 , A = ( ~x ij (1)) ≤ i,j ≤ l , B = ( ~y ij (1)) ≤ i,j ≤ l } . Proof.
It can be proved similarly as Proposition 3.6. (cid:3)
For any k , k ∈ Z + such that k ≥ k , and ~x = ( x , · · · , x k ), ~y = ( y , · · · , y k ) ∈ R k , define H ( ~x, ~y ) as the 2 k × k matrix (cid:18) K ( x , · · · , x k ) 0 k × ( k − k ) K ( y , · · · , y k ) 0 k × ( k − k ) − K ( y , · · · , y k ) 0 k × ( k − k ) K ( x , · · · , x k ) 0 k × ( k − k ) (cid:19) , and define H ( ~x, ~y ) as the 2 k × k matrix ( k − k ) × k ( k − k ) × k K ( x , · · · , x k ) K ( y , · · · , y k )0 ( k − k ) × k ( k − k ) × k − K ( y , · · · , y k ) K ( x , · · · , x k ) . Proposition 3.14.
For any a, b ∈ R , b = 0 , k , k , l , l ∈ Z + , k < k , M l ,l k ,k = { X ∈ M k l × k l ( R ) | X ( J l k ( a, b )) = ( J l k ( a, b )) X } = { H ( ~x , ~y ) · · · H ( ~x l , ~y l ) ... . . . ... H ( ~x l , ~y l ) · · · H ( ~x l l , ~y l l ) | ~x ij , ~y ij ∈ R k , ∀ ≤ i ≤ l , ≤ j ≤ l } ,N l ,l k ,k = { X ∈ M k l × k l ( R ) | X ( J l k ( a, b )) = ( J l k ( a, b )) X } = { H ( ~x , ~y ) · · · H ( ~x l , ~y l ) ... . . . ... H ( ~x l , ~y l ) · · · H ( ~x l l , ~y l l ) | ~x ij , ~y ij ∈ R k , ∀ ≤ i ≤ l , ≤ j ≤ l } . Proof.
It is easy to check. (cid:3)
For convenience, we fix a 1 ≤ j ≤ u and set r = r j , k i = k ji , l i = l ji ,for 1 ≤ i ≤ r j in the Proposition 3.15. Proposition 3.15.
The stabilizer L j = Stab GL nj ( R ) ( ξ j ) = { X X · · · X r X X · · · X r ... ... . . . ... X r X r · · · X rr | X ss ∈ H k s ,l s , ∀ ≤ s ≤ r,X st ∈ M l s ,l t k s ,k t , ∀ ≤ s < t ≤ r, X st ∈ N l s ,l t k s ,k t , ∀ ≤ t < s ≤ r } . Proof.
It can be checked directly as Proposition 3.8. (cid:3)
Now we go to calculate ( R n − { } ) /L . As L is block diagonal, weonly need to calculate ( R n j − { } ) /L j . For u + 1 ≤ j ≤ u + v , we canget the result from the case of G n ( C ). For 1 ≤ j ≤ u , we have thefollowing result.We fix a j and adopt the same notation as Proposition 3.15. Proposition 3.16.
Write R = { , · · · , r } . For ∅ 6 = R ⊂ R , let ≤ x i ≤ k i , for i ∈ R , such that (1) x i < x i ′ for any i, i ′ ∈ R such that i < i ′ , (2) k i − x i < k i ′ − x i ′ for any i, i ′ ∈ R such that i < i ′ ,we define I = { i − X t =0 k t l t + k i (2 l i −
1) + x i , i ∈ R } ⊆ { , , · · · , n j } , and v I = ( y , y , · · · , y n j ) , y i = 1 when i ∈ I and y i = 0 when i / ∈ I .Let I a be the set of all I ’s constructed as above. Then { v I | I ∈ I a } represent all different L j -orbits in R n j − { } .Proof. It can be proved similarly as Proposition 3.9. (cid:3)
Now we can get the representatives of ( R n − { } ) /L . Proposition 3.17.
Set K = { , · · · , u + v } , R j = { , · · · , r j } , ∀ ≤ j ≤ u + v . Let ∅ 6 = K ⊆ K , and ∅ 6 = R j ⊆ R j for any j ∈ K . Choosean integer ≤ x ji ≤ k ji for j ∈ K, i ∈ R j , such that (1) x ji < x ji , for any i , i ∈ R j such that i < i , (2) k ji − x ji < k ji − x ji , for any i , i ∈ R j such that i < i .Set I = G j ∈ K, ≤ j ≤ u { j − X l =0 n l + i − X p =0 k jp l jp + k ji (2 l ji −
1) + x ji , i ∈ R j } GG j ∈ K,u +1 ≤ j ≤ u + v { j − X l =0 n l + i − X p =0 k jp l jp + k ji ( l ji −
1) + x ji , i ∈ R j } , and v I = ( v , · · · , v n ) with v i = 1 when i ∈ I and v i = 0 when i / ∈ I .Let I a be the set of all I ’s constructed above, then { v I | I ∈ I a } formall different representatives of ( R n − { } ) /L . GENERALIZATION OF DUFLO’S CONJECTURE 27
Proof.
It can be proved similarly as Proposition 3.10. (cid:3)
We can define g I in the same way as the contents about regular orbitsand such g I · f ’s form all different representatives of P \ O f and theorbit of g I o · f , I o = { n , n + n , · · · , n } , is the unique dense open orbitamong them.We go to calculate the moment map p : O f → p ∗ . Let’s fix some I corresponding to the given K , R j ( j ∈ K ) and 1 ≤ x ji ≤ r j ( i ∈ R j , j ∈ K ). For any 1 ≤ j ≤ u + v , let q j = R j , and R j = { c j , · · · , c jq j } , c j < c j < · · · < c jq j . Set x jh = x jc jh , k jh = k jc jh , l jh = l jc jh for1 ≤ h ≤ q j , set d j = x jq j and x j = 0. Proposition 3.18.
We have that p( g I · f ) has depth d = X j ∈ M, ≤ j ≤ u d j + X j ∈ M,u +1 ≤ j ≤ u + v d j , and is P -conjugated to the element pr ′ ( η ) , where η = diag( A , · · · , A m , J d ) , where A j = ξ j if j / ∈ M , and A j = diag( A j , A j , · · · , A jr j ) , if j ∈ M ,where A ji ’s are defined as follows, when i / ∈ R j , A ji = ( J l ji k ji ( a j , b j ) , ≤ j ≤ u,J l ji k ji ( a j ) , u + 1 ≤ j ≤ u + v, when i = c jh ∈ R j for some ≤ h ≤ q j , A ji = ( diag( J l ji − k ji ( a j , b j ) , J t ( a j , b j )) , ≤ j ≤ u, diag( J l ji − k ji ( a j ) , J t ( a j )) , u + 1 ≤ j ≤ u + v, here t = k ji − x jh + x j ( h − .Proof. We regard p n ( R ) as the real form of p n ( C ). Use the result of G n ( C ), we get that p( g I · f ) is P n ( C )-conjugated to pr ′ ( η ). So p( g I · f )is also P n ( R )-conjugated to pr ′ ( η ). (cid:3) Theorem 3.19. p : O f → p ∗ sends different P -orbits of O f to different P -orbits of p ∗ , the image p ( O f ) contains a unique dense orbit P · ( g I o · f ) ,and the restriction of p on P · ( g I o · f ) is proper. The reduce space ofthe unique open dense orbit is singleton.Proof. It follows from Proposition 3.18 and the similar result of the G n ( C ) case (see Theorem 3.12). (cid:3) Results of Kirillov’s conjecture
Let k be the field R or C , let P n ( k ) be the mirabolic subgroupof GL n ( k ). The Kirillov’s conjecture states that the irreducible uni-tary representations of GL n ( k ) remains irreducible upon restrictionto P n ( k ) and was proved by Sahi ([19]) for tempered representations of GL n ( R ) or GL n ( C ), and Sahi-Stein ([21]) for Speh representationsof GL n ( R ), and Baruch ([4]) in archimedean fields general. LaterAizenbud-Gourevitch-Sahi ([21]) calculated the adduced representa-tions of the Speh complementary series.Sahi’s and Sahi-Stein’s proofs are based on Mackey’s theory of theunitary representations of semi-direct products and Vogan’s result aboutthe classification of the irreducible unitary representations of GL n ( k ),and their key method is constructing the intertwining operator byFourier transform. Baruch proved by studying the P n ( k )-invariant dis-tribution. Aizenbud-Gourevitch-Sahi’s used annihilator variety and de-generate Whittaker models to get the adduced representations of theSpeh complementary series. Here we summarize some results of Kir-illov’s conjecture.Firstly, we show how to get the unitary dual of P n ( k ), denoted by \ P n ( k ). The following two facts are easy to see.(i) P n ( k ) ∼ = GL n − ( k ) ⋉ k n − .(ii) GL n − ( k ) has two orbits in ( k n − ) ∗ : { } and ( k n − ) ∗ − { } . Let ξ ∈ ( k n − ) ∗ − { } be defined by ξ ( x , · · · , x n − ) = x n − . ThenStab GL n − ( k ) ( ξ ) ∼ = P n − ( k ).Based on Mackey’s theory, we have the followings.Every irreducible unitary representation of P n ( k ) is obtained in oneof the following two ways:(i) by trivially extending an irreducible unitary representation ofGL n − ( k ),(ii) by extending an irreducible unitary representation of P n − ( k )to P n − ( k ) ⋉ k n − by the character ξ and then unitarily inducingto P n ( k ).We use E and I to denote for functors from the above constructions(i) and (ii) respectively. Then, \ P n ( k ) = E ( \ GL n − ( k )) G I ( \ P n − ( k )) . Moreover, we have the following fact: each irreducible unitary repre-sentation τ of P n ( k ) is of the form τ = I j − Eσ , where the integer j ≥ σ ∈ \ GL n − ( k ) are uniquely determined by τ .Now we describe the unitary dual of GL n ( k ), which is finally ob-tained by Vogan [23]. Let π and π be the representations of GL n ( k )and GL n ( k ), π × π denotes the unitary parabolic induction fromthe representation π ⊗ π of the Levi subgroup GL n ( k ) × GL n ( k ) toGL n + n ( k ). Then every irreducible unitary representation of GL n ( R )is a × -product of unitary characters, Stein representations, Speh repre-sentations, and Speh complementary series representations. And everyirreducible unitary representation of GL n ( C ) is a × -product of unitarycharacters and Stein representations. GENERALIZATION OF DUFLO’S CONJECTURE 29
For convenience, we give the descriptions of Speh representations,Stein representations and Speh complementary series representations.They can be described as the (subrepresentations of) degenerated prin-ciple series.(i) Speh representations. Let Q be the subgroup of GL n ( R ) de-fined as Q = { (cid:18) A B D (cid:19) | A, D ∈ GL n ( R ) , B ∈ M n × n ( R ) } . For any m ∈ Z + , we define a character χ m : Q → C × as (cid:18) A B D (cid:19)
7→ | det( A ) | m/ sgn(det( A )) m +1 | det( D ) | − m/ . The Speh representation of GL n ( R ), denoted by δ ( n, m ), isthe unique nonzero irreducible subrepresentation of Ind GQ ( χ m ),where the smooth vector in Ind GQ ( χ m ) is { f ∈ C ∞ ( G ) | f ( qg ) = λ ( q ) χ m ( q ) f ( g ) , ∀ q ∈ Q, g ∈ G } , here λ is the modular character defined as (cid:18) A B D (cid:19)
7→ | det( A ) | n/ | det( D ) | − n/ . Simply speaking, the Speh representation δ ( n, m ) is the uniquenonzero irreducible subrepresentation of( | det | m/ · sgn(det) m +1 ) | GL n ( R ) × | det | − m/ | GL n ( R ) . (ii) Stein representation. For any s ∈ (0 , / n ( R ), denoted by σ ( n, s ) (or σ ( n, s ) | GL n ( R ) when there is ambiguity), as the representation | det | s | GL n ( R ) × | det | − s | GL n ( R ) . The Stein representation of GL n ( C ), denoted by σ ( n, s )(or σ ( n, s ) | GL n ( C ) when there is ambiguity), can be defined thesame as that of GL n ( R ), | det | s | GL n ( C ) × | det | − s | GL n ( C ) . (iii) Speh complementary series representation. For any m ∈ Z + ,and s ∈ (0 , / n ( R ), denoted by ∆( n, m, s ), as therepresentation | det | s δ ( n, m ) × | det | − s δ ( n, m ) . We now state some results on Kirillov’s conjecture. Firstly, S.Sahi[19] established Kirillov’s conjecture of GL n ( k ) for the × -product ofunitary characters and Stein representations.Sahi [19] defined that (i) a unitary representation of P n ( k ), denoted by τ , is homogeneousof depth j if τ = I j − Eσ for some unitary representation σ ofGL n − ( k ),(ii) a unitary representation of GL n ( k ), denoted by ρ , is adducibleof depth j if ρ | P n ( k ) is homogeneous of depth j , and if ρ | P n ( k ) = I j − Eσ , we shall write σ = Aρ and call it the adduced repre-sentation of ρ .Then Sahi obtained the following key fact based on Mackey’s theoryand the partial Fourier transform, and got the result in that paper. Theorem 4.1 ([19], Theorem 2.1) . If ρ and σ are adducible represen-tation of GL r ( k ) and GL s ( k ) of depths l and m , then ρ × σ is adducibleof depth l + m . Moreover, A ( ρ × σ ) = ( Aρ ) × ( Aσ ) . We also have the following result.
Theorem 4.2 ([19], Lemma 3.1) . Let π be a unitary character of GL n ( k ) , then π is adducible of depth and Aπ = π | GL n − ( k ) , where GL n − ( k ) is imbedded on the top left corner of GL n ( k ) . Latter, Sahi [20], Sahi-Stein [21] obtained the adduced represen-tations of Stein representations and Speh representations. The keymethod is constructing the intertwining operator by partial Fouriertransform.
Theorem 4.3 ([20], 2.4) . Let σ ( n + 1 , s ) be the Stein representation of GL n +2 ( R ) , s ∈ (0 , / , then Aσ ( n + 1 , s ) = σ ( n, s ) . Theorem 4.4 ([21], Theorem 3) . Let δ ( n + 1 , m ) be the m -th Spehrepresentation of GL n +2 ( R ) , m ∈ Z + , then Aδ ( n + 1 , m ) = δ ( n, m ) . After Baruch [4] proved the Kirillov’s conjecture in archimedeanfields, Aizenbud-Gourevitch-Sahi [21] calculated the adduced represen-tations of the Speh complementary series.
Theorem 4.5 ([2], Theorem 4.2.4) . Let ∆( n + 1 , m, s ) be the Spehcomplementary series representation of GL n +4 ( R ) , m ∈ Z + and s ∈ (0 , / , then A ∆( n + 1 , m, s ) = ∆( n, m, s ) . Example 4.1.
In the case of GL n ( C ) , let a i ∈ R , s i ∈ Z + , ≤ i ≤ m , P mi =1 s i = n , let π = (det) ia | GL s ( C ) × · · · × (det) ia m | GL sm ( C ) be theirreducible unitary representation of GL n ( C ) , then π | P n = I m − E ((det) ia | GL s − ( C ) × · · · × (det) ia m | GL sm − ( C ) ) . Orbit method
The orbit method tries to establish a close connection existed be-tween irreducible unitary representations of a Lie group and its orbitsin the coadjoint representation, and to provide a clear geometric pic-ture of irreducible unitary representations. In 1950’s , before the orbit
GENERALIZATION OF DUFLO’S CONJECTURE 31 method developed, Mackey and others had obtained some wonderfulresults on the irreducible unitary representations of Lie groups, show-ing how to use induced representation to obtain irreducible unitaryrepresentations, see Mackey [18]. In 1960’s, Kirillov found out a wayto think about the set of all irreducible unitary representations of asimply connected nilpotent Lie group, establishing a bijection betweenthe coadjoint orbits and the irreducible unitary representations (seeKirillov [12]), and brought up the orbit method. Later, Kostant de-veloped a quantization theory to obtain irreducible unitary representa-tions, and generalized the results of Kirillov to solvable Lie groups (seeAuslander-Kostant [3]). In 1970’s-1980’s, Duflo got all the irreducibleunitary representations of (almost) algebraic groups assuming the uni-tary dual of reductive Lie groups, see Duflo [6]. The orbit method(or the quantization problem) of reductive groups is not completelysolved until now. Roughly speaking, the quantization of coadjoint or-bits can be reduced to the case of nilpotent coadjoint orbits (definedbelow), and such irreducible unitary representations attached to nilpo-tent coadjoint orbits, called unipotent representations, are not easy tounderstood in general, see Vogan [22]. We will summarize some resultsabout the orbit method of reductive Lie groups and algebraic groupsin the following two parts. The method of constructing representa-tions is using induced representation (includes parabolic induction andcohomological induction).5.1.
Representations attached to coadjoint orbits of GL n ( k ) . Jordan decomposition.
To begin with, we use Jordan decompo-sition to describe the coadjoint orbits of reductive groups (see Vogan[22], Lecture 2).Let G be a reductive Lie group and let g be its Lie algebra. Let θ be a Cartan involution of g and let G = K exp s be the Cartan decomposition, where K = G θ and s = − θ on g .Let B be a nondegenerate G -invariant and θ -invariant bilinear formon g such that the quadratic form g ∋ X
7→ − B ( X, θX )on g is positive definite. Let g ∗ be the dual of g . Then we have a G -isomorphism φ : g → g ∗ , X ( Y B ( X, Y ) , ∀ Y ∈ g ) , ∀ X ∈ g . For any λ ∈ g ∗ , we call it semisimple (resp. nilpotent, hyperbolic,elliptic) if φ − ( λ ) is a semisimple (resp. nilpotent, hyperbolic, elliptic) element in g . By the Jordan decomposition, we can decompose any λ ∈ g ∗ uniquely as λ = λ h + λ e + λ n , where λ h is hyperbolic, λ e is elliptic, λ n is nilpotent and φ − ( λ h ), φ − ( λ e ), φ − ( λ n ) commute with each other.Set X h = φ − ( λ h ), X e = φ − ( λ e ) and X n = φ − ( λ n ). Let G ( X )(resp. G ( λ )) be the stabilizer of X (resp. of λ ), and g ( X ) (resp. g ( λ )) be the Lie algebra of G ( X ) (resp. of G ( λ )). Then we have thefollowings. Proposition 5.1 (Vogan [23], Proposition 2.10, 2.11 and 2.12) . (i) Any hyperbolic element in g is conjugated into s . If X ∈ s ishyperbolic, then G ( X ) is reductive group with Cartan involution θ | G ( X ) . (ii) There is a bijection between the G -orbits of elements in g , whichhyperbolic part are G -conjugated to X h , and the G ( X h ) -orbitsof elements in g ( X h ) , which hyperbolic part is zero. (iii) Any elliptic element in g is conjugated into k . If X ∈ k iselliptic, then G ( X ) is a reductive group with Cartan involution θ | G ( X ) . (iv) There is a bijection between the G -orbits of elements in g , whichsemisimple part are G -conjugated to X h + X e , and the G ( X h + X e ) -orbits of elements in g ( X h + X e ) , which semisimple part iszero. Therefore we can assume X h ∈ s and X e ∈ k .An irreducible unitary representation which is attached to a coad-joint orbit has a kind of “Jordan decomposition”. To attach an irre-ducible unitary representation to the orbit of λ , we firstly attach anirreducible representation π n of G ( λ h + λ e ) to the G ( λ h + λ e )-orbit of λ n | g ( λ h + λ e ) ∗ , which is called as nilpotent step; then using λ e and ap-plying cohomological induction to π n , we get an irreducible unitaryrepresentation π h of G ( λ h ), which is called as elliptic step; finally, us-ing λ h and applying parabolic induction to π h , we get an irreducibleunitary representation of G , which is called as hyperbolic step.5.1.2. Hyperbolic step.
The easiest step is hyperbolic step, which usesparabolic induction, see Vogan [22], Lecture 2.Set L = G ( λ h ) with its Lie algebra l . Set g r = { Y ∈ g | [ X h , Y ] = rY } , r ∈ R and u = ⊕ r> g r . Set U = exp u and Q = LU . Define unitary character χ ( λ h ) of L as follows, χ ( λ h )( k · exp( Z )) = exp( iλ h ( Z )) , k ∈ L ∩ K, Z ∈ l ∩ s . Suppose that π L is any unitary representation of G ( λ h ) attached to the G ( λ h )-coadjoint orbit of ( λ e + λ n ) | l , then the unitary representation of GENERALIZATION OF DUFLO’S CONJECTURE 33 G (this unitary representation may be reducible) π G = Ind GQ ( π L ⊗ χ ( λ h ))is attached to the coadjoint orbit of λ .5.1.3. Elliptic step.
The elliptic step is more complicated, which usescohomological induction, see Vogan [24]. Set λ s = λ h + λ e be semisimplepart of λ , and set X s = X h + X e .Let g C be the complexification of g , and let q be the parabolic subal-gebra having g C ( λ s ) as a Levi factor with nilpotent radical u such thatthe eigenvalues of X s acting on u are in { z ∈ C | Re( z ) > , or Re( z ) = 0 , Im( z ) > } . Let e ρ ( u ) be the character of the adjoint action of G ( λ s ) on the topexterior power of u .Assume that τ is any irreducible representation of G ( λ s ) such that dτ = iλ s + ρ ( u ) . Then there is attached to λ a unitary representation π λ such that theunderlying ( g C , K )-module Π( λ ) is defined as follows,(1) Π( λ ) = (Γ g C ,K g C ,G ( λ s ) ∩ K ) d (pro g C ,G ( λ s ) ∩ K q ,G ( λ s ) ∩ K ( τ ⊗ π n )) , where Γ is the Zuckerman functor, pro is a kind of Hom functor, d is the dimension of u ∩ k , π n is the unipotent representation of G ( λ s )which is attached to λ n | ( g ( λ s )) ∗ .To define the Zuckerman functor (see Knapp-Vogan [13], Introduc-tion), we need to define the Hecke algebra. Assume that h is a complexreductive Lie algebra of Lie group H , M is a compact subgroup of H ,then the Hecke algebra R ( h , M ) can be defined as the algebra of left M -finite distributions on H with support in M , with convolution asmultiplication.Let’s fix some notations to define the Zuckerman functor and thefunctor pro. Let G be a linear connected reductive Lie group and G C be the complexification of G . Let K be the maximal compact subgroupof G . Let g denote the Lie algebra of G and g C be the complexificationof g .Assume that T is a torus in G , and L = Z G ( T ) with Lie algebra l .Let L C be the analytic subgroup of G C with Lie subalgebra l C . Let Q be a parabolic subgroup in G C containing L C as Levi subgroup and let q be the Lie algebra of Q .Assume that V is a ( g C , L ∩ K )-module, W is an ( l C , L ∩ K )-module,hence is also a ( q , L ∩ K )-module by trivial extension to the radical u of q .Define the functor pro from ( q , L ∩ K )-module to ( g C , L ∩ K )-moduleas pro g C ,L ∩ K q ,L ∩ K ( W ) = Hom q ( U ( g C ) , W ) L ∩ K . Here the subscript L ∩ K means taking all the L ∩ K finite vectors.Define the functor from ( g C , L ∩ K )-module to ( g C , K )-module asΓ g C ,K g C ,L ∩ K ( V ) = Hom R ( g C ,L ∩ K ) ( R ( g C , K ) , V ) K . It is left exact and has i -th right derivative functors, denoted by(Γ g C ,K g C ,L ∩ K ) i .Let ( u R g C ,K q ,L ∩ K ) i ( W ) denote(Γ g C ,K g C ,K ∩ L ) i (pro g C ,L ∩ K q ,L ∩ K ( W )) , then the equation (1) can be written as( u R g C ,K q ,G ( λ s ) ∩ K ) d ( τ ⊗ π n ) . Nilpotent step.
We focus on the special unipotent representa-tions which are defined by Arthur, Barbasch and Vogan (see Adams-Barbasch-Vogan [1], chapter 27).Let’s introduce notation about partition firstly. Let n ∈ Z + , let r ∈ Z + , k i , l i ∈ Z + such that P ≤ i ≤ r k i l i = n and k i > k j for any1 ≤ i < j ≤ r . We use { k l · · · k l r r } to denote the partition of n , { k , · · · , k | {z } l , · · · , k r , · · · , k r | {z } l r } . Let P denote this partition, we will use J P to denote diag( J l k , · · · , J l r k r ),use J P ( a ) to denote diag( J l k ( a ) , · · · , J l r k r ( a )), use J P ( a, b ) to denotediag( J l k ( a, b ) , · · · , J l r k r ( a, b )).For the case of G = G n ( R ), let f = pr( ξ ) with ξ = J P . The specialunipotent representation attached to O f is π = sgn(det) w | G t ( R ) × · · · × sgn(det) w p | G tp ( R ) , where { t , · · · , t p } is the dual partition of { k l · · · k l r r } and w i = 0 , ≤ i ≤ p .For the case of G = G n ( C ), let f = pr( ξ ) with ξ = J P . The specialunipotent representation attached to O f is π = 1 | G t ( C ) × · · · × | G tp ( C ) where { t , · · · , t p } is the dual partition of { k l · · · k l r r } .In general, we have the following correspondence between coadjointorbits and irreducible unitary representations.For the case of G = G n ( R ).When n = 2 m , assume that P = { k l · · · k l r r } is a partition of m . Let f ∈ g ∗ and f = pr( ξ ) with ξ = J P (0 , b/ b ∈ Z + .Firstly, we use the nilpotent step and attach the orbit f n = pr( ξ n ), ξ n = J P (0 , π n of G ( f s ) ∼ = G m ( C ) 1 | G t ( C ) × · · · × | G tp ( C ) , GENERALIZATION OF DUFLO’S CONJECTURE 35 where { t , · · · , t p } is the dual partition of { k l · · · k l r r } .By direct calculation, we have τ = ( det | det | ) a + m | G m ( C ) and e ρ ( u ) = ( det | det | ) m | G m ( C ) . Using Vogan [23], Theorem 17.6, we have( u R g C ,K q ,G ( f s ) ∩ K ) d ( τ ⊗ π n ) = p × i =1 δ ( t i , b ) . So the irreducible unitary representation p × i =1 δ ( t i , b ) is attached to thecoadjoint orbit of f = pr( ξ ) with ξ = J P (0 , b/ f = pr( ξ ) with ξ = diag( J P (0 , b / , · · · , J P r (0 , b r / , J P ) , where r ∈ Z + , b i ∈ Z + , for 1 ≤ i ≤ r , b > b > · · · > b r , P i is apartition of n i and P is a partition of n with P ri =1 n i + n = n .Let { t i , · · · , t is i } be the dual partition of P i for 1 ≤ i ≤ r . Let { t , · · · , t s } be the dual partition of P .Let H = G n ( R ) × · · · × G n p ( R ) × G n ( R ) , then G ( f s ) ⊂ H . Let h be the Lie algebra of H and h C be the complex-ification of h . Let q = h C ∩ q , and let u be the nilpotent radical of q . Let Q be the parabolic subgroup containing H and the invertibleupper triangular matrices. Set d = dim( u ∩ k ).We use Vogan [23], Theorem 17.6, and get that( u R g C ,K q ,G ( f s ) ∩ K ) d ( τ ⊗ π n ) = Ind GQ (( u R h C ,K ∩ H q ,G ( f s ) ∩ K ) d ( τ ⊗ π n ))= ( p × j =1 s j × i =1 δ ( t ji , b j )) × ( s × j =1 sgn(det) w j | G tj ( R ) ) , where w j = 0 , ≤ j ≤ s .5.1.5. Representations of GL n ( k ) attached to some kind of coadjointorbits. In general, for the case of G n ( R ), let f = pr( ξ ) with ξ =diag( ξ , · · · , ξ m ), m ∈ Z + , and ξ i = diag( J P i ( a i , b i / , · · · , J P ipi ( a i , b ip i / , J P i ( a i )) , ≤ i ≤ m, where p i ∈ Z + , a i ∈ R for 1 ≤ i ≤ m , a > · · · > a m , b ij ∈ Z + , and b i > · · · > b ip i , P ij is a partition of n ij for 1 ≤ i ≤ m and 1 ≤ j ≤ p i , P i is a partition of n i for 1 ≤ i ≤ m , n ij , n i satisfy X ≤ i ≤ m X ≤ j ≤ p i n ij + X ≤ i ≤ m n i = n. Assume that { t ij, , · · · , t ij,s ij } is the dual partition of P ij , and { t i, , · · · , t i,s i } is the dual partition of P i . The representation attached to O f is π = π × · · · × π m , where π l = | det | ia l ⊗ (( p l × j =1 s lj × k =1 δ ( t lj,k , b lj )) × ( s l × k =1 sgn(det) w l,k | G tl,k ( R ) )) , with w l,k = 0 , ≤ l ≤ m , 1 ≤ k ≤ s l .For the case of G = G n ( C ), let f = pr( ξ ) with ξ = diag( J P ( a ) , · · · , J P m ( a m )) , where a i ∈ R for any 1 ≤ i ≤ m , and a > · · · > a m , P i is a partitionof n i for 1 ≤ i ≤ m , and n i (1 ≤ i ≤ m ) satisfy P ≤ i ≤ m n i = n .The representation attached to O f is π = m × j =1 s j × k =1 | det | ia j | G pjk ( C ) , where { p j , · · · , p js j } is the dual partition of P j for 1 ≤ j ≤ m .5.2. Representations attached to coadjoint orbits of P n ( k ) . Givenan (almost) algebraic group G , Duflo [6] constructed all the irreducibleunitary representations of G assuming the unitary dual of some re-ductive groups, basing on the Mackey’s theory. It suggests that theproblem of attaching irreducible unitary representations with coadjointorbits can be reduced to the problem in the case of reductive groups.We summarize some results of Duflo [6] without proofs in the follow-ings.5.2.1. Coadjoint orbits of algebraic groups.
Duflo showed how to get allthe coadjoint orbits of g ∗ assuming that one knew the coadjoint orbitsof reductive groups.Let g be the Lie algebra of G , and let g ∗ be the dual of g . For any x ∈ g ∗ , let G ( x ) be the stabilizer of x in G and let g ( x ) be the Liealgebra of G ( x ), let u G ( x ) be the unipotent radical of G ( x ) and let u g ( x ) be the Lie algebra of u G ( x ).On g / g ( x ), there is a natural symplectic form, denoted by ω x , ω x ( α, β ) = x ([ α, β ]) , where α, β ∈ g and α, β denote the image of α and β in g / g ( x ).Firstly, we need to define the linear forms of unipotent type (see[6], I). If G is unipotent, all of g ∗ are of unipotent type, and if G isreductive, 0 is the unique element of g ∗ of unipotent type. In general,its definition depends on the following things. Definition 5.1.
Let x ∈ g ∗ and b be a subalgebra of g . We say that (i) b is coisotropic relative to x (or coisotropic) if the orthogonalof b in g with respect to ω x , denoted by b ⊥ , is contained in b . (ii) b is a polarization of x if b ⊥ = b . (iii) An coisotropic subalgebra b is of strongly unipotent type if b isalgebraic (i.e. there exists algebraic subgroup with Lie subalge-bra b ) and b = g ( x )+ u b . GENERALIZATION OF DUFLO’S CONJECTURE 37
Definition 5.2.
An element x in g ∗ is said to be of unipotent type ifit satisfies (i) There exists a reductive factor of g ( x ) contained in ker x . (ii) There is a subalgebra of strongly unipotent type relative to x . Let x ∈ g ∗ be a form of unipotent type, then we can construct asubalgebra of strongly unipotent type relative to x by induction ondim g canonically (see [6], I.20). Let u = x | u g and let h = g ( u ). If h = g , then set b = h . If h = g , then y = x | h is a form of unipotenttype. Let l ⊂ h be the canonical subalgebra of strongly unipotent typerelative to y , set b = u g + h . Then b is the canonical subalgebra ofstrongly unipotent type relative to x .Let D be the set of pairs ( x, λ ), where x is a linear form of unipotenttype on g , and λ is an element in L ( x ), L ( x ) is the set of linear formsover g ( x ) whose restriction to u g ( x ) is equal to x | u g ( x ) .Let r be a reductive factor of g ( x ), then the restriction map establisha bijection of L ( x ) and r ∗ , sending y ∈ L ( x ) to y | r .Let ( x, λ ) ∈ D . Let b be a subalgebra of g , which is of unipotenttype relative to x (see [6],I.8 for definition, from [6], I.9, I.22, we havethat when x is of unipotent type, a subalgebra is of unipotent typerelative to x if and only if it is strongly unipotent type relative to x ).Let f be an element of g ∗ whose restriction to u b is equal to x | u b , andwhose restriction restriction to g ( x ) is equal to λ . Such an element f exists by the definition of L ( x ).Let G acts on D naturally, then we have the followings. Proposition 5.2 ([6], II.5) . (i) The orbit G · f in g ∗ does not de-pend on the choices of b and f . We denote it by O x,λ . (ii) The map ( x, λ ) O x,λ induces a bijection from D/G to g ∗ /G . Construction of unitary dual of algebraic groups.
Duflo showedhow to construct the irreducible unitary representations of G using thecoadjoint orbits and the irreducible unitary representations of somereductive groups.The construction of representations used Metaplectic groups.Since G ( x ) acts on g / g ( x ) and keeps the form ω x , we get a morphism φ : G ( x ) → Sp( g / g ( x ) , ω x ) . Let Mp( g / g ( x ) , ω x ) be the metaplectic group, i.e., there is a nontrivialconnected double cover ϕ : Mp( g / g ( x ) , ω x ) → Sp( g / g ( x ) , ω x ) . We define a double cover of G ( x ) as G ( x ) g = { ( g, t ) | g ∈ G ( x ) , t ∈ Mp( g / g ( x ) , ω x ) and φ ( g ) = ϕ ( t ) } . Let ψ : G ( x ) g → G ( x ) be the covering morphism, and let (1 , −
1) denotethe nontrivial element of ker ψ . Similarly, for any group H , which acts on a symplectic vector space m and keep the symplectic form, we have a morphism φ ′ : H → Sp( m )and a double covering morphism ϕ ′ : Mp( m ) → Sp( m ). Define a doublecover e H = { ( h, t ) | h ∈ H, t ∈ Mp( m ) and φ ′ ( h ) = ϕ ′ ( t ) } . By Segal-Shale-Weil representation (see Duflo [6], II.6), any ˆ s ∈ Mp( m ) which is mapped to s ∈ Sp( m ) can be represented by ( s, θ ˆ s )(or just denoted by ( s, θ )), where θ ˆ s is a function depended on ˆ s , withcomplex value module 1 over the set of maximal isotropic subspacesin m . Therefore, any element ( h, t ) ∈ e H ( h ∈ H, t ∈ Mp( m )) can berepresented by ( h, θ t ). We will also use H m to denote e H .Since u G ( x ) is unipotent, u G ( x ) can be embedded into G ( x ) g . Let Y irr ( x ) = { irreducible unitary representation π of G ( x ) g / u G ( x ) | π ((1 , − − } . Let C denote the set of couples ( x, τ ), where x is a linear form ofunipotent type, and τ ∈ Y irr ( x ). Then the group G acts on C naturally.Now, we can construct an irreducible unitary representation T x,τ of G (see [6], III for details).Let b be a subalgebra of strongly unipotent type relative to x whichis stable under G ( x ), and let v denote the nilpotent radical of b . Let V be the subgroup corresponding to v . Set B = G ( x ) V . Let v denotethe restriction of x to v , and G ( x ) v is defined.Let l ⊂ v be a polarization relative to v , and let L be the unipotentsubgroup with Lie algebra l . Define the irreducible unitary representa-tion of V attached to v as T v = Ind VL e iv | l . Define T x,τ = Ind GB ( τ ′ ⊗ S v T v ) , where τ ′ and S v are defined as follows, and we have that T x,τ is inde-pendent of the choice of B (see [6], III.16).Define τ ′ to be the representation of G ( x ) v as τ ′ ( y, ψ ) = ( ϕψ − ) τ ( y, ϕ ) , here τ = τ ⊗ χ x of ( G ( x ) g / u G ( x )) ⋊ u G ( x ) ∼ = G ( x ) g with χ x defined byd χ x = ix | u g ( x ) , and ϕψ − = ϕ ( l ′ + g ⊥ ) ψ ( l ′ ) − is a constant which is independent on the choice of the maximal totallyisotropic subspace l ′ of v (see [6], II.8).Define S v to be the action of G ( x ) v on T v by S v ( y, ϕ ) = ϕ ( l ) · S ′ v, l ( y ) , GENERALIZATION OF DUFLO’S CONJECTURE 39 and S ′ v, l is the action of G ( x ) on T v defined by S ′ v, l ( y ) = || A ( y ) || − F l ,y l A ( y ) ,F l ,y l is the intertwining operator from T yv to T v , and A ( y ) is the operatorfrom T v to T yv with A ( y ) α ( z ) = α ( y − ( z )) , α ∈ T v , z ∈ V. It can be checked that S v is independent of the choice of l (see [6],II.10). Proposition 5.3 ([6], III.12) . For any element ( x, τ ) ∈ C , we getan irreducible unitary representation T x,τ of G . Moreover, the map ( x, τ ) T x,τ induces a bijection from C/G to b G . Now, we can obtain the correspondence between coadjoint orbitsand irreducible unitary representations of an algebraic group G (see[6], III.19, III.20).Let f ∈ g ∗ , by Proposition 5.2, we get O f = O x,λ for some x ∈ g ∗ of unipotent type and λ ∈ L ( x ) ∼ = ( g ( x ) / u g ( x )) ∗ . Assume that τ is anirreducible unitary representation of G ( x ) g / u G ( x ) attached to λ suchthat τ ((1 , − −
1, then T x,τ is an irreducible unitary representationof G attached to O f .5.2.3. Duflo’s construction in the P n ( k ) case. Set G = P n ( k ), we showhow to attach representations with coadjoint orbits by the above method(using the same notation).For any element f ∈ g ∗ , applying the results in section 2, we havethat f is conjugated to an element pr ′ (diag( A, J m )), for some m ∈ Z + and A ∈ gl n − m ( k ).Let x = pr ′ (diag(0 , J m )), then x is of unipotent type. Actually, g ( x )has reductive factor g ′ = { diag( X, m × m ) | X ∈ gl n − m ( k ) } ⊂ ker( x ) . Let b = { (cid:18) X Y Z (cid:19) | X ∈ gl n − m ( k ) , Y ∈ M ( n − m ) × m ( k ) , Z ∈ u m ( k ) } , where u m ( k ) is the set of strictly upper triangular matrices over k , then b is a subalgebra of strongly unipotent type relative to x . Therefore, x is of unipotent type.Let λ ′ = pr ′ (diag( A, λ := λ ′ | g ( x ) ∈ L ( x ), since x | u g ( x ) = 0and λ | u g ( x ) = 0. Moreover, L ( x ) ∼ = ( g ( x ) / u g ( x )) ∗ ∼ = gl n − m ( k ) ∗ . Also it is easy to see that O f = O x,λ . Furthermore, we have that G ( x ) g is a trivial double cover of G ( x ),that is G ( x ) g ∼ = G ( x ) × Z . Actually, G ( x ) = { Y Z
00 1 00 0 I m − | Y ∈ GL n − m ( k ) , Z ∈ M ( n − m ) × ( k ) } . It is not hard to see that the image of the reductive factor { diag( Y, I m ) | Y ∈ GL n − m ( k ) } in Sp( g / g ( x ) , ω x ) ∼ = Sp(( n − m m − , k )has trivial double cover in the metaplectic group Mp(( n − m )( m − , k ).Therefore, G ( x ) g is a trivial double cover of G ( x ).To attach O f with representation of P n ( k ), we assume that τ is theirreducible unitary representation of G ( x ) / u G ( x ) ∼ = GL n − m ( k )which is attached to the GL n − m ( k )-orbit of λ ∈ L ( x ) ∼ = gl n − m ( k ) ∗ .Define an irreducible unitary representation of G ( x ) / u G ( x ) × Z , de-noted by τ , such that τ | G ( x ) / u G ( x ) = τ and τ ((1 , − −
1. Thenthe irreducible representation of the irreducible unitary representationattached to O f is T x,τ = Ind GB ( τ ′ ⊗ S v T v ) . The subalgebra of unipotent type relative to x constructed above, b ,is the canonical one. And the corresponding subgroup B is B = { (cid:18) Y Z U (cid:19) | Y ∈ GL n − m ( k ) , Z ∈ M ( n − m ) × m ( k ) , U ∈ U m ( k ) } . Here U m ( k ) is the set of upper triangular unipotent matrices over k .Moreover, v = u b itself is the polarization of v = x | v , and we get a onedimensional unitary representation of V , T v = e ix | v .Since ω x is trivial on b , we get G ( x ) v ∼ = G ( x ) × Z and G ( x ) b ∼ = G ( x ) × Z , and therefore S v | G ( x ) is the one dimensional trivial representation,so we have τ ′ ⊗ S v T v = τ ⊗ e ix . Therefore, the irreducible unitary representation of G attached to O f with f = pr ′ (diag( A, J m )) is T x,τ = Ind GB ( τ ⊗ e ix ) = I m − E τ , where τ is an irreducible unitary representation of GL n − m ( k ) attachedto the GL n − m ( k )-orbit of pr n − m ( A ). GENERALIZATION OF DUFLO’S CONJECTURE 41 A generalization of Duflo’s conjecture
We establish a generalization of Duflo’s conjecture for the restric-tion of an irreducible unitary representation π of G = GL n ( k ) to themirabolic subgroup P = P n ( k ), k = R or C , where π is attached to a G -coadjoint orbit of f ∈ g ∗ in section 5. Set O π = O f . Let p : O π → p ∗ be the moment map. Then we have the following theorem. Theorem 6.1.
There are only finitely many P -orbits in p( O π ) , includ-ing a unique open P -orbit Ω in p( O π ) . Moreover, (1) the moment map p : O π → p ∗ is proper over Ω , (2) the restriction of π to P , π | P is irreducible, and is attached to Ω in the sense of Duflo, (3) the reduced space of Ω (with respect to the moment map p ) is asingle point.Proof. By Theorem 3.12, Theorem 3.19, we get (1),(3) immediately.To get (2), we firstly get π | P from section 4 and get Ω = p( g I o · f )from the results of moment maps in section 3. It remains to verify if π P is attached to Ω by Duflo’s construction in section 5. We verify oneexample and one can verify the other cases easily.Let 1 ≤ t p ≤ · · · ≤ t ∈ Z + such that P pi =1 t i = n . Let π be theunipotent representation1 | G t ( C ) × · · · × | G tp ( C ) of G n ( C ), which is attached to the coadjoint orbit of f = pr( ξ ) with ξ = diag( J l k , · · · , J l r k r ) , where k i , l i ∈ Z + for 1 ≤ i ≤ r , and { k , · · · , k | {z } l , · · · , k r , · · · , k r | {z } l r } is thedual partition of { t , · · · , t p } (so k ≥ · · · ≥ k r ).Then π | P = I p E (1 | G t − ( C ) × · · · × | G tp − ( C ) ) . To get the unique dense open P -orbit in p( O f ), we set ˜ f = pr( ˜ ζ ) with˜ ζ = diag( J l r k r , · · · , J l k ). Then ˜ f is G n ( C )-conjugated to f , so O f = O ˜ f .By Theorem 3.12, we have Ω = P · p( g I o · ˜ f ) is the unique dense open P -orbit in p( O f ). Since I o = { n } and g I o = I n , we have g I o · ˜ f = ˜ f . Itis easy to see that p( ˜ f ) is P -conjugated to the element pr ′ ( ζ ) with ζ = diag( J l − k , J l k , · · · , J l r k r , J k ) , So Ω = P · pr ′ ( ζ ).Based on Duflo’s construction, we get that the irreducible unitaryrepresentation of P attached to Ω is I k E (1 | G s ( C ) × · · · × | G sq ( C ) ) , where { s , · · · , s q } is the dual partition of { k , · · · , k | {z } l − , k , · · · , k | {z } l , · · · , k r , · · · , k r | {z } l r } .By definition of dual partition, we have p = q , s i = t i − ≤ i ≤ p ,and k = p . Hence we get that π | P is attached to Ω. (cid:3) Acknowledgements
The author would like to thank Jun Yu for suggesting this problemand many discussions, thank Daniel Kayue Wong for discussions onthe orbit method of reductive groups and thank Huajian Xue for manydiscussions, etc.
References [1] J.Adams, D.Barbasch, D.Vogan:
The Langlands Classification and Irre-ducible Characters for Real Reductive Groups , Progress in Mathematics, .Birkh¨auser Boston, Inc., Boston, MA, 1992.[2] A.Aizenbud, D.Gourevitch, S.Sahi:
Derivatives for smooth representations of
GL( n, R ) and GL( n, C ), Israel J. Math. (2015), no. 1, 1-38.[3] L.Auslander, B.Kostant: Quantization and representations of solvable Liegroups , Bull. Amer. Math. Soc. (1967), 692-695.[4] E.M.Baruch: A proof of Kirillov’s conjecture , Ann. of Math. (2) (2003),no. 1, 207-252.[5] J.Bernstein: P -invariant distributions on GL( N ) and the classification of uni-tary representations of GL( N ) (non-Archimedean case) , Lie group represen-tations, II (College Park, Md., 1982/1983), 50-102, Lecture Notes in Math., , Springer, Berlin, 1984.[6] M.Duflo: Th´eorie de Mackey pour les groupes de Lie alg´ebriques , (French)[Mackey theory for algebraic Lie groups] Acta Math. (1982), no. 3-4, 153-213.[7] M.Duflo:
On the Plancherel formula for almost algebraic real Lie groups , Liegroup representations, III (College Park, Md., 1982/1983) 101-165, LectureNotes in Math., , Springer, Berlin, 1984.[8] M.Duflo, M.Vergne:
Kirillov’s formula and Guillemin-Sternberg conjecture , C.R. Math. Acad. Sci. Paris (2011), no. 23-24, 1213-1217.[9] H.Fujiwara:
Sur les restrictions des repr´esentations unitaires des groupes deLie r´esolubles exponentiels , (French) [On restrictions of unitary representationsof exponential solvable Lie groups] Invent. Math. (1991), no.3, 647-654.[10] V.Guillemin, S.Sternberg:
Geometric quantization and multiplicities of grouprepresentations , Invent. Math. (1982), no.3, 515-538.[11] G.Heckman: Projections of orbits and asymptotic behavior of multiplicities forcompact connected Lie groups , Invent. Math. (1982) no.2, 333-356.[12] A.Kirillov: Lectures on the Orbit Method , Graduate Studies in Mathematics, . American Mathematical Society, Providence, RI, 2004.[13] A.W.Knapp, D.Vogan: Cohomological induction and unitary representations ,Princeton Mathematical Series, . Princeton University Press, Princeton, NJ,1995.[14] T. Kobayashi: Restrictions of unitary representations of real reductive groups ,Lie theory, 139-207, Progr. Math., , Birkh¨auser Boston, Boston, MA, 2005.[15] G.Liu:
Duflo’s conjecture for the branching problem to the Iwasawa AN -subgroup of a semisimple Lie group , Math. Ann. (2015), no. 1-2, 107-120. GENERALIZATION OF DUFLO’S CONJECTURE 43 [16] G.Liu, Y.Oshima, J.Yu:
Restriction of irreducible unitary representations of
Spin( N, to parabolic subgroups , arXiv: 2010.01026.[17] G.Liu, J.Yu: A geometric interpretation of Kirillov’s conjecture , arXiv:1806.06318.[18] G.W.Mackey:
Induced representations of locally compact groups I , Ann. ofMath. (2) (1952), 101-139.[19] S.Sahi: On Kirillov’s conjecture for Archimedean fields , Compositio Math. (1989), no.1, 67-86.[20] S.Sahi: A simple construction of Stein’s complementary series representations ,Proc. Amer. Math. Soc. (1990), no.1, 257-266.[21] S.Sahi, E.M.Stein:
Analysis in matrix space and Speh’s representations , Invent.Math. (1990), no.2, 379-393.[22] D.Vogan:
The method of coadjoint orbits for real reductive groups , Represen-tation theory of Lie groups (Park City, UT, 1998), 179-238, IAS/Park CityMath. Ser., , Amer. Math. Soc., Providence, RI, 2000.[23] D.Vogan: The unitary dual of
GL( n ) over an Archimedean field , Invent.Math. (1986), no. 3, 449-505.[24] D.Vogan: Unitary representations of reductive Lie groups and the orbit method ,Based on notes prepared by Jorge Vargas. Progr. Math., , New develop-ments in Lie theory and their applications (C´ordoba, 1989), 87-114, Birkh¨auserBoston, Boston, MA, 1992.[25] D.Vogan:
Unitary Representations of Reductive Lie Groups , Annals of Math-ematics Studies, . Princeton University Press, Princeton, NJ, 1987.(Hongfeng Zhang)
BICMR, Peking University, No. 5 Yiheyuan Road,Haidian District, Beijing 100871, China.
Email address ::