aa r X i v : . [ m a t h . N T ] J un A PROOF OF THE HOWE DUALITY CONJECTURE
WEE TECK GAN AND SHUICHIRO TAKEDA to Professor Roger Howewho started it allon the occasion of his 70th birthday
Abstract.
We give a proof of the Howe duality conjecture in the theory of local thetacorrespondence for symplectic-orthogonal or unitary dual pairs in arbitrary residual charac-teristic. Introduction
Let F be a nonarchimedean local field of characteristic not 2 and residue characteristic p .Let E be F itself or a quadratic field extension of F . For ǫ = ± , we consider a − ǫ -Hermitianspace W over E of dimension n and an ǫ -Hermitian space V of dimension m .Let G ( W ) and H ( V ) denote the isometry group of W and V respectively. Then the group G ( W ) × H ( V ) forms a dual reductive pair and possesses a Weil representation ω ψ whichdepends on a nontrivial additive character ψ of F (and some other auxiliary data which weshall suppress for now). To be precise, when E = F and one of the spaces, say V , is odddimensional, one needs to consider the metaplectic double cover of G ( W ); we shall simplydenote this double cover by G ( W ) as well. The various cases are tabulated in [GI, § ω ψ into irreducible representations of G ( W ) × H ( V ). More precisely, for any irreducibleadmissible representation π of G ( W ), one may consider the maximal π -isotypic quotient of ω ψ . This has the form π ⊗ Θ W,V,ψ ( π ) for some smooth representation Θ W,V,ψ ( π ) of H ( V );we shall frequently suppress ( W, V, ψ ) from the notation if there is no cause for confusion.It was shown by Kudla [K1] that Θ ( π ) has finite length (possibly zero), so we may considerits maximal semisimple quotient θ ( π ). One has the following fundamental conjecture due toHowe [H1, H2]: Howe Duality Conjecture for G ( W ) × H ( V )(i) θ ( π ) is either 0 or irreducible.(ii) If θ ( π ) = θ ( π ′ ) = 0, then π = π ′ . Date : August 14, 2018.2000
Mathematics Subject Classification.
Primary 11F27, Secondary 22E50.
A concise reformulation is: for any irreducible π and π ′ ,(HD) dim Hom H ( V ) ( θ ( π ) , θ ( π ′ )) ≤ δ π,π ′ := ( , if π ∼ = π ′ ;0 , if π ≇ π ′ .We take note of the following theorem: Theorem 1.1. (i) If π is supercuspidal, then Θ ( π ) is either zero or irreducible (and thus isequal to θ ( π ) ). Moreover, for any irreducible supercuspidal π and π ′ , Θ ( π ) ∼ = Θ ( π ′ ) = 0 = ⇒ π ∼ = π ′ . (ii) θ ( π ) is multiplicity-free.(iii) If p = 2 , the Howe duality conjecture holds. The statement (i) is a classic theorem of Kudla [K1] (see also [MVW]), whereas (iii) is awell-known result of Waldspurger [W]. The statement (ii), on the other hand, is a result of Li-Sun-Tian [LST]. We note that the techniques for proving the three statements in the theoremare quite disjoint from each other. For example, the proof of (i) is based on arguments usingthe doubling see-saw and Jacquet modules of the Weil representation: these have becomestandard tools in the study of the local theta correspondence. The proof of (iii) is based on K -type analysis and uses various lattice models of the Weil representation. Finally, the proofof (ii) is based on an argument using the Gelfand-Kazhdan criterion for the (non-)existenceof equivariant distributions.In this paper, we shall not assume any of the statements in Theorem 1.1. Indeed, thepurpose of this paper is to give a simple proof of the Howe duality conjecture, following astrategy initiated by Howe in his Corvallis article [H1, §
11] and using essentially the sametools in the proof of Theorem 1.1(i) [K1], as developed further in [MVW]. Thus, our maintheorem is:
Theorem 1.2.
The Howe duality conjecture (HD) holds for the pair G ( W ) × H ( V ) . Let us make a few remarks:(1) From our brief sketch of the history of the Howe duality conjecture above, the reader candiscern two distinct lines of attack on the Howe duality conjecture, the genesis of whichcan both be found in [H1]. The first is a study of K-types, which was used to establishthe conjecture in the archimedean cases and adapted to the p-adic case (with p = 2)by Howe and Waldspurger. The second is a doubling see-saw argument outlined in [H1, § From this doubling construction, we see that the space ( Y ⊗ Y ∨ ) defined above isdescribed as a G ′ -module by Theorem 9.2. By restriction we can investigate its structureas a ˜ G ′ × ˜ G ′ -module. Doing so we find that the duality conjecture is certainly true if G or G ′ is compact, and in general is “almost” true. It remains in the general case to removethe “almost”. ”It has taken almost 40 years to remove the “almost”. HE HOWE DUALITY CONJECTURE 3 (2) The above setup makes sense even when E = F × F is a split quadratic algebra, in whichcase the groups G ( W ) and H ( V ) are general linear groups. In that case, the Howe dualityconjecture has been shown by Minguez in [M]. As we shall see, the proof of Theorem 1.2is essentially analogous to the one given by Minguez.(3) In an earlier paper [GT], we had extended Theorem 1.1(i) (with Θ ( π ) replaced by θ ( π ))from supercuspidal to tempered representations. Using this, we had shown the Howeduality conjecture for almost equal rank dual pairs. The argument in Section 4 of thispaper (doubling see-saw) is the same as that in [GT, § §
3] and uses a key technique of Minguez[M].(4) We remark that in the papers [M1, M2, M3, M4], Mui´c has conducted detailed studies ofthe local theta correspondence for symplectic-orthogonal dual pairs. In [M1], for example,he explicitly determined the theta lift of discrete series representations π in terms of theMoeglin-Tadi´c classification and observed as a consequence the irreducibility of θ ( π ). TheMoeglin-Tadi´c classification was conditional at that point, and we are not sure where itstands today. In [M3, M4], Mui´c proved various general properties of the theta lifting oftempered representations (such as the issue of whether Θ ( π ) = θ ( π )), and obtained veryexplicit information about the theta lifting under the assumption of the Howe dualityconjecture. The main tools he used are Jacquet modules analysis and Kudla’s filtration.Since the Howe duality conjecture is a simple statement without reference to classification,it seems desirable to have a classification-free proof. Indeed, our result renders mostresults in [M3, M4] unconditional.As is well-known, there is another family of dual pairs associated to quaternionic Hermitianand skew-Hermitian spaces. (See [W] or [K2] for more details.) Our proof, unfortunately,does not apply to these quaternionic dual pairs, because we have made use of the MVW-involution π π MV W on the category of smooth representations of G ( W ) and H ( V ). Forthe same reason, the result of [LST] in Theorem 1.1(ii) is not known for these quaternionicdual pairs. Unlike the contragredient functor, which is contravariant in nature, the MVW-involution is covariant and has the property that π MV W = π ∨ if π is irreducible. It wasshown in [LnST] that such an involution does not exist for quaternionic unitary groups.Nonetheless, even in the quaternionic case, our proof gives a partial result which is oftensufficient for global applications. Namely, if π is an irreducible Hermitian representation (i.e. π = π ∨ ) and we let θ her ( π ) ⊂ θ ( π ) denote the submodule generated by irreducible Hermitiansummands, then the results of Theorem 1.2 hold for Hermitian π ’s and with θ ( π ) replacedby θ her ( π ). Namely we have: Theorem 1.3.
Consider a quaternionic dual pair G ( W ) × H ( V ) and let π and π ′ be irre-ducible Hermitian representations of G ( W ) . Let θ her ( π ) ⊂ θ ( π ) be the submodule generatedby irreducible Hermitian summands. Then we have dim Hom H ( V ) ( θ her ( π ) , θ her ( π ′ )) ≤ δ π,π ′ . In particular, if π and π ′ are unitary, we have dim Hom H ( V ) ( θ unit ( π ) , θ unit ( π ′ )) ≤ δ π,π ′ , WEE TECK GAN AND SHUICHIRO TAKEDA where θ unit ( π ) ⊂ θ her ( π ) consists of irreducible unitary summands of θ ( π ) . We give a proof of this theorem in the last section of the paper.
Acknowledgements
This project was begun during the authors’ participation in the Oberwolfach workshop “Mod-ular Forms” in April 2014 and completed while both authors were participating in the work-shop “The Gan-Gross-Prasad conjecture” at Jussieu in June-July 2014. We thank the EcoleNormale Superieur and the Institut des Hautes ´Etudes Scientifiques for hosting our respectivevisits. We also thank Goran Mui´c and Marcela Hanzer for several useful conversations andemail exchanges about [M1, M2, M3, M4] and the geometric lemma respectively. We areextremely grateful to Alberto Minguez for explaining to us the key idea in his paper [M] fortaking care of the representations on the boundary. Finally, we thank an anonymous refereewho pointed out several embarrassing errors in a first version of this paper.The first author is partially supported by an MOE Tier 1 Grant R-146-000-155-112 and anMOE Tier Two grant R-146-000-175-112, whereas the second author is partially supportedby NSF grant DMS-1215419.2.
Basic Notations and Conventions
Fields.
Throughout the paper, F denotes a nonarchimedean local field of characteristicdifferent from 2 and residue characteristic p . Once and for all, we fix a non-trivial additivecharacter ψ on F . Let E be F itself or a quadratic field extension of F . For ǫ = ±
1, we set ǫ = ( ǫ if E = F ;0 if E = F .2.2. Spaces.
Let W = W n = a − ǫ -Hermitian space over E of dimension n over EV = V m = an ǫ -Hermitian space over E of dimension m over E. We also set: s m,n = m − ( n + ǫ )2 . Groups.
We will consider the isometry groups associated to the pair (
V, W ) of ± ǫ -Hermitian spaces. More precisely, we set: G ( W ) = ( the metaplectic group Mp( W ), if W is symplectic and dim V is odd;the isometry group of W , otherwise.We define H ( V ) similarly by switching the roles of W and V . Occasionally we write G n := G ( W n ) H m := H ( V m ) . For the general linear group, we shall write GL n for the group GL n ( E ). Also for a vectorspace X over E , we write det GL( X ) or sometimes simply det X for the determinant on GL( X ). HE HOWE DUALITY CONJECTURE 5
Representations.
For a p -adic group G , let Rep( G ) denote the category of smoothrepresentations of G and denote by Irr( G ) the set of equivalence classes of irreducible smoothrepresentations of G .For a parabolic P = M N of G , we have the normalized induction functorInd GP : Rep( M ) −→ Rep( G ) . On the other hand, we have the normalized Jacquet functor R P : Rep( G ) −→ Rep( M ) . If P = M N denotes the opposite parabolic subgroup to P , we likewise have the functor R P .We shall frequently exploit the following two Frobenius’ reciprocity formulas:Hom G ( π, Ind GP σ ) ∼ = Hom M ( R P ( π ) , σ ) (standard Frobenius reciprocity)and Hom G (Ind GP σ, π ) ∼ = Hom M ( σ, R P ( π )) (Bernstein’s Frobenius reciprocity) . Moreover, Bernstein’s Frobenius reciprocity is equivalent to the statement: R P ( π ∨ ) ∨ ∼ = R P ( π )for any smooth representation π with contragredient π ∨ , where P denotes the opposite par-abolic subgroup to P .2.5. Parabolic induction.
When G is a classical group, we shall use Tadi´c’s notation forinduced representations. Namely, for general linear groups, we set ρ × · · · × ρ a := Ind GL n ··· + na Q ρ ⊗ · · · ⊗ ρ a where Q is the standard parabolic subgroup with Levi subgroup GL n × · · · × GL n a . For aclassical group such as G ( W ), its parabolic subgroups are given as the stabilizers of flags ofisotropic spaces. If X t is a t -dimensional isotropic space of W = W n and we decompose W = X t ⊕ W n − t ⊕ X ∗ t , the corresponding maximal parabolic subgroup Q ( X t ) = L ( X t ) · U ( X t ) has Levi factor L ( X t ) = GL( X t ) × G ( W n − t ). If ρ is a representation of GL( X t ) and σ is a representa-tion of G ( W n − t ), we write ρ ⋊ σ = Ind G ( W ) Q ( X t ) ρ ⊗ σ. More generally, a standard parabolic subgroup Q of G ( W ) has the Levi factor of the formGL n × · · · × GL n a × G ( W n ′ ) and we set ρ × · · · × ρ a ⋊ σ := Ind G ( W ) Q ρ ⊗ · · · ⊗ ρ a ⊗ σ, where ρ i is a representation of GL n i and σ is a representation of G ( W n ′ ). When G ( W ) =Mp( W ) is a metaplectic group, we will follow the convention of [GS, § H ( V m ). For example, a maximal parabolic subgroup of H ( V m ) has the form P ( Y t ) = M ( Y t ) · N ( Y t ) and is the stabilizer of a t -dimensional isotropic subspace Y t of V m . WEE TECK GAN AND SHUICHIRO TAKEDA
To distinguish between representations of G ( W ) and H ( V ), we will normally use lowercase Greek letters such as π, σ etc to denote representations of G ( W ), and upper case Greekletters such as Π, Σ etc to denote representations of H ( V ).2.6. Conjugation Action.
Let X be a vector space over E of dimension n and let c be thegenerator of Gal( E/F ). For each representation ρ of GL( X ), we define the c -conjugate c ρ of ρ as follows. First, fix a basis of X , which gives an isomorphism α : GL( X ) ∼ = GL n ( E ).The natural action of c on GL n ( E ) induces an action on GL( X ), which we write as c · g := α − ◦ c ◦ α ( g ) for g ∈ GL( X ).Then we define c ρ by c ρ ( g ) := ρ ( c · g )for g ∈ GL( X ). Of course, a different choice of the basis gives a different α and thus a differentautomorphism g c · g of GL( X ). But these different automorphisms differ from each otherby inner automorphisms of GL( X ), and hence the isomorphism class of c ρ is independent ofthe choice of the basis. Of course if E = F , then c ρ = ρ .2.7. MVW.
In [MVW, p. 91], Moeglin, Vigneras and Waldspurger introduced a functor
M V W : Rep( G ( W )) −→ Rep( G ( W ))which is an involution and satisfies π MV W = π ∨ if π is irreducible.Unlike the contragredient functor, this MVW involution is covariant. It will be useful toobserve that(2.1) ( ρ ⋊ σ ) MV W = c ρ ⋊ σ MV W . Weil representations.
To consider the Weil representation of the pair G ( W ) × H ( V ),we need to specify extra data to give a splitting G ( W ) × H ( V ) → Mp( W ⊗ V ) of the dualpair. Such splittings were constructed and parametrized by Kudla [K2] and we shall use hisconvention here, as described in [GI, § χ = ( χ V , χ W ), which are certain unitary characters of E × .Pulling back the Weil representation of Mp( W ⊗ V ) to G ( W ) × H ( V ) via the splitting, weobtain the associated Weil representation ω W,V, χ ,ψ of G ( W ) × H ( V ). Note that the character χ V satisfies(2.2) c χ − V = χ V and likewise for χ W . We shall frequently suppress χ and ψ from the notation, and simplywrite ω W,V for the Weil representation.3.
Some Lemmas
In this section, we record a couple of standard lemmas which will be used in the proof ofthe main theorem. We also introduce the important notion of “occurring on the boundary”.
HE HOWE DUALITY CONJECTURE 7
Kudla’s filtration.
The first lemma describes the computation of the Jacquet modulesof the Weil representation. This is a well-known result of Kudla [K1]; see also [MVW].
Lemma 3.1.
The Jacquet module R Q ( X a ) ( ω W n ,V m ) has an equivariant filtration R Q ( X a ) ( ω W n ,V m ) = R ⊃ R ⊃ · · · ⊃ R a ⊃ R a +1 = 0 whose successive quotients J k = R k /R k +1 are described in [GI, Lemma C.2] . More precisely, J k = Ind GL( X a ) × G ( W n − a ) × H ( V m ) Q ( X a − k ,X a ) × G ( W n − a ) × P ( Y k ) (cid:16) χ V | det X a − k | λ a − k ⊗ C ∞ c (Isom E,c ( X k , Y k )) ⊗ ω W n − a ,V m − k (cid:17) , where · λ a − k = s m,n + a − k ; · V m = Y k + V m − k + Y ∗ k with Y k a k -dimensional isotropic space; · X a = X a − k + X ′ k and Q ( X a − k , X a ) is the maximal parabolic subgroup of GL( X a ) stabilizing X a − k ; · Isom
E,c ( X k , Y k ) is the set of E -conjugate-linear isomorphisms from X k to Y k ; · GL( X k ) × GL( Y k ) acts on C ∞ c (Isom E,c ( X k , Y k )) as (( b, c ) · f )( g ) = χ V (det b ) χ W (det c ) f ( c − gb ) for ( b, c ) ∈ GL( X k ) × GL( Y k ) , f ∈ C ∞ c (Isom E,c ( X k , Y k )) and g ∈ GL k . · J k = 0 for k > min { a, q } , where q is the Witt index of V m .In particular, the bottom piece of the filtration (if nonzero) is: J a ∼ = Ind GL( X a ) × G ( W n − a ) × H ( V m )GL( X a ) × G ( W n − a ) × P ( Y a ) (cid:0) C ∞ c (Isom E,c ( X a , Y a )) ⊗ ω W n − a ,V m − a (cid:1) . A degenerate principal series.
Another key ingredient used later is the decompo-sition of a certain degenerate principal series representation. Consider the “doubled” space W + W − . This “doubled” space contains the diagonally embedded ∆W ⊂ W + W − as a maximal isotropic subspace whose stabilizer Q ( ∆W ) is a maximal parabolic subgroup of G ( W + W − ) and has Levi factor GL( ∆W ). We may consider the degenerate principal seriesrepresentation I ( s ) = Ind G ( W + W − ) Q ( ∆W ) χ V | det | s of G ( W + W − ) induced from the character χ V | det | s of Q ( ∆W ) (normalized induction).The following lemma (see [KR]) describes the restriction of I ( s ) to the subgroup G ( W ) × G ( W ) ⊂ G ( W + W − ). Lemma 3.2.
As a representation of G ( W ) × G ( W − ) , I ( s ) possesses an equivariant filtration ⊂ I ( s ) ⊂ I ( s ) ⊂ · · · ⊂ I q W ( s ) = Ind G ( W + W − ) Q ( ∆W ) χ V · | det | s WEE TECK GAN AND SHUICHIRO TAKEDA whose successive quotients are given by R t ( s ) = I t ( s ) /I t − ( s )= Ind G ( W ) × G ( W − ) Q t × Q t (cid:16) χ V | det X t | s + t ⊠ χ V | det X t | s + t (cid:17) (3.1) ⊗ (cid:16) ( χ V ◦ det W − n − t ) ⊗ C ∞ c ( G ( W n − t )) (cid:17) ! . Here, the induction is normalized and · q W is the Witt index of W ; · Q t is the maximal parabolic subgroup of G ( W ) stabilizing a t -dimensional isotropicsubspace X t of W , with Levi subgroup GL( X t ) × G ( W n − t ) , where dim W n − t = n − t . · G ( W n − t ) × G ( W n − t ) acts on C ∞ c ( G ( W n − t )) by left-right translation.In particular, R = R ( s ) = ( χ V ◦ det W − ) ⊗ C ∞ c ( G ( W )) is independent of s . Boundary.
The above lemma suggests a notion which plays an important role in thispaper. For π ∈ Irr( G ( W )), we shall say that π occurs on the boundary of I ( s ) if there exists0 < t ≤ q W such that Hom G ( W ) ( R t ( s ) , π ) = 0 . By Bernstein’s Frobenius reciprocity, this is equivalent to(3.2) Hom
GL( X t ) ( χ V | det | s + t , R Q ( X t ) ( π )) = 0 , where Q ( X t ) = L ( X t ) · U ( X t ) stands for the parabolic subgroup opposite to Q ( X t ). Dualizingand using the MVW involution, this is in turn equivalent to(3.3) π ֒ → χ V | det GL( X t ) | − s − t ⋊ σ for some irreducible representation σ of G ( W n − t ). This terminology is due to Kudla andRallis and the reader may consult [KR, Definition 1.3] for the explanation of the use of“boundary”.3.4. Outline of proof.
With the basic notations introduced, we can now give a brief outlineof the proof of Theorem 1.2. First of all, there is no loss of generality in assuming that(3.4) m ≤ n + ǫ , so that(3.5) s m,n = m − ( n + ǫ )2 ≤ , because otherwise one can switch the roles of G ( W ) and H ( V ). We shall assume this hence-forth.The proof proceeds by induction on dim W , with the base case with dim W = 0 beingtrivial. The inductive step is divided into two different parts. HE HOWE DUALITY CONJECTURE 9
The first part, which is given in Section 4, deals with the case when π does not occur onthe boundary of I ( − s m,n ), which cover “almost all” representations. The argument for thiscase has been sketched by Howe [H1, § π occurson the boundary of I ( − s m,n ). To use the induction hypothesis, we use Lemma 3.1 and a keyidea of Minguez [M] in his proof of the Howe duality conjecture for general linear groups.In Section 6, we assemble the results of Section 4 and Section 5 to complete the proof ofTheorem 1.2. 4. A See-Saw Argument
In this section, we will give a proof of (HD) when at least one of π or π ′ ∈ Irr( G ( W ))does not occur on the boundary of I ( − s m,n ) by assuming (3.4) or equivalently (3.5). Asis mentioned at the end of the last section, though we prove (HD) by induction on dim W ,it turns out that for this case, one can prove (HD) without appealing to the inductionhypothesis. Namely, we shall prove Theorem 4.1.
Assume that m ≤ n + ǫ and suppose that π ∈ Irr( G ( W )) does not occur onthe boundary of I ( − s m,n ) . Then for any π ′ ∈ Irr( G ( W )) , dim Hom H ( V ) ( θ ( π ) , θ ( π ′ )) ≤ δ π,π ′ . In particular, θ ( π ) is either zero or irreducible, and moreover for any irreducible π ′ , = θ ( π ) ⊂ θ ( π ′ ) = ⇒ π ∼ = π ′ . Proof.
First, we consider the following see-saw diagram G ( W + W − ) ❘❘❘❘❘❘❘❘❘❘❘❘❘ H ( V ) × H ( V ) G ( W ) × G ( W − ) ❧❧❧❧❧❧❧❧❧❧❧❧❧ H ( V ) ∆ , where W − denotes the space obtained from W by multiplying the form by −
1, so that G ( W − ) = G ( W ). Given irreducible representations π and π ′ of G ( W ), the see-saw identity[GI, § G ( W ) × G ( W ) ( Θ V,W + W − ( χ W ) , π ′ ⊗ π ∨ χ V )(4.1) = Hom H ( V ) ∆ ( Θ ( π ′ ) ⊗ Θ ( π ) MV W , C ) . Here Θ V,W + W − ( χ W ) denotes the big theta lift of the character χ W of H ( V ) to G ( W + W − ).We analyze each side of the see-saw identity in turn. For the RHS, one hasHom H ( V ) ∆ ( Θ ( π ′ ) ⊗ Θ ( π ) MV W , C ) ⊇ Hom H ( V ) ∆ ( θ ( π ′ ) ⊗ θ ( π ) ∨ , C )(4.2) = Hom H ( V ) ( θ ( π ′ ) , θ ( π )) . For the LHS of the see-saw identity (4.1), we need to understand Θ V,W + W − ( χ W ). It isknown that Θ V,W + W − ( χ W ) is irreducible (see [GI, Prop. 7.2]). Moreover, it was shown byRallis that(4.3) Θ V,W + W − ( χ W ) ֒ → I ( s m,n ) = Ind G ( W + W − ) Q ( ∆W ) χ V | det | s m,n . Since s m,n ≤
0, there is a surjective map (see [GI, Prop. 8.2]) I ( − s m,n ) −→ Θ V,W + W − ( χ W ) . Hence the see-saw identity (4.1) gives:Hom G ( W ) × G ( W ) ( I ( − s m,n ) , π ′ ⊗ π ∨ χ V ) ⊇ Hom H ( V ) ( θ ( π ′ ) , θ ( π )) . To prove the theorem, it suffices to prove that the LHS has dimension ≤ π ∼ = π ′ .Applying Lemma 3.2, we claim that if π is not on the boundary of I ( − s m,n ), the naturalrestriction mapHom G ( W ) × G ( W ) ( I ( − s m,n ) , π ′ ⊗ π ∨ χ V ) −→ Hom G ( W ) × G ( W ) ( R , π ′ ⊗ π ∨ χ V )is injective. This will imply the theorem since the RHS has dimension ≤
1, with equality ifand only if π = π ′ .To deduce the claim, it suffices to to show that for each 0 < t ≤ q W ,(4.4) Hom G ( W ) × G ( W ) ( R t ( − s m,n ) , π ′ ⊗ π ∨ χ V ) = 0 . By Bernstein’s Frobenius reciprocity, this Hom space is equal toHom L ( X t ) × L ( X t ) (cid:16)(cid:0) χ V | det | − s m,n + t ⊠ χ V | det | − s m,n + t (cid:1) ⊗ (cid:16) ( χ V ◦ det W − n − t ) ⊗ C ∞ c ( G ( W n − t )) (cid:17) , R Q t ( π ′ ) ⊗ R Q t ( π ∨ χ V ) (cid:17) . Hence we deduce that the equation (4.4) holds ifHom
GL( X t ) ( χ V | det | − s m,n + t , R Q t ( π ∨ χ V )) = 0 . By dualizing and Bernstein’s Frobenius reciprocity, this is equivalent toHom
GL( X t ) ( R Q t ( π · ( χ − V ◦ det W )) , ( χ − V ◦ det GL( X t ) ) · | det | s m,n − t ) = 0 . Now note that χ V ◦ det W | GL( X t ) = χ V ◦ det GL( X t ) . Hence the above condition is equivalent toHom
GL( X t ) ( R Q t ( π ) , χ V | det | s m,n − t ) = 0 . Since this condition holds when π is not on the boundary of ω W,V , the equation (4.4) isproved. This completes the proof of Theorem 4.1. (cid:3)
Finally, let us note that the above argument also gives the following proposition, which wewill use later.
HE HOWE DUALITY CONJECTURE 11
Proposition 4.2.
Assume m ≤ n + ǫ . If π = π ′ ∈ Irr( G ( W )) are such that (4.5) Hom H ( V ) ( θ W,V ( π ) , θ W,V ( π ′ )) = 0 , then there exists t > such that π ֒ → χ V | det GL( X t ) | s m,n − t ⋊ τ and π ′ ֒ → χ V | det GL( X t ) | s m,n − t ⋊ τ ′ for some τ and τ ′ ∈ Irr( G ( W n − t )) .Proof. If π = π ′ but Hom H ( V ) ( θ W,V ( π ) , θ W,V ( π ′ )) = 0, then arguing as above one must haveHom G ( W ) × G ( W ) ( R t ( − s m,n ) , π ⊗ π ′∨ χ V ) = 0 for some t >
0, which gives the conclusion of theproposition. (cid:3)
Of course, the hypothesis of this proposition contradicts the Howe duality (HD), and henceis never satisfied. So what this proposition is saying is that if (HD) is to be violated as in(4.5), it must be violated by representations π and π ′ which occur on the boundary for “thesame t ”. 5. Life on the Boundary
After Theorem 4.1, we see that to prove Theorem 1.2 or equivalently (HD), it remains toconsider the case when both π and π ′ occur on the boundary of I ( − s m,n ), as in Proposition4.2. In this section, we examine what life looks like on the boundary. We continue to assumethat m ≤ n + ǫ , or equivalently that s m,n ≤ An idea of Minguez.
Since π occurs on the boundary of I ( − s m,n ), we have π ֒ → χ V | det GL ( Xt ) | s m,n − t ⋊ π for some π ∈ Irr( G ( W n − t )) and some t >
0. By induction in stages, we have π ֒ → ( χ V | − | s m,n − t + × · · · × χ V | − | s m,n − ) ⋊ π . For simplicity, let us set s m,n,t = s m,n − t + 12 < . Now let us use a key idea of Minguez [M]. Let a > π ֒ → χ V | − | s m,n,t · × a ⋊ σ, where 1 × a = 1 × · · · × | {z } a times = Ind GL a B ⊗ · · · ⊗ σ is an irreducible representation of G ( W n − a ). To simplify notation, let us set(5.1) σ a := χ V | − | s m,n,t · × a ⋊ σ. Note that the representation 1 × a is irreducible and generic by [Z, Theorem 9.7].Observe that if π ֒ → σ a with a > σ * χ V | − | s m,n,t ⋊ σ ′ for any σ ′ . In fact, the converse is also true, as we shall verify in Corollary 5.3(ii) below.The chief reason for considering σ a with a > Proposition 5.1.
Suppose σ a = χ V | − | s m,n,t · × a ⋊ σ is such that σ * χ V | − | s m,n,t ⋊ σ ′ for any σ ′ . Then σ a has a unique irreducible submodule. The proof of Proposition 5.1 relies on the following key technical lemma, which we statein slightly greater generality as it may be useful for other purposes. The proof of the lemmais deferred to the appendix.
Lemma 5.2.
Let ρ be a supercuspidal representation of GL r ( E ) and consider the inducedrepresentation σ ρ,a = ρ × a ⋊ σ of G ( W n ) where σ is an irreducible representation of G ( W n − ra ) . Assume that (a) c ρ ∨ = ρ ; (b) σ * ρ ⋊ σ for any σ .Then we have the following:(i) One has a natural short exact sequence −−−−→ T −−−−→ R Q ( X ra ) σ ρ,a −−−−→ ( c ρ ∨ ) × a ⊗ σ −−−−→ and T does not contain any irreducible subquotient of the form ( c ρ ∨ ) × a ⋊ σ ′ for any σ ′ . In particular, R Q ( X ra ) σ ρ,a contains ( c ρ ∨ ) × a ⊗ σ with multiplicity one, and does notcontain any other subquotient of the form ( c ρ ∨ ) × a ⊗ σ ′ . Likewise, R Q ( X ra ) σ ρ,a contains ρ × a ⊗ σ with multiplicity one and does not contain any other subquotient of the form ρ × a ⊗ σ ′ .(ii) The induced representation σ ρ,a has a unique irreducible submodule.Proof of Proposition 5.1. We shall apply this lemma with r = 1 , ρ = χ V | − | s m,n,t and σ a = χ V | − | s m,n,t · × a ⋊ σ. Here, condition (a) holds since s m,n,t <
0, whereas condition (b) holds by the maximality of a . This proves Proposition 5.1. (cid:3) We shall have another occasion to use Lemma 5.2 later on. We also note the followingcorollary:
Corollary 5.3.
Suppose that π ֒ → σ a and σ * χ V | − | s m,n,t ⋊ σ ′ for any σ ′ .(i) If π ֒ → δ a := χ V | − | s m,n,t · × a ⋊ δ for some δ , then δ ∼ = σ .(ii) Moreover, a is maximal with respect to the property that π ֒ → δ a for some irreducible δ . HE HOWE DUALITY CONJECTURE 13
Proof.
By the exactness of the Jacquet functor, R Q ( X a ) ( π ) is a submodule of R Q ( X a ) ( σ a ). ByLemma 5.2, it follows that R Q ( X a ) ( π ) contains χ V | − | s m,n,t · × a ⊗ σ with multiplicity one,and does not contain any other subquotient of the form χ V | − | s m,n,t · × a ⊗ σ ′ with σ ′ = σ .This key fact will imply both (i) and (ii).(i) If π ֒ → δ a , then R Q ( X a ) ( π ) contains χ V | − | s m,n,t · × a ⊗ δ as a quotient. By the key factobserved above, it follows that δ ∼ = σ .(ii) Suppose for the sake of contradiction that π ֒ → χ V | − | s m,n,t · × ( a +1) ⋊ σ ′ for some irreducible σ ′ . Then by induction in stages, one has π ֒ → χ V | − | s m,n,t · × a ⋊ δ with δ = χ V | − | s m,n,t ⋊ σ ′ . By the Frobenius reciprocity, one has a nonzero equivariant map R Q ( X a ) ( π ) −→ χ V | − | s m,n,t · × a ⊗ δ. By the key fact observed above, the image of this nonzero map must be isomorphic to χ V | − | s m,n,t · × a ⊗ σ . Hence, σ ֒ → δ = χ V | − | s m,n,t ⋊ σ ′ which is a contradiction to the hypothesis of the corollary. (cid:3) A key computation.
We are now ready to launch into a computation needed to com-plete the proof of Theorem 1.2 for the representations on the boundary. The following is thekey proposition:
Proposition 5.4.
Assume = Π ⊂ θ ( π ) and Π is irreducible.(i) If π ֒ → χ V | − | s m,n,t · × a ⋊ σ with a maximal (and for some σ ), then Π ֒ → χ W | − | s m,n,t · × a ⋊ Σ for some Σ and where a is also maximal for Π .(ii) Moreover, whenever Π is presented as a submodule as above, one has = Hom G n × H m ( ω W n ,V m , π ⊗ Π ) ֒ → Hom G n − a × H m − a ( ω W n − a ,V m − a , σ ⊗ Σ ) , so that Σ ⊆ θ W n − a ,V m − a ( σ ) .Proof. (i) Since 0 = Π ⊆ θ ( π ), we have0 = Hom G n × H m ( ω W,V , π ⊗ Π ) ֒ → Hom G n × H m ( ω W,V , σ a ⊗ Π )= Hom GL( X a ) × G n − a × H m ( R Q ( X a ) ( ω W,V ) , χ V | − | s m,n,t × a ⊗ σ ⊗ Π ) , where we used the Frobenius reciprocity for the last step. Now the Jacquet module R Q ( X a ) ( ω W,V )of the Weil representation is computed as in the lemma, which implies that there is a naturalrestriction mapHom
GL( X a ) × G n − a × H m ( R Q ( X a ) ( ω W,V ) , χ V | − | s m,n,t · × a ⊗ σ ⊗ Π ) −→ Hom
GL( X a ) × G n − a × H m ( J a , χ V | − | s m,n,t · × a ⊗ σ ⊗ Π ) . We claim that this map is injective. To see this, it suffices to show that for all 0 ≤ k < a ,Hom GL( X a ) × G n − a × H m ( J k , χ V | − | s m,n,t · × a ⊗ σ ⊗ Π ) = 0 . By the above lemma, this Hom space is equal toHom M ( X a − k ,X a ) × G n − a × H m (cid:16) Ind H ( V m ) P ( Y k ) χ V | det X a − k | λ a − k ⊗ C ∞ c (Isom E,c ( X k , Y k )) ⊗ ω W n − a ,V m − k ,R Q ( X a − k ,X a ) ( χ V | − | s m,n,t · × a ) ⊗ σ ⊗ Π (cid:17) , where M ( X a − k , X a ) is the Levi factor of the parabolic subgroup of GL( X a ) stabilizing X a − k .Because 1 × a is generic, the second representation in this Hom space has a nonzero Whittakerfunctional when viewed as a representation of GL( X a − k ) and hence the first one must alsohave a non-zero Whittaker functional, which is possible only when a − k = 1. Therefore, ifthis Hom space is nonzero, we must have a − k = 1. But in that case, one has: λ = s m,n + 12 > s m,n − t + 12 = s m,n,t , so that the above Hom space is zero even when a − k = 1.Therefore we have J a = 0 and0 = Hom GL( X a ) × G n − a × H m ( J a , χ V | − | s m,n,t · × a ⊗ σ ⊗ Π )= Hom H m ( χ W | − | − s m,n,t · × a ⋊ Θ W n − a ,V m − a ( σ ) , Π ) . Dualizing and applying MVW along with (2.1) and (2.2), this shows that
Π ֒ → χ W | − | s m,n,t · × a ⋊ ( Θ W n − a ,V m − a ( σ ) ∨ ) MV W , and hence(5.2) Π ֒ → χ W | − | s m,n,t · × a ⋊ Σ for some irreducible representation Σ of H ( V m − a ) which is a subquotient of the representa-tion ( Θ W n − a ,V m − a ( σ ) ∨ ) MV W and hence of Θ W n − a ,V m − a ( σ ).To prove (i), it remains to show that in (5.2), the integer a is maximal for Π . Let b ≥ a be maximal such that Π ֒ → χ W | − | s m,n,t · × b ⋊ Σ for some irreducible representation Σ of H ( V m − b ). Then we have0 = Hom G n × H m ( ω W n ,V m , π ⊗ Π )(5.3) ֒ → Hom G n × H m ( ω W n ,V m , π ⊗ ( χ W | − | s m,n,t · × b ⋊ Σ ))= Hom G n × GL( Y b ) × H m − b ( R P ( Y b ) ( ω W n ,V m ) , π ⊗ χ W | − | s m,n,t · × b ⊗ Σ ) . HE HOWE DUALITY CONJECTURE 15
We can compute the Jacquet module R P ( Y b ) ( ω W n ,V m ) by using Lemma 3.1 with the roles of H ( V m ) and G ( W n ) switched. But for this, it should be noted that the exponent λ b − k (for k < b ) in Lemma 3.1 satisfies:(5.4) λ b − k = − s m,n + b − k > > s m,n,t . Keeping this in mind, the last Hom space in (5.3) can be computed asHom G n × GL( Y b ) × H m − b ( R P ( Y b ) ( ω W n ,V m ) , π ⊗ χ W | − | s m,n,t · × b ⊗ Σ ) ֒ → Hom G n × GL( Y b ) × H m − b ( J b , π ⊗ χ W | − | s m,n,t · × b ⊗ Σ )= Hom GL( X b ) × G n − b × H m − b ( χ V | − | − s m,n,t · × b ⊗ ω W n − b ,V m − b , R Q ( X b ) ( π ) ⊗ Σ )= Hom GL( X b ) × G n − b ( χ V | − | − s m,n,t · × b ⊗ Θ W n − b ,V m − b ( Σ ) , R Q ( X b ) ( π ))= Hom G n ( χ V | − | − s m,n,t · × b ⋊ Θ W n − b ,V m − b ( Σ ) , π )where to obtain the second injection, we used the genericity of 1 × b and (5.4) as before. Thenagain by dualizing and applying MVW, we have π ֒ → χ V | − | s m,n,t · × b ⋊ σ for some σ which is a subquotient of Θ W n − b ,V m − b ( Σ ). By the maximality of a , we concludethat b ≤ a and hence b = a . This completes the proof of (i).(ii) Suppose that Π is given as in (5.2) with a maximal and some Σ . Now that we know that b = a in the proof of (i), we revisit the computations starting from (5.3):0 = Hom G n × H m (cid:0) ω W n ,V m , π ⊗ Π (cid:1) ֒ → Hom
GL( X b ) × G n − b × H m − b ( χ V | − | − s m,n,t · × b ⊗ ω W n − b ,V m − b , R Q ( X b ) ( π ) ⊗ Σ ) ֒ → Hom
GL( X a ) × G n − a × H m − a (cid:0) χ V | − | − s m,n,t · × a ⊗ ω W n − a ,V m − a ,R Q ( X a ) ( χ V | − | s m,n,t · × a ⋊ σ ) ⊗ Σ (cid:1) . To show the proposition, it suffices to show that the last Hom space embeds intoHom G n − a × H m − a ( ω W n − a ,V m − a , σ ⊗ Σ ) . To show this inclusion, we shall make use of Lemma 5.2. In Lemma 5.2(i), set σ ρ,a = σ a ,namely set ρ = χ V | − | s m,n,t . Tensoring the short exact sequence with Σ and then applyingthe functor Hom GL( X a ) × G n − a × H m − a ( χ V · | − | − s m,n,t · × a ⊗ ω W n − a ,V m − a , − ) , one sees that the desired inclusion follows from the assertion:Hom GL( X a ) × G n − a × H m − a ( χ V · | − | − s m,n,t · × a ⊗ ω W n − a ,V m − a , T ⊗ Σ ) = 0 . But this follows from Lemma 5.2(ii) which asserts that T does not contain any irreduciblesubquotient of the form χ W | − | − s m,n,t · × a ⊗ Σ ′ for any Σ ′ .This completes the proof of (ii). (cid:3) Proof of Theorem 1.2
We can now assemble the results of the last two sections and complete our proof of Theorem1.2. As is mentioned in Section 3.4, we may assume that m ≤ n + ǫ , or equivalently that s m,n ≤ G ( W ) and H ( V ).Further, we shall argue by induction on dim W . Thus, by induction hypothesis, we assumethat Theorem 1.2 is known for dual pairs G ( W ′ ) × H ( V ′ ) with dim V ′ ≤ dim W ′ + ǫ < n + ǫ .6.1. Irreducibility.
We first show the irreducibility of θ ( π ). If π does not occur on theboundary of I ( − s m,n ), this follows from Theorem 4.1. Thus, we assume π occurs on theboundary, so that π ֒ → χ V | det GL( X t ) | s m,n − t ⋊ σ for some t > σ . Let us write π ֒ → χ V | − | s m,n,t · × a ⋊ σ with a maximal. Let Π ⊆ θ ( π ) be an irreducible submodule. But by Proposition 5.4,0 = Hom G n × H m ( ω W n ,V m , π ⊗ Π ) ֒ → Hom G n − a × H m − a ( ω W n − a ,V m − a , σ ⊗ Σ ) , where Σ ⊂ θ W n − a ,V m − a ( σ ) is an irreducible representation such that Π ֒ → χ W | − | s m,n,t · × a ⋊ Σ where a is also maximal for Π . By induction hypothesis, we have Σ = θ W n − a ,V m − a ( σ ), andhence Π ֒ → χ W | − | s m,n,t · × a ⋊ θ W n − a ,V m − a ( σ ) . By Proposition 5.1, this induced representation has a unique submodule. This shows that θ ( π ) is an isotypic representation. Further, sincedim Hom G n − a × H m − a ( ω W n − a ,V m − a , σ ⊗ Σ ) = 1 , by the induction hypothesis, we conclude by Proposition 5.4(ii) thatdim Hom G n × H m ( ω W n ,V m , π ⊗ Π ) = 1 . This shows that Π occurs with multiplicity one in θ ( π ), so that θ ( π ) is irreducible.6.2. Disjointness.
It remains to prove that if θ ( π ) = θ ( π ′ ) = Π = 0, then π = π ′ . ByProposition 4.2, this holds unless both π and π ′ occur on the boundary for the same t ,namely there exists t > π ֒ → χ V | det GL( X t ) | s m,n − t ⋊ τ and π ′ ֒ → χ V | det GL( X t ) | s m,n − t ⋊ τ ′ . This means that we may write π ֒ → χ V | − | s m,n,t · × a ⋊ σ and π ′ ֒ → χ V | − | s m,n,t · × a ′ ⋊ σ ′ HE HOWE DUALITY CONJECTURE 17 with a and a ′ maximal (and for some σ and σ ′ ). But then by Proposition 5.4(i) one musthave a = a ′ , where a is maximal such that Π ֒ → χ W | − | s m,n,t · × a ⋊ Σ for some Σ .Moreover, with Proposition 5.4(ii) and the induction hypothesis, we have θ W n − a ,V m − a ( σ ) = Σ = θ W n − a ,V m − a ( σ ′ ) , so that σ ∼ = σ ′ . We then deduce by Proposition 5.1 that π ∼ = π ′ is the unique irreduciblesubmodule of σ a = χ V | − | s m,n,t · × a ⋊ σ . This completes the proof of Theorem 1.2.7. Quaternionic Dual Pairs
In this final section, we consider the case of the quaternionic dual pairs. As mentionedin the introduction, due to the lack of the MVW involution, we are not able to prove theHowe duality conjecture in full generality. The best we can prove is Theorem 1.3, where weconsider only Hermitian representations (i.e. those π such that π ∼ = π ∨ , where π is the complexconjugate of π ). The idea of the proof is essentially the same as for the non-quaternioniccase. But in place of the MVW involution, we use the involution π π. In what follows, we will outline how to modify the proof.7.1.
Setup.
Let us briefly recall the setup in the quaternionic case, with emphasis on theaspects which are different from before.Let B be the unique quaternion division algebra over F . For ǫ = ± , let W = W n be a rank n B -module equipped with a − ǫ -Hermitian form and V m a rank m B -module equipped withan ǫ -Hermitian form. Then the product G ( W n ) × H ( V m ) of isometry groups is a dual pair,with a Weil representation ω W,V associated to a pair of splitting characters ( χ V , χ W ). In thiscase, the characters χ V and χ W are simply (possibly trivial) quadratic characters determinedby the discriminants of the corresponding spaces V and W .For an isotropic subspace X t of rank t over B , let Q ( X t ) be the stabilizer of X t , which isa maximal parabolic subgroup of G ( W ) with the Levi factor L ( X t ) ∼ = GL( X t ) × G ( W n − t ),where GL( X t ) ∼ = GL t ( B ). We shall denote by det GL( X t ) : GL( X t ) −→ F × the reduced normmap. Likewise, a maximal parabolic subgroup P ( Y t ) of H ( V m ) is the stabilizer of an isotropicsubspace Y t of V m . As before, we have Tadi´c’s notation for parabolic induction.We set s m,n = m − n + ǫ . In the quaternionic case, we say that an irreducible representation π of G ( W n ) lies on theboundary of I ( − s m,n ) if there exists 0 < t ≤ q W such that π ֒ → χ V | det GL( X t ) | s m,n − t ⋊ σ for some irreducible representation σ of G ( W n − t ). If π ∼ = π ∨ , i.e. if π is Hermitian, then bydualizing and complex conjugating, we see that this is equivalent to:Hom GL( X t ) ( χ V | det GL( X t ) | − s m,n + t , R Q ( X t ) ( π )) = 0 . Non-boundary case.
We can now begin the proof of Theorem 1.3, starting with thecase when the Hermitian representation π does not lie on the boundary. For the sake ofproving Theorem 1.3, there is no loss of generality in assuming that m < n − ǫ , so that s m,n < π , with the following modifications: • The first place where the MVW involution is used in Section 4 is the see-saw identity (4.1).But one can see from the proof of the see-saw identity in [GI, § G ( W ) × G ( W ) ( Θ V,W + W − ( χ W ) , π ′ ⊗ π ∨ ) = Hom H ( V ) ∆ ( Θ ( π ′ ) ⊗ Θ ( π ) , C ) . Then (4.2) can be written asHom H ( V ) ∆ ( Θ ( π ′ ) ⊗ Θ ( π ) , C ) ⊇ Hom H ( V ) ∆ ( θ her ( π ′ ) ⊗ θ her ( π ) ∨ , C )= Hom H ( V ) ( θ her ( π ′ ) , θ her ( π )) . • There is an evident analogue of Lemma 3.2 in the quaternionic case. The statementis as given in Lemma 3.2, except that the terms | det GL( X t ) | s + t should be replaced by | det GL( X t ) | s + t .With these provisions, the rest of the argument in Section 4 does not use the MVW involution,and hence apply to the quaternionic case without any modification. One also has the analogueof Proposition 4.2, with the exponent s m,n − t replaced by s m,n − t .7.3. Boundary case.
Suppose that π is an irreducible Hermitian representation of G ( W )which lies on the boundary of I ( − s m,n ). Then, for some t >
0, one has π ֒ → χ V | − | s m,n,t · × a ⋊ σ with a maximal (and for some σ ) and s m,n,t = s m,n − t + 1 < . Now one has an analogue of Lemma 5.2, based on the explicit Geometric Lemma in thequaternionic case (which is written down in the thesis of M. Hanzer [Ha, Theorem 2.2.5]).Using this, one deduces the analogue of Proposition 5.1 by the same argument. However, itis essential to note the following lemma:
Lemma 7.1.
Suppose that π ֒ → χ V | − | s m,n,t · × a ⋊ σ with a maximal and for some σ . If π is Hermitian, so is σ .Proof. To see this, starting from π ֒ → ρ × a ⋊ σ (with a maximal and ρ a real-valued 1-dimensional character for which ρ ∨ = ρ ), one deduces (by dualizing and complex-conjugating)that ( ρ ∨ ) × a ⋊ σ ∨ ։ π ∨ ∼ = π. Now note that for the case at hand, the supercuspidal representation ρ satisfies ρ = ρ ; indeed, ρ is a real-valued character for our application. Thus, Bernstein’s Frobenius reciprocityimplies that ( ρ ∨ ) × a ⊗ σ ∨ ֒ → R Q ( X ra ) ( π ) . HE HOWE DUALITY CONJECTURE 19
However, the analogue of Lemma 5.2(i) says that the only irreducible subquotient of R Q ( X ra ) ( π )of the form ( ρ ∨ ) × a ⊗ σ is ( ρ ∨ ) × a ⊗ σ . Hence, we see that σ ∨ ∼ = σ . (cid:3) Then one has the following analogue of Proposition 5.4:
Proposition 7.2.
Assume that π is an irreducible Hermitian representation of G ( W ) and = Π ⊂ θ her ( π ) .(i) If π ֒ → χ V | − | s m,n,t · × a ⋊ σ with a maximal (and for some σ , necessarily Hermitian by Lemma 7.1), then Π ֒ → χ W | − | s m,n,t · × a ⋊ Σ for some Σ (necessarily Hermitian by Lemma 7.1) and where a is also maximal for Π .(ii) Moreover, whenever Π is presented as a submodule as above, one has = Hom G n × H m ( ω W n ,V m , π ⊗ Π ) ֒ → Hom G n − a × H m − a ( ω W n − a ,V m − a , σ ⊗ Σ ) , so that Σ ⊆ θ W n − a ,V m − a ,her ( σ ) . Proposition 7.2 is proved by the same argument as that for Proposition 5.4, using theanalogue of Lemma 3.1 (see [MVW, Chap. 3, Sect. IV, Thm. 5, Pg. 70]). We only take notethat in the statement of Lemma 3.1, the quantity λ a − k should be equal to s m,n + a − k inthe quaternionic case.With the above provisions, the rest of the proof goes through for the quaternionic case,which completes the proof of Theorem 1.3. Appendix: Proof of Lemma 5.2
The goal of this appendix is to prove the technical Lemma 5.2. We restate the lemma herefor the convenience of the reader.
Lemma 5.2.
Let ρ be a supercuspidal representation of GL r ( E ) and consider the inducedrepresentation σ ρ,a = ρ × a ⋊ σ of G ( W n ) where σ is an irreducible representation of G ( W n − ra ). Assume that(a) c ρ ∨ = ρ ;(b) σ * ρ ⋊ σ for any σ .Then we have the following:(i) One has a natural short exact sequence0 −−−−→ T −−−−→ R Q ( X ra ) σ ρ,a −−−−→ ( c ρ ∨ ) × a ⊗ σ −−−−→ T does not contain any irreducible subquotient of the form ( c ρ ∨ ) × a ⋊ σ ′ for any σ ′ . In particular, R Q ( X ra ) σ ρ,a contains ( c ρ ∨ ) × a ⊗ σ with multiplicity one, and does notcontain any other subquotient of the form ( c ρ ∨ ) × a ⊗ σ ′ . Likewise, R Q ( X ra ) σ ρ,a contains ρ × a ⊗ σ with multiplicity one and does not contain any other subquotient of the form ρ × a ⊗ σ ′ . (ii) The induced representation σ ρ,a has a unique irreducible submodule. Proof.
We shall use an explication of the Geometric Lemma of Bernstein-Zelevinsky due toTadi´c [T, Lemmas 5.1 and 6.3]. (See [HM] for the metaplectic group.) Tadi´c’s results implythat any irreducible subquotient δ ⊗ σ ′ of R Q ( X ra ) σ ρ,a is obtained in the following way.For any partition k + k + k = ra , write the semisimplification of the normalized Jacquetmodule of ρ × a to the Levi subgroup GL k × GL k × GL k as a sum of δ ⊗ δ ⊗ δ . Similarly,write the semisimplification of the normalized Jacquet module of σ to the Levi subgroupGL k × G ( W n − ra − k ) as a sum of δ ⊗ σ . Then δ is a subquotient of δ × c δ ∨ × c δ ∨ whereas σ ′ is a subquotient of δ ⊗ σ .For the case at hand, since ρ is supercuspidal, we can assume the partition of ra is of theform rk + rk + rk = ra , and the (semisimplified) normalized Jacquet module of ρ × a is theisotypic sum of ρ × k ⊗ ρ × k ⊗ ρ × k . Hence we see that for any irreducible subquotient δ ⊗ σ ′ of R Q ( X ra ) σ ρ,a , δ is a subquotient of ρ × k × ( c ρ × k ) ∨ × c δ ∨ .Now the irreducible subquotients of T correspond to those partitions with k > k > k = k = 0 corresponds to the closed cell in Q \ G/Q , which gives the thirdterm in the short exact sequence.) The conditions (a) and (b) then imply that δ = ( c ρ ∨ ) × a .This proves the statements about T in (i). Now Q ( X ra ) and Q ( X ra ) are conjugate in G ( W )by an element w which normalizes the Levi subgroup L ( X ra ) = GL( X ra ) × G ( W n − ra ), actingas the identity on G ( W n − ra ) and via g c ( t g − ) on GL( X ra ). Then one has w R Q ( X ra ) ( σ ρ,a ) = R Q ( X ra ) ( σ ρ,a )where the LHS is the representation of the Levi L ( X ra ) obtained by twisting R Q ( X ra ) ( σ ρ,a )by w . Hence, one deduces that R Q ( X ra ) ( σ ρ,a ) contains ρ × a ⊗ σ with multiplicity one.Finally, for (ii), let π ⊆ σ ρ,a be any irreducible submodule. Then the Frobenius reciprocityimplies that the semisimplification of R Q ( X ra ) ( π ) contains ρ × a ⊗ σ . Thus, if σ ρ,a containsmore than one irreducible submodule, the exactness of the Jacquet functor implies that R Q ( X ra ) ( σ ρ,a ) contains ρ × a ⊗ σ with multiplicity ≥
2, which contradicts (i). (cid:3)
References [GI] W. T. Gan and A. Ichino,
Formal degrees and local theta correspondence , Invent. Math. 195 (2014), no.3, 509-672.[GS] W. T. Gan and G. Savin,
Representations of metaplectic groups I: epsilon dichotomy and local Langlandscorrespondence , Compos. Math. (2012), 1655–1694.[GT] W. T. Gan and S. Takeda,
On the Howe duality conjecture in classical theta correspondence , to appearin Contemporary Math., volume in honor of J. Cogdell (2014), arXiv:1405.2626.[H1] R. Howe, θ -series and invariant theory , in Automorphic Forms, Representations and L-functions , Pro-ceedings of Symposia in Pure Mathematics Vol. 33 (1979), part 1, pp. 275-285.[H2] R. Howe,
Transcending classical invariant theory , J. Amer. Math. Soc. (1989), 535–552.[Ha] M. Hanzer, Inducirane reprezentacije hermitskih kvaternionskih grupa , Ph.D. thesis, University of Zagreb(2005).[HM] M. Hanzer, and G. Mui´c,
Parabolic induction and Jacquet functors for metaplectic groups , J. Algebra323 (2010), no. 1, 241–260.[K1] S. S. Kudla,
On the local theta-correspondence , Invent. Math. (1986), 229–255.[K2] S. S. Kudla, Splitting metaplectic covers of dual reductive pairs , Israel J. Math. (1994), 361–401. HE HOWE DUALITY CONJECTURE 21 [KR] S. S. Kudla and S. Rallis,
On first occurrence in the local theta correspondence , Automorphic represen-tations, L -functions and applications: progress and prospects, Ohio State Univ. Math. Res. Inst. Publ. , de Gruyter, Berlin, 2005, pp. 273–308.[LST] J.-S. Li, B. Sun, and Y. Tian, The multiplicity one conjecture for local theta correspondences , Invent.Math. (2011), 117–124.[LnST] Y. N. Lin, B. Y. Sun and S. B. Tan,
MVW-extensions of real quaternionic classical groups , Math. Z.277 (2014), no. 1-2, 81-89.[M] A. Minguez,
Correspondance de Howe explicite: paires duales de type II , Ann. Sci. ´Ec. Norm. Sup´er. (2008), 717–741.[MVW] C. Mœglin, M.-F. Vign´eras, and J.-L. Waldspurger, Correspondances de Howe sur un corps p -adique ,Lecture Notes in Mathematics , Springer-Verlag, Berlin, 1987.[M1] G. Mui´c, Howe correspondence for discrete series representations; the case of (Sp( n ) , O( V )), J. ReineAngew. Math. 567 (2004), 99-150.[M2] G. Mui´c, On the structure of the full lift for the Howe correspondence of (Sp( n ) , O( V )) for rank-onereducibilities , Canad. Math. Bull. (2006), 578–591.[M3] G. Mui´c, On the structure of theta lifts of discrete series for dual pairs (Sp( n ) , O( V )), Israel J. Math.164 (2008), 87–124[M4] G. Mui´c, Theta lifts of tempered representations for dual pairs (Sp n , O( V )), Canadian J. Math. 60 No.6 (2008), 1306-1335.[T] M. Tadi´c, Structure arising from induction and Jacquet modules of representations of classical p -adicgroups , Journal of Algebra vol. 177 (1995), Pg. 1-33.[W] J.-L. Waldspurger, D´emonstration d’une conjecture de dualit´e de Howe dans le cas p -adique, p = 2,Festschrift in honor of I. I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part I, Israel Math.Conf. Proc. , Weizmann, Jerusalem, 1990, pp. 267–324.[Z] A. V. Zelevinsky, Induced representations of reductive p-adic groups. II. On irreducible representations of
GL( n ), Ann. Sci. ´Ecole Norm. Sup. 13 (1980), 165–210. Department of Mathematics, National University of Singapore, 10 Lower Kent Ridge Road,Singapore 119076
E-mail address : [email protected] Mathematics Department, University of Missouri, Columbia, 202 Math Sciences Building,Columbia, MO, 65211
E-mail address ::