Uniqueness of certain Fourier-Jacobi models over finite fields
aa r X i v : . [ m a t h . R T ] O c t UNIQUENESS OF CERTAIN FOURIER-JACOBI MODELS OVER FINITEFIELDS
BAIYING LIU AND QING ZHANG
Abstract.
In this paper, we prove the uniqueness of certain Fourier-Jacobi models for the splitexceptional group G over finite fields with odd characteristic. Similar results are also proved forSp and U . Introduction
Uniqueness of Bessel models and Fourier-Jacobi models for classical groups over local fields,recently proved in [AGRS, LS, Su, SZ, GGP1] for various cases, has played very important roles in thestudy of automorphic representations and L -functions for classical groups. These uniqueness resultsare the starting points of the local Gan-Gross-Prasad conjectures [GGP1, Conjecture 17.1], and thekey ingredients to construct new Rankin-Selberg integrals on these groups (see [GPSR, GJRS, JZ]for some examples). They also give local functional equations, and thus local gamma factors, formany long-known local zeta integrals for these groups (see [Ka] for example).After proving the uniqueness of Bessel models and Fourier-Jacobi models for classical groups, anatural question to ask is whether one could define similar models for exceptional groups and proveanalogous uniqueness. For the simplest exceptional group, the split group of type G , we can definea Fourier-Jacobi model which is quite similar to the Sp case.Let k be a local field, and α, β be the two roots of G ( k ) with α the short root and β the longroot, and let P = M V be the parabolic subgroup of G ( k ) with β in its Levi subgroup M , where V is the unipotent part of P . Then one has M ∼ = GL ( k ) . Let J = SL ( k ) ⋉ V ⊂ P . Then onecan check that there is a projection map J → SL ( k ) ⋉ H , where H is the Heisenberg groupwith 3 variables. For a non-trivial additive character ψ of k , one then has a Weil representation ω ψ of f SL ( k ) ⋉ H , where f SL ( k ) is the metaplectic cover of SL ( k ). Thus we can view ω ψ as arepresentation of e J = f SL ( k ) ⋉ V . Given a genuine irreducible representation π of f SL ( k ), thetensor product π ⊗ ω ψ gives a representation of J . Let Π be an irreducible representation of G ( k ),a non-trivial element in Hom G ( k ) (Π , Ind G ( k ) J ( π ⊗ ω ψ )) is called a Fourier-Jacobi model of Π. ByFrobenius reciprocity, Hom G ( k ) (Π , Ind G ( k ) J ( π ⊗ ω ψ )) = Hom J (Π , π ⊗ ω ψ ) . We conjecture that these Hom spaces should have dimension at most one over local fields, at least forself-dual irreducible representation Π, see Conjecture 6.2. It is worthwhile to mention that Ginzburg[Gi] has constructed a local zeta integral which naturally lies in these Hom spaces.The main goal of this paper is to consider the analogue conjecture over finite fields k with oddcharacteristic and verify certain uniqueness of Fourier-Jacobi models for G ( k ). Precisely, we provethe following Theorem A (Theorem 7.1) . Let k be a finite field with odd characteristic. Let π be an irreduciblerepresentation of SL ( k ) which is not fully induced from the Borel subgroup. Then Ind G ( k ) J ( π ⊗ ω ψ ) is multiplicity free. Note that, over finite fields, the metaplectic cover f SL ( k ) splits and the Weil representation canbe defined over SL ( k ). Moreover, over finite fields, every finite dimensional representation is semi-simple, and thus the above theorem is equivalent to that for any irreducible representation Π of Mathematics Subject Classification.
Primary 20C33; Secondary 20G40.
Key words and phrases. multiplicity one, Fourier-Jacobi model, exceptional group G . G ( k ), one has dim Hom J (Π , π ⊗ ω ψ ) ≤ π is an irreducible representation of SL ( k ) which isnot fully induced from the Borel subgroup. However, if π is indeed an irreducible representationwhich is fully induced from the Borel subgroup, then the representation Ind G ( k ) J ( π ⊗ ω ψ ) may notbe multiplicity free, see Remark 7.2, which is quite different from the conjectural local fields case.On the other hand, if π is fully induced representation of SL ( k ) and if Π is an irreducible cuspidal representation of G ( k ), in [LZ], we prove that Hom J (Π , π ⊗ ω ψ ) ≤
1, which is used to prove theexistence of GL -twisted gamma factors.When we were working on the above result for the split exceptional group G , we realized that evenfor classical groups over finite fields, the uniqueness of Fourier-Jacobi models has not been settled ingeneral. Thus we decided to include some results on the uniqueness of certain Fourier-Jacobi modelsfor Sp and U over finite fields, which are quite similar to the G case: Theorem B (Theorem 4.1 and Theorem 5.2) . Let k be a finite field with odd characteristic and let E/k be a quadratic extension. Let G = Sp ( k ) ( respectively U ( k )) and let J be a subgroup of G which is isomorphic to SL ( k ) ⋉ H ( respectively U ( k ) ⋉ H E with H E the Heisenberg group of theform E ⊕ k ) . Let π be an irreducible representation of SL ( k ) ( respectively U ( k ) ) which is notfully induced from the Borel subgroup. Then Ind GJ ( π ⊗ ω ψ ) is multiplicity free. As in the G case, if π is an irreducible representation which is fully induced from the Borelsubgroup, then Ind GJ ( π ⊗ ω ψ ) may not be multiplicity free in general (see Remark 4.2). On the otherhand, since unitary groups are inner forms of general linear groups, one might expect that similarresults also hold for GL ( k ). But it turns out that even the Weil representation itself of GL ( k ) isnot multiplicity free (see [Ge, Proposition 4.2]), thus we could not expect similar results for generallinear groups. This also shows how tricky things can be over finite fields.We remark that, when π is a fully induced representation of SL ( k ) (resp. U ( k )) from the Borelsubgroup, the fact that Ind GJ ( π ⊗ ω ψ ), where G can be G ( k ) , Sp ( k ) (resp. U ( k )), may not bemultiplicity free comes from the fact that π ⊗ ω ψ | SL ( k ) (resp. π ⊗ ω ψ | U ( k ) ) is not multiplicity free.This is quite different from the local fields case and makes the uniqueness problem of Fourier-Jacobimodels over finite fields more complicated and thus more interesting. However, in this case, we stillexpect irreducible cuspidal representations of G occur with multiplicity one in Ind GJ ( π ⊗ ω ψ ) (seeour subsequent work [LZ]).Theoretically speaking, all of the above multiplicity one results could be checked using the knowncharacter tables for these groups, see [Sr] for the character table of Sp ( F q ) when q is odd, and[CR, En, EY] for the character table of G . However, although the groups in our consideration arerelatively small, their character tables are already too complicated to be used to give a proof ofthe above multiplicity one results practically. Alternatively, we use a variant of Gelfand-Kazhdanmethod (see Section 3.2) to prove the above results. In [T], using this approach, Teicher provedcertain multiplicity one results for GSp n and O n over finite fields.In general, let k be a finite field with odd characteristic and E/k be a quadratic extension,G n ( k ) = Sp n ( k ) or U n ( k ), and let ω ψ be the Weil representation of G n ( k ). A natural questionis then for what kind of irreducible representations π of G n ( k ), the tensor product π ⊗ ω ψ is stillmultiplicity free? If π is the trivial representation or a character, this is known to be true. If π isthe Steinberg representation, it is proved by Hiss and Zalesski ([HZ]) that π ⊗ ω ψ is multiplicity free.It seems that little is known for general π . According to the decomposition of π ⊗ ω ψ | SL ( k ) givenin Section 2.5, we guess that if π is an irreducible cuspidal representation of G n ( k ), then π ⊗ ω ψ should be multiplicity free. This uniqueness of general Fourier-Jacobi models towards analogues ofthe local Gan-Gross-Prasad conjectures over finite fields (using Deligne-Lusztig character theory) iscurrently our work in progress.These multiplicity one results have potential applications in establishing certain functional equa-tions over finite fields and proving the existence of certain twisted gamma factors as done by Rodittyin [Ro] for GL n over finite fields. In [Ni], Nien proved a local converse theorem for GL n ( k ) usingthe local gamma factors in [Ro]. We expect to construct the gamma factors (using an analogue ofRankin-Selberg method) and prove the converse theorems for other groups over finite fields in thefuture. NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 3
Finally, we remark that, although few results on the multiplicities of Fourier-Jacobi models overfinite fields have been obtained in the literature, analogue problems for Bessel models have beenvastly studied, see [GGP2, GP, Ha, R, Th, Za] for example.The paper is organized as follows. In Section 2, we give a review of representations of SL ( k )over finite fields. In Section 3 and 4, we prove Theorem B for Sp ( k ). The U ( k ) case is included inSection 5. We then introduce the Fourier-Jacobi models for G ( k ) in Section 6, and prove TheoremA in Section 7. Acknowledgement
The first-named author is partially supported by NSF grant DMS-1702218 and start-up fundsfrom the Department of Mathematics at Purdue University. The second-named author was partiallysupported by Guangdong Natural Science Foundation 2018A030310081, NSFC grant 11801577 anda PIMS posdtoc fellowship. The authors would like to thank Professor Meinolf Geck for usefuldiscussion and pointing out the reference [HZ]. Part of the work was finished when the second-named author was a research associate in Sun Yat-Sen University, Guangzhou, China, visited Bin Xuin Sichuan University and attended the workshop “Introductory Workshop: Group RepresentationTheory and Applications” at MSRI, where he had a chance to talk to Professor Meinolf Geck, hewould like to thank the support from these institutes.2.
Review of representations of SL(2) over finite fields
In this section, we collect some well-known facts on representations of SL over finite fields withodd characteristic.Throughout the paper, unless otherwise specified, we let q = p r with an odd prime p , and let k = F q , the unique (up to isomorphism) finite field with q elements. Let ψ be a fixed non-trivialadditive character of k . Write ǫ = (cid:16) − q (cid:17) and ǫ ( x ) = (cid:16) xq (cid:17) for x ∈ k × , where (cid:16) · q (cid:17) is the Legendresymbol. Note that ǫ is the unique non-trivial quadratic character of k × and the kernel of ǫ is k × , := (cid:8) x : x ∈ k × (cid:9) . We fix a generator κ of the cyclic group k × . Then a set of representatives of k × /k × , can be taken as { , κ } .Let E be the unique (up to isomorphism) quadratic extension of k . Let Fr : E → E be theFrobenius map defined by Fr( x ) = x q . Then Fr is the unique non-trivial element in the Galois groupGal( E/k ). Let Nm : E → k be the norm map and let Tr : E → k be the trace map. Note thatNm( x ) = x · Fr( x ) = x q +1 and Tr( x ) = x + x q . We can realize E as k [ √ κ ]. Under this realization,we have Fr( x + y √ κ ) = x − y √ κ, Nm( x + y √ κ ) = x − y κ, and Tr( x + y √ κ ) = 2 x, for x, y ∈ k. Let E = { x ∈ E × : Nm( x ) = 1 } . Then the norm map Nm : E × → k × induces an exact sequence1 → E → E × → k × → . Conjugacy classes and induced representations of SL(2).
Table 1 gives the conjugacyclasses of SL ( k ) (see [FH, § (cid:18) x x − (cid:19) and (cid:18) x − x (cid:19) are in the same class. Inthe last row, (cid:18) x yκy x (cid:19) and (cid:18) x − y − κy x (cid:19) are in the same class, and (cid:18) x yκy x (cid:19) can be mappedto x + y √ κ = ± E . Thus these representatives can be indexed by ( E − {± } ) / h Fr i . For (cid:18) x yκy x (cid:19) ∈ SL ( k ), we write ξ x,y = x + y √ κ .The simplest class of irreducible representations of SL ( k ) is that induced from characters ofthe Borel subgroup. Let B SL = A SL N SL be the Borel subgroup of SL ( k ) consisting of uppertriangular matrices, with diagonal torus A SL ∼ = k × and upper triangular unipotent subgroup N SL .Let χ be a character of k × ∼ = A SL . We can view χ as a character of B SL such that its actionon N SL is trivial. Let I ( χ ) be the induced representation Ind SL ( k ) B SL2 ( χ ), then dim I ( χ ) = q + 1. If χ = 1, then it is well-known that I ( χ ) is irreducible (see [Pr, § I ( χ ) ∼ = I ( χ − ), andthere are totally q − of them. If χ = 1, the trivial character, then I (1) = 1 ⊕ St, where 1 denotesthe trivial representation of SL ( k ) and St denotes the Steinberg representation. Hence, dim St = q . BAIYING LIU AND QING ZHANG
Representative Number of elements in class Number of classes (cid:18) (cid:19) (cid:18) − − (cid:19) (cid:18) (cid:19) ( q − / (cid:18) κ (cid:19) ( q − / (cid:18) − − (cid:19) ( q − / (cid:18) − κ − (cid:19) ( q − / (cid:18) x x − (cid:19) , x = ± q ( q + 1) ( q − / (cid:18) x yκy x (cid:19) , x = ± , y = 0 q ( q −
1) ( q − / Table 1.
Conjugacy class of SL ( k )If χ = ǫ , then I ( ǫ ) is a direct sum of two non-equivalent irreducible representations, each of whichis a Weil representation (cf. the next subsection), and has dimension q +12 ,2.2. Weil representations.
Since N SL ∼ = k , we could view ψ as a character of N SL . Up toconjugation by A SL , there are two non-trivial additive characters of N SL , which are ψ and ψ κ ,where ψ κ is the character given by ψ κ ( x ) = ψ ( κx ).Let W = k , endowed with the symplectic structure h , i defined by h ( x , y ) , ( x , y ) i = 2 x y − x y . Let H be the Heisenberg group associated with W . Explicitly, H = W ⊕ k with addition[ x , y , z ] + [ x , y , z ] = [ x + x , y + y , z + z + x y − x y ] . Let SL ( k ) act on H such that it acts on W by right multiplication and acts on the third component k in H trivially. Then we can form the semi-direct product SL ( k ) ⋉ H . It is well-known that thereis a Weil representation ω ψ of SL ( k ) ⋉ H on S ( k ), where S ( k ) is the space of C -valued functionson k . This representation is determined by the following formulas ω ψ ([ x, , z ]) φ ( ξ ) = ψ ( z ) φ ( ξ + x ) ,ω ψ ([0 , y, φ ( ξ ) = ψ (2 ξy ) φ ( ξ ) ,ω ψ (diag( a, a − )) φ ( ξ ) = ǫ ( a ) φ ( aξ ) ,ω ψ (cid:18)(cid:18) b (cid:19)(cid:19) φ ( ξ ) = ψ ( bξ ) φ ( ξ ) ,ω ψ (cid:18)(cid:18) b − b − (cid:19)(cid:19) φ ( ξ ) = 1 γ ( b, ψ ) X x ∈ k ψ (2 bxξ ) φ ( x ) , (2.1)where γ ( b, ψ ) = P x ∈ k ψ ( − bx ). The construction of Weil representations for more general groupsover finite fields was given in [Ge] after the seminal work of Weil [W]. The above formulas also couldbe found in [GH, p. 220].The representation ω ψ | SL ( k ) is reducible. In fact, let S ± ( k ) = { φ ∈ S ( k ) : φ ( − ξ ) = ± φ ( ξ ) , ∀ ξ ∈ k } . Then S + ( k ) and S − ( k ) are invariant under the action of SL ( k ). Denote the corresponding rep-resentations by ω + ψ and ω − ψ respectively. Then ω + ψ and ω − ψ are irreducible, dim ω + ψ = q +12 and NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 5 ch (cid:18) x x (cid:19) , x = ± (cid:18) x yx (cid:19) , x = ± y = 1 , κ (cid:18) x x − (cid:19) , x = ± (cid:18) x yκy x (cid:19) , x = ± y = 01 1 1 1 1St q − I ( χ ) ( q + 1) χ ( x ) χ ( x ) χ ( x ) + χ ( x − ) 0 ω ψ,µ , µ = 1 ( q − µ ( x ) − µ ( x ) 0 − ( µ + µ q )( ξ x,y ) ,ω + ψ , ω + ψ κ ǫ ( x ) 0 ω − ψ , ω − ψ κ − µ ( ξ x,y ) Table 2.
Character table of SL ( k )dim ω − ψ = q − . Similarly, from the character ψ κ , one can construct the Weil representation ω ψ κ = ω + ψ κ ⊕ ω − ψ κ . Moreover, we have I ( ǫ ) = ω + ψ ⊕ ω + ψ κ . One can also check that ω − ψ and ω − ψ κ are cuspidal, in the sense that they are not subrepresentations of I ( χ ) for any character χ of k × .2.3. Cuspidal representations.
We already have two cuspidal representations ω − ψ , ω − ψ κ . The re-maining cuspidal representations are also constructed from Weil representations.Let µ be a non-trivial character of E . Let W ( µ ) = (cid:8) f : E → C : f ( yx ) = µ − ( y ) f ( x ) , ∀ x ∈ E, y ∈ E (cid:9) . For f ∈ W ( µ ), we have f (0) = µ − ( y ) f (0), for all y ∈ E . Since µ is non-trivial, we get f (0) = 0.For any a ∈ k × , let x a ∈ E × be such that Nm( x a ) = a . Note that a function f ∈ W ( µ ) is uniquelydetermined by its values on the set { x a : a ∈ k × } . Thus dim W ( µ ) = q − ω ψ,µ of SL ( k ) on W ( µ ) such that ω ψ,µ (cid:18)(cid:18) a a − (cid:19)(cid:19) f ( ξ ) = f ( aξ ) ,ω ψ,µ (cid:18)(cid:18) b (cid:19)(cid:19) f ( ξ ) = ψ (Nm( ξ ) b ) f ( ξ ) ,ω ψ,µ (cid:18)(cid:18) − (cid:19)(cid:19) f ( ξ ) = − q − X y ∈ E ψ (Tr(Fr( y ) ξ )) f ( y ) . (2.2)See [Bu, § ω ψ,µ is cuspidal. If µ = 1, then ω ψ,µ is irreducible. Furthermore, ω ψ,µ isisomorphic to ω ψ,µ ′ if and only if µ ′ = µ ± . One can also check that ω ψ,µ ∼ = ω ψ κ ,µ , and thus therepresentation ω ψ,µ is independent of the choice of ψ . We then get q − equivalence classes in thefamily n ω ψ,µ , µ ∈ b E , µ = 1 o . Let µ be the unique non-trivial quadratic character of E , then onecan check that ω ψ,µ = ω − ψ ⊕ ω − ψ κ .So far, we get a list of irreducible representations1 , St , I ( χ ) ( χ = 1) , ω ± ψ , ω ± ψ κ , ω ψ,µ ( µ = 1) , and the only non-trivial relations among them are I ( χ ) ∼ = I ( χ − ) and ω ψ,µ ∼ = ω ψ,µ − . One caneasily see that this is a complete list of irreducible representations of SL ( k ) by checking that thecardinality of this list is exactly the same as the number of conjugacy classes of SL ( k ).2.4. Character table.
For a finite dimensional representation ρ of a finite group G , we denote bych ρ the character function of G . Recall that ch ρ ( g ) = Tr ρ ( g ) = P i r ii , if ρ ( g ) is identified with amatrix ( r ij ). The function ch ρ is a class function, i.e., it is constant on a conjugacy class. Table 2gives the character table of SL ( k ), which is taken from [FH, § s, t, u, v, s ′ , t ′ , u ′ , v ′ in Table 3 BAIYING LIU AND QING ZHANG (cid:18) (cid:19) (cid:18) − − (cid:19) (cid:18) (cid:19) (cid:18) κ (cid:19) (cid:18) − − (cid:19) (cid:18) − κ − (cid:19) ω + ψ q +12 q +12 ǫ s t s ′ t ′ ω + ψ κ q +12 q +12 ǫ t s t ′ s ′ ω − ψ q − − q − ǫ u v u ′ v ′ ω − ψ κ q − − q − ǫ v u v ′ u ′ Table 3.
Missing part of Table 2are given below: s = 12 + 12 √ ǫ q, t = 12 − √ ǫ q,u = −
12 + 12 √ ǫ q, v = − − √ ǫ q,s ′ = ǫ s, t ′ = ǫ t, u ′ = t ′ = ǫ t, v ′ = s ′ = ǫ s. The tensor product π ⊗ ω ψ . Given an irreducible representation π of SL ( k ), we want todetermine the decomposition of π ⊗ ω ψ . We first notice that ch ω ψ = ch ω + ψ + ch ω − ψ . Thus we havech (cid:18) (cid:19) (cid:18) − − (cid:19) (cid:18) (cid:19) (cid:18) κ (cid:19) (cid:18) − − (cid:19) (cid:18) − κ − (cid:19) (cid:18) x x − (cid:19) (cid:18) x yyκ x (cid:19) ω ψ q ǫ √ ǫ q −√ ǫ q ǫ ǫ ǫ ( x ) − µ ( ξ x,y )Let π be an irreducible representation of SL ( k ), we are going to compute ch π ⊗ ω ψ . It is well-knownthat ch π ⊗ ω ψ ( g ) = ch π ( g )ch ω ψ ( g ). Hence, we have the following character tablech St ⊗ ω ψ I ( χ ) ⊗ ω ψ ω ψ,µ ⊗ ω ψ ω + ψ ⊗ ω ψ ω + ψ κ ⊗ ω ψ ω − ψ ⊗ ω ψ ω − ψ κ ⊗ ω ψ (cid:18) (cid:19) q q ( q + 1) q ( q − q ( q +1)2 q ( q +1)2 q ( q − q ( q − (cid:18) − − (cid:19) ǫ q ( q + 1) χ ( − ǫ ( q − µ ( − ǫ q +12 q +12 − q +12 − q +12 (cid:18) (cid:19) √ ǫ q −√ ǫ q s √ ǫ q t √ ǫ q u √ ǫ q v √ ǫ q (cid:18) κ (cid:19) −√ ǫ q √ ǫ q − t √ ǫ q − s √ ǫ q − v √ ǫ q − u √ ǫ q (cid:18) − − (cid:19) ǫ χ ( − − ǫ µ ( − s t t s (cid:18) − κ − (cid:19) ǫ χ ( − − ǫ µ ( − t s s t (cid:18) x x − (cid:19) ǫ ( x ) ǫχ ( x ) + ǫχ ( x − ) 0 1 1 0 0 (cid:18) x yyκ x (cid:19) µ ( ξ x,y ) 0 ( µ + µ q ) µ ( ξ x,y ) 0 0 1 1where in the last two rows x = ±
1, and in the last row y = 0 as usual.Let A ⊂ c k × be a set of representatives of ( c k × − { , ǫ } ) / h χ = χ − i , where 1 is the trivial character.In other words, for each χ ∈ c k × − { , ǫ } , there is one and only one of χ, χ − is in A . Thus | A | = q − .Let B be a set of representatives of ( c E − { , µ } ) / h µ = µ − i . Then | B | = q − . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 7
Proposition 2.1.
Let χ ( resp. µ ) be a character of k × ( resp. E ) such that χ = 1 ( resp. µ = 1) . We have the following decomposition of representations of SL ( k ) : St ⊗ ω ψ = St M M χ ∈ A I ( χ ) M M µ ∈ B ω ψ,µ M ω + ψ M ω + ψ κ ,I ( χ ) ⊗ ω ψ = St M I ( ǫχ ) M M χ ∈ A,χ = ǫχ ± I ( χ ) M M µ ∈ B ω ψ,µ M ǫ ω + ψ ⊕ ω − ψ ) M − ǫ ω + ψ κ ⊕ ω − ψ κ ) ,ω ψ,µ ⊗ ω ψ = St M M χ ∈ A I ( χ ) M M µ ∈ B,µ = µ µ ± ω ψ,µ M − ǫ ω + ψ ⊕ ω − ψ ) M ǫ ω + ψ κ ⊕ ω − ψ κ ) ,ω + ψ ⊗ ω ψ = St M ǫ ⊕ M χ ∈ A I ( χ ) M ω + ψ M − ǫ M µ ∈ B ω ψ,µ M ω − ψ κ ,ω − ψ ⊗ ω ψ = 1 + ǫ ⊕ M χ ∈ A I ( χ ) M ω + ψ κ M − ǫ M µ ∈ B ω ψ,µ M ω − ψ ,ω + ψ κ ⊗ ω ψ = St M − ǫ ⊕ M χ ∈ A I ( χ ) M ω + ψ κ M ǫ M µ ∈ B ω ψ,µ M ω − ψ ,ω − ψ κ ⊗ ω ψ = 1 − ǫ ⊕ M χ ∈ A I ( χ ) M ω + ψ M ǫ M µ ∈ B ω ψ,µ M ω − ψ κ . Proof.
Denote by { V i } a complete set of irreducible representations of SL ( k ). Then any represen-tation V of SL ( k ) can be written as V = ⊕ m i V i . We have m i = (ch V , ch V i ), where(ch V , ch V i ) = 1 | SL ( k ) | X g ∈ SL ( k ) ch V ( g )ch V i ( g )is the standard inner product of ch V and ch V i . Thus to prove the proposition, one needs to computevarious (ch π ⊗ ω ψ , ch π ′ ) for any pair of irreducible representations π, π ′ of SL ( k ) using the abovetables. In the following, we only give the computation of (ch I ( χ ) ⊗ ω ψ , ch I ( χ ) ) for a character χ with χ = 1, and omit similar calculations of other cases.By the definition of the standard inner product, we have | SL ( k ) | (ch I ( χ ) ⊗ ω ψ , ch I ( χ ) )= q ( q + 1) + ( q + 1) χ χ ( − ǫ + q − √ ǫ q + q −
12 ( −√ ǫ q )+ q − χ χ ( − ǫ + q − χ χ ( − ǫ + q ( q + 1) X x ∈ ( k × −{± } ) / ( ± ( χ ( x ) + χ ( x − ))( ǫχ ( x ) + ǫχ − ( x ))= q ( q + 1) + 2 q ( q + 1) χ χǫ ( −
1) + q ( q + 1) X x ∈ k × −{± } ( χχ ǫ ( x ) + ǫχ − χ ( x )) . BAIYING LIU AND QING ZHANG
Since χ ( −
1) = χ − ( −
1) and q ( q + 1) = q ( q + 1)( q −
1) + 2 q ( q + 1), we get | SL ( k ) | (ch I ( χ ) ⊗ ω ψ , ch I ( χ ) )= q ( q + 1)( q −
1) + q ( q + 1) X x ∈ k × ( χχ ǫ ( x ) + χ − χ ǫ ( x )) . Note that when χ = ǫχ ± , we have P x ∈ k × ( χχ ǫ ( x ) + χ − χ ǫ ( x )) = 0, and if χ = ǫχ , or χ = ǫχ − ,we have P x ∈ k × ( χχ ǫ ( x ) + χ − χ ǫ ( x )) = q −
1. Since | SL ( k ) | = q ( q + 1)( q − I ( χ ) ⊗ ω ψ , ch I ( χ ) ) = 1 if χ = ǫχ ± , and (ch I ( χ ) ⊗ ω ψ , ch I ( χ ) ) = 2 if χ = ǫχ ± . (cid:3) Corollary 2.2.
Let π be an irreducible representation SL ( k ) . If π = I ( χ ) with χ = 1 , then thetensor product π ⊗ ω ψ is multiplicity free.Proof. The assertion follows from Proposition 2.1 and the fact that ǫ , − ǫ ∈ { , } . (cid:3) Remark 2.3.
Let k be a p -adic field, n be a positive integer. Let f Sp n ( k ) be the metaplectic coverof the symplectic group Sp n ( k ), and ω ψ be the Weil representation of f Sp n ( k ). Then by the maintheorem of [Su], for any irreducible smooth genuine representation π of f Sp n ( k ), the tensor productrepresentation π ⊗ ω ψ of Sp n ( k ) is multiplicity free. While when k is a finite field, Proposition 2.1shows that Hom SL ( k ) ( I ( ǫχ ) , I ( χ ) ⊗ ω ψ ) = 2. This shows that the multiplicity one result of π ⊗ ω ψ fails in general over finite fields. This kind of phenomenon is also known for Bessel models over finitefields (see [GGP2, R] for examples). (cid:3) Remark 2.4.
Over finite fields, few results are known on the multiplicities of decomposition of π ⊗ ω ψ | Sp n ( k ) for general n and general irreducible representations π of Sp n ( k ). To the author’sknowledge, only when π is the Steinberg representation of Sp n ( k ), it is shown in [HZ, Corollary1.3] that π ⊗ ω ψ is multiplicity free. Based on the SL ( k ) case given in Proposition 2.1 and thecorresponding results for Bessel models [GGP2, R], one might guess that π ⊗ ω ψ | Sp n ( k ) should bemultiplicity free when π is an irreducible cuspidal representation on Sp n ( k ). (cid:3) Dual representation.
For g ∈ SL ( k ), we define ι g = d gd , where d = diag( − , ι isan involution of SL ( k ). Given an irreducible representation π of SL ( k ), let ι π be the representationsuch that its space is the same with π and its action is given by ι π ( g ) = π ( ι g ). For later use, werecord the following result Lemma 2.5.
Let π be an irreducible representation of SL ( k ) , and ˜ π be its dual representation.Then we have ˜ π ∼ = ι π. Proof.
We have ch ˜ π ( g ) = ch π ( g − ) and ch ι π ( g ) = ch π ( ι g ). From the character table given in Section2.4, we can check case by case that ch ˜ π = ch ι π . Thus ˜ π ∼ = ι π. (cid:3) The involution ι is called an MVW involution of SL ( k ) and can be defined in a more generalsetting, see [MVW, p.91].3. Construction of transpose on certain End spaces
In this section, we introduce the multiplicity one problem of certain Fourier-Jacobi models on Sp and general strategies to attack such a problem. We then construct transpose operators on certainEnd spaces as preparations for proving the multiplicity one theorems for Sp , G , and U .3.1. The problem.
For a positive integer n , let J n be the matrix defined by J n = (cid:18) J n − (cid:19) , J = (1) . Let Sp n ( k ) = (cid:26) g ∈ GL n ( k ) | g (cid:18) J n − J n (cid:19) t g = (cid:18) J n − J n (cid:19)(cid:27) . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 9
Note that Sp ( k ) = SL ( k ). We will mainly focus on Sp ( k ) in this section. A typical element inthe torus of Sp ( k ) has the form t = diag( a, b, b − , a − ), a, b ∈ k × . Let α, β be the two simple rootsdefined by α ( t ) = a/b, β ( t ) = b , for a, b ∈ k × . Denote s α = − − , and s β = − . Then s α , s β are representatives of the reflections defined by α and β , respectively. We write x β ( b ) = b , b ∈ k. Let P = M U be the parabolic subgroup of Sp ( k ) with Levi subgroup M = a g a − , a ∈ k × , g ∈ SL ( k ) , and unipotent subgroup U = ∗ ∗ ∗ ∗ ∗ ∈ Sp ( k ) . Let M = { diag(1 , g, , g ∈ SL ( k ) } and let J = M U ⊂ P . We view SL ( k ) as a subgroup ofSp ( k ) via SL ( k ) ֒ → M .There is an isomorphism SL ( k ) ⋉ H → J defined by( g, [ v, z ]) → v zg v ∗ , where v = ( x, y ) ∈ W and v ∗ = (cid:18) y − x (cid:19) . We identify [ x, y, z ] as an element in J in this way.Thus the Weil representation ω ψ of SL ( k ) ⋉ H gives a representation of J . Given an irreduciblerepresentation π of SL ( k ), we consider the representation π ⊗ ω ψ of J . The action of π ⊗ ω ψ isgiven by π ⊗ ω ψ ( j )( v ⊗ v ) = π ( p ( j )) v ⊗ ω ψ ( j ) v , for j ∈ J, v ∈ π, v ∈ ω ψ , where p : J → SL ( k )is the natural projection. It is known that π ⊗ ω ψ is irreducible as a representation of J , see [Su] fora proof in the p -adic case which is also valid in the finite fields case.We might ask the question: for which irreducible representation π of SL ( k ), the induced repre-sentation Ind Sp ( k ) J ( π ⊗ ω ψ ) is multiplicity free?For a p -adic field k , given an irreducible genuine irreducible representation π of f SL ( k ), and anirreducible smooth representation σ of Sp ( k ), it is always true thatdim Hom J ( σ, π ⊗ ω ψ ) ≤ , which is the main theorem of [BR]. Any nonzero element in Hom J ( σ, π ⊗ ω ψ ) gives an embedding σ ֒ → Ind Sp ( k ) J ( π ⊗ ω ψ ), which is called a Fourier-Jacobi model of σ for the given datum ( π ⊗ ω ψ , J ).Thus the above result of [BR] says that the Fourier-Jacobi model of σ is unique (if it exists). Fourier-Jacobi models over local fields were defined in a more general context for many classical groups suchas Sp n , unitary groups and GL n in [GGP1]; the uniqueness of Fourier-Jacobi model in the p -adicfield case was proved in [GGP1, Su], and was proved in [LS] in the Archimedean case.For a finite field k , our purpose is to show that Ind Sp ( k ) J ( π ⊗ ω ψ ) is multiplicity free when π isan irreducible representation not of the form I ( χ ), where χ is a non-quadratic character of k × , see Section 4. When π = I ( χ ), it turns out that the induced representation Ind Sp ( k ) J ( I ( χ ) ⊗ ω ψ ) is ingeneral not multiplicity free, see Remark 4.2. This in fact is not surprising after Proposition 2.1.3.2. The general strategy.
Given a group G , an anti-involution τ on G is a map τ : G → G suchthat τ ( τ g ) = g and τ ( g g ) = τ g τ g , for all g, g , g ∈ G .In this subsection, let G be an arbitrary finite group and H be a subgroup of G . Let σ be arepresentation of H . We consider the algebra A ( G, H, σ ) = { K : G → End C ( σ ) : K ( h gh ) = σ ( h ) K ( g ) σ ( h ) } . The product in A ( G, H, σ ) is given by convolution: K ∗ K ( g ) = X x ∈ G K ( gx − ) K ( x ) . For K ∈ A ( G, H, σ ) and f ∈ Ind GH ( σ ), we define a function K ∗ f : G → σ by( K ∗ f )( x ) = 1 | G | X g ∈ G K ( xg − ) f ( g ) . One can check that K ∗ f ∈ Ind GH ( σ ). Denote by L K ∈ End(Ind GH ( σ )) the endomorphism f K ∗ f . Theorem 3.1 (Mackey, see [Pr, p.3]) . The assignment K L K defines an isomorphism between A ( G, H, σ ) and End G (Ind GH ( σ )) . Corollary 3.2.
The induced representation
Ind GH ( σ ) is multiplicity free if and only if the algebra A ( G, H, σ ) is commutative.Proof. By Schur’s Lemma, the representation Ind GH ( σ ) is multiplicity free if and only if End G (Ind GH ( σ ))is commutative. Then the assertion follows from Mackey’s Theorem, Theorem 3.1, directly. (cid:3) We are going to use the Gelfand-Kazhdan method to prove the commutativity of certain A ( G, H, σ ).We assume that there exists an anti-involution τ of G such that τ H = H , and there exists an anti-involution t on End C ( σ ) such that t ( σ ( h )) = σ ( τ h ) for all h ∈ H . Then for K ∈ A ( G, H, σ ), we candefine τ K : G → End C ( σ ) by ( τ K )( g ) = t ( K ( τ g )) . Lemma 3.3.
For
K, K , K ∈ A ( G, H, σ ) , we have (1) τ K ( h gh ) = σ ( h ) τ K ( g ) σ ( h ) , ∀ h , h ∈ H, g ∈ G ; thus τ K ∈ A ( G, H, σ );(2) τ ( τ K ) = K ;(3) τ ( K ∗ K ) = τ K ∗ τ K . Thus, τ is an anti-involution on A ( G, H, σ ) . The proof is routine and thus omitted. Certain detailed computation could be found in [T].
Corollary 3.4.
Let { g i , ≤ i ≤ n } be a subset of G such that G = ∪ i, ≤ i ≤ n Hg i H . If τ K ( g i ) = K ( g i ) for all i with ≤ i ≤ n for all K ∈ A ( G, H, σ ) , then A ( G, H, σ ) is commutative and thus Ind GH ( σ ) is multiplicity free.Proof. Since G = ∪ Hg i H and then τ K ( g ) = K ( g ) for all g ∈ G, K ∈ A ( G, H, σ ) from the assump-tion. Thus τ K = K . Since τ is an anti-involution, we get that A ( G, H, σ ) is commutative. (cid:3)
An anti-involution.
Denote d = diag( − , − , , . For g ∈ Sp ( k ), we define ι g = d − gd. τ g = ι g − . Then ι is the MVW involution on Sp ( k ), see [MVW, p.91], and τ is an anti-involution on Sp ( k ).We have ι [ x, y, z ] = [ x, − y, − z ] , and ι a bc d = a − b − c d . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 11
In particular, ι J = J . Note that the restriction of the involution ι to SL ( k ) is exactly the MVWinvolution considered in Section 2.6. Given a representation σ of J , we denote by ι σ the representation ι σ ( g ) = σ ( ι g ).Let π be an irreducible representation of SL ( k ) which is not fully induced from the Borel subgroupand let σ π = π ⊗ ω ψ . The aim of the rest of this section is to define a transpose t on End( σ π ) suchthat t ( σ ( j )) = σ ( τ j ) for all j ∈ J . First we need to define pairs between ω ψ and ι ω ψ , π and ι π .3.4. A pair on ω ψ × ι ω ψ . For a, b ∈ k , recall the delta function δ a,b = (cid:26) , if a = b ;0 , if a = b. Then the space S ( k ) has a basis { δ s , s ∈ k } , where δ s ( t ) = δ s,t . Lemma 3.5.
Consider the pair h φ, φ ′ i = X ξ ∈ k φ ( ξ ) φ ′ ( ξ ) , φ, φ ′ ∈ S ( k ) . We have h ω ψ ( j ) φ, ω ψ ( ι j ) φ ′ i = h φ, φ ′ i , ∀ j ∈ J, φ, φ ′ ∈ S ( k ) . Proof. If j = [ x, , z ], we have ω ψ ( j ) φ ( ξ ) = ψ ( z ) φ ( ξ + x ). On the other hand, we have ι [ x, , z ] =[ x, , − z ]. Thus ι ω ψ ( j ) φ ′ ( ξ ) = ω ψ ([ x, , − z ]) φ ′ ( ξ ) = ψ ( − z ) φ ′ ( ξ + x ). By changing variable on thesummation, we get h ω ψ ([ x, , z ] φ ) , ι ω ψ ([ x, , z ]) φ ′ i = X ξ ∈ k φ ( ξ ) φ ′ ( ξ ) = h φ, φ ′ i . Similarly, we can show that h ω ψ ( j ) φ, ι ω ψ ( j ) φ ′ i = h φ, φ ′ i , for j = [0 , y, , diag( a, a − ) , x β ( b ) . Let w = (cid:18) − (cid:19) and w ′ = (cid:18) − (cid:19) , then under the embedding SL ( k ) ֒ → Sp ( k ), we have ι w = w ′ . We need to show that h ω ψ ( w ) φ, ω ψ ( w ′ ) φ ′ i = h φ, φ ′ i . Since S ( k ) is spanned by { δ s } and the pair h , i is bilinear, it suffices to show that h ω ψ ( w ) δ s , ω ψ ( w ′ ) δ t i = h δ s , δ t i = δ s,t . By the formula (2.1), we have ω ψ ( w ) δ s ( ξ ) = 1 γ (1 , ψ ) ψ (2 sξ ) , ω ψ ( w ′ ) δ t ( ξ ) = 1 γ ( − , ψ ) ψ ( − tξ ) . Thus we get h ω ψ ( w ) δ s , ω ψ ( w ′ ) δ t i = 1 γ (1 , ψ ) γ ( − , ψ ) X ξ ∈ k ψ (2( s − t ) ξ ) = qγ (1 , ψ ) γ ( − , ψ ) δ s,t , where the last step follows from P ξ ∈ k ψ (2( s − t ) ξ ) = qδ s,t . On the other hand, it is well-known that γ (1 , ψ ) γ ( − , ψ ) = q , see [Bu, Exercise 4.1.14, p.420] for example. Thus we get h ω ψ ( w ) δ s , ω ψ ( w ′ ) δ t i = h δ s , δ t i = δ s,t . Since J = SL ( k ) ⋉ H is generated by diag( a, a − ) , x β ( b ), w , [ x, , z ] and [0 , y, h ω ψ ( j ) φ, ι ω ψ ( j ) φ ′ i = h φ, φ ′ i , ∀ j ∈ J. This completes the proof of the lemma. (cid:3)
Note that the pair constructed in Lemma 3.5 is symmetric and satisfies the property(3.1) h δ s , δ t i = δ s,t . A pair on St ⊗ ι St . Let 1 be the trivial character of k × . We consider the induced representation I (1) of SL ( k ). An element f ∈ I (1) is a function f : SL ( k ) → C such that f ( bg ) = f ( g ) , ∀ b ∈ B SL , g ∈ SL ( k ) , where B SL is the upper triangular subgroup of SL ( k ). For f , f ∈ I (1), we define a pair h f , f i = X g ∈ B SL2 \ SL ( k ) f ( g ) f ( d gd ) , where d = (cid:18) − (cid:19) ∈ GL ( k ). Note that this pair is well-defined and symmetric. Moreover, itsatisfies the property h r ( g ) f , r ( ι g ) f i = h f , f i , ∀ g ∈ SL ( k ) , f , f ∈ I (1) , (3.2)where r ( g ) f denotes the right translation action of g on the sections f .Recall that we have the Bruhat decomposition SL ( k ) = B SL ∪ B SL wN SL , where w = (cid:18) − (cid:19) .Let f ∈ I (1) whose support is in B SL and f ( b ) = 1 , for all b ∈ B SL . For r ∈ k , let f w,r ∈ I (1) bethe function such that its support is B SL w (cid:18) r (cid:19) , and f w,r (cid:18) bw (cid:18) r (cid:19)(cid:19) = 1 , ∀ b ∈ B SL . Then { f , f w,r : r ∈ k } forms a basis of I (1). Let f = f + P r ∈ k f w,r , i.e., f ( g ) = 1 for all g ∈ SL ( k ). Lemma 3.6.
We have following formulas: h f , f i = q + 1; h f , f w,r i = 1 , ∀ r ∈ k ; h f w,r , f w,s i = δ s, − r , ∀ r, s ∈ k. (3.3) Proof.
This follows from direct computations. (cid:3)
Recall that I (1) = 1 ⊕ St as a representation of SL ( k ). The subspace of I (1) spanned by f isan invariant subspace of I (1) and it corresponds to the trivial representation of SL ( k ). For r ∈ k ,define F r = f − f w,r . One can check that the space generated by { F r , r ∈ k } is also invariant underthe action of SL ( k ), which is the space of St. By Lemma 3.6, we have(3.4) h F r , F r ′ i = q − δ r, − r ′ . A pair on ω ± ψ ⊗ ι ω ± ψ . Recall that the space S + ( k ) of ω + ψ consists of functions φ ∈ S ( k )with φ ( − x ) = φ ( x ) for all x ∈ k . Let A be a set of representatives of k × / {± } . Then the set { δ , δ s + δ − s , s ∈ A } forms a basis of S + ( k ). For simplicity we write ∆ s = δ s + δ − s for s ∈ k × and∆ = 2 δ . The pair h i defined in Lemma 3.5 gives a pair on ω + ψ which satisfies h ω + ψ ( g ) φ, ω + ψ ( ι g ) φ ′ i = h φ, φ ′ i , ∀ g ∈ SL ( k ) , φ, φ ′ ∈ S + ( k ) . We have the formula(3.5) h ∆ s , ∆ t i = 2( δ s,t + δ s, − t ) . Recall that the space of ω − ψ consists of φ ∈ S ( k ) such that φ ( − x ) = − φ ( x ) for all x ∈ k . For s ∈ k × , write ∆ ′ s = δ s − δ − s . Then { ∆ ′ s , s ∈ A } forms a basis of ω − ψ . The pair h i defined in Lemma3.5 also gives a bilinear symmetric pair on ω − ψ which satisfies h ω − ψ ( g ) φ, ω − ψ ( ι g ) φ ′ i = h φ, φ ′ i , ∀ g ∈ SL ( k ) , φ, φ ′ ∈ ω − ψ . And we also have a formula(3.6) h ∆ ′ s , ∆ ′ t i = 2( δ s,t − δ s, − t ) . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 13
A pair on ω ψ,µ ⊗ ι ω ψ,µ . Let µ be a character of E with µ = 1. We then have an irreduciblecuspidal representation ω ψ,µ of SL ( k ). We need an explicit pair h i : ω ψ,µ × ω ψ,µ → C such that h ω ψ,µ ( g ) v , ω ψ,µ ( ι g ) v i = h v , v i , ∀ g ∈ SL ( k ) , v , v ∈ ω ψ,µ . Recall that the space of ω ψ,µ consists of functions f : E × → C such that f ( yx ) = µ − ( y ) f ( x ) forall y ∈ E , x ∈ E × . As in Section 2.3, for each a ∈ k × , we fix an element x a ∈ E × such thatNm( x a ) = a , then a function f ∈ ω ψ,µ is uniquely determined by its values on the set { x a , a ∈ k × } .For each a ∈ k × , we define a function f a ∈ ω ψ,µ such that f a ( x b ) = δ a,b . Then { f a : a ∈ k × } formsa basis of ω ψ,µ .For φ, φ ′ ∈ ω ψ,µ , notice that the function x φ ( x ) φ ′ ( x q ) on E × is E -invariant. We define apair h φ, φ ′ i = X x ∈ E \ E × φ ( x ) φ ′ ( x q ) = 1 q + 1 X x ∈ E φ ( x ) φ ′ ( x q ) , where we used φ (0) = 0 for φ ∈ ω ψ,µ . Then, we have(3.7) h f a , f b i = µ − ( x q − a ) δ a,b , ∀ a, b ∈ k × . Lemma 3.7.
We have h ω ψ,µ ( g ) φ, ω ψ,µ ( ι g ) φ ′ i = h φ, φ ′ i , ∀ φ, φ ′ ∈ ω ψ,µ , g ∈ SL ( k ) . Proof.
Note that a q = a for a ∈ k × , and Nm( ξ ) = Nm( ξ q ) = ξ q +1 for ξ ∈ E . By Eq.(2.2) and asimple changing of variables, we get h ω ψ,µ (cid:18)(cid:18) a a − (cid:19)(cid:19) φ, ω ψ,µ (cid:18)(cid:18) a a − (cid:19)(cid:19) φ ′ i = h φ, φ ′ i , and h ω ψ,µ (cid:18)(cid:18) b (cid:19)(cid:19) φ, ω ψ,µ (cid:18)(cid:18) − b (cid:19)(cid:19) φ i = h φ, φ ′ i , for all a ∈ k × , b ∈ k, φ, φ ′ ∈ ω ψ,µ .We now check h ω ψ,µ ( w ) φ, ω ψ,µ ( ι w ) φ ′ i = h φ, φ ′ i . It suffices to show that h ω ψ,µ ( w ) f a , ω ψ,µ ( ι w ) f b i = µ − ( x q − a ) δ a,b , ∀ a, b ∈ k × . We have ω ψ,µ ( w ) f a ( ξ ) = − q − X x ∈ E ψ (Tr( x q x qa ξ )) µ − ( x ) , and ω ψ,µ ( ι w ) f b ( ξ ) = − q − X y ∈ E ψ ( − Tr( y q x qb ξ )) µ − ( y ) . Thus h ω ψ,µ ( w ) f a , ω ψ,µ ( ι w ) f b i = q − q + 1 X ξ ∈ E X x,y ∈ E ψ (Tr( x q x qa ξ ) − Tr( y q x qb ξ q )) µ − ( xy )= q − q + 1 X ξ ∈ E X x,y ∈ E ψ (Tr( x − y − x qa ξ ) − Tr( x qb ξ q )) µ − ( xy ) , ξ ξy − = q − X ξ ∈ E X x ∈ E ψ (Tr( x − x qa ξ ) − Tr( x qb ξ q )) µ − ( x ) , x xy − = q − X ξ ∈ E X x ∈ E ψ (Tr( x − x qa ξ ) − Tr( x b ξ )) µ − ( x ) , since Tr( x qb ξ q ) = Tr( x b ξ )= q − µ − ( x q − a ) X ξ ∈ E X x ∈ E ψ (Tr( x − x a ξ ) − Tr( x b ξ )) µ − ( x ) , x xx q − a . If a = b , for any x ∈ E , the character ξ ψ (Tr( x − x a ξ ) − Tr( x b ξ )) on E is non-trivial, and thus h ω ψ,µ ( w ) f a , ω ψ,µ ( ι w ) f b i = 0. If a = b , the character ξ ψ (Tr( x − x a ξ ) − Tr( x b ξ )) on E is non-trivialunless x = 1, and thus h ω ψ,µ ( w ) f a , ω ψ,µ ( ι w ) f b i = µ − ( x q − a ). This completes the proof. (cid:3) Transpose operators on
End( π ⊗ ω ψ ) . Let π = 1 , St , ω ± ψ , ω ± ψ κ , or ω ψ,µ , and σ π = π ⊗ ω ψ .We have constructed pairs on π × ι π and on ω ψ × ι ω ψ in previous subsections. We then can definea pair on σ π by h f ⊗ φ , f ⊗ φ i = h f , f ih φ , φ i , f , f ∈ π, φ , φ ∈ ω ψ . From the construction, we have h σ π ( j )Φ , ι σ π ( j )Φ ′ i = h Φ , Φ ′ i , ∀ j ∈ J, Φ , Φ ′ ∈ σ π . For A ∈ End C ( σ π ), we define t A ∈ End C ( σ π ) by h t A (Φ) , Φ ′ i = h Φ , A (Φ ′ ) i , ∀ Φ , Φ ′ ∈ σ π . Lemma 3.8.
The assignment A t A is an anti-involution on End C ( σ π ) and satisfies t ( σ π ( j )) = σ π ( τ j ) , ∀ j ∈ J. Proof.
Note that the pair h , i on σ π is in fact symmetric from the construction, and it is routine tocheck that A t A is an anti-involution on End C ( σ π ).For Φ , Φ ′ ∈ σ π , j ∈ J , we have that h t ( σ π ( j ))Φ , Φ ′ i = h Φ , σ π ( j )Φ ′ i = h σ π ( τ j )Φ , Φ ′ i . Thus we get t ( σ π ( j )) = σ π ( τ j ) . (cid:3) Let σ π | SL ( k ) = L i V i be the decomposition given in Proposition 2.1, where V i is an irreduciblerepresentation of SL ( k ). Note that each V i occurred at most once in the decomposition. Letid V i ∈ End( σ π ) be the element such that id V i | V j = 0 if j = i , and id V i | V i is the identity. Lemma 3.9.
We have t id V i = id V i .Proof. Suppose that j = i , we need to show that t id V i | V j = 0. Suppose that this is false, thenthere exists v j ∈ V j such that t id V i ( v j ) = 0. To get a contradiction, it suffices to show that h t id V i ( v j ) , v i = 0 for all v ∈ σ π . If v / ∈ V i , we have h t id V i ( v j ) , v i = h v j , id V i v i = 0 since id V i v = 0.If v ∈ V i , we have h t id V i ( v j ) , v i = h v j , v i . If this is not zero, we then get a non-trivial pair between V i and V j such that h σ π ( g ) v, σ π ( ι g ) v ′ i 6 = 0, which would imply that e V i = ι V j . But we know that e V i = ι V i by Lemma 2.5 and ι V j is not isomorphic to ι V i by assumption. This proves the lemma. (cid:3) Certain multiplicity one theorems for Sp ( k )Our main theorem for Sp ( k ) is the following Theorem 4.1.
The representation
Ind Sp ( k ) J ( π ⊗ ω ψ ) of Sp ( k ) is multiplicity free if π = 1 , St , ω ± ψ , ω ± ψ κ , ω ψ,µ , where µ is a character of E with µ = 1 . Remark 4.2.
Before proving Theorem 4.1, we show that for q large, the representation Ind Sp J ( I ( χ ) ⊗ ω ψ ) is not multiplicity free. In the following, we write Sp ( k ) as G for simplicity. Recall that P isthe Kilingen parabolic subgroup with Levi M ∼ = GL ( k ) × SL ( k ). Given characters χ , χ of k × ,view χ ⊗ I ( χ ) as a representation of M ∼ = GL ( k ) × SL ( k ) and consider the parabolic inductionInd GP ( χ ⊗ I ( χ )). We claim that, if χ = ǫχ , thenHom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( χ ⊗ I ( χ ))) ≥ . In fact, by Frobenius reciprocity law,Hom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( χ ⊗ I ( χ ))) = Hom J ( I ( χ ) ⊗ ω ψ , Ind GP ( χ ⊗ I ( χ )) | J ) . By Mackey’s Theorem (see [Se, Proposition 22, p.58]),Ind GP ( χ ⊗ I ( χ )) | J = M s ∈ J \ G/P
Ind JP s (( χ ⊗ I ( χ s )) s ) , NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 15 where P s = sP s − ∩ J , and for a representation ρ of P , the representation ρ s of P s is defined by ρ s ( h ) = ρ ( s − hs ). Considering the element s = s α s β s α ∈ J \ G/P , we have thatHom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( χ ⊗ I ( χ ))) ⊃ Hom J ( I ( χ ) ⊗ ω ψ , Ind JP s ( χ ⊗ I ( χ )) s )= Hom P s ( I ( χ ) ⊗ ω ψ | P s , ( χ ⊗ I ( χ )) s ) . We have P s ∼ = SL ( k ) ֒ → M , and ( χ ⊗ I ( χ )) s = I ( χ ). Thusdim Hom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( χ ⊗ I ( χ ))) ≥ dim Hom SL ( k ) ( I ( χ ) ⊗ ω ψ , I ( χ )) . By Proposition 2 .
1, if χ = ǫχ , thendim Hom SL ( k ) ( I ( χ ) ⊗ ω ψ , I ( ǫχ )) = 2 . Thus dim Hom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( χ ⊗ I ( ǫχ ))) ≥ . By Mackey’s irreducibility criterion (see [Se, p.59]), if χ and ǫχ are in “general position”, theinduced representation Ind GP ( χ ⊗ I ( ǫχ )) is irreducible. Here two characters χ , χ are said to be ingeneral position, if ( χ ⊗ χ ) = ( χ ⊗ χ ) w for all w ∈ W (Sp ) − { } , where W (Sp ) denotes theWeyl group of Sp and ( χ ⊗ χ ) is viewed as a character of the maximal torus of G via( χ ⊗ χ )(diag( a, b, b − , a − )) = χ ( a ) χ ( b ) , and ( χ ⊗ χ ) w ( t ) = ( χ ⊗ χ )( w.t ) for t in the maximal torus. In fact, it is not hard to check that χ , χ are in general position if and only if χ = 1 , χ = 1 , χ = χ ± . For q large (in fact, q ≥ χ such that χ , ǫχ are in general position, so that Ind GP ( χ ⊗ I ( ǫχ )) isirreducible. Hence Ind GJ ( I ( χ ) ⊗ ω ψ ) is not multiplicity free for q large. (cid:3) Before we start the proof of Theorem 4.1, we also need to give the double coset decomposition J \ Sp ( k ) /J . Denote t ( a ) = diag( a, , , a − ) for a ∈ k × . From the decompositionSp ( k ) = P s α s β s α P ∪ P s α P ∪ P, and P = ∪ a ∈ k × t ( a ) J = ∪ a ∈ F × Jt ( a ) , we can get a set of representatives of the double coset J \ Sp ( k ) /J given by t ( a ) , η ( a ) := aa − aa − , ξ ( a ) := aI − a − , a ∈ k × . Proof of Theorem . . If π = 1, the multiplicity-freeness of Ind Sp ( k ) J ( ω ψ ) could be deduced fromthe main result of [T]. In the following, for completeness, we still give details of the proof in thiscase.Denote σ π = π ⊗ ω ψ for π listed in Theorem 4.1. In Section 3.8, we have constructed ananti-involution t on End( σ π ) such that t ( σ π ( j )) = σ π ( τ j ). We can define an anti-involution τ on A (Sp ( k ) , J, σ π ) by ( τ K )( g ) = t ( K ( τ g )) , K ∈ A (Sp ( k ) , J, σ π ) , g ∈ Sp ( k ) . By Corollary 3.4, it suffices to show that ( τ K )( g ) = K ( g ) for g = t ( a ) , η ( a ) , ξ ( a ) for all a ∈ k × and all K ∈ A (Sp ( k ) , J, σ π ). Replacing K by K − τ K , it suffices to show that for K ∈ A ( G, H, σ π )with τ K = − K , K ( g ) = 0 for g = t ( a ) , η ( a ) , ξ ( a ) , ∀ a ∈ k × . We shall assume τ K = − K and showthat K ( g ) = 0 for g = t ( a ) , η ( a ) , ξ ( a ) , a ∈ k × , case by case.Step (1), we show that K ( t ( a )) = 0 for all a ∈ k × . We first consider t ( a ) , a = ±
1. Since t ( a )[0 , , z ] = [0 , , a z ] t ( a ) , from the definition of A (Sp ( k ) , J, σ π ), ψ ( z ) K ( t ( a )) = ψ ( a z ) K ( t ( a )).Since ψ is non-trivial and a = 1, one can choose z ∈ k such that ψ ( z ) = ψ ( a z ) . Hence, K ( t ( a )) = 0 . Next, we show that K ( t ( a )) = 0 if a = 1. Since t ( a ) g = gt ( a ) , ∀ g ∈ SL ( k ), K ( t ( a )) σ π ( g ) = σ π ( g ) K ( t ( a )) , ∀ g ∈ SL ( k ) . This implies that K ( t ( a )) ∈ End SL ( k ) ( σ π ). As a representation of SL ( k ), by Proposition 2.1, wecan write σ π = M V i , where V i is an irreducible representation of SL ( k ) and i runs in certain index set. By Schur’sLemma, we can write K ( t ( a )) = X i C i id V i , with C i ∈ C depending on a . On the other hand, we have τ ( t ( a )) = t ( a ) if a = 1. Thus ( τ K )( t ( a )) = t ( K ( t ( a ))) by definition. By Lemma 3.9, the idempodents id V i are invariant under transpose, whichimplies that t ( K ( t ( a ))) = K ( t ( a )). Then the assumption τ K = − K implies that K ( t ( a )) = 0 . Thiscompletes Step (1).Step (2), we show that K ( η ( a )) = 0 for all a ∈ k × . We first record the following relations η ( a ) x β ( y ) = [0 , , a y ] η ( a ) , (4.1) η ( a )[0 , , a y ] = x β ( y ) η ( a ) , (4.2) η ( a )[0 , y,
0] = [0 , y, η ( a ) . (4.3)We now consider the cases π = 1 , St , ω ± ψ , ω ± ψ κ , ω ψ,µ , respectively.Case (2.1), π = 1. Recall that σ π = ω ψ has a basis { δ s , s ∈ k } . Applying formulas (2.1), we havethat(4.4) ω ψ ( x β ( y )) δ s = ψ ( ys ) δ s , ω ψ ([0 , y, δ s = ψ (2 sy ) δ s , ∀ y ∈ k. Assume that K ( η ( a )) δ s = P t ∈ k C s ( t ) δ t , for C s ( t ) ∈ C . From the relation (4.3), K ( η ( a )) σ π [0 , y,
0] = σ π [0 , y, K ( η ( a )) . Applying the above formula to δ s we have that ψ (2 sy ) X t C s ( t ) δ t = X t ψ (2 ty ) C s ( t ) δ t . Hence, C s ( t ) = 0 if t = s , and K ( η ( a )) δ s = C s ( s ) δ s .We now show that C s ( s ) = 0 using the assumption that K + τ K = 0. Since τ η ( a ) = η ( a ), τ K ( η ( a )) = t ( K ( η ( a ))). Thus we have t K ( η ( a )) + K ( η ( a )) = 0, which then implies that h t ( K ( η ( a )) δ s , δ s i + h K ( η ( a )) δ s , δ s i = 0 , ∀ s ∈ k. From the definition of t ( K ( η ( a ))), the above condition implies that h K ( η ( a )) δ s , δ s i = 0 . FromEq.(3.1), we then get C s ( s ) = 0. This shows that K ( η ( a )) = 0, for all a ∈ k × .Case (2.2), π = St. Recall that σ π = St ⊗ ω ψ has a basis { F r ⊗ δ s , r, s ∈ k } , see Section 3.5. Notethat(4.5) St( x β ( b )) F r = F r − b , St([ x, y, z ]) F r = F r , where the action of St is given by right translation.From the relation Eq.(4.1), we get K ( η ( a )) σ ( x β ( r )) = ψ ( a y ) K ( η ( a )). Applying this formula to F r ⊗ δ s and using Eq.(4.4) (4.5), we have ψ ( ys ) K ( η ( a )) F r − y ⊗ δ s = ψ ( a y ) K ( η ( a )) F r ⊗ δ s . In particular, K ( η ( a )) F r ⊗ δ s = ψ (( s − a ) r ) K ( η ( a )) F ⊗ δ s . Thus to show that K ( η ( a )) = 0 itsuffices to show that K ( η ( a )) F ⊗ δ s = 0 , ∀ s ∈ k. Assume that K ( η ( a )) F ⊗ δ s = P b,t ∈ k C s ( b, t ) F b ⊗ δ t . Here C s ( b, t ) ∈ C might also depend on a .From the relation (4.2), we have ψ ( a y ) K ( η ( a )) = σ π ( x β ( y )) K ( η ( a )). Applying this to F ⊗ δ s , weget that X b,t ψ ( a y ) C s ( b, t ) F b ⊗ δ t = X b,t C s ( b, t ) ψ ( yt ) F b − y ⊗ δ t = X b,t C s ( b + y, t ) ψ ( yt ) F b ⊗ δ t . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 17
Hence,(4.6) C s ( b + y, t ) = ψ (( a − t ) y ) C s ( b, t ) , ∀ b, t, y ∈ k. On the other hand, using the relation (4.3), we have K ( η ( a )) σ ([0 , y, σ ([0 , y, K ( η ( a )). Ap-plying this to F ⊗ δ s and using Eqs.(4.4), (4.5), we obtain that X b,t ∈ k ψ (2 sy ) C s ( b, t ) F b ⊗ δ t = X b,t ∈ k C s ( b, t ) ψ (2 ty ) F b ⊗ δ t . Thus we get ψ (2 sy ) C s ( b, t ) = ψ (2 ty ) C s ( b, t ) for all b, t, y ∈ k . If s = t , one can choose y such that ψ (2 sy ) = ψ (2 ty ), hence, C s ( b, t ) = 0. Write D s = C s (0 , s ), then C s ( b, s ) = ψ (( a − s ) b ) D s byEq.(4.6). We get that K ( η ( a )) F ⊗ δ s = X b C s ( b, s ) F b ⊗ δ s = X b ψ (( a − s ) b ) D s F b ⊗ δ s . Since τ η ( a ) = η ( a ), and τ K = − K , we get h K ( η ( a )) F ⊗ δ s , F ⊗ δ s i = 0 , where the pair h , i isdefined in Section 3.8. By Eqs.(3.1) and (3.4), the above equation is equivalent to D s + ( q − X b ∈ k ψ (( a − s ) b ) D s = 0 . Note that P b ∈ k ψ (( a − s ) b ) is either q or 0 depending on a = s or not. We then have D s = 0 . This shows that K ( η ( a )) = 0 when π = St.Case (2.3), π = ω ± ψ u for u = 1 , κ . In these cases, the proofs are similar, and we only give detailsfor π = ω + ψ . Recall that σ π = ω + ψ ⊗ ω ψ has a basis ∆ r ⊗ δ s , where r runs over A ∪ { } and s ∈ k .Recall that A is a set of representatives of k × / {± } (see Section 3.6).We first record the following formulas(4.7) σ ([0 , y, z ])∆ r ⊗ δ s = ψ (2 sy )∆ r ⊗ δ s , σ ( x β ( b ))∆ r ⊗ δ s = ψ ( b ( r + s ))∆ r ⊗ δ s . From the relation η ( a ) x β ( y ) = [0 , , a y ] η ( a ), we get K ( η ( a )) σ ( x β ( y )) = ψ ( a y ) K ( η ( a )). Applyingthis to ∆ r ⊗ δ s , we get that ψ (( r + s ) y ) K ( η ( a ))∆ r ⊗ δ s = ψ ( a y ) K ( η ( a ))∆ r ⊗ δ s . Since y is arbitrary, we get that(4.8) K ( η ( a ))∆ r ⊗ δ s = 0 if r + s = a If r + s = a , assume that K ( η ( a ))∆ r ⊗ δ s = P b,t C r,s ( b, t )∆ b ⊗ δ t , where t runs over k , and b runs over { } ∪ A . From the relations (4.2) and (4.7), we can get ψ ( a y ) K ( η ( a ))∆ r ⊗ δ s = σ ( x β ( y )) K ( η ( a ))∆ r ⊗ δ s , or ψ ( a y ) X b,t C r,s ( b, t )∆ b ⊗ δ t = X b,t C r,s ( b, t ) ψ ( y ( b + t ))∆ b ⊗ δ t . Since y is arbitrary, we can get(4.9) C r,s ( b, t ) = 0 , if b + t = a . From the relation (4.3), we have K ( η ( a )) σ ([0 , y, σ ([0 , y, K ( η ( a )). By Eq.(4.7), we have ψ (2 sy ) X b,t C r,s ( b, t )∆ b ⊗ δ t = X b,t C r,s ( b, t ) ψ (2 ty )∆ b ⊗ δ t . Hence, C r,s ( b, t ) = 0 if t = s , and(4.10) K ( η ( a ))∆ r ⊗ δ s = X b C r,s ( b, s )∆ b ⊗ δ s . Let b ∈ k be such that b = a − s . If b = b , then C r,s ( b, s ) = 0 by Eq.(4.9). Thus we get K ( η ( a ))∆ r ⊗ δ s = C r,s ( b , s )∆ b ⊗ δ s . On the other hand, if r = ± b , then C r,s ( b , s ) = 0 byEq.(4.8). Thus to show K ( η ( a )) = 0, it suffices to show that K ( η ( a ))∆ b ⊗ δ s = 0. Note that K ( η ( a ))∆ b ⊗ δ s = C b ,s ( b , s )∆ b ⊗ δ s . Since τ K = − K and τ η ( a ) = η ( a ), we have h K ( η ( a ))∆ b ⊗ δ s , ∆ b ⊗ δ s i = 0 , By the definition of the h , i (see Section 3.8) and Eq.(3.5), we get that0 = h K ( η ( a ))∆ b ⊗ δ s , ∆ b ⊗ δ s i = 2 C b ,s ( b , s ) . Thus we get C b ,s ( b , s ) = 0. This shows that K ( η ( a ))∆ b ⊗ δ s = 0, and hence K ( η ( a )) = 0.Therefore, we get K ( η ( a )) = 0, for all a ∈ k × , when π = ω + ψ .Case (2.4), π = ω ψ,µ for a character µ of E with µ = 1. Recall that π has a basis { f a , a ∈ k × } ,see Section 3.7. We record the following formulas σ ([0 , y, z ]) f a ⊗ δ s = ψ (2 sy + z ) f a ⊗ δ s ; σ ( x β ( b )) f a ⊗ δ s = ψ ( b ( a + s )) f a ⊗ δ s . (4.11)From the relation (4.1), we can get K ( η ( a )) σ ( x β ( y )) = ψ ( a y ) K ( η ( a )) . Applying this to f r ⊗ δ s ,we get ψ ( y ( r + s )) K ( η ( y )) f r ⊗ δ s = ψ ( a y ) f r ⊗ δ s . Since y is arbitrary, we get K ( η ( a )) f r ⊗ δ s = 0 if r + s = a . Assume that K ( η ( a )) f a − s ⊗ δ s = X b ∈ k × ,t ∈ k C s ( b, t ) f b ⊗ δ t . From the relations (4.2) and (4.11), we can obtain that ψ ( a y ) K ( η ( a )) f a − s ⊗ δ s = σ ( x β ( y )) K ( η ( a )) f a − s ⊗ δ s , i.e., ψ ( a y ) X b,t C s ( b, t ) f b ⊗ δ t = X b,t C s ( b, t ) ψ ( y ( b + t )) f b ⊗ δ t . Thus we get(4.12) C s ( b, t ) = 0 if b + t = a . Furthermore, using (4.3), we have K ( η ( a )) σ ([0 , y, f a − s ⊗ δ s = σ ([0 , y, K ( η ( a )) f a − s ⊗ δ s ,which is equivalent to X b,t C s ( b, t ) ψ (2 sy ) f b ⊗ δ t = X b,t C s ( b, t ) ψ (2 ty ) f b ⊗ δ t . Thus we get(4.13) C s ( b, t ) = 0 , if t = s. Thus we get K ( η ( a )) f a − s ⊗ δ s = C s ( a − s , s ) f a − s ⊗ δ s . Since τ K = − K and τ η ( a ) = η ( a ), we get h K ( η ( a )) f r ⊗ δ s , f r ⊗ δ s i = 0. By Eq.(3.7), it is easy tosee that C s ( a − s , s ) = 0. This shows that K ( η ( a )) = 0, for all a ∈ k × , in the case π = ω ψ,µ . Thisalso completes the proof of Step (2).Step (3), we show that K ( ξ ( a )) = 0 for all a ∈ k × . One can check that τ ( ξ ( a )) = ξ ( a ) and ξ ( a ) g = gξ ( a ) , ∀ g ∈ SL ( k ) . Thus K ( ξ ( a )) σ π ( g ) = σ π ( g ) K ( ξ ( a )) , ∀ g ∈ SL ( k ) . Hence, K ( ξ ( a )) ∈ End SL ( k ) ( σ π ). Let σ π | SL ( k ) = ⊕ V i be the irreducible decomposition as inProposition 2.1. As in the proof of K ( t ( a )) = 0 when a = 1 in Step (1), we can write K ( ξ ( a )) = P i C i id V i , with C i ∈ C depending on a . Since id V i is invariant under the transpose t by Lemma 3.9and t K ( ξ ( a )) + K ( ξ ( a )) = 0, we can get C i = 0 and thus K ( ξ ( a )) = 0 . This completes the proof of Theorem 4.1. (cid:3)
NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 19 A multiplicity one theorem for U over finite fields In this section, we briefly introduce a multiplicity one result for the unitary group U ( k ), which isquite similar to the Sp ( k ) case. Note that some notations which were used for subgroups of Sp ( k )in previous sections will be used for subgroups of U in this subsection.Recall that k is a finite field with odd cardinality q and E is the quadratic extension of k . DefineU n ( k ) = (cid:26) g ∈ GL n ( E ) : g (cid:18) J n − J n (cid:19) t ¯ g = (cid:18) J n − J n (cid:19)(cid:27) . Conjugacy classes and some simple representations of U ( k ) . The conjugacy classes ofU ( k ) is given in the following table (see [Ca]):Representative Number of elements in class Number of classes (cid:18) x x (cid:19) , x ∈ E q + 1 (cid:18) x xx (cid:19) , x ∈ E ( q − q + 1) q + 1 (cid:18) x ¯ x − (cid:19) , x ∈ E × − E q ( q + 1) ( q +1)( q − (cid:18) x yκy x (cid:19) , y = 0 q ( q − q ( q +1)2 Note that the norm map Nm : E × → k × is surjective and thus (cid:18) (cid:19) and (cid:18) κ (cid:19) are in the sameconjugacy classes. This is different from the SL ( k ) case. We explain a little bit about the last row.The condition (cid:18) x yκy x (cid:19) ∈ U ( k ) is equivalent to ¯ xy = x ¯ y and x ¯ x − κy ¯ y = 1, which implies that x ± y √ κ ∈ E . Note that unlike in the SL ( k ) case, here we don’t require that x, y ∈ k . We nowcount the number of representatives of the form (cid:18) x yκy x (cid:19) . If x = 0, we get y √ κ ∈ E and there aretotally q + 1 such y . If x = 0 , y = 0, let u = x + y √ κ, u = x − y √ κ . Then u , u ∈ E , u = ± u .There are totally ( q + 1)( q −
1) choices of u , u , and hence such many of x, y . Note that x + y √ κ and x − y √ κ give the same conjugacy class. Thus we totally have ( q + 1 + ( q + 1)( q − q ( q + 1) / q ( q −
1) by counting the centralizer of each representative. Note that there are ( q + 1) conjugacy classes and thus there are ( q + 1) irreducible representations of U ( k ).Let η be a character of E . View η as a representation of U ( k ) via the determinant mapdet : U ( k ) → E . We then have total q + 1 irreducible 1-dimensional representations of U ( k ).Let B U = A U ⋉ N U be the upper triangular Borel subgroup of U ( k ) with torus A U = (cid:8) diag( a, ¯ a − ) , a ∈ E × (cid:9) and unipotent subgroup N U . Given a character χ of E × , view it as acharacter on A U ∼ = E × and hence on B U such that the action of N U is trivial. We then considerthe induced representation I ( χ ) := Ind U ( k ) B U2 ( χ ) which is irreducible if and only if χ = ¯ χ − , where¯ χ − is the character of E × defined by ¯ χ − ( a ) = χ (¯ a − ) , a ∈ E × . Note that the condition χ = ¯ χ − is equivalent to χ ◦ Nm E/k = 1. Since the norm map Nm
E/k : E × → k × is surjective, the condition χ = ¯ χ − is equivalent to that χ | k × = 1 and giving such a character is amount to giving a character η of E via η ( a/ ¯ a ) = χ ( a ). On the other hand, we have I ( χ ) ∼ = I ( ¯ χ − ) and thus there are totally ( q + 1)( q −
2) irreducible representations of the form I ( χ ) , χ = ¯ χ − . Note that dim I ( χ ) = q + 1 . Consider the induced representation I (1), where 1 is the trivial character of E × . We have I (1) =1 L St, where 1 denotes the trivial representation of U ( k ) by abuse of notation, St is the Steinbergrepresentation and dim St = q. Given a character η of U ( k ), form the tensor product η ⊗ St, which isstill an irreducible representation of U ( k ) of dimension q . We then get another family of irreduciblerepresentations of U ( k ) given by n η ⊗ St , η ∈ b E o and there are total q + 1 of them. Note that if χ is a character of E × with χ | k × = 1, then I ( χ ) = η ⊗ I (1) = η L ( η ⊗ St), where η is the character of E determined by η ( a/ ¯ a ) = χ ( a ) for a ∈ E × . For simplicity, we write η ⊗ St as St η . The followingis the character table of the representations η, I ( χ ) , St η for η ∈ b E , χ ∈ b E × , χ = ¯ χ − :ch η ch I ( χ ) ch St η (cid:18) x x (cid:19) , x ∈ E η ( x ) ( q + 1) χ ( x ) qη ( x ) (cid:18) x xx (cid:19) , x ∈ E η ( x ) χ ( x ) 0 (cid:18) x ¯ x − (cid:19) , x ∈ E × − E η ( x ¯ x − ) χ ( x ) + χ (¯ x − ) η ( x ¯ x − ) (cid:18) x yκy x (cid:19) , y = 0 η ( x − κy ) 0 − η ( x − κy )5.2. Cuspidal representation.
Recall that ψ is a fixed non-trivial additive character of k , and weidentify ψ as a character of N U by the isomorphism N U ∼ = k .Let µ be a non-trivial character of E and let W ( µ ) = (cid:8) f : E → C : f ( yx ) = µ − ( y ) f ( x ) , ∀ x ∈ E, y ∈ E (cid:9) . Recall that we have a representation ω ψ,µ of SL ( k ) on W ( µ ). Let η be a character of E , one canextend the representation ω ψ,µ to a representation ω ψ,µ,η of U ( k ) such that ω ψ,µ,η (cid:18)(cid:18) a ¯ a − (cid:19)(cid:19) φ ( x ) = η ( a ¯ a − ) φ ( xa ) , φ ∈ W ( µ ) . Since any g ∈ U ( k ) can be written as g = diag( a, ¯ a − ) g for some a ∈ E × , g ∈ SL ( k ), the aboverelation uniquely determines an extension of ω ψ,µ . It is not hard to check that the extension ω ψ,µ,η is indeed a representation. Let 1 be the trivial representation of E , then ω ψ,µ,η = η ⊗ ω ψ,µ, . Wenow compute the character of ω ψ,µ,η .For each a ∈ k × , we fix an element x a ∈ E × with Nm( x a ) = a . Let f a ∈ W ( µ ) be the functionsuch that f a ( x b ) = δ a,b . Then { f a , a ∈ k × } becomes a basis of W ( µ ). Using this basis, we cancompute the following character table: ch ω ψ,µ,η (cid:18) x x (cid:19) , x ∈ E ( q − η ( x ) µ − ( x ) (cid:18) x xx (cid:19) , x ∈ E − η ( x ) µ − ( x ) (cid:18) x ¯ x − (cid:19) , x ∈ E × − E (cid:18) x yκy x (cid:19) , y = 0 − η ( x − κy )( µ − ( x + y √ κ ) + µ − ( x − y √ κ ))From this table, we can check that the representation ω ψ,µ,η is irreducible if µ is non-trivial (whichis always assumed). On the other hand, we have ω ψ,µ,η ∼ = ω ψ,µ ,η if and only if ( µ , η ) = ( µ, η )or ( µ − , ηµ − ). Thus there are totally ( q + 1) q/ ω ψ,µ,η . The followingfamily η, St η , I ( χ ) , ω ψ,µ,η , with µ, η ∈ b E , µ = 1 , χ ∈ b E × , χ = ¯ χ − , is a complete list of irreducible representations of U ( k ).5.3. The Weil representation.
Let W = E ⊕ E , endowed with the skew-Hermitian structure h u, v i = uJ t ¯ v, where u, v are viewed as row vectors. We consider the Heisenburg group H = W ⊕ k with addition[ u , t ] + [ u , t ] = [ u + u , t + t + 12 Tr E/k ( h u , u i )] , u , u ∈ W, t , t ∈ k. The group U ( k ) act on H by g. [ u, t ] = [ gu, t ] , g ∈ U ( k ) , u ∈ W, t ∈ F . Thus we can form thesemi-direct product U ( k ) ⋉ H . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 21
There is a Weil representation ω ψ of U ( k ) ⋉ H on S ( E ) determined by the formulas ω ψ ([ x, , z ]) f ( ξ ) = ψ ( z ) f ( ξ + x ) , x ∈ E, z ∈ k,ω ψ ([0 , y, f ( ξ ) = ψ (Tr(¯ yξ )) f ( ξ ) , y ∈ E,ω ψ (cid:18)(cid:18) a ¯ a − (cid:19)(cid:19) f ( ξ ) = f ( aξ ) , a ∈ E × ,ω ψ (cid:18)(cid:18) b (cid:19)(cid:19) f ( ξ ) = ψ (Nm( ξ ) b ) f ( ξ ) , b ∈ k,ω ψ ( w ) f ( ξ ) = − q − X y ∈ E ψ (Tr(¯ yξ )) f ( y ) . Here w = (cid:18) − (cid:19) as usual. The Weil representation for general unitary group over finite fields isconstructed in [Ge]. The above formulas could be found in [Bu, Section 4.1].We next consider the restriction ω ψ | U ( k ) . Let W (1) be the subspace of S ( E ) which consists offunctions f ∈ S ( E ) such that f ( ux ) = f ( x ) , ∀ u ∈ E , x ∈ E. Then it is not hard to check thatas representations of U ( k ), ω ψ | W (1) ∼ = St. Thus we have the following decomposition as in [Ge,Corollary 4.5] ω ψ | U ( k ) = St M M µ ∈ b E ,µ =1 ω ψ,µ, . From this decomposition, the character table of ω ψ is as followingch ω ψ (cid:18) (cid:19) q (cid:18) x x (cid:19) , x ∈ E , x = 1 1 (cid:18) (cid:19) − q (cid:18) x xx (cid:19) , x ∈ E , x = 1 1 (cid:18) x ¯ x − (cid:19) , x ∈ E × − E (cid:18) x yκy x (cid:19) , x ± y √ κ = 1 , y = 0 − q (cid:18) x yκy x (cid:19) , x ± y √ κ = 1 , y = 0 1One can compare the above table with [Ge, Theorem 4.5, Corollary 4.8.2 and Theorem 4.9.2].5.4. On the tensor product of an irreducible representation with the Weil representa-tion.
In this section, we consider the decomposition of π ⊗ ω ψ for an irreducible representation π of U ( k ).Let A be a set of representatives of characters of E × such that χ | k × = 1 modulo the relation χ = ¯ χ − . Then the cardinality of A is ( q + 1)( q − /
2. Let B be a set of representatives of pairs( µ, η ) ∈ b E × b E with µ = 1 modulo the relation ( µ, η ) = ( µ − , ηµ − ). Then the cardinality of B is( q + 1) q/
2. The following proposition is U ( k ) analogue of Proposition 2.1 and we omit its proof. Proposition 5.1.
Let χ ∈ b E × with χ = ¯ χ − , and η , µ ∈ b E with µ = 1 . We have η ⊗ ω ψ = St η M M µ ∈ b E ,µ =1 ω ψ,µ,η ,I ( χ ) ⊗ ω ψ = 2 I ( χ ) M M χ ∈ A,χ = χ , ¯ χ − I ( χ ) M M η ∈ b E St η M M ( µ,η ) ∈ B ω ψ,µ,η , St ⊗ ω ψ = 1 M M χ ∈ A I ( χ ) M M η ∈ b E St η M M ( µ,η ) ∈ B,η =1 ,η = µ ω ψ,µ,η ,ω ψ,µ , ⊗ ω ψ = 1 M µ − M M χ ∈ A I ( χ ) M M η =1 ,η = µ St η M M ( µ,η ) ∈ B,η =1 ,η = µ,η = µ ,η = µ µ ω ψ,µ,η Note that St η ⊗ ω ψ = η ⊗ St ⊗ ω ψ , and ω ψ,µ,η ⊗ ω ψ = η ⊗ ω ψ,µ, ⊗ ω ψ , the decomposition ofSt η ⊗ ω ψ (resp. ω ψ,µ,η ⊗ ω ψ ) can be given using the decomposition of St ⊗ ω ψ (resp. ω ψ,µ, ⊗ ω ψ ).5.5. A multiplicity one result for U ( k ) . In the group U ( k ), we consider the subgroup J = ∗ ∗ g ∗ , g ∈ U . Then there is an isomorphism U ⋉ H → J defined by( g, [ v, z ]) → v z − ¯ xy + x ¯ yg v ∗ , g ∈ U , v = ( x, y ) ∈ E , z ∈ k where v ∗ = (cid:18) ¯ y − ¯ x (cid:19) . Under this isomorphism, we view ω ψ as a representation of J . Given anirreducible representation π of U ( k ), we consider the tensor product representation π ⊗ ω ψ of J .Similar to the Sp ( k ) case, we have the following Theorem 5.2.
Let π be an irreducible representation of U which is not of the form I ( χ ) . Thenthe induced representation Ind U J ( π ⊗ ω ψ ) is multiplicity free. Since the proof is similar to that of Theorem 4.1, we omit the details.6.
The group G In this section, we introduce the multiplicity one problem of certain Fourier-Jacobi models for thesplit exceptional group G ( k ), which is quite similar to what we considered in Section 3 for Sp ( k ).In this section, the notations J, P, M, etc. , will be used as subgroups of G ( k ) rather than subgroupsof Sp ( k ) as in Sections 3 and 4. For simplicity, denote G = G ( k ).6.1. Roots and commutator relations.
The group G has two simple roots, the short root α and the long root β . The set of the positive roots is Σ + = { α, β, α + β, α + β, α + β, α + 2 β } .Let ( , ) be the inner product in the root system and h , i be the pair defined by h γ , γ i = γ ,γ )( γ ,γ ) .For G , we have the relations: h α, β i = − , h β, α i = − . For a root γ , let s γ be the reflection defined by γ , i.e., s γ ( γ ′ ) = γ ′ − h γ ′ , γ i γ . We have the relation s α ( β ) = 3 α + β, s β ( α ) = α + β. Let W (G ) be the Weyl group of G , which is generated by s α , s β and has size 12. NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 23
We use the following standard notations from Chevalley group theory (see [St]). For a root γ , let U γ ⊂ G be the root space of γ , and let x γ : k → U γ be a fixed isomorphism whichsatisfies various Chevalley relations (see [St, Chapter 3]). Following [St], for t ∈ k × , denote w γ ( t ) = x γ ( t ) x − γ ( − t − ) x γ ( t ) and w γ = w γ (1). Note that w γ is a representative of s γ . Let h γ ( t ) = w γ ( t ) w − γ . For simplicity, we denote w = w α w β w − α , w = w α w β w α w − β w − α . Let T be the subgroup of G which consists of elements of the form h α ( t ) h β ( t ) , t , t ∈ k × and U be the subgroup of G generated by U γ for all γ ∈ Σ + . Let B = T U , which is a Borel subgroup of G . For t , t ∈ F × , denote h ( t , t ) = h α ( t t ) h β ( t t ). We can check the following relations h − ( t , t ) x α ( r ) h ( t , t ) = x α ( t − r ) ,h − ( t , t ) x β ( r ) h ( t , t ) = x β ( t − t r ) . (6.1)The notation h ( a, b ) agrees with that of [Gi], and our h ( a, b ) is h ( a, b, a − b − ) in the notation of[CR]. One can also check that(6.2) w α h ( t , t ) w − α = h ( t t , t − ) , w β h ( t , t ) w − β = h ( t , t ) . For g , g ∈ G , denote [ g , g ] = g − g − g g . We have the following commutator relations (see [Re,p.443]): [ x α ( x ) , x β ( y )] = x α + β ( − xy ) x α + β ( − x y ) x α + β ( x y ) x α +2 β ( − x y ) , [ x α ( x ) , x α + β ( y )] = x α + β ( − xy ) x α + β (3 x y ) x α +2 β (3 xy ) , [ x α ( x ) , x α + β ( y )] = x α + β (3 xy ) , [ x β ( x ) , x α + β ( y )] = x α +2 β ( xy ) , [ x α + β ( x ) , x α + β ( y )] = x α +2 β (3 xy ) . (6.3)For all the other pairs of positive roots γ , γ , we have [ x γ ( x ) , x γ ( y )] = 1.We also need the Chevalley relation w γ x γ ( r ) w − γ = x w γ ( γ ) ( c ( γ , γ ) r ) (see [St, Lemma 20,(b)]), where c ( γ , γ ) ∈ {± } and c ( γ , γ ) = c ( γ , − γ ). The numbers c ( γ , γ ) are given in thefollowing: c ( α, α ) = c ( α, α + β ) = c ( α, α + β ) = − , c ( α, β ) = c ( α, α + β ) = c (3 α + 2 β ) = 1 , (6.4) c ( β, β ) = c ( β, α + β ) = c ( β, α + 2 β ) = − , c ( β, α ) = c ( β, α + β ) = c ( β, α + β ) = 1 . Subgroups.
The group G has two proper parabolic subgroups. Let P = M ⋉ V be theparabolic subgroup of G with Levi M and unipotent V , such that U β ⊂ M ∼ = GL ( k ). Theisomorphism M ∼ = GL is determined by x β ( r ) (cid:18) r (cid:19) , h ( a, b ) (cid:18) a b (cid:19) . The unipotent subgroup V of P consists of root spaces of α, α + β, α + β, α + β, α + 2 β , and atypical element of V is of the form x α ( r ) x α + β ( r ) x α + β ( r ) x α + β ( r ) x α +2 β ( r ) , r i ∈ k. To ease the notation, we write the above element as ( r , r , r , r , r ). Denote by J the followingsubgroup of P J = SL ( k ) ⋉ V. We always view SL ( k ) as a subgroup of G via the embedding SL ( k ) ⊂ GL ( k ) ∼ = M ֒ → G . Let V (resp. Z ) be the subgroup of V which consists of root spaces of 3 α + β and 3 α + 2 β (resp.2 α + β, α + β and 3 α + 2 β ). Note that P and hence J normalize V and Z . An anti-involution on G . For g ∈ G , we define ι g = h (1 , − gh (1 , −
1) and τ g = ι g − . Thenwe can check that ι ( r , r , r , r , r ) = ( − r , r , − r , r , − r ) , and ι (cid:18) a bc d (cid:19) = (cid:18) a − b − c d (cid:19) . In particular, both the involution ι and the anti-involution τ preserve J .6.4. Weil representations and the problem.
Let W = k , endowed with the symplectic struc-ture h , i defined by(6.5) h ( x , y ) , ( x , y ) i = − x y + 2 x y . Note that the symplectic form on W here is different from the one defined in Section 2.2. The reasonfor choosing this non-standard symplectic structure on W will be explained below.Let H be the Heisenberg group associated with the symplectic space W . Explicitly, H = W ⊕ k with addition [ x , y , z ] + [ x , y , z ] = [ x + x , y + y , z + z − x y + x y ] . Let SL ( k ) act on H such that it acts on W from the right and acts on the third component k in H trivially. Then we can form the semi-direct product SL ( k ) ⋉ H . The product map in SL ( k ) ⋉ H is given by ( g , v )( g , v ) = ( g g , v .g + v ) . Let ψ be a fixed non-trivial additive character of k , and let ω ψ be the Weil representation ofSL ( k ) ⋉ H on S ( k ), where S ( k ) is the space of C -valued functions on k . The formulas (2.1) shouldbe adapted to our new symplectic structure on W .Define a map pr : V → H pr(( r , r , r , r , r )) = [ r , r , r − r r ] . From the commutator relations in (6.3), we can check that pr is a group homomorphism and definesan exact sequence 0 → V → V → H → . Here, to ensure that pr is a group homomorphism, we need to choose the symplectic form on W inthe non-standard way (6.5). It seems that there is a typo in the formula of the projection map prin [Gi, p.316].For g = (cid:18) a bc d (cid:19) ∈ SL ( F ) ⊂ M , we have g − ( r , r , r , , g = ( r ′ , r ′ , r ′ , r ′ , r ′ ) , where r ′ = ar − cr , r ′ = − br + dr , r ′ − r ′ r ′ = r − r r . This implies that the map pr : J =SL ( F ) ⋉ V → SL ( F ) ⋉ H , defined by,( g, v ) ( g ∗ , pr( v )) , g ∈ SL , v ∈ V, is a group homomorphism, where g ∗ = (cid:18) a − b − c d (cid:19) = d gd − and d = diag( − , ∈ GL ( k ). Now,we can view the Weil representation ω ψ as a representation of J by composing with the map pr.Given an irreducible representation π of SL ( k ), view it as a representation of J via the quotientmap J → SL ( k ).Similarly as the Sp ( k ) case, we consider the problem: for what irreducible representation π ofSL ( k ), the induced representation Ind G ( k ) J ( π ⊗ ω ψ ) is multiplicity free? The answer is similar tothe Sp ( k ) case and the proof will be given in next section. Remark 6.1. If k is a local field, let f SL ( k ) be the metaplectic double cover of SL ( k ). Then thereis a Weil representation ω ψ of f SL ( k ) ⋉ H . By composing with the projection map, we could view ω ψ as a representation of f SL ( k ) ⋉ V . Given a genuine irreducible representation π of f SL ( k ), the NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 25 tensor product π ⊗ ω ψ can be viewed as a representation of J = SL ( k ) ⋉ V . Due to the similaritybetween the G case and the Sp case, we propose the following Conjecture 6.2.
For any self-dual irreducible representation σ of G ( k ) and any irreducible genuinerepresentation π of f SL ( k ) , we have dim Hom J ( σ, π ⊗ ω ψ ) ≤ . Here the “self-dual” condition might be removable and we add it due to certain technical diffi-culties in the application of the Gelfand-Kazhdan method. A nonzero element in Hom J ( σ, π ⊗ ω ψ )defines an embedding σ ֒ → Ind G ( k ) J ( π ⊗ ω ψ ). Such a realization of σ will be called a Fourier-Jacobi model of σ with respect to the datum ( π ⊗ ω ψ , J ). Given a character χ of GL ( k ), consider thegenuine indueced representation ˜ I ( s, χ ) of f SL ( k ), s ∈ C , given an irreducible generic representa-tion σ of G ( k ), Ginzburg ([Gi]) has contructed a local zeta integral which defines an elements inHom J ( σ, ˜ I ( s, χ ) ⊗ ω ψ ). Thus the above conjecture would imply the local functional equation forGinzburg’s local zeta integral in [Gi]. We can prove that dim Hom J ( σ, ˜ I ( s, χ ) ⊗ ω ψ ) ≤ non-supercuspidal representations σ , which could be viewed as a special case and an evidence of theabove conjecture. (cid:3) As preparations of our multiplicity results in next section, we record some useful facts on theWeil representations. From the above description of the projection map and modified versions ofEq.(2.1), we have the following formulas ω ψ (( r , , r , r , r )) φ ( ξ ) = ψ ( r ) φ ( ξ + r ) ,ω ψ ((0 , r , , , φ ( ξ ) = ψ ( − ξr ) φ ( ξ ) ,ω ψ ( h ( a, a − )) φ ( ξ ) = ǫ ( a ) φ ( aξ ) ,ω ψ ( x β ( b )) φ ( ξ ) = ψ ( bξ ) φ ( ξ ) ,ω ψ (cid:18)(cid:18) b − b − (cid:19)(cid:19) φ ( ξ ) = 1 γ ( b, ψ ) X x ∈ k φ ( x ) ψ ( − xbξ ) . (6.6)The space S ( k ) has a basis { δ s , s ∈ k } , where δ s ( t ) = δ s,t . From formulas (6.6), we have ω ψ ( x β ( b )) δ s = ψ ( bs ) δ s ,ω ψ ((0 , y, , , δ s = ψ ( − sy ) δ s ,ω ψ (( r , , r , r , r )) δ s = ψ ( r ) δ s − r . (6.7) Lemma 6.3.
For φ, φ ′ ∈ S ( k ) , we define a pair h φ, φ ′ i = X ξ ∈ k φ ( ξ ) φ ′ ( − ξ ) . Then we have h ω ψ ( j ) φ, ω ψ ( ι j ) φ ′ i = h φ, φ ′ i , ∀ j ∈ J, φ, φ ′ ∈ S ( k ) . For A ∈ End C ( S ( k )) , we define t A by h t Aφ, φ ′ i = h φ, Aφ ′ i . The operator t is an anti-involution on End C ( S ( k )) and satisfies t ω ψ ( j ) = ω ψ ( τ j ) .Proof. The proof is similar to that of Lemma 3.5, and thus is omitted. (cid:3)
Note that the above pair on S ( k ) satisfies the property(6.8) h δ a , δ b i = δ a, − b , a, b ∈ k. See [Ku], for example, for the dependence of these formulas on the symplectic form.
Transposes on
End( π ⊗ ω ψ ) for an irreducible representation π of SL ( k ) . Let π =1 , St , ω ± ψ , ω ± ψ κ , or ω ψ,µ , where µ is a character of E such that µ = 1. Then π is an irreducible rep-resentation of SL ( k ). We have defined a pair π × ι π → C in Section 3. Considering the representation σ π := π ⊗ ω ψ of J , we can then define a bilinear pair σ π × ι σ π → C by h v ⊗ φ , v ⊗ φ i = h v , v ih φ , φ i , v , v ∈ π, φ , φ ∈ ω ψ , where h φ , φ i is defined in Lemma 6.3. This pair satisfies the property h σ π ( j ) v, σ π ( ι j ) v ′ i = h v, v ′ i , v, v ′ ∈ σ π , j ∈ J. As in the Sp case, we define a transpose t : End C ( σ π ) → End C ( σ π ) by h t Av, v ′ i = h v, Av ′ i , A ∈ End C ( σ π ) , v, v ′ ∈ σ π . Then we have t σ π ( j ) = σ π ( t j ) , ∀ j ∈ J. By Proposition 2.1, the representation σ π | SL ( k ) is multiplicity free. Let σ π | SL ( k ) = ⊕ i V i be thedecomposition of irreducible representations of SL ( k ). As in Lemma 3.9, the idempotent id V i ∈ End C ( σ π ) is invariant under the transpose t defined above.7. Certain Multiplicity one theorems for G In this section, we continue to let G = G ( k ) and let J be the Fourier-Jacobi subgroup of G defined in Section 6.2. For an irreducible representation π of SL ( k ), as in Section 6.5, we write σ π = π ⊗ ω ψ . Theorem 7.1.
The representation
Ind GJ ( σ π ) is multiplicity free, if π = 1 , St , ω ψ,µ , ω ± ψ , ω ± ψ κ for anycharacter µ of E with µ = 1 . Remark 7.2. (1) If π = I ( χ ) for a character χ of k × with χ = 1, we can also show that for q large,the induced representation Ind GJ ( I ( χ ) ⊗ ω ψ ) is not multiplicity free as in Remark 4.2. In fact, let χ , χ be two characters of k × , we consider the induced representation I ( χ , χ ) = Ind GL ( k ) B GL2 ( χ ⊗ χ ) ofGL ( k ), where B GL is the upper triangular subgroup of GL ( k ), and χ ⊗ χ is viewed as a characterof B GL by χ ⊗ χ (cid:18)(cid:18) a ba (cid:19)(cid:19) = χ ( a ) χ ( a ) . Recall that P = M ⋉ V with M ∼ = GL ( k ). View I ( χ , χ ) as a representation of P by making V act trivially on it. Then we consider the induced representation Ind GP ( I ( χ ⊗ χ )) andHom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( I ( χ , χ ))) . By Frobenius reciprocity, we haveHom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( I ( χ , χ ))) = Hom J ( I ( χ ) ⊗ ω ψ , Ind GP ( I ( χ , χ )) | J ) . By Mackey’s Theory, see [Se, p.58], we haveInd GP ( I ( χ , χ )) | J = M s ∈ J \ G/P
Ind JP s ( I ( χ , χ s ) s ) , where P s = sP s − ∩ J and for a representation ρ of P , the representation ρ s of P s is defined by ρ s ( h ) = ρ ( s − hs ). Considering the element w = w α w β w α w − β w − α ∈ J \ G/P , we have P w ∼ =SL ( k ) ֒ → M and I ( χ , χ ) w = I ( χ , χ ) | SL ( k ) . ThusHom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( I ( χ , χ ))) ⊃ Hom J ( I ( χ ) ⊗ ω ψ , Ind JP w ( I ( χ , χ ) | SL ( k ) ))= Hom SL ( k ) ( I ( χ ) ⊗ ω ψ | SL ( k ) , I ( χ , χ ) | SL ( k ) ) . Note that I ( χ , χ ) | SL ( k ) = I ( χ χ − ). We take χ = ǫχχ . By Proposition 2.1, we havedim Hom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( I ( ǫχχ , χ ))) ≥ dim Hom SL ( k ) ( I ( χ ) ⊗ ω ψ | SL ( k ) , I ( ǫχ )) = 2 . From Mackey’s irreducibility criterion, see [Se, p.59], for χ , χ in “general positions”, the inducedrepresentation Ind GP ( I ( χ , χ )) is irreducible. Here χ , χ are called in general position, if ( χ ⊗ χ ) = NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 27 ( χ ⊗ χ ) w for all w ∈ W ( G ) − { } , where ( χ ⊗ χ ) is viewed as a character of the maximaltorus of G via ( χ ⊗ χ )( h ( a, b )) = χ ( a ) χ ( b ). One can check that, for q large, it is easy tofind χ such that ǫχχ , χ are in general positions. Then Ind GP ( I ( ǫχχ , χ )) is irreducible anddim Hom G (Ind GJ ( I ( χ ) ⊗ ω ψ ) , Ind GP ( I ( ǫχχ , χ ))) ≥
2. Thus Ind GJ ( I ( χ ) ⊗ ω ψ ) is not multiplicity free.(2) For an irreducible representation Π of G , it is in general false that dim Hom J (Π , I ( χ ) ⊗ ω ψ ) ≤ cuspidal , in [LZ], we are able to showthat the multiplicity one result dim Hom J (Π , I ( χ ) ⊗ ω ψ ) ≤ G . (cid:3) Before proving Theorem 7.1, we give a set of representatives of the double coset decomposition J \ G/J . Recall that w = w α w β w − α , w = w α w β w α w − β w − α . Lemma 7.3.
A set of representatives of J \ G/J is given by (cid:8) h ( a, , h ( a, a − ) w α , h ( − , a ) w , h ( a, w , a ∈ k × (cid:9) . Proof.
Considering the double coset decomposition P \ G/P , we have G = P w P ∪ P w P ∪ P w α P ∪ P. From the fact P = ∪ a ∈ k × h ( a, J = ∪ b ∈ k × Jh (1 , b ), we can get the statement of the lemma. Notethat h ( a, /a ) ∈ J . Thus h ( a, b ) w , h ( ab, w and h (1 , ab ) w are in the same double coset Jh ( a, b ) wJ for any Weyl element w . We take h ( a, a − ) w α and h ( − , a ) w as representatives because we have τ ( h ( a, a − ) w α ) = h ( a, a − ) w α and τ ( h ( − , a ) w ) = h ( − , a ) w . (cid:3) Before we start the proof of Theorem 7.1, we show one more lemma as follows. Let ( ρ, V ) bean irreducible representation of SL ( k ) and a ∈ k × . Let ( ρ a , V a ) be the representation of SL ( k )defined by V a = V, ρ a ( g ) = ρ ( g a ), where g a = diag( a, g diag( a − , ρ, V ), we fix anon-trivial pair h , i on V such that h ρ ( g ) v, ρ ( ι g ) v ′ i = h v, v ′ i , ∀ v, v ′ ∈ V, g ∈ SL ( k ) . Note that defining such a pair is equivalent to defining an isomorphism ι ρ ∼ = ˜ ρ , and thus such a pairis unique up to a scalar. Lemma 7.4. If Hom SL ( k ) ( ρ, ρ a ) = 0 , there exists a unique nonzero λ a ∈ Hom SL ( k ) ( ρ, ρ a ) suchthat h λ a v , λ a v i = h v , v i , ∀ v , v ∈ V, where λ a v is viewed as an element in V under the identification V = V a for v ∈ V .Moreover, for the unique λ a defined above, let d aV be the constant such that ρ (diag( a − , a )) ◦ λ a ◦ λ a = d aV id V , then d aV = 1 . Here, the middle λ a is viewed as an element of Hom SL ( k ) ( ρ a , ρ a ) .Proof. We first fix any nonzero λ a ∈ Hom SL ( k ) ( ρ, ρ a ) and consider the pair h , i a on V defined by h v , v i a := h λ a v , λ a v i . For g ∈ SL ( k ), one can check that h ρ ( g ) v , ρ ( ι g ) v i a = h λ a ρ ( g ) v , λ a ρ ( ι g ) v i = h ρ ( g a )( λ a v ) , ρ (( ι g ) a )( λ a v ) i = h ρ ( g a )( λ a v ) , ρ ( ι ( g a ))( λ a v ) i = h λ a v , λ a v i = h v , v i a , where we used the relation ι ( g a ) = ( ι g ) a . Thus by the uniqueness of the pair, we have that thereexists a constant c a ∈ C × such that h v , v i a = c a h v , v i . Then λ a := √ c a λ a ∈ Hom SL ( k ) ( ρ, ρ a )satisfies the property h λ a v , λ a v i = h v , v i , ∀ v , v ∈ V, where √ c a is a square root of c a . The uniqueness of such λ a follows from the fact thatdim Hom SL ( k ) ( ρ, ρ a ) = 1 . The “moreover” part seems very delicate and we don’t have a uniform proof at this moment. Wewill check it case by case.We first consider that ( ρ, V ) = ω ψ,µ for a quadratic character µ of E with µ = 1. For a ∈ F × ,we fix x a ∈ E × with Nm( x a ) = x q +1 a = a . Recall that an element f ∈ V is a function on E × suchthat f ( uy ) = µ − ( u ) f ( y ) , ∀ u ∈ E , y ∈ E × . For f ∈ V , we consider λ a f ( ξ ) = q µ − ( x q − a ) f ( x a ξ ) . Then we can check that h λ a f , λ a f i = h f , f i , where h f , f i is the pair defined in § λ a ◦ λ a f ( ξ ) = µ − ( x q − a ) f ( x a ξ )= µ − ( x q − a ) µ − ( x a a − ) f ( aξ )= µ − ( x q +1 a a − ) f ( aξ )= f ( aξ )= ρ (diag( a, a − )) f ( ξ ) . Thus d aV = 1 in this case.Next, we consider the case when ( ρ, V ) = I ( χ ) for a character χ of k × . If χ is not quadratic, wehaven’t constructed a h , i on I ( χ ) in previous sections. We first define a pair below. Consider theintertwining operator ∆ : I ( χ ) → I ( χ − ) defined by∆ ∗ f ( g ) = q − X x ∈ k f (cid:18) w (cid:18) x (cid:19) g (cid:19) , ∀ f ∈ I ( χ ) , where w = (cid:18) − (cid:19) . One can check that ∆ ∗ f ∈ I ( χ − ). For f , f ∈ I ( χ ), we define h f , f i = X g ∈ B SL2( k ) \ SL ( k ) ∆ ∗ f ( g ) f ( ι g ) . Then, one can check that h ρ ( g ) f , ρ ( ι g ) f i = h f , f i , ∀ f , f ∈ I ( χ ) , g ∈ SL ( k ) . For a ∈ k × , f ∈ I ( χ ), we define λ a f ( g ) = p χ ( a ) f ( g a − ) . Then λ a ∈ Hom SL ( k ) ( V, V a ), and∆ ∗ λ a f ( g ) = χ ( a − )∆ ∗ f ( g a − ) . It follows that h λ a f , λ a f i = h f , f i . On the other hand, we have λ a ◦ λ a f ( g ) = χ ( a ) f ( g a − )= χ ( a ) f (diag( a − , a ) g diag( a, a − ))= f ( g diag( a, a − ))= ρ (diag( a, a − )) f ( g ) . Thus we get d aV = 1.We omit the cases when ρ = 1 , St , and just remark that the proofs in these cases are similar. (cid:3) In the following, for a ∈ k × , we write η ( a ) = h ( a, a − ) w α , and ξ ( a ) = h ( − , a ) w , for simplicity. Proof of Theorem . . We have defined an anti-involution τ on G by τ g = h (1 , − g − h (1 , − t on End( σ π ) such that t ( σ π ( j )) = σ π ( τ j ). We define an anti-involution τ on A ( G, J, σ π ) by( τ K )( g ) = t ( K ( τ g )) , ∀ g ∈ G, K ∈ A ( G, J, σ π ) . By Corollary 3.4 and Lemma 7.3, it suffices to show that for all K ∈ A ( G, J, ω ψ ) with τ K = − K , K ( g ) = 0 for all g = h ( a, , η ( a ) , ξ ( a ) , h ( a, w , ∀ a ∈ k × . We now fix a K ∈ A ( G, J, ω ψ ) with τ K = − K . NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 29
Step (1), K ( h ( a, h ( a, x α + β ( z ) = x α + β ( az ) h ( a, , see Eq.(6.1). Thus we get ψ ( z ) K ( h ( a, ψ ( az ) K ( h ( a, . If a = 1, we can take z such that ψ ( z ) = ψ ( az ). Then we get K ( h ( a, a = 1. If a = 1, wehave h (1 , g = gh (1 ,
1) for all g ∈ J . Thus we get K ( h (1 , Cσ π (1) for some C ∈ C by Schur’sLemma. We can get C = 0 from the condition τ K = − K. Step (2), K ( η ( a )) = 0 . From Eqs.(6.1)-(6.4), we can check the relations η ( a )(0 , x, y, , ,
0) = (0 , − ay, x/a, , xy ) η ( a ) , ∀ x, y ∈ k,η ( a ) x β ( b ) = (0 , , , a − b, η ( a ) , ∀ b ∈ k, x β ( x ) η ( a ) = η ( a )(0 , , , − x/a , , ∀ x ∈ k. Thus, we have that K ( η ( a )) σ π ((0 , x, y, , , σ π ((0 , − ay, a − x, , xy )) K ( η ( a )) , ∀ x, y ∈ k, (7.1) K ( η ( a )) σ π ( x β ( b )) = K ( η ( a )) , ∀ b ∈ k, (7.2) σ π ( x β ( x )) K ( η ( a )) = K ( η ( a )) , ∀ x ∈ k. (7.3)We now consider different π separately.Case (2.1), π = 1. Put y = 0 in Eq.(7.1) and apply Eq.(6.7), we can get ψ ( − sx ) K ( η ( a )) δ s = ψ ( a − x ) K ( η ( a )) δ s . We then get K ( η ( a )) δ s = 0 if s = − / (2 a ). Applying Eq.(7.2) and Eq.(6.7), we get ψ ( bs ) K ( η ( a )) δ s = K ( η ( a )) δ s . Thus if s = 0, we get K ( η ( a )) δ s = 0. Since − / (2 a ) = 0, we have K ( η ( a )) δ s = 0 for all s ∈ k .Case (2.2), π = ω ψ,µ . Recall that ω ψ,µ has a basis { f b , b ∈ k × } , where f a ( x b ) = δ a,b , see Section3.7. We can check that σ π ( x β ( b )) f r ⊗ δ s = ψ ( b ( r + s )) f r ⊗ δ s . Plugging y = 0 into Eq.(7.1), wecan get ψ ( − xs ) K ( η ( a )) f r ⊗ δ s = ψ ( a − x ) K ( η ( a )) f r ⊗ δ s . Thus we get K ( η ( a )) f r ⊗ δ s = 0 if s = − / (2 a ). By Eq.(7.2), we can get ψ ( b ( r + s )) K ( η ( a )) f r ⊗ δ s = K ( η ( a )) f r ⊗ δ s . Thus we get K ( h ( a, w α ) f r ⊗ δ s = 0 if r = − s . We assume that K ( h ( a, w α ) f − / (4 a ) ⊗ δ − / (2 a ) = X b ∈ k × ,t ∈ k C ( b, t ) f b ⊗ δ t for C ( b, t ) ∈ C . Applying Eq.(7.1) when x = 0, we can get X b,t ψ ( y ) C ( b, t ) f b ⊗ δ t = X b,t ψ (2 ayt ) C ( b, t ) f b ⊗ δ t . Thus we get C ( b, t ) = 0 if t = 1 / (2 a ). Applying Eq.(7.3), we can get C ( b, / (2 a )) = 0 if b = − / (4 a ).We denote D ( a ) = C ( − / (4 a ) , / (2 a )). To summarize, we have that K ( η ( a )) f r ⊗ δ s = 0 , if s = − / (2 a ) , or r = − / (4 a ) ,K ( η ( a )) f − / (4 a ) ⊗ δ − / (2 a ) = D ( a ) f − / (4 a ) ⊗ δ / (2 a ) . Note that τ K ( η ( a )) = t K ( η ( a )) since τ η ( a ) = η ( a ). Since τ K + K = 0 and the pair h , i on σ π issymmetric, we get h K ( η ( a )) f − / (4 a ) ⊗ δ − / (2 a ) , f − / (4 a ) ⊗ δ − / (2 a ) i = 0. By Eq.(3.7) and Eq.(6.8),we have h K ( η ( a )) f − / (4 a ) ⊗ δ − / (2 a ) , f − / (4 a ) ⊗ δ − / (2 a ) i = D ( a ) µ − ( x q − − / (4 a ) ) . Thus we get D ( a ) = 0, which implies that K ( η ( a )) = 0 . Case (2.3), π = ω + ψ u for u = 1 , κ . Recall that ω + u has a basis { ∆ , ∆ x , x ∈ A } , where A ⊂ k × is still a set of representatives of k × / {± } , see Section 3.6. We can check that σ π ( x β ( b ))∆ r ⊗ δ s = ψ ( b ( ur + s ))∆ r ⊗ δ s , see Eq.(4.7) and Eq.(6.6). Using Eq.(7.1) when y = 0, we can get K ( η ( a ))∆ r ⊗ δ s = 0 if s = − / (2 a ) as above. Applying Eq.(7.2), we can get K ( η ( a ))∆ r ⊗ δ s = 0 if ur + s = 0. Note that there is at most one r ∈ A ∪ { } such that ur + 1 / (4 a ) = 0. If there is no such r , we are done. Now assume that there exists r with ur + 1 / (4 a ) = 0. We assume that K ( η ( a ))∆ r ⊗ δ − / (2 a ) = P b,t C ( b, t )∆ b ⊗ δ t . Applying Eq.(7.1) when x = 0 we get X b,t ψ ( y ) C ( b, t )∆ b ⊗ δ t = X b,t ψ (2 ayt ) C ( b, t )∆ b ⊗ δ t , ∀ y ∈ k. By choosing appropriate y we can see C ( b, t ) = 0 if t = 1 / (2 a ). Applying Eq.(7.3) to the equation K ( η ( a ))∆ r ⊗ δ − / (2 a ) = P b C ( b, / (2 a ))∆ b ⊗ δ / (2 a ) , we can obtain that X b C ( b, / (2 a )) ψ ( x ( ub + 1 / (4 a )))∆ b ⊗ δ / a = X b C ( b, / (2 a ))∆ b ⊗ δ / (2 a ) , ∀ x ∈ k, which implies that C ( b, / (2 a )) = 0 if b = r . Thus we have K ( η ( a ))∆ r ⊗ δ − / (2 a ) = C ( r , / (2 a ))∆ r ⊗ δ / (2 a ) . From the condition τ K + K = 0 and the symmetry of the pair h , i on σ π , we have h K ( η ( a ))∆ r ⊗ δ − / (2 a ) , ∆ r ⊗ δ − / (2 a ) i = 0 . By Eq.(3.5), Eq.(4.9) and the above discussion, the above equation implies that C ( r , / (2 a )) = 0.Thus K ( h ( a, w α ) = 0 . Case (2.4), π = ω − ψ u . The proof is similar to that the Case (2.3) and thus omitted.Case (2.5), π = St. Recall that St has a basis { F r , r ∈ k } , see Section 3.5. By Eqs.(4.5, 6.7, 7.2),we have ψ ( bs ) K ( η ( a )) F r − b ⊗ δ s = K ( η ( a )) F r ⊗ δ s . In particular, we have K ( η ( a )) F r ⊗ δ s = ψ ( rs ) K ( η ( a )) F ⊗ δ s . Plugging y = 0 into Eq.(7.1) andapplying it to F ⊗ δ s , we get ψ ( − sx ) K ( η ( a )) F ⊗ δ s = ψ ( a − x ) K ( η ( a )) F ⊗ δ s using Eq.(4.5) andEq.(6.7). By choosing appropriate x , we see that K ( h ( a, w α ) F ⊗ δ s = 0 if s = − / (2 a ). Denote a = − / (2 a ).We assume that K ( η ( a )) F ⊗ δ a = P b,t ∈ k C s ( b, t ) F b ⊗ δ t . Plugging x = 0 into Eq.(7.1) andapplying it to F ⊗ δ a , we can get P b,t ψ ( y ) C s ( b, t ) F b ⊗ δ t = P b,t C s ( b, t ) ψ (2 ayt ) F b ⊗ δ t , ∀ y ∈ k. By choosing appropriate y , we have C s ( b, t ) = 0 if t = − a . Thus we get K ( η ( a )) F ⊗ δ a = P b ∈ k D b F b ⊗ δ − a , where D b = C a ( b, − a ).Note that τ ( η ( a )) = η ( a ) , thus the condition τ K + K = 0 implies that h K ( η ( a )) F r ⊗ δ a , F ⊗ δ a i + h F r ⊗ δ a , K ( η ( a ) F ⊗ δ a ) i = 0 , ∀ r ∈ k. In particular, we have h K ( η ( a )) F ⊗ δ a , F ⊗ δ a i = 0 and thus, h K ( η ( a )) F r ⊗ δ a , F ⊗ δ a i = ψ ( ra ) h K ( η ( a )) F ⊗ δ a , F ⊗ δ a i = 0 . The above two equations then imply that h F r ⊗ δ a , K ( η ( a ) F ⊗ δ a ) i = 0 , ∀ r ∈ k. By Eq.(3.4) and Eq.(6.8), h F r ⊗ δ a , K ( η ( a ) F ⊗ δ a ) i = D − r + ( q − P b ∈ k D b . Thus we get D − r + ( q − P b ∈ k D b = 0 , ∀ r ∈ k. A direct calculation shows that D b = 0 , ∀ b ∈ k . Hence, K ( η ( a )) = 0 . This completes the proof of Step (2).Step (3), K ( ξ ( a )) = 0. Recall that ξ ( a ) = h ( − , a ) w and w = w α w β w − α . We can check therelations ξ ( a ) x β ( y ) = (0 , , , , − ay ) ξ ( a ) ,ξ ( a )(0 , , , , − ay ) = x β ( y ) ξ ( a ) ,ξ ( a ) x α + β ( x ) = x α + β ( − x ) ξ ( a ) . Thus we have K ( ξ ( a )) σ π ( x β ( y )) = K ( ξ ( a )) , ∀ y ∈ k, (7.4) K ( ξ ( a )) = σ π ( x β ( y )) K ( ξ ( a )) , ∀ y ∈ k, (7.5) K ( ξ ( a )) σ π ( x α + β ( x )) = σ π ( x α + β ( − x )) K ( ξ ( a )) , ∀ x ∈ k. (7.6)In the following, we still consider different π separately.Case (3.1), π = 1 . Applying Eq.(7.4) to δ s , we get ψ ( ys ) K ( ξ ( a )) δ s = K ( ξ ( a )) δ s , ∀ y ∈ k. NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 31
For s = 0, we can take y such that ψ ( ys ) = 1. Thus we have K ( ξ ( a )) δ s = 0 , if s = 0 . Suppose that K ( ξ ( a )) δ = P s ∈ k C ( s ) δ s , where C ( s ) ∈ C . Applying Eq.(7.5) to δ , we have X s ∈ k C ( s ) δ s = K ( ξ ( a )) δ = σ π ( x β ( y )) K ( ξ ( a )) δ = X s ∈ k ψ ( ys ) C ( s ) δ s , ∀ y ∈ k. If s = 0, take y = 0 such that ψ ( ys = 0). By comparing coefficients, we have C ( s ) = 0 unless s = 0.Therefore, K ( ξ ( a )) δ s = 0 , s = 0; K ( ξ ( a )) δ = C (0) δ . From the relation τ ( ξ ( a )) = ξ ( a ), τ K + K = 0 and the symmetry of the pair h i on σ π , we get h K ( ξ ( a )) δ , δ i = 0. By Eq.(6.8), we have h K ( ξ ( a )) δ , δ i = h C (0) δ , δ i = C (0) . It follows that C (0) = 0 and K ( η ( a )) = 0.Case (3.2), π = ω ψ,µ . Applying Eq.(7.4) to f r ⊗ δ s , we have ψ ( y ( r + s )) K ( ξ ( a )) f r ⊗ δ s = K ( ξ ( a )) f r ⊗ δ s . This implies that K ( ξ ( a )) f r ⊗ δ s = 0 if r + s = 0. Suppose that K ( ξ ( a )) f − s ⊗ δ s = P b,t C s ( b, t ) f b ⊗ δ t . Applying Eq.(7.6) to f − s ⊗ δ s , we have X b,t ψ ( − sx ) C s ( b, t ) f b ⊗ δ t = X b,t ψ (2 tx ) C s ( b, t ) f b ⊗ δ t , ∀ x ∈ k. An appropriate choice of x ∈ k implies that C s ( b, t ) = 0 if t = − s . We get K ( ξ ( a )) f − s ⊗ δ s = P b C s ( b, − s ) f b ⊗ δ − s . Applying Eq.(7.5) to f − s ⊗ δ s , we have X b C s ( b, − s ) f b ⊗ δ − s = X b C s ( b, − s ) ψ ( y ( b + s )) f b ⊗ δ − s , ∀ y ∈ k, which implies that C s ( b, − s ) = 0 unless b = − s . Thus we get K ( ξ ( a )) f − s ⊗ δ s = D ( s ) f − s ⊗ δ − s ,where D ( s ) = C s ( − s , − s ). The condition τ K + K = 0 implies that h f − s ⊗ δ s , K ( ξ ( a )) f − s ⊗ δ s i = 0 . By Eqs.(3.7) and (6.8), the above equation implies that D ( s ) = 0. Hence, we have K ( ξ ( a )) = 0 . Case (3.3), π = ω ± ψ u for u = 1 , κ . The proof is similar to that of Case (3.2) and thus omitted.Case (3.4), π = St. As in the proof of case (2.5), an application of Eq.(7.4) to F r ⊗ δ s shows that K ( ξ ( a )) F r ⊗ δ s = ψ ( rs ) K ( ξ ( a )) F ⊗ δ s . Suppose that K ( ξ ( a )) F ⊗ δ s = P b,t C s ( b, t ) F b ⊗ δ t for C s ( b, t ) ∈ C . Applying Eq.(7.6) to F ⊗ δ s and using Eqs.(4.5), (6.7), we get X b,t ψ ( − xs ) C s ( b, t ) F b ⊗ δ t = X b,t C s ( b, t ) ψ (2 xt ) F b ⊗ δ t , ∀ x ∈ k. If t = − s , we can choose x ∈ k such that ψ ( − xs ) = ψ (2 xt ). Consequently, C s ( b, t ) = 0 unless t = − s , and K ( ξ ( a )) F ⊗ δ s = P b ∈ k C s ( b ) F b ⊗ δ − s , where C s ( b ) = C s ( b, − s ). An application ofEq.(7.5) to F ⊗ δ s shows that X b ∈ k C s ( b ) F b ⊗ δ − s = X b ∈ k C s ( b ) ψ ( ys ) F b − y ⊗ δ − s , ∀ y ∈ k. By comparing the coefficients of both sides of the above identity, we get C s ( b ) = ψ ( ys ) C s ( b + y ) . In particular, we have C s ( b ) = ψ ( − bs ) C s (0). Thus K ( ξ ( a )) F ⊗ δ s = C s (0) P b ∈ k ψ ( − bs ) F b ⊗ δ − s . As usual, the condition τ K + K = 0 implies that h K ( ξ ( a )) F ⊗ δ s , F ⊗ δ s i = 0. By Eq.(3.4) andEq.(6.8), we get C s (0) ( q − X b ∈ k ψ ( − bs ) + ψ (0) ! = h K ( ξ ( a )) F ⊗ δ s , F ⊗ δ s i = 0 . Because P b ∈ k ψ ( − bs ) = 0 if s = 0, and P b ∈ k ψ ( − bs ) = q if s = 0, we have C s (0) = 0. Hence K ( ξ ( a )) F ⊗ δ s = 0 and K ( ξ ( a )) F r ⊗ δ s = ψ ( rs ) K ( ξ ( a )) F ⊗ δ s = 0. This shows K ( ξ ( a )) = 0 andcompletes the proof of Step (3).Step (4), K ( h ( a, w ) = 0. We have h ( a, w g = g a h ( a, w , where g a = (cid:18) x ax a − x x (cid:19) , for g = (cid:18) x x x x (cid:19) ∈ SL ( k ) . Thus, K ( h ( a, w ) σ π ( g ) = σ aπ ( g ) K ( h ( a, w ) , ∀ g ∈ SL ( k ) , where σ aπ ( g ) = σ π ( g a ) . The above equation implies that K ( h ( a, w ) ∈ Hom SL ( k ) ( σ π , σ aπ ) . We first assume that a / ∈ k × , .If π = 1, we have σ π | SL = ω + ψ ⊕ ω − ψ and σ aπ | SL ( k ) ∼ = ω + ψ κ ⊕ ω − ψ κ . We haveHom SL ( k ) ( σ π , σ aπ ) = 0 , and thus K ( h ( a, w ) = 0.If π = ω ψ,µ , ω ± ψ u for u = 1 , κ , then Hom SL ( k ) ( σ π , σ aπ ) = 0. In fact, from the decompositiongiven in Proposition 2.1, we can write σ π | SL ( k ) = L i ∈ I V i , where I is an index set and each V i is an irreducible representation of SL ( k ) and occurred with multiplicity one. If V i is one of1 , I ( χ ) , ω ψ,µ , St, then V i ∼ = V ai . By Lemma 7.4, we fix an isomorphism λ aV i ∈ Hom SL ( k ) ( V i , V ai ) suchthat h λ aV i v , λ aV i v i = h v , v i , ∀ v , v ∈ V . If V i ∼ = ω ± ψ (resp. ω ± ψ κ ), then V ai ∼ = ω ± ψ κ (resp. ω ± ψ ). From the decomposition of σ π | SL ( k ) , one seesthat Hom SL ( k ) ( σ π , σ aπ ) = M i ∈ I Hom( V i , V ai ) , where I is the subset of I such that V i is 1 , St , I ( χ ) or ω ψ,µ for i ∈ I . For example, when π = ω ψ,µ and ǫ = 1, there are factors ω + ψ κ ⊕ ω − ψ κ in the decomposition σ π | SL ( k ) , but these factors are notindexed by I , in fact, one has Hom SL ( k ) ( ω + ψ κ ⊕ ω − ψ κ , ( ω + ψ κ ⊕ ω − ψ κ ) a ) = 0 . Hence, there are constants C i ∈ C such that K ( h ( a, w ) = X i ∈ I C i λ aV i . Note that τ ( h ( a, w ) = h (1 , a ) w = h ( a − , a ) h ( a, w . Thus K ( h (1 , a ) w ) = X i C i σ π ( h ( a − , a )) λ aV i , and the condition K = − τ K implies that K ( h ( a, w ) = − ( τ K )( h ( a, w ) = − t K ( h (1 , a ) w ). Fromthe definition of the transpose operator, we get h v , K ( h ( a, w ) v i + h K ( h (1 , a ) w ) v , v i = 0 , ∀ v , v ∈ π ⊗ ω ψ . In particular, if we choose v , v ∈ V i for a fixed i ∈ I , we have C i (cid:0) h v , λ aV i v i + h σ π ( h ( a − , a )) λ aV i v , v i (cid:1) = 0 , ∀ v , v ∈ V i . If we replace v by λ aV i v , we get C i (cid:0) h λ aV i v , λ aV i v i + h σ π ( h ( a − , a )) λ aV i λ aV i v , v i (cid:1) = 0 , ∀ v , v ∈ V i . By Lemma 7.4, we have σ π ( h ( a − , a )) λ aV i ◦ λ aV i = id V i . We then get C i (cid:0) h λ aV i v , λ aV i v i + h v , v i (cid:1) = 0 , ∀ v , v ∈ V i , i.e., 2 C i h v , v i = 0 , ∀ v , v ∈ V i . This implies that C i = 0 for all i and thus K ( h ( a, w ) = 0. NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 33
Next, we assume that a / ∈ k × , and π = St. Without loss of generality, we assume that a = κ − .The proof is almost identical to the above case, but for completeness we still provide the detailshere. Note that we have that σ π | SL ( k ) ∼ = σ aπ | SL ( k ) . We consider the decomposion σ π | SL ( k ) given inProposition 2.1:St ⊗ ω ψ | SL ( k ) = St M M χ ∈ A I ( χ ) M M µ ∈ B ω ψ,µ M ω + ψ M ω + ψ κ . Let V be a summand in the above decomposition. If V = St , I ( χ ) or ω ψ,µ , we have V ∼ = V a . Inthese cases, by Lemma 7.4, we fix a nonzero λ aV ∈ Hom SL ( k ) ( V, V a ) such that h λ aV v , λ aV v i = h v , v i , ∀ v , v ∈ V. If V = ω + ψ , then V a ∼ = ω + ψ κ ; and if V = ω + ψ κ , then V a ∼ = ω + ψ . In fact, a direct calculation showsthat ( ω ψ κ ) a is exactly ω ψ since we assumed a = κ − (if a = a κ − for some a ∈ k × , the tworepresentations ( ω ψ κ ) a and ω ψ would differ by an inner automorphism). Let λ aω + ψ ∈ Hom( ω + ψ , ( ω + ψ κ ) a )and λ aω + ψκ ∈ Hom SL ( k ) ( ω + ψ κ , ( ω + ψ ) a ) be the identity maps.From the decomposition of St ⊗ ω ψ | SL ( k ) , we haveHom SL ( k ) ( σ π , σ aπ ) = M χ ∈ A Hom SL ( k ) ( I ( χ ) , I ( χ ) a ) M M µ ∈ B Hom SL ( k ) ( ω ψ,µ , ω aψ,µ ) M Hom SL ( k ) (St , St a ) M Hom SL ( k ) ( ω + ψ , ( ω + ψ κ ) a ) M Hom SL ( k ) ( ω + ψ κ , ( ω + ψ ) a ) . By Schur’s Lemma, each Hom space on the right hand side of the above equation has dimension 1and is generated by λ aV for the corresponding V in the decomposition of St ⊗ ω ψ | SL ( k ) . Thus wecan write K ( h ( a, w ) = X V C V λ aV , for some constants C V ∈ C .Since τ ( h ( a, w ) = h (1 , a ) w = h ( a − , a ) h ( a, w and h ( a − , a ) ∈ J , we have that K ( h (1 , a ) w ) = σ π ( h ( a − , a )) K ( h ( a, w ) . Let λ a − V = σ π ( h ( a − , a )) λ aV , which is an isomorphism in Hom SL ( k ) ( V, V a − ) if V = St , I ( χ ) or ω ψ,µ ,and is an isomorphism in Hom SL ( k ) ( ω + ψ , ( ω + ψ κ ) a − ) (resp. Hom SL ( k ) ( ω + ψ κ , ( ω + ψ ) a − )) if V = ω + ψ (resp. ω + ψ κ ). Then K ( h (1 , a ) w ) = X V C V λ a − V . Since K is τ -skew-invariant, K ( h ( a, w ) = − ( τ K )( h ( a, w ) = − t K ( h (1 , a ) w ) . By the definition of the transpose, h v , K ( h ( a, w ) v i + h K ( h (1 , a ) w ) v , v i = 0 , ∀ v , v ∈ St ⊗ ω ψ . In particular, choosing v , v ∈ V , where V is still a component of σ π | SL ( k ) , we have that(7.7) C V (cid:16) h v , λ aV v i + h λ a − V v , v i (cid:17) = 0 , ∀ v , v ∈ V. As in the previous case, one can show that C V = 0 for each V = ω + ψ or ω + ψ κ . If V = ω + ψ or ω + ψ κ , then λ aV is simply the identity map and λ a − V = σ π ( h ( a − , a )). Taking v = ∆ s = v for some s ∈ k × inEq.(7.7), then λ a − V ∆ s = ǫ ( κ )∆ κ − s = − ∆ κ − s . Thus we have h v , λ aV v i = 2 and h λ a − V v , v i = 0.Then Eq.(7.7) implies that C V = 0. Thus K ( h ( a, w ) = 0.At last, we assume that a = a , a ∈ k × . Since h ( a, w = h ( a , a − ) h ( a , a ) w and h ( a , a − ) ∈ J , we have K ( h ( a, w ) = σ π ( h ( a , a − )) K ( h ( a , a ) w ) . Thus it suffices to show K ( h ( a , a ) w ) = 0 . Note that we have the relation h ( a , a ) w g = gh ( a , a ) w , ∀ g ∈ SL ( k ) , which implies that K ( h ( a , a ) w ) σ π ( g ) = σ π ( g ) K ( h ( a , a ) w ) , ∀ g ∈ SL ( k ) . Thus K ( h ( a , a ) w ) ∈ Hom SL ( k ) ( σ π , σ π ). Let σ π | SL ( k ) = ⊕ V i be the decomposition as in Propo-sition 2.1. We have Hom SL ( k ) ( σ π , σ π ) = ⊕ End SL ( k ) ( V i ). Thus K ( h ( a , a ) w ) = X C i id V i . Note that τ ( h ( a , a ) w ) = h ( a , a ) w , and t id V i = id V i by the discussion in Section 6.5. Thus( τ K )( h ( a , a ) w ) = t K ( h ( a , a ) w ) = X i C it id V i = K ( h ( a , a ) w ) . The condition τ K = − K implies that K ( h ( a , a ) w ) = 0 . This completes the proof of Theorem7.1. (cid:3)
References [AGRS] A. Aizenbud, D. Gourevitch, S. Rallis and G. Schiffmann,
Multiplicity one theorems,
Ann. Math. ,(2010), 1407-1434[BR] E.M Baruch, S. Rallis,
A uniqueness theorem of Fourier Jacobi models for representations of
Sp(4) , J.London Math. Soc. (2000), 183-197.[Bu] D. Bump, Automorphic forms and representations,
Cambridge Studies in Advanced Mathematics, , Cam-bridge University Press, Cambridge (1997)[Ca] J. Campbell, , The irreducible characters of × unitary matrix groups over finite fields , Master Thesis,University of Alberta, 2014.[CR] B. Chang and R. Ree, The characters of G ( q ), Symposia Mathematica XIII, In stituto Nazionale di AltaMathematica, 1974, 395-413.[En] H. Enomoto, The characters of the finite Chevalley group G ( q ) , q = 3 f , Japan J. Math. (1976), 191-248.[EY] H. Enomoto, H. Yamada, The characters of G (2 n ), Japan J. Math. , 1986, 325-377.[FH] W. Fulton and J. Harris, Representation theory, a first course,
Graduate Texts in Mathematics, ,Springer-Verlag, 1991.[GGP1] W.T. Gan, B.H. Gross, D. Prasad,
Symplectic local root numbers, central critical L-values and restrictionproblems in the representation theory of classical groups , Ast´erisque (2012), 1-109.[GGP2] W. T. Gan, B. Gross, D. Prasad.
Restrictions of representations of classical groups: Examples.
Ast´erisque (2012), 111-170.[Ge] P. G´erardin,
Weil representations associated to finite fields,
J. Algebra , (1977), 54-101[Gi] D. Ginzburg, On the standard L -function for G , Duke Mathematical Journal, , (1993), 315-333.[GPSR] D. Ginzburg, I. Piatetski-Shapiro, S. Rallis, L-functions for the orthogonal group , Mem. Amer. Math. Soc. (611) (1997) viii+218.[GJRS] D. Ginzburg, D. Jiang, S. Rallis, D. Soudry,
L-functions for symplectic groups using FourierJacobi models, in:Arithmetic Geometry and Automorphic Forms, in: Adv. Lect. Math. (ALM), vol. , Int. Press, Somerville,MA, 2011, pp. 18320[GP] B. Gross and D. Prasad, On the decomposition of a representation of SO n when restricted to SO n − , Canad.J. Math., (1992), pp. 974-1002. MR1186476[GH] S. Gurevich, R. Howe, Small representations of finite classical groups , pp.209-234 in “
Representation Theory,Number Theory, and Invariant Theory: In Honor of Roger Howe on the occasion of his 70th Birthday ”,edited by Jim Cogdell, Ju-Lee Kim and Chen-bo Zhu, Progress in Math, , Birkh¨auser, 2017.[Ha] T. Hagedorn,
Multiplicities in Restricted Representations of GL n ( F q ) , U n ( F q ) , SO n ( F q ) , Ph.D. thesis,Harvard University (1994).[HZ] G. Hiss, A. Zalesski,
The Weil-Steinberg character of finite classical groups , with an appendix by O. Brunat,Representation theory, (2009), 427-459.[JZ] D. Jiang, L. Zhang, A Product of Tensor Product L-functions of Quasi-splits Classical Groups of HermitianType , Geom. Funct. Anal. (2014), no. 2, 552-609.[Ka] E. Kaplan, Complementary results on the Rankin-Selberg gamma factors of classical groups , Journal ofNumber Theory, (2015), 390-447.[Ku] S. Kudla,
Notes on the local theta correspondence
Degenerate principal series representations for U( n, n ), Israel Journal of Math, (1997), 253-306.[LZ] B. Liu, Q. Zhang, On a converse theorem for G over finite fields , preprint. NIQUENESS OF CERTAIN FOURIER-JACOBI MODELS 35 [LS] Y. Liu, B. Sun,
Uniqueness of Fourier-Jacobi models: The Archimedean case,
Journal of Functional Analysis, (2013), 3325-3344.[MVW] C. Moeglin, M-F. Vign´eras, J-L Waldspurger,
Correspondances de Howe sur un corps p-adique , SpringerLecture Notes in Mathematics, , Springer, 1987.[Ni] C. Nien,
A proof of the finite field analogue of Jacquets conjecture , American Journal of Mathematics, (2014), pp.653-674[Pr] A. Prasad,
Representations of GL ( F q ) and SL ( F q ) , and some remarks about GL n ( F q ) , arXiv: 0712.4051.[Re] R. Ree, A Family of Simple Groups Associated with the Simple Lie Algebra of Type G , American Journalof Math. , (1961), 432-462.[R] M. Reeder, On the restriction of Deligne-Lusztig characters , J. Amer. Math. Soc. (2007), 573-602.[Ro] E-A, Roditty, On Gamma factors and Bessel functions for representations of general linear groups overfinite fields , M.s.c. Thesis, Tel-Aviv University.[Se] J-P., Serre,
Linear representations of finite groups , Graduate Texts in Mathematics, Vol , Springer-Verlag,1971.[Sr] B. Srinivasan, The characters of the finite symplectic group
Sp(4 , q ) , Transaction of the AMS, (1968),488-525.[St] R. Steiberg,
Lectures on Chevalley groups , Yale University, 1967.[Su] B. Sun,
Multiplicity one theorems for Fourier-Jacobi models,
American Journal of Mathematics (2012),1655-1678.[SZ] B. Sun, C. Zhu,
Multiplicity one theorems: the Archimedean case,
Ann. Math. (2012), 23-44.[T] M. Teicher,
Multiplicity one theorems for
GSp( k, n ) and O( k, n ) where k is a finite field , Journal of Algebra (1987), 436-465.[Th] E. Thoma, Die Einschr¨ankung der Charaktere von
GL( n, q ) auf GL( n − l, q ) , Math. Z., (1971), pp.321-338. MR0288190[W] A. Weil,
Sur certains groupes d’op´erateurs unitaires , Acta Math. (1964), 143-211.[Za] A. Zalesski,
On the Steinberg character of an orthogonal group over a finite field , Journal of Algebra andIts Applications, (2014), 1450033-1—1450033-46 Department of Mathematics, Purdue University, West Lafayette, IN, USA, 47907
E-mail address : [email protected] Department of Mathematics and Statistics, University of Calgary, Calgary, Alberta, Canada, T2N1N4
E-mail address ::