Fourier-Jacobi cycles and arithmetic relative trace formula (with an appendix by Chao Li and Yihang Zhu)
FFOURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACEFORMULA (WITH AN APPENDIX BY CHAO LI AND YIHANG ZHU)
YIFENG LIU
Abstract.
In this article, we develop an arithmetic analogue of Fourier–Jacobi period integrals fora pair of unitary groups of equal rank. We construct the so-called Fourier–Jacobi cycles, which arealgebraic cycles on the product of unitary Shimura varieties and abelian varieties. We propose the arith-metic Gan–Gross–Prasad conjecture for these cycles, which is related to central derivatives of certainRankin–Selberg L -functions, and develop a relative trace formula approach toward this conjecture. Asa necessary ingredient, we propose the conjecture of the corresponding arithmetic fundamental lemma,and confirm it for unitary groups of rank at most two and for the minuscule case. Contents
1. Introduction 21.1. Fourier–Jacobi cycles and arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n ) 21.2. A relative trace formula approach 51.3. Arithmetic fundamental lemma for U( n ) × U( n ) 71.4. Relation between U( n ) × U( n ) and U( n ) × U( n + 1) 91.5. Relation with arithmetic triple product formula 101.6. Structure of the article 111.7. Notation and conventions 122. Albanese variety 132.1. Albanese variety and its polarization 142.2. Picard motive from almost ample divisor 183. Algebraic cycles and height pairings 213.1. Cycles and correspondences 213.2. Beilinson–Bloch–Poincaré height pairing 223.3. Künneth–Chow projectors 244. Fourier–Jacobi cycles and derivative of L -functions 254.1. Motives for CM characters 254.2. Albanese of unitary Shimura varieties 284.3. Construction of Fourier–Jacobi cycles 354.4. Arithmetic Gan–Gross–Prasad conjecture 385. Arithmetic relative trace formula 415.1. A doubling formula for CM data 425.2. Arithmetic invariant functional 465.3. Orbital decomposition of local arithmetic invariant functional 49Appendix A. Proof of arithmetic fundamental lemma for the minuscule caseby Chao Li and Yihang Zhu 54A.1. Derivatives of orbital integrals via lattice counting 54A.2. The minuscule case 56Appendix B. Poles of Eisenstein series and theta lifting for unitary groups 57B.1. Discrete automorphic spectrum 57 Date : February 24, 2021.2010
Mathematics Subject Classification. a r X i v : . [ m a t h . N T ] F e b YIFENG LIU
B.2. Main theorem and consequences 58B.3. Proof of Theorem B.4 60Appendix C. Shimura varieties for hermitian spaces 64C.1. Case of isometry 64C.2. Case of similitude 66C.3. Their connection 68C.4. Integral models and uniformization 70Appendix D. Cohomology of unitary Shimura curves 75D.1. Oscillator representations of local unitary groups 75D.2. Setup for cohomology of Shimura varieties 77D.3. Statements for cohomology of unitary Shimura curves 78D.4. Proof of Theorem D.6 81References 851.
Introduction
Fourier–Jacobi cycles and arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n ) . We first recall the classical notion of Fourier–Jacobi periods for U( n ) × U( n ) and their relation with L -functions (see [GGP12a, Section 24] for more details). Let E/F be a quadratic extension of numberfields with the nontrivial Galois involution c and the associated quadratic character µ E/F : F × \ A × F →{± } . Let V be a (non-degenerate) hermitian space over E of rank n (cid:62) c , withthe unitary group U(V). Consider two irreducible cuspidal automorphic representations π and π ofU(V)( A F ). To define the Fourier–Jacobi periods for π × π , we need an auxiliary conjugate symplecticautomorphic character µ of A × E , that is, an automorphic character of A × E whose restriction to A × F coincides with µ E/F . The character µ (together with a nontrivial additive character of ( E + A F ) \ A E )will give us a Weil representation of U(V)( A F ), with natural automorphic realization via theta series θ φµ attached to certain Schwartz functions φ . We define the Fourier–Jacobi period integral for f ∈ π , f ∈ π , and φ to be P µ ( f , f ; φ ) := Z U(V)( F ) \ U(V)( A F ) f ( g ) f ( g ) θ φµ ( g ) d g where d g is the Tamagawa measure on U(V)( A F ). Readers may realize that the above formula is veryclose to the Rankin–Selberg integral for GL( n ) × GL( n ) in which the role of the theta series is replacedby a mirabolic Eisenstein series (see [Liu14, Section 3] for a systematic discussion). In particular, it isnot surprising that P µ ( f , f ; φ ) is related to L -values. In fact, if we assume that π and π are bothtempered, then as a special case of the global Gan–Gross–Prasad (GGP) conjecture, one expects anIchino–Ikeda type relation |P µ ( f , f ; φ ) | ∼ L ( , π × π ⊗ µ ) · α ( f , f ; φ )(1.1)where ∼ means that the two sides are differed by an explicit nonzero factor which depends only on π , π , and µ ; L ( s, π × π ⊗ µ ) denotes the complete Rankin–Selberg L -function (of symplectic type)centered at s = ; and α ( f , f ; φ ) is some explicit period integral of local matrix coefficients. See[Xue16, Conjecture 1.1.2] for a precise conjecture. Suppose that the central (cid:15) -factor (cid:15) ( , π × π ⊗ µ )is 1. By the refined local GGP conjecture, which is known in this case by [GI16], if we consider theentire Vogan L -packet of the triple (V , π , π ), then there is a unique member for which α is a nonzerofunctional. Thus, the global GGP conjecture asserts that the global period P µ vanishes on the entireVogan L -packet if and only if L ( , π × π ⊗ µ ) = 0. Note that the L -function depends only on theVogan L -packet.Now suppose that (cid:15) ( , π × π ⊗ µ ) = −
1. Then the local GGP conjecture already forces P µ tobe zero; and the first possible nonzero term in the Taylor expansion of L ( s, π × π ⊗ µ ) at s = is OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 3 L ( , π × π ⊗ µ ). Thus, it is curious to find a replacement of P µ that encodes information about thefirst central derivative. This is the main goal of this article. In fact, the same question can be askedfor all types of periods in the global GGP conjecture, namely,(1) O( m ) × O( n ) with n − m odd, which is of Bessel type,(2) U( m ) × U( n ) with n − m odd, which is of Bessel type,(3) U( m ) × U( n ) with n − m even, which is of Fourier–Jacobi type,(4) Sp(2 m ) × Mp(2 n ), which is of Fourier–Jacobi type.A replacement of the period integral (under certain assumptions on the field E/F and archimedeancomponents of the representations) is only known before for Case (1) with | m − n | = 1 and m, n (cid:62) | m − n | = 1 and m, n (cid:62)
1. They are both realized as height pairings of certaindiagonal cycles. See [GGP12a, Section 27] for more details. For example, the celebrated Gross–Zagierformula [GZ86] is responsible for O(2) × O(3); see [YZZ13] for a generalization. Now we give aformulation for Case (3) with n = m (cid:62) E/F is a CM extension, and n (cid:62)
2. We first state a resultconcerning the Albanese variety of a unitary Shimura variety. Let V be a totally positive definiteincoherent hermitian space over A E of rank n . We have the associated system of Shimura varieties { Sh( V ) K } K indexed by sufficiently small open compact subgroups K ⊆ U( V )( A ∞ F ), each being smoothof dimension n − E . Let X K be the canonical (smooth) toroidal compactification of Sh( V ) K (which is just Sh( V ) K if it is already proper). Put X ∞ := lim ←− K X K . Let A K be the Albanese varietyof X K ; see Section 2. Put A ∞ := lim ←− K A K , which is an abelian group pro-scheme over E . To everyconjugate symplectic automorphic character µ of A × E of weight one (Definition 4.3), we associate anumber field M µ ⊆ C , and an abelian variety A µ over E with CM action i µ : M µ → End E ( A µ ) Q ,unique up to isogeny, in Subsection 4.1. In particular, A µ has dimension [ M µ : Q ] /
2; and the setΩ( µ ) := Hom E ( A ∞ , A µ ) Q is naturally an M µ [U( V )( A ∞ F )]-module depending only on µ . Theorem 1.1 (Theorem 4.18, Corollary 4.20) . Let the notation be as above. There is an isomorphism Ω( µ ) ⊗ M µ C ’ M ε M χ ω ( µ, ε, χ ) of C [U( V )( A ∞ F )] -modules. Here, { ω ( µ, ε, χ ) } ε,χ , introduced in Definition 4.11, is a certain collection ofWeil representations of U( V )( A ∞ F ) which are isomorphic to the finite part of the Weil representationsappeared in the definition of P µ . We refer to Theorem 4.18 for the precise statement. Moreover, atevery finite level K , there is an isogeny decomposition A K ∼ Y µ A d ( µ,K ) µ , resp. A end K ∼ Y µ A d ( µ,K ) µ over E when n (cid:62) (resp. n = 2 ), where • the product is taken over representatives of Gal( C / Q ) -orbits of all conjugate symplectic auto-morphic characters of A × E of weight one, • d ( µ, K ) := P ε P χ dim C ω ( µ, ε, χ ) K with the same index set for ε, χ , and • A end K is the endoscopic part of A K when n = 2 , defined in (D.3) . The above theorem suggests that if we want to replace P µ by algebraic cycles, then the Albanesevariety should be involved. Definition 1.2.
We say that a complex representation Π of GL n ( A E ) is relevant if(1) Π = (cid:1) s (Π) i =1 Π i is an isobaric sum of s (Π) irreducible cuspidal automorphic representations { Π , . . . , Π s (Π) } , which we call cuspidal factors of Π, satisfying Π i ◦ c ’ Π ∨ i for every 1 (cid:54) i (cid:54) s (Π); YIFENG LIU (2) for every archimedean place v of E , Π v is isomorphic to the (irreducible) principal series repre-sentation induced by the characters (arg n − , arg n − , . . . , arg − n , arg − n ), where arg : C × → C × is the argument character defined by the formula arg( z ) := z/ √ zz .Note that (2) implies that the cuspidal factors Π , . . . , Π s (Π) in (1) are mutually non-isomorphic.Now we fix our (tempered) Vogan L -packet by choosing two relevant representations Π and Π of GL n ( A E ). We also fix a conjugate symplectic automorphic character µ of A × E of weight one. Let V be a totally positive definite incoherent hermitian space over A E of rank n . Consider irreducibleadmissible representations π ∞ and π ∞ of U( V )( A ∞ F ) whose base change to GL n ( A ∞ E ) are Π ∞ andΠ ∞ , respectively.Take a level subgroup K ⊆ U( V )( A ∞ F ). Let α K : X K → A K be “the Albanese morphism” sendingthe zero-dimensional cycle D n − K to zero, where D K is the canonical extension of the Hodge divisor.For test functions f , f ∈ C ∞ c ( K \ U( V )( A ∞ F ) /K, C ) for π ∞ and π ∞ , respectively (Definition 4.26),and a homomorphism φ : A K → A µ , we define a Chow cycleFJ( f , f ; φ ) K := | π (( X K ) E ac ) | · ( T f K ⊗ T f K ⊗ T can µ ) ∗ (id X K × X K × ( φ ◦ α K )) ∗ ∆ X K on X K × X K × A µ , where T f i K denotes the normalized Hecke correspondence on X K attached to f i ; T can µ is a specific correspondence on A µ (Definition 4.9); and ∆ X K ⊆ X K is the diagonal cycle. For i ∈ Z , put CH i ( X ∞ × X ∞ × A µ ) C := lim −→ K CH i ( X K × X K × A µ ) C , and denote by CH iµ ( X ∞ × X ∞ ) C the subspace of CH i ( X ∞ × X ∞ × A µ ) C on which M µ acts via theinclusion M µ , → C , which depends only on µ . Theorem 1.3 (Subsections 4.3 and 4.4) . The Chow cycle
FJ( f , f ; φ ) K is homologically trivial,compatible under pullback when changing K , hence defines an element FJ( f , f ; φ ) ∈ CH n − M µ : Q ] / ( X ∞ × X ∞ × A µ ) C . If we assume the conjecture on the injectivity of the ‘ -adic Abel–Jacobi map, then the assignment ( f , f , φ ) FJ( f , f ; φ ) induces a complex linear map FJ ε : π ∞ ⊗ C π ∞ ⊗ C Ω( µ, ε ) → Hom C [U( V )( A ∞ F ) × U( V )( A ∞ F )] (cid:16) π ∞ ⊗ C π ∞ , CH n − M µ : Q ] / µ ( X ∞ × X ∞ ) C (cid:17) which is invariant under the diagonal action of U( V )( A ∞ F ) on the left-hand side, for every given µ -admissible collection ε (Definition 4.12). Here, Ω( µ, ε ) is the sum of the factors of Ω( µ ) ⊗ M µ C in thedecomposition in Theorem 1.1 corresponding to ε, χ with χ arbitrary. We propose the following unrefined version of the arithmetic Gan–Gross–Prasad conjecture forU( n ) × U( n ). Conjecture 1.4 (Conjecture 4.31) . Let the notation be as above. Then for every given µ -admissiblecollection ε (Definition 4.12), the following three statements are equivalent:(a) We have FJ ε = 0 .(b) We have FJ ε = 0 , and dim C Hom C [U( V )( A ∞ F ) × U( V )( A ∞ F )] (cid:16) π ∞ ⊗ C π ∞ , CH n − M µ : Q ] / µ ( X ∞ × X ∞ ) C (cid:17) = 1 . (c) We have L ( , Π × Π ⊗ µ ) = 0 , and Hom C [U( V )( A ∞ F )] ( π ∞ ⊗ C π ∞ ⊗ C Ω( µ, ε ) , C ) = { } . This is not exactly what we do. Here, we state it in this ideally correct but technically wrong way only for simplicityand for emphasizing the main idea. See Subsection 4.3 for the rigorous construction.
OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 5
In view of the local GGP conjecture, the above conjecture implies that if (cid:15) ( , Π × Π ⊗ µ ) = − L ( , Π × Π ⊗ µ ) = 0 if and only if one can find a triple ( V , π ∞ , π ∞ ) as above such thatFJ ε = 0. Moreover, if this is the case, then such triple is uniquely determined for every fixed ε .See Remark 4.32 for more details. In fact, in the actual discussion in Subsection 4.4, we replaceCH n − M µ : Q ] / µ ( X ∞ × X ∞ ) C by its quotient by the common kernel of ‘ -adic Abel–Jacobi maps for all ‘ , in order to avoid Beilinson’s conjecture and make the discussion unconditional. Moreover, we alsotrack the rationality of the functional FJ ε via replacing C by a certain subfield of C .We now propose the following refined version of the arithmetic Gan–Gross–Prasad conjecture forU( n ) × U( n ), which is an explicit formula relating the Beilinson–Bloch–Poincaré heights (See Subsection3.2) of the cycles FJ( f , f ; φ ) K with the central derivative of L ( s, Π × Π ⊗ µ ). Conjecture 1.5 (Conjecture 4.33) . Let the notation be as above. For test functions f , f ∨ , f , f ∨ for π ∞ , ( π ∞ ) ∨ , π ∞ , ( π ∞ ) ∨ , respectively, and φ ∈ Hom E ( A K , A µ , ε ) , φ c ∈ Hom E ( A K , A µ c , − ε ) , theidentity vol( K ) · h FJ( f , f ; φ ) K , FJ( f ∨ , f ∨ ; φ c ) K i BBP X K × X K ,A µ = Q ni =1 L ( i, µ iE/F )2 s (Π )+ s (Π ) · L ( , Π × Π ⊗ µ ) L (1 , Π , As ( − n ) · L (1 , Π , As ( − n ) · β ( f , f ∨ , f , f ∨ , φ, φ c ) holds. Here, • µ c := µ ◦ c is the c -twist of µ ; and we may identify A µ c with A ∨ µ (Proposition 4.6); • Hom E ( A K , A µ , ε ) (resp. Hom E ( A K , A µ c , − ε ) ) is the intersection of Hom E ( A K , A µ ) (resp. Hom E ( A K , A µ c ) ) and Ω( µ, ε ) (resp. Ω( µ c , − ε ) ); • vol( K ) is the normalized volume of K (Definition 4.22); • h , i BBP X K × X K ,A µ is a variant of the (conjectural) Beilinson–Bloch height pairing, which we callBeilinson–Bloch–Poincaré height pairing, which is a bilinear map CH n − M µ : Q ] / ( X K × X K × A µ ) C × CH n − M µ : Q ] / ( X K × X K × A ∨ µ ) C → C ; • s (Π i ) has appeared in Definition 1.2; • As ± stands for the two Asai representations (see, for example, [GGP12a, Section 7] ); and • β is a certain normalized matrix coefficient integral defined immediately after Conjecture 4.33. In order to transfer the height pairing in the above conjecture to some other pairing without A µ , weintroduce a variant of the cycle FJ( f , f ; φ ) K by simply replacing the diagonal ∆ X K by a modifieddiagonal ∆ z X K , which we denote by FJ( f , f ; φ ) zK . It is actually equal to FJ( f , f ; φ ) K as elementsin CH n − M µ : Q ] / ( X K × X K × A µ ) C if Beilinson’s conjecture is granted. Thus, we also formulate avariant of the above refined arithmetic Gan–Gross–Prasad conjecture as Conjecture 4.37. Remark . We expect that the Fourier–Jacobi cycles can also be used to bound Selmer groups forthe Rankin–Selberg motive associated to Π × Π ⊗ µ , just as what we have done for O(3) × O(4)[Liu16] and for U( n ) × U( n + 1) [LTXZZ] using diagonal cycles.1.2. A relative trace formula approach.
For the case of central L -values for U( n ) × U( n ), namelythe relation (1.1), the author developed a relative trace formula approach in [Liu14] generalizing theJacquet–Rallis relative trace formula, which was later carried out by Hang Xue [Xue14, Xue16]. Thus,it is natural to expect a relative trace formula approach toward Conjecture 1.5 as well, as what WeiZhang did for the case U( n ) × U( n + 1) [Zha12]. However, our situation is much more complicatedboth due to the construction of the cycle FJ( f , f ; φ ) and the height pairing itself. Nevertheless, westill find such an approach after several reduction steps for the height pairing in Conjecture 1.5, orrather the variant Conjecture 4.37. In order to avoid extra technical difficulty, in this article, we onlydiscuss the relative trace formula for the case where Sh( V ) K is already proper, which we will nowassume.The first reduction step is the following theorem, which we refer as the doubling formula for CMdata . YIFENG LIU
Theorem 1.7 (Proposition 5.10, (5.8), and Proposition 5.15) . Let the notation be as in Conjecture1.5 (or rather Conjecture 4.37). For i = 1 , , let f i := f t i ∗ f ∨ i be the convolution of the transpose of f i and f ∨ i . If we write f = P s d s g − s K ∩ Kg − s as a finite sum with d s ∈ C and g s ∈ U( V )( A ∞ F ) , thenthe identity vol( K ) · h FJ( f , f ; φ ) zK , FJ( f ∨ , f ∨ ; φ c ) zK i BBP X K × X K ,A µ = X s d s · I K s (L g s f , φ s ) holds, in which • K s := K ∩ g s Kg − s ; • L g s f is the left translation of f by g s ; • φ s ∈ S ( V ( A ∞ E )) K s is a Schwartz function determined by ( φ, g s φ c ) via (5.4) ; and • for general K ⊆ U( V )( A ∞ F ) , f ∈ C ∞ c ( K \ U( V )( A ∞ F ) /K, C ) , and φ ∈ S ( V ( A ∞ E )) K , we put I zK ( f , φ ) := h p ∗ ∆ z X K , (∆ X K × T f K × Z ♥ K ) . p ∗ ∆ z X K i BB X K where Z ♥ K is a (formal sum of) divisor on X K × X K such that its restriction to the diagonal ∆ X K is Kudla’s generating series of special divisors associated to φ (Definition 5.3).Moreover, if f ⊗ φ is regularly supported at some nonarchimedean place v of F (Definition 5.14), thenthe cycles p ∗ ∆ X K and (∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K have empty intersection on X K . Thus, it suffices to study the functional I K ( f , φ ). We now assume that f ⊗ φ is regularly supportedat some nonarchimedean place v of F . Then the definition of the Beilinson–Bloch height pairingprovides us with a decomposition I zK ( f , φ ) = X u I zK ( f , φ ) u into local heights over all places u of E . In what follows, we will study an approximation of the localterm I zK ( f , φ ) u at certain places u by ignoring z .To continue the discussion, we need some notation. For integers r, s (cid:62)
1, denote by Mat r,s thescheme over Z of r -by- s matrices. For n (cid:62)
1, we put M n := Mat n, × Mat ,n ; and let S n be the F -subscheme of Res E/F
Mat n,n consisting of matrices g satisfying g · g c = I n , known as the symmetricspace. In view of the relative trace formula developed in [Liu14], we are looking for test functions˜ f ∈ S (S n ( A F )) and ˜ φ ∈ S (M n ( A F )) such that I K ( f , φ ) can be compared to another functional J (˜ f , ˜ φ ) which encodes the right-hand side of Conjecture 1.5. In this article, we only discuss theterm I K ( f , φ ) p and local components ˜ f p , ˜ φ p when p is a good inert prime of F (Definition 5.16), alsoregarded as a place of E .Let p be a good inert prime. Then X K has a canonical integral smooth model X K over O E p ; theHecke operator T f K extends naturally to X K by taking Zariski closure; and we also have a naturalextension of Z ♥ K to a (formal sum of) divisor Z ♥ K on X K × O E p X K . We define the local arithmeticinvariant functional at p to be I K ( f , φ ) p := 2 log | O F / p | · χ (cid:18) O ( p ∗ ∆ X K ) ⊗ L O X K O ((∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K ) (cid:19) where χ denotes the Euler–Poincaré characteristic, as an intersection number of cycles on X K , thesixfold self fiber product of X K over O E p . The following result provides an orbital decomposition of I K ( f , φ ) p , which is the key for the comparison of relative trace formulae. Theorem 1.8 (Theorem 5.25) . Let K, f , φ be as above such that f ⊗ φ is regularly supported at somenonarchimedean place v of F . Then for a good inert prime p , the identity I K ( f , φ ) p = 2 log | O F / p | · X (¯ ξ, ¯ x ) ∈ [U( ¯V)( F ) × ¯V( E ))] rs e − π · Tr F/ Q (¯ x, ¯ x ) ¯V Orb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) · χ (cid:16) O Γ ¯ ξ ⊗ L O N O ∆ Z (¯ x ) (cid:17) holds, where OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 7 • ¯V is a hermitian space over E satisfying ¯V ⊗ F A p F ’ V ⊗ A F A p F ; • the orbital integral is defined as Orb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) := Z U( ¯V)( A ∞ F , p ) ¯ f p (¯ g − ¯ ξ ¯ g ) ¯ φ p (¯ g − ¯ x ) d¯ g ; • χ (cid:16) O Γ ¯ ξ ⊗ L O N O ∆ Z (¯ x ) (cid:17) is a certain intersection number defined on a relative Rapoport–Zinkspace.We refer to Theorem 5.25 for the precise meaning of all notation. The term χ (cid:16) O Γ ¯ ξ ⊗ L O N O ∆ Z (¯ x ) (cid:17) is the one that is related to the derivative of L -function, moreprecisely, to the derivative of local orbital integrals at p in the decomposition of J (˜ f , ˜ φ ). The preciserelation is the content of the arithmetic fundamental lemma for U( n ) × U( n ), which we introduce inthe next subsection.1.3. Arithmetic fundamental lemma for U( n ) × U( n ) . In this subsection, we introduce the arith-metic fundamental lemma for U( n ) × U( n ). Since the question is purely local, we will shift our notationslightly from the previous discussion. Moreover, we will allow n to be an arbitrary positive integersince the discussion makes sense even for n = 1.Let F be a finite extension of Q p , with residue cardinality q . Let E/F be an unramified quadraticextension, and ˘ E a completed maximal unramified extension of E with k its residue field.We recall some definitions and facts from [Liu14, Section 5.3]. We say that a pair ( ζ, y ) ∈ S n ( F ) × M n ( F ) is regular semisimple if the matrix ( y ζ i + j − y ) ni,j =1 is non-degenerate, where y = ( y , y ) ∈ Mat n, ( F ) × Mat ,n ( F ). If ( ζ, y ) is regular semisimple, we define its transfer factor to be ω ( ζ, y ) := µ E/F (det( y , ζy , . . . , ζ n − y )) . The group GL n ( F ) acts on S n ( F ) × M n ( F ) by the formula ( ζ, y ) g = ( g − ζg, g − y , y g ), which preservesregular semisimple elements. We denote by [S n ( F ) × M n ( F )] rs the set of regular semisimple GL n ( F )-orbits.Let V + n (resp. V − n ) be a hermitian space over E of rank n whose determinant has even (resp.odd) valuation. For δ = ± , we say that a pair ( ξ, x ) ∈ U(V δn )( F ) × V δn ( E ) is regular semisimple if { x, ξx, . . . , ξ n − x } are linearly independent. The group U(V δn )( F ) acts on U(V δn )( F ) × V δn ( E ) bythe formula ( ξ, x ) g = ( g − ξg, g − x ), which preserves regular semisimple elements. We denote by[U(V δn )( F ) × V δn ( E )] rs the set of regular semisimple U(V δn )( F )-orbits. We say that ( ζ, y ) ∈ [S n ( F ) × M n ( F )] rs and ( ξ, x ) ∈ [U(V δn )( F ) × V δn ( E )] rs match if ζ and ξ have the same characteristic polynomialand y ζ i y = ( ξ i x, x ) for 0 (cid:54) i (cid:54) n −
1. The matching relation induces a bijection[S n ( F ) × M n ( F )] rs ’ [U(V + n )( F ) × V + n ( E )] rs a [U(V − n )( F ) × V − n ( E )] rs . Denote by [S n ( F ) × M n ( F )] ± rs ⊆ [S n ( F ) × M n ( F )] rs the subset corresponding to orbits in [U(V ± n )( F ) × V ± n ( E )] rs . Then a regular semisimple orbit ( ζ, y ) belongs to [S n ( F ) × M n ( F )] δ rs for δ = + (resp. δ = − )if and only if the det(( y ζ i + j − y ) ni,j =1 ) has even (resp. odd) valuation.Now we introduce the relevant orbital integral. For a regular semisimple pair ( ζ, y ) ∈ S n ( F ) × M n ( F )and a pair of Schwartz functions f ∈ S (S n ( F )), φ ∈ S (M n ( F )), we defineOrb( s ; f, φ ; ζ, y ) := Z GL n ( F ) f ( g − ζg ) φ ( g − y , y g ) µ E/F (det g ) | det g | sE d g where d g is the Haar measure under which GL n ( O F ) has volume 1. It is clear that the product ω ( ζ, y ) Orb(0; f, φ ; ζ, y ) depends only on the GL n ( F )-orbit of ( ζ, y ). We recall the following conjecturefrom [Liu14]. Conjecture 1.9 (relative fundamental lemma for U( n ) × U( n )) . For every regular semisimple orbit ( ζ, y ) ∈ [S n ( F ) × M n ( F )] rs , we have that YIFENG LIU (1) if ( ζ, y ) ∈ [S n ( F ) × M n ( F )] − rs , then ω ( ζ, y ) Orb(0; S n ( O F ) , M n ( O F ) ; ζ, y ) = 0; (2) if ( ζ, y ) ∈ [S n ( F ) × M n ( F )] +rs , then ω ( ζ, y ) Orb(0; S n ( O F ) , M n ( O F ) ; ζ, y ) = Z U(V + n ) K n ( g − ξg ) Λ n ( g − x ) d g (1.2) where ( ξ, x ) ∈ [U(V + n )( F ) × V + n ( E )] rs is the unique orbit that matches ( ζ, y ) ; Λ n is a self-duallattice in V + n ; K n is the stabilizer of Λ n ; and d g is the Haar measure on U(V + n ) under which K n has volume .Remark . Conjecture 1.9(1) is known by [Liu14, Proposition 5.14]. Conjecture 1.9(2) is known for p sufficiently large by [Liu14, Theorem 5.15].Now we describe the arithmetic fundamental lemma, where in (1.2) we replace the left-hand side byits derivative and the right-hand side by a certain intersection number on a (relative) Rapoport–Zinkspace. We start by recalling the notion of relative Rapoport–Zink spaces. For an O ˘ E -scheme S , a unitary O F -module of signature ( r, s ) with integers r, s (cid:62) X, i, λ ) in which • X is a strict O F -module over S of dimension r + s and O F -height 2( r + s ) over S ; • i : O E → End S ( X ) is an action compatible with the O F -module structure satisfying that forevery e ∈ O E the characteristic polynomial of i ( e ) on Lie S ( X ) is given by ( T − a c ) r ( T − a ) s ∈O S [ T ]; • λ : X → X ∨ is a principal polarization such that the associated Rosati involution induces theconjugation on O E .We say that ( X, i, λ ) is supersingular if X is a supersingular strict O F -module.We fix a supersingular unitary O F -module ( X , i , λ ) of signature (1 ,
0) over O ˘ E , which is uniqueup to isomorphism. For every integer n (cid:62)
1, we also choose a supersingular unitary O F -module( X n , i n , λ n ) of signature ( n − ,
1) over k , which is unique up to O E -linear isogeny preserving thepolarization up to scalars. Let N n be the relative Rapoport–Zink space parameterizing quasi-isogeniesof ( X n , i n , λ n ) of height zero. More precisely, it is a formal scheme over O ˘ E such that for every scheme S over O ˘ E on which p is locally nilpotent, N n ( S ) is the set of isomorphism classes of quadruples( X, i, λ ; ρ ) where • ( X, i, λ ) is a unitary O F -module over S of signature ( n − , • ρ : X × S S k → X n × k S k is an O E -linear quasi-isogeny (of height zero), such that ρ ∗ λ n = λ .Here, we put S k := S ⊗ O ˘ E k .It is known that N n is formally smooth over O ˘ E of relative dimension n −
1. See [Mih, Section 3.1]for more details.We recall the notion of formal special divisors from [KR11]. Put Λ n := Hom k (( X k , i k ) , ( X n , i n ))and V − n := (Λ n ) Q . Then V − n is an E -vectors space of rank n equipped with a hermitian form( x, y ) = i − k (cid:16) λ − k ◦ y ∨ ◦ λ n ◦ x (cid:17) ∈ E. If we denote by Λ ∗ n the dual lattice of Λ n under the above hermitian form, then we have Λ n ⊆ Λ ∗ n and that the length of the O E -module Λ ∗ n / Λ n is odd. In particular, the determinant of V − n has oddvaluation, justifying its notation. Definition 1.11.
For every x ∈ V − n that is nonzero, we define Z n ( x ) to be the subfunctor of N n suchthat for every scheme S over O ˘ E on which p is locally nilpotent, Z n ( x )( S ) consists of ( X, i, λ ; ρ ) suchthat the composite homomorphism X k × k S k x −→ X n × k S k ρ − −−→ X × S S k extends to an O E -linear homomorphism X × O ˘ E S → X over S . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 9
By [RZ96, Proposition 2.9], Z n ( x ) is a closed sub-formal scheme of N n . For every g ∈ U(V n )( F ),let ρ g : X n → X n be the unique O E -linear quasi-isogeny (of height zero) such that gx = ρ g ◦ x forevery x ∈ V n ; and, by abuse of notation, let g : N n → N n be the (auto)morphism sending ( X, i, λ ; ρ )to ( X, i, λ ; ρ g ◦ ρ ). We denote by Γ g ⊆ N n := N n × O ˘ E N n the graph of g . Conjecture 1.12 (arithmetic fundamental lemma for U( n ) × U( n )) . For every regular semisimpleorbit ( ζ, y ) ∈ [S n ( F ) × M n ( F )] − rs , we have − ω ( ζ, y ) dd s (cid:12)(cid:12)(cid:12)(cid:12) s =0 Orb( s ; S n ( O F ) , M n ( O F ) ; ζ, y ) = 2 log q · χ (cid:18) O Γ ξ ⊗ L O N n O ∆ Z n ( x ) (cid:19) where ( ξ, x ) ∈ [U(V − n )( F ) × V − n ( E )] rs is the unique orbit that matches ( ζ, y ) ; and χ denotes the Euler–Poincaré characteristic. In Conjecture 1.12, it follows from Conjecture 1.9(1), which is known, that the left-hand side dependsonly on the GL n ( F )-orbit of ( ζ, y ). Remark . During the referee process of this article, Wei Zhang [Zha, Proposition 4.12 & Re-mark 3.1] has shown that his arithmetic fundamental lemma for U( n ) × U( n + 1) is equivalent to ourarithmetic fundamental lemma for U( n ) × U( n ) (with respect to the same field extension E/F ) whenthe residue cardinality of F is greater than n . In particular, we find(1) Conjecture 1.12 holds when n (cid:54) q is odd, by [Zha12, Theorem 2.10 & Theorem 5.5].(2) Conjecture 1.12 holds when F = Q p with p > n , by [Zha, Theorem 15.1]. Remark . In Section A, Chao Li and Yihang Zhu proved Conjecture 1.12 (for arbitrary
E/F ) inthe so-called minuscule case, just like the arithmetic fundamental lemma for U( n ) × U( n + 1). In thecase of U( n ) × U( n ), we say that a regular semisimple pair ( ξ, x ) ∈ U(V − n )( F ) × V − n ( E ) is minuscule if the O E -lattice L ξ,x generated by { x, ξx, · · · , ξ n − x } satisfies $L ∗ ξ,x ⊆ L ξ,x ⊆ L ∗ ξ,x where $ is auniformizer of F and L ∗ ξ,x denotes the dual lattice.1.4. Relation between U( n ) × U( n ) and U( n ) × U( n + 1) . In this subsection, we make an informalcomparison between the two scenarios of U( n ) × U( n ) and U( n ) × U( n + 1), for both automorphicperiods/central values and arithmetic periods/central derivatives.The following diagram compares the automorphic periods and relative trace formula approachestoward the global GGP conjectures in the two scenarios.Automorphic Fourier–Jacobiperiods for U( n ) × U( n )[Xue14, Xue16] (cid:111) (cid:111) global theta lifting (cid:47) (cid:47) (cid:79) (cid:79) relativetrace formula[Liu14] (cid:15) (cid:15) Automorphic Besselperiods for U( n ) × U( n + 1)[Zha14a, Zha14b] (cid:79) (cid:79) relativetrace formula [JR11] (cid:15) (cid:15) relativefundamental lemma+ (cid:111) (cid:111) “equivalent”[Liu14] (cid:47) (cid:47) relativefundamental lemma[Yun11]+relativesmooth matching (cid:111) (cid:111) “equivalent”[Xue14] (cid:47) (cid:47) relativesmooth matching[Zha14a] In the first line, if we assume Conjecture 1.15 below, then the method of global theta lifting shouldprovide an equivalence between the two sides when all n are considered. In fact, Xue [Xue14,Xue16] hasessentially verified the deduction for both directions starting from two stable tempered representationson U( n ) that satisfy Conjecture 1.15. Conjecture 1.15.
Let V be a hermitian space over E of rank n (cid:62) . Let π be a tempered cuspidalautomorphic representation of U(V)( A F ) . If n is even (resp. odd), then there exists a conjugateorthogonal (resp. conjugate symplectic) automorphic character µ of A × E such that L ( , Π ⊗ µ ) = 0 , where Π is the standard base change of π to GL n ( A E ) . The following diagram compares the arithmetic periods and relative trace formula approaches to-ward the arithmetic GGP conjectures in the two scenarios.Arithmetic Fourier–Jacobiperiods for U( n ) × U( n )[this article] (cid:111) (cid:111) motivic endoscopictransfer? (cid:47) (cid:47) (cid:79) (cid:79) arithmetictrace formula[this article] (cid:15) (cid:15) Arithmetic Besselperiods for U( n ) × U( n + 1)[Zha12, RSZ20] (cid:79) (cid:79) arithmetictrace formula [Zha12] (cid:15) (cid:15) arithmeticfundamental lemma+ (cid:111) (cid:111) “equivalent” Remark 1.13 (cid:47) (cid:47) arthmeticfundamental lemma+arithmeticsmooth matching (cid:111) (cid:111) perhaps related? (cid:47) (cid:47) arithmeticsmooth matchingIn the first line, the Tate conjecture over number fields predicts a motivic endoscopic lifting (ormotivic theta lifting) that transfers algebraic cycles from one side to the other. Thus, we expect thatour Fourier–Jacobi cycles should be related to the diagonal cycle considered in [Zha12] in a certain way.However, at this moment the motivic endoscopic lifting seems far out of reach. For the two dashedbubbles surrounding “arithmetic smooth matching”, we do not how to formulate a precise conjecturein general. However, in some special cases for U( n ) × U( n + 1), there are some results [RSZ17, RSZ18].1.5. Relation with arithmetic triple product formula.
In this subsection, we compare our arith-metic GGP conjecture for n = 2 with the (conjectural) arithmetic triple product formula, which can beregarded as the arithmetic GGP for O(3) × O(4) in which O(4) has trivial discriminant. Lots of progresshas been made toward the arithmetic triple product formula; see, for example, [GK92, GK93, YZZ].We first make a quick review of the arithmetic triple product formula following the line of [YZZ].Consider three irreducible cuspidal automorphic representations σ , σ , σ of GL ( A F ) of parallelweight 2 such that the product of their central characters is trivial and (cid:15) ( , σ × σ × σ ) = −
1. Thenthe local dichotomy of triple product invariant functionals provides us with a totally definite incoherentquaternion algebra B over A F , unique up to isomorphism. Let { Y U } U be the system of compactifiedShimura curves over F associated to B indexed by open compact subsets U ⊆ ( B ⊗ A F A ∞ F ) × . For i = 1 , ,
3, let A i be the abelian variety of strict GL(2)-type over F associated to σ i . For morphisms g i : Y U → A i for i = 1 , ,
3, we have the Gross–Kudla–Schoen cycle, which is essentiallyGKS( g , g , g ) U := ( g × g × g ) ∗ ∆ Y U ∈ CH ( A × A × A ) Q . Recently, Dihua Jiang and Lei Zhang [JZ20] have confirmed this conjecture when n (cid:54)
4. Of course, when n (cid:54)
2, itwas already known before.
OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 11
The arithmetic triple product formula predicts a relation between the Beilinson–Bloch height ofGKS( g , g , g ) U and the central derivative L ( , σ × σ × σ ).Now we discuss its connection with our case. Suppose that(1) σ is the theta lifting of µ alg := µ · | | − / E for an automorphic character µ of A × E , which isnecessarily conjugate symplectic of weight one;(2) for i = 1 ,
2, the base change of σ i to GL ( A E ), denoted by Π i , has trivial central character.Then Π and Π are both relevant; and we may take A µ to be A ⊗ F E . Let V be the unique upto isomorphism totally positive definite (incoherent) hermitian space over A E of rank 2 such that forevery nonarchimedean place v of F , V ⊗ A F F v is isotropic if and only if B ⊗ A F F v is split. We recallour compactified unitary Shimura curve { X K } K associated to V . For morphisms f i : X K → A i ⊗ F E for i = 1 , φ : X K → A µ = A ⊗ F E , we have the Fourier–Jacobi cycle, which is essentiallyFJ( f , f ; φ ) K := ( f × f × φ ) ∗ ∆ X K ∈ CH (( A × A × A ) ⊗ F E ) Q . Conjecture 1.5 predicts a relation between the Beilinson–Bloch height of FJ( f , f ; φ ) K and the centralderivative L ( , Π × Π ⊗ µ ). It is possible to show a priori that the height of GKS( g , g , g ) U forsome choice of U, g , g , g is related to the height of FJ( f , f ; φ ) K for some choice of K, f , f , φ . Thisis not surprising as in this case we have the equality L ( s, σ × σ × σ ) = L ( s, Π × Π ⊗ µ )between L -functions. In other words, our work in the special case where n = 2 provides a relativetrace formula approach toward the arithmetic triple product formula in the situation where (1) and(2) are satisfied. But note that, the cases for U(2) × U(2) that have overlap with the arithmetic tripleproduct formula is also a proper subset since Π and Π are necessarily base change from GL ( A F ).1.6. Structure of the article.
The main part of the article contains five sections.In Section 2, we study Albanese varieties. In Subsection 2.1, we introduce Albanese varieties ofproper smooth varieties over a general base field, and study their polarizations. In Subsection 2.2,we generalize the construction of Picard motives using not necessarily ample divisors as the cuttingdivisor, which will be used in Subsection 3.3.In Section 3, we make some preparation for algebraic cycles and height pairings for general varieties.In Subsection 3.1, we review the notion of algebraic cycles and correspondences. In Subsection 3.2,we review the construction of the Beilinson–Bloch height pairing and introduce our variant, namely,the Beilinson–Bloch–Poincaré height pairing. In Subsection 3.3, we discuss the construction of someKünneth–Chow projectors for curves and surfaces, which can be used in the modified diagonal ∆ z X K later.In Section 4, we construct Fourier–Jacobi cycles and state our main conjectures. In Subsection 4.1,we construct the category of CM data for a conjugate symplectic automorphic character µ of weightone. In Subsection 4.2, we introduce our Shimura varieties and study their Albanese varieties; inparticular, we prove Theorem 1.1. In Subsection 4.3, we construct Fourier–Jacobi cycles and showthat they are homologically trivial. In Subsection 4.4, we prove the remaining part of Theorem 1.3,and propose various versions of the arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n ).In Section 5, we discuss the relative trace formula approach toward the arithmetic GGP conjecturefor U( n ) × U( n ). In Subsection 5.1, we prove the doubling formula for CM data in Theorem 1.7. InSubsection 5.2, we introduce the arithmetic invariant functional I ( f , φ ), and its local version I ( f , φ ) p at a good inert prime for which we perform some preliminary computation. In Subsection 5.3, weprove Theorem 1.8.The article also contains four appendices.In Section A, provided by Chao Li and Yihang Zhu, we confirm the arithmetic fundamental lemmain the minuscule case.In Section B, we prove some results about global theta lifting for unitary groups, namely, TheoremB.4 and its two corollaries. Those results are only used in the proof of Proposition 4.13. Thus, if readers are willing to admit these results from the theory of automorphic forms, they are welcome toskip the entire section except the very short Subsection B.1 where we introduce some notation for thediscrete automorphic spectrum.In Section C, we summarize different versions of unitary Shimura varieties. In Subsection C.1,we recall Shimura varieties associated to isometry groups of hermitian spaces, which are of abeliantype; we also introduce the Shimura varieties associated to incoherent hermitian spaces – they aremain geometric objects studied in this article. In Subsection C.2, we recall the well-known PELtype Shimura varieties associated to groups of rational similitude of skew-hermitian spaces, and theirintegral models at good primes, after Kottwitz. These Shimura varieties are only for the preparation ofthe next subsection, which are not logically needed in the main part of the article. In Subsection C.3,we summarize the connection of these two kinds of unitary Shimura varieties via the third one whichpossesses a moduli interpretation but is not of PEL type in the sense of Kottwitz, after [BHKRY,RSZ20]. In Subsection C.4, we discuss integral models of the third unitary Shimura varieties at goodinert primes and uniformization along the basic locus. The last two subsections are crucial to thediscussion in Subsections 5.2 and 5.3.In Section D, we compute the cohomology of Shimura curves associated to isometry groups ofhermitian spaces of rank 2, as Galois–Hecke modules. In Subsection D.1, we collect some resultsabout local oscillator representations of unitary groups of general rank. In Subsection D.2, we recallsome facts and introduce some notation about cohomology of Shimura varieties in general. These twosubsections will be used both in Section D and in the main part of the article. The last two subsectionsconcern the cohomology of unitary Shimura curves, for the statements and for the proof, respectively.These statements are only used in the proof of Theorem 4.15 and Theorem 4.18 in the main part ofthe article, and are probably known to experts. However, we can not find any reference for the proofsor even for the statements themselves.For readers’ convenience, we summarize the dependence relation of the article in the followingdiagram. 2 (cid:33) (cid:41) B (cid:15) (cid:15) C (cid:15) (cid:15) (cid:43) (cid:51) (cid:53) (cid:61) D (cid:79) (cid:79) A (cid:79) (cid:79) Notation and conventions.
General notation. • For a set S , we denote by S the characteristic function of S . • Suppose that we are in a category with finite products. Then – for a finite collection { X , . . . , X n } of objects, the notation p abc ··· : X × · · · × X n → X a × X b × X c × · · · will, by default, stand for the projection to the factors labeled by { a, b, c, . . . } ⊆ { , . . . , n } ; – for an abject X and an integer r (cid:62)
0, we denote ∆ r : X → X r the diagonal morphism,and simply write ∆ for ∆ . • All rings are commutative and unital; and ring homomorphisms preserve unity. • For an abelian group A and a ring R , we put A R := A ⊗ Z R as an R -module. • For a field k , we denote by k ac an abstract algebraic closure of k . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 13 • The bar only denotes the complex conjugation in C . For example, for an element x ∈ C ⊗ Q E , x is obtained by only applying conjugation to the first factor. • We denote by C the subgroup of C × consisting of z satisfying zz = 1. Notation in number theory. • A reflex field is always a subfield of C . • Denote by A ∞ := b Z Q the ring of finite adèles of Q , and put A := R × A ∞ . • For a number field k , we put A k := A ⊗ Q k and A ∞ k := A ∞ ⊗ Q k . – Denote by | | Q : Q × \ A × → R × > the character, uniquely determined by the propertiesthat | x | Q = | x | for x ∈ R × and that | | Q is trivial on b Z × . For every s ∈ C , Put | | sk := | | s Q ◦ N k/ Q : k × \ A × k → R × > . – Denote by ψ Q : Q \ A → C × the character, uniquely determined by the properties that ψ Q ( x ) = exp(2 πix ) for x ∈ R and that ψ Q is trivial on b Z . Put ψ k := ψ Q ◦ Tr k/ Q : k \ A k → C × , which we call the standard additive character for k . • In local or global class field theory, the Artin reciprocity map always sends a uniformizer at anonarchimedean place v to a geometric Frobenius element at v . Notation in algebraic geometry. • For a scheme S and a rational prime p , we denote Sch /S the category of schemes over S andSch /S the subcategory of those that are locally Noetherian. If S = Spec R is affine, then wesimply replace S by R in above notations. • We denote by G m := Spec Z [ T, T − ] the multiplicative group scheme over Z . For integers r, s (cid:62)
1, denote by Mat r,s the scheme over Z of r -by- s matrices. For an integer n (cid:62)
1, we putM n := Mat n, × Mat ,n . • For a ring R , a scheme X in Sch /R , and a ring R over R , we usually write X R ∈ Sch /R instead of X × Spec R Spec S . • For a ring R , a scheme X ∈ Sch /R that is locally of finite type, a homomorphism τ : R → C ,and an abelian group Λ, we denote by H i B ,τ ( X, Λ) the degree i singular cohomology of theunderlying topological space of X ⊗ R,τ C with coefficient Λ. If R is a subring of C and τ is theinclusion, then we suppress τ in the subscript. • For a ring R , we denote by D bfl ( R ) the bounded derived category of R -modules whose coho-mology consists of R -modules of finite length. For F ∈ D bfl ( R ), we have the Euler–Poincarécharacteristic χ ( F ) := P i ( − i length R H i F . In general, for a (formal) scheme X over R andan element F in the derived category of O X -modules, we define its Euler–Poincaré character-istic χ ( F ) to be χ (R s ∗ F ) (resp. ∞ ) if R s ∗ F belongs to D bfl ( R ) (resp. otherwise), where s isthe structure morphism. Acknowledgements.
The author would like to thank Sungyoon Cho, Benedict Gross, Kai-Wen Lan,Chao Li, Xinyi Yuan, Shouwu Zhang, and Wei Zhang for helpful comments and discussion, andChao Li and Yihang Zhu for providing Section A for the proof of the arithmetic fundamental lemmain the minuscule case. He thanks the anonymous referee for careful reading and valuable comments.The research of the author is partially supported by NSF grant DMS–1702019 and a Sloan ResearchFellowship. 2.
Albanese variety
In this section, we study Albanese varieties. In Subsection 2.1, we introduce Albanese varieties ofproper smooth varieties over a general base field, and study their polarizations. In Subsection 2.2,we generalize the construction of Picard motives using not necessarily ample divisors as the cuttingdivisor.Let k be a field. We work in the category Sch /k . Albanese variety and its polarization.Definition 2.1.
Consider schemes
X, Y ∈ Sch /k of finite type.(1) We denote by ∇ X the smallest open and closed subscheme of X × X containing the diagonal∆ X . For every morphism u : Y → X , u × u restricts to a morphism ∇ u : ∇ Y → ∇ X .(2) We say that a field k over k splits X if every connected component of X k is geometricallyconnected. For such k , we regard π ( X k ) as a scheme in Sch /k , which induces a factorization X k → π ( X k ) → Spec k in Sch /k . In particular, giving an element in X ( π ( X k )) is equivalentto giving an element in X i ( k ) for every connected component X i of X k .(3) Let f : ∇ X → Y be a morphism. For every field k over k that splits X and every point x ∈ X ( π ( X k )), we denote by f x : X k → Y k the morphism such that its restriction to a connected component X i is the restriction of f k to X i × k ( x ∩ X i ) ’ X i .The following proposition on the Albanese variety without rational base point is probably well-known. Since we can not find a definitive reference for it, we include a proof. Proposition 2.2.
Let X be a proper smooth scheme in Sch /k . Consider the functor Alb X on thecategory of abelian varieties A over k such that Alb X ( A ) is the set of morphisms f : ∇ X → A over k such that ∆ X is contained in f − A . Then Alb X is corepresentable.Proof. Let k be a separable closure of k . Then k splits X (Definition 2.1). Put X := X k . Wefirst consider the problem for X . Pick up a point x ∈ X ( π ( X )), which is possible as X is smoothover k . By Serre’s construction [Ser59] of Albanese variety (see [Wit08, Appendix A] for a versionover separably closed field), we have a morphism g x : X → Alb X , universal among all morphisms g : X → A to an abelian variety A over k such that g ( x ) = 0 A . Now it is easy to see that thecomposite morphism ∇ k X g x × g x −−−−→ Alb X × k Alb X − −→ Alb X does not depend on the choice of x , and corepresents the functor Alb X . Here, ∇ k X is definedsimilarly as in Definition 2.1, but with the base field k . As ( ∇ X ) k ’ ∇ k X , the statement for X then follows by Galois descent. (cid:3) Definition 2.3.
Let X be a proper smooth scheme in Sch /k . The abelian variety that corepresentsthe functor Alb X is called the Albanese variety of X , denoted by Alb X . The canonical morphism,denoted by α X : ∇ X → Alb X , is called the Albanese morphism . For a morphism u : Y → X of proper smooth schemes over k , wehave the induced morphism Alb u : Alb Y → Alb X by the universal property, which satisfies Alb u ◦ α X = α Y ◦ ∇ u . Lemma 2.4.
Suppose that k has characteristic zero. Then(1) for every homomorphism τ : k → C , we have a canonical isomorphism H ,τ (Alb X , Q ) ’ H ,τ ( X, Q ) ;(2) for every prime ‘ , we have a canonical isomorphism H ((Alb X ) k ac , Q ‘ ) ’ H ( X k ac , Q ‘ ) of Gal( k ac /k ) -modules.Proof. For (1), we pick a point x ∈ X ( π ( X ⊗ k,τ C )). Then we have the morphism( α X ) x : X ⊗ k,τ C → Alb X ⊗ k,τ C from Definition 2.3 and Definition 2.1. By the property of complex Albanese varieties, the inducedmap ( α X ) ∗ x : H ,τ (Alb , Q ) → H ,τ ( X, Q ) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 15 is an isomorphism; it is independent of the choice of x since translation acts trivially on H ,τ (Alb , Q ).For (2), we extend the morphism α X to α X : X × X → Alb X by letting X × X \ ∇ X map to 0 Alb X .By Künneth formula, we have the map( α X ) ∗ : H ((Alb X ) k ac , Q ‘ ) → H ( X k ac , Q ‘ ) ⊗ Q ‘ H ( X k ac , Q ‘ )of Gal( k ac /k )-modules. Taking cup product, we obtain a map α X : H ((Alb X ) k ac , Q ‘ ) → H ( X k ac , Q ‘ )of Gal( k ac /k )-modules. It suffices to show that this is an isomorphism. Since k has characteristic 0,by the Lefschetz principle and the comparison theorem for singular and étale cohomology, α X ⊗ Q ‘ C via any embedding Q ‘ , → C is isomorphic to the canonical map in (1). Thus, (2) follows. (cid:3) Now we study polarizations of Alb X . Let k be an arbitrary field. Proposition 2.5.
Let X be a proper smooth scheme and A an abelian variety, both over k . For every f ∈ Alb X ( A ) and every divisor D on X , there is a unique homomorphism θ f,D : A ∨ → A (over k ) satisfying the following property: for every field k over k , every geometric point a of A ∨ ( k ) corresponding to a line bundle L a on A := A k , and every point x ∈ X ( π ( X k )) , we have θ f,D ( a ) = Σ A (cid:16) c ( L a ) .f x ∗ ( D dim X − ) (cid:17) , where Σ A : CH ( A ) → A ( k ) is the (classical) Albanese map for A . Moreover, θ f,D is symmetric,depends only on the rational equivalence class of D , and satisfies θ f,nD = [ n dim X − ] A ◦ θ f,D for n ∈ Z . This is previously known when D is a hyperplane section. See for example [Mur90, Section 2]. Proof.
The uniqueness is clear. Now we show the existence. We may assume that k is separably closed.In fact, for every point x ∈ X ( π ( X )), we are going to define a homomorphism θ f,D,x satisfying therequirement in the proposition. Then we will show that θ f,D,x does not depend on the choice of x .Therefore, by Galois descent, we conclude for general field k .We start from constructing θ f,D,x . Let P be the Poincaré line bundle on A ∨ × A . Consider thefollowing diagram of projection homomorphisms A ∨ × A ∨ × A p (cid:119) (cid:119) p (cid:15) (cid:15) p (cid:15) (cid:15) p (cid:38) (cid:38) A ∨ × A ∨ A ∨ × A A. (2.1)For every z ∈ CH ( A ), put D z := p ∗ ( p ∗ c ( P ) . p ∗ c ( P ) . p ∗ z )which belongs to CH ( A ∨ × A ∨ ). Then we put L z := O A ∨ × A ∨ ( D z ). We show(1) L z is symmetric, that is, L z is invariant under the obvious involution of A ∨ × A ∨ ;(2) the restrictions of L z to 0 A ∨ × A ∨ and A ∨ × A ∨ are both trivial;(3) for every point a ∈ A ∨ ( k ), the restriction of L z to a × A ∨ corresponds to the point Σ A ( c ( L a ) .z )under the canonical isomorphism A ∨∨ ’ A .Part (1) is straightforward from the definition. For (2), it suffices to show that the restricted linebunlde L z | A ∨ × A ∨ is trivial by (1). However, this is a special case of (3).Now we show (3). We expand the previous commutative diagram (2.1) to the following one A ∨ × A ’ a × A ∨ × A j (cid:47) (cid:47) q (cid:117) (cid:117) A ∨ × A ∨ × A p (cid:117) (cid:117) p (cid:15) (cid:15) p (cid:15) (cid:15) p (cid:37) (cid:37) A ∨ ’ a × A ∨ i (cid:47) (cid:47) A ∨ × A ∨ A ∨ × A A in which the parallelogram is Cartesian. By [Ful98, Proposition 1.7], we have L z | a × A ∨ ’ i ∗ L z ’ O A ∨ ( q ∗ j ∗ ( p ∗ c ( P ) . p ∗ c ( P ) . p ∗ z )) . (2.2)We put q := p ◦ j : A ∨ × A → A which is simply the projection to the second factor. Since j ∗ p ∗ P isisomorphic to q ∗ L a , we have(2.2) ’ O A ∨ ( q ∗ ( q ∗ c ( L a ) .c ( P ) .q ∗ z )) ’ O A ∨ ( q ∗ ( c ( P ) .q ∗ ( c ( L a ) .z ))) . It remains to show that the line bundle L on A ∨ corresponding the the point Σ A ( c ( L a ) .z ) is O A ∨ ( q ∗ ( c ( P ) .q ∗ ( c ( L a ) .z ))). Choose a representative P i m i a i of the 0-cycle c ( L a ) .z ; it has de-gree zero since L a is algebraically equivalent to zero. Then we have L ’ O i L ⊗ m i a i , where L a i is the line bundle on A ∨ corresponding to a i which, by the property of the Poincaré bundle,is isomorphic to O A ∨ ( q ∗ ( c ( P ) .q ∗ a i )). Thus, we have L ’ O i O A ∨ ( q ∗ ( c ( P ) .q ∗ a i )) ⊗ m i ’ O A ∨ ( q ∗ ( c ( P ) .q ∗ ( c ( L a ) .z ))) ;and (3) is proved.By (1) and (2), the line bundle L z induces a symmetric homomorphism θ z : A ∨ → A . Now taking z = f x ∗ D dim X − , we obtain a symmetric homomorphism θ f,D,x : A ∨ → A satisfying the requirementin the proposition. To construct θ f,D , it suffices to show that θ f,D,x = θ f,D,y for any other choice of y .This amounts to showing thatΣ A (cid:16) c ( L a ) .f x ∗ ( D dim X − ) (cid:17) = Σ A (cid:16) c ( L a ) .f y ∗ ( D dim X − ) (cid:17) (2.3)Put b := f x ( y ) ∈ A ( k ). Then f y = t b ◦ f x where t b is the translation morphism on A by b . Since L a isalgebraically equivalent to zero, c ( L a ) .f y ∗ ( D dim X − ) is a degree zero divisor. Thus, we haveΣ A (cid:16) c ( L a ) .f y ∗ ( D dim X − ) (cid:17) = Σ A (cid:16) t − b ∗ c ( L a ) .f x ∗ ( D dim X − ) (cid:17) . Again, since L a is algebraically equivalent to zero, we have t − b ∗ c ( L a ) = c ( L a ) ∈ CH ( A ). Thus,(2.3) follows.The last assertion of the proposition is already clear. (cid:3) In the case (
A, f ) = (Alb X , α X ), we will write θ X,D := θ α X ,D : Alb ∨ X → Alb X . Remark . If dim X = 1, then θ X,D is simply the canonical polarization of Alb X (which is simplythe Jacobian of X ), hence is an isomorphism and independent of D .We have the following result on the functoriality of θ X,D . Proposition 2.7.
Let u : Y → X be a generically finite dominant morphism of proper smooth schemesover k . Let D be a divisor on X . Then we have [deg u ] Alb X ◦ θ X,D = Alb u ◦ θ Y,u ∗ D ◦ Alb ∨ u . Here, deg u is regarded as a function on π ( X ) whose value on a connected component of X isthe total degree of u over it; and if we write X = ‘ X i , then [deg u ] Alb X is the endomorphism Q i [(deg u )( X i )] Alb Xi on Alb X ’ Q i Alb X i .Proof. We may assume that k is algebraically closed and both X and Y are connected. Put d :=dim X = dim Y . Take points a ∈ Alb ∨ X ( k ) and y ∈ Y ( k ). Put b := Alb ∨ u ( a ) ∈ Alb ∨ Y ( k ) and x := OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 17 u ( y ) ∈ X ( k ). Put f := α X,x : X → Alb X and g := α Y,y : Y → Alb Y for short. By the functoriality ofAlbanese morphisms, the following diagram Y g (cid:47) (cid:47) u (cid:15) (cid:15) Alb Y Alb u (cid:15) (cid:15) X f (cid:47) (cid:47) Alb X commutes. To prove the proposition, it suffices to show that[deg u ] Alb X ( θ X,D ( a )) = Alb u ( θ Y,u ∗ D ( b )) . (2.4)By Proposition 2.5 and the projection formula [Ful98, Example 8.1.7], the left-hand side of (2.4) equals[deg u ] Alb X (cid:16) Σ Alb X (cid:16) c ( L a ) .f ∗ ( D d − ) (cid:17)(cid:17) = Σ Alb X (cid:16) deg u · f ∗ (cid:16) f ∗ c ( L a ) .D d − (cid:17)(cid:17) . (2.5)Again by the projection formula, we havedeg u · f ∗ c ( L a ) .D d − = f ∗ c ( L a ) .u ∗ ( u ∗ D d − ) . Repeatedly applying the projection formula, we have(2.5) = Σ
Alb X (cid:16) f ∗ (cid:16) f ∗ c ( L a ) .u ∗ ( u ∗ D d − ) (cid:17)(cid:17) = Σ Alb X (cid:16) f ∗ (cid:16) u ∗ (cid:16) u ∗ f ∗ c ( L a ) . ( u ∗ D ) d − (cid:17)(cid:17)(cid:17) = Σ Alb X (cid:16) Alb u ∗ g ∗ (cid:16) g ∗ Alb ∗ u c ( L a ) . ( u ∗ D ) d − (cid:17)(cid:17) = Alb u (cid:16) Σ Alb Y (cid:16) g ∗ (cid:16) g ∗ c ( L b ) . ( u ∗ D ) d − (cid:17)(cid:17)(cid:17) = Alb u (cid:16) Σ Alb Y (cid:16) c ( L b ) .g ∗ ( u ∗ D ) d − (cid:17)(cid:17) = Alb u ( θ Y,u ∗ D ( b )) . The proposition follows. (cid:3)
Definition 2.8.
We say that a divisor D on a proper smooth scheme X over k is almost ample if thereexists m ∈ Z > such that | mD | is base point free and the induced morphism φ mD : X → P ( | mD | ) is agenerically finite morphism onto its image. Proposition 2.9.
Suppose that k has characteristic zero. Let X be a proper smooth scheme in Sch /k and D a divisor on X such that D is almost ample. Then the symmetric homomorphism θ X,D : Alb ∨ X → Alb X is a polarization.Proof. Since k has characteristic zero, by the Lefschetz principle, we may assume that k is embeddableinto C . To check whether θ X,D is a polarization, we may take base change to C . Thus, we may justassume k = C and assume that X is connected. Since D is almost ample, by replacing D by mD forsome m ∈ Z > , we may assume that | D | is base point free and the induced morphism φ D : X → P ( | D | )is a generically finite morphism onto its image.Put A := Alb X , d := dim X and h := dim A for short. We choose a point x ∈ X ( C ), and put f := α X,x : X → A . We have canonical isomorphisms A ∨ ( C ) ’ H ( A, O A ) / H ( A, Z ) , A ( C ) ’ H h ( A, Ω h − A ) / H h − ( A, Z )of complex manifolds. From the construction, the following diagramH ( A, O A ) ∧ f ∗ c ( D ) d − (cid:47) (cid:47) (cid:15) (cid:15) H h ( A, Ω h − A ) (cid:15) (cid:15) H ( A, O A ) / H ( A, Z ) θ X,D (cid:47) (cid:47) H h ( A, Ω h − A ) / H h − ( A, Z ) commutes, where the vertical arrows are quotient maps. Here, c ( D ) is regarded as the Chern class inH ( X, Ω X ). Then the symmetric homomorphism θ X,D is a polarization if and only if for every nonzero ∂ -closed smooth (0 , ω on A , we have Z A ( C ) ω ∧ ω ∧ f ∗ c ( D ) d − > . By the property that D satisfies, we may find a smooth hermitian metric k k D on O X ( D ) such thatits Chern (1 , c ( k k D ) is semi-positive on X ( C ) and strictly positive on a Zariski dense opensubset. Therefore, Z A ( C ) ω ∧ ω ∧ f ∗ c ( D ) d − = Z X ( C ) f ∗ ω ∧ f ∗ ω ∧ c ( D ) d − = Z X ( C ) f ∗ ω ∧ f ∗ ω ∧ c ( k k D ) d − > . The proposition follows. (cid:3)
Remark . There is a byproduct in proof of Proposition 2.9: For an almost ample divisor D ona proper smooth scheme X over a field k of characteristic zero, the degree of the top intersectiondeg D dim X is strictly positive on every irreducible component of X . Remark . We are curious whether one can find an algebraic proof of Proposition 2.9, and whetherthe proposition holds for an arbitrary field k or a weaker condition on D . Note that if D is a hyperplane,then it is known previously that θ X,D is an isogeny for arbitrary field k .2.2. Picard motive from almost ample divisor.
Let k be a field of characteristic zero. Let X bea proper smooth scheme in Sch /k of pure dimension d (cid:62)
1. For every almost ample divisor D on X ,we now define a correspondence e X,D ∈ CH d ( X × X ) Q such that the induced endomorphismcl ∗ dR ( e X,D ) : d M i =0 H i dR ( X/k ) → d M i =0 H i dR ( X/k )on the de Rham cohomology of X is the projection onto H ( X/k ). In particular, when X is projective,( X, e
X,D ) is a Grothendieck motive, which is a Picard motive for X . The construction generalizes theone in [Mur90, Section 3]. We use such construction only in Subsection 3.3 when the Shimura varietyis a non-proper surface; so readers may choose to skip this subsection for now.Let θ := θ X,D : Alb ∨ X → Alb X be the polarization obtained from Proposition 2.9. Let ϑ : Alb X → Alb ∨ X be an isogeny such that θ ◦ ϑ = [ n ] Alb X for some integer n (cid:62)
1. We obtain a morphism β := ( ϑ ◦ α X ) × α X : ∇ X × ∇ X → Alb ∨ X × Alb X . Let P be the Poincaré line bundle on the target. We put E X,D := p ∗ (cid:16) β ∗ c ( P ) . ( D d × X × D d × D d − ) (cid:17) ∈ CH d ( X × X ) Q where the intersection is taken in X × X × X × X , and e X,D := 1 n (deg D d ) E X,D ∈ CH d ( X × X ) Q where deg D d is understood as a function on π ( X ). We leave the readers an easy exercise to showthat e X,D does not depend on the choice of ϑ . Proposition 2.12.
Let X be a proper smooth scheme in Sch /k of pure dimension d (cid:62) , and D analmost ample divisor on X .(1) The map cl ∗ dR ( e X,D ) coincides with the projection to H ( X/k ) . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 19 (2) Let u : Y → X be a generically finite dominant morphism of proper smooth schemes over k .Then u ∗ D is an almost ample divisor on Y , and we have (id Y × u ) ∗ e Y,u ∗ D = ( u × id X ) ∗ e X,D in CH d ( Y × X ) Q .Proof. For both assertions, we may assume that k is algebraically closed and X is connected.For (1), recall that for every x ∈ X ( k ), we have the induced morphism ( α X ) x : X → Alb X byrestriction. Now take two arbitrary points x, x ∨ ∈ X ( k ). We have the induced morphism( ϑ ◦ ( α X ) x ∨ ) × ( α X ) x : X × X → Alb ∨ X × Alb X . Put E := (( ϑ ◦ ( α X ) x ∨ ) × ( α X ) x ) ∗ c ( P ) . ( X × D d − ) ∈ CH d ( X × X ) Q . It suffices to show that theinduced map cl ∗ dR ( E ) on the de Rham cohomology of X is the projection onto H ( X/k ) multipliedby n .As cl dR ( c ( P )) ∈ H (Alb ∨ X /k ) ⊗ k H (Alb X /k ), we have cl dR ( E ) ∈ H ( X/k ) ⊗ k H d − ( X/k ),which implies that cl ∗ dR ( E ) | H i dR ( X/k ) = 0 unless i = 1. It remains to show that cl ∗ dR ( E ) acts onH ( X/k ) by the multiplication of n . By Lemma 2.4 and the comparison theorem, it suffices to showthat the correspondence( ϑ × id Alb X ) ∗ c ( P ) . (Alb X × ( α X ) x ∗ D d − ) ∈ CH h (Alb X × Alb X ) Q induces the multiplication by n on H (Alb X /k ), where h is the dimension of Alb X . This in turn isequivalent to that( θ × id Alb X ) ∗ ( ϑ × id Alb X ) ∗ c ( P ) . (Alb ∨ X × ( α X ) x ∗ D d − ) ∈ CH h (Alb ∨ X × Alb X ) Q induces the map n · θ ∗ : H (Alb X /k ) → H (Alb ∨ X /k ). However, we have θ ◦ ϑ = [ n ] Alb X whichimplies ϑ ◦ θ = [ n ] Alb ∨ X , hence( θ × id Alb X ) ∗ ( ϑ × id Alb X ) ∗ c ( P ) = ([ n ] Alb ∨ X × id Alb X ) ∗ c ( P ) = n · c ( P ) . On the other hand, the construction of θ in Proposition 2.5 implies that the correspondence c ( P ) . (Alb ∨ X × ( α X ) x ∗ D d − ) exactly induces the restriction θ ∗ : H (Alb X /k ) → H (Alb ∨ X /k ). Thus,(1) is proved.For (2), the assertion that u ∗ D is almost ample follows directly from Definition 2.8. For the rest,we may assume that k ( Y ) /k ( X ) is Galois. In fact, by resolution of singularity, we can always findanother generically finite dominant morphism of connected proper smooth schemes v : Z → Y suchthat k ( Z ) /k ( X ) is Galois. Now if (2) holds for v and u ◦ v , then it holds for u . Thus, we may assumethat k ( Y ) /k ( X ) is Galois with the Galois group Γ.Choose two arbitrary points y, y ∨ ∈ Y ( k ), and put x := u ( y ) and x ∨ := u ( y ∨ ). Put β X := ( α X ) x ∨ × ( α X ) x : X × X → Alb X × Alb X , and similarly for β Y . We choose a homomorphism ϑ X : Alb X → Alb ∨ X (resp. ϑ Y : Alb Y → Alb ∨ Y ) suchthat θ X,D ◦ ϑ X = [ n X ] Alb X (resp. θ Y,u ∗ D ◦ ϑ Y = [ n Y ] Alb Y ). Put E X := β ∗ X (id Alb X × ϑ X ) ∗ c ( P X ) . ( X × D d − ) ,E Y := β ∗ Y (id Alb Y × ϑ Y ) ∗ c ( P Y ) . ( Y × u ∗ D d − ) , where P X (resp. P Y ) is the Poincaré line bundle on Alb X × Alb ∨ X (resp. Alb Y × Alb ∨ Y ). Then theformula in (2) follows from the symmetry of Poincaré bundles, and the identity(id Y × u ) ∗ E Y = n Y n X · ( u × id X ) ∗ E X in CH d ( Y × X ) Q . By the projection formula, this in turn follows from(id Y × u ) ∗ β ∗ Y (id Alb Y × ϑ Y ) ∗ c ( P Y ) = n Y n X · ( u × id X ) ∗ β ∗ X ( ϑ X × id Alb X ) ∗ c ( P X ) . (2.6) Consider the following diagram Y × Y β Y (cid:47) (cid:47) id Y × u (cid:15) (cid:15) Alb Y × Alb Y id Alb Y × ϑ Y (cid:47) (cid:47) id Alb Y × Alb u (cid:15) (cid:15) Alb Y × Alb ∨ Y Y × X β (cid:47) (cid:47) u × id X (cid:15) (cid:15) Alb Y × Alb X id Alb Y × ϑ X (cid:47) (cid:47) Alb u × id Alb X (cid:15) (cid:15) Alb Y × Alb ∨ X id Alb Y × Alb ∨ u (cid:79) (cid:79) Alb u × id Alb ∨ X (cid:15) (cid:15) X × X β X (cid:47) (cid:47) Alb X × Alb X id Alb X × ϑ X (cid:47) (cid:47) Alb X × Alb ∨ X where β := ( α Y ) y × ( α X ) x ∨ . Note that squares involving the dash arrow do not necessarily commute.By the isomorphism (Alb u × id Alb ∨ X ) ∗ P X ’ (id Alb Y × Alb ∨ u ) ∗ P Y , (2.6) is equivalent to(id Y × u ) ∗ β ∗ Y (id Alb Y × ϑ Y ) ∗ c ( P Y ) = n Y n X · β ∗ (id Alb Y × ϑ X ) ∗ (id Alb Y × Alb ∨ u ) ∗ c ( P Y ) . (2.7)By the projection formula, (2.7) is equivalent to thatdeg u · n X · (id Alb Y × ϑ Y ) ∗ c ( P Y ) − n Y · (id Alb Y × (Alb ∨ u ◦ ϑ X ◦ Alb u )) ∗ c ( P Y )is contained in the kernel of (id Y × u ) ∗ ◦ β ∗ Y . Now the Galois group Γ acts on Alb Y via homomorphisms γ : Alb Y → Alb Y for γ ∈ Γ. We have a similar action on Alb ∨ Y by duality, and the homomorphism ϑ Y is Γ-equivariant since the divisor u ∗ D is Γ-invariant. For a line bundle L on Alb Y × Alb Y , we havethe trace line bundle L Γ := O γ ∈ Γ (id Alb Y × γ ) ∗ L . Moreover, if L Γ is torsion, then c ( L ) is in the kernel of (id Y × u ) ∗ ◦ β ∗ Y . We define similarly L Γ forline bundles L on Alb Y × Alb ∨ Y .In all, (2.7) will follow from the following claim: For P := P Y on Alb Y × Alb ∨ Y , the line bundle(id Alb Y × ϑ Y ) ∗ P ⊗ n X Γ ⊗ (id Alb Y × (Alb ∨ u ◦ ϑ X ◦ Alb u )) ∗ P ⊗− n Y is torsion. An easy diagram chasing implies that the claim will follow if we can show that the twohomomorphisms Alb ∨ u ◦ ϑ X ◦ Alb u ◦ [ n Y ] Alb Y , X γ ∈ Γ γ ∨ ◦ ϑ Y ◦ [ n X ] Alb Y (2.8)from Alb Y to Alb ∨ Y coincide. However, this can be checked on the level point as the base field hascharacteristic zero. Then we have a homomorphism u ∗ : Alb ∨ Y ( k ) → Alb ∨ X ( k ) induced by pushforwardof divisors along u : Y → X as we have Alb ∨ X ’ Pic X and Alb ∨ Y ’ Pic Y . By the definition ofpushforward, the diagram Alb ∨ X ( k ) Alb ∨ u (cid:37) (cid:37) Alb ∨ Y ( k ) u ∗ (cid:57) (cid:57) P Γ γ ∨ (cid:47) (cid:47) Alb ∨ Y ( k )commutes; and by the projection formula, the diagramAlb ∨ Y ( k ) θ Y,u ∗ D (cid:47) (cid:47) u ∗ (cid:15) (cid:15) Alb Y ( k ) Alb u (cid:15) (cid:15) Alb ∨ X ( k ) θ X,D (cid:47) (cid:47)
Alb X ( k )commutes as well. The two diagrams imply the coincidence of the two homomorphisms in (2.8). Thus,the claim hence (2) follow. (cid:3) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 21 Algebraic cycles and height pairings
In this section, we make some preparation for algebraic cycles and height pairings for generalvarieties. In Subsection 3.1, we review the notion of algebraic cycles and correspondences. In Subsec-tion 3.2, we review the construction of the Beilinson–Bloch height pairing and introduce our variant,namely, the Beilinson–Bloch–Poincaré height pairing. In Subsection 3.3, we discuss the constructionof some Künneth–Chow projectors for curves and surfaces, which can be used in the modified diagonal∆ z X K later.Let k be a field of characteristic zero. We work in the category Sch /k .3.1. Cycles and correspondences.
Consider a proper smooth scheme X ∈ Sch /k of pure dimension d . Let Z i ( X ) (resp. CH i ( X )) be the abelian group of cycles (resp. Chow cycles) on X of codimension i , with a natural surjective map Z i ( X ) → CH i ( X ). For example, we have the diagonal cycle∆ r X ∈ Z ( r − d ( X r )for r (cid:62) r : X → X r . We write ∆ X for ∆ X for simplicity.We have the de Rham cycle class mapcl dR : CH i ( X ) Q → H i dR ( X/k ) , whose kernel we denote by CH i ( X ) Q . By various comparison theorems, CH i ( X ) Q coincides with thekernel of the Betti cycle class map cl B ,τ : CH i ( X ) Q → H i B ,τ ( X, Q )for every embedding τ : k , → C , and the ‘ -adic cycle class mapcl ‘ : CH i ( X ) Q → H i ´et ( X k ac , Q ‘ ( i ))for every rational prime ‘ . Moreover, by the Hochschild–Serre spectral sequence, we obtain the ‘ -adicAbel–Jacobi map AJ ‘ : CH i ( X ) Q → H ( k, H i − ( X k ac , Q ‘ ( i ))) . Definition 3.1.
We put CH i ( X ) Q := T ‘ ker AJ ‘ as a subspace of CH i ( X ) Q , andCH i ( X ) R := CH i ( X ) Q ⊗ Q R, CH i ( X ) \R := (CH i ( X ) Q / CH i ( X ) Q ) ⊗ Q R for every ring R containing Q . We call elements in CH i ( X ) \R natural cycles (of codimension i ). Remark . In [Be˘ı87], Beilinson conjectures that ker AJ ‘ = { } for every ‘ if k is a number field and X is projective, which implies CH i ( X ) R = CH i ( X ) \R .We introduce the following definition, which will be used in Section 5. Definition 3.3.
We say that a formal series P j c j Z j with c j ∈ C and Z j ∈ Z i ( X ) is Chow convergent if the image of { Z j } j in CH i ( X ) C generates a finite dimensional subspace, and the induced formalseries in this finite dimensional space is absolutely convergent. We denote by CZ i ( X ) the set of Chowconvergent formal series in Z i ( X ), which is a complex vector space and admits a natural complexlinear map CZ i ( X ) → CH i ( X ) C .Now we recall the notation of correspondences. A (Chow self-)correspondence of X is an element z ∈ CH d ( X × X ). It induces a graded map z ∗ : d M i =0 CH i ( X ) → d M i =0 CH i ( X ) sending α to p ∗ ( z. p ∗ α ), where p i : X × X → X is the projection to the i -th factor, a conventionrecalled from Subsection 1.7. On the level of various cohomology, it induces graded mapscl ∗ dR ( z ) : d M i =0 H i dR ( X/k ) → d M i =0 H i dR ( X/k ) , cl ∗ B ,τ ( z ) : d M i =0 H i B ,τ ( X, Q ) → d M i =0 H i B ,τ ( X, Q ) , and cl ∗ ‘ ( z ) : d M i =0 H i ´et ( X k ac , Q ‘ ( j )) → d M i =0 H i ´et ( X k ac , Q ‘ ( j ))for every prime ‘ and j ∈ Z . They are compatible with various comparison theorems and cycle classmaps. When we regard the diagonal ∆ X ⊆ X × X as a correspondence, we usually write it as id X .3.2. Beilinson–Bloch–Poincaré height pairing.
We review the theory of height pairing betweencycles of Beilinson and Bloch. Now suppose that k is a number field. Consider a projective smoothscheme X ∈ Sch /k of pure dimension d . Beilinson [Be˘ı87] and Bloch [Blo84] have defined, via twoapproaches, a bilinear pairing h , i BB X : CH i ( X ) Q × CH d +1 − i ( X ) Q → C . However, both approaches relies on some assumptions that are still unknown even today.We review briefly one construction of Beilinson: For z ∈ CH i ( X ) Q and z ∈ CH d +1 − i ( X ) Q , wechoose their representatives Z ∈ Z i ( X ) Q and Z ∈ Z d +1 − i ( X ) Q that have disjoint support. Forevery place v of k , there is a local index h Z , Z i X v on X v := X k v . For v archimedean, this isdefined in [Be˘ı87, Section 3] via the potential theory on Kähler manifolds; it is unconditional. For v nonarchimedean such that X v has good reduction, this is defined via intersection theory on anarbitrary smooth model of X v over O k v . For general nonarchimedean v , the definition of h Z , Z i X v isconditional: Choose a rational prime ‘ not underlying v and an isomorphism ι ‘ : C ∼ −→ Q ac ‘ such thatH i ´et ( X k ac v , Q ‘ ) satisfies the Monodromy–Weight conjecture, which implies that the cycle class of Z inthe absolute étale cohomology H i ´et ( X v , Q ‘ ( i )) vanishes (same for Z ). Then one can define h Z , Z i X v as a “link pairing” valued in Q ‘ followed by the map ι − ‘ . See [Be˘ı87, Section 2.1] for more details.We then define h Z , Z i BB X := X v r ( v ) · h Z , Z i X v , (3.1)where the sum is taken over all places v of k , and r ( v ) equals 1, 2, and log q v when v is real, complex,and nonarchimedean (with q v the residue cardinality of k v ), respectively.For every intermediate ring Q ⊆ R ⊆ C , we obtain a pairing h , i BB X : CH i ( X ) R × CH d +1 − i ( X ) R → C via R -bilinear extension. Beilinson conjectures that the pairing h , i BB X is independent of ‘ and theisomorphism ι ‘ . Remark . As we have mentioned, if X v satisfies the Monodromy–Weight conjecture for every nonar-chimedean place v of k (for example, when X is a product of curves, surfaces, or abelian varieties),then the Beilinson–Bloch height pairing h , i BB X is unconditionally defined (but may a priori dependon the choices of ‘ and ι ‘ ). When X is a curve, the Beilinson–Bloch height pairing coincides with theNéron–Tate height pairing up to −
1. When X is an abelian variety, the Beilinson–Bloch height pairingcoincides with the pairing defined in [Kün01]. In particular, in these two cases, the independence of ‘ and ι ‘ is known. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 23
Lemma 3.5.
Suppose that the Beilinson–Bloch height pairing is defined for X . Take Z ∈ CH ( X ) R for some intermediate ring Q ⊆ R ⊆ C . Then we have h Z , Z.Z i BB X = 0 for every Z ∈ CH i ( X ) R and Z ∈ CH d − i ( X ) R .Proof. We fix an embedding k , → C . Since Z is homologically trivial, it is algebraically equivalentto zero; so is Z.Z . By [Be˘ı87, Lemma 4.0.7], it suffices to show that the image of Z.Z under thecomplex Abel–Jacobi map CH d − i +1 ( X ) R → J d − i +1 ( X C ) R is zero, where J d − i +1 ( X C ) is the ( d − i +1)-thintermediate Jacobian of X C (as an abelian group). We replace Z and Z by their representatives inZ ( X ) R and Z d − i ( X ) R with proper intersection. Since Z is homologically trivial, we may choose a(singular) chain C Z of (real) dimension 2 i + 1 with boundary Z . Then Z.C Z is a chain of dimension2 i − Z.Z . It suffices to show that Z Z.C Z ω = 0for every closed differential form ω whose class belongs to the Hodge filtration Fil i H i − ( X, C ). Since(the underlying cycle of) Z is homologous to zero, we can take a (1 , η such that d η is the classrepresented by Z by the ∂∂ -lemma from Hodge theory. Thus, Z Z.C Z ω = Z C Z d η ∧ ω = Z C Z d( η ∧ ω ) = Z Z η ∧ ω = 0 , in which the last equality follows as we may take a representative of ω as a sum of ( p, i − − p )-formswith p (cid:62) i . The lemma follows. (cid:3) Recall that if A is an abelian variety over k of dimension h (cid:62)
1, and Q ⊆ R ⊆ C is an immediatering, then we also have the Néron–Tate (bilinear) height pairing h , i NT A : A ( k ) R × A ∨ ( k ) R → C . Composing with the Albanese maps CH h ( A ) R → A ( k ) R and CH h ( A ∨ ) R → A ∨ ( k ) R , we may regardthe above pairing as a map h , i NT A : CH h ( A ) R × CH h ( A ∨ ) R → C . (3.2) Remark . The Néron–Tate height pairing (3.2) is related to the Beilinson–Bloch height pairing viathe following commutative diagram:CH h ( A ) R × CH ( A ) R (cid:15) (cid:15) h , i BB A (cid:47) (cid:47) C CH h ( A ) R × CH h ( A ∨ ) R −h , i NT A (cid:47) (cid:47) C in which CH ( A ) R → CH h ( A ∨ ) R is the tautological map.Now we will combine the Beilinson–Bloch height pairing on X and the Néron–Tate height pairingon A to give a height pairing h , i BBP
X,A : CH h + i ( X × A ) R × CH h + d − i ( X × A ∨ ) R → C using the Poincaré bundle, for every intermediate ring Q ⊆ R ⊆ C . The process is easy: Let P be thePoincaré line bundle on A × A ∨ . We have projection morphisms p : X × A × A ∨ → X × A, p : X × A × A ∨ → X × A ∨ , and recall the Fourier–Mukai transform ℘ : CH h + d − i ( X × A ∨ ) R → CH d +1 − i ( X × A ) R sending z to p ∗ (( X × c ( P )) . p ∗ z ). We then define h z , z i BBP
X,A := h z , ℘ ( z ) i BB X × A . Definition 3.7.
We call h , i BBP
X,A the
Beilinson–Bloch–Poincaré height pairing for (
X, A ). Remark . The Beilinson–Bloch–Poincaré height pairing is unconditionally defined if X v satisfiesthe Monodromy–Weight conjecture for every nonarchimedean place v of k (but may a priori dependon the choices of ‘ and ι ‘ ). When X = Spec k (resp. h = 1, that is, A is an elliptic curve hencecanonically isomorphic to A ∨ ), the Beilinson–Bloch–Poincaré height pairing for ( X, A ) reduces to theNéron–Tate height pairing (3.2) for A up to − X × A ). Remark . The Beilinson–Bloch–Poincaré height pairing can be defined more generally for an abelianscheme A of relative dimension h (cid:62) X as a pairing h , i BBP A : CH h + i ( A ) R × CH h + d − i ( A ∨ ) R → C such that h z , z i BBP A = h z , p ∗ ( c ( P ) . p ∗ z ) i BB A , where p : A × X A ∨ → A and p : A × X A ∨ → A ∨ areprojection morphisms, and P is the relative Poincaré bundle on A × X A ∨ .3.3. Künneth–Chow projectors.
In this subsection, we will construct some Künneth–Chow pro-jectors, which will be used in Subsection 4.4. The readers may skip it at this moment.Consider a proper smooth scheme X ∈ Sch /k of pure dimension d . PutH evendR ( X/k ) := M i even H i dR ( X/k ); H odddR ( X/k ) := M i odd H i dR ( X/k ) . Definition 3.10.
We say that a correspondence z ∈ CH d ( X × X ) Q is an even (resp. odd ) projector ifthe map cl ∗ dR ( z ) is the projection map to H evendR ( X/k ) (resp. H odddR ( X/k )).We introduce the following convention: for a zero cycle D on X , we regard its degree deg D as afunction on π ( X ). The following two lemmas and one definition will be used later. Lemma 3.11.
We have(1) Suppose that d = 1 . Let D ∈ CH ( X ) Q be a cycle such that deg D is nonzero on every connectedcomponent of X . Then z X,D := ∆ X − D ( X × D + D × X ) is an odd projector for X .(2) Suppose that d = 2 . Let D ∈ CH ( X ) Q be a cycle that is an almost ample divisor (Definition2.8). Then z X,D := e X,D + e t X,D is an odd projector for X , where e X,D is the correspondence in Proposition 2.12 and e t X,D isits transpose.Proof.
Part (1) is obvious. Part (2) follows from Proposition 2.12(1). (cid:3)
Lemma 3.12.
Let z be an odd projector for X . Then(1) the image of the induced map z ∗ : CH i ( X ) Q → CH i ( X ) Q is contained in CH i ( X ) Q ;(2) the cycle z × z × z + z × (∆ X − z ) × (∆ X − z ) + (∆ X − z ) × z × (∆ X − z ) + (∆ X − z ) × (∆ X − z ) × z is an odd projector for X × X × X .Proof. For (1), since cl dR (Im z ∗ ) ⊆ Im(cl ∗ dR ( z )) = H odddR ( X/k ), we know that the image of z ∗ is con-tained in CH i ( X ) Q .For (2), note that if z is an odd projector, then ∆ X − z is an even projector. Thus, (2) follows fromthe Künneth decomposition for the algebraic de Rham cohomology. (cid:3) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 25
Definition 3.13.
Let z be an odd projector for X . We definepr [3] z : CH i ( X × X × X ) Q → CH i ( X × X × X ) Q to be the map induced by the odd projector for X × X × X as in Lemma 3.12(2).4. Fourier–Jacobi cycles and derivative of L -functions In this section, we construct Fourier–Jacobi cycles and state our main conjectures. In Subsection4.1, we construct the the category of CM data for a conjugate symplectic automorphic character µ ofweight one. In Subsection 4.2, we introduce our Shimura varieties and study their Albanese varieties.In Subsection 4.3, we construct Fourier–Jacobi cycles and show that they are homologically trivial.In Subsection 4.4, we propose various versions of the arithmetic Gan–Gross–Prasad conjecture forU( n ) × U( n ).Let F be a totally real number field of degree d (cid:62)
1, and
E/F a totally imaginary quadraticextension. We denote by • c the nontrivial Galois involution of E over F , • E − the subgroup of E consisting of e satisfying e + e c = 0, and E the subgroup of E × consisting of e satisfying ee c = 1, • by µ E/F : F × \ A × F → C × the quadratic character associated to E/F via global class fieldtheory, • E v the base change E ⊗ F F v for every place v of F , • Φ F the set of real embeddings of F , Φ E the set of complex embeddings of E , and π : Φ E → Φ F the projection map given by restriction.Recall that a CM type (of E ) is a subset Φ of Φ E such that π induces a bijection from Φ to Φ F .In this section, we work in the category Sch /E .4.1. Motives for CM characters.
In this subsection, we generalize some constructions in [Den89,Section 2].
Definition 4.1.
We say that an automorphic character µ : E × \ A × E → C × is conjugate self-dual if µ istrivial on N A E / A F A × E . We say that µ is conjugate orthogonal (resp. conjugate symplectic ) if µ | A × F = 1(resp. µ | A × F = µ E/F ). Remark . A conjugate self-dual automorphic character is necessarily strictly unitary (DefinitionB.2). It is either conjugate orthogonal or conjugate symplectic, but not both.For a conjugate symplectic (resp. conjugate orthogonal) automorphic character µ , there exist a CMtype Φ µ and a unique tuple w µ = ( w τ ) τ ∈ Φ of odd (resp. even) nonnegative integers such that thecomponent µ τ : ( E ⊗ F,τ R ) × → C × is the character z arg( z ) − w τ , where we have identified ( E ⊗ F,τ R ) × with C × via the unique element τ ∈ Φ µ above τ . If w µ does notcontain 0, then Φ µ is also unique. In what follows, we put µ c := µ ◦ c . Definition 4.3.
Let µ be a conjugate self-dual automorphic character.(1) We call w µ the weight of µ . If w µ is a constant m , then we say that µ is of weight m .(2) If w µ does not contain zero, then we call Φ µ the CM type of µ . Furthermore, we denote by M µ ⊆ C the reflex field of ( E, Φ µ ), with the induced CM type Ψ µ .Now let µ be a conjugate symplectic automorphic character. Then it is not algebraic. We put µ alg := µ · | | − / E , which is then algebraic. Denote by M µ ⊆ C the subfield generated by values µ alg ( x ) for x ∈ ( A ∞ E ) × ,which is a number field containing M µ . Remark . It is clear that µ c is conjugate symplectic of the same weight as µ . Moreover, we have M µ c = M µ , M µ c = M µ , and that Ψ µ c is the opposite CM type of Ψ µ . Definition 4.5.
Let µ be a conjugate symplectic automorphic character of weight one.(1) We denote by η µ : Res M µ / Q G m → Res E/ Q G m the reciprocity map, and put η µ := η µ ◦ N M µ /M µ : Res M µ / Q G m → Res E/ Q G m . (2) We define a CM data for µ to be a quadruple D µ = ( A µ , i µ , λ µ , r µ ) in which • A µ is an abelian variety over E ; • i µ : M µ → End E ( A µ ) Q is a CM structure such that – for every x ∈ M µ , the determinant of the action of i µ ( x ) on the E -vector spaceLie E ( A µ ) equals η µ ( x ); – the associated CM character of A µ with respect to the inclusion M µ , → C coincideswith µ alg ; • λ µ : A µ → A ∨ µ is a polarization satisfying λ ◦ i µ ( x ) = i µ ( x ) ∨ ◦ λ for every x ∈ M µ ; • r µ : M µ ⊗ Q E → H dR1 ( A µ /E ) is an isomorphism of M µ ⊗ Q E -modules satisfying that thereexist an element β ∈ M µ and an isomorphism c : H dR2 dim A µ ( A µ /E ) → E of E -modules,such that for every x, y ∈ M µ ⊗ Q E , we have c ( h r µ ( x ) , r µ ( y ) i λ ) = Tr M µ ⊗ Q E/E ( xβy ), where h , i λ : H dR1 ( A µ /E ) × H dR1 ( A µ /E ) → H dR2 dim A µ ( A µ /E ) denotes the pairing induced by λ .(3) We denote by A ( µ ) the category of CM data for µ , whose objects are CM data D µ , andmorphisms from D µ = ( A µ , i µ , λ µ , r µ ) to D µ = ( A µ , i µ , λ µ , r µ ) are isogenies ϕ : A µ → A µ satisfying ϕ ◦ i µ ( x ) = i µ ( x ) ◦ ϕ for every x ∈ M µ , ϕ ∨ ◦ λ µ ◦ ϕ = cλ µ for some element c ∈ Q × ,and r µ = ϕ ∗ ◦ r µ .(4) From a CM data D µ = ( A µ , i µ , λ µ , r µ ) for µ , we define another quadruple D ∨ µ = ( A ∨ µ , i ∨ µ , λ ∨ µ , r ∨ µ )in which ( A ∨ µ , λ ∨ µ ) is simply the dual of ( A µ , λ µ ); i ∨ µ is defined by the formula i ∨ µ ( x ) = i µ ( x ) ∨ for x ∈ M µ ; and r ∨ µ := ( λ µ ) ∗ ◦ r µ . Proposition 4.6.
Let µ be as in Definition 4.5.(1) The category A ( µ ) is a nonempty and connected partially ordered set.(2) The assignment sending D µ to D ∨ µ induces an equivalence A ( µ ) op ∼ −→ A ( µ c ) of categories.Proof. For (1), we first show that A ( µ ) is nonempty. Take a finite abelian extension E /E such thatthe character µ alg := µ alg ◦ N E /E satisfies [Shi71, (1.12) & (1.13)] for ( K , Φ ) = ( E, Φ µ ), k = E ,( K, Φ) = ( M µ , Ψ µ ) with ∞ the archimedean place of M µ induced by the inclusion M µ , → C , and a = O M µ . For example, we may take an open compact subgroup U of A ∞ E on which µ (hence µ alg ) istrivial and take E to be the abelian extension corresponding to U via the global class field theory. ByCasselman’s theorem [Shi71, Theorem 6], we have a pair ( A , i ) where A is an abelian variety over E and i : M µ → End E ( A ) Q is a CM structure such that • the determinant of the action of i ( x ) on the E -vector space Lie E ( A ) is η µ ( x ) for every x ∈ M µ ; • the associated CM character of A with respect to the inclusion M µ , → C coincides with µ alg .By [Shi71, Lemma 1 & Lemma 2], A is simple hence i is an isomorphism. The same argument in[Den89, (2.1)] implies that there is an isogeny factor A µ of the abelian variety Res E /E A over E together with a CM structure i µ : M µ → End E ( A µ ) Q satisfying the conditions in the proposition. Inother words, we have obtained the part ( A µ , i µ ) for a CM data for µ . By [Shi71, Theorem 5], we knowthe existence of λ µ . The existence of r µ is obvious. Thus, we obtain an object D µ = ( A µ , i µ , λ, r µ ) of A ( µ )The connectedness of A ( µ ) also follows from [Shi71, Theorem 5]. Finally, the compatibility condition r µ = ϕ ∗ ◦ r µ ensures that A ( µ ) is a partially ordered set.Part (2) is clear from the definition. (cid:3) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 27
Remark . Let µ be as in Definition 4.5. Although we will not use in the main body of the article,we propose the definition of the motive for µ , denoted by L µ , as a Grothendieck motive. For a CMdata D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ), let h ( A µ , M µ ) be the Picard motive of A µ with coefficient M µ ,and h ( D µ ) the direct summand of h ( A µ , M µ ) on which the induced action of M µ via i µ coincideswith the underlying linear action of M µ . The assignment D µ h ( D µ ) is a functor from A ( µ ) tothe category of Grothendieck motives over E . We put L µ := lim −→ D µ ∈A ( µ ) h ( D µ ), which is of rank 1with coefficient M µ . It follows from Proposition 4.6(1) that the canonical map L µ → h ( D µ ) is anisomorphism for every D µ ∈ A ( µ ).To end this subsection, we construct some canonical projector on A µ , which will be used in Subsec-tion 4.3. Let µ be as in Definition 4.5. Denote by I µ the set of all complex embeddings of M µ . We takea CM data D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) for µ . For every element x ∈ M µ such that i µ ( x ) ∈ End E ( A µ ),we denote by h x i the correspondence A µ i µ ( x ) ←−−− A µ id −→ A µ of A µ . In particular, h x i ∗ = i µ ( x ) ∗ . For every τ : E , → C and 0 (cid:54) i (cid:54) [ M µ : Q ], we have a canonicaldecomposition H [ M µ : Q ] − i B ,τ ( A µ , C ) = M I ⊆ I µ , | I | = i H [ M µ : Q ] − i B ,τ ( A µ , C ) I where H [ M µ : Q ] − i B ,τ ( A µ , C ) I denotes the subspace on which cl ∗ B ,τ ( h x i ) acts by Q ι ∈ I ι ( x ) for every x ∈ M µ satisfying i µ ( x ) ∈ End E ( A µ ). Let T B ,τ µ be the endomorphism of L i H i B ,τ ( A µ , C ) such that • the restriction T B ,τ µ | H i B ,τ ( A µ , C ) is zero if i = [ M µ : Q ] − • the restriction T B ,τ µ | H [ M µ : Q ] − ,τ ( A µ , C ) is the canonical projection to the direct summandH [ M µ : Q ] − ,τ ( A µ , C ) I , where I ⊆ I µ is the subset consisting only of the inclusion M µ , → C . Definition 4.8.
Let I ⊆ I µ be a subset.(1) We say that x ∈ M µ is an I -generator if x generates the field M µ with i µ ( x ) ∈ End E ( A µ ) suchthat Y ι ∈ I ι ( x ) = Y ι ∈ J ι ( x )for every J ⊆ I µ other than I .(2) For an I -generator x , we put T xµ := Y J = I h x i − Q ι ∈ J ι ( x ) Q ι ∈ I ι ( x ) − Q ι ∈ J ι ( x ) ∈ CH [ M µ : Q ] / ( A µ × A µ ) C , We now choose an I -generator x . It is easy to see that T xµ lies in CH [ M µ : Q ] / ( A µ × A µ ) M µ , andmoreover cl ∗ B ,τ ( T xµ ) = T B ,τ µ . In particular, the numerical equivalence class of T xµ is independent of thechoice of x , which we denote by T num µ . Applying the main theorem of [O’S11] to A µ × A µ , we knowthat T num µ has a canonical lift in CH [ M µ : Q ] / ( A µ × A µ ) M µ , which we denote by T can µ . Definition 4.9.
We call T can µ ∈ CH [ M µ : Q ] / ( A µ × A µ ) M µ the canonical projector of A µ . Lemma 4.10.
For every τ : E , → C , we have cl ∗ B ,τ ( T can µ ) = T B ,τ µ .Proof. By part (iii) of the main theorem of [O’S11], we know that T can µ hence T can µ − T xµ commutes with h y i for all y ∈ M µ such that i µ ( y ) ∈ End E ( A µ ). Now we show that T can µ − T xµ is homologically trivial. The author states the theorem with coefficient Q . However, by [O’S11, Corollary 6.2.6], one may replace Q by anyfield of characteristic zero, for example, M µ . If not, then there exist some 0 (cid:54) i (cid:54) [ M µ : Q ] and a set I ⊆ I µ with | I | = [ M µ : Q ] − i such that therestriction cl ∗ B ,τ ( T can µ − T xµ ) | H i B ,τ ( A, C ) I is the canonical embedding.Now we take an I c -generator y ∈ M µ , where I c := I µ \ I . Then the cycle T yµ (Definition 4.8) hasnonzero intersection number with T can µ − T xµ . This contradicts with the fact that T can µ − T xµ is numericallytrivial. Thus, the lemma follows. (cid:3) Albanese of unitary Shimura varieties.
Let n (cid:62) V be a totally definiteincoherent hermitian space over A E of rank n (Definition C.3). We distinguish between two cases: • Noncompact Case : d = 1, and either n (cid:62) n = 2 and the hermitian space V ⊗ A Q p isisotropic for every rational prime p ; • Compact Case : if it is not in the Noncompact Case.Let G := U( V ) be the unitary group of V , which is a reductive group over A F . Let { Sh( V ) K } K bethe projective system of Shimura varieties for V indexed by sufficiently small open compact subgroupsof G ( A ∞ F ) (Definition C.6). Every scheme Sh( V ) K is smooth, quasi-projective, and of dimension n − E ; it is projective if and only if we are in the Compact Case. In all cases, we have the compactifiedShimura variety f Sh( V ) K (Definition C.8). Put X K := f Sh( V ) K for short. Then { X K } K is a projective system of smooth projective schemes in Sch /E of dimension n −
1. For K ⊆ K , we denote the transition morphism by u K K : X K → X K , which is a genericallyfinite dominant morphism. Put X ∞ := lim ←− K X K .We denote by A K the Albanese variety Alb X K of X K (Definition 2.3) for short, and by α K := α X K : ∇ X K → A K (4.1)the Albanese morphism (see Definition 2.1 for the meaning of ∇ ). By functoriality, we obtain aprojective system { A K } K . Put A ∞ := lim ←− K A K , which is an abelian group pro-object in Sch /E . Then the Hecke correspondences provide a homomor-phism G ( A ∞ F ) → Aut E ( A ∞ ).To study isogeny factors of A K , it suffices to study the L -function of H (( A K ) E ac , Q ac ‘ ) by Faltings’isogeny theorem. We start from describing its Betti cohomology H ,τ ( A K , C ). For every embedding τ : E → C , put H ,τ ( A ∞ , C ) := lim −→ K H ,τ ( A K , C ) , which is an admissible representation of G ( A ∞ F ). To study this representation, we need to recalloscillator representations of unitary groups. Definition 4.11. An adèlic oscillator triple is a triple ( µ, ε, χ ) consisting of • a conjugate symplectic automorphic character (Definition 4.1) µ = ⊗ µ v : E × \ A × E → C × (whosevalue is necessarily in C ), • a collection ε = ( ε v ∈ E −× v / N E v /F v E × v ) v for every nonarchimedean place v of F such that ε v ∈ O × E v N E v /F v E × v for all but finitely many v , and • an automorphic character χ = ⊗ χ v : E \ ( A ∞ E ) → C × (whose value is necessarily in C ).For an adèlic oscillator triple ( µ, ε, χ ), the local oscillator representation ω ( µ v , ε v , χ v ) of G ( F v ) intro-duced in Subsection D.1 is unramified for all but finitely many v . Thus, it makes sense to define the adèlic oscillator representation attached to ( µ, ε, χ ) ω ( µ, ε, χ ) := O v ω ( µ v , ε v , χ v )which is an irreducible admissible representation of G ( A ∞ F ). OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 29
Definition 4.12.
In an adèlic oscillator triple ( µ, ε, χ ), we say that ε is µ -admissible if there existssome e ∈ E ×− such that • ε v = e N E v /F v E × v for every nonarchimedean place v of F ; and • τ ( e ) has negative imaginary part for every τ ∈ Φ µ .It is clear by Remark 4.4 that ε is µ -admissible if and only if − ε is µ c -admissible. Proposition 4.13.
Suppose that n (cid:62) . Then for every embedding τ : E → C , there is an isomor-phism H ,τ ( A ∞ , C ) ’ M ( µ,ε,χ ) ω ( µ, ε, χ ) of C [ G ( A ∞ F )] -modules, where the direct sum is taken over all adèlic oscillator triples in which µ is ofweight one and ε is µ -admissible.Proof. By Lemma 2.4, we have a canonical isomorphismH ,τ ( A ∞ , C ) ’ H ,τ ( X ∞ , C ) := lim −→ K H ,τ ( X K , C )of C [ G ( A ∞ F )]-modules. We regard E as a subfield of C via a fixed embedding τ ∈ Φ E and put τ := τ | F . We choose a CM type Φ that contains τ . Take a hermitian space V that is τ -nearby to V (Definition C.4). Put G := Res F/ Q U(V) and h := h [ V , Φ for short. Then by Propositions C.5 and(C.2), we have an isomorphism H ,τ ( X ∞ , C ) ’ lim −→ K H ( f Sh(G , h) K , C )of C [ G ( A ∞ F )]-modules. By [MR92, Lemma 1], for every K , there is a canonical isomorphismH ( f Sh(G , h) K , C ) ’ IH (Sh(G , h) K , C ) , where the right-hand side is the complex analytic intersection cohomology of the Baily–Borel com-pactification Sh(G , h) K of Sh(G , h) K . Combining (D.2) and (D.1), we have an isomorphismH ,τ ( A ∞ , C ) ’ M π m disc ( π )H ( g , K G ; π ∞ ) ⊗ π ∞ of C [ G ( A ∞ F )]-modules. We say that an irreducible admissible representation π of G( A ) contributesto the Albanese if m disc ( π ) > ( g , K G ; π ∞ ) = { } . We determine all such π together with thevalue m disc ( π ). By the proof of [BMM16, Proposition 13.4] (with m = n , p = n − q = 1, a + b = 1),we know that there exists a strictly unitary automorphic character (Definition B.2) µ : E × \ A × E → C × such that the partial L -function L S ( s, π × µ ) has a simple pole at s with s (cid:62) n where S is a finite setof places of F containing all archimedean ones and such that for v S , both π v and µ v are unramified. We separate the discussion into two cases.
Case 1.
Suppose that π contributes to the Albanese and m cusp ( π ) >
0. Let V π be a cuspidalrealization of π (Definition B.1). By Corollary B.5, s is either n +12 or n , not both. If s = n +12 ,then Theorem B.4 implies that Θ W( µ,ν ) , V ( V π ) = { } , where W is the zero skew-hermitian space. ByCorollary B.6(1), V π is a character hence H ( g , K G ; π ∞ ) = { } , which is a contradiction. Thus, wemust have s = n . By Theorem B.4, we have a one-dimensional skew-hermitian space W such thatΘ W( µ,ν ) , V ( V π ) = { } which is cuspidal. By Corollary B.6(1), we have V π = Θ − W( µ − ,ν − ) ( π W ). Note that thecentral character χ of π satisfies χ ∞ = 1. In other words, there is a unique element e ∈ E −× / N E/F E × determined by W such that π ∞ ’ ω ( µ, ε e , χ ) where ε e is the collection given by e . To determine π ∞ , we suppose that Φ F = { τ = τ, τ , . . . , τ d } and Φ = { τ +1 , . . . , τ + d } with π ( τ + i ) = τ i . Using Φ, weobtain an isomorphism G R ’ U( n − , R × U( n, R × · · · × U( n, R , and accordingly a decomposition Note that π is assumed to be cuspidal in the statement of [BMM16, Proposition 13.4]. However, this step works for π discrete. Here, we have replaced µ by its inverse to match notation in the statement of the proposition. π ∞ = ⊗ di =1 π ∞ i . Under the notation from Subsection D.1, we have π ∞ ’ ω m , ± , n − , and π ∞ i ’ ω m i , ± , n, for i (cid:62)
2, where ( m , . . . , m d ) is the weight of µ and the sign in the parameter is the sign of i − τ + i ( e ).By Lemma D.2, we know that µ is of weight one and ε e is µ -admissible. Moreover, H ( g , K G ; π ∞ ) isof dimension 1. By Corollary B.6(3), we have m cusp ( π ) = 1. Case 2.
Suppose that π contributes to the Albanese and m disc ( π ) − m cusp ( π ) >
0. This mighthappen only when F = Q . In this case, V has Witt index 1. Write V = V ⊕ D where V isanisotropic and D is a hyperbolic hermitian plane. Let V π be a discrete realization (Definition B.1)of π that is perpendicular to L (G). By the Langlands theory of Eisenstein series [MW95], thereexist a strictly unitary automorphic character µ : E × \ A × E → C × , an irreducible subrepresentation V π ⊆ L (U(V )) of U(V )( A F ) with the underlying representation π , and a real number s > V π is contained in R s ( V π (cid:2) µ ): the space generated by residues of { E Q ( g ; f s ) | f ∈ I( V π (cid:2) µ ) } at s = s . Here, we adopt the notation in Subsection B.2. Since L S ( s, π ) = L S ( s − s , µ ) · L S ( s, π ),and L S ( s, π × µ ) can not have poles at s (cid:62) n by Corollary B.5, we must have s = s − µ = µ − . Again by Corollary B.5, µ is conjugate self-dual, so is µ , and µ = µ c . The appearanceof the residue implies that { E Q ( g ; f s ) | f ∈ I( V π (cid:2) µ c ) } has a pole at s −
1. If s = n +12 , then bythe similar argument in Case 1, we conclude that π is a character, so is π . This contradicts withH ( g , K G ; π ∞ ) = { } . Thus, s = n and s = n − . By Corollary B.6(2) and the similar argument inCase 1, we conclude that V π = Θ − W( µ − ,ν − ) ( π W ) for a unique one-dimensional skew-hermitian space Wand a unique character π W ; and π ∞ ’ ω ( µ, ε e , χ ) in which µ is of weight one and ε e is µ -admissible.Moreover, we have m cusp ( π ) = 1 and m disc ( π ) = 1 by Corollary B.6(3).To summarize, we have shown that if an irreducible admissible representation π of G( A ) contributesto the Albanese, then π ∞ ’ ω ( µ, ε, χ ) for a unique adèlic oscillator triple in which µ is of weight one and ε is µ -admissible, and m disc ( π ) = 1. Conversely, for every such adèlic oscillator triple ( µ, ε, χ ), thereexists a pair (W , π W ), unique up to isomorphism, such that if we denote by π an irreducible subrep-resentation of Θ V( µ,ν ) , W ( π W ), then ω ( µ, ε, χ ) is isomorphic to π ∞ and H ( g , K G ; π ∞ ) = { } . Moreover,by the Rallis inner product formula, Θ V( µ,ν ) , W ( π W ) is contained in L (G). Thus, we may apply theabove discussions to the representation π to conclude that the dimension of H ,τ ( A ∞ , C )[ ω ( µ, ε, χ )]is 1. The proposition follows. (cid:3) Remark . When n = 3, Proposition 4.13 can be deduced from [GR91, Rog92].Now we study the ‘ -adic cohomology of A ∞ . Take an embedding τ : E → C , a rational prime ‘ ,and an isomorphism ι ‘ : C ∼ −→ Q ac ‘ . Then we have a canonical isomorphismH ( A K ⊗ E,τ C , Q ac ‘ ) ’ H ,τ ( A K , C ) ⊗ C ,ι ‘ Q ac ‘ by the comparison theorem. PutH ( A ∞ ⊗ E,τ C , Q ac ‘ ) := lim −→ K H ( A K ⊗ E,τ C , Q ac ‘ ) , which is a Q ac ‘ [Gal( C /τ ( E )) × G ( A ∞ F )]-module.Suppose that n (cid:62) µ, ε, χ ) in which µ is of weight one and ε is µ -admissible. Then Hom Q ac ‘ [ G ( A ∞ F )] (cid:16) ι ‘ ◦ ω ( µ, ε, χ ) , H ( A ∞ ⊗ E,τ C , Q ac ‘ ) (cid:17) is a representation of Gal( C /τ ( E )) over Q ac ‘ . By Proposition 4.13, such representation is an ‘ -adiccharacter, denoted by ρ τ ,ι ‘ ( µ, ε, χ ) : Gal( C /τ ( E )) → ( Q ac ‘ ) × . The pair (V , V ) correspond to the pair (V , V) in Subsection B.2. Note that the global theta lifting Θ V( µ,ν ) , W ( π W ) is always in Weil’s convergent range. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 31
It induces, via the isomorphism ι ‘ , an automorphic character ρ τ ,‘ ( µ, ε, χ ) : τ ( E ) × \ A × τ ( E ) → C × . It is easy to see that the character ρ τ ,‘ ( µ, ε, χ ) does not depend on the isomorphism of ι ‘ , whichjustifies its notation. Theorem 4.15.
Suppose that n (cid:62) and let ( µ, ε, χ ) be an adèlic oscillator triple in which µ is ofweight one and ε is µ -admissible. Then we have ρ τ ,‘ ( µ, ε, χ ) ◦ τ = µ alg for every τ ∈ Φ µ and ‘ .Proof. We fix a prime ‘ and an isomorphism ι ‘ : C ∼ −→ Q ac ‘ . We also fix an element τ ∈ Φ µ , andidentify E as a subfield of C via τ . Let V be the hermitian space that is τ -nearby to V as in theproof of Proposition 4.13, and Sh(G , h) the corresponding Shimura variety from Subsection C.1 withG := Res F/ Q U(V) and h := h [ V , Φ . Then in view of the discussion in Subsection D.2, we have canonicalisomorphismsH ( A ∞ ⊗ E,τ C , Q ac ‘ ) ⊗ Q ac ‘ ,ι − ‘ C ’ H (Sh(G , h) , C ) ’ M χ H ( g , K G ; L (G( Q ) \ G( A ) , χ )) . For every orthogonal decomposition V = V ? ⊕ V ⊥ ? of hermitian spaces such that V ⊥ ? is totally positivedefinite, we have similarly the Shimura variety Sh(G ? , h ? ) together with the morphism Sh(G ? , h ? ) → Sh(G , h) over E . For an element e ∈ E ×− , we choose a maximal isotropic F -subspace V e of thesymplectic space (Res E/F V , Tr E/F e ( , ) V ). Then V e? := V e ∩ Res
E/F V ? is a maximal isotropic F -subspace of (Res E/F V ? , Tr E/F e ( , ) V ? ). Denote by V ( µ, e ) ⊆ C ∞ (G( Q ) \ G( A ) , C ) the subspace oftheta functions θ φµ ( g ) := X v ∈ V e ( ω µ,ε ( g ) φ )( v )on G( A ), where φ is in the Schwartz space S (V e ( A F )) in which we use the Fock model at archimedeanplaces. Similarly, we define the subspace V ? ( µ, e ) ⊆ C ∞ (G ? ( Q ) \ G ? ( A )).We claim that the restriction map C ∞ (G( Q ) \ G( A ) , C ) → C ∞ (G ? ( Q ) \ G ? ( A ) , C ) induced by theinclusion G ? , → G sends V ( µ, e ) to V ? ( µ, e ). In fact, we can find finitely many pairs ( φ ?,i , φ ⊥ ?,i ) with φ ?,i ∈ S (V e? ( A F )) and φ ⊥ ?,i ∈ S (V ⊥ e? ( A F )) where V ⊥ e? := V e ∩ Res
E/F V ⊥ ? , such that φ ( v ? , v ⊥ ? ) = X i φ ?,i ( v ? ) · φ ⊥ ?,i ( v ⊥ ? )for every v ? ∈ V ⊥ e? ( A F ) and v ⊥ ? ∈ V ⊥ e? ( A F ). Then for g ? ∈ G ? ( A ) ⊆ G( A ), we have θ φµ ( g ? ) = X i X v ? ∈ V e? ( ω µ,ε ( g ? ) φ ?,i )( v ? ) X v ⊥ ? ∈ V ⊥ e? φ ⊥ ?,i ( v ⊥ ? ) = X i X v ⊥ ? ∈ V ⊥ e? φ ⊥ ?,i ( v ⊥ ? ) θ φ ?,i µ ( g ? ) . Thus, the claim follows.To prove the theorem, note that for every class c ∈ H (Sh(G , h) , C ), using the same proof of[MR92, Proposition 6], one can find a decomposition V = V ? ⊕ V ⊥ ? as above with dim V ? = 2 suchthat the image of c under the restriction map H (Sh(G , h) , C ) in H (Sh(G ? , h ? ) , C ) is nonzero. Wedenote such image of c ? , and note that c ? actually belongs to (the image of) H (Sh(G ? , h ? ) , C ). Thenthe theorem follows from the above claim, Remark D.5, and Theorem D.6(1). (cid:3) Although [MR92, Proposition 6] only implies the existence of such V ? with dim V ? = 3, its proof actually shows theexistence of such V ? with dim V ? = 2 by only changing the term n − n − ? = 3 simply because the property they aimed to reduce does not hold whendim V ? = 2. Definition 4.16.
Let µ : E × \ A × E → C × be a conjugate symplectic character of weight one. Forevery object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) (Definition 4.5), the Q -vector space Hom E ( A ∞ , A µ ) Q is an M µ [ G ( A ∞ F )]-module where M µ acts via i µ and G ( A ∞ F ) acts M µ -linearly via its action on A ∞ . PutΩ( µ ) := lim −→ D µ ∈A ( µ ) Hom E ( A ∞ , A µ ) Q in the category of M µ [ G ( A ∞ F )]-modules. Remark . It follows from Proposition 4.6(1) that for every object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ),the canonical map Hom E ( A ∞ , A µ ) Q → Ω( µ ) is an isomorphism. Theorem 4.18.
There is an isomorphism Ω( µ ) ⊗ M µ C ’ M ε M χ ω ( µ, ε, χ ) of C [ G ( A ∞ F )] -modules, where the direct sum is taken over all ε, χ such that ε is µ -admissible. Moreover,the following statements hold.(1) For every object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) , we have a canonical isomorphism Ω( µ ) K =Hom E ( A K , A µ ) Q for every sufficiently small open compact subgroup K ⊆ G ( A ∞ F ) .(2) The C [ G ( A ∞ F )] -modules in the direct sum in Theorem 4.18 are mutually non-isomorphic.(3) For every given ε that is µ -admissible, the subspace L χ ω ( µ, ε, χ ) is stable under the action of Gal( C /M µ ) .Proof. Take an arbitrary object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) and identify Ω( µ ) with Hom E ( A ∞ , A µ ) Q by Remark 4.17.Take an embedding τ : E → C in Φ µ . It is clear that the maximal subspace of the complex vectorspace H ,τ ( A µ , C ) over which M µ acts via the inclusion M µ , → C has dimension 1. We choose abasis α of this subspace. Then we obtain a map Ω( µ ) → H ,τ ( A ∞ , C ) by pulling back α , which is C [ G ( A ∞ F )]-linear. It canonically extends to a mapΩ( µ ) ⊗ M µ C → H ,τ ( A ∞ , C ) . (4.2)To compute this map, we choose an isomorphism ι ‘ : C ∼ −→ Q ac ‘ . By the comparison theorem, (4.2)induces the following map Ω( µ ) ⊗ M µ Q ac ‘ → H ( A ∞ ⊗ E,τ C , Q ac ‘ )by pulling back α , as a class in H ( A µ ⊗ E,τ C , Q ac ‘ ). By Faltings’ isogeny theorem [Fal83], we have acanonical isomorphismΩ( µ ) ⊗ M µ Q ac ‘ ’ Hom Q ac ‘ [Gal( C /τ ( E ))] (cid:16) Q ac ‘ · α, H ( A µ ⊗ E,τ C , Q ac ‘ ) (cid:17) . However, by Definition 4.5(2), the action of Gal( C /τ ( E )) on the line Q ac ‘ · α spanned by α is givenby the automorphic character µ alg ◦ ( τ ) − : τ ( E ) × \ A × τ ( E ) → C × via ι ‘ . When n (cid:62) n = 2),by Proposition 4.13 (resp. Proposition D.4(1) with Remark D.5) and Theorem 4.15 (resp. TheoremD.6(1)), we have an isomorphismHom Q ac ‘ [Gal( C /τ ( E ))] (cid:16) Q ac ‘ · α, H ( A µ ⊗ E,τ C , Q ac ‘ ) (cid:17) ’ M ε M χ ω ( µ, ε, χ ) ⊗ C ,ι ‘ Q ac ‘ of Q ac ‘ [ G ( A ∞ F )]-modules induced by pulling back α , where the direct sum is taken over all ε, χ suchthat ε is µ -admissible. Thus, we obtain an isomorphism as in the theorem, which depends only on α ,not on ‘ , ι ‘ , and τ .The additional statement (1) follows from the above discussion as well. Statement (2) follows fromLemma D.1.Now we consider statement (3). Since Gal( C /M µ ) stabilizes µ and by (2), it suffices to showthat for every rational prime p , the image of Gal( C /M µ ) under the p -adic cyclotomic character χ p : Gal( C / Q ) → Z × p is contained in Z × p ∩ N E p /F p E × p for every prime p of F above p . This only OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 33 becomes a problem if p is ramified in E . To ease notation, we suppress the subscript p . So we havea ramified quadratic extension E/F where F/ Q p is a finite extension. Put U E/F := Z × p ∩ N E/F E × ,which we may assume a subgroup of Z × p of index 2. Denote by M E/F ⊆ C the subfield correspondingto the kernel of the composite homomorphism Gal( C / Q ) χ p −→ Z × p → Z × p /U E/F , which is a quadraticfield. Thus, our goal is to show that M E/F is contained in M µ .We first assume p odd. Then the residue extension degree f of F/ Q p must be odd. Write E = F ( √ u )for a uniformizer u of F . Then µ ( √ u ) = µ ( √ u ) = µ ( − N E/F u ) = µ ( − • If µ ( −
1) = 1, then − p hence M E/F = Q ( √ p ). On the otherhand, µ ( √ u ) = ± µ alg ( √ u ) = ± p f/ . Thus, M µ contains √ p as f is odd. • If µ ( −
1) = −
1, then − p hence M E/F = Q ( √− p ). On theother hand, µ ( √ u ) = ±√− µ alg ( √ u ) = ±√− p f/ . Thus, M µ contains √− p as f isodd.We now assume p = 2. Write v F : F → Z ∪ {∞} for the valuation function on F . We choose anEisenstein polynomial X + aX + b for E/F with v F ( a ) (cid:62) v F ( b ) = 1. Put d := min { v F ( a ) − , v F (4) } , which is an invariant of E/F . There are three cases. • Suppose that M E/F = Q ( √− U E/F = 1 + 4 Z . If d = v F (4), then by [BH06,Proposition 41.2(2)], 3 is contained in N E/F E × , which is a contradiction. Thus, we have d 1) = − − U E/F . Thus, µ alg ( √ u ) = ±√− M µ . • Suppose that M E/F = Q ( √ U E/F = ± Z . In particular, U E/F does not contain5, hence the residue extension degree f of F/ Q must be odd. Moreover, by [BH06, Proposi-tion 41.2(2)] again, we must have d = v F (4) hence v F ( a ) (cid:62) v F (2) + 1. Then we can find a uni-formizer u of F such that E = F ( √ u ). We have µ ( √ u ) = µ ( − 1) = 1, and µ alg ( √ u ) = ± f/ .In particular, √ M µ . • Suppose that M E/F = Q ( √− U E/F = ± Z . In particular, U E/F does notcontain 5 or − 1. The remaining discussion is same as the above case, which we omit.Statement (3) is proved. (cid:3) Theorem 4.18(2,3) allows us to make the following definition. Definition 4.19. For every collection ε that is µ -admissible, we denote by Ω( µ, ε ) the unique M µ [ G ( A ∞ F )]-submodule of Ω( µ ), such that Ω( µ, ε ) ⊗ M µ C is isomorphic to L χ ω ( µ, ε, χ ) as a C [ G ( A ∞ F )]-module. Corollary 4.20. Take an arbitrary object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) . For every sufficiently smallopen compact subgroup K of G ( A ∞ F ) , there is an isogeny decomposition A K ∼ Y µ A d ( µ,K ) µ , resp. A end K ∼ Y µ A d ( µ,K ) µ of abelian varieties over E when n (cid:62) (resp. n = 2 ), where the product is taken over representativesof Gal( C / Q ) -orbits of all conjugate symplectic automorphic characters of A × E of weight one. Here, A end K is the endoscopic part of A K when n = 2 , defined in (D.3) , and d ( µ, K ) := X ε X χ dim C ω ( µ, ε, χ ) K where the sum is taken over all ε, χ such that ε is µ -admissible. It is clear that the integer d ( µ, K ) depends only on the Gal( C / Q )-orbit of µ . Proof. This is a direct consequence of Theorem 4.18. (cid:3) Remark . Corollary 4.20 has a very interesting implication. Namely, if n (cid:62) X K has exoticsmooth reduction, that is, X K has proper smooth reduction at some nonarchimedean place of E thatis ramified over F , then H ( X K /E ) = { } since A µ cannot have good reduction at such a place.At the end of this subsection, we will construct a canonical pairing( , ) µ : Ω( µ ) × Ω( µ c ) → M µ (4.3)that is M µ -bilinear, non-degenerate, and G ( A ∞ F )-invariant. Definition 4.22. We define(1) the Hodge divisor D K on X K , as an element in CH ( X K ) Q , to be • the usual Hodge divisor on the Shimura variety Sh( V ) K if Sh( V ) K is proper (CompactCase); • the canonical extension of the usual Hodge divisor on Sh( V ) K to X K if Sh( V ) K is notproper (Noncompact Case).(2) the canonical volume of K to bevol( K ) := 1deg D n − K · | π (( X K ) E ac ) | in which deg D n − K is regarded as a constant positive integer by Lemma 4.23(4) below. Lemma 4.23. We have(1) The Hodge divisor D K is almost ample (Definition 2.8).(2) For every transition morphism u K K : X K → X K , ( u K K ) ∗ D K is rationally equivalent to D K .(3) For every g ∈ G ( A ∞ F ) , T ∗ g D K is rationally equivalent to D gKg − , where T g : X gKg − → X K isthe Hecke translation.(4) The degree function deg D n − K is a constant positive integer on π ( X K ) .Proof. Consider (1) first. If n = 2, then (for sufficiently small K ) X K has genus at least 2. Since D K has positive degree on every connected component, it is ample hence almost ample. Now suppose that n (cid:62) 3. If we are in the Compact Case, then the usual Hodge divisor is already ample. If we are in theNoncompact Case, then D K is the pullback of the Hodge divisor on the Baily–Borel compactificationof Sh( V ) K . Since the latter is ample, D K is almost ample (and in fact, not ample).For (2,3), since D K is the (canonical extension of the) usual Hodge divisor of Sh( V ) K , it is functorialunder pullback and Hecke translation. For (4), the positivity follows from (1) and Remark 2.10; theconstancy is a consequence of (2,3). (cid:3) Thus, by Proposition 2.9, we obtain a polarization θ K := θ X K ,D K : A ∨ K → A K . Now we define the pairing (4.3). We choose an object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) hence D ∨ µ =( A ∨ µ , i ∨ µ , λ ∨ µ , r ∨ µ ) ∈ A ( µ c ), and obtain Ω( µ ) = Hom E ( A ∞ , A µ ) Q and Ω( µ c ) = Hom E ( A ∞ , A ∨ µ ) Q . Itsuffices to consider elements φ ∈ Hom E ( A ∞ , A µ ) and φ c ∈ Hom E ( A ∞ , A ∨ µ ). Since both A µ and A µ c are of finite type, we may choose some K such that both φ and φ c factor through A K . The compositemap A µ ’ A ∨∨ µ = ( A µ c ) ∨ φ ∨ c −−→ A ∨ K θ K −−→ A K φ −→ A µ belongs to End E ( A µ ) Q = i µ ( M µ ). Now we define( φ, φ c ) Kµ := vol( K ) · i − µ ( φ ◦ θ K ◦ φ ∨ c ) ∈ M µ . For sufficiently small K and K ⊆ K , the degree of the transition morphism u K K equals vol( K ) · vol( K ) − by Lemma 4.23(2). Thus, by Lemma 4.23(2) and Proposition 2.7, we know that ( φ, φ c ) Kµ does not depend on the choice of K , which we define as ( φ, φ c ) µ . It is clear from the construction that(4.3) is bilinear, independent of the choice of D µ , non-degenerate since θ K is a polarization for every K , and G ( A ∞ F )-invariant since { D K } K is functorial under Hecke translation. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 35 Construction of Fourier–Jacobi cycles. Let V be a totally definite incoherent hermitianspace over A E of rank n (cid:62) 2, with G := U( V ). From now on to the end of Section 5, we • fix a conjugate symplectic automorphic character µ : E × \ A × E → C × of weight one; • will only consider sufficiently small open compact subgroups K ⊆ G ( A ∞ F ) that are decompos-able, that is, K can be written as Q v K v when v runs over all nonarchimedean places of F ; wecall such K a level subgroup .Let R be a ring containing Q . Let H R := C ∞ c ( G ( A ∞ F ) , R )be the full Hecke algebra with coefficient R , whose multiplication is given by the convolution withrespect to the canonical volume (Definition 4.22(3)). It is know that H R is an R [ G ( A ∞ F ) × G ( A ∞ F )]-module via left and right translations. For g ∈ G ( A ∞ F ), we denote by L g and R g the left and righttranslation on H R , respectively.For a level subgroup K ⊆ G ( A ∞ F ), we have the Hecke (sub)algebra H K,R := C ∞ c ( K \ G ( A ∞ F ) /K, R ),which admits an R -linear map T K : H K,R → Z n − ( X K × X K ) R sending f to the Hecke correspondence T fK , normalized by vol( K ). For example, if f = K , then T fK = vol( K ) · ∆ X K ∈ Z n − ( X K × X K ) R . The induced map (with the same notation) T K : H K,R → CH n − ( X K × X K ) R is a homomorphism of R -algebras. It is clear that H R = lim −→ K H K,R . Definition 4.24. Let Π be a relevant representation of GL n ( A E ) (Definition 1.2). We define Φ Π tobe the set of isomorphism classes of pairs ( V , π ∞ ) where • V is a totally positive definition incoherent hermitian space over A E of rank n ; • π ∞ is an irreducible admissible representation of G ( A ∞ F ) such that – for a nonarchimedean place v of F either split in E or at which π ∞ v is unramified, we haveBC( π ∞ v ) ’ Π v ; – π ∞ appears in H i B ,τ ( f Sh( V ) , C ) as a subquotient representation of G ( A ∞ F ) for some i ∈ Z and some place τ : E → C . Proposition 4.25. Let Π be a relevant representation of GL n ( A E ) . For ( V , π ∞ ) ∈ Φ Π , we have forevery τ : E → C that(1) π ∞ appears in H i B ,τ ( f Sh( V ) , C ) semisimply for every i ;(2) π ∞ does not appear in H i B ,τ ( f Sh( V ) , C ) if i = n − ;(3) H i B ,τ ( f Sh( V ) , C )[ π ∞ ] = IH i B ,τ ( f Sh( V ) , C )[ π ∞ ] for every i .Proof. Put τ := τ | F , and fix a hermitian space V that is τ -nearby to V (Definition C.4). PutG := Res F/ Q U(V), and identify f Sh( V ) ⊗ E,τ τ ( E ) with the (compactified) Shimura variety f Sh(G , h V ,τ )under the notation in Subsection C.1.We first note that π ∞ does not appear in H i B ( f Sh(G , h V ,τ ) , C ) / IH i B ( f Sh(G , h V ,τ ) , C ) as a subquotient,since otherwise Π will have two isomorphic cuspidal factors by Definition 1.2(1), which can not happenby Definition 1.2(2). Then (1) and (3) follow by the discussion in Subsection D.2.If π ∞ appears in IH i B ( f Sh(G , h V ,τ ) , C ), then there is an automorphic representation π ∞ ⊗ π ∞ ofG( A ) with m disc ( π ∞ ⊗ π ∞ ) (cid:62) i ( g , K G ; π ∞ ) = { } . By [Car12, Theorem 1.2], we knowthat Π is everywhere tempered. By Arthur’s endoscopic classification [Art13], which has been workedout in [Mok15] and [KMSW] for tempered representations for unitary groups, we know that the localbase change of π ∞ must be Π ∞ , which implies that π ∞ is a discrete series representation. In particular, i has to be the middle dimension n − 1. Thus, (2) follows. (cid:3) Definition 4.26. Let Π and ( V , π ∞ ) be as in Proposition 4.25. Let K ⊆ G ( A ∞ F ) be a level subgroup.We say that a function f ∈ H K, L , where L is some subfield of C , is a test function for π ∞ , if theelement cl B ,τ ( T fK ) ∈ H n − ,τ ( f Sh( V ) K × f Sh( V ) K , C )belongs to the subspace H n − ,τ ( f Sh( V ) K , C )[( π ∞ ) K ] ⊗ C H n − ,τ ( f Sh( V ) K , C )[(( π ∞ ) ∨ ) K ] under the Künnethdecomposition for every τ : E → C .Now we begin to construct Fourier–Jacobi cycles. We fix two relevant representations Π and Π of GL n ( A E ), and consider pairs ( V , π ∞ i ) ∈ Φ Π i for i = 1 , V . Let L ⊆ C be a subfieldcontaining M µ over which Π ∞ and Π ∞ (hence π ∞ and π ∞ ) are both defined. In what follows, we willregard π ∞ and π ∞ as irreducible L [ G ( A ∞ F )]-modules. Take a CM data D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ).Let K ⊆ G ( A ∞ F ) be a level subgroup; and we now write X K for f Sh( V ) K as in Subsection 4.2. Step 1: We start from the cycle∆ X K × D n − K ∈ CH n − ( X K × X K × X K × X K ) Q where we recall that D K is the Hodge divisor on X K (Definition 4.22(1)). Put(∆ X K × D n − K ) ∇ := ∆ X K × D n − K ∩ X K × X K × ∇ X K as an element in CH n − ( X K × X K × ∇ X K ) Q (see Definition 2.1 for the meaning of ∇ ). Step 2: Choose an element φ ∈ Hom E ( A K , A µ ). We push the above cycle along the morphismid X K × X K × ( φ ◦ α K ) : X K × X K × ∇ X K → X K × X K × A µ to obtain a cycle(id X K × X K × ( φ ◦ α K )) ∗ (∆ X K × D n − K ) ∇ ∈ CH n − M µ : Q ] / ( X K × X K × A µ ) Q , where we recall that α K is the Albanese morphism (4.1). Step 3: To proceed, we need to homologically trivialize the cycle in Step 2. Moreover, heuristically,the Chow group CH n − M µ : Q ] / ( X K × X K × A µ ) Q should be encoded in the cohomology/motiveH n − M µ : Q ] − ( X K × X K × A µ ). The motive we study comes from the product Π × Π ⊗ µ ,which appears in the cohomology H n − ( X K ) ⊗ H n − ( X K ) ⊗ H [ M µ : Q ] − ( A µ ) as a direct summandof the previous cohomology by suitable Künneth decomposition. To make sense of it, we needto introduce certain correspondences serving as projectors to the correct piece of cohomology.For the factor A µ , we use the canonical projector T can µ in Definition 4.9. For Shimura varieties,we choose test functions f and f in H K, L for π ∞ and π ∞ (Definition 4.26), respectively. PutFJ( f , f ; φ ) K := | π (( X K ) E ac ) | · ( T f K ⊗ T f K ⊗ T can µ ) ∗ (id X K × X K × ( φ ◦ α K )) ∗ (∆ X K × D n − K ) ∇ as an element in CH n − M µ : Q ] / ( X K × X K × A µ ) L .For i ∈ Z , we denote by CH i ( X K × X K × A µ ) \ L [ i µ ] the subspace of CH i ( X K × X K × A µ ) \ L on which i µ ( M µ ) acts via the inclusion M µ , → L . Proposition 4.27. Let the notation be as above.(1) The cycle FJ( f , f ; φ ) K belongs to CH n − M µ : Q ] / ( X K × X K × A µ ) L .(2) The image of FJ( f , f ; φ ) K in CH n − M µ : Q ] / ( X K × X K × A µ ) \ L belongs to the subspace CH n − M µ : Q ] / ( X K × X K × A µ ) \ L [ i µ ] and depends only on the homological equivalence class of T f K ⊗ T f K .Proof. Take an embedding τ : E → C .For (1), we realize that the image of cl ∗ B ,τ ( T f i K ) for i = 1 , n − ,τ ( X K , C ), while theimage of cl ∗ B ,τ ( T can µ ) is contained in L i (cid:54) [ M µ : Q ] − H i B ,τ ( A µ , C ) by Lemma 4.10. Thus, FJ( f , f ; φ ) K ishomologically trivial. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 37 For (2), by construction, it is clear that the image belongs to CH n − M µ : Q ] / ( X K × X K × A µ ) \ L [ i µ ].For the other part, we pick another pair of test functions ( f , f ) such that T f K ⊗ T f K is homologicallyequivalent to T f K ⊗ T f K . By (1), it suffices to show that for every ‘ and ι ‘ : C ∼ −→ Q ac ‘ , the pullbacks( T f K ⊗ T f K ⊗ T can µ ) ∗ and ( T f K ⊗ T f K ⊗ T can µ ) ∗ induce the same map from CH n − M µ : Q ] / ( X K × X K × A µ ) L to H ( E, H n − M µ : Q ] − (( X K × X K × A µ ) E ac , Q ac ‘ ( n − M µ : Q ] / ⊗ Q ac ‘ ,ι − ‘ C . (4.4)We denote the difference by ζ ‘ . Again, since T f K ⊗ T f K ⊗ T can µ and T f K ⊗ T f K ⊗ T can µ are homologicallyequivalent, the kernel of ζ ‘ contains CH n − M µ : Q ] / ( X K × X K × A µ ) L . Thus, ζ ‘ induces a complexlinear map fromCH n − M µ : Q ] / ( X K × X K × A µ ) C / CH n − M µ : Q ] / ( X K × X K × A µ ) C (4.5)to (4.4). We now explain that such map much be zero.In fact, let Σ be a finite set of places of F such that for v Σ, K v is hyperspecial maximal. Let H Σ K, C be the partial Hecke algebra away from Σ. Then H Σ K, C ⊗ C H Σ K, C acts on both (4.4) and (4.5) via thefactor X K × X K under which ζ ‘ is equivariant. In other words, ζ ‘ is a map of H Σ K, C ⊗ C H Σ K, C -modules.Since f and f are test functions for π ∞ and π ∞ , respectively, the image of ζ ‘ is isomorphic to a finitecopy of ( π ∞ , Σ1 ) K Σ ⊗ C ( π ∞ , Σ2 ) K Σ as an H Σ K, C ⊗ C H Σ K, C -module. Therefore, by Proposition 4.25(2), ζ ‘ must factor through the image of cycle map from (4.5) to (cid:16) H n − (( X K × X K ) E ac , Q ac ‘ ( n − ⊗ H [ M µ : Q ]´et (( A µ ) E ac , Q ac ‘ ([ M µ : Q ] / (cid:17) ⊗ Q ac ‘ ,ι − ‘ C . However, ζ ‘ also commutes with the action of M µ through the factor A µ by the functoriality of T can µ .As the actions of Q ac ‘ [ M µ ] on H [ M µ : Q ]´et (( A µ ) E ac , Q ac ‘ ) and H [ M µ : Q ] − (( A µ ) E ac , Q ac ‘ ) have disjoint support,we conclude that ζ ‘ must be zero. (cid:3) Definition 4.28 (Fourier–Jacobi cycles) . We call FJ( f , f ; φ ) K a Fourier–Jacobi cycle for Π × Π ⊗ µ .We call the image of FJ( f , f ; φ ) K in CH n − M µ : Q ] / ( X K × X K × A µ ) \ L [ i µ ], denoted by FJ( f , f ; φ ) \K ,a natural Fourier–Jacobi cycle for Π × Π ⊗ µ .The following lemma states that Fourier–Jacobi cycles are compatible with changing level subgroups. Lemma 4.29. The following statements hold.(1) Let K ⊆ K be a smaller level subgroup. Then we have ( u K K × u K K × id A µ ) ∗ FJ( f , f ; φ ) K = FJ( f , f ; φ ) K . (2) For g ∈ G ( A ∞ F ) , we have ( T g × T g × id A µ ) ∗ FJ( f , f ; φ ) K = FJ(R g L g f , R g L g f ; gφ ) gKg − where T g : X gKg − → X K is the Hecke translation.Proof. For (1), put u := u K K : X K → X K for short. Note that by definition, we have( u × u ) ∗ T f i K = vol( K )vol( K ) · T f i K for i = 1 , 2. Thus, for every α ∈ CH n − M µ : Q ] / ( X K × X K × A µ ) Q , we have( T f K ⊗ T f K ⊗ T can µ ) ∗ α = (cid:18) vol( K )vol( K ) (cid:19) · ( u × u × id A µ ) ∗ ( T f K ⊗ T f K ⊗ T can µ ) ∗ ( u × u × id A µ ) ∗ α by a standard computation of correspondences. Therefore, it suffices to show that( u × u × id A µ ) ∗ (id X K × X K × ( φ ◦ α K )) ∗ (∆ X K × D n − K ) ∇ = vol( K )vol( K ) · deg D n − K deg D n − K · (id X K × X K × ( φ ◦ α K )) ∗ (∆ X K × D n − K ) ∇ . This is an easy consequence of the equality α K ◦ ∇ u = α K . Thus, (1) follows.Part (2) follows from the same argument for (1), together with the relations T ∗ g f i = R g L g f i for i = 1 , φ ◦ Alb T g = gφ , and | π (( X K ) E ac ) | = | π (( X gKg − ) E ac ) | . (cid:3) For i ∈ Z , put CH i ( X ∞ × X ∞ × A µ ) ? L := lim −→ K CH i ( X K × X K × A µ ) ? L for ? = 0 , \ . The above lemma implies that we have well-defined elementsFJ( f , f ; φ ) ∈ CH n − M µ : Q ] / ( X ∞ × X ∞ × A µ ) L , FJ( f , f ; φ ) \ ∈ CH n − M µ : Q ] / ( X ∞ × X ∞ × A µ ) \ L [ i µ ] . Lemma 4.30. For every elements g, g , g ∈ G ( A ∞ F ) , we have FJ(R g f , R g f ; φ ) = ( T g × T g × id A µ ) ∗ FJ( f , f ; φ ) , FJ(L g f , L g f ; gφ ) = FJ( f , f ; φ ) , where T g : X ∞ → X ∞ denotes the Hecke translation by g .Proof. The first equality is obvious. For the second one, we haveFJ(R g L g f , R g L g f ; gφ ) = ( T g × T g × id A µ ) ∗ FJ(L g f , L g f ; gφ )from the first one. Thus,FJ(L g f , L g f ; gφ ) = ( T g − × T g − × id A µ ) ∗ FJ(R g L g f , R g L g f ; gφ )= ( T g − × T g − × id A µ ) ∗ ( T g × T g × id A µ ) ∗ FJ( f , f ; φ )= FJ( f , f ; φ )where the second equality is due to Lemma 4.29(2). (cid:3) Arithmetic Gan–Gross–Prasad conjecture. We first summarize the construction of naturalFourier–Jacobi cycles in a more functorial way. Let Π , Π , ( V , π ∞ ), ( V , π ∞ ), and L be as in theprevious subsection.Similar to Definition 4.16, for i ∈ Z , we putCH iµ ( X ∞ × X ∞ ) \ L := lim −→ D µ =( A µ ,i µ ,λ µ ,r µ ) ∈A ( µ ) CH i ( X ∞ × X ∞ × A µ ) \ L [ i µ ](4.6)in the category of L [ G ( A ∞ F ) × G ( A ∞ F )]-modules. It follows from Proposition 4.6(1) that the canonicalmap CH i ( X ∞ × X ∞ × A µ ) \ L [ i µ ] → CH iµ ( X ∞ × X ∞ ) \ L from (4.6) is an isomorphism for every object D µ ∈ A ( µ ), similar to Remark 4.17.Then it is clear that the assignment ( f , f , φ ) FJ( f , f , φ ) \ defines an L -linear mapFJ \ : H L ⊗ L H L ⊗ M µ Ω( µ ) → CH n − M µ : Q ] / µ ( X ∞ × X ∞ ) \ L , which is independent of the choice of D µ ∈ A ( µ ).The Hecke actions induce canonical surjective maps H L → π ∞ i ⊗ L ( π ∞ i ) ∨ (4.7)of L [ G ( A ∞ F ) × G ( A ∞ F )]-modules for i = 1 , 2. Proposition 4.27(2) implies that FJ \ factors through thequotient (cid:0) π ∞ ⊗ L ( π ∞ ) ∨ (cid:1) ⊗ L (cid:0) π ∞ ⊗ L ( π ∞ ) ∨ (cid:1) ⊗ M µ Ω( µ ) . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 39 Together with Lemma 4.30, we conclude that FJ \ is actually an L -linear mapFJ \ : π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ ) → Hom L [ G ( A ∞ F ) × G ( A ∞ F )] (cid:16) ( π ∞ ) ∨ ⊗ L ( π ∞ ) ∨ , CH n − M µ : Q ] / µ ( X ∞ × X ∞ ) \ L (cid:17) , which is invariant under the diagonal action of G ( A ∞ F ) on the left-hand side. For every µ -admissiblecollection ε (Definition 4.12), we denote byFJ \ε : π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ, ε ) → Hom L [ G ( A ∞ F ) × G ( A ∞ F )] (cid:16) ( π ∞ ) ∨ ⊗ L ( π ∞ ) ∨ , CH n − M µ : Q ] / µ ( X ∞ × X ∞ ) \ L (cid:17) the restriction of FJ \ to π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ, ε ) (Definition 4.19). Conjecture 4.31 (Unrefined arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n )) . Let Π and Π be two relevant representations of GL n ( A E ) (Definition 1.2). Let µ : E × \ A × E → C × be aconjugate symplectic automorphic character of weight one, and ε a µ -admissible collection. Let L ⊆ C be a subfield containing M µ over which both Π ∞ and Π ∞ are defined. For pairs ( V , π ∞ ) ∈ Φ Π and ( V , π ∞ ) ∈ Φ Π , the following three statements are equivalent:(a) We have FJ \ε = 0 .(b) We have FJ \ε = 0 , and dim L Hom L [ G ( A ∞ F ) × G ( A ∞ F )] (cid:16) ( π ∞ ) ∨ ⊗ L ( π ∞ ) ∨ , CH n − M µ : Q ] / µ ( X ∞ × X ∞ ) \ L (cid:17) = 1 . (c) We have L ( , Π × Π ⊗ µ ) = 0 , and Hom L [ G ( A ∞ F )] ( π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ, ε ) , L ) = { } . Remark . We have the following remarks concerning Conjecture 4.31.(1) The equivalence between (a) and (b) can be regarded as a generalization of Kolyvagin’s theoremfor Heegner points.(2) The assertion FJ \ε = 0 immediately implies Hom L [ G ( A ∞ F )] ( π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ, ε ) , L ) = { } .(3) By the multiplicity one part of the local Gan–Gross–Prasad conjecture, which is proved in[Sun12] for our particular Fourier–Jacobi model, we know thatdim L Hom L [ G ( A ∞ F )] ( π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ, ε ) , L ) (cid:54) . (4) By the (refined) local Gan–Gross–Prasad conjecture, which is proved in [GI16] for our partic-ular Fourier–Jacobi model, we know that ifdim L Hom L [ G ( A ∞ F )] ( π ∞ ⊗ L π ∞ ⊗ M µ Ω( µ, ε ) , L ) = 1(4.8) from some µ -admissible collection ε , then the global root number of Π × Π ⊗ µ is − 1, thatis, L ( s, Π × Π ⊗ µ ) has odd vanishing order at the center s = . Moreover, we have • If n is even, then the triple ( V , π ∞ , π ∞ ) is uniquely determined; but ε could be an arbitrary µ -admissible collection. • If n is odd, then V could be arbitrary; but once V is chosen, π ∞ , π ∞ , and ε are uniquelydetermined.In other words, in both cases, once ε is given, the triple ( V , π ∞ , π ∞ ) is uniquely determined.Now we state a refined version of the arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n ).We will now assume that all height pairings are defined, which is the case if ( X K ) v satisfies theMonodromy–Weight conjecture for every nonarchimedean place v of E .We take a level subgroup K ⊆ G ( A ∞ F ). For every object D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) and every µ -admissible collection ε , putHom E ( A K , A µ , ε ) := Hom E ( A K , A µ ) ∩ Ω( µ, ε );Hom E ( A K , A ∨ µ , − ε ) := Hom E ( A K , A ∨ µ ) ∩ Ω( µ c , − ε ) . Conjecture 4.33 (Refined arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n )) . Let the setup beas in Conjecture 4.31. Moreover, let K ⊆ G ( A ∞ F ) be a level subgroup, and D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ) a CM data for µ (Definition 4.5). For every test functions f , f ∨ , f , f ∨ ∈ H K, L for π ∞ , ( π ∞ ) ∨ , π ∞ , ( π ∞ ) ∨ , respectively, and every elements φ ∈ Hom E ( A K , A µ , ε ) and φ c ∈ Hom E ( A K , A ∨ µ , − ε ) , theequality vol( K ) · h FJ( f , f ; φ ) K , FJ( f ∨ , f ∨ ; φ c ) K i BBP X K × X K ,A µ (4.9) = Q ni =1 L ( i, µ iE/F )2 s (Π )+ s (Π ) · L ( , Π × Π ⊗ µ ) L (1 , Π , As ( − n ) · L (1 , Π , As ( − n ) · β ( f , f ∨ , f , f ∨ , φ, φ c ) holds. Here, s (Π i ) has appeared in Definition 1.2; As ± stands for the two Asai representations (see,for example, [GGP12a, Section 7] ); and β is a certain normalized matrix coefficient integral definedimmediately below. For i = 1 , 2, we have L -linear maps H L → π ∞ i ⊗ L ( π ∞ i ) ∨ → L ( ⊆ C )in which the first is (4.7) and the second is the evaluation map. For every f ∈ H L and g ∈ G ( A ∞ F ),we denote by ev( π ∞ i ( g ) , f ) the image of L g f under the above composite map. In particular, theassignment g ev( π ∞ i ( g ) , f ) is a matrix coefficient of π ∞ i .Consider a finite set Σ of nonarchimedean places of F such that K v is hyperspecial maximal for v Σ. Let d Σ be the unique Haar measure on G ( F Σ ) under which the volume of K Σ equals 2 vol( K ).For f , f ∨ , f , f ∨ ∈ H L , φ ∈ Ω( µ, ε ) and φ c ∈ Ω( µ c , − ε ), we define β Σ ( f , f ∨ , f , f ∨ , φ, φ c ) := Y v ∈ Σ ∪ Φ F Q ni =1 L ( i, µ iE/F,v ) · L ( , Π ,v × Π ,v ⊗ µ v ) L (1 , Π ,v , As ( − n ) · L (1 , Π ,v , As ( − n ) − Z G ( F Σ ) ev( π ∞ ( g ) , f t1 ∗ f ∨ ) · ev( π ∞ ( g ) , f t2 ∗ f ∨ ) · ( gφ, φ c ) µ · d Σ g in which • f t i is the transpose of f i , that is, f t i ( g ) = f i ( g − ); • f t i ∗ f ∨ i denotes the convolution product in H K, L ; and • ( , ) µ is the pairing (4.3).By [Xue16, Proposition 1.1.1(1,3)], the value of β Σ ( f , f ∨ , f , f ∨ , φ, φ c ) is finite and stabilizes when Σis large enough; and we denote the stable value by β ( f , f ∨ , f , f ∨ , φ, φ c ). Remark . We have the following remarks concerning Conjecture 4.31.(1) The left-hand side of (4.9) is insensitive to K . More precisely, if we take a smaller levelsubgroup K contained in K , then the left-hand side of (4.9) is equal tovol( K ) · h FJ( f , f ; φ ) K , FJ( f ∨ , f ∨ ; φ c ) K i BBP X K × X K ,A µ by the projection formula.(2) The refined Gan–Gross–Prasad conjecture for the central value formula in this case is formu-lated by Hang Xue [Xue16, Conjecture 1.1.2].(3) It is known by [Xue16, Proposition 1.1.1(2)] that (4.8) holds if and only if β is nonvanishingas a functional.At the end of this subsection, we state a variant of Conjecture 4.33. The following definition (withslightly different terminology) is taken from [RSZ20]. Definition 4.35. We say that a collection of correspondences z = ( z K ∈ CH n − ( X K × X K ) Q ) K is a Hecke system of projectors if(1) z K is an odd projector (Definition 3.10) for every K ; OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 41 (2) we have (id X K × u K K ) ∗ z K = ( u K K × id X K ) ∗ z K ∈ CH n − ( X K × X K ) Q for every transitionmorphism u K K : X K → X K ;(3) for every g ∈ G ( A ∞ F ), we have T ∗ g z K = z gKg − where T g : X gKg − → X K is the Hecke transla-tion. Remark . We have the following remarks concerning the existence of Hecke system of projectors.(1) If n = 2, then z = ( z X K ,D K ) K constructed in Lemma 3.11(1) is a Hecke system of projectorsby Lemma 4.23.(2) If n = 3, then z = ( z X K ,D K ) K constructed in Lemma 3.11(2) is a Hecke system of projectorsby Lemma 4.23 and Proposition 2.12(2).(3) If n (cid:62) F = Q , then odd projectors exist by [MS, Theorem 1.3]. Note that since weconsider trivial coefficients, there is no need to require the Shimura data to be of PEL type inthat theorem; see [MS, Remark 2.7].(4) If n (cid:62) F = Q , then one probably needs to use projectors for intersection cohomology;see [MS, Theorem 1.4].Now take a Hecke system of projectors z = ( z K ) K . We will use z to modify Step 1 in the constructionof FJ( f , f ; φ ) K . Namely, we consider∆ z X K := pr [3] z K ∆ X K ∈ CH n − ( X K × X K × X K ) Q where pr [3] z K is defined in Definition 3.13. Then we replace ∆ X K by ∆ z X K in every later steps, anddenote the final outcome byFJ( f , f ; φ ) zK ∈ CH n − M µ : Q ] / ( X K × X K × A µ ) L . Conjecture 4.37 (Refined arithmetic Gan–Gross–Prasad conjecture for U( n ) × U( n ), variant) . Letthe setup be as in Conjecture 4.33. Take a Hecke system of projectors z = ( z K ) K . Then the equality vol( K ) · h FJ( f , f ; φ ) zK , FJ( f ∨ , f ∨ ; φ c ) zK i BBP X K × X K ,A µ (4.10) = Q ni =1 L ( i, µ iE/F )2 s (Π )+ s (Π ) · L ( , Π × Π ⊗ µ ) L (1 , Π , As ( − n ) · L (1 , Π , As ( − n ) · β ( f , f ∨ , f , f ∨ , φ, φ c ) holds.Remark . We have the following remarks concerning Conjecture 4.37.(1) We have a similar statement for FJ( f , f ; φ ) zK as in Lemma 4.29. In particular, the left-handside of (4.10) is independent of K .(2) By a similar argument for Proposition 4.27(2), one can show that the image of FJ( f , f ; φ ) zK in CH n − M µ : Q ] / ( X K × X K × A µ ) \ L equals FJ( f , f ; φ ) \K . This is why we expect the variantconjecture to hold as well, in view of Remark 3.2.(3) One of the advantages of introducing the auxiliary projector z is that one can show that theleft-hand side of (4.10), regarded as a functional in ( φ, φ c ), factors through the mapHom E ( A K , A µ , ε ) × Hom E ( A K , A ∨ µ , − ε ) → Ω( µ, ε ) ⊗ M µ Ω( µ c , − ε )and becomes M µ -linear. See Remark 5.11.5. Arithmetic relative trace formula In this section, we discuss the relative trace formula approach toward the arithmetic GGP conjecturefor U( n ) × U( n ). In Subsection 5.1, we prove the doubling formula for CM data. In Subsection 5.2,we introduce the arithmetic invariant functional I ( f , φ ), and its local version I ( f , φ ) p at a good inertprime for which we perform some preliminary computation. In Subsection 5.3, we prove the formulafor the orbital decomposition of I ( f , φ ) p .We keep the notation from Section 4. We fix a conjugate symplectic automorphic character µ : E × \ A × E → C × of weight one, and an µ -admissible collection ε (Definition 4.12). From now on, we will restrict ourselves to the Compact Case. We will identify E as a subfield of C via a fixed complex embedding τ ∈ Φ µ . Put τ := τ | F , and fix a hermitian space V that is τ -nearbyto V (Definition C.4). In particular, V is anisotropic. Put G := Res F/ Q U(V), and identify X K withthe (proper) Shimura variety Sh(G , h V ,τ ) K under the notation in Remark C.2.5.1. A doubling formula for CM data. We start by performing some preliminary computation ofthe Beilinson–Bloch–Poincaré height pairingvol( K ) · h FJ( f , f ; φ ) zK , FJ( f ∨ , f ∨ ; φ c ) zK i BBP X K × X K ,A µ (5.1)for a level subgroup K ⊆ G( A ∞ ) and a CM data D µ = ( A µ , i µ , λ µ , r µ ) ∈ A ( µ ), as in Conjecture 4.37.Consider an intermediate number field E ⊆ E ⊆ C such that E splits X K (Definition 2.1). Put X K := ( X K ) E , A K := ( A K ) E , A µ := ( A µ ) E , φ := φ E , and φ c := ( φ c ) E . We will suppress E in thefiber product X × E Y of schemes if X and Y are obviously over E . For every point P ∈ X K ( π ( X K )),we have the induced morphism α P := ( α K ) P : X K → A K from Definition 2.3 and Definition 2.1. We put∆ φ,Pz X K := (id X K × X K × ( φ ◦ α P )) ∗ ∆ z X K ∈ CH n − M µ : Q ] / ( X K × X K × A µ ) Q , ∆ φ c ,Pz X K := (id X K × X K × ( φ c ◦ α P )) ∗ ∆ z X K ∈ CH n − M µ : Q ] / ( X K × X K × A µ ) Q . Lemma 5.1. Suppose that E is sufficiently large such that D n − K can be represented by a finite sum P i c i P i with c i ∈ Q and P i ∈ X K ( π ( X K )) . Then we have (5.1) = 1[ E : E ](deg D n − K ) X i,j c i c j · h ( T f K ⊗ T f K ⊗ T can µ ) ∗ ∆ φ,P i z X K , ( T f ∨ K ⊗ T f ∨ K ⊗ T can µ c ) ∗ ∆ φ c ,P j z X K i BBP X K × X K ,A µ . Proof. This follows immediately from the definition of FJ( f ∨ , f ∨ ; φ c ) zK and Definition 4.22(2). (cid:3) Let P µ ∈ CH ( A µ × A ∨ µ ) be the Poincaré class on A µ × A ∨ µ . Put Q µ := ( T can , t µ ⊗ T can µ c ) ∗ P µ ∈ CH ( A µ × A ∨ µ ) M µ , and for P, Q ∈ X K ( π ( X K )), put Q φ,φ c ,P,Qµ,K := ( φ ◦ α P × φ c ◦ α Q ) ∗ Q µ ∈ CH ( X K × X K ) M µ . Lemma 5.2. For P, Q ∈ X K ( E ) , we have h ( T f K ⊗ T f K ⊗ T can µ ) ∗ ∆ φ,Pz X K , ( T f ∨ K ⊗ T f ∨ K ⊗ T can µ c ) ∗ ∆ φ c ,Qz X K i BBP X K × X K ,A µ = h p ∗ ∆ z X K , ( X K × X K × Q φ,φ c ,P,Qµ,K ) . p ∗ ∆ z,f t1 ∗ f ∨ ,f t2 ∗ f ∨ X K i BB X K × X K × X K × X K where ∆ z, f , f X K := ( T f K ⊗ T f K ⊗ id X K ) ∗ ∆ z X K ∈ CH n − ( X K × X K × X K ) L for f , f ∈ H K, L .Proof. Consider the following commutative diagram of in the category Sch /E X K × X K × X K × X Kγ (cid:116) (cid:116) γ (cid:42) (cid:42) δ (cid:15) (cid:15) X K × X K × X K × A µr (cid:117) (cid:117) β (cid:42) (cid:42) X K × X K × A µ × X Kβ (cid:116) (cid:116) r (cid:41) (cid:41) X K × X K × X K α (cid:41) (cid:41) X K × X K × A µ × A µq (cid:116) (cid:116) q (cid:42) (cid:42) X K × X K × X Kα (cid:117) (cid:117) X K × X K × A µ X K × X K × A µ OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 43 in which all diamonds are Cartesian, and α := id X K × X K × ( φ ◦ α P ), α := id X K × X K × ( φ c ◦ α Q ), q := p , q := p .Put P µ := ( X K × X K ) × P µ and Q µ := ( X K × X K ) × Q µ . By the definition of the Beilinson–Bloch–Poincaré height pairing, we have h ( T f K ⊗ T f K ⊗ T can µ ) ∗ ∆ φ,Pz X K , ( T f ∨ K ⊗ T f ∨ K ⊗ T can µ c ) ∗ ∆ φ c ,Qz X K i BBP X K × X K ,A µ = h ( T f K ⊗ T f K ⊗ T can µ ) ∗ ∆ φ,Pz X K , q ∗ ( P µ .q ∗ ( T f ∨ K ⊗ T f ∨ K ⊗ T can µ c ) ∗ ∆ φ c ,Qz X K ) i BB X K × X K × A µ = h q ∗ ( T f K ⊗ T f K ⊗ T can µ ) ∗ ∆ φ,Pz X K , P µ .q ∗ ( T f ∨ K ⊗ T f ∨ K ⊗ T can µ c ) ∗ ∆ φ c ,Qz X K i BB X K × X K × A µ × A µ (5.2)where we have used [Be˘ı87, 4.0.3] for the last equality. Note that we have q ∗ ( T f K ⊗ T f K ⊗ T can µ ) ∗ ∆ φ,Pz X K = ( T f K ⊗ T f K ⊗ T can µ ⊗ id A µ ) ∗ q ∗ ∆ φ,Pz X K = ( T f K ⊗ T f K ⊗ T can µ ⊗ id A µ ) ∗ q ∗ α ∗ ∆ z X K = ( T f K ⊗ T f K ⊗ T can µ ⊗ id A µ ) ∗ β ∗ r ∗ ∆ z X K , and similarly q ∗ ( T f ∨ K ⊗ T f ∨ K ⊗ T can µ c ) ∗ ∆ φ c ,Qz X K = ( T f ∨ K ⊗ T f ∨ K ⊗ id A µ ⊗ T can µ c ) ∗ β ∗ r ∗ ∆ z X K . Then it follows that(5.2) = h ( T can µ ) ∗ β ∗ r ∗ ∆ z,f ,f X K , P µ . ( T can µ c ) ∗ β ∗ r ∗ ∆ z,f ∨ ,f ∨ X K i BB X K × X K × A µ × A µ (5.3)where we have suppressed the expression id ? in the notation of correspondences as it is clear whichfactor the correspondence acts on. Using [Be˘ı87, 4.0.3] again, we further have(5.3) = h β ∗ r ∗ ∆ z,f ,f X K , Q µ .β ∗ r ∗ ∆ z,f ∨ ,f ∨ X K i BB X K × X K × A µ × A µ = h γ ∗ r ∗ ∆ z,f ,f X K , δ ∗ Q µ .γ ∗ r ∗ ∆ z,f ∨ ,f ∨ X K i BB X K × X K × X K × X K = h γ ∗ r ∗ ∆ z X K , δ ∗ Q µ .γ ∗ r ∗ ∆ z,f t1 ∗ f ∨ ,f t2 ∗ f ∨ X K i BB X K × X K × X K × X K = h p ∗ ∆ z X K , δ ∗ Q µ . p ∗ ∆ z,f t1 ∗ f ∨ ,f t2 ∗ f ∨ X K i BB X K × X K × X K × X K . The lemma follows by noting that δ = β ◦ γ = β ◦ γ = id X K × X K × ( φ ◦ α P ) × ( φ c ◦ α Q ). (cid:3) Lemma 5.2 suggests us to compute the class Q φ,φ c ,P,Qµ,K ∈ CH ( X K × X K ) M µ , or rather its homologicalequivalence class by Lemma 3.5. To do this, we first review some doubling construction and Kudla’sgenerating series of special divisors, introduced by Kudla [Kud97] in the context of orthogonal Shimuravarieties. Definition 5.3. Let V( E ) + ⊆ V( E ) be the subset consisting of x such that ( x, x ) V is totally positive.Take x ∈ V( E ) + , and denote its orthogonal complement in V by V x . For g ∈ G( A ∞ ), we have thecomposite morphism s x,g := Sh(G x , h V x ,τ ) gKg − ∩ G x ( A ∞ ) → Sh(G , h V ,τ ) gKg − = X gKg − T g −→ X K where G x := Res F/ Q U(V x ) and the first arrow is induced by the inclusion V x ⊆ V of hermitiansubspaces. The morphism s x,g is finite and unramified. We define Z ( x, g ) K := ( s x,g ) ∗ Sh(G x , h V x ,τ ) gKg − ∩ G x ( A ∞ ) as an element in Z ( X K ).We denote by S (V( A ∞ E )) the space of complex valued Schwartz functions on V( A ∞ E ), which admitsan action by G( A ∞ ) via the variable. For every φ ∈ S (V( A ∞ E )), we define the generating series of special divisors attached to φ (of level K ) to be Z ( φ ) K := − φ (0) D K + X x ∈ U(V)( F ) \ V( E ) + e − π · Tr F/ Q ( x,x ) V X g ∈ G x ( A ∞ ) \ G( A ∞ ) /K φ ( g − x ) Z ( x, g ) K as a formal series in Z ( X K ) C , where D K is (some representative of) the Hodge divisor (Definition4.22). Lemma 5.4. The generating series of special divisors Z ( φ ) K is Chow convergent, that is, an elementin CZ ( X K ) (Definition 3.3).Proof. This is [Liu11a, Theorem 3.5(2)] (with g = 1), together with the fact that CH ( X K ) C is offinite dimension. (cid:3) We study the relation between generating series of special divisors and the spaces Ω( µ, ε ) andΩ( µ c , − ε ). Choose a nonzero element α (resp. α c ) in H ( A µ ( C ) , Ω ) (resp. H ( A ∨ µ ( C ) , Ω )) on which M µ acts via the inclusion M µ , → C , such that under the canonical pairing H ( A µ , C ) × H ( A ∨ µ , C ) → C , α and α c pair to one. It is clear that for φ ∈ Ω( µ, ε ) and φ c ∈ Ω( µ c , − ε ), the (1 , φ (cid:5) φ c := φ ∗ α ∧ φ ∗ c α c on X K ( C ) does not depend on the choice of the pair ( α, α c ), which is moreover in H ( X K , M µ (1)).By [BMM16, Proposition 5.19] and [Liu14, Lemma 5.3], φ (cid:5) φ c is a Kudla–Milson form which, in thenotation of [BMM16, (8.8)], equals θ ψ F ,µ, ˜ φ ( − , 1) where˜ φ = ϕ , ⊗ O Φ F \{ τ } ϕ ⊗ φ for a unique φ ∈ S (V( A ∞ E )) as in [BMM16, (8.9)]. The assignment ( φ, φ c ) φ gives rise to a map d : Ω( µ, ε ) ⊗ M µ Ω( µ c , − ε ) ⊗ M µ C → S (V( A ∞ E )) . (5.4) Lemma 5.5. The map (5.4) is an isomorphism of C [G( A ∞ )] -modules.Proof. For g ∈ G( A ∞ ), we have gφ (cid:5) gφ c = T ∗ g ( φ (cid:5) φ c ) = T ∗ g θ ψ F ,µ, ˜ φ ( − , 1) = θ ψ F ,µ, T ∗ g ˜ φ ( − , T ∗ g ˜ φ = ϕ , ⊗ (cid:16)N Φ F \{ τ } ϕ (cid:17) ⊗ ( g. φ ). Thus, d is G( A ∞ )-equivariant. The map d is apparently injective, so is surjective by [Liu14, Lemma 5.3]. The lemma follows. (cid:3) Lemma 5.6. We have(1) The cohomology class cl B ( Q φ,φ c ,P,Qµ,K ) ∈ H ( X K × X K , C ) depends only on d ( φ ⊗ φ c ) .(2) There is a unique C -linear map c Q µ,K : S (V( A ∞ E )) K → H ( X K × X K , C ) φ c Q φ µ,K such that(a) c Q d ( φ ⊗ φ c ) µ,K = cl B ( Q φ,φ c ,P,Qµ,K ) for every pair ( φ, φ c ) ∈ Hom E ( A K , A µ , ε ) × Hom E ( A K , A ∨ µ , − ε ) and every pair P, Q ∈ X K ( E ) ;(b) ∆ ∗ c Q φ µ,K = cl B ( Z ( φ ) K ) ∈ H ( X K , C ) for every φ ∈ S (V( A ∞ E )) K .Proof. The class cl B ( Q φ,φ c ,P,Qµ,K ) ∈ H ( X K × X K , C ) is given by the (1 , p ∗ φ ∗ α ∧ p ∗ φ ∗ c α c on X K ( C ) × X K ( C ). Part (1) follows immediately.For (2), by (1) and Lemma 5.5, there is a unique C -linear map c Q µ,K satisfying (a). However, italso satisfies (b) due to [BMM16, Proposition 8.3]. (cid:3) Remark . The maps { c Q µ,K } K in Lemma 5.6 are clearly compatible under restriction, hence inducea C -linear map c Q µ : S (V( A ∞ E )) → H ( X ∞ × X ∞ , C ) := lim −→ K H ( X K × X K , C ). OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 45 Definition 5.8. We say that an element Z K ∈ Z ( X K × X K ) C is a doubling divisor (of level K ) foran element φ ∈ S (V( A ∞ E )) K if cl B ( Z K ) = c Q φ µ,K , and Z K has proper intersection with ∆ X K . Lemma 5.9. For every element φ ∈ S (V( A ∞ E )) K , there exists a doubling divisor of level K .Proof. By linearity, it suffices to consider the case φ = d ( φ ⊗ φ c ) for ( φ, φ c ) ∈ Hom E ( A K , A µ , ε ) × Hom E ( A K , A ∨ µ , − ε ). Take an intermediate number field E ⊆ E ⊆ C such that E is Galois over E ,splits X K , and satisfies X K ( π ( X E )) = ∅ . We choose a point P ∈ X K ( π ( X E )). Then Z K := 1[ E : E ] X σ ∈ Gal( E /E ) Q φ,φ c ,σP,σPµ,K is an element in Z ( X K × X K ) M µ such that cl B ( Z K ) = c Q φ µ,K by Lemma 5.6(2). By Chow’s movinglemma, we may replace Z K by another rationally equivalent cycle that has proper intersection with∆ X K . The lemma follows. (cid:3) Now we can state and prove our doubling formula for CM data. Proposition 5.10. Put f i := f t i ∗ f ∨ i for i = 1 , . If we write f = P s d s g − s K ∩ Kg − s as a finite sumwith d s ∈ L and g s ∈ U(V)( A ∞ F ) , then the identity (5.1) = X s d s · h p ∗ ∆ z X K s , (∆ X K s × T L gs f K s × Z sK s ) . p ∗ ∆ z X K s i BB X Ks holds, where K s := K ∩ g s Kg − s , and Z sK s ∈ Z ( X K × X K ) C is an arbitrary doubling divisor for d ( φ ⊗ g s φ c ) (which exists by Lemma 5.9).Proof. To shorten notation, we put δ K := (deg D n − K ) − .By Lemma 5.1, Lemma 5.2, we have(5.1) = δ K [ E : E ] X i,j c i c j h p ∗ ∆ z X K , ( X K × X K × Q φ,φ c ,P i ,P j µ,K ) . p ∗ ∆ z, f , f X K i BB( X K ) . (5.5)By Remark 4.38(1), we may replace K by K := T s ( g − s Kg s ∩ K s ) and possibly enlarge E to obtain(5.5) = δ K [ E : E ] X i,j c i c j h p ∗ ∆ z X K , ( X K × X K × Q φ,φ c ,P i ,P j µ,K ) . p ∗ ∆ z, f , f X K i BB( X K ) = d K [ E : E ] X i,j,s c i c j d s h p ∗ ∆ z X K , ( X K × X K × Q φ,φ c ,P i ,P j µ,K ) . p ∗ ∆ z, g − s K ∩ Kg − s , f X K i BB( X K ) . (5.6)Since L g s g − s K ∩ Kg − s = K ∩ g s Kg − s = K s , by Lemma 4.30, we have(5.6) = δ K [ E : E ] X i,j,s c i c j d s h p ∗ ∆ z X K , ( X K × X K × Q φ,g s φ c ,P si ,P sj µ,K ) . p ∗ ∆ z, Ks , L gs f X K i BB( X K ) = X s d s · δ K [ E : E ] X i,j c i c j h p ∗ ∆ z X K , ( X K × X K × Q φ,g s φ c ,P si ,P sj µ,K ) . p ∗ ∆ z, Ks , L gs f X K i BB( X K ) . (5.7)For each individual s , we may descend the corresponding term down to X K s again by Remark 4.38(1).Choose a representative P i c si P si of D n − K s with c si ∈ Q and P si ∈ X K s ( π ( X K s )). Moreover, by Lemma3.5 and Lemma 5.6, we may replace Q φ,g s φ c ,P si ,P sj µ,K s by Z sK s ⊗ E E . Then we have(5.7) = X s d s · δ K s · X i,j c si c sj h p ∗ ∆ z X K s , ( X K s × X K s × Z sK s ) . p ∗ ∆ z, Ks , L gs f X K s i BB( X Ks ) = X s d s h p ∗ ∆ z X K s , ( X K s × X K s × Z sK s ) . p ∗ ∆ z, Ks , L gs f X K s i BB( X Ks ) It is elementary to see that every element in H K, L can be written in this way. where in the second equality, we use the fact that P i c si = deg D n − K s = δ − K s . The proposition thenfollows by [Be˘ı87, 4.0.3]. (cid:3) Remark . Proposition 5.10 implies that, for given data f , f , f ∨ , f ∨ , z , the assignmentHom E ( A K , A µ , ε ) × Hom E ( A K , A ∨ µ , − ε ) → C ( φ, φ c ) vol( K ) · h FJ( f , f ; φ ) zK , FJ( f ∨ , f ∨ ; φ c ) zK i BBP X K × X K ,A µ factors through Ω( µ, ε ) ⊗ M µ Ω( µ c , − ε ) and extends uniquely to an M µ -linear mapΩ( µ, ε ) ⊗ M µ Ω( µ c , − ε ) → C by considering all level subgroups K .5.2. Arithmetic invariant functional. In view of Proposition 5.10, we need to study global arith-metic invariant functionals defined as follows. Definition 5.12 (Global arithmetic invariant functional) . Let K ⊆ G( A ∞ ) be a level subgroup. Fortest functions f ∈ H K, C and φ ∈ S (V( A ∞ E )) K , we define the global arithmetic invariant functional to be I zK ( f , φ ) := h p ∗ ∆ z X K , (∆ X K × T f K × Z K ) . p ∗ ∆ z X K i BB X K where Z K ∈ Z ( X K × X K ) C is an arbitrary doubling divisor for φ (Definition 5.8, linearly extendedto coefficient C ).We introduce two important conventions, which will be adopted from now on.(1) We will regard T f K as a cycle, rather than a Chow cycle, in X K × X K .(2) Whenever we have two cycles A and B in a regular scheme X that have proper intersection, A.B will be regarded as the cycle P C m C ( A, B ) · C (rather than the associated Chow cycle),where the sum is taken over all irreducible components in A ∩ B with the correct dimension,and m C ( A, B ) is the intersection multiplicity. Definition 5.13. Let K ⊆ G( A ∞ ) be a level subgroup. For a doubling divisor Z K ∈ Z ( X K × X K ) C for φ ∈ S (V( A ∞ E )) K , we define the calibration of Z K to be Z ♥ K := Z K − p ∗ (∆ X K .Z K − Z ( φ ) K )where we regard ∆ X K .Z K as in Z ( X K ) C and recall that p : X K × X K → X K is the projection tothe second factor.It is clear that Z ♥ K ∈ CZ ( X K × X K ) (Definition 3.3), ∆ X K .Z ♥ K = Z ( φ ) K , cl B ( Z ♥ K ) = cl B ( Z K ) byLemma 5.6(2), and I zK ( f , φ ) = h p ∗ ∆ z X K , (∆ X K × T f K × Z ♥ K ) . p ∗ ∆ z X K i BB X K (5.8)by Lemma 3.5.To proceed, we introduce the notation of (relative) regular semisimple elements. Definition 5.14. Consider a field extension F /F and put E := E ⊗ F F .(1) We say that a pair of elements ( ξ, x ) ∈ U(V)( F ) × V( E ) is regular semisimple if the vectors { ξ i x | i = 0 , . . . , n − } span the E -module V( E ).(2) The group U(V)( F ) acts on U(V)( F ) × V( E ) via the formula ( ξ, x ) g = ( g − ξg, g − x ), whichpreserves regular semisimple pairs. Denote by [U(V)( F ) × V( E )] the orbits of U(V)( F ) × V( E )under the above action, and by [U(V)( F ) × V( E )] rs the subset of regular semisimple orbits.(3) We say that a function on U(V)( F ) × V( E ) is regularly supported if its support consists ofonly regular semisimple pairs.(4) We say that a function F on U(V)( A ∞ F ) × V( A ∞ E ) is regularly supported at some nonar-chimedean place v of F if we can write F = F v ⊗ F v in which F v , a function onU(V)( F v ) × V( E v ), is regularly supported in the sense of (3). OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 47 Proposition 5.15. Let K, f , φ , Z K be as in Definition 5.12.(1) The cycles ∆ X K × T f K × Z ♥ K and p ∗ ∆ X K intersect properly in X K .(2) If f ⊗ φ is regularly supported at some nonarchimedean place v of F , then p ∗ ∆ X K and (∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K have empty intersection on X K .Proof. For (1), we have to show that every irreducible component C of the intersection of ∆ X K × T f K × Z ♥ K and p ∗ ∆ X K has dimension 2 n − 3. However, it is easy to see that C is a closed subschemeof the fiber product ∆ X K × ( X K × X K × X K ) ( X K × Y × Z ) ’ Y × X K Z where Y (resp. Z ) is an irreducible component in the support of T f K (resp. Z ♥ K ). But then themorphism Y → X K is finite étale, and Z has dimension 2 n − 3. Thus, C has dimension at most 2 n − p ∗ ∆ X K is a regular subscheme, the dimension of C is at least 2 n − X K ∩ T f K ∩ Z ♥ K is empty in X K × X K .As ∆ X K ∩ Z ♥ K = Z ( φ ) K , we have to show T f K ∩ Z ( φ ) K = ∅ , which can be checked on X K ( C ) × X K ( C ).By complex uniformization, we have X K ( C ) = U(V)( F ) \ ( D × U(V)( A ∞ F ) /K )where D is the corresponding hermitian domain of dimension n − f ≡ 0, then there is nothing to prove. Otherwise, we have φ (0) = 0. Thus, we need to showthat for every x ∈ V( E ) + and g, h ∈ U(V)( A ∞ F ), if f ( h ) φ ( g − x ) = 0, then T KhK ∩ Z ( x, g ) = ∅ in X K ( C ) × X K ( C ). We prove by contradiction. Let D x ⊆ D be the subdomain that is perpendicularto x . If T KhK ∩ Z ( x, g ) = ∅ , then we may find z ∈ D , g ∈ U(V)( A ∞ F ), h ∈ KhK , z x ∈ D x , g x ∈ U(V x )( A ∞ F ), and ξ ∈ U(V)( F ), such that ( z x , g x g ) = ( z , g ) and ( z x , g x g ) = ξ ( z , g h ). Theserelations imply that z x = ξz x and g x g = ξg x gh . The second equality implies that h i g − x = g − g − x ξ − i x for i (cid:62) 0. Now since z x = ξz x , the vectors { x, ξ − x, . . . , ξ − ( n − x } ⊆ V( E ) are linearly dependent. Inparticular, the pair ( h v , g − v x ) ∈ U(V)( F v ) × V( E v ) is not regular semisimple, which is a contradiction.Thus, (2) follows. (cid:3) Of course, one would like to know whether p ∗ ∆ z X K and (∆ X K × T f K × Z ♥ K ) . p ∗ ∆ z X K have emptyintersection, by choosing a cycle representative of z K . At this moment, we do not find a uniform answerto this question since it highly depends on the projector z K . On the other hand, the contribution ofthe difference ∆ X K − ∆ z X K in the height pairing should be negligible in the comparison of relativetrace formulae. In what follows, we will only consider ∆ X K in the decomposition into local heights,suggested by Proposition 5.15. In this article, we only consider the local heights at good inert primes,which we now explain. Definition 5.16. We say that a prime p of F is a good inert prime (with respect to K, f , φ ) if • p is inert in E ; • the underlying rational prime p is odd and unramified in E ; • if we denote by p the set of all primes of F above p that are inert in E , then there exists aself-dual lattice Λ q ⊆ V( F q ) for every q ∈ p such that – K = K p × Q q ∈ p K q in which K q is the stabilizer of Λ q for every q ∈ p , – f = f p ⊗ N q ∈ p f q in which f q = K q , – φ = φ p ⊗ N q ∈ p φ q in which φ q = Λ q .We fix a good inert prime p . From now on, we work in the category Sch /O E p .Let X K be the canonical integral model of X K over O E p (Definition C.21), which is a proper smoothscheme in Sch /O E p of relative dimension n − 1. Then the Zariski closure of T f K in X K × X K is an étalecorrespondence, which will be denoted by the same notation. Let Z K (resp. Z ( φ ) K ) be the Zariski closure of Z K (resp. Z ( φ ) K ) in X K × X K (resp. X K ). Similar to Z ♥ K , we put Z ♥ K := Z K − p ∗ (∆ X K . Z K − Z ( φ ) K ) , (5.9)which is a formal series of divisors on X K , whose generic fiber is Z ♥ K .From now on, we work in the category Sch /O E p . Definition 5.17 (Local arithmetic invariant functional) . Let K, f , φ , Z K be as in Definition 5.12 suchthat f ⊗ φ is regularly supported at some nonarchimedean place v of F , we define the local arithmeticinvariant functional at (a good inert prime) p to be I K ( f , φ ) p := 2 log | O F / p | · χ (cid:18) O ( p ∗ ∆ X K ) ⊗ L O X K O ((∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K ) (cid:19) where χ denotes the Euler–Poincaré characteristic (see Remark 5.18 below); and for a formal series P j c j Z j of cycles on X K , we put O ( P j c j Z j ) := P j c j O Z j as a formal series of O X K -modules. Remark . For a Noetherian scheme X , we denote by D bcoh ( X ) the bounded derived category of O X -modules with coherent cohomology. By Proposition 5.15(2), O ( p ∗ ∆ X K ) ⊗ L O X K O ((∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K )is a formal series in D bcoh ( X K ⊗ Z F p ), which implies that its Euler–Poincaré characteristic is a formalseries in C . Proposition 5.19. In the situation of Definition 5.17, we have I K ( f , φ ) p = 2 log | O F / p | · χ (cid:18) O ( T f K ) ⊗ L O X K O (∆ Z ( φ ) K ) (cid:19) . Proof. First, by the same argument for Proposition 5.15(1), we know that ∆ X K × T f K × Z ♥ K and p ∗ ∆ X K have proper intersection on X K . Since ∆ X K , every component of T f K , and Z ♥ K are allCohen–Macaulay schemes, we have O ((∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K ) = O (∆ X K × T f K × Z ♥ K ) ⊗ O X K O ( p ∗ ∆ X K )= O (∆ X K × T f K × Z ♥ K ) ⊗ L O X K O ( p ∗ ∆ X K ) . Thus, we have O ( p ∗ ∆ X K ) ⊗ L O X K O ((∆ X K × T f K × Z ♥ K ) . p ∗ ∆ X K )= O ( p ∗ ∆ X K ) ⊗ L O X K O (∆ X K × T f K × Z ♥ K ) ⊗ L O X K O ( p ∗ ∆ X K )= (cid:18) O ( p ∗ ∆ X K ) ⊗ L O X K O ( p ∗ ∆ X K ) (cid:19) ⊗ L O X K O (∆ X K × T f K × Z ♥ K )= O (∆ ( X K × X K )) ⊗ L O ( X K ×X K )3 O (∆ X K × T f K × Z ♥ K ) . Restricting to X K , we have I K ( f , φ ) p = 2 log | O F / p | · χ (cid:18) O (∆ X K ) ⊗ L O X K O ( T f K ) ⊗ L O X K O ( Z ♥ K ) (cid:19) . By (5.9), ∆ X K and Z ♥ K have proper intersection. Since both have Cohen–Macaulay components, wehave O (∆ X K ) ⊗ L O X K O ( Z ♥ K ) ’ O (∆ X K ) ⊗ O X K O ( Z ♥ K ) = O (∆ X K ∩ Z ♥ K ) = O (∆ Z ( φ ) K ) . It is clear that v can not be in p . The reason we add the factor 2 in front of log | O F / p | is the following: I K ( f , φ ) p is supposed to “approximate” thelocal term of I zK ( f , φ ) at the unique place u of E above p , hence the factor c ( u ) in (3.1) is log | O E / p O E | = 2 log | O F / p | . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 49 The proposition then follows. (cid:3) Orbital decomposition of local arithmetic invariant functional. To further study theintersection number in Proposition 5.19, we need certain moduli interpretation of the integral model X K and Z ( φ ) K . We will follow the discussion and notation in Subsection C.4. Definition 5.20. A frame for the (good inert) prime p (with the underlying rational prime p ) containsthe following • a CM type Φ of E containing the fixed embedding τ , • a rational skew-hermitian space W ∞ over A ∞ E of rank 1 such that W ( W ∞ , Φ c ) is nonemptyand W ∞ ⊗ A ∞ Q p admits a self-dual lattice, • a sufficiently small open compact subgroup L = L p × ( L ) p of H ∞ ( A ∞ ) in which ( L ) p is thestabilizer of a self-dual lattice in W ∞ ⊗ A ∞ Q p , where H ∞ is the group of similitude of W ∞ , • an isomorphism between the two E -extensions C and E ac p , • a point P : Spec O E nr p → M (V , W ∞ , Φ) nr K p ,L as (C.8), whose reduction is in the supersingularlocus, where E nr p is the maximal unramified extension of E p contained in E ac p .Now we take a frame. Put X nr K := X K ⊗ O E p O E nr p . By Remark C.22, the point P provides us witha Cartesian diagram X nr K (cid:47) (cid:47) (cid:15) (cid:15) Spec O E nr p q ◦ P (cid:15) (cid:15) M (V , W ∞ , Φ) nr K,L q (cid:47) (cid:47) M ( W ∞ , Φ c ) L ⊗ O E Φ , ( p ) O E nr p (5.10)of schemes over O E nr p . In particular, for every locally Noetherian scheme S over O E nr p , the set X nr K ( S )is the set of equivalent classes of nonuples ( A , i , λ , η p ; A, i, λ, η p , η spl p ) in which ( A , i , λ , η p ) is thebase change of ( A , i , λ , η p ) to S .We introduce the moduli interpretation of integral special divisors. Denote by Spl p the set of primesof F above p that are split in E . Definition 5.21. For x ∈ V( E ) + and g p = ( g p , g q | q ∈ Spl p ) ∈ U(V)( A ∞ F , p ), we define a relativefunctor s x,g p : Z ( x, g p ) nr K → X nr K in the way that the fiber over a point ( A , i , λ , η p ; A, i, λ, η p , η spl p ) ∈ X nr K ( S ) consists of ρ ∈ Hom S (( A , i ) , ( A, i A )) ⊗ O F O F, ( p ) such that for every geometric point s of S , • ρ ∗ ∈ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A s , A ∞ ,p ) , H ´et1 ( A s , A ∞ ,p )) belongs to η p (( g p ) − x ); • ρ ∗ ∈ Q q ∈ Spl p Hom O E q − ( A s [( q − ) ∞ ] , A s [( q − ) ∞ ]) ⊗ O E q − E q − belongs to η spl p (( g − q x ) q ∈ Spl p ). Proposition 5.22. For x ∈ V( E ) + and g p ∈ U(V)( A ∞ F , p ) , we have(1) The relative morphism s x,g p is representable, finite, and unramified.(2) There is an isomorphism s x,g p ⊗ O E nr p E nr p ’ a ( g q | q ∈ p ) ,g q ∈ U(V x )( F q ) \ U(V)( F q ) /K q ,g − q x ∈ Λ q s x, ( g p ,g q | q ∈ p ) ⊗ E E nr p of relative functors over X K ⊗ E E nr p , where s x, ( g p ,g q | q ∈ p ) is defined in Definition 5.3.(3) For every point z ∈ Z ( x, g ) nr K ( k ) , the induced ring homomorphism R y → R z is surjective whosekernel is a principal ideal that is not contained in pR y . Here R z (resp. R y ) denotes completedlocal ring of X nr K (resp. Z ( x, g ) nr K ) at z (resp. y := s x,g p ( z ) ). Proof. Part (1) follows from the same argument in the proof of [KR14, Proposition 2.9].For (2), put X nr K := X K ⊗ E p E nr p . For every point P = ( A , i , λ , η p ; A, i, λ, η p , η spl p ) ∈ X nr K ( S ), wewill construct a functorial bijection s − x,g p P ∼ −→ ‘ g s − x,g P between the fibers.For the forward direction, take an element ρ as in Definition 5.21. Let ( A ρ , i ρ ) be the quotient abelianscheme ( A/ρ ( A ) , i ), which is naturally an ( E, sig V , Φ − Φ c )-abelian scheme (Definition C.10). Denoteby % : A → A ρ the quotient homomorphism, and define a homomorphism ρ := cλ − ◦ ρ ∨ ◦ λ : A → A for some c ∈ Z × ( p ) . Then we obtain a prime-to- p isogeny ( %, ρ ) : A → A ρ × A . Let λ ρ be the induced p -principal polarization of ( A ρ , i ρ ). Choose a representative η p in its K p -class such that % ∗ ◦ η p (( g p ) − x )is the zero map. We define η pρ to be the compositionV x ( A ∞ ,pF ) , → V( A ∞ ,pF ) η p −→ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A s , A ∞ ,p ) , H ´et1 ( A s , A ∞ ,p )) % ∗ ◦ −−→ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A s , A ∞ ,p ) , H ´et1 ( A ρs , A ∞ ,p )) . Let η px : A ∞ ,pE x → Hom E ⊗ Q A ∞ ,p (H ´et1 ( A s , A ∞ ,p ) , H ´et1 ( A s , A ∞ ,p )) be the homomorphism sending x to c · ( x, x ) V . Then we have ( η pρ ⊕ η px ) ◦ ( g p ) − = ( %, ρ ) ∗ ◦ η p . We have a similar construction for η spl pρ , whose details we omit. Finally, we obtain ( A , i , λ , η p ; A ρ , i ρ , λ ρ , η pρ , η spl pρ ) together with the O E -linear prime-to- p isogeny ( %, ρ ) : A → A ρ × A , which provides an element in the fiber s − x,g P where g = ( g p , g q | q ∈ p ) is (a representative in) the unique double coset in the disjoint union satisfying g − q x = ρ ∗ under any isomorphism Λ q ’ Hom O E q (T q A s , T q A s ) of hermitian lattices over O E q , whereT q denotes the q -adic Tate module.For the backward direction, take an element in the fiber s − x,g P for g in the disjoint union, given bydata ( A , i , λ , η p ; A , i , λ , η p , η spl p ) ∈ Sh(G x , h V x ,τ ) gKg − ∩ G x ( A ∞ ) ( S )together with an O E -linear prime-to- p isogeny ρ : A → A × A satisfying relevant properties. We justtake ρ as the composite homomorphism A λ −→ A ∨ ρ | A ∨ −−−−→ A ∨ cλ − −−−→ A for some c ∈ O × F, ( p ) . It is straightforward to check that the above constructions are inverse to eachother; hence (2) is proved.For (3), let I be the kernel of R y → R z . To show that I is principal, we follow the strategy in theproof of [How15, Proposition 3.2.3] for the case F = Q . Let ( A , i , λ , η p ; A, i, λ, η p , η spl p ) be theuniversal object over R y , which is equipped with the universal O E -linear homomorphism ρ : ( A ) R z → A R z . It suffices to study the obstruction to lifting ρ to a homomorphism A S → A S where S := R y / m I with m the maximal ideal of R y . Note that the Hodge exact sequence0 → Fil H dR1 ( A ) → H dR1 ( A ) → Lie( A ) → → Fil H dR1 ( A ) q → H dR1 ( A ) q → Lie( A ) q → q of F above p , in which H dR1 ( A ) q is the direct summand of H dR1 ( A ) on which O F, ( p ) acts via the prime q . We have a similar splitting for A . Moreover H dR1 ( A ) q (resp. H dR1 ( A ) q ) isa free O E q ⊗ Z p R y -module of rank 1 (resp. n ). By the signature condition, the obstruction to lifting ρ coincides with the obstruction for the canonical lifting ˜ ρ ∗ : H dR1 ( A S ) p → H dR1 ( A S ) p to respect Hodgefiltration. The remaining argument is then same as [How15, p.668] by taking j := π ⊗ S − ⊗ π S forsome π ∈ O × E p ∩ E − p . Note that, j . Lie( A k ) is always nonzero in our case.Finally, we show that I is not contained in pR y . If it is, then by (1) the image of s x,g p contains theentire connected component of ( X nr K ) k at y . Thus, for every k point ( A , i , λ , η p ; A, i, λ, η p , η spl p ) in Note that [How15] considers all residue characteristics; while we only consider p that is unramified in E . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 51 this connected component, there exists a nonzero homomorphism from ( A , i ) to ( A, i ). In particular,( A, i ) is not µ -ordinary, which contradicts to the main theorem of [Wed99] saying that µ -ordinary locusis dense. Here, we apply [Wed99] to the PEL type moduli scheme in Remark C.13 parameterizing( A, i, λ, ˜ η p ) where ˜ η p is an away-from- p level structure induced from λ p and η p . Thus, (3) is proved. (cid:3) By the above proposition, ( s x,g p ) ∗ Z ( x, g p ) nr K is a relative divisor on X nr K . By abuse of notation, westill denote Z ( x, g p ) nr K for the cycle ( s x,g p ) ∗ Z ( x, g p ) nr K . The following corollary is immediate. Corollary 5.23. Let p be a good inert prime. If φ (0) = 0 , then we have Z ( φ ) K ⊗ O E p O E nr p = X x ∈ U(V)( F ) \ V( E ) + e − π · Tr F/ Q ( x,x ) V X g p ∈ U(V x )( A ∞ F , p ) \ U(V)( A ∞ F , p ) /K p φ p (( g p ) − x ) Z ( x, g p ) nr K as a formal series in Z ( X nr K ) C .Proof. By Proposition 5.22, the relative divisor Z ( x, g p ) nr K is the Zariski closure of X ( g q | q ∈ p ) ,g q ∈ U(V x )( F q ) \ U(V)( F q ) /K q Y q ∈ p Λ q ( g − q x ) · Z ( x, ( g p , g q | q ∈ p )) K in X nr K . The corollary follows since φ = φ p ⊗ N q ∈ p φ q in which φ q = Λ q . (cid:3) Lemma 5.24. Let K, f , φ be as in Definition 5.12 such that f ⊗ φ is regularly supported at somenonarchimedean place v of F . For a point y = ( A , i , λ , η p ; A, i, λ, η p , η spl p ) ∈ X nr K ( k ) , if ( y, y ) belongsto both T f K and the support of ∆ Z ( φ ) K , then the strict O F p -module A [ p ∞ ] is supersingular.Proof. By Corollary 5.23, it suffices to consider T KhK ∩ Z ( x, g ) nr K for some g, h ∈ U(V)( A ∞ F , p ) and x ∈ V( E ) + such that ( h v , g − v x ) is regular semisimple for some nonarchimedean place v p . We onlyconsider the case where v is not above p , and leave the similar case where v ∈ Spl p to the reader.Let ( y, y ) be a k -point in T KhK ∩ Z ( x, g ) nr K with y as in the lemma. By the moduli inter-pretation, there is a coprime-to- p isogeny ξ : A → A such that ξ ∗ η p = η p ◦ h p , and an element ρ ∈ Hom k (( A , i ) , ( A, i )) ⊗ O F O F, ( p ) such that ρ ∗ ∈ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A , A ∞ ,p ) , H ´et1 ( A, A ∞ ,p )) be-longs to η p (( g p ) − x ). Consider the situation at v . We may choose a representation κ v in the K v -class of η pv such that ρ ∗ v = η pv ( g − v x ). Possibly replacing h v by some element in K v h v K v , we have( ξ i ◦ ρ ) ∗ v = η pv ( h iv g − v x ) for every integer i (cid:62) 0. Since ( h v , g − v x ) is regular semisimple, { ( ξ i ◦ ρ ) ∗ v , i (cid:62) } generates Hom O Ev (T v A , T v A ) Q as an E v -module where T v denotes the v -adic Tate module. In par-ticular, Hom k (( A , i ) , ( A, i )) Q has dimension n over E . Thus, A [ p ∞ ] is isogenous to A [ p ∞ ] ⊕ n , whichis then supersingular. (cid:3) In Subsection C.4, we define the supersingular locus M (V , W ∞ , Φ) ss K p ,L . For K as above, wedefine M (V , W ∞ , Φ) ss K,L to be the image of M (V , W ∞ , Φ) ss K p ,L under the natural quotient morphism M (V , W ∞ , Φ) K p ,L → M (V , W ∞ , Φ) K,L . Define X ss K as the preimage of M (V , W ∞ , Φ) ss K,L underthe left vertical morphism in the diagram (5.10), which is a Zariski closed subset of X nr K ⊗ O E nr p k .Finally, let X ss , ∧ K be the completion of X nr K along X ss K . Proposition C.26 provides us with the followinguniformization isomorphism X ss , ∧ K ’ U( ¯V)( F ) \ (cid:16) N × U( ¯V)( A ∞ F , p ) / ¯ K p (cid:17) (5.11)depending on the frame we chose, in particular, the point P . Here, ¯ K p = ¯ K p × Q q ∈ p \{ p } ¯ K q ,where ¯ K p = K p under the isomorphism ι P ; and ¯ K q is the stabilizer of ¯Λ q in Lemma C.25(6). The However, one can show that the supersingular locus X ss K itself is intrinsic, which does not depend on the choice offrame. uniformization isomorphism is functorial in K p and Hecke translations. We recall the new hermitianspace ¯V := Hom k (( A k , i k ) , ( A k , i k )) Q equipped with the hermitian form (C.11), satisfying Lemma C.25, which is “ p -nearby to V ”. Inparticular, we have an isomorphism¯V ⊗ F F p ’ Hom k (( X k , i k ) , ( X k , i k )) Q . Applying the constructions from Subsection 1.3, we have for every nonzero ¯ x ∈ ¯V( E p ), a sub-formal scheme Z (¯ x ) of N ; and for every ¯ g ∈ U( ¯V)( F p ), an isomorphism ¯ g : N → N with its graphΓ ¯ g ⊆ N := N × O ∧ E nr p N . Now we arrive at the theorem on the orbital decomposition. Theorem 5.25. Let K, f , φ be as in Definition 5.12 such that f ⊗ φ is regularly supported at somenonarchimedean place v of F . For a good inert prime p , we have I K ( f , φ ) p = 2 log | O F / p | · X (¯ ξ, ¯ x ) ∈ [U( ¯V)( F ) × ¯V( E ))] rs e − π · Tr F Q (¯ x, ¯ x ) ¯V Orb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) · χ (cid:16) O Γ ¯ ξ ⊗ L O N O ∆ Z (¯ x ) (cid:17) after choosing a frame (Definition 5.20). Here, we define the orbital integral as Orb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) := Z U( ¯V)( A ∞ F , p ) ¯ f p (¯ g − ¯ ξ ¯ g ) ¯ φ p (¯ g − ¯ x ) d¯ g where • ¯ f p = ¯ f p ⊗ N q ∈ p \{ p } ¯ f q in which ¯ f p = f p under the isomorphism ι P (C.13) , and ¯ f q = ¯ K q ; • ¯ φ p = ¯ φ p ⊗ N q ∈ p \{ p } ¯ φ q in which ¯ φ p = φ p under the isomorphism ι P , and ¯ φ q = ¯Λ q ; • d¯ g is the Haar measure on U( ¯V)( A ∞ F , p ) such that ¯ K p has volume vol( K ) .In particular, the Euler–Poincaré characteristic appearing in the formula is finite for every regularsemisimple pair ( ¯ ξ, ¯ x ) .Proof. We choose a representative x ∈ V( E ) + in the coset U(V)( F ) \ V( E ) + . We first compute X g p =( g p ,g q | q ∈ Spl p ) ∈ U(V x )( A ∞ F , p ) \ U(V)( A ∞ F , p ) /K p φ p (( g p ) − x ) Z ( x, g p ) ss , ∧ K (5.12)where Z ( x, g p ) ss , ∧ K is the formal completion of Z ( x, g p ) nr K along the supersingular locus. Let S be a connected scheme in Sch /O nr E p on which p is locally nilpotent, and take a point( A , i , λ , η p ; A, i, λ, η p , η spl p ) ∈ Z ( x, g p ) ss , ∧ K ( S ). Then we can choose an element ¯ x ∈ ¯V and an E -linear quasi-isogeny ρ : A → A k × k S over S such that • the image of ρ − ∗ ◦ ¯ x ∗ in Hom E ⊗ Q A ∞ ,p (H ´et1 ( A s , A ∞ ,p ) , H ´et1 ( A s , A ∞ ,p )) belongs to η p (( g p ) − x ); • the image of ρ − ∗ ◦ ¯ x ∗ in Q q ∈ Spl p Hom O E q − ( A s [( q − ) ∞ ] , A s [( q − ) ∞ ]) ⊗ O E q − E q − belongs to η spl p (( g − q x ) q ∈ Spl p ); • ρ − ◦ ¯ x lifts to an O E -linear homomorphism A [ q ∞ ] → A [ q ∞ ] for every q ∈ p .Here, we note that ( A , i , λ , η p ) is identified with the base change of ( A , i , λ , η p ) to the closedpoint s . By Proposition C.26, ρ is given by an element ¯ g p ∈ U( ¯V)( A ∞ F , p ) on S . In particular, we have(¯ x, ¯ x ) ¯V = ( x, x ) V . Choose a representative ¯ x in the coset U( ¯V)( F ) \ ¯V( E ) of this norm. Then underthe isomorphism (5.11), we have(5.12) = X ¯ g p ∈ U( ¯V ¯ x )( A ∞ F , p ) \ U( ¯V)( A ∞ F , p ) / ¯ K p ¯ φ p ((¯ g p ) − ¯ x ) · [ Z (¯ x ) , ¯ g p ] , p where [ Z (¯ x ) , ¯ g p ] denotes the corresponding double coset in the right-hand side of (5.11). Comparing the notations with those in Subsection 1.3, we have ( X k , i k , λ k ) = ( X n , i n , λ n ), N = N n , and ¯V ⊗ F F p =V − n . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 53 By linearity, we may assume f = KhK for some h ∈ U(V)( A ∞ F ) with h p = 1. In particular, T f K = vol( K ) T KhK . By Proposition C.26, the formal completion of T KhK in ( X ss , ∧ K ) is simply the set-theoretical Hecke correspondence T ¯ K p ¯ h p ¯ K p under the isomorphism (5.11) by Proposition C.26, where¯ h pq = 1 for q ∈ p \ { p } . We first analyze the intersection T ¯ K p ¯ h p ¯ K p ∩ ∆[ Z (¯ x ) , ¯ g p ]. If the intersectionis nonempty, then [ Z (¯ x ) , ¯ g p ¯ h p ] and [ Z (¯ x ) , ¯ g p ] are in the same connected component. By (5.11), thereexists ¯ ξ ∈ U( ¯V)( F ) such that ¯ ξ ¯ g p ¯ K p = ¯ g p ¯ h p ¯ K p , that is, KhK ((¯ g p ) − ¯ ξ ¯ g p ) = 1. Moreover, if we fixa set of representatives of the orbit of U( ¯V)( F ) under conjugation, then one can always choose ¯ ξ tobe one of the representative. Now we think conversely, for any such representative ¯ ξ , the cosets ¯ g p ¯ K p satisfying ¯ ξ ¯ g p ¯ K p = ¯ g p ¯ h p ¯ K p are those satisfying KhK ((¯ g p ) − ¯ ξ ¯ g p ) = 1. In this case, the intersection T ¯ K p ¯ h p ¯ K p ∩ ∆[ Z (¯ x ) , ¯ g p ] is isomorphic to the image of Γ ¯ ξ ∩ ∆ Z (¯ x ) under the quotient morphism N → ( C \N ) for some subgroup C ⊆ U( ¯V)( F p ) acting on N discretely.Now we claim that Γ ¯ ξ ∩ ∆ Z (¯ x ) is a proper scheme in Sch /k . By definition, we have ¯ ξ Z (¯ x ) = Z ( ¯ ξ ¯ x ). Itfollows that Γ ¯ ξ ∩ ∆ Z (¯ x ) is isomorphic to a closed sub-formal scheme of T n − i =0 Z ( ¯ ξ i ¯ x ), whose underlyingreduced scheme is a proper scheme in Sch /k by [KR11, Theorem 4.12] for F p = Q p and [Cho] ingeneral. Thus, the underlying reduced scheme of Γ ¯ ξ ∩ ∆ Z (¯ x ) is of finite type over k . By the previousdiscussion, it suffices to show that T ¯ K p ¯ h p ¯ K p ∩ ∆[ Z (¯ x ) , ¯ g p ] is a scheme of finite type over k . However,this follows from Lemma 5.24. As a consequence, χ (cid:16) O Γ ¯ ξ ⊗ L O N O ∆ Z (¯ x ) (cid:17) is finite. Moreover, it is equalto χ (cid:16) O T ¯ K p ¯ h p ¯ K p ⊗ L O N O ∆[ Z (¯ x ) , ¯ g p ] (cid:17) . Therefore, the theorem follows from (5.12) and Lemma 5.24. (cid:3) Remark . We believe that a more general notion of “good inert prime”, for which a result similarto Theorem 5.25 holds, should just be a prime p of F that is inert in E , and such that there is aself-dual lattice Λ p ⊆ V( F p ) satisfying • K = K p × K p in which K p is the stabilizer of Λ p , • f = f p ⊗ f p in which f p = K p , • φ = φ p ⊗ φ p in which φ p = Λ p .In the formula for I K ( f , φ ) p in Theorem 5.25, the orbital integral has the decompositionOrb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) = Orb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) · Y q ∈ p \{ p } Orb( ¯ K q , ¯Λ q ; ¯ ξ, ¯ x )in which we decompose the Haar measure on U( ¯V)( A ∞ F , p ) such that ¯ K q has volume 1 for every q ∈ p \ { p } .We now compare the term2 log | O F / p | · Y q ∈ p \{ p } Orb( ¯ K q , ¯Λ q ; ¯ ξ, ¯ x ) · χ (cid:16) O Γ ¯ ξ ⊗ L O N O ∆ Z (¯ x ) (cid:17) with orbital integrals on the general linear side. Recall notations Mat r,s and M n from Subsection 1.7.Let S n be the F -subscheme of Res E/F Mat n,n consisting of matrices g satisfying g · g c = I n . Definition 5.27 ([Liu14, Section 5.3] ) . Consider a field extension F /F and put E := E ⊗ F F .(1) We say that a pair of elements ( ζ, y ) ∈ S n ( F ) × M n ( F ) is regular semisimple if the matrix( y ζ i + j − y ) ni,j =1 is invertible in E , where we write y = ( y , y ) for y ∈ Mat n, ( F ) and y ∈ Mat ,n ( F ).(2) The group GL n ( F ) acts on S n ( F ) × M n ( F ) via the formula ( ζ, y , y ) .g = ( g − ζg, g − y , y g ),which preserves regular semisimple pairs. Denote by [S n ( F ) × M n ( F )] the orbits of S n ( F ) × M n ( F ) under the above action, and by [S n ( F ) × M n ( F )] rs the subset of regular semisimpleorbits. Note that we have changed the roles of rows and columns from [Liu14], in order to match the convention of generatingseries. (3) Suppose that F = F v for some place v of F . For a regular semisimple pair ( ζ, y ) ∈ S n ( F ) × M n ( F ), we define its local transfer factor to be ω v ( ζ, y ) := µ E/F (det( y , ζy , . . . , ζ n − y )).We denote by [S n ( F ) × M n ( F )] ± rs the subset of [S n ( F ) × M n ( F )] rs of orbits ( ζ, y ) such that µ E/F (det( y ζ i + j − y ) ni,j =1 ) = ± ζ, y ) ∈ [S n ( F ) × M n ( F )] rs and ( ¯ ξ, ¯ x ) ∈ [U( ¯V)( F ) × ¯V( E )] rs (Definition 5.14) match if • ζ and ¯ ξ have the same characteristic polynomial as elements in Mat n,n ( E ); • y ζ i y = ( ¯ ξ i ¯ x, ¯ x ) ¯V for 0 (cid:54) i (cid:54) n − Corollary 5.28. In the situation of Theorem 5.25, suppose that for every orbit ( ¯ ξ, ¯ x ) ∈ [U( ¯V)( F ) × ¯V( E ))] rs , Conjecture 1.9(2) for E q /F q for every q ∈ p \ { p } and Conjecture 1.12 for E p /F p hold. Thenwe have I K ( f , φ ) p = − X (¯ ξ, ¯ x ) ∈ [U( ¯V)( F ) × ¯V( E ))] rs e − π · Tr F/ Q (¯ x, ¯ x ) ¯V Orb(¯ f p , ¯ φ p ; ¯ ξ, ¯ x ) · dd s (cid:12)(cid:12)(cid:12)(cid:12) s =0 Y q ∈ p ω q ( ζ, y ) Orb( s ; S n ( O F q ) , M n ( O F q ) ; ζ, y ) where ( ζ, y ) ∈ [S n ( F ) × M n ( F )] rs is the unique orbit that matches ( ¯ ξ, ¯ x ) .Proof. It suffices to note that Orb(0; S n ( O F p ) , M n ( O F p ) ; ζ, y ) = 0, which is Conjecture 1.9(1) and isknown (see Remark 1.10). (cid:3) Remark . To obtain a global result, we would like to find test functions ˜ f p , ˜ φ p on the general linearside, in order to obtain some matching relation with the local intersection number I K ( f , φ ) v at everyplace v of F . If v is split in E , then it is expected that I K ( f , φ ) v should vanish, and the matching testfunctions ˜ f v , ˜ φ v should be obtained from f v , φ v in an elementary way as in [Liu14, Proposition 5.11].If v is neither split nor a good inert prime, then we do not know how to do in general at this moment. Appendix A. Proof of arithmetic fundamental lemma for the minuscule caseby Chao Li and Yihang Zhu The purpose of this appendix is to prove the arithmetic fundamental lemma for U( n ) × U( n ), namely,Conjecture 1.12, in the minuscule case. We follow the setup and notation in Subsection 1.3. A.1. Derivatives of orbital integrals via lattice counting. We take a regular semisimple orbit( ζ, y ) ∈ [S n ( F ) × M n ( F )] − rs , where y = ( y , y ) ∈ Mat n, ( F ) × Mat ,n ( F ). Let ( ξ, x ) ∈ [U(V − n )( F ) × V − n ( E )] rs be the unique orbit that matches ( ζ, y ). By definition, ζ and ξ have the same characteristicpolynomial; and we have y ζ i y = ( ξ i x, x ) , i = 0 , . . . , n − . (A.1)Recall that we denote v ( ζ, y ) := val(det( y , ζy , . . . , ζ n − y )), and define the transfer factor as ω ( ζ, y ) =( − v ( ζ,y ) . We also denote ∆( ζ, y ) := det( y ζ i + j − y ) ni,j =1 and δ ( ζ, y ) = val(∆( ζ, y )). As ( ζ, y ) ∈ [S n ( F ) × M n ( F )] − rs , we know that δ ( ζ, y ) is odd. This can be rigorously shown if the level structure at v is hyperspecial maximal; but we decide not to include thedetails in this article. The typesetting editor should feel free to place the contact information of the authors of the appendix accordingto journal’s rule: Chao Li: Department of Mathematics, Columbia University, 2990 Broadway, New York, NY 10027.Email: [email protected]. Yihang Zhu: Department of Mathematics, Columbia University, 2990 Broadway,New York, NY 10027. Email: [email protected]. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 55 Define two O E -lattices L = L ζ,y := O E y ⊕ O E ζy ⊕ · · · ⊕ O E ζ n − y ⊆ Mat n, ( E ) ,L = L ζ,y := O E y ⊕ O E y ζ ⊕ · · · ⊕ O E y ζ n − ⊆ Mat ,n ( E ) . For every integer i (cid:62) 0, we define the set M i ( ζ, y ) := { O E -lattice Λ ⊆ Mat n, ( E ) | L ⊆ Λ , L ⊆ Λ ∨ , Λ c = Λ , ζ Λ = Λ , length O E (Λ /L ) = i } where ∨ denotes dual lattice under the standard sesquilinear formMat n, ( E ) × Mat ,n ( E ) → E, ( x , x ) x c · x . (A.2) Lemma A.1. We have dd s (cid:12)(cid:12)(cid:12)(cid:12) s =0 Orb( s ; S n ( O F ) , M n ( O F ) ; ζ, y ) = − q · ω ( ζ, y ) X i (cid:62) ( − i ( v ( ζ, y ) − i ) · M i ( ζ, y ) . Proof. By definition, we haveOrb( s ; S n ( O F ) , M n ( O F ); ζ, y ) = Z GL n ( F ) S n ( O F ) ( g − ζg ) M n ( O F ) ( g − y , y g ) µ E/F (det g ) | det g | sE d g. Notice that ( g − y , y g ) belongs to M n ( O F ) if and only if y ∈ g Mat n, ( O F ) and y ∈ Mat ,n ( O F ) g − hold. We also notice that g − ζg belongs to S n ( O F ) if and only if ζg Mat n, ( O E ) = g Mat n, ( O E ) andMat ,n ( O E ) g − ζ = Mat ,n ( O E ) g − hold. Moreover, the O E -lattice g Mat n, ( O E ) is invariant underthe involution c , and is dual to Mat ,n ( O E ) g − under the pairing (A.1). It follows that the assignment g Λ = Λ( g ) := g Mat n, ( O E )induces a bijection between the set { g ∈ GL n ( F ) / GL n ( O F ) | ( g − y , y g ) ∈ M n ( O F ) , g − ζg ∈ S n ( O F ) } , and { O E -lattice Λ ⊆ Mat n, ( E ) | y ∈ Λ , y ∈ Λ ∨ , Λ c = Λ , ζ Λ = Λ } which is equal to { O E -lattice Λ ⊆ Mat n, ( E ) | L ⊆ Λ , L ⊆ Λ ∨ , Λ c = Λ , ζ Λ = Λ } . Clearly it further induces a bijection between such elements g with val(det g ) = i and such O E -latticesΛ with length O E (Λ /L ) = i , namely, the set M i ( ζ, y ). Now notice that we haveval(det g ) = length O E (Mat n, ( O E ) /L ) − length O E (Λ /L )and length O E (Mat n, ( O E ) /L ) = val(det( y , ζy , . . . , ζ n − y )) = v ( ζ, y ) . It follows that if length O E (Λ /L ) = i , then we have µ E/F (det g ) = ( − v ( ζ,y ) − i = ( − i · ω ( ζ, y ) , dd s (cid:12)(cid:12)(cid:12)(cid:12) s =0 | det( g ) | sE = − q · ( v ( ζ, y ) − i ) . The lemma is proved. (cid:3) Define isomorphisms of E -vector spaces φ : Mat n, ( E ) → V − n ( E ) , ζ i y ξ i x, i = 0 , . . . , n − , and φ : Mat ,n ( E ) → V − n ( E ) , y ζ i ξ i x, i = 0 , . . . , n − . By (A.1), the standard sesquilinear form (A.2) transfers to the hermitian form on V − n ( E ) under φ × φ .It is clear that under φ , the unique F -linear involution Mat n, ( E ) → Mat n, ( E ) sending a · ζ i y to a c · ( ζ i ) c y = a c · ζ − i y for every a ∈ E and i = 0 , . . . , n − F -linear involution τ : V − n ( E ) → V − n ( E ) satisfying τ ( a · ξ i x ) = a c · ξ − i x for every a ∈ E and i = 0 , . . . , n − Define the O E -lattice L = L ξ,x := O E x ⊕ O E ξx ⊕ · · · ⊕ O E ξ n − x ⊆ V − n ( E ) . Then we have φ i ( L ( ζ, y i )) = L for i = 1 , 2. For every integer i ≥ 0, we define the set N i ( ζ, y ) := { O E -lattice Λ ⊆ V − n ( E ) | L ⊆ Λ ⊆ L ∗ , ξ Λ = Λ , Λ τ = Λ , length O E (Λ /L ) = i } where ∗ denotes dual lattice under the hermitian form on V − n ( E ). Proposition A.2. We have dd s (cid:12)(cid:12)(cid:12)(cid:12) s =0 Orb( s ; S n ( O F ) , M n ( O F ) ; ζ, y ) = − q · ω ( ζ, y ) δ ( ζ,y ) X i =0 ( − i ( − i ) · N i ( ζ, y ) . Proof. Notice that the isomorphisms φ and φ induce a bijection between the sets M i ( ζ, y ) and N i ( ζ, y ) for every i . As length O E ( L ∗ /L ) = δ ( ζ, y ), we know that N i is empty unless 0 ≤ i ≤ δ ( ζ, y ).Moreover, the assignment Λ Λ ∗ induces an isomorphism between N i ( ζ, y ) and N δ ( ζ,y ) − i ( ζ, y ). Since δ ( ζ, y ) is odd, we have δ ( ζ,y ) X i =0 ( − i v ( ζ, y ) = 0 . Thus, we have δ ( ζ,y ) X i =0 ( − i ( v ( ζ, y ) − i ) · M i ( ζ, y ) = δ ( ζ,y ) X i =0 ( − i ( − i ) · N i ( ζ, y ) . The proposition then follows from Lemma A.1. (cid:3) Remark A.3 . There seems to be a sign error in [RTZ13, Corollary 7.3(2)], which is corrected in themore general Proposition A.2.A.2. The minuscule case. Choose a uniformizer $ of F . From now on we assume that ( ξ, x ) is minuscule , namely, we assume $L ∗ ⊆ L ⊆ L ∗ , where L = L ξ,x as we recall. In this case, L ∗ /L is a vector space over the residue field κ E ∼ = F q of E , which is equipped with an hermitian form induced from V − n . Since ξ ∈ U(V − n )( F ) stabilizes L and L ∗ , we know that ξ induces an action ¯ ξ on L ∗ /L , which is an element in U( L ∗ /L ). We denote by P ( T ) the characteristic polynomial of ¯ ξ on L ∗ /L . Since ¯ ξ belongs to U( L ∗ /L ), we know that P ( T ) is self-reciprocal . Here we recall that for a polynomial R ( T ) = a k T k + · · · + a T + a ∈ κ E [ T ]with a a k = 0, we define its reciprocal polynomial as R ∗ ( T ) := ( a c ) − · T k · R (1 /T ) c ;and we say that R ( T ) is self-reciprocal if R ( T ) = R ∗ ( T ).Now for any irreducible factor R ( T ) of P ( T ), for P ( T ) defined above, we denote the multiplicityof R ( T ) in P ( T ) by m ( R ( T )). Since P ( T ) is self-reciprocal, if R ( T ) is an irreducible factor of P ( T ),then R ∗ ( T ) is also an irreducible factor of P ( T ). Thus, taking reciprocal R ( T ) R ∗ ( T ) inducesan involution on the set of irreducible factors of P ( T ). We denote by NSR the set of all orbits of non-self-reciprocal monic irreducible factors of P ( T ) under this involution. Lemma A.4. If P ( T ) has a unique self-reciprocal monic irreducible factor Q ( T ) such that m ( Q ( T )) is odd, then δ ( ζ,y ) X i =0 ( − i ( − i ) · N i ( ζ, y ) = deg Q ( T ) · m ( Q ( T )) + 12 · Y { R ( T ) ,R ∗ ( T ) }∈ NSR (1 + m ( R ( T ))) . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 57 Otherwise, we have δ ( ζ,y ) X i =0 ( − i ( − i ) · N i ( ζ, y ) = 0 . Proof. This follows from the same proof as [RTZ13, Proposition 8.2]. (cid:3) Put Λ := L ∗ . Since ( ξ, x ) is minuscule, we know that Λ is a vertex lattice, namely, it satisfies $ Λ ⊆ Λ ∗ ⊆ Λ. Let V (Λ) be the Deligne–Lusztig variety associated to the vertex lattice Λ as in[LZ17, Section 2.5] and [RTZ13, Section 3], which is a smooth projective variety over k , where k is analgebraic closure of κ E as in Subsection 1.3. Lemma A.5. We have a canonical isomorphism Γ ξ ∩ ∆ Z n ( x ) ∼ = V (Λ) ¯ ξ . Proof. Notice that we have a canonical isomorphism Γ ξ ∩ ∆ Z n ( x ) ∼ = Z n ( x ) ∩ N ξn . Let N Λ ⊆ N n bethe closed Bruhat–Tits stratum associated to the vertex lattice Λ as in [LZ17, Section 2.6]. Then bydefinition, we have Z n ( x ) ∩ N ξn ∼ = N ξ Λ . By [LZ17, Corollary 3.2.3 & Section 2.6], we have N Λ ∼ = V (Λ). The lemma then follows. (cid:3) Lemma A.6. We have that V (Λ) ¯ ξ is empty unless P ( T ) has a unique self-reciprocal monic irreduciblefactor Q ( T ) such that m ( Q ( T )) is odd. Assume that V (Λ) ¯ ξ is non-empty. Then V (Λ) ¯ ξ is an Artinian k -scheme, and χ ( O Γ ξ ⊗ L O N n O ∆ Z n ( x ) ) = length k V (Λ) ¯ ξ = deg Q ( T ) · m ( Q ( T )) + 12 · Y { R ( T ) ,R ∗ ( T ) }∈ NSR (1 + m ( R ( T ))) . Proof. The result follows directly from Lemma A.5, [LZ17, Corollary 3.2.3], [RTZ13, Proposition 8.1],and [HLZ19, Lemma 5.1.1 & Theorem 4.6.3]. Strictly speaking, these references assume F = Q p ,but the same proof works for general F as long as one replaces results related to the Bruhat–Titsstratification and special cycles by more general ones in [Cho]. (cid:3) Theorem A.7. Conjecture 1.12 holds when ( ξ, x ) is minuscule.Proof. This follows immediately from Proposition A.2, Lemma A.4, and Lemma A.6. (cid:3) Appendix B. Poles of Eisenstein series and theta lifting for unitary groups In this appendix, we prove some results about global theta lifting for unitary groups, namely,Theorem B.4 and its two corollaries. These results are only used in the proof of Proposition 4.13.Thus, if readers are willing to admit these results from the theory of automorphic forms, they arewelcome to skip the entire section except the very short Subsection B.1 where we introduce somenotation for the discrete automorphic spectrum.B.1. Discrete automorphic spectrum. We recall some setup about the discrete automorphic spec-trum. Let G be a reductive group over a number field F . Let Z G be the center of G. For a character χ : Z G ( F ) \ Z G ( A F ) → C × , we denote by L (G( F ) \ G( A F ) , χ ) the space of measurable complex valuedfunctions f on G( F ) \ G( A F ) satisfying f ( gz ) = χ ( z ) f ( g ) for z ∈ Z G ( A F ) such that | f ( g ) χ ( g ) | isintegrable on G( F ) \ G( A F ) /Z G ( A F ) for some (hence every) character χ : G( F ) \ G( A F ) → C × suchthat χ · ( χ | Z G ( A F )) is unitary. The group G( A F ) acts on L (G( F ) \ G( A F ) , χ ) by right translation.Denote by L (G( F ) \ G( A F ) , χ ) the maximal subspace of L (G( F ) \ G( A F ) , χ ) that is a direct sumof irreducible representations of G( A F ). We putL (G) := M χ L (G( F ) \ G( A F ) , χ ) where χ runs through all automorphic characters of Z G ( A F ). Finally, denote by L (G) the subspaceof L (G) consisting of cusp forms. Both L (G) and L (G) are representations of G( A F ) viaright translation. Definition B.1. Let π be an irreducible admissible representation of G( A F ).(1) We define the discrete (resp. cuspidal) multiplicity m disc ( π ) (resp. m cusp ( π )) to be the dimen-sion of Hom G( A F ) ( π, L (G)) (resp. Hom G( A F ) ( π, L (G))).(2) We define a discrete (resp. cuspidal) realization of π to be an irreducible subrepresentation V π contained in L (G) (resp. L (G)) that is isomorphic to π .It is known that 0 (cid:54) m cusp ( π ) (cid:54) m disc ( π ) < ∞ .B.2. Main theorem and consequences. Now we let F be a totally real number field, and E/F atotally imaginary quadratic extension. Denote by c the nontrivial involution of E over F . Definition B.2. We say that an automorphic character µ : E × \ A × E → C × is strictly unitary if µ ∞ takes value 1 on the diagonal ∆ [ F : Q ] R × > ⊆ ( R × > ) [ F : Q ] as a subgroup of E ×∞ ⊆ A ∞ E . Remark B.3 . It is clear that a strictly unitary automorphic character is unitary. For every automorphiccharacter µ of A × E , there exists a unique complex number s such that µ | | sE is strictly unitary.Let V , ( , ) V be a (non-degenerate) hermitian space over E (with respect to c ) of rank n and letW , h , i W be a skew-hermitian space over E of rank m . Let G := U(V) and H := U(W) be theunitary groups of V and W, respectively. We form the symplectic space Res E/F V ⊗ E W, and letMp(Res E/F V ⊗ E W) be the metaplectic cover of Sp(Res E/F V ⊗ E W)( A F ) with center C . Then wehave the oscillator representation ω of Mp(Res E/F V ⊗ E W) using the standard additive character ψ F . Let µ = ( µ V , µ W ) be a pair of splitting characters for (V , W), that is, ( µ V , µ W ) is a pair of automorphiccharacters of A × E satisfying µ V | A × F = µ mE/F and µ W | A × F = µ nE/F . Then it induces an embedding ι µ : G( A F ) × H( A F ) , → Mp(Res E/F V ⊗ E W). By restriction, we obtain the Weil representation ω V , W µ := ω ◦ ι µ (B.1)of G( A F ) × H( A F ). It induces the global theta lifting map Θ W µ, V : for an irreducible subrepresentation V ⊆ L (G) of G( A F ), we obtain a subrepresentation Θ W µ, V ( V ) ⊆ C ∞ (H( F ) \ H( A F ) , C ) of H( A F ).More precisely, there is a space of theta functions θ µ ( g, h ) on G( F ) \ G( A F ) × H( F ) \ H( A F ), which isan automorphic realization of ω V , W µ . Then Θ W µ, V ( V ) is spanned by functions h Z G( F ) \ G( A F ) θ µ ( g, h ) f ( g ) dg on H( F ) \ H( A F ) for f ∈ V . Similarly, we have the reverse global theta lifting map Θ V µ, W .We consider an automorphic representation π of G( A F ) and a strictly unitary automorphic character µ : E × \ A × E → C × . We study three objects associated to π and µ as follows. • Let S be a finite set of places of F containing all archimedean ones and such that for v S ,both π v and µ v are unramified. We then have the partial standard L -function L S ( s, π × µ ). • Let G be the unitary group of the hermitian space V := V ⊕ D, where D is the hyperbolichermitian plane, let Q be a parabolic subgroup of G stabilizing an isotropic line in D, and let K ⊆ G ( A F ) be a maximal compact subgroup such that the Cartan decomposition G ( A F ) =Q( A F ) K holds. Let V π be a cuspidal realization of π (Definition B.1). Let I( V π (cid:2) µ c ) bethe space of functions f on G ( A F ) such that for every k ∈ K the function p f ( pk ) In this article, we will always use ψ F to form oscillator representations. Thus, in the sequel, we will no longermention the dependence of ψ F when discussing oscillator representations. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 59 is a K ∩ Q( A F )-finite vector in V π (cid:2) ( µ c · | | ( n +1) / E ). For every f ∈ I( V π (cid:2) µ c ), we canform an Eisenstein series E Q ( g ; f s ) normalized such that Re( s ) = 0 is the unitary line (see[Sha88, Section 2] for details). By the Langlands theory of Eisenstein series [Lan71, MW95], E Q ( g ; f s ) is absolutely convergent for Re( s ) > n +12 and has a meromorphic continuation to theentire complex plane. • Let V π be a cuspidal realization of π (Definition B.1). Then we have the global theta liftingΘ W( µ,ν ) , V ( V π ). We will adopt the convention that if µ | A × F = µ mE/F with m := dim E W, thenΘ W( µ,ν ) , V ( V π ) = 0.We have the following theorem, which is the unitary version of a weaker form of [GJS09, Theo-rem 1.1]. Theorem B.4. Let π be an irreducible admissible representation of G( A F ) , let V π be a cuspidalrealization of π (Definition B.1), and let µ : E × \ A × E → C × be a strictly unitary automorphic character.We have(1) For s ∈ C with Re( s ) > , consider the following:(a) L S ( s, π × µ ) · L S (2 s, µ, As ( − n ) has a pole at s , where As ± stands for the two Asairepresentations (see, for example, [GGP12a, Section 7] ).(b) { E Q ( g ; f s ) | f ∈ I( V π (cid:2) µ c ) } has a pole at s + j for some integer j (cid:62) .(c) Θ W( µ,ν ) , V ( V π ) = 0 for some skew-hermitian space W of dimension n + 1 − s and some ν with ν | A × F = µ nE/F . Then (a) ⇒ (b) ⇒ (c).(2) The skew-hermitian space W in (1c) is unique up to isomorphism. We will prove the theorem in Subsection B.3. Corollary B.5. Let Pol Sπ,µ be the set of poles of L S ( s, π × µ ) · L S (2 s, µ, As ( − n ) in the region Re( s ) > .(1) If µ is not conjugate self-dual, then Pol Sπ,µ is empty.(2) If µ is conjugate orthogonal, then Pol Sπ,µ is contained in { n +12 , n − , . . . , n +12 − b n c} .(3) If µ is conjugate symplectic, then Pol Sπ,µ is contained in { n , n − , . . . , n − b n − c} .Proof. This is a direct consequence of Theorem B.4. (cid:3) Let V π be a cuspidal realization of π such that { E Q ( g ; f s ) | f ∈ I( V π (cid:2) µ c ) } has the largest pole at s max . By Theorem B.4, there is a skew-hermitian space W of dimension n + 1 − s max , unique up toisomorphism, such that Θ W( µ,ν ) , V ( V π ) is nonzero. Corollary B.6. Let the notation be as above. Suppose that Θ W( µ,ν ) , V ( V π ) is cuspidal. Then(1) The space Θ W( µ,ν ) , V ( V π ) is an irreducible representation of U(W)( A F ) ; and V π = Θ V( µ − ,ν − ) , − W (Θ W( µ,ν ) , V ( V π ))(B.2) where − W , h , i − W denotes the skew-hermitian space W , −h , i W ; and we naturally identify U(W) with U( − W) .(2) The space R s max ( V π (cid:2) µ c ) generated by residues of { E Q ( g ; f s ) | f ∈ I( V π (cid:2) µ c ) } at s = s max isan irreducible representation of G ( A F ) ; and R s max ( V π (cid:2) µ c ) = Θ V ( µ − ,ν − ) , − W (Θ W( µ,ν ) , V ( V π )) . (B.3) (3) In the situation of (1) (resp. (2)), let π W (resp. π ) be the underlying (irreducible) represen-tation of Θ W( µ,ν ) , V ( V π ) (resp. R s max ( V π (cid:2) µ c ) ). If π W has unique realization as a subquotient inthe space of automorphic forms on U(W) , then m cusp ( π ) = 1 (resp. m cusp ( π ) = 0 ). Here, we regard vectors in V π (cid:2) ( µ c · | | ( n +1) / E ) as functions on Q( A F ) via the Levi quotient map. The choice of such ν is insensitive since changing ν results in a twist of Θ W( µ,ν ) , V ( V π ) by a character. Proof. Put m := n + 1 − s max for simplicity.For (1), the irreducibility follows from [Wu13, Theorem 5.3], and (B.2) follows from [Wu13, Theo-rem 5.1].For (2), since the space generated by the constant terms of forms in R s max ( V π (cid:2) µ c ) is an irreduciblerepresentation of M Q ( A F ) where M Q is the Levi quotient of Q, the space R s max ( V π (cid:2) µ c ) is anirreducible representation of G ( A ). Then (B.3) follows from [Wu13, Proposition 5.9].For (3), we first study m cusp ( π ). Let V π be an arbitrary cuspidal realization of π . By TheoremB.4, there exists a skew-hermitian space W of the same dimension m such that Θ W ( µ,ν ) , V ( V π ) = 0.As m (cid:54) n , by the local theta dichotomy [SZ15, Theorem 1.10], we have W ’ W. By the Howeduality [GT16, Theorem 1.2], the underlying representation of Θ W ( µ,ν ) , V ( V π ) must be isomorphic to afinite sum of π W . Then the assumption of π W implies Θ W( µ,ν ) , V ( V π ) = Θ W ( µ,ν ) , V ( V π ). Thus, V π = V π by[Wu13, Theorem 5.1]. In particular, m cusp ( π ) = 1.Then we study m cusp ( π ). Let V π be a cuspidal realization of π . By the identity L S ( s, π ) = L S ( s, π ) · L S ( s − s max , µ c ), we know that L S ( s, π × µ ) has a pole at s max + 1. By Theorem B.4, thereexists a skew-hermitian space W of dimension ( n + 2) + 1 − s max + 1) = m such that Θ W ( µ,ν ) , V ( V π ) =0. Again by the local theta dichotomy and the Howe correspondence, we have W ’ W and thatthe underlying representation of Θ W ( µ,ν ) , V ( V π ) must be isomorphic to a finite sum of π W . Then theassumption of π W implies Θ W ( µ,ν ) , V ( V π ) = Θ W( µ,ν ) , V ( V π ). Thus, V π = Θ V ( µ − ,ν − ) , − W (Θ W( µ,ν ) , V ( V π )) by[Wu13, Theorem 5.1], which is simply R s max ( V π (cid:2) µ ) by (B.3). This is a contradiction. Therefore, m cusp ( π ) = 0. (cid:3) Remark B.7 . In fact, in Corollary B.6, the space Θ W( µ,ν ) , V ( V π ) is always cuspidal, which follows froman analogous statement of [GJS09, Theorem 5.1], whose proof can be adopted to the unitary case aswell. Since we do not need this fact, we will leave the details to interested readers as an exercise.B.3. Proof of Theorem B.4. We follow the strategy in [GJS09]. We first prove the followingproposition, which is a part of Theorem B.4. Proposition B.8. Suppose that µ c = µ − . Let s be the maximal positive real pole of { E Q ( g ; f s ) | f ∈ I( V π (cid:2) µ c ) } . Then(1) there is some skew-hermitian space W of dimension n + 1 − s such that Θ W( µ,ν ) , V ( V π ) = 0 ;(2) all other positive real poles of E Q ( g ; f s ) have the form s − j for some integer j (cid:62) . Part (1) of this proposition is the unitary version of [GJS09, Theorem 3.1]. The proof is very similarto the argument in [Mœg97, Section 2.1] and [GJS09, Section 3], which are for orthogonal groups. Wewill only sketch the proof with necessary modification for the unitary case.We first introduce some notation. Fix a polarization D = δ + ⊕ δ − of the hyperbolic hermitianplane D. For an integer a (cid:62) 0, put δ ± a := ( δ ± ) ⊕ a and V a := V ⊕ ( δ + a ⊕ δ − a ). Put G a := U(V a ) andlet Q a ⊆ G a be the parabolic subgroup stabilizing the subspace δ + a . In particular, we may identifyQ with Q. Note that the Levi quotient of Q a is isomorphic to G × Res E/F GL a . In particular, wehave the space of functions V π (cid:2) ( µ c · | | ( n + a ) / E ) ◦ det a on Q a ( A F ), where det a : GL a → G m is thedeterminant map. Similar to I( V π (cid:2) µ c ), we have the space I a ( V π (cid:2) µ c ) of functions on G a ( A F ); andfor f a ∈ I a ( V π (cid:2) µ c ), one can form the Eisenstein series E Q a ( ; f a,s ) on G a ( A F ), which is absolutelyconvergent for Re( s ) > n + a . In particular, I ( V π (cid:2) µ c ) = I( V π (cid:2) µ c ). Let Pol a ( V π (cid:2) µ c ) be the set ofpositive real poles of E Q a ( ; f a,s ). Then s is the largest number in Pol ( V π (cid:2) µ c ) by our assumption. Lemma B.9. Let s be an element in Pol ( V π (cid:2) µ c ) such that s + j Pol ( V π (cid:2) µ c ) for every integer j > . Then s + a − lies in Pol a ( V π (cid:2) µ c ) .Proof. This is the unitary analogue of [Mœg97, Remarque 1.1] and [GJS09, Proposition 1.1]. Theargument for [Mœg97, Remarque 1.1] works in the unitary case as well. However, we would like At a nonarchimedean place, the local theta dichotomy is also proved in [GG11]. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 61 to remark that in [Mœg97, Remarque 1.1], the author assumes that s is the maximal element ofPol ( V π (cid:2) µ c ). This is unnecessary since the argument only uses the fact that s + j Pol ( V π (cid:2) µ c )for every integer j > (cid:3) Now we recall the generalized doubling method for unitary groups. Again let − V be the hermitianspace with the negative hermitian form on V. Let V (cid:5) be the doubling space V ⊕ ( − V). For an integer a (cid:62) 0, put V (cid:5) a := V (cid:5) ⊕ ( δ + a ⊕ δ − a ) = V a ⊕ ( − V) . Via this decomposition, we have a canonical embedding ι : G a × G → U(V (cid:5) a )where we have identified G with U( − V). Put V ± := { ( v, ± v ) ∈ V (cid:5) | v ∈ V } and V ± a := V ± ⊕ δ ± a . LetP a be the parabolic subgroup of U(V (cid:5) a ) stabilizing the maximal totally isotropic subspace V + a of V (cid:5) a .Then the Levi quotient of P a is isomorphic to Res E/F GL n + a . We have the space of degenerate seriesJ a ( s, µ c ) which is the normalized induced representation Ind U(V (cid:5) a )( A F )P a ( A F ) ( µ c · | | s ) ◦ det n + a . Let f (cid:5) a,s be astandard section in J a ( s, µ c ). Then we can form the Siegel–hermitian Eisenstein series E P a ( ; f (cid:5) a,s ) onU(V (cid:5) a )( A F ), which is absolutely convergent for Re( s ) > n + a . See [Tan99, Section 1] for more details.Now for a standard section f (cid:5) a,s ∈ J a ( s, µ c ) and a cusp form φ ∈ V π , we have the function f (cid:5) ,φa,s ( g ) := Z G( A F ) f (cid:5) a,s ( ι ( g − g , φ ( g )d g (B.4)on G a ( A F ). The following lemma is analogous to [GJS09, Proposition 3.2]. Lemma B.10. Suppose that µ | A × F = µ iE/F for i ∈ { , } . We have(1) The poles of the Siegel–hermitian Eisenstein series E P a ( ; f (cid:5) a,s ) in the region Re( s ) > areall simple, and contained in the set { n + a − i , n + a − i − , . . . } .(2) The integral (B.4) is absolutely convergent for Re( s ) > n + a .(3) The function f (cid:5) ,φa,s has a meromorphic continuation to the entire complex plane, whose possiblepoles in the region Re( s ) > are contained in the set { n − i , n − i − , . . . } .(4) If s is not a pole of f (cid:5) ,φa,s , then f (cid:5) ,φa,s is a section in the normalized induced representation Ind G a ( A F )Q a ( A F ) V π (cid:2) ( µ c · | | sE ) ◦ det a .Proof. Part (1) follows from Main Theorem of [Tan99]. The proof of (2–4) is same as in [Mœg97,Section 2.1]. In particular, the poles of f (cid:5) ,φa,s are contained in the set of poles of the Eisenstein series E P ( g ; f s | G( A F )). Thus, (3) follows from Main Theorem of [Tan99]. (cid:3) The following lemma is analogous to [Mœg97, Proposition 2.1] and [GJS09, Proposition 3.3]. Lemma B.11. For a standard section f (cid:5) a,s ∈ J a ( s, µ c ) and a cusp form φ ∈ V π , we have the identity Z G( F ) \ G( A F ) E P a ( ι ( g , g ); f (cid:5) a,s ) φ ( g ) µ (det g )d g = E Q a ( g ; f (cid:5) ,φa,s ) for g ∈ G a ( A F ) , as meromorphic functions in s away from the poles of f (cid:5) ,φa,s .Proof. The proof is almost same to the argument on [Mœg97, p.214–215]. We will sketch the process.To ease notation, we identify G a × G as a subgroup of U(V (cid:5) a ) via ι . We consider the double cosetP a ( F ) \ U(V (cid:5) a )( F ) / G a ( F ) × G( F ) . (B.5)We identify P a ( F ) \ U(V (cid:5) a )( F ) with the set of maximal isotropic subspaces of V (cid:5) a . Let L be such asubspace. Put d L := dim E ( L ∩ ( − V)). Then L and L are in the same double coset of (B.5) if and onlyif d L = d L . In other words, we have a canonical bijection between (B.5) and { , , . . . , r } where r isthe Witt index of V. Moreover, the identity double coset corresponds to 0. For every d = 0 , , . . . , r , we fix a representative γ d of the corresponding double coset (we take γ to be the identity matrix).Then for g ∈ G a ( A F ), we have Z G( F ) \ G( A F ) E P a ( ι ( g , g ); f (cid:5) a,s ) φ ( g ) µ (det g )d g = r X d =0 Z G( F ) \ G( A F ) X ( γ ,γ ) ∈ γ − d P a ( F ) γ d ∩ (G a × G)( F ) \ (G a × G)( F ) f (cid:5) a,s ( γ d ( γ g , γg )) φ ( g ) µ (det g )d g = X γ ∈ γ − d P a ( F ) γ d G( F ) ∩ G a ( F ) \ G a ( F ) Z G( F ) ∩ γ − d P a ( F ) γ d \ G( A F ) f (cid:5) a,s ( γ d ( γ g , g )) φ ( g ) µ (det g )d g. It is easy to see that, since φ is cuspidal, the integration vanishes unless d = 0. Thus, we have Z G( F ) \ G( A F ) E P a ( ι ( g , g ); f (cid:5) a,s ) φ ( g ) µ (det g )d g = X γ ∈ P a ( F )G( F ) ∩ G a ( F ) \ G a ( F ) Z G( A F ) f (cid:5) a,s ( γ d ( γ g , g )) φ ( g ) µ (det g )d g = X γ ∈ P a ( F )G( F ) ∩ G a ( F ) \ G a ( F ) Z G( A F ) f (cid:5) a,s (( g − γ g , φ ( g )d g = X γ ∈ P a ( F )G( F ) ∩ G a ( F ) \ G a ( F ) Z G( A F ) f (cid:5) ,φa,s ( γ g )= E Q a ( g ; f (cid:5) ,φa,s ) . Here the last equality is due to the fact that P a ( F )G( F ) ∩ G a ( F ) = Q a ( F ). The lemma follows. (cid:3) The following lemma suggests that sections of the form f (cid:5) ,φa,s detect poles of E Q a when a is sufficientlylarge. Lemma B.12. There exists an integer a depending only on V π and µ such that for every integer a (cid:62) a , if s is not a pole of { f (cid:5) ,φa,s } , then the functions { f (cid:5) ,φa,s } for all standard sections f (cid:5) a,s ∈ J a ( s, µ c ) and φ ∈ V π span the whole space Ind G a ( A F )Q a ( A F ) V π (cid:2) ( µ c · | | sE ) ◦ det a .Proof. This follows from the same discussion after [GJS09, Proposition 3.3]. (cid:3) Proof of Proposition B.8. Let s be an element in Pol ( V π (cid:2) µ c ) such that s + j Pol ( V π (cid:2) µ c ) forevery integer j > 0. Put s a := s + a − . Let a be an integer such that s a > n and a (cid:62) a where a isas in Lemma B.12. By Lemma B.10, f (cid:5) ,φa,s is holomorphic at s = s a . By Lemma B.9 and Lemma B.12,we may find some standard section f (cid:5) a,s ∈ J a ( s, µ c ) and φ ∈ V π such that E Q a ( ; f (cid:5) ,φa,s ) has a pole at s = s a . By Lemma B.11, we know that E P a ( ; f (cid:5) a,s ) has a pole at s = s a for such f (cid:5) a,s . Therefore, s has to be the maximal element in Pol ( V π (cid:2) µ c ), that is, s = s . In particular, (2) follows.We continue for (1). By Lemma B.10(1), the pole must be simple, that is, Res s = s a E P a ( ; f (cid:5) a,s ) = 0.Put m := n +1 − s and m a := 2( n + a ) − m . Let W a be a skew-hermitian space over E of rank m a . Wehave a Weil representation of U(V (cid:5) a )( A F ) × U(W a )( A F ) on the Schwartz space S ((V + a ⊗ E W a )( A F )),and a U(V (cid:5) a )( A F )-equivariant map f ( s a ) : S ((V + a ⊗ E W a )( A F )) → Ind U(V (cid:5) a )( A F )P a ( A F ) ( µ c · | | s a ) ◦ det n + a sending Φ to f ( s a )Φ , which is known as a Siegel–Weil section. For more details, see, for example, [Ich04].Since m a (cid:62) n + a , by [KS97, Theorem 1.2 & Theorem 1.3] and [Lee94, Theorem 6.10], the map f ( s a ) : M W a S ((V + a ⊗ E W a )( A F )) → Ind U(V (cid:5) a )( A F )P a ( A F ) ( µ c · | | s a ) ◦ det n + a , OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 63 by considering all possible skew-hermitian spaces W a of rank m a up to isomorphism, is surjective.Thus, there exists some W a in the summand and an element Φ ∈ S ((V + a ⊗ E W a )( A F )) such that f ( s a )Φ = f (cid:5) a,s and hence Res s = s a E P a ( ; f ( s a )Φ ) = 0. In particular, the Witt index of W a is at least m a − ( n + a ). Now by the main theorem on [Ich04, p.243], we have the identityRes s = s a E P a ( ; f ( s a )Φ ) = c · Z U(W)( F ) \ U(W)( A F ) θ ( µ c , ) ( , h )d h as functions on U(V (cid:5) a )( A F ). Here, c is a nonzero constant; W is a certain skew-hermitian space of rank2( n + a ) − m a = m determined by W a ; and θ ( µ c , ) is a certain theta series on U(V (cid:5) a )( A F ) × U(W)( A F )with respect to the pair of splitting characters ( µ c , ) in which denotes the trivial character. ByLemma B.11 and our choices of f (cid:5) a,s and φ , the integral Z G( F ) \ G( A F ) Z U(W)( F ) \ U(W)( A F ) θ ( µ c , ) ( ι ( g , g ) , h ) φ ( g ) µ (det g )d h d g (B.6)is nonzero for some g ∈ G a ( A F ). Now we need to separable the variables g and g in the above thetaseries. Choose an arbitrary automorphic character ν of A × E such that ν | A × F = µ nE/F . We have twoembeddings ι := ι × id U(W) : G a × G × U(W) , → U(V (cid:5) a ) × U(W) ,ι : G a × G × U(W) , → (G a × U(W)) × (G × U(W))in which the second one is induced by the diagonal embedding of U(W). It follows from [HKS96,Lemma 1.1] that ω V (cid:5) a , W( µ c , ) ◦ ι ’ ( ω V a , W( µ c ,ν c ) ⊗ ω V , W( µ c ,ν ) ) ◦ ι for the the restriction of Weil representations (B.1). Therefore, there exist finitely many pairs( θ ( i )( µ c ,ν c ) , θ [ i ]( µ c ,ν ) ) in which θ ( i )( µ c ,ν c ) (resp. θ [ i ]( µ c ,ν ) ) is a theta series on G a ( A F ) × U(W)( A F ) (resp.G( A F ) × U(W)( A F )) with respect to ( µ c , ν c ) (resp. ( µ c , ν )) such that θ ( µ c , ) ( ι ( g , g ) , h ) = X i θ ( i )( µ c ,ν c ) ( g , h ) θ [ i ]( µ c ,ν ) ( g, h ) . Then we have(B.6) = Z G( F ) \ G( A F ) Z U(W)( F ) \ U(W)( A F ) θ µ ( ι ( g , g ) , h ) φ ( g ) µ (det g )d h d g = Z G( F ) \ G( A F ) Z U(W)( F ) \ U(W)( A F ) X i θ ( i )( µ c ,ν c ) ( g , h ) θ [ i ]( µ c ,ν ) ( g, h ) φ ( g ) µ (det g )d h d g = X i Z U(W)( F ) \ U(W)( A F ) θ ( i )( µ c ,ν c ) ( g , h ) Z G( F ) \ G( A F ) θ [ i ]( µ c ,ν ) ( g, h ) φ ( g ) µ (det g )d g ! d h = X i Z U(W)( F ) \ U(W)( A F ) θ ( i )( µ c ,ν c ) ( g , h ) Z G( F ) \ G( A F ) θ [ i ]( µ,ν ) ( g, h ) φ ( g )d g ! d h. In particular, there exists some i such that Z G( F ) \ G( A F ) θ [ i ]( µ,ν ) ( g, h ) φ ( g )d g . In other words, Θ W( µ,ν ) , V ( V π ) = 0, and (1) follows. (cid:3) Proof of Theorem B.4. By the Langlands–Shahidi theory, the poles of Eisenstein series E Q ( ; f s ) arecontrolled by its constant term, which in term are control by the intertwining operator attached to the longest Weyl element in Q \ G / Q. By the Gindikin–Karpelevich formula, we know that the polesof the L -function L S ( s, π × µ ) · L S (2 s, µ, As ( − n ) L S ( s + 1 , π × µ ) · L S (2 s + 1 , µ, As ( − n )(B.7)in the region Re( s ) > ( V π (cid:2) µ c ). See the proof of [GJS09, Proposition 2.2]for a similar discussion in the orthogonal case.We first consider the case where µ c = µ − . Then L S ( s, µ, As ( − n ) has no pole for Re( s ) > ( V π (cid:2) µ c ) is empty. Thus, it follows easily that L S ( s, π (cid:2) µ ) has no pole for Re( s ) > µ c = µ − . In other words, µ | A × F = µ iE/F for a unique i ∈ { , } . Part (2) is aconsequence of the local theta dichotomy [SZ15, Theorem 1.10]. It remains to consider (1). Let s bea pole of L S ( s, π × µ ) · L S (2 s, µ, As ( − n ) as in (a). Let j (cid:62) s + j is a pole of L S ( s, π × µ ) · L S (2 s, µ, As ( − n ). Then the L -function (B.7) has a pole at s + j .Thus, we have s + j ∈ Pol ( V π (cid:2) µ c ), and (b) holds. For the implication (b) ⇒ (c), by the Rallis towerproperty for global theta lifting, we may assume that j = 0 in (b) and s + j Pol ( V π (cid:2) µ c ) for everyinteger j > 0. Then by Proposition B.8(2), s = s . Then (c) follows from Proposition B.8(1). (cid:3) Appendix C. Shimura varieties for hermitian spaces In this appendix, we summarize different versions of unitary Shimura varieties. In Subsection C.1,we recall Shimura varieties associated to isometry groups of hermitian spaces, which are of abeliantype; we also introduce the Shimura varieties associated to incoherent hermitian spaces. In SubsectionC.2, we recall the well-known PEL type Shimura varieties associated to groups of rational similitudeof skew-hermitian spaces, and their integral models at good primes, after Kottwitz. These Shimuravarieties are only for the preparation of the next subsection, which are not logically needed in themain part of the article. In Subsection C.3, we summarize the connection of these two kinds of unitaryShimura varieties via the third one which possesses a moduli interpretation but is not of PEL typein the sense of Kottwitz, after [BHKRY, RSZ20]. In Subsection C.4, we discuss integral models of thethird unitary Shimura varieties at good inert primes and uniformization along the basic locus.Let F be a totally real number field of degree d (cid:62) 1, and E/F a totally imaginary quadraticextension. Denote by c the nontrivial involution of E over F . Denote by Φ F the set of real embeddingsof F and by Φ E the set of complex embeddings of E . Let N [Φ E ] be the commutative monoid freelygenerated by Φ E . The Galois group Gal( C / Q ) acts on Φ E hence on N [Φ E ]. We have the projectionmap π : Φ E → Φ F given by restriction. Recall that a CM type (of E ) is a subset Φ of Φ E such that π induces a bijection from Φ to Φ F . For a CM type Φ, put Φ c := Φ E \ Φ which is again a CM type.C.1. Case of isometry. Let V be a (non-degenerate) hermitian space over E (with respect to c ) ofrank n (cid:62) 1, with the hermitian form ( , ) V : V × V → E that is E -linear in the first variable. Forevery τ ∈ Φ F , let ( p τ , q τ ) be the signature of V ⊗ F,τ R . We take a CM type Φ ⊆ Φ E . Then we havetwo elements sig V , Φ := X τ ∈ Φ F p τ τ + + X τ ∈ Φ F q τ τ − , sig [ V , Φ := X τ ∈ Φ F q τ τ − (C.1)in N [Φ E ]. Here, τ − (resp. τ + ) is the unique element in Φ (resp. Φ c ) whose image under π is τ . Definition C.1. We define the reflex field (resp. reduced reflex field ) of the pair (V , Φ) to be the fixedfield of the stabilizer in Gal( C / Q ) of the element sig V , Φ (resp. sig [ V , Φ ), denoted by E V , Φ (resp. E [ V , Φ ).Let U(V) be the unitary group (of isometry) of V, that is, the reductive group over F such that forevery F -algebra R , we haveU(V)( R ) = { g ∈ Aut R (V ⊗ F R ) | ( gx, gy ) V = ( x, y ) V for all x, y ∈ V ⊗ F R } . OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 65 For every τ ∈ Φ F , we may identify V ⊗ E,τ − C with C ⊕ n ; and hence U(V) ⊗ F,τ R is identified with thesubgroup of Res C / R GL n of elements preserving the hermitian form given by the matrix (cid:16) I pτ − I qτ (cid:17) .Put G := Res F/ Q U(V). We define the Hodge maph [ V , Φ : Res C / R G m → G R to be the one sending z ∈ C × = (Res C / R G m )( R ) to I p τ ( z/z )I q τ ! , · · · , I p τd ( z/z )I q τd !! ∈ G R ( R ) , where we identify G R ( R ) as a subgroup of GL n ( C ) d via { τ − , . . . , τ − d } . Then we obtain a Shimuradata (G , h [ V , Φ ). It is of abelian type but not Hodge type; and its reflex field coincides with E [ V , Φ . Thetheory of Shimura varieties provides us with a projective system of schemes { Sh(G , h [ V , Φ ) K } K , quasi-projective and smooth over E [ V , Φ of dimension P τ ∈ Φ F p τ q τ , indexed by neat open compact subgroups K of G( A ∞ ) = U(V)( A ∞ F ). Remark C.2 . Suppose that there is an element τ ∈ Φ F such that V has signature ( n − , 1) at τ and( n, 0) at other places. The Hodge map h [ V , Φ and hence the Shimura variety Sh(G , h [ V , Φ ) K dependonly on Φ ∩ π − τ , that is, the unique element contained in Φ above τ . Thus, for an element τ ∈ Φ E above τ , we may write h V ,τ and Sh(G , h V ,τ ) K for those Φ containing τ . In particular, the reflexfield of h V ,τ is τ ( E ). The Galois group Gal( C /τ ( E )) acts on the set of connected components ofSh(G , h [ V , Φ ) K ⊗ ι ( E ) C via the homomorphismGal( C /τ ( E )) rec −−→ τ ( E ) × \ ( A ∞ τ ( E ) ) × ( τ ) − −−−−→ E × \ ( A ∞ E ) × e e/e c −−−−→ E \ ( A ∞ E ) , where rec is the global reciprocity map for the number field τ ( E ).Now we would like to attach Shimura varieties to an incoherent hermitian space, a concept originatedfrom [KR94] in the orthogonal case. This observation generalizes the case of Shimura curves in [YZZ13],and has already appeared in some old work [Liu11a, Liu11b], with more details explained by Gross[Gro] recently. Definition C.3. An incoherent hermitian space over A E is a free A E -module V of some rank n (cid:62) , ) V : V × V → A E with respect to the (induced)involution c on A E such that its determinant belongs to A × F \ F × N A E / A F A × E . We say that V istotally positive definite if for every τ ∈ Φ F , V ⊗ A F ,τ R is positive definite.Let V be a totally positive definite incoherent hermitian space over A E of rank n (cid:62) 1, and let G := U( V ) be its group of isometry, which is a reductive group over A F . Definition C.4. For τ ∈ Φ F , we say that a hermitian space V over E is τ -nearby to V if V ⊗ F A τF ’ V ⊗ A F A τF , and V ⊗ F,τ R has signature ( n − , τ ∈ Φ F , there exists a hermitian space that is τ -nearby to V , unique upto isomorphism. We fix such a space V( τ ). Put G( τ ) := Res F/ Q U(V( τ )). We fix an isomorphism V ⊗ A F A ∞ F ’ V( τ ) ⊗ F A ∞ F hence an isomorphism G ( A ∞ F ) ’ G( τ )( A ∞ ). Proposition C.5. There is a projective system of schemes { Sh( V ) K } K over E indexed by sufficientlysmall open compact subgroups K of G ( A ∞ F ) , such that for every τ ∈ Φ F and every τ ∈ Φ E above it,we have { Sh( V ) K ⊗ E,τ τ ( E ) } K ’ { Sh(G( τ ) , h V( τ ) ,τ ) K } K as projective systems of schemes over τ ( E ) . Here, we use the fixed isomorphism G ( A ∞ F ) ’ G( τ )( A ∞ ) to regard K as a subgroup of G( τ )( A ∞ ) .Proof. See [Gro, Section 10]. (cid:3) Definition C.6. We call the projective system of schemes { Sh( V ) K } K over E in Proposition C.5 the Shimura variety associated to V . Remark C.7 . One can also interpret Proposition C.5 in the following way: The scheme Y τ ∈ Φ F Y τ ∈ π − τ Sh(G( τ ) , h V( τ ) ,τ ) K over Y τ ∈ Φ F Y τ ∈ π − τ Spec τ ( E ) = Y τ ∈ Φ E Spec τ ( E )descends to a scheme Sh( V ) K over Spec E , where the above fiber products are taken over Spec Q .The scheme Sh( V ) K (for K sufficiently small) is quasi-projective and smooth over E of dimension n − 1. It is projective if d > n = 1. In all cases, we denote by Sh( V ) K the Baily–Borelcompactification of Sh( V ) K over E . Then Sh( V ) K \ Sh( V ) K is either empty or consists of isolatedsingular points. Let f Sh( V ) K be the blow-up of Sh( V ) K along Sh( V ) K \ Sh( V ) K . If Sh( V ) K is proper,then f Sh( V ) K = Sh( V ) K . Otherwise, we must have d = 1, that is, F = Q . In this case, there is onlyone choice for τ ∈ Φ F , for which we will suppress from various notation like V( τ ), G( τ ), etc. However,there are still two choices of Φ, say, { τ + } and { τ − } . We have isomorphisms f Sh( V ) K ⊗ E,τ ± τ ± ( E ) ’ f Sh(G , h V ,τ ± ) K (C.2)extending those in Proposition C.5. Here, f Sh(G , h V ,τ ± ) K is the unique toroidal compactification ofSh(G , h V ,τ ± ) K over E [AMRT10, Pin90]. Definition C.8. We call the projective system of schemes { f Sh( V ) K } K over E the compactifiedShimura variety associated to V (even when Sh( V ) K is already proper). Remark C.9 . The boundary f Sh( V ) K \ Sh( V ) K is a smooth divisor.C.2. Case of similitude. In this subsection, we recall the notion of Shimura varieties attached to thegroup of similitude of a hermitian space, which are of PEL type hence more familiar to most readers.They will not be used in the main part of the article, but it is instructional to invoke them for thelater discussion.Let Ψ = X τ ∈ Φ F p τ τ + + X τ ∈ Φ F q τ τ − be an element of N [Φ E ] such that p τ + q τ = n for every τ ∈ Φ F . Let E Ψ be the fixed field of thestabilizer of Ψ in Gal( C / Q ). Definition C.10. Let S be an E Ψ -scheme.(1) An ( E, Ψ) -abelian scheme over S is a pair ( A, i ) where A is an abelian scheme over S ; and i : E → End S ( A ) Q is a homomorphism of Q -algebras such that for every e ∈ E , the character-istic polynomial of i ( e ) on the locally free sheaf Lie S ( A ) on S is equal to Y τ ∈ Φ F ( T − τ + ( e )) p τ ( T − τ − ( e )) q τ ∈ O S [ T ] . (2) A polarization of an ( E, Ψ)-abelian scheme ( A, i ) is a polarization λ : A → A ∨ satisfying λ ◦ i ( e ) = i ( e c ) ∨ ◦ λ for every e ∈ E . Definition C.11. For a ring R containing Q , a rational skew-hermitian space over E ⊗ Q R of rank n is a free E ⊗ Q R -module W of rank n together with a R -bilinear skew-symmetric non-degeneratepairing h , i W : W × W → R OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 67 satisfying h ex, y i W = h x, e c y i W for every e ∈ E and x, y ∈ W. We say that two rational skew-hermitianspaces W and W over E ⊗ Q R is similar if there exists an isomorphism f : W → W of E ⊗ Q R -modulessuch that there exists some ν ( f ) ∈ R × satisfying h f ( x ) , f ( y ) i W = ν ( f ) h x, y i W for every x, y ∈ W.We take a rational skew-hermitian space W ∞ over A ∞ E = E ⊗ Q A ∞ of rank n . Let H ∞ be thegroup of similitude of W ∞ , which is a reductive group over A ∞ . We denote by W ( W ∞ , Ψ) the set ofsimilarity classes of rational skew-hermitian spaces W over E of rank n such that • W ⊗ E A ∞ E is similar to W ∞ as a rational skew-hermitian space over A ∞ E = E ⊗ Q A ∞ (and wefix a similarity isomorphism); • the signature of the hermitian form h , i · i W on the C -vector space W ⊗ E,τ − C is ( p τ , q τ ).It is a finite set; and its cardinality is at most one if n is even.For every W ∈ W ( W ∞ , Ψ), let H be its group of similitude, that is, the reductive group over Q such that for every ring R containing Q , we haveH( R ) = { h ∈ Aut E ⊗ Q R (W ⊗ Q R ) | h hx, hy i W = ν ( h ) h x, y i W for some ν ( h ) ∈ R × } . We define the Hodge map h W , Ψ : Res C / R G m → H R to be the one sending z ∈ C × = (Res C / R G m )( R ) to z I p τ z I q τ ! , · · · , z I p τd z I q τd ! ; zz ! ∈ H R ( R ) , where we identify H R ( R ) as a subgroup of GL n ( C ) d × C × via { τ − , . . . , τ − d } . Then we have a Shimuradata (H , h W , Ψ ) with the reflex field E Ψ . We obtain a projective system of schemes { Sh(H , h W , Ψ ) L } L ,quasi-projective and smooth over E Ψ of dimension P τ ∈ Φ F p τ q τ , indexed by neat open compact sub-groups L of H ∞ ( A ∞ ) ’ H( A ∞ ).The Shimura data (H , h W , Ψ ) is of PEL type. In particular, it has a moduli interpretation which weroughly recall in the following definition. Definition C.12 ([Kot92]) . For an open compact subgroup L ⊆ H ∞ ( A ∞ ), we define a presheafM( W ∞ , Ψ) L on Sch /E Ψ as follows: for every object S ∈ Sch /E Ψ , we let M( W ∞ , Ψ) L ( S ) be the set ofequivalence classes of quadruples ( A, i, λ, η ) where • ( A, i ) is an ( E, Ψ)-abelian scheme over S (Definition C.10); • λ is a polarization of ( A, i ) (Definition C.10); • η is an L -level structure (see [Kot92, Section 5] for more details).Two quadruples ( A, i, λ, η ) and ( A , i , λ , η )) are equivalent if there is an isogeny ϕ : A → A taking i, λ, η to i , cλ , η for some c ∈ Q × .From [Kot92], it is known that M( W ∞ , Ψ) L is a scheme if L is sufficiently small, and we have acanonical isomorphism M( W ∞ , Ψ) L ’ a W ∈W ( W ∞ , Ψ) Sh(H , h W , Ψ ) L functorial in L . Remark C.13 . Let p be a rational prime unramified in E such that we may write L = L p × L p in which L p is the stabilizer of a self-dual lattice in W ∞ ⊗ A ∞ Q p . Then the presheaf M( W ∞ , Ψ) L admits anextension M ( W ∞ , Ψ) L to a presheaf on Sch /O E Ψ , ( p ) as follows: for every object S ∈ Sch /O E Ψ , ( p ) , welet M ( W ∞ , Ψ) L ( S ) be the set of equivalence classes of quadruples ( A, i, λ, η p ) where • ( A, i ) is an ( E, Ψ)-abelian scheme over S in the sense similar to Definition C.10 but with i : O E, ( p ) → End S ( A ) ⊗ Z Z ( p ) being a homomorphism of Z ( p ) -algebras; • λ is a p -principal polarization of ( A, i ); • η p is an L p -level structure. The equivalence relation is defined in a similar way as in Definition C.12 except that we require theisogenies to be coprime to p and c ∈ Z × ( p ) . The functor M ( W ∞ , Ψ) L is a smooth separated scheme inSch /O E Ψ , ( p ) if L is sufficiently small; and is functorial in L .C.3. Their connection. In this subsection, we study the connection between Shimura varieties inthe case of isometry and those in the case of similitude. Consider • a hermitian space V , ( , ) V over E of rank n , • a rational skew-hermitian space W ∞ , h , i over A ∞ E = E ⊗ Q A ∞ of rank 1 with the group ofsimilitude H ∞ , • a CM type Φ of E such that W ( W ∞ , Φ c ) is nonempty.We now equip W ∞ := V ⊗ E W ∞ with a rational skew-hermitian form over A ∞ E = E ⊗ Q A ∞ . For x, y ∈ W ∞ , let h x, y i † ∈ A ∞ E be the unique element such that Tr E/ Q ( e · h x, y i † ) = h ex, y i for every e ∈ A ∞ E . Thus, we obtain a non-degenerate pairing h , i † : W ∞ × W ∞ → A ∞ E that is A ∞ E -linear in thefirst variable. Now we equip the pairing Tr E/ Q ( , ) V ⊗ E h , i † to W ∞ , which becomes a rational skew-hermitian space over E ⊗ Q A ∞ . By a similar construction, we obtain a map W ( W ∞ , Φ c ) → W ( W ∞ , Ψ)sending W to W, where Ψ = sig V , Φ (C.1). Take an element W ∈ W ( W ∞ , Φ c ) with H its groupof similitude. We obtain three Shimura data: (G , h [ V , Φ ), (H , h W , Φ c ) and (H , h W , Ψ ) with reflex fields E [ V , Φ , E Φ and E Ψ = E V , Φ , respectively. Lemma C.14. Let E ] V , Φ be the subfield of C generated by E [ V , Φ and E Φ . Then E ] V , Φ contains E V , Φ .Proof. By definition, the subgroup of Gal( C / Q ) fixing E ] V , Φ stabilizes both sig [ V , Φ and Φ. Thus, itstabilizes sig V , Φ . The lemma follows. (cid:3) Remark C.15 . In the main part of the article, the hermitian space V we encounter will have signature( n − , 1) at one place τ ∈ Φ F and ( n, 0) elsewhere for some n (cid:62) 2. Then for whatever Φ, we have E [ V , Φ = τ ( E ) where τ ∈ Φ E is either place above τ . However, it is possible that T Φ E ] V , Φ strictlycontains τ ( E ), where Φ runs over all CM types of E .Now we consider the reductive group G ] := G × H over Q . Put h ] Φ := (h [ V , Φ , h W , Φ c ). Thenwe have a product Shimura data (G ] , h ] Φ ), whose reflex field is E ] V , Φ . On the other hand, thereis a homomorphism q W : G ] = G × H → H induced by taking tensor product. It is clear that q W ◦ h ] Φ = h W , Ψ . To summarize, we have the following diagram of Shimura data(G , h [ V , Φ ) (G ] , h ] Φ ) q V (cid:102) (cid:102) q W0 (cid:120) (cid:120) q W (cid:47) (cid:47) (H , h W , Ψ ) . (H , h W , Φ c )(C.3) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 69 For neat open compact subgroups K ⊆ G( A ∞ ), L ⊆ H ( A ∞ ), and L ⊆ H( A ∞ ) satisfying q W ( K × L ) ⊆ L , we have the following diagram of Shimura varieties induced from (C.3)Sh(G , h [ V , Φ ) K ⊗ E [ V , Φ E ] V , Φ Sh(G ] , h ] Φ ) K × L q V (cid:105) (cid:105) q W0 (cid:117) (cid:117) q W (cid:47) (cid:47) Sh(H , h W , Ψ ) L ⊗ E V , Φ E ] V , Φ Sh(H , h W , Φ c ) L ⊗ E Φ E ] V , Φ (C.4)in view of Lemma C.14, in which ( q V , q W ) induces an isomorphismSh(G ] , h ] Φ ) K × L ’ (cid:18) Sh(G , h [ V , Φ ) K ⊗ E [ V , Φ E ] V , Φ (cid:19) × E ] V , Φ (cid:16) Sh(H , h W , Φ c ) L ⊗ E Φ E ] V , Φ (cid:17) . (C.5)The Shimura variety Sh(G ] , h ] Φ ) K × L has a moduli interpretation as well. Definition C.16. For open compact subgroups K ⊆ G( A ∞ ) and L ⊆ H ∞ ( A ∞ ), we define a presheafM(V , W ∞ , Φ) K,L on Sch /E ] V , Φ as follows: for every object S ∈ Sch /E ] V , Φ , we let M(V , W ∞ , Φ) K,L ( S )be the set of equivalence classes of octuples ( A , i , λ , η ; A, i, λ, η ) where • ( A , i ) is an ( E, Φ c )-abelian scheme over S ; • λ is a polarization of ( A , i ); • η is an L -level structure for ( A , i , λ ); • ( A, i ) is an ( E, sig V , Φ )-abelian scheme over S ; • λ is a polarization of ( A, i ); • for chosen geometric point s on every connected component of S , η is a π ( S, s )-invariant K -orbit of isometriesV ⊗ Q A ∞ ∼ −→ Hom E ⊗ Q A ∞ (H ´et1 ( A s , A ∞ ) , H ´et1 ( A s , A ∞ ))of hermitian spaces over A ∞ E . Here, the hermitian pairing on the latter space is given by theformula( x, y ) i − (cid:16) ( λ ∗ ) − ◦ y ∨ ◦ λ ∗ ◦ x (cid:17) ∈ i − End E ⊗ Q A ∞ (H ´et1 ( A s , A ∞ )) = A ∞ E . Two octuples ( A , i , λ , η ; A, i, λ, η ) and ( A , i , λ , η ; A , i , λ , η ) are equivalent if there are isogenies ϕ : A → A and ϕ : A → A such that • there exists c ∈ Q × such that ϕ ∨ ◦ λ ◦ ϕ = cλ and ϕ ∨ ◦ λ ◦ ϕ = cλ ; • for every e ∈ E , we have ϕ ◦ i ( e ) = i ( e ) ◦ ϕ and ϕ ◦ i ( e ) = i ( e ) ◦ ϕ ; • the K -orbit of maps x ϕ ∗ ◦ η ( x ) ◦ ( ϕ ∗ ) − for x ∈ V ⊗ Q A ∞ coincides with η . Remark C.17 . The Shimura variety Sh(G ] , h ] Φ ) K × L and its moduli interpretation were first introducedin [BHKRY] when F = Q , and in [RSZ20] for more general CM extension E/F . Lemma C.18. Let the notation be as above. We have a canonical isomorphism M(V , W ∞ , Φ) K,L ’ (cid:18) Sh(G , h [ V , Φ ) K ⊗ E [ V , Φ E ] V , Φ (cid:19) × E ] V , Φ (cid:16) M( W ∞ , Φ c ) L ⊗ E Φ E ] V , Φ (cid:17) in Sch /E ] V , Φ , functorial in K , L , and Hecke translations.Proof. We have canonical morphisms q : M(V , W ∞ , Φ) K,L → M( W ∞ , Ψ) L ⊗ E V , Φ E ] V , Φ q : M(V , W ∞ , Φ) K,L → M( W ∞ , Φ c ) L ⊗ E Φ E ] V , Φ0 YIFENG LIU of functors obtained from moduli interpretation. Since ( q , q ) induces a closed embedding, the functorM(V , W ∞ , Φ) K,L is representable. Moreover, we have a canonical isomorphismM(V , W ∞ , Φ) K,L ’ a W ∈W ( W ∞ , Φ c ) Sh(G ] , h ] Φ ) K × L (C.6)functorial in K , L , and Hecke translations. The morphisms q and q are compatible with q W and q W in (C.4), respectively. Combining with (C.5), we have(C.6) ’ (cid:18) Sh(G , h [ V , Φ ) K ⊗ E [ V , Φ E ] V , Φ (cid:19) × E ] V , Φ a W ∈W ( W ∞ , Φ) Sh(H , h W , Φ c ) L ⊗ E Φ E ] V , Φ ’ (cid:18) Sh(G , h [ V , Φ ) K ⊗ E [ V , Φ E ] V , Φ (cid:19) × E ] V , Φ (cid:16) M( W ∞ , Φ c ) L ⊗ E Φ E ] V , Φ (cid:17) . The lemma follows. (cid:3) C.4. Integral models and uniformization. In this subsection, we study integral models and uni-formization of the Shimura varieties introduced previously, which are only used in Subsection 5.2 andSubsection 5.3 for the main part of the article. We identify E as a subfield of C via an element τ ∈ Φ E .We fix a hermitian space V over E that has signature ( n − , 1) at τ := τ | F and ( n, 0) at other places.We first review the integral models of M(V , W ∞ , Φ) K,L in Definition C.16 at good primes, wherewe assume τ ∈ Φ. Let p be a prime of F such that • p is inert E ; • the underlying rational prime p is odd and unramified in E ; • we may choose a self-dual lattice Λ q in V ⊗ F F q for every q ∈ p , where p denotes the set of allprimes of F above p that are inert in E ; • L = L p × ( L ) p in which ( L ) p is the stabilizer of a self-dual lattice in W ∞ ⊗ A ∞ Q p , and L p is sufficiently small.We fix an isomorphism between E -extensions C and E ac p . We also assume that elements in Φ inducingthe same place in Spl p induce the same place of E .Denote by E ] V , Φ , p the completion of E ] V , Φ in E ac p . We now consider subgroups K of the form K = K p × K p p × K p where K p = Q q ∈ p K q in which K q is the stabilizer of Λ q , and K p is sufficientlysmall. We denote by Spl p the set of primes of F above p that are split in E . For q ∈ Spl p , we denoteby q − the unique prime of E that is in Φ (under the isomorphism between C and E ac p ) and regard K p p a subgroup of GL( Q q ∈ Spl p V ⊗ E E q − ).The following definition is a special case of the discussion in [RSZ20, Section 4.1] (but with slightlyfiner level structure at Spl p ). Definition C.19. We define a presheaf M (V , W ∞ , Φ) K,L on Sch /O E] V , Φ , p as follows: for every ob-ject S ∈ Sch /O E] V , Φ , p , we let M (V , W ∞ , Φ) K,L ( S ) be the set of equivalence classes of nonuples( A , i , λ , η p ; A, i, λ, η p , η spl p ) where • ( A , i ) is an ( E, Φ c )-abelian scheme over S (in the sense of Remark C.13); • λ is a p -principal polarization of ( A , i ); • η p is an L p -level structure for ( A , i , λ ); • ( A, i ) is an ( E, sig V , Φ )-abelian scheme over S (in the sense of Remark C.13); • λ is a p -principal polarization of ( A, i ); • for chosen geometric point s on every connected component of S , – η p is a π ( S, s )-invariant K p -orbit of isometriesV ⊗ Q A ∞ ,p ∼ −→ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A s , A ∞ ,p ) , H ´et1 ( A s , A ∞ ,p )) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 71 of hermitian spaces over E ⊗ Q A ∞ ,p . Here, the hermitian pairing is defined similarly asin Definition C.16; – η spl p is a π ( S, s )-invariant K p p -orbit of isomorphisms Y q ∈ Spl p V ⊗ E E q − ∼ −→ Y q ∈ Spl p Hom O E q − (cid:0) A s [( q − ) ∞ ] , A s [( q − ) ∞ ] (cid:1) ⊗ O E q − E q − of Q q ∈ p E q − -modules. Note that due to the signature condition in Definition C.12, both A s [( q − ) ∞ ] and A s [( q − ) ∞ ] are étale O E q − -modules.The equivalence relation is defined in a similar way as in Definition C.16 except that we require theisogeny ϕ (resp. ϕ ) to be coprime to p (resp. p ), and c ∈ Z × ( p ) .The presheaf M (V , W ∞ , Φ) K,L is a separated scheme in Sch /O E] V , Φ , p , which is proper if and onlyif V is anisotropic. By Definition C.19 and Remark C.13, we have a canonical morphism q : M (V , W ∞ , Φ) K,L → M ( W ∞ , Φ c ) L ⊗ O E Φ , ( p ) O E ] V , Φ , p (C.7)extending the projection to the second factor in Lemma C.18. Proposition C.20. Let V be as in the beginning of this subsection. Let p be a prime of F inert in E such that its underlying rational prime is unramified in E . Denote by p the set of all primes of F with the same residue characteristic of p that are inert in E . We fix a subgroup K q ⊆ U(V)( F q ) thatis the stabilizer of a self-dual lattice in V ⊗ F F q for every q ∈ p , and put K p := Q q ∈ p K q . Then theShimura variety Sh(G , h V ,τ ) K p := lim ←− K p Sh(G , h V ,τ ) K p K p (see Remark C.2 for the notation) over E has a (smooth) integral canonical model over O E p in thesense of [Mil92, Definition 2.9] .Proof. Let p be the underlying rational prime of p . Choose auxiliary data Φ, W ∞ and L as in theprevious discussion, such that L = L p × ( L ) p in which ( L ) p is the stabilizer of a self-dual lattice in W ∞ ⊗ A ∞ Q p and L p is sufficiently small. Write K for K p × K p . It suffices to consider those K p thatare of the form K p × K p p with K p sufficiently small.Put M := M ( W ∞ , Φ c ) L ⊗ O E Φ , ( p ) O E ] V , Φ , p as in (C.7), which is a finite étale scheme over O E p . Put M := M ⊗ Z p Q p . Then we have canonical isomorphisms M (V , W ∞ , Φ) K,L × M M ’ M (V , W ∞ , Φ) K,L ⊗ O E] V , Φ , p E ] V , Φ , p ’ Sh(G , h V ,τ ) K × Spec E (cid:16) M( W ∞ , Φ c ) L ⊗ E Φ E ] V , Φ , p (cid:17) ’ Sh(G , h V ,τ ) K × Spec E M by Lemma C.18. Take a connected component M of M , which is isomorphic to Spec O E for someunramified finite extension E /E p , with the generic fiber M := M ⊗ Z p Q p . Put M (V , W ∞ , Φ) K,L := q − M . Then we have a canonical isomorphism M (V , W ∞ , Φ) K,L × M M ’ Sh(G , h V ,τ ) K × Spec E M . Thus, it suffices to show that lim ←− K p M (V , W ∞ , Φ) K p K p ,L is an integral canonical model over M .We now modify the proof of [Mil92, Theorem 2.10].Let Y be an integral regular scheme over M such that U := Y × M M is dense in Y , witha given morphism α : U → lim ←− K p M (V , W ∞ , Φ) K p K p ,L × M M . This is equivalent to giving data( A , i , λ , η p ; A, i, λ, η p , η spl p ) as in Definition C.19, but with η p : V ⊗ Q A ∞ ,p ∼ −→ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A η , A ∞ ,p ) , H ´et1 ( A η , A ∞ ,p )) being a π ( U, η )-invariant isometry, and η spl p : Y q ∈ p V ⊗ E E q − ∼ −→ Y q ∈ p Hom O E q − (cid:0) A η [( q − ) ∞ ] , A η [( q − ) ∞ ] (cid:1) ⊗ O E q − E q − being a π ( U, η )-invariant isomorphism, where η is a geometric generic point of Y ; and with the partialdata ( A , i , λ , η p ) extending uniquely to Y . In particular, the action of π ( U, η ) on H ´et1 ( A η , A ∞ ,p )factors through π ( Y, η ), and that on Hom E ⊗ Q A ∞ ,p (H ´et1 ( A η , A ∞ ,p ) , H ´et1 ( A η , A ∞ ,p )) is trivial. Thus,the action of π ( U, η ) on H ´et1 ( A η , A ∞ ,p ) factors through π ( Y, η ). By [Mil92, Propositions 2.11, 2.13,2.14], the triple ( A, i A , θ A ) extends uniquely to Y . Then it is clear that ( η p , η spl p ) extends uniquely aswell.We then conclude that lim ←− K p M (V , W ∞ , Φ) K p K p ,L is an integral canonical model over M . Theproposition follows. (cid:3) Definition C.21. We denote by S (G , h V ,τ ) K p the integral canonical model of Sh(G , h V ,τ ) K p over O E p in Proposition C.20, on which the action of U(V)( A ∞ F , p ) extends uniquely by the extension property.For an open compact subgroup K ⊆ G( A ∞ ) = U(V)( A ∞ F ) of the form K = K p × K p , we put S (G , h V ,τ ) K := S (G , h V ,τ ) K p /K p which we refer as the canonical integral model of Sh(G , h V ,τ ) K over O E p . It is proper/smooth ifSh(G , h V ,τ ) K is. Remark C.22 . The extension property of integral canonical models together with Lemma C.18 impliesthat we have a canonical isomorphism M (V , W ∞ , Φ) K,L ’ S (G , h V ,τ ) K × O E p (cid:18) M ( W ∞ , Φ c ) L ⊗ O E Φ , ( p ) O E ] V , Φ , p (cid:19) under which q (C.7) corresponds to the projection to the second factor. Remark C.23 . Proposition C.20 is slightly stronger than the main result in [Kis10], as the latter hasto assume that K p is hyperspecial maximal.At last, we review the uniformization of M (V , W ∞ , Φ) K,L along the basic locus, which is only usedin Subsection 5.3. Let E nr p be the maximal unramified extension of E p inside E ac p . Let k := O E nr p ⊗ Z F p be the residue field of E nr p . Put M (V , W ∞ , Φ) nr K,L := M (V , W ∞ , Φ) K,L ⊗ O E] V , Φ , p O E nr p , and M (V , W ∞ , Φ) nr K p ,L := lim ←− K p lim ←− K p p M (V , W ∞ , Φ) nr K p K p p K p ,L . Definition C.24. For an algebraically closed field k containing k , we say that a k -point( A , i , λ , η p ; A, i, λ, η p , η spl p ) ∈ M (V , W ∞ , Φ) nr K p ,L ( k )is supersingular if A [ p ∞ ], regarded as a strict O F p -module by [Mih, Theorem 3.3], is supersingular(that is, isoclinic).Denote by M (V , W ∞ , Φ) ss K p ,L the supersingular locus of M (V , W ∞ , Φ) nr K p ,L ⊗ O E nr p k , which isa Zariski closed subset. Denote by M (V , W ∞ , Φ) ss , ∧ K p ,L the completion of M (V , W ∞ , Φ) nr K,L along M (V , W ∞ , Φ) ss K p ,L , which is a formal scheme over O ∧ E nr p , where O ∧ E nr p is the completion of O E nr p . Thedescription of the uniformization of M (V , W ∞ , Φ) ss , ∧ K p ,L depends on the choice of a point P = ( A , i , λ , η p ; A , i , λ , η p , η spl p ) ∈ M (V , W ∞ , Φ) nr K p ,L ( O E nr p )(C.8) OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 73 such that P k is supersingular. In particular, we have the induced section P ∧ : Spf O ∧ E nr p → M (V , W ∞ , Φ) ss , ∧ K p ,L . (C.9)We denote the base change of ( A , i , λ ; A , i , λ ) to k by ( A k , i k , λ k ; A k , i k , λ k ). Moreover, we useSpec k as the reference point in the level structures ( η p ; η p , η spl p ). We now attach to P two objects: aformal scheme N over O ∧ E nr p , and a new hermitian space ¯V over E . • Put X := A [ p ∞ ] equipped with an induced homomorphism i : O E p → End O E nr p ( X ) and aninduced principal polarization λ . Similarly, we have ( X , i , λ ) induced from ( A , i , λ ).Let N be the relative Rapoport–Zink space parameterizing quasi-isogenies of the supersingularunitary O F p -module ( X k , i k , λ k ) of signature ( n − , 1) as introduced in Subsection 1.3, whichis a formal scheme over O ∧ E nr p . In particular, the point P induces a section P ∧ loc : Spf O ∧ E nr p → N . (C.10) • Now we define the new hermitian space. Put¯V := Hom k (( A k , i k ) , ( A k , i k )) Q which is an E -vector space through i k . We define a map( , ) ¯V : ¯V × ¯V → E given by the formula( x, y ) ¯V = i − k (cid:0) λ ∨ k ◦ y ∨ ◦ λ k ◦ x (cid:1) ∈ i − k End k (( A k , i k )) = E, (C.11) which is a hermitian form on ¯V. Lemma C.25. The hermitian space ¯V , ( , ) ¯V has the following properties:(1) ¯V is of dimension n over E ;(2) ¯V is totally positive definite;(3) the composite map ¯V ⊗ Q A ∞ ,p → Hom E ⊗ Q A ∞ ,p (H ´et1 ( A k , A ∞ ,p ) , H ´et1 ( A k , A ∞ ,p )) ( η p ) − −−−−→ V ⊗ Q A ∞ ,p is an isomorphism of hermitian spaces over F ⊗ Q A ∞ ,p ;(4) the composite map Y q ∈ Spl p ¯V ⊗ E E q − → Y q ∈ Spl p Hom O E q − (cid:0) A k [( q − ) ∞ ] , A k [( q − ) ∞ ] (cid:1) ⊗ O E q − E q − ( η spl p ) − −−−−−→ Y q ∈ Spl p V ⊗ E E q − is an isomorphism of Q q ∈ Spl p E q − -modules;(5) for every q ∈ p , the canonical map ¯V ⊗ F F q → Hom k (( A k [ q ∞ ] , i k ) , ( A k [ q ∞ ] , i k )) ⊗ O F q F q is an isomorphism of E q -vector spaces.(6) for every q ∈ p \ { p } , ¯Λ q := Hom k (( A k [ q ∞ ] , i k ) , ( A k [ q ∞ ] , i k )) is a self-dual lattice in ¯V ⊗ F F q ;(7) ¯V ⊗ F F p does not admit a self-dual lattice.Proof. We first show that the canonical map¯V ⊗ Q Q p → Hom k (( A k [ p ∞ ] , i k ) , ( A k [ p ∞ ] , i k )) ⊗ Z p Q p (C.12)is an isomorphism. Let O D be the O F, ( p ) -algebra of endomorphisms of ( A k , i k | O F, ( p ) , λ k ), andput D := O D ⊗ O F, ( p ) F . Then D is a totally definite (division) quaternion algebra over F whichcontains E via i k . We write D = E ⊕ Ej for some element j ∈ O D \ pO D such that j − ej = e c forevery e ∈ E . Choose an element f ∈ O F such that f ∈ p but f q for every other prime q of F above p ; and put j := j + f . We define a new action i k of O E, ( p ) on B k via the formula i k ( e ) = j ◦ i k ( e ) ◦ j . Then ( A k , i k ) is an ( E, Φ )-abelian scheme over k , where Φ := (Φ c \ { τ c } ) ∪ { τ } ,with a polarization λ k := ( j ) ∗ λ k . Now by [RZ96, Proposition 6.29], ( A k , i k ) is quasi-isogenous to( A k , i k ) n − × ( A k , i k ). In particular, (C.12) is an isomorphism. From this, (1), (3), (4), and (5)follow immediately. Part (2) can be proved in the same way as [KR14, Lemma 2.7]. Part (7) is aconsequence of (1–6) and the Hasse principle.It remains to show (6). By the above discussion, ( A k [ q ∞ ] , i k ) is quasi-isogenous to ( A k [ q ∞ ] , i k ) n for q ∈ p \ { p } . Then they must be isomorphic. In particular, the induced hermitian form on ¯Λ q isgiven by the identity matrix under some basis. Thus, we obtain (6). (cid:3) Lemma C.25(3,4) gives rise to an isomorphism ι P : ¯V ⊗ F A ∞ F , p → V ⊗ F A ∞ F , p (C.13)of hermitian spaces over A ∞ F , p . Let ¯ K q be the stabilizer of ¯Λ q in Lemma C.25(6) for every q ∈ p \ { p } ,which is a hyperspecial maximal subgroup of U( ¯V)( F q ).Let M ( W ∞ , Φ c ) ∧ L be the completion of M ( W ∞ , Φ c ) L ⊗ O E Φ , ( p ) O E nr p along the special fiber, whichis isomorphic to a finite disjoint union of Spf O ∧ E nr p . Then (C.7) induces a morphism q ∧ : M (V , W ∞ , Φ) ss , ∧ K p ,L → M ( W ∞ , Φ c ) ∧ L of formal schemes over O ∧ E nr p . Proposition C.26. The chosen point P (C.8) induces the following Cartesian diagram U( ¯V)( F ) \ (cid:16) N × U( ¯V)( A ∞ F , p ) / Q q ∈ p \{ p } ¯ K q (cid:17) (cid:47) (cid:47) u P (cid:15) (cid:15) Spf O ∧ E nr p q ∧ ◦ P ∧ (cid:15) (cid:15) M (V , W ∞ , Φ) ss , ∧ K p ,L q ∧ (cid:47) (cid:47) M ( W ∞ , Φ c ) ∧ L of formal schemes over O ∧ E nr p , satisfying • u P ◦ ( P ∧ loc , 1) = P ∧ (see (C.9) and (C.10) ); • u P ◦ T ¯ g = T g ◦ u P for every g ∈ U(V)( A ∞ F , p ) and ¯ g ∈ U( ¯V)( A ∞ F , p ) that correspond under ι P (C.13) , where T g (resp. T ¯ g ) denotes the Hecke translation on the target (resp. source) of u P .Proof. The proof is very similar to [RZ96, Theorem 6.30]. For readers’ convenience, we will describethe morphism v P : M P → U( ¯V)( F ) \ N × U( ¯V)( A ∞ F , p ) / Y q ∈ p \{ p } ¯ K q , (C.14)where M P is the pullback of q ∧ along q ∧ ◦ P ∧ , for which u P is the inverse. This is the hardest step;and in particular, we will see how this morphism depends on P .Let S be a connected scheme in Sch /O ∧ E nr p on which p is locally nilpotent, with a chosen geometricpoint s ∈ S ( k ). Take a point P = ( A , i , λ , η p ; A, i, λ, η p , η spl p ) ∈ M P ( S ), where ( A , i , λ , η p ) isthe base change of ( A , i , λ , η p ) to S . By [RZ96, Proposition 6.29], we can choose an O E -linearquasi-isogeny ρ : A × S S k → A k × k S k such that ρ ∗ λ k = λ k . Then ( A [ p ∞ ] , i [ p ∞ ] , λ [ p ∞ ]; ρ [ p ∞ ]) is an element in N ( S ) by [Mih, Theorem 3.3].The composite map¯V ⊗ Q A ∞ ,p ι P −→ V ⊗ Q A ∞ ,p η p −→ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A k , A ∞ ,p ) , H ´et1 ( A s , A ∞ ,p )) ρ s ∗ ◦ −−→ Hom E ⊗ Q A ∞ ,p (H ´et1 ( A k , A ∞ ,p ) , H ´et1 ( A k , A ∞ ,p )) = ¯V ⊗ Q A ∞ ,p OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 75 is an isometry, which gives rise to an element g pP ∈ U( ¯V)( A ∞ F ,p ). The same process will produce anelement g spl P,p ∈ Q q ∈ Spl p U( ¯V)( F q ). For every q ∈ p \ { p } , the image of the map ρ s ∗ ◦ : Hom k (( A k [ q ∞ ] , i k ) , ( A s [ q ∞ ] , i s )) → Hom k (( A k [ q ∞ ] , i k ) , ( A k [ q ∞ ] , i k )) ⊗ O F q F q = ¯V ⊗ F F q is a self-dual lattice, say Λ P, q . Therefore, there exists a unique element g P, q ∈ U( ¯V)( F q ) / ¯ K q such that g P, q Λ P, q = ¯Λ q . Together, we obtain an element (cid:16) ( A [ p ∞ ] , i [ p ∞ ] , λ [ p ∞ ]; ρ [ p ∞ ]) , g pP , g spl P,p , ( g P, q ) q (cid:17) ∈ N ( S ) × U( ¯V)( A ∞ F , p ) / Y q ∈ p \{ p } ¯ K q depending on the choice of ρ . However, changing ρ will result the left multiplication by an element inU( ¯V)( F ). Thus, the element v P ( P ) := (cid:16) ( A [ p ∞ ] , i [ p ∞ ] , λ [ p ∞ ]; ρ [ p ∞ ]) , g pP , g spl P,p , ( g P, q ) q (cid:17) is a well-defined element in the right-hand side of (C.14). The construction of the inverse of v P ,which is nothing but u P , is easy by Dieudonné theory. We leave the details to readers; it is the sameargument in [RZ96]. (cid:3) Remark C.27 . In fact, the morphism u P in Proposition C.26 is compatible with more Hecke operators.Consider a prime q ∈ p \ { p } . For every double coset K q gK q ⊆ U(V)( F q ), we have the Heckecorrespondence T K q gK q on the target of u P which is simply the Zariski closure of the usual Heckecorrespondence on the generic fiber; it is in fact étale. Then we have u ∗ P T K q gK q = T ¯ K q ¯ g ¯ K q if K q gK q =¯ K q ¯ g ¯ K q under the canonical isomorphism K q \ U(V)( F q ) /K q ’ ¯ K q \ U( ¯V)( F q ) / ¯ K q . Here, T ¯ K q ¯ g ¯ K q denotesthe set-theoretical Hecke correspondence on the source of u P . Appendix D. Cohomology of unitary Shimura curves In this appendix, we compute the cohomology of Shimura curves associated to isometry groups ofhermitian spaces of rank 2, as Galois–Hecke modules. In Subsection D.1, we collect some results aboutlocal oscillator representations of unitary groups of general rank. In Subsection D.2, we recall somefacts and introduce some notation about cohomology of Shimura varieties in general. The last twosubsections concern the cohomology of unitary Shimura curves, for the statements and for the proof,respectively. These statements are only used in the proof of Theorem 4.15 and Theorem 4.18 in themain part of the article.D.1. Oscillator representations of local unitary groups. Let F be a local field whose charac-teristic is not 2. Let E be an étale F -algebra of rank 2. Denote by c the unique nontrivial involutionon E that fixes F , and put E − := { x ∈ E | x + x c = 0 } and E := { x ∈ E | xx c = 1 } . Let V , ( , ) V bea (non-degenerate) hermitian space over E (with respect to c ) of rank n (cid:62) Step 1: Choose an element ε ∈ E −× / N E/F E × . Let V ε be the underlying F -vector space of Vequipped with the form Tr E/F ε ( , ) V , which becomes a symplectic space. Let Mp(V ε ) bethe metaplectic group of V ε with center C . Then we have the oscillator representation ω ( ε )of Mp(V ε ) using the standard additive character ψ F . Step 2: Choose a character µ : E × → C such that µ | F × is the unique character whose kernel isexactly N E/F E × . Then we have the induced homomorphism ι µ : U(V) → Mp(V ε ) (see, forexample, [HKS96, Section 1]). Put ω ( µ, ε ) := ω ( ε ) ◦ ι µ . Step 3: Choose a character χ : E → C . Let ω ( µ, ε, χ ) be the maximal quotient of the representation ω ( ε, µ ) of U(V) with central character χ .For χ as above, we define a character ˇ χ of E × via the formula ˇ χ ( x ) = χ ( x/x c ). More precisely, we have to choose an element in E −× in the coset ε ; and it is known that the resulting oscillatorrepresentation depends only on ε . Lemma D.1. Suppose that F is nonarchimedean. Then ω ( µ, ε, χ ) is irreducible and admissible.Moreover,(1) ω ( µ, ε, χ ) is zero if and only if E is a field; V is anisotropic (in particular n = 2 ); and ˇ χ = µ ;(2) the contragredient representation of ω ( µ, ε, χ ) is isomorphic to ω ( µ c , − ε, χ − ) , where µ c := µ ◦ c as usual;(3) if n (cid:62) , then ω ( µ , ε , χ ) is isomorphic to ω ( µ, ε, χ ) if and only if ( µ , ε , χ ) = ( µ, ε, χ ) ;(4) if n = 2 and ω ( µ, ε, χ ) is nonzero, then ω ( µ , ε , χ ) is isomorphic to ω ( µ, ε, χ ) if and only ifeither ( µ , ε , χ ) = ( µ, ε, χ ) , or µ = µ c ˇ χ , χ = χ and ε = ε (resp. ε = ε ) when V is isotropic(resp. anisotropic).Proof. We consider first the case where E = F × F . We identify U(V) with GL n ( F ) and E − with F through the first factor; and write µ = ν (cid:2) ν − . Note that the first component of ˇ χ is simply χ .Let Q n − , be the standard parabolic subgroup of GL n whose Levi is GL n − × GL . Then ω ( µ, ε, χ ) isisomorphic to the unitary induction from Q n − , ( F ) to GL n ( F ) of the (unitary) character ( ν ◦ det) (cid:2) χν − n of GL n − ( F ) × GL ( F ) (hence of Q n − , ( F )). See for example [GR90, 2.6]. The lemma followsfrom such description.Now we assume that E is a field. The fact that ω ( µ, ε, χ ) is irreducible is a special case of the Howeduality; see for example [GT16, Theorem 1.1(1)].For (1), the fact that ω ( µ, ε, χ ) is nonzero unless in the exceptional case in (1) follows from thepersistence property [HKS96, Proposition 5.1(iii)], and the first occurrence speculation [HKS96, Spec-ulation 7.5 & Speculation 7.6] (which has been proved as [SZ15, Theorem 1.10]). Note that in theexceptional case, the first occurrence of the theta lifting of the trivial character in the split even toweris 0; therefore its first occurrence in the nonsplit even tower is 4. See [HKS96, p.986] for more details.Note that, since E is compact, we have a canonical isomorphism of representations of U(V) ω ( µ, ε ) ’ M χ ω ( µ, ε, χ ) . For (2), note that under the canonical isomorphism Mp(V ε ) ’ Mp(V − ε ), the contragredient of ω ( ε ) is isomorphic to ω ( − ε ). Moreover, under such isomorphism, ι µ coincides with ι µ c by [HKS96,Lemma 1.1 & (1.8)]. Therefore, ω ( µ, ε ) is contragredient to ω ( µ c , − ε ). Since E is compact, we havea canonical isomorphism ω ( µ, ε ) ’ L χ ω ( µ, ε, χ ). Thus, ω ( µ, ε, χ ) is contragredient to ω ( µ c , − ε, χ − )as both are irreducible with inverse central character, or both are zero.For (3), it is known when n = 3 by [GR90, Proposition 5.1.4]. In fact, the same proof also worksfor n > χ = χ . By the description of endoscopic packets for in [GGP12b, Section 8],we must have either µ = µ or µ = µ c ˇ χ . There are two cases.Suppose that µ c ˇ χ = µ . Then ω ( µ, ε, χ ) = { } when V is anisotropic; and ω ( µ, ε, χ ) is not isomorphicto ω ( µ, ε , χ ) when V is isotropic and ε = ε . Thus, (4) follows.Suppose that µ c ˇ χ = µ . Then the packet has four members, and we need to show that ω ( µ, ε, χ ) ’ ω ( µ c ˇ χ, ε , χ ) for ε = ε (resp. ε = ε ) when V is isotropic (resp. anisotropic). We adopt the notation in[GGP12b, Section 8]. Let M be the two-dimensional conjugate symplectic representation associatedto the packet. Then M has two non-isomorphic one dimensional conjugate symplectic representations.We write M = M • ⊕ M • = M ◦ ⊕ M ◦ for the two different ways of ordering of direct summands.Thus, we obtain two ways of labelling for the four members in the packet, say { π ++ • , π −−• , π + −• , π − + • } and { π ++ ◦ , π −−◦ , π + −◦ , π − + ◦ } , respectively. Then (4) is equivalent to the isomorphisms π ++ • ’ π ++ ◦ , π −−• ’ π −−◦ , π + −• ’ π − + ◦ , and π − + • ’ π + −◦ . However, these isomorphisms are consequences of[GGP12b, Theorem 10.2]. (cid:3) Now we take F = R and E = C . Let ( p, q ) be the signature of V. Then we may identify U(V)with U( p, q ) R , the subgroup of Res C / R GL n of elements preserving the hermitian form given by thematrix (cid:16) I p − I q (cid:17) . Denote by u p,q the Lie algebra of U( p, q ) R and fix a maximal compact subgroup K p,q OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 77 of U( p, q ) R ( R ). In the construction of ω ( µ, ε, χ ), the three parameters have the following possibilities: µ m ( z ) = arg( z ) m , m odd integer; ε = ± i ; χ l ( z ) = z l , l ∈ Z . To shorten notation, we denote by ω m, ± ,lp,q the representation ω ( µ m , ± i, χ l ) of U( p, q ) R . It is well-known(see, for example [SW78, Section 4]) that ω m, ± ,lp,q is irreducible.By the computation in [BMM16, Section 5], up to equivalence, there are only two irreducibleunitary representations π of U( n − , R such that H ( u n − , , K n − , ; π ) = { } , in which case thecohomology has dimension 1 for both. Let us label them by π , n − , and π , n − , in the way thatH ( u n − , , K n − , ; π , n − , ) has Hodge type (1 , ( u n − , , K n − , ; π , n − , ) has Hodge type (0 , Lemma D.2. Let the notation be as above.(1) Among the representations ω m, ± ,ln, , only ω , + , n, and ω − , − , n, are the trivial character.(2) If n (cid:62) , then in the set { ω m, ± ,ln − , } , only ω − , − , n − , (resp. ω , + , n − , ) is isomorphic to π , n − , (resp. π , n − , ).(3) If n = 2 , then in the set { ω m, ± ,l , } , only ω − , − , , and ω , − , , (resp. ω , + , , and ω − , + , , ) areisomorphic to π , , (resp. π , , ).Proof. The explicit formulae for the K p,q -type of ω m, ± ,lp,q can be found in, for example, [KK07, Theo-rem 5.4] with p + q = 1. In particular, (1) follows directly.For (2) and (3), it is shown in [BMM16, Section 5] that both π , n − , and π , n − , are isomorphic tosome ω m, ± ,ln − , . Comparing the formula for the highest weights in [BMM16, 5.7] with p = n − , q =1 , a + b = 1( (cid:54) p ) with [KK07, Theorem 5.4], we obtain the assertions. (cid:3) D.2. Setup for cohomology of Shimura varieties. Let us recall some general facts about coho-mology of Shimura varieties. Let (G , h) be a Shimura data with E ⊆ C its reflex field. In particular,G is a reductive group over Q . Let ξ be an algebraic complex representation of G. Then it induces acomplex local system L ξ on { Sh(G , h) K ⊗ E C } . Let H i (2) (Sh(G , h) K ( C ) , L ξ ) be the i -th L -cohomologyof the complex manifold Sh(G , h) K ( C ) with coefficient L ξ . PutH i (2) (Sh(G , h) , L ξ ) := lim −→ K H i (2) (Sh(G , h) K ( C ) , L ξ )which is a smooth representation of G( A ∞ ). By the Matsushima formula for L -cohomology, we havean isomorphism H i (2) (Sh(G , h) , L ξ ) ’ M π m disc ( π )H i ( g , K G ; ξ ∞ ⊗ π ∞ ) ⊗ π ∞ (D.1)of G( A ∞ )-modules, where • g := Lie G R , and K G is a maximal connected compact subgroup of G( R ); • ξ ∞ is the associated ( g , K G )-module of ξ ; and • π = π ∞ ⊗ π ∞ runs through isomorphism classes of irreducible admissible representations ofG( A ), where m disc ( π ) is the discrete multiplicity of π (Definition B.1).Here, we have to use [BC83, Section 4] to conclude that the continuous part of L (G( Q ) \ G( A ) , χ ) doesnot contribute to the L -cohomology in the case of Shimura varieties. By Zucker’s conjecture (provedindependently by Looijenga [Loo88] and Saper–Sturn [SS90]), we have a canonical isomorphismH i (2) (Sh(G , h) , L ξ ) ’ IH i (Sh(G , h) , L ξ )(D.2)of G( A ∞ )-modules, whereIH i (Sh(G , h) , L ξ ) := lim −→ K IH i (Sh(G , h) K ⊗ E C , L ξ ) , In this article, we only need the case where ξ is the trivial representation. is the direct limit over K of the complex analytic intersection cohomology of Sh(G , h) K ⊗ E C , whereSh(G , h) K is the Baily–Borel compactification of Sh(G , h) K (over E ).Now let ‘ be a rational prime and choose an isomorphism ι ‘ : C ∼ −→ Q ac ‘ p. Then the Q ac ‘ -local system L ξ ⊗ C ,ι ‘ Q ac ‘ descends to an (étale) Q ac ‘ -local system L ξ,ι ‘ on { Sh(G , h) K } . We then have a comparisonisomorphism IH i ´et (Sh(G , h) , L ξ,ι ‘ ) ’ IH i (Sh(G , h) , L ξ ) ⊗ C ,ι ‘ Q ac ‘ where IH i ´et (Sh(G , h) , L ξ,ι ‘ ) := lim −→ K IH i ´et (Sh(G , h) K ⊗ E C , L ξ,ι ‘ ) . For an irreducible admissible representation π ∞ of G( A ∞ ), putIH iξ,ι ‘ ( π ∞ ) := Hom Q ac ‘ [G( A ∞ )] (cid:16) ι ◦ π ∞ , IH i ´et (Sh(G , h) , L ξ,ι ‘ ) (cid:17) which is a finite dimensional representation of Gal( C /E ), whose dimension is equal to X π ∞ m disc ( π ∞ ⊗ π ∞ ) dim C H i ( g , K G ; ξ ∞ ⊗ π ∞ ) , where π ∞ runs through all irreducible admissible representations of G( R ). We suppress ξ in thenotation if it is the trivial representation.D.3. Statements for cohomology of unitary Shimura curves. We fix a CM number field E andregard E as a subfield of C via a fixed complex embedding τ : E , → C . Let c ∈ Gal( E/ Q ) be theinduced complex conjugation and put F := E c =1 . Write Φ F = { τ , . . . , τ d } with d = [ F : Q ] as the setof real embeddings of F , in which τ is induced by τ .Let V be a hermitian space over E of rank 2 of signature (1 , 1) at τ and (2 , 0) elsewhere. As inSubsection C.1 especially Remark C.2, we have the Hodge map h := h V ,τ and the Shimura varieties { Sh(G , h) K } defined over E , and their Baily–Borel compactification Sh(G , h) K . They are all smoothcurves over E . By the discussion from Subsection D.2, we have an isomorphismH (Sh(G , h) , C ) ’ M π m disc ( π )H ( g , K G ; π ∞ ) ⊗ π ∞ of G( A ∞ )-modules, where H (Sh(G , h) , C ) := lim −→ K H (Sh(G , h) K , C ). By Lemma D.2, up to equiva-lence, there are only two representations π ∞ of G( R ) with H ( g , K G ; π ∞ ) = { } , namely, π (1 , ∞ := π , , ⊗ ⊗ · · · ⊗ π (0 , ∞ := π , , ⊗ ⊗ · · · ⊗ . Definition D.3. Let π ∞ be an irreducible admissible representation of G( A ∞ ). • We say that π ∞ is stable cohomological if both π (1 , ∞ ⊗ π ∞ and π (0 , ∞ ⊗ π ∞ have positive cuspidalmultiplicity. • We say that π ∞ is endoscopic cohomological if exactly one of π (1 , ∞ ⊗ π ∞ and π (0 , ∞ ⊗ π ∞ haspositive cuspidal multiplicity.Denote C stV (resp. C endV ) the set of isomorphism classes of stable (resp. endoscopic) cohomologicalirreducible admissible representations of G( A ∞ ). Put C V := C stV ‘ C endV . Proposition D.4. Let π ∞ be an irreducible admissible representation of G( A ∞ ) .(1) If π ∞ is endoscopic cohomological, then there exists a unique adèlic oscillator triple ( µ, ε, χ ) (Definition 4.11) with µ of weight one and satisfying τ ∈ Φ µ , such that π ∞ is isomorphic to ω ( µ, ε, χ ) . Moreover, Hom C [G( A ∞ )] ( π ∞ , H (Sh(G , h) , C )) has dimension .(2) If π ∞ is stable cohomological, then Hom C [G( A ∞ )] ( π ∞ , H (Sh(G , h) , C )) has dimension .Proof. Let V ∗ be an isotropic skew-hermitian space over E of rank 2, which is unique up to isomor-phism. The global inner transfer from U(V) to U(V ∗ ) is known; see, for example, [Har93]. Moreprecisely, let V π be an irreducible U(V)( A F )-submodule of L (U(V)) and denote V π its complexconjugate space. We may choose an automorphic character ξ : E \ ( A ∞ E ) → C × such that the global OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 79 theta lifting Θ V ∗ ψ F , (1 , , V ( V π ⊗ ξ ) is nonzero. Then JL( V π ) := Θ V ∗ ψ F , (1 , , V ( V π ⊗ ξ ) ⊗ ξ is a subspace ofL (U(V ∗ )), which is an irreducible U(V ∗ )( A F )-module and independent of the choice of ξ . Denoteby JL( π ) the representation of U(V ∗ )( A F ) on JL( V π ). Since the complement of L in L consistsof automorphic characters, we have m disc ( π ) = m disc (JL( π )).The Langlands–Arthur classification for U(V ∗ ) is known by [Rog90, Section 11]. Let π ∗ be anirreducible cuspidal automorphic representation of U(V ∗ )( A F ). We have the (standard) base changeΠ of π ∗ , which is an irreducible isobaric automorphic representation of GL ( A E ). We say that π ∗ isstable (resp. endoscopic) if Π is cuspidal (resp. Π ’ Π (cid:1) Π for two conjugate symplectic automorphiccharacters Π and Π ).Suppose that π ∞ is stable cohomological. Then both JL( π (1 , ∞ ⊗ π ∞ ) and JL( π (0 , ∞ ⊗ π ∞ ) havepositive multiplicity and the same base change Π. By Arthur’s multiplicity formula, Π has to cuspidal,and m disc ( π (1 , ∞ ⊗ π ∞ ) = m disc ( π (0 , ∞ ⊗ π ∞ ) = 1. In particular, (2) follows.Suppose that π ∞ is endoscopic cohomological. Then by the same reasoning, we have Π ’ Π (cid:1) Π ,and m disc ( π (1 , ∞ ⊗ π ∞ ) + m disc ( π (0 , ∞ ⊗ π ∞ ) = 1. Let π be the unique member in { π (1 , ∞ ⊗ π ∞ , π (0 , ∞ ⊗ π ∞ } such that m disc ( π ) = 1. Since JL( π ) is endoscopic, both Π and Π are conjugate symplecticautomorphic characters of weight one. Thus, there exists a conjugate symplectic automorphic character µ of weight one such that L ( s, Π ⊗ µ ) has a simple pole at s = 1. By Theorem B.4, we have a skew-hermitian space W over E of rank 1 of determinant e ∈ E −× / N E/F E × and an automorphic character χ of U(W)( A F ), such that π is realized in the space of global theta lifting Θ V ψ F , ( µ,ν ) , W ( χ ). Let χ bethe central character of π . Then it is trivial at infinity. Thus, by Lemma D.1(4), there exist exactlytwo adèlic oscillator triples, which are ( µ, ε, χ ) and ( µ c ˇ χ, ε , χ ), such that π ∞ is isomorphic to theassociated oscillator representation. In particular, the condition that µ is of weight one and satisfies τ ∈ Φ µ picks up exactly one of the two triples. Therefore, (1) follows. (cid:3) Remark D.5 . The proof of Proposition D.4(1) implies that for π ∞ ’ ω ( µ, ε, χ ) that is endoscopiccohomological, we have m cusp ( π (1 , ∞ ⊗ π ∞ ) = 1 (resp. m cusp ( π (0 , ∞ ⊗ π ∞ ) = 1) if and only if there existssome e ∈ E ×− such that • ε v = e N E v /F v E × v for every nonarchimedean place v of F ; • τ i ( e ) has negative imaginary part for i = 2 , . . . , d , where τ i is the unique element in Φ µ above τ i ; and • τ ( e ) has negative (resp. positive) imaginary part.Now we study ‘ -adic cohomology of { Sh(G , h) K } K . Take a rational prime ‘ and an isomorphism ι ‘ : C ∼ −→ Q ac ‘ . Put H (Sh(G , h) , Q ac ‘ ) := lim −→ K H (Sh(G , h) K ⊗ E C , Q ac ‘ ) . By the comparison theorem, we have a canonical isomorphismH (Sh(G , h) , Q ac ‘ ) ’ H (Sh(G , h) , C ) ⊗ C ,ι ‘ Q ac ‘ of G( A ∞ )-modules. For an irreducible admissible representation π ∞ of G( A ∞ ), the Q ac ‘ -vector spaceH ι ‘ ( π ∞ ) := Hom Q ac ‘ [G( A ∞ )] (cid:16) ι ‘ ◦ π ∞ , H (Sh(G , h) , Q ac ‘ ) (cid:17) is a representation of Gal( C /E ), which we denote by ρ ι ‘ ( π ∞ ).Suppose that π ∞ is endoscopic cohomological. Then by Proposition D.4, we obtain an ‘ -adiccharacter ρ ι ‘ ( π ∞ ) : Gal( C /E ) → ( Q ac ‘ ) × . It induces, via the isomorphism ι ‘ , an automorphic character ρ ‘ ( π ∞ ) : E × \ A × E → C × . It is easy to see that the character ρ ‘ ( π ∞ ) does not depend on the choice ofthe isomorphism ι ‘ , which justifies its notation. Theorem D.6. Let π ∞ be an irreducible admissible representation of G( A ∞ ) , and ‘ a rational prime. (1) Suppose that π ∞ is endoscopic cohomological, which is isomorphic to ω ( µ, ε, χ ) with µ of weightone and satisfying τ ∈ Φ µ as in Theorem D.4(1). Then ρ ‘ ( π ∞ ) = µ · | | − / E if m cusp ( π (1 , ∞ ⊗ π ∞ ) = 1; µ c ˇ χ · | | − / E if m cusp ( π (0 , ∞ ⊗ π ∞ ) = 1 . (2) Suppose that π ∞ is stable cohomological. Then for every ι ‘ : C ∼ −→ Q ac ‘ , we have(a) ρ ι ‘ ( π ∞ ) is an irreducible two-dimensional representation of Gal( C /E ) ;(b) ρ ι ‘ ( π ∞ ) ∨ ’ ρ ι ‘ ( π ∞ ) c (1) ;(c) if we let Π ∞ be the irreducible admissible representation of GL ( A ∞ E ) that is the standardbase change of π ∞ , then for every nonarchimedean place w of E coprime to ‘ , WD( ρ ι ‘ ( π ∞ ) | Gal( E ac w /E w )) F - ss ’ ι ‘ ◦ L ,E w (Π ∞ w ) holds, where L ,E w denotes the local Langlands correspondence for GL ,E w . The proof of the theorem will be given in Subsection D.4.The theorem reveals some information about the Albanese variety (Jacobian) A K of Sh(G , h) K .We have a homomorphism C ∞ c ( K \ G ( A ∞ ) /K, Q ) → End( A K ) Q of Q -algebras induced by the Heckeactions. Note that Gal( C / Q ) acts on C V by acting on the coefficient C , which preserves the two subsets C stV and C endV . Therefore, we obtain an isogeny decomposition A K ∼ A st K × A end K (D.3)(over E ) such that under the canonical isomorphism in Lemma 2.4(1), we have isomorphismsH ( A st K , C ) ’ M π ∞ ∈C stV H (Sh(G , h) K , C )[( π ∞ ) K ] , H ( A end K , C ) ’ M π ∞ ∈C endV H (Sh(G , h) K , C )[( π ∞ ) K ]of C ∞ c ( K \ G ( A ∞ ) /K, Q )-modules.Put C stV := C stV / Gal( C / Q ), the set of Gal( C / Q )-orbits in C stV . For every orbit π ∞ , denote M ( π ∞ ) ⊆ C its field of definition, namely, the fixed field of the stabilizer of π ∞ in Gal( C / Q ); it is a number field,either totally real or CM. By Theorem D.6(2) and a standard argument, we may associate π ∞ a(simple) abelian variety A ( π ∞ ) over E , which satisfies • dim A ( π ∞ ) = [ M ( π ∞ ) : Q ], • End E ( A ( π ∞ )) Q ’ M ( π ∞ ), and • A ( π ∞ ) ⊗ E, c E is isogenous to A ( π ∞ ) ∨ .In fact, A ( π ∞ ) is of strict GL(2)-type in the terminology of [YZZ13, Section 3.2.1]. Finally, note thatfor every open compact subgroup K of G( A ∞ ), the dimension of K -fixed vectors in a representationsin π ∞ depends only on the orbit, which we denote by dim C ( π ∞ ) K . Theorem D.6(2) has the followingcorollary. Corollary D.7. For every sufficiently small open compact subgroup K of G( A ∞ ) , we have an isogenydecomposition A st K ∼ Y π ∞ ∈C stV A ( π ∞ ) dim C ( π ∞ ) K compatible with changing K in the obvious way. In particular, A st K does not have factors that are ofCM type. Moreover, A ( π ∞ ) is isogenous to A ( π ∞ ) for π ∞ , π ∞ ∈ C stV if and only if there exist π ∞ ∈ π ∞ and π ∞ ∈ π ∞ such that their standard base change representations of GL ( A ∞ E ) are isomorphic. The isogeny decomposition of A end K is a special case of Corollary 4.20. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 81 D.4. Proof of Theorem D.6. We prove Theorem D.6 by first establishing a congruence relation forthe Shimura curve Sh(G , h) K over a set of primes of E of density 1.To state the congruence relation, we fix a prime q of E , with the underlying rational prime p , suchthat • G ⊗ Q Q p is unramified; in particular, p is unramified in E ; • q = q c , that is, q has degree 1 over F .Denote by p the prime of F underlying q . We identify F p with E q . Choose a uniformizer $ of F p . Put O p := O F p , κ := O p /$ O p , and q := κ . Fix a maximal unramified extension F nr p of F p with O nr p thering of integers and κ ac := O nr p /$ O nr p the residue field. Let σ : O nr p → O nr p be the q -th Frobenius map.Fix a basis of the E q -vector space V ⊗ E E q under which we identify U(V ⊗ F F p ) with GL ,F p .Let Iw p := (cid:16) O p O p $ O p O p (cid:17) ⊆ GL ( O p ) be an Iwahori subgroup. We consider open compact subgroups K ⊆ G( A ∞ ) of the form GL ( O p ) × K p p × K p where GL ( O p ) × K p p is a hyperspecial maximal subgroupof G( Q p ) and K p is a sufficiently small open compact subgroup of G( A ∞ ,p ). For such K , we put K Iw := Iw p × K p p × K p . We have the projection morphism π : Sh(G , h) K Iw → Sh(G , h) K , and anisomorphism t $ : Sh(G , h) K Iw ∼ −→ Sh(G , h) K Iw induced by the Hecke translation of elements ( $ ). In view of the reciprocity map in Remark C.2,the morphism t $ ⊗ σ : Sh(G , h) K Iw ⊗ F p F nr p → Sh(G , h) K Iw ⊗ F p F nr p preserves every connected component.We will show in Proposition D.8 that Sh(G , h) K (resp. Sh(G , h) K Iw ) admits a smooth model (resp.a stable model) S K (resp. S K Iw ) over O p . By [LL99, Proposition 4.4(a)], the morphism t $ extends(uniquely) to a morphism t $ : S K Iw → S K Iw which has to be an isomorphism; and π extends (uniquely)to a morphism π : S K Iw → S K . Finally, to ease notation, we put T K := S K ⊗ O p κ and T K Iw := S K Iw ⊗ O p κ for the special fibers. Proposition D.8. Let the notation be as above. We have(1) The smooth projective F p -curve Sh(G , h) K admits a smooth model S K over O p .(2) The smooth projective F p -curve Sh(G , h) K Iw admits a stable model S K Iw over O p .(3) The κ -scheme T K Iw has two irreducible components T + K Iw and T − K Iw , satisfying that(a) π + := π | T + K Iw : T + K Iw → T K is an isomorphism;(b) π − := π | T − K Iw : T − K Iw → T K is a finite flat morphism of degree q ;(c) t $ ⊗ σ induces an isomorphism between T + K Iw ⊗ κ κ ac and T − K Iw ⊗ κ κ ac ;(d) the morphism ( π − ⊗ id) ◦ (t $ ⊗ σ ) ◦ ( π + ⊗ id) − coincides with the absolute q -th Frobeniusmorphism of T K ⊗ κ κ ac .Proof. We first assume F = Q . Then we have Sh(G , h) K = Sh(G , h) K . We will reduce the propositionto (a weak form of) the congruence relation in [Car86, Proposition 10.3] by changing the Shimuradatum. Choose a quaternion algebra over B together with an embedding E , → B of F -algebras,such that the induced hermitian form on B is isomorphic to V. In particular, B is indefinite at τ anddefinite at all other places of F ; B is division at a nonarchimedean place v of F if and only if V v isanisotropic. We identify V with B as hermitian spaces. Let ( B × × E × ) be the subgroup of B × × E × consisting of elements ( b, e ) such that N B/F b · N E/F e = 1, viewing as a reductive group over F . Thenwe have a short exact sequence1 → G m ,F → ( B × × E × ) → U(V) → In [Car86], the initial Shimura variety is a quaternionic Shimura curve, and the auxiliary Shimura variety is (quater-nionic) unitary Shimura curve of PEL type. However, in our case, the initial Shimura variety is unitary Shimura curveof non-PEL type, and the auxiliary Shimura variety we introduce below is the quaternionic Shimura curve, which is theinitial Shimura variety of Carayol. So strictly speaking, to obtain this proposition, we have to change Shimura data twicebut the second step has already been carried out by Carayol. Such consideration was also used in [Liu11b]. where the homomorphism G m ,F → ( B × × E × ) is given by e ( e, e − ). The fixed basis of V ⊗ E E q identifies B ⊗ F F p with Mat ( F p ), and further ( B ⊗ F F p ) × with U(V)( F p ).Put G := Res F/ Q B × and let h be the Hodge map that is inverse to the one given in [Car86, 0.1].We have the Shimura curve Sh(G , h ) K defined over F . Here, the open compact subgroup K ⊆ G ( A ∞ ) is of the form K p × K p , where K p is hyperspecial maximal of the form GL ( O p ) × K p p .Replacing GL ( O p ) by Iw p , we obtain K Iw and hence Sh(G , h ) K Iw . Repeat the constructions for theShimura data (G , h) to (G , h ), we obtain Sh(G , h ) K , Sh(G , h ) K Iw , π and t $ . By Deligne’s theoryof connected Shimura varieties [Del79] (or see [Car86, Section 4]), for every connected componentSh(G , h) † K of Sh(G , h) K ⊗ F p F nr p , there exists some K p and a connected component Sh(G , h ) † K ofSh(G , h ) K ⊗ F p F nr p such that there is a commutative diagramSh(G , h) † K Iw ’ (cid:47) (cid:47) π (cid:15) (cid:15) Sh(G , h ) † K Iw π (cid:15) (cid:15) Sh(G , h) † K ’ (cid:47) (cid:47) Sh(G , h ) † K where Sh(G , h) † K Iw := π − Sh(G , h) † K and Sh(G , h ) † K Iw := π Sh(G , h ) † K , under which the automor-phism t $ ⊗ σ of Sh(G , h) † K Iw coincide with the automorphism t $ ⊗ σ of Sh(G , h ) † K Iw respectively.Therefore, the proposition will follow from the version for (G , h ). To release ourselves from the clumsy notation, we will now suppress the “prime” in all superscripts;in particular, the group G now is Res F/ Q B × . Then (1) follows from [Car86, Proposition 6.1]. For theremaining claims, we need some preparation.For n (cid:62) 1, put K n := (I + $ n GL ( O p )) × K p p × K p . In [Car86, 1.4.4], Carayol constructed an O p -divisible group E ∞ over Sh(G , h) K , such that the pullback of E ∞ [ p n ] to Sh(G , h) K n is trivial. Bythe construction, the subgroup Sh(G , h) K × ( ∗ ) ⊆ Sh(G , h) K × ( p − / O p ) is stable under the action(given in [Car86, 1.4.2]) of Iw p . In particular, it defines an O p -stable subgroup C Iw of E ∞ [ p ] overSh(G , h) K Iw of rank q . By [Car86, Proposition 6.4], the O p -divisible group E ∞ extends uniquely to an O p -divisible group E ∞ over S K such that E ∞ | T K is of dimension 1 and O p -height 2.We define a functor S K Iw over S K such that for every S K -scheme u : S → S K , the set S K Iw ( S )consists of O p -stable finite flat S -subgroups of u ∗ E ∞ [ p ] of rank q . As pointed out in [Car86, Sec-tion 6.7], the supersingular locus of E ∞ is discrete. Thus, it follows from [Car86, Proposition 6.6]and the Grothendieck–Messing theory that the above functor is represented by a finite flat morphism π : S K Iw → S K of schemes (of degree q + 1), satisfying that S K Iw is a semi-stable curve over O p .Moreover, since the special fiber T K Iw := S K Iw ⊗ O p κ does not contain genus zero curves as irreduciblecomponents, S K Iw is a stable curve over O p . The subgroup C Iw constructed above induces a morphism ι : Sh(G , h) K Iw → S K Iw ⊗ O p F p of schemes over Sh(G , h) K . By the construction of E ∞ , it is easy tosee that the morphism π : S K Iw ⊗ O p F p → S K ⊗ O p F p = Sh(G , h) K is étale and generically irreducible.Thus, ι is an isomorphism since both sides are finite étale of degree q + 1 and generically irreducibleover Sh(G , h) K . Thus, (2) follows, and we will identify S K Iw ⊗ O p F p with Sh(G , h) K Iw via ι .Let ( π ∗ E ∞ , C Iw ) be the universal object over S K Iw . Denote by T + K Iw (resp. T − K Iw ) the Zariski closure ofthe locus in T K Iw where C Iw is continuous (resp. étale). Then T ± K Iw are union of irreducible componentsand they cover T K Iw . To prove (3), we have to consider full Drinfeld level structures at p . For n (cid:62) S K n be the functor over S K such that for every S K -scheme u : S → S K , the set S K n ( S ) consistsof Drinfeld level structures ϕ : ( p − n / O p ) → Mor S ( S, u ∗ E ∞ [ p n ]) (see [Car86, Section 7.2] for more This is to make sure that the actions of σ on the connected components of Sh(G , h) ⊗ E C and Sh(G , h ) ⊗ F C arecompatible with the map F × p ’ E × q e ( e,e − ) −−−−−−−→ E p . Here, we have to use the fact that constructing smooth (resp. stable) models of smooth projective curves over O p is equivalent to constructing them after base change to O nr p ; see, for example, [DM69, Section 1]. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 83 details). By [Car86, Proposition 7.4], it is represented by a finite flat morphism π n : S K n → S K ofschemes (of degree ( O p / p n )), such that π n ⊗ O p F p is canonically isomorphic to the projectionSh(G , h) K n → Sh(G , h) K . Now we take n = 1, we define a morphism π Iw : S K → S K Iw by sending aDrinfeld level structure ϕ to the subgroup P α ∈ A + [ ϕ ( α )] where A + ⊆ ( p − / O p ) is the line with thesecond coordinate zero. Then π Iw ⊗ O p F p is canonically isomorphic to the projection Sh(G , h) K → Sh(G , h) K Iw . Let T red K be the induced reduced subscheme of S K ⊗ O p κ ac . Then by [Car86, 9.4.1], themorphism T red K → T K ⊗ κ κ ac is finite flat of degree ( q − q ( q + 1). For every line A in ( p − / O p ) ,let T red K ,A be the locus where ϕ | A = 0. Then by [Car86, Proposition 9.4.4], {T red K ,A } A is the set of allirreducible components of T red K . Since GL ( κ ) acts transitively on {T red K ,A } A , each T red K ,A is of degree q ( q − 1) over T K ⊗ κ κ ac . By definition, the image of T red K ,A under π Iw is contained in T + K Iw (resp. T − K Iw ) if and only if A = A + (resp. A = A + ). If A = A + , then π Iw : T red K ,A → T − K Iw ⊗ κ κ ac is étale ofdegree q − A + . Thus,deg( π | T − K Iw ) (cid:62) q . Since deg( π | T + K Iw ) (cid:62) 1, we must have deg( π | T − K Iw ) = q and deg( π | T + K Iw ) = 1, andboth T − K Iw and T + K Iw are irreducible. Thus, (3a) has been verified as a finite flat morphism of degree1 must be an isomorphism, and (3b) also follows. For (3c,3d), put S K ∞ := lim ←− n S K n . Let A + ∞ (resp. A −∞ ) be the subspace of F p with the second (resp. first) coordinate zero. In view of the notation of[Car86, Section 10.3], we have subschemes ( S K ∞ ⊗ κ ac ) A ±∞ of S K ∞ ⊗ κ ac , which map surjectively to T ± K Iw ⊗ κ κ ac under the composite map S K ∞ → S K π Iw −−→ S K Iw , respectively. Note that the endomorphismt $ lifts to S K ∞ by Hecke translation. By [Car86, Proposition 10.3], the morphism t $ ⊗ σ and theHecke translation by ( ) induce the same map on the underlying set of ( S K ∞ ⊗ κ ac ) A + ∞ . Since theHecke translation by ( ) maps ( S K ∞ ⊗ κ ac ) A + ∞ to ( S K ∞ ⊗ κ ac ) A −∞ , we obtain (3c). For (3d), since ( )acts trivially on T K , we know, again by [Car86, Proposition 10.3], that ( π ∞ ⊗ id) ◦ (t $ ⊗ σ ) coincideswith π ∞ ⊗ id on the underlying set of ( S K ∞ ⊗ κ ac ) A + ∞ where π ∞ : S K ∞ → S K is the obvious projection.This implies that t := ( π − ⊗ id) ◦ (t $ ⊗ σ ) ◦ ( π + ⊗ id) − induces the identity map on the underlyingset of T K ⊗ κ κ ac , which has to be purely inseparable. We factors t as the composite map T K ⊗ κ κ ac t −→ ( T K ⊗ κ κ ac ) ( q ) id ⊗ σ −−−→ T K ⊗ κ κ ac . Now t is κ ac -linear, purely inseparable, inducing the identity map on the underlying set, and ofdegree q by (3b,3c), so it has to be the relative q -th Frobenius morphism by [SP, 0CCZ]. Thus, t isthe absolute q -th Frobenius morphism. The proposition is finally proved in the case where F = Q .When F = Q , we can still deduce the proposition to the one for Sh(G , h ) K , which is either: (i)a Shimura curve associated to a division rational quaternion algebra, or (ii) a compactified modularcurve. In both cases, Sh(G , h ) K is already a moduli space. In case (i), the conclusions of theproposition can be found in [Buz97]. In case (ii), the proposition is well-known (see [DR73,KM85]). (cid:3) Corollary D.9. For every rational prime ‘ = p , the action ( σ − ) ∗ of the geometric Frobenius at q on H (Sh(G , h) K ⊗ E C , Q ac ‘ ) satisfies the equation X − t ∗ $ X + q h $ i ∗ = 0 where h $ i : Sh(G , h) K → Sh(G , h) K is the Hecke translation given by ( $ $ ) . Here, we regard t $ as acorrespondence on Sh(G , h) K .Proof. It suffices to show t ∗ $ = ( σ − ) ∗ + q h $ i ∗ ◦ σ ∗ on H (Sh(G , h) K ⊗ E C , Q ac ‘ ). By comparison, itsuffices to prove this identity on T K . However, by Proposition D.8(3), the correspondence t $ on T K decomposes as the sum of T K π + ←−− T + K Iw t + $ −→ T − K Iw π − −−→ T K , T K π − ←−− T − K Iw t − $ −→ T + K Iw π + −−→ T K , Our S K ∞ is Carayol’s M . Here, our σ is the (arithmetic) Frobenius, which is inverse to the one that should appear in [Car86, Proposition 10.3].Such difference is due to the fact that our choice of the Hodge map for Res F/ Q B × is inverse to Carayol’s. where t ± $ is the restriction of t $ to T ± K Iw , respectively. By Proposition D.8(3d), the action of ( π − ◦ t + $ ◦ ( π + ) − ) ∗ coincides with ( σ − ) ∗ on H (Sh(G , h) K ⊗ E C , Q ac ‘ ); and the action of ( π − ◦ t + $ ◦ ( π + ) − ) ∗ coincides with q ( σ − ) ∗ = qσ ∗ on H (Sh(G , h) K ⊗ E C , Q ac ‘ ).For the first part, we have on H ( T K ⊗ κ κ ac , Q ac ‘ ) that π + ∗ ◦ (t + $ ) ∗ ◦ ( π − ) ∗ = π + ∗ ◦ ( π + ) ∗ ◦ (( π + ) − ) ∗ ◦ (t + $ ) ∗ ◦ ( π − ) ∗ = ( π + ∗ ◦ ( π + ) ∗ ) ◦ ( π − ◦ t + $ ◦ ( π + ) − ) ∗ = ( π + ∗ ◦ ( π + ) ∗ ) ◦ ( σ − ) ∗ = ( σ − ) ∗ as π + is an isomorphism by Proposition D.8(3a).For the second part, we have on H ( T K ⊗ κ κ ac , Q ac ‘ ) that π −∗ ◦ (t − $ ) ∗ ◦ ( π + ) ∗ = π −∗ ◦ ((t + $ ) − ) ∗ ◦ h $ i ∗ ◦ ( π + ) ∗ = π −∗ ◦ (t + $ ) ∗ ◦ h $ i ∗ ◦ (( π + ) − ) ∗ = π −∗ ◦ (t + $ ) ∗ ◦ (( π + ) − ) ∗ ◦ h $ i ∗ = ( π − ◦ t + $ ◦ ( π + ) − ) ∗ ◦ h $ i ∗ = qσ ∗ ◦ h $ i ∗ = q h $ i ∗ ◦ σ ∗ . Adding the two parts, we obtain the desired identity. (cid:3) Proof of Theorem D.6. Let π ∞ be an irreducible admissible representation of G( A ∞ ). Denote byΣ( π ∞ ) the set of primes q of E such that q has degree 1 over F , and π ∞ p is an unramified representationof G( Q p ) where p is the underlying rational prime of q . It is clear that Σ( π ∞ ) Chebotarev density 1among all primes of E .We consider (2) first. Let ‘ be a rational prime and let ι ‘ : C ∼ −→ Q ac ‘ be an isomorphism. Let Πbe the standard base change of either π (1 , ∞ ⊗ π ∞ or π (0 , ∞ ⊗ π ∞ . Then Π is cuspidal. Suppose that ρ ι ‘ ( π ∞ ) is not irreducible, then it is the direct sum of two characters µ and µ since it is semisimple.Now using ι , we obtain two automorphic characters Π , Π : E × \ A × E → C × . Corollary D.9 impliesthat Π q is isomorphic to (Π (cid:1) Π ) q for every q ∈ Σ( π ∞ ) that is coprime to ‘ . However, this is notpossible by [Ram, Theorem A]. Therefore, ρ ι ‘ ( π ∞ ) is irreducible; and (2a) follows. For (2b), CorollaryD.9 implies that ρ ι ‘ ( π ∞ ) ∨ ’ ρ ι ‘ ( π ∞ ) c (1) when restricted to q ∈ Σ( π ∞ ) that is coprime to ‘ . Thus,(2b) follows from the Chebotarev density theorem. For (2c), Corollary D.9 already implies (2c) for w = q ∈ Σ( π ∞ ) that is coprime to ‘ . By the Chebotarev density theorem, ρ ι ‘ ( π ∞ ) coincides with therepresentation ρ constructed in [BR93, Section 4] associated to the L -packet Π of π (1 , ∞ ⊗ π ∞ and ι ‘ . Then (2c) follow from [Car12, Theorem 1.1].Now we consider (1). Put ˜ µ := ρ ‘ ( π ∞ ) : E × \ A × E → C × for simplicity. We also put the set { µ , µ } := { µ | | − / E , µ c ˇ χ | | − / E } . Then Corollary D.9 implies that for every q ∈ Σ( π ∞ ) that is coprime to ‘ , wehave ˜ µ q ∈ { µ q , µ q } . We claim that ˜ µ ∈ { µ , µ } . For i = 1 , 2, let Σ i be the set of primes v of E such that ˜ µ v = µ iv , and let δ i be the upper density of Σ i . Then we have δ + δ (cid:62) 1. Without lost ofgenerality, we assume that δ > 0. Then by [Raj00, Theorem 1], there exists a Dirichlet character η of E such that ˜ µ = µ η . If η = 1, then we are done. Otherwise, δ < 1, and then δ > 0. By thesame argument, we have another Dirichlet character η of E such that ˜ µ = µ η . Thus, µ µ − is aDirichlet character, which is not true. Therefore, we must have µ ∈ { µ , µ } . We are left to determinewhich one ˜ µ is.We fix an open compact subgroup K ⊆ G( A ∞ ) such that ( π K ) ∞ = { } . Let A K be the Jacobianof Sh(G , h) K . Let π ∞ be the Gal( C / Q )-orbit of π ∞ . Using Hecke operator, we may find a surjec-tive homomorphism ϕ : A K → B of abelian varieties over E such that φ induces an isomorphism φ ∗ : H ( B, Q ) ∼ −→ H ( A K , Q )[ π ∞ ]. Let B be some simple factor of B over E . Then B is of CM typeby some subfield M ⊆ C , which has to contain M µ (Definition 4.3). If m cusp ( π (1 , ∞ ⊗ π ∞ ) = 1, thenH ( X, C )[ π ∞ ] has Hodge type (1 , µ is the associated CM character of B . In particular,we have ˜ µ τ ( z ) = 1 /z where we have identified C with E ⊗ τ R through the embedding τ , whichimplies that ˜ µ = µ | | − / E . If m cusp ( π (0 , ∞ ⊗ π ∞ ) = 1, then H ( X, C )[ π ∞ ] has Hodge type (0 , Note that our Π satisfies the assumption in [BR93, Lemma 4.2.1]. But Π (as a cuspidal automorphic representationof GL ( A E )) is not slightly regular in the sense of [Shi11]. See [CH13] for more comments on this subtlety. OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 85 ˜ µ c is the associated CM character of B . In particular, we have ˜ µ τ ( z ) = 1 /z , which implies that˜ µ = µ c ˇ χ | | − / E .Theorem D.6 is all proved. (cid:3) References [SP] The Stacks Project Authors, Stacks Project . Available at http://math.columbia.edu/algebraic_geometry/stacks-git/ . ↑ The endoscopic classification of representations , American Mathematical Society Colloquium Pub-lications, vol. 61, American Mathematical Society, Providence, RI, 2013. Orthogonal and symplectic groups.MR3135650 ↑ Smooth compactifications of locally symmetric varieties , 2nded., Cambridge Mathematical Library, Cambridge University Press, Cambridge, 2010. With the collaborationof Peter Scholze. MR2590897 ↑ Height pairing between algebraic cycles , Current trends in arithmetical algebraic geometry(Arcata, Calif., 1985), Contemp. Math., vol. 67, Amer. Math. Soc., Providence, RI, 1987, pp. 1–24. MR902590(89g:11052) ↑ 21, 22, 23, 43, 46[BMM16] N. Bergeron, J. Millson, and C. Mœglin, The Hodge conjecture and arithmetic quotients of complex balls ,Acta Math. (2016), no. 1, 1–125. MR3508219 ↑ 29, 44, 77[BR93] D. Blasius and J. D. Rogawski, Motives for Hilbert modular forms , Invent. Math. (1993), no. 1, 55–87.MR1235020 ↑ Height pairings for algebraic cycles , Proceedings of the Luminy conference on algebraic K -theory(Luminy, 1983), 1984, pp. 119–145, DOI 10.1016/0022-4049(84)90032-X. MR772054 (86h:14015) ↑ L -cohomology of locally symmetric manifolds of finite volume , Duke Math. J. (1983), no. 3, 625–647. MR714821 ↑ Modularity of generating series of divisors onunitary Shimura varieties . arXiv:1702.07812. ↑ 12, 64, 69[BH06] C. J. Bushnell and G. Henniart, The local Langlands conjecture for GL(2), Grundlehren der MathematischenWissenschaften [Fundamental Principles of Mathematical Sciences], vol. 335, Springer-Verlag, Berlin, 2006.MR2234120 ↑ Integral models of certain Shimura curves , Duke Math. J. (1997), no. 3, 591–612. MR1446619 ↑ Local-global compatibility and the action of monodromy on nearby cycles , Duke Math. J. (2012), no. 12, 2311–2413. MR2972460 ↑ 35, 84[Car86] H. Carayol, Sur la mauvaise réduction des courbes de Shimura , Compositio Math. (1986), no. 2, 151–230(French). MR860139 ↑ 81, 82, 83[CH13] G. Chenevier and M. Harris, Construction of automorphic Galois representations, II , Camb. J. Math. (2013), no. 1, 53–73. MR3272052 ↑ The basic locus of unitary Shimura varieties with unramified parahoric level, and special cycles .arXiv:1807.09997. ↑ 53, 57[Del79] P. Deligne, Variétés de Shimura: interprétation modulaire, et techniques de construction de modèles canon-iques , Automorphic forms, representations and L -functions (Proc. Sympos. Pure Math., Oregon State Univ.,Corvallis, Ore., 1977), Proc. Sympos. Pure Math., XXXIII, Amer. Math. Soc., Providence, R.I., 1979, pp. 247–289 (French). MR546620 ↑ The irreducibility of the space of curves of given genus , Inst. Hautes Études Sci.Publ. Math. (1969), 75–109. MR0262240 ↑ Les schémas de modules de courbes elliptiques , Modular functions of one variable,II (Proc. Internat. Summer School, Univ. Antwerp, Antwerp, 1972), Springer, Berlin, 1973, pp. 143–316.Lecture Notes in Math., Vol. 349 (French). MR0337993 ↑ Higher regulators and Hecke L -series of imaginary quadratic fields. I , Invent. Math. (1989),no. 1, 1–69. MR981737 ↑ 25, 26[Fal83] G. Faltings, Endlichkeitssätze für abelsche Varietäten über Zahlkörpern , Invent. Math. (1983), no. 3, 349–366 (German). MR718935 ↑ Intersection theory , 2nd ed., Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. ASeries of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Seriesof Modern Surveys in Mathematics], vol. 2, Springer-Verlag, Berlin, 1998. MR1644323 ↑ 16, 17[GGP12a] W. T. Gan, B. H. Gross, and D. Prasad, Symplectic local root numbers, central critical L values, and restrictionproblems in the representation theory of classical groups , Astérisque (2012), 1–109 (English, with Englishand French summaries). Sur les conjectures de Gross et Prasad. I. MR3202556 ↑ 2, 3, 5, 40, 59 [GGP12b] , Restrictions of representations of classical groups: examples , Astérisque (2012), 111–170 (English,with English and French summaries). Sur les conjectures de Gross et Prasad. I. MR3202557 ↑ The Gross-Prasad conjecture and local theta correspondence , Invent. Math. (2016), no. 3, 705–799, DOI 10.1007/s00222-016-0662-8. MR3573972 ↑ 2, 39[GT16] W. T. Gan and S. Takeda, A proof of the Howe duality conjecture , J. Amer. Math. Soc. (2016), no. 2,473–493. MR3454380 ↑ 60, 76[GR90] S. S. Gelbart and J. D. Rogawski, Exceptional representations and Shimura’s integral for the local unitarygroup U(3), Festschrift in honor of I. I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part I(Ramat Aviv, 1989), Israel Math. Conf. Proc., vol. 2, Weizmann, Jerusalem, 1990, pp. 19–75. MR1159099 ↑ L -functions and Fourier-Jacobi coefficients for the unitary group U(3), Invent. Math. (1991),no. 3, 445–472. MR1117148 ↑ Poles of L -functions and theta liftings for orthogonal groups , J. Inst.Math. Jussieu (2009), no. 4, 693–741. MR2540878 ↑ 59, 60, 61, 62, 64[GG11] Z. Gong and L. Grenié, An inequality for local unitary theta correspondence , Ann. Fac. Sci. Toulouse Math.(6) (2011), no. 1, 167–202 (English, with English and French summaries). MR2830396 ↑ Incoherent definite spaces and Shimura varieties . arXiv:2005.05188. ↑ On the intersection of modular correspondences , Invent. Math. (1993), no. 2,225–245, DOI 10.1007/BF01232433. MR1213101 ↑ Heights and the central critical values of triple product L -functions , CompositioMath. (1992), no. 2, 143–209. MR1145805 ↑ Heegner points and derivatives of L -series , Invent. Math. (1986), no. 2,225–320, DOI 10.1007/BF01388809. MR833192 ↑ L -functions of × unitary groups and factorization of periods of Hilbert modular forms , J. Amer.Math. Soc. (1993), no. 3, 637–719. MR1186960 ↑ Theta dichotomy for unitary groups , J. Amer. Math. Soc. (1996),no. 4, 941–1004. MR1327161 ↑ 63, 75, 76[HLZ19] X. He, C. Li, and Y. Zhu, Fine Deligne-Lusztig varieties and arithmetic fundamental lemmas , Forum Math.Sigma (2019), e47, 55, DOI 10.1017/fms.2019.45. MR4061970 ↑ Complex multiplication cycles and Kudla-Rapoport divisors, II , Amer. J. Math. (2015), no. 3,639–698, DOI 10.1353/ajm.2015.0021. MR3357118 ↑ A regularized Siegel-Weil formula for unitary groups , Math. Z. (2004), no. 2, 241–277.MR2064052 ↑ 62, 63[JR11] H. Jacquet and S. Rallis, On the Gross-Prasad conjecture for unitary groups , On certain L -functions, ClayMath. Proc., vol. 13, Amer. Math. Soc., Providence, RI, 2011, pp. 205–264. MR2767518 ↑ On the non-vanishing of the central value of certain L -functions: unitary groups , J.Eur. Math. Soc. (JEMS) (2020), no. 6, 1759–1783, DOI 10.4171/jems/955. MR4092898 ↑ Endoscopic Classification of Representations: InnerForms of Unitary Groups . arXiv:1409.3731. ↑ Arithmetic moduli of elliptic curves , Annals of Mathematics Studies, vol. 108,Princeton University Press, Princeton, NJ, 1985. MR772569 ↑ Langlands-Shahidi method and poles of automorphic L -functions: application to exterior square L -functions , Canad. J. Math. (1999), no. 4, 835–849. MR1701344 ↑ Integral models for Shimura varieties of abelian type , J. Amer. Math. Soc. (2010), no. 4, 967–1012.MR2669706 ↑ On doubling construction for real unitary dual pairs , Kyushu J. Math. (2007),no. 1, 35–82. MR2317282 ↑ Points on some Shimura varieties over finite fields , J. Amer. Math. Soc. (1992), no. 2,373–444. MR1124982 ↑ Algebraic cycles on Shimura varieties of orthogonal type , Duke Math. J. (1997), no. 1, 39–78,DOI 10.1215/S0012-7094-97-08602-6. MR1427845 ↑ A regularized Siegel-Weil formula: the first term identity , Ann. of Math. (2) (1994), no. 1, 1–80. MR1289491 ↑ Special cycles on unitary Shimura varieties I. Unramified local theory , Invent.Math. (2011), no. 3, 629–682, DOI 10.1007/s00222-010-0298-z. MR2800697 ↑ 8, 53[KR14] , Special cycles on unitary Shimura varieties II: Global theory , J. Reine Angew. Math. (2014),91–157, DOI 10.1515/crelle-2012-0121. MR3281653 ↑ 50, 74[KS97] S. S. Kudla and W. J. Sweet Jr., Degenerate principal series representations for U( n, n ), Israel J. Math. (1997), 253–306. MR1459856 ↑ OURIER–JACOBI CYCLES AND ARITHMETIC RELATIVE TRACE FORMULA 87 [Kün01] K. Künnemann, Height pairings for algebraic cycles on abelian varieties , Ann. Sci. École Norm. Sup. (4) (2001), no. 4, 503–523 (English, with English and French summaries). MR1852008 ↑ Euler products , Yale University Press, New Haven, Conn.-London, 1971. A James K. Whitte-more Lecture in Mathematics given at Yale University, 1967; Yale Mathematical Monographs, 1. MR0419366 ↑ On some degenerate principal series representations of U( n, n ), J. Funct. Anal. (1994), no. 2,305–366. MR1305072 ↑ Remarks on the arithmetic fundamental lemma , Algebra Number Theory (2017), no. 10,2425–2445, DOI 10.2140/ant.2017.11.2425. MR3744362 ↑ Models of curves and finite covers , Compositio Math. (1999), no. 1, 61–102.MR1705977 ↑ Arithmetic theta lifting and L -derivatives for unitary groups, I , Algebra Number Theory (2011),no. 7, 849–921. MR2928563 ↑ 44, 65[Liu11b] , Arithmetic theta lifting and L -derivatives for unitary groups, II , Algebra Number Theory (2011),no. 7, 923–1000. MR2928564 ↑ 65, 81[Liu14] , Relative trace formulae toward Bessel and Fourier-Jacobi periods on unitary groups , ManuscriptaMath. (2014), no. 1-2, 1–69. MR3244725 ↑ 2, 5, 6, 7, 8, 9, 44, 53, 54[Liu16] , Hirzebruch-Zagier cycles and twisted triple product Selmer groups , Invent. Math. (2016), no. 3,693–780. MR3539925 ↑ On the Beilinson–Bloch–Kato conjecture for Rankin–Selbergmotives . arXiv:1912.11942. ↑ L -cohomology of locally symmetric varieties , Compositio Math. (1988), no. 1, 3–20.MR949269 ↑ Relative unitary RZ-spaces and the Arithmetic Fundamental Lemma . arXiv:1611.06520. ↑ 8, 72,74[Mil92] J. S. Milne, The points on a Shimura variety modulo a prime of good reduction , The zeta functions of Picardmodular surfaces, Univ. Montréal, Montreal, QC, 1992, pp. 151–253. MR1155229 ↑ 71, 72[Mœg97] C. Mœglin, Non nullité de certains relêvements par séries théta , J. Lie Theory (1997), no. 2, 201–229(French, with English summary). MR1473165 ↑ 60, 61[MW95] C. Mœglin and J.-L. Waldspurger, Spectral decomposition and Eisenstein series , Cambridge Tracts in Math-ematics, vol. 113, Cambridge University Press, Cambridge, 1995. Une paraphrase de l’Écriture [A paraphraseof Scripture]. MR1361168 ↑ 30, 59[Mok15] C. P. Mok, Endoscopic classification of representations of quasi-split unitary groups , Mem. Amer. Math. Soc. (2015), no. 1108, vi+248, DOI 10.1090/memo/1108. MR3338302 ↑ The standard sign conjecture on algebraic cycles: the case of Shimura varieties .arXiv:1408.0461. ↑ On the motive of an algebraic surface , J. Reine Angew. Math. (1990), 190–204. MR1061525 ↑ 15, 18[MR92] V. K. Murty and D. Ramakrishnan, The Albanese of unitary Shimura varieties , The zeta functions of Picardmodular surfaces, Univ. Montréal, Montreal, QC, 1992, pp. 445–464. MR1155237 ↑ 29, 31[O’S11] P. O’Sullivan, Algebraic cycles on an abelian variety , J. Reine Angew. Math. (2011), 1–81, DOI10.1515/CRELLE.2011.025. MR2795752 ↑ Arithmetical compactification of mixed Shimura varieties , Bonner Mathematische Schriften [BonnMathematical Publications], vol. 209, Universität Bonn, Mathematisches Institut, Bonn, 1990. Dissertation,Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, 1989. MR1128753 ↑ Refinement of strong multiplicity one for automorphic representations of GL( n ), Proc. Amer.Math. Soc. (2000), no. 3, 691–700. MR1707005 ↑ A theorem on GL( n ) à là Tchebotarev . , preprint. ↑ On the arithmetic transfer conjecture for exotic smooth formalmoduli spaces , Duke Math. J. (2017), no. 12, 2183–2336, DOI 10.1215/00127094-2017-0003. MR3694568 ↑ Regular formal moduli spaces and arithmetic transfer conjectures , Math. Ann. (2018), no. 3-4,1079–1175, DOI 10.1007/s00208-017-1526-2. MR3770164 ↑ Arithmetic diagonal cycles on unitary Shimura varieties , Compos. Math. (2020), no. 9, 1745–1824, DOI 10.1112/s0010437x20007289. MR4167594 ↑ 10, 12, 40, 64, 69, 70[RTZ13] M. Rapoport, U. Terstiege, and W. Zhang, On the arithmetic fundamental lemma in the minuscule case ,Compos. Math. (2013), no. 10, 1631–1666. MR3123304 ↑ 56, 57[RZ96] M. Rapoport and Th. Zink, Period spaces for p -divisible groups , Annals of Mathematics Studies, vol. 141,Princeton University Press, Princeton, NJ, 1996. MR1393439 ↑ 9, 74, 75 [Rog90] J. D. Rogawski, Automorphic representations of unitary groups in three variables , Annals of MathematicsStudies, vol. 123, Princeton University Press, Princeton, NJ, 1990. MR1081540 ↑ Analytic expression for the number of points mod p , The zeta functions of Picard modular surfaces,Univ. Montréal, Montreal, QC, 1992, pp. 65–109. MR1155227 ↑ L -cohomology of arithmetic varieties , Ann. of Math. (2) (1990), no. 1, 1–69.MR1059935 ↑ Morphismes universels et variété d’Albanese , Séminaire Claude Chevalley, Vol. 4 (1958–1959),1959, pp. Talk No. 10, 1–22 (French). ↑ On the Ramanujan conjecture and finiteness of poles for certain L -functions , Ann. of Math. (2) (1988), no. 3, 547–584. MR942520 ↑ On the zeta-function of an abelian variety with complex multiplication , Ann. of Math. (2) (1971), 504–533. MR0288089 ↑ Galois representations arising from some compact Shimura varieties , Ann. of Math. (2) (2011), no. 3, 1645–1741. MR2800722 ↑ Hermitian Lie algebras and metaplectic representations. I , Trans. Amer. Math.Soc. (1978), 1–43. MR0486325 ↑ Multiplicity one theorems for Fourier-Jacobi models , Amer. J. Math. (2012), no. 6, 1655–1678,DOI 10.1353/ajm.2012.0044. MR2999291 ↑ Conservation relations for local theta correspondence , J. Amer. Math. Soc. (2015),no. 4, 939–983. MR3369906 ↑ 60, 64, 76[Tan99] V. Tan, Poles of Siegel Eisenstein series on U( n, n ), Canad. J. Math. (1999), no. 1, 164–175. MR1692899 ↑ Ordinariness in good reductions of Shimura varieties of PEL-type , Ann. Sci. École Norm. Sup.(4) (1999), no. 5, 575–618, DOI 10.1016/S0012-9593(01)80001-X (English, with English and French sum-maries). MR1710754 ↑ On Albanese torsors and the elementary obstruction , Math. Ann. (2008), no. 4, 805–838.MR2372739 ↑ Irreducibility of theta lifting for unitary groups , J. Number Theory (2013), no. 10, 3296–3318.MR3071813 ↑ The Gan-Gross-Prasad conjecture for U( n ) × U( n ), Adv. Math. (2014), 1130–1191. MR3228451 ↑ 5, 9, 10[Xue16] , Fourier-Jacobi periods and the central value of Rankin-Selberg L -functions , Israel J. Math. (2016), no. 2, 547–633. MR3505397 ↑ 2, 5, 9, 10, 40[YZZ13] X. Yuan, S.-W. Zhang, and W. Zhang, The Gross-Zagier formula on Shimura curves , Annals of MathematicsStudies, vol. 184, Princeton University Press, Princeton, NJ, 2013. MR3237437 ↑ 3, 65, 80[YZZ] , Triple product L -series and Gross–Kudla–Schoen cycles . , preprint. ↑ The fundamental lemma of Jacquet and Rallis , Duke Math. J. (2011), no. 2, 167–227. With anappendix by Julia Gordon. MR2769216 ↑ On arithmetic fundamental lemmas , Invent. Math. (2012), no. 1, 197–252. MR2897697 ↑ Fourier transform and the global Gan-Gross-Prasad conjecture for unitary groups , Ann. of Math. (2) (2014), no. 3, 971–1049. MR3245011 ↑ Automorphic period and the central value of Rankin-Selberg L-function , J. Amer. Math. Soc. (2014), no. 2, 541–612. MR3164988 ↑ Weil representation and Arithmetic Fundamental Lemma . arXiv:1909.02697. ↑ Department of Mathematics, Yale University, New Haven CT 06511, United States Email address ::