Euler characters and super Jacobi polynomials
aa r X i v : . [ m a t h . R T ] D ec EULER CHARACTERS AND SUPER JACOBIPOLYNOMIALS
A.N. SERGEEV AND A.P. VESELOV
Abstract.
We prove that Euler supercharacters for orthosymplecticLie superalgebras can be obtained as a certain specialization of superJacobi polynomials. A new version of Weyl type formula for super Schurfunctions and specialized super Jacobi polynomials play a key role in theproof.
Contents
1. Introduction 12. Weyl type formulas for super Schur polynomials 33. Euler supercharacters for Lie superalgebra osp (2 m + 1 , n ) 64. Super Jacobi polynomials for k = − osp (2 m, n ) 219. Concluding remarks 2710. Acknowledgements 27References 271. Introduction
The main purpose of this paper is to develop further the link between thetheory of the deformed Calogero-Moser systems and representation theoryof Lie superalgebras [1, 2]. Recall that in the BC ( m, n ) case the deformedCalogero-Moser systems depend on 3 parameters k, p, q. For generic valuesof these parameters they have polynomial eigenfunctions S J λ ( u, v ; k, p, q )called super Jacobi polynomials [3]. The special case k = p = − , q = 0corresponds to the orthosymplectic Lie superalgebra osp (2 m + 1 , n ).It turns out that this case is singular in the sense that the correspondinglimit does not always exist. However if we consider first the limit when k → − p, q and then let ( p, q ) → ( − ,
0) then this limit doesalways exist and gives what we call specialized super Jacobi polynomials S J λ ( u, v ; − , − , . A natural question is what do they correspond to inthe representation theory of orthosymplectic Lie superalgebras. We show hat the answer is given by the so-called Euler characters studied by Penkovand Serganova [4, 5].There is a classical construction due to Borel, Weil and Bott of the ir-reducible representations of the complex semisimple Lie groups G in termsof the cohomology of the holomorphic line bundles over the correspondingflag varieties G/B (see e.g. [6], section 23.3). Such line bundles L λ are de-termined by the weight λ ∈ h ∗ , where h is Cartan subalgebra of Lie algebraof G and B is Borel subgroup of G. The cohomology groups H i ( G/B, L λ )are finite-dimensional and have natural actions of G on them. By Kodairavanishing theorem all of them are zero except one depending on the Weylchamber the weight λ belongs to (see details in [6, 7]). In particular, for adominant weight λ the space of sections H ( G/B, L λ ) gives the irreduciblerepresentation with highest weight λ. In the Lie supergroup case in general there is no vanishing property, sothis construction does not work [4]. The idea is to consider the virtualrepresentation given by the Euler characteristic E λ = X i ( − i H i ( G/B, O λ )for certain sheaf cohomology groups (see [5]). For the generic (typical)highest weights λ this leads to the Kac character formula [8].One can generalise this construction for any parabolic subgroup P of G in a natural way. In the orthosymplectic case G = OSP (2 m + 1 , n ) thereis a natural choice of P with the reductive part GL ( m, n ) . The correspond-ing supercharacter E λ (called Euler supercharacter ) can be given by thegeneral explicit formula due to Serganova [5]. Our main result is that E λ up to a constant factor coincides with the specialized super Jacobi polyno-mials S J λ ( u, v ; − , − , . A similar result holds for the Lie superalgebra osp (2 m, n ) and super Jacobi polynomials S J λ ( u, v ; − , , . The proof is based on a new formula for super Schur polynomials and thesuper version [3] of Okounkov’s formula for Jacobi polynomials [9]. We provealso the Pieri and Jacobi–Trudy formulas for the corresponding specializedsuper Jacobi polynomials and Euler supercharacters.It turns out that we can simplify the relations with super Jacobi poly-nomials if we choose a different Borel subalgebra and the correspondingparabolic subalgebras following recent work by Gruson and Serganova [10].In that case the Euler supercharaters coincide with specialised super Jacobipolynomials without non-trivial factor (see the last section for the details).This shows that the super Jacobi polynomials can be considered as a nat-ural deformation of the Euler supercharacters and gives one more evidence ofa close relationship between quantum integrable systems and representationtheory. . Weyl type formulas for super Schur polynomials
We start with the new formula for super Schur polynomials, which willplay an important role in this work.Let H ( m, n ) be the set of partitions with λ m +1 ≤ n, which means thatthe corresponding Young diagram belongs to the fat ( m, n ) -hook . Let λ besuch a partition and let d = m − n be the superdimension . Introduce thefollowing quantities i ( λ ) = max { i | λ i + d − i ≥ , ≤ i ≤ m } , (1) j ( λ ) = max { j | λ ′ j − d − j ≥ , ≤ j ≤ n } . (2)If all λ i + d − i < i ( λ ) = 0 (and similarly for j ( λ )). Itis easy to verify that in all cases m − i ( λ ) = n − j ( λ ), so i ( λ ) − j ( λ ) = d. We should mention that Moens and van der Jeugt introduced a similarquantity k ( λ ) , which they called ( m, n )-index of λ (see Definition 2.2 in[19]). They were motivated by Kac-Wakimoto formula [11]. It is related toour i ( λ ) by i ( λ ) = k ( λ ) − . Moens and van der Jeugt used this quantityto write down a new determinantal formula for super Schur polynomialsdifferent from Sergeev-Pragacz formula [12].Our formula (7) below is another new formula of Weyl type, which gen-eralizes Sergeev-Pragacz formula. Let us denote by π λ the set of pairs ( i, j )such that i ≤ i ( λ ) or j ≤ j ( λ ) and fix a partition ν such that λ ∩ Π m,n ⊆ ν ⊆ π λ , where Π m,n is the rectangle of the size m × n . When ν = π λ the formula(7) coincides with Serganova’s formula (11) for a special choice of parabolicsubalgebra depending on the weight (although that was not the way wecame to this).Introduce the following quantities by l i = λ i + m − ν i − i, ≤ i ≤ i ( λ ) , l i = m − i, i ( λ ) < i ≤ m, (3) k j = λ ′ j + n − ν ′ j − j, ≤ j ≤ j ( λ ) , k j = n − j, j ( λ ) < j ≤ n. (4)Now we can formulate the main result of this section. Recall that superSchur polynomial is the supercharacter of the polynomial representation M = V λ of gl ( m, n ) determined by a Young diagram λ from the fat hook H ( m, n ). It can be given by the following Jacobi–Trudy formula (see [13]): SP λ ( x, y ) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) h λ h λ +1 . . . h λ + l − h λ − h λ . . . h λ + l − ... ... . . . ... h λ l − l +1 h λ l − l +2 . . . h λ l (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (5)where l = l ( λ ) is the number of non-zero parts in the partition λ , h k = h k ( x, y ) are determined by Q nj =1 (1 − ty j ) Q mi =1 (1 − tx i ) = ∞ X a =0 h a ( x, y ) t a (6) nd h a = 0 if a <
0. One can check that the highest coefficient of SP λ ( x, y )is equal to ( − b , where b = P j>m λ j . This explains the appearance of thissign below (see also [14]).
Theorem 2.1.
The super Schur polynomial SP λ ( x , . . . , x m , y , . . . , y n ) canbe expressed by the following Weyl type formula for any choice of partition ν such that λ ∩ Π m,n ⊆ ν ⊆ π λ : SP λ ( x , . . . , x m , y , . . . , y n ) =( − b X w ∈ S m × S n w Y ( i,j ) ∈ ν ( x i − y j ) x l . . . x l m m y k . . . y k n n ∆( x )∆( y ) , (7) where ∆( x ) = Q mi
For any function f ( x, y ) define the following alternation operations { f ( x, y ) } = X w ∈ S m × S n ε ( w ) w ( f ( x, y ))and { f ( x, y ) } x = X w ∈ S m ε ( w ) w ( f ( x, y )) , where S m and S n permute x i and y j respectively. Introduce also the nota-tions x ρ m = x m − x m − . . . x m , y ρ n = y n − y n − . . . y n . First let us prove the following equality { h a ( x, y ) x ρ m y ρ n } = n Y j =1 ( x − y j ) x a + d − x m − . . . x m y n − . . . y n , (8)where a is an integer such that a + d − ≥ . Indeed, we have from the usualWeyl formula for a ≥ { h a ( x ) x m − x m − . . . x m } x = { x a + m − x m − . . . x m } x This is true also for all a ≥ − m because the left hand side for negative a is zero by definition. From (6) we have h k ( x, y ) = n X j =0 ( − j h k − j ( x ) e j ( y ) , where h k and e j are complete symmetric and elementary symmetric poly-nomials respectively. Note now that if a + d − ≥ ≤ j ≤ n we have a − j ≥ − m . Therefore in that case { h a ( x, y ) x m − . . . x m } x = n X j =0 { h a − j ( x ) x m − . . . x m } x ( − j e j ( y ) = X j =0 { x a − j x ρ m } x ( − j e j ( y ) = { n X j =0 x n − j ( − j e j ( y ) x a + d − x m − . . . x m } x = { n Y j =1 ( x − y j ) x a + d − x m − . . . x m } x . This implies the formula (8).We prove now the theorem by induction. For this we need the following
Lemma 2.2. If λ + d − ≥ we have the following equality { SP λ ( x, y ) x ρ m y ρ n } = n Y j =1 ( x − y j ) x λ + d − SP ˆ λ (ˆ x, y ) x m − . . . x m y ρ n (9) where ˆ x = ( x , . . . , x m ) , ˆ λ = ( λ , λ , . . . ) . Proof.
To prove it we use the Jacobi-Trudy formula (5). Let λ + d − ≥ , then we have (cid:8) SP λ ( x , . . . , x m , y , . . . , y n ) x m − . . . x m y n − . . . y n (cid:9) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) h λ h λ +1 . . . h λ + l − h λ − h λ . . . h λ + l − ... ... . . . ... h λ l − l +1 h λ l − l +2 . . . h λ l (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) x m − . . . x m y n − . . . y n = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) (cid:8) h λ x m − . . . y n (cid:9) (cid:8) h λ +1 x m − . . . y n (cid:9) . . . (cid:8) h λ + l − x m − . . . y n (cid:9) h λ − h λ . . . h λ + l − ... ... . . . ... h λ l − l +1 h λ l − l +2 . . . h λ l (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = n Y j =1 ( x − y j ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) x λ + d − . . . y n x λ + d . . . y n . . . x λ + d + l − . . . y n h λ − h λ . . . h λ + l − ... ... . . . ... h λ l − l +1 h λ l − l +2 . . . h λ l (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) . Now multiplying every column except the last one by x and subtracting itfrom the next column taking into account the equality h a − ( x, y ) − x h a ( x, y ) = h a (ˆ x, y )we get n Y j =1 ( x − y j ) x λ + d − (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ˆ h λ ˆ h λ +1 . . . ˆ h λ + l − ... ... . . . ...ˆ h λ l − l +2 ˆ h λ l − l +3 . . . ˆ h λ l (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , where ˆ h k = h k (ˆ x, y ) and ˆ x = ( x , . . . , x m ) as before. This implies theformula (9). (cid:3) emma 2.3. Let λ < n , then we have the following equality { SP λ ( x, y , . . . , y n ) x ρ m y ρ n } = { SP λ ( x, y , . . . , y n − ) x ρ m y ρ n } . Proof.
We have (see formulas (5.9) in [18] and (25) in [14]) SP λ ( x, y ) = X µ ⊆ λ ( − | µ | S λ/µ ( x ) S µ ′ ( y )So, if λ < n then µ ′ n = 0 and lemma follows. (cid:3) Now we finish the proof by induction in m + n. Note first that the case n = 0 follows from the classical Weyl’s formula, while in the case m = 0 wehave also the sign ( − λ m +1 + λ m +2 + ... determined as in the formula (25) in[14]. Let us assume now that m > . Rewrite formula (7) in the form { SP λ ( x, y ) x ρ m y ρ n } = ( − b Y ( i,j ) ∈ ν ( x i − y j ) x l . . . x l m m y k . . . y k n n . (10)If ν < n (and hence λ < n ) by induction assumption we have { SP λ ( x, y , . . . , y n − ) x ρ m y ρ n − } =( − b Y ( i,j ) ∈ ν ( x i − y j ) x l . . . x l m m y k − . . . y k n − − n − . Multiplying both sides by the product y . . . y n and using lemma 2.3, weprove the theorem in this case.If ν = n then the box (1 , n ) belongs to π λ and therefore by definitioneither i ( λ ) > j ( λ ) = n. If i ( λ ) = 0 then j ( λ ) = n − m < n since m > . Thus, i ( λ ) ≥ , which implies that λ + d − ≥ . This means that we canapply lemma 2.2. By induction we have (cid:8) SP λ ( x , . . . , x m ) x m − . . . x m y ρ n (cid:9) =( − b Y ( i,j ) ∈ ˆ ν ( x i − y j ) x l . . . x l m m y k . . . y k n n , where ˆ ν is the partition ν without the first row. Now the theorem followsfrom lemma 2.2. (cid:3) Euler supercharacters for Lie superalgebra osp (2 m + 1 , n )In this section we consider the case of orthosymplectic Lie superalgebra osp (2 m + 1 , n ) , the case of osp (2 m, n ) is considered in section 7.Recall the description of the root system of Lie superalgebra g = osp (2 m +1 , n ). We have g = g ⊕ g , where g = so (2 m +1) ⊕ sp (2 n ) and g = V ⊗ V where V and V are the identical representations of so (2 m + 1) and sp (2 n ) espectively. Let ± ε , . . . , ± ε m , ± δ , . . . , ± δ n be the non-zero weights of theidentical representation of g . The root system of osp (2 m + 1 , n ) consists of R = {± ε i ± ε j , ± ε i , ± δ p ± δ q , ± δ p , i = j, ≤ i, j ≤ m , p = q, ≤ p, q ≤ n } ,R = {± ε i ± δ p , ± δ p } , R iso = {± ε i ± δ p } , where R , R and R iso are even, odd and isotropic parts respectively. Theinvariant bilinear form is given by( ε i , ε i ) = 1 , ( ε i , ε j ) = 0 , i = j, ( δ p , δ p ) = − , ( δ p , δ q ) = 0 , p = q, ( ε i , δ p ) = 0 . The Weyl group W = ( S m ⋉ Z m ) × ( S n ⋉ Z n ) acts on the weights by sep-arately permuting ε i , j = 1 , . . . , m and δ p , p = 1 , . . . , n and changing theirsigns. A distinguished system of simple roots can be chosen as B = { δ − δ , . . . , δ n − − δ n , δ n − ε , ε − ε , . . . , ε m − − ε m , ε m } . Introduce the variables x i = e ε i , x / i = e ε i / , u i = x i + x − i , i = 1 , . . . , m and y p = e δ p , v p = y p + y − p , p = 1 , . . . , n .For any parabolic subalgebra p ⊂ g and any finite-dimensional represen-tation M of p by a superversion of Borel-Weil-Bott construction one candefine the corresponding Euler supercharacter E p ( M ). According to thegeneral formula due to Serganova [5] E p ( M ) = X w ∈ W w D e ρ schM Q α ∈ R p ∩ R +1 (1 − e − α ) ! (11)with D = Q α ∈ R +1 ( e α/ − e − α/ ) Q α ∈ R +0 ( e α/ − e − α/ ) . Here ρ is the half-sum of the even positive roots minus the half-sum of oddpositive roots, R p is the set of roots α such that g ± α ⊂ p (see formula (3.1)in [5]). Note that we use here the supercharacter rather than the characterused by Serganova. This leads simply to the change of signs in some places.Consider now the parabolic subalgebra p with R p = { ε i − ε j , δ p − δ q , ± ( ε i − δ p ) } . The algebra p is isomorphic to the sum of the Lie superalgebra gl ( m, n ) andsome nilpotent Lie superalgebra. In that case Serganova’s formula (11) hasthe form E ( M ) = X w ∈ W w Y ( i,j ) ∈ Π m,n (1 − x − i y − j ) x m − . . . x m y n − . . . y n schM ∆( u )∆( v ) Q mi =1 ( x i − x − i ) Q nj =1 ( y j + y − j ) , (12)where Π m,n is the rectangle of the size m × n , ∆( u ) = Q mi For the trivial even representation M we have E ( M ) = 2 min( m,n ) . (13) Proof. It is well-known (see e.g. [18], Ch. 1, formula (4.3’)) that Y i,j (1 − x − i y − j ) = X ( − | λ | S λ ( x − , . . . , x − m ) S λ ′ ( y − , . . . , y − n ) , where S λ is the Schur polynomial and the sum is over all the Young diagrams λ , which are contained in the ( m × n ) rectangle, λ ′ is the diagram transposedto λ. Using the Weyl formula for Schur polynomials we can replace in theformula (12) the product Q i,j (1 − x − i y − j ) by the sum X ( − | λ | x − λ m + m − / . . . x − λ +1 / m y − λ ′ n + n − / . . . y − λ ′ +1 / n . One can check that the only non-vanishing terms correspond to the symmet-ric Young diagrams λ = λ ′ . Now the proposition follows from the fact thatthe number of the symmetric diagrams contained in the ( m × n ) rectangleis equal to 2 min( m,n ) , which can be easily proved by induction. (cid:3) Consider now the polynomial representations M = V λ of gl ( m, n ) deter-mined by a Young diagram λ ∈ H ( m, n ) with the supercharacter given bythe super Schur polynomial: sch V λ = SP λ ( x , . . . , x m , y , . . . , y n ) . The next step is to rewrite the formula for the Euler supercharacter E λ = E ( V λ ) , (14)using the formula (7) for the super Schur polynomials in the case when ν = π λ . In this case k i , l j are defined by l i = λ i + d − i, ≤ i ≤ i ( λ ) , l i = m − i, i ( λ ) < i ≤ m, (15) k j = λ ′ j − d − j, ≤ j ≤ j ( λ ) , k j = n − j, j ( λ ) < j ≤ n, (16)where d = m − n. Introduce the following polynomials (which are particular cases of classicalJacobi polynomials, see the next section) ϕ a ( z ) = w a +1 / − w − a − / w / − w − / , ψ a ( z ) = w a +1 / + w − a − / w / + w − / , (17)where z = w + w − . Define also Π λ ( u, v ) asΠ λ ( u, v ) = Y ( i,j ) ∈ π λ ( u i − v j ) . heorem 3.2. The Euler supercharacters can be given by the followingformula E λ ( u, v ) = C ( λ ) X w ∈ S m × S n w (cid:20) Π λ ( u, v ) ϕ l ( u ) . . . ϕ l m ( u m ) ψ k ( v ) . . . ψ k n ( v n )∆( u )∆( v ) (cid:21) , (18) where C ( λ ) = ( − b m − i ( λ ) = ( − b n − j ( λ ) , b = P j>m λ j and i ( λ ) , j ( λ ) , l i , k j are defined by (1), (2),(15), (16).Proof. According to Theorem 2.1 for M = V λ the corresponding superchar-acter schM = SP λ can be given by (7), or, equivalently, by (10). Substitut-ing this into Serganova’s formula (12) we get E ( M ) = ( − b X w ∈ W w Y ( i,j ) ∈ Π m,n (1 − x − i y − j ) Y ( i,j ) ∈ π λ ( x i − y j ) x l + y k + ∆( u )∆( v ) Q mi =1 ( x i − x − i ) Q nj =1 ( y j + y − j ) , where we use the notations x l + = x l + . . . x l m + m , y k + = y k + . . . y k n + n . Since Y ( i,j ) ∈ π λ (1 − x − i y − j ) Y ( i,j ) ∈ π λ ( x i − y j ) = Y ( i,j ) ∈ π λ ( u i − v j ) = Π λ ( u, v ) , we have E ( M ) = ( − b X w ∈ W w Π λ ( u, v ) Y ( i,j ) ∈ Π m,n \ π λ (1 − x − i y − j ) x l + y k + ∆( u )∆( v ) Q mi =1 ( x i − x − i ) Q nj =1 ( y j + y − j ) . Now proposition 3.1 and Weyl’s formula for the BC n root system (which isthe root system of the Lie superalgebra osp (1 , n )) allow us to replace here Q ( i,j ) ∈ Π m,n \ π λ (1 − x − i y − j ) by 2 m − i ( λ ) = 2 n − j ( λ ) to come to E ( M ) = C ( λ ) X w ∈ W w Π λ ( u, v ) x l + y k + ∆( u )∆( v ) m Y i =1 ( x i − x − i ) n Y j =1 ( y j + y − j ) . (19)Summing now over the subgroup Z m × Z n ⊂ W and using (17) we have theclaim. (cid:3) We will use this formula now to show the relation with super Jacobipolynomials. . Super Jacobi polynomials for k = − k = − . Let us introduce the followingpolynomials f l ( z, p, q ), which are certain normalised versions of the classicalJacobi polynomials P α,βl ( z ) with α = − p − q − , β = q − : f l ( z, p, q ) = l X i =0 C l,i ( z − i , where C l,l = 1 and C l,i = 4 l − i ( i + 1) . . . ( l − l ( l − i )! ( i + 1 − p − q − / . . . ( l − p − q − / l + i − p − q ) . . . (2 l − − p − q )for i < l. Introduce also g k ( w, p, q ) = f k ( w, − p, − − q ) . Note that the polynomials ϕ ( z ) , ψ ( z ) from the previous section are the par-ticular cases: ϕ l ( z ) = f l ( z, − , , ψ k ( z ) = g k ( z, − , 0) = f k ( z, , − . Later we drop the parameters for brevity, writing simply f k ( z ) , g k ( z ) . Let λ ∈ H ( m, n ) be a partition from the fat hook and π λ , l i , k j be thesame as in the previous section (see formulas (15),(16) above). Theorem 4.1. The super Jacobi polynomials for special value of parameter k = − can be given by the following formula SJ λ ( u, v, − , p, q ) =( − b X w ∈ S m × S n w (cid:20) Π λ ( u, v )∆( u )∆( v ) f l ( u ) . . . f l m ( u m ) g k ( v ) . . . g k n ( v n ) (cid:21) , (20) where as before b = P j>m λ j and Π λ ( u, v ) = Q ( i,j ) ∈ π λ ( u i − v j ) . We prove this first in the particular case when λ contains the m × n rectangle, i.e. λ m ≥ n. The corresponding formula for super Jacobi polynomials can be consideredas a natural analogue of Berele-Regev factorisation formula for super Schurpolynomials [15]. For such a diagram λ one can consider its sub-diagram µ, which is the diagram λ without first n columns. Define w i ( λ ) = µ i , i = 1 , . . . , m, z j ( λ ) = λ ′ j , j = 1 , . . . , n. (21)In other words, w i is the length of i -th row of µ and z j is the length of j -thcolumn of λ ∈ H m,n . he super Jacobi polynomials [3] in the special case k = − p, q in terms of super Schur polynomials SP λ by Okounkov’sformula SJ λ ( u, v, − , p, q ) = X ˜ λ ⊆ λ K λ, ˜ λ SP ˜ λ (ˆ u, ˆ v ) (22)where u = ( u , . . . , u m ) , v = ( v , . . . , v n ) , ˆ u = ( u − , . . . , u m − , ˆ v =( v − , . . . , v n − 2) and K λ, ˜ λ = 4 | λ |−| ˜ λ | C λ ( d ) C λ ( d − p − q − ) C λ ( d ) C λ ( d − p − q − ) I ˜ λ ( w ( λ ) , z ( λ ) , − , h ) C − λ (1) C + λ (2 h − . (23)Here d = m − n is the superdimension, h = d − p − q,C + λ ( x ) = Y ( ij ) ∈ λ (cid:0) λ i + j − ( λ ′ j + i ) + x (cid:1) , (24) C − λ ( x ) = Y ( ij ) ∈ λ (cid:0) λ i − j + ( λ ′ j − i ) + x (cid:1) , (25) C λ ( x ) = Y ( ij ) ∈ λ ( j − − ( i − 1) + x ) , (26) I λ ( w, z, − , h ) is the specialisation of the deformed interpolation BC poly-nomial I λ ( w, z, k, h ) (see Proposition 6.3 in [3]), w ( λ ) and z ( λ ) are definedby (21) above. We should note that we are using here more convenientvariables u i = x i + x − i , v j = y j + y − j , rather than u i = ( x i + x − i − , v j = ( y j + y − j − 2) used in [3]. Theorem 4.2. Let λ ∈ H ( m, n ) contains the m × n rectangle. Then thesuper Jacobi polynomials SJ λ ( u, v, − , p, q ) can be expressed in terms of theusual Jacobi polynomials as SJ λ ( u, v, − , p, q ) = ( − | ν | m Y i =1 n Y j =1 ( u i − v j ) J µ ( u, − , p, q ) J ν ( v, − , − p, − − q ) , (27) where µ i = λ i − n, i = 1 , . . . , m, ν j = λ ′ j − m, j = 1 , . . . , n. (28) Proof. To prove this we need the following factorisation formula for thedeformed interpolation BC polynomials. We should mention that in thespecial case k = − emma 4.3. If λ ∈ H ( m, n ) contains the m × n rectangle, then we havethe following formula for deformed interpolation BC polynomials I λ ( w, z, − , h ) = ( − | ν | m Y i =1 n Y j =1 (cid:2) ( w i + n + h − i ) − ( z j − h − j + 1) (cid:3) × I µ ( w , . . . , w m , − , h + n ) I ν ( z − m, . . . , z n − m, − , − h + m ) , where I µ ( w, k, h ) is the usual interpolation BC polynomial and µ, ν are thesame as in the theorem.Proof. Let us denote by ˆ I λ the right hand side of the previous equality.According to proposition 6.3 from [3] it is enough to prove that ˆ I λ satisfythe following properties:1) ˆ I λ is a polynomial in variables ( w i + h + n − i ) , i = 1 , . . . , m and( z j − h + 1 − j ) , j = 1 , . . . , n ;2) the degree of this polynomial is 2 | λ | ;3) ˆ I λ ( w (˜ λ ) , z (˜ λ )) = 0 if ˜ λ ∈ H ( m, n ) and λ * ˜ λ ;4) ˆ I λ ( w ( λ ) , z ( λ )) = Q ( i,j ) ∈ λ (1 + λ i − j + λ ′ j − i )(2 h − λ i + j − λ ′ j − i ) . First two statements are obvious from the explicit form of ˆ I λ . The fourthstatement can be checked directly. Let us prove the third property.Let us suppose that λ * ˜ λ . Consider two possible cases depending onwhether ˜ λ contains the m × n rectangle or not. In the first case we have µ * ˜ µ or ν * ˜ ν . Therefore by definition of the interpolation BC polynomials[9] I µ (˜ µ, − , h + n ) I ν (˜ ν, − , − h + m ) = 0 . In the second case consider the box ( i, n ) , ≤ i ≤ m such that ( i, n ) / ∈ ˜ λ ,but ( i − , n ) ∈ ˜ λ (if i=1, we require only first condition). Note that fromour assumptions on λ it follows that such a box does exist. Then we have( w i + n + h − i )+( z n − h − n +1) = ˜ µ i + n − i +˜ ν n − n +1 = 0+ n − i +( i − − n +1 =0, which means that ˆ I λ ( w (˜ λ ) , z (˜ λ )) = 0. Lemma is proved. (cid:3) Now we can prove the theorem 4.2. First we note that in Okounkov’s for-mula we can always assume that the diagram ˜ λ contains the m × n rectangle.Indeed, otherwise K λ, ˜ λ = 0 since in that case one can easily see that C λ ( d ) C λ ( d ) = 0 . Therefore by lemma 4.3 I ˜ λ ( w ( λ ) , z ( λ ) , − , h ) = ( − | ˜ ν | m Y i =1 n Y j =1 ( λ i + λ ′ j − i − j +1)( λ i − λ ′ j + j − i +2 h − × I ˜ µ ( µ, − , h + n ) I ˜ ν ( ν, − , − h + m ) , here ˜ µ, ˜ ν are defined by ˜ λ as in (28). We rewrite now the coefficient K λ, ˜ λ in terms of the diagrams µ and ν. We have C − λ (1) = m Y i =1 n Y j =1 ( λ i + λ ′ j − i − j + 1) C − µ (1) C − ν ′ (1) ,C + λ (2 h − 1) = m Y i =1 n Y j =1 ( λ i − λ ′ j + j − i +2 h − C + µ (2( h + n ) − C + ν ′ (2( h − m ) − ,C λ ( m − n ) C λ ( m − n ) = C µ ( m ) C µ ( m ) C ν ′ ( − n ) C ν ′ ( − n ) ,C λ ( m − n − p − q − ) C λ ( m − n − p − q − ) = C µ ( m − p − q − ) C µ ( m − p − q − ) C ν ′ ( − n − p − q − ) C ν ′ ( − n − p − q − ) . Now we use the Berele-Regev factorisation formula [15] for super Schurpolynomials SP ˜ λ ( u, v, − 1) = ( − | ˜ ν | m Y i =1 n Y j =1 ( u i − v j ) P ˜ µ ( u ) P ˜ ν ( v ) , where P λ ( u ) are usual Schur polynomials. Comparing (22) and (27) andusing Okounkov’s formula for usual Jacobi polynomials [9] we see that inorder to prove the theorem we need to show that J ν ( v , . . . , v n , − , − p, − − q ) = ( − | ν | X ˜ ν ⊆ ν D ν, ˜ ν P ˜ ν (ˆ v ) , where D ν, ˜ ν = 4 | ν |−| ˜ ν | C ν ′ ( − n ) C ν ′ ( − n ) C ν ′ ( − n − p − q − ) C ν ′ ( − n − p − q − ) I ˜ ν ( ν, − , − h + m ) C − ν ′ (1) C + ν ′ (2( h − m ) − . Since C ν ( x ) = ( − | ν | C ν ′ ( − x ) , C − ν ( x ) = C − ν ′ ( − x ) , C + ν ( x ) = ( − | λ | C + λ ( − x ) , we have( − | ν | D ν, ˜ ν = 4 | ν |−| ˜ ν | C ν ( n ) C ν ( n ) C ν ( n + p + q + ) C ν ( n + p + q + ) I ˜ ν ( ν, − , − h + m ) C − ν (1) C + ν (1 − h − m ))and the proof now follows from Okounkov’s formula. (cid:3) Let’s come to the proof of the main theorem 4.1. We need the followingresult. Denote the right hand side of the formula (20) as RHS. Proposition 4.4. The right hand side of the formula (20) can be writtenin terms of super Schur functions as RHS = X µ ⊆ λ C µ ( d, p, q ) SP µ ( u − , . . . , u m − , v − , . . . , v n − , where the coefficients C µ ( d, p, q ) are some rational functions of µ, d, p, q. o prove this let us denote r = i ( λ ) , s = j ( λ ) and expand every polyno-mial f l i , ≤ i ≤ r in terms of the powers of u i − g k j , ≤ j ≤ s in terms of the powers of v j − 2. Then RHS is the sum ofterms X w ∈ S m × S n w " Π λ ( u, v ) ( u − ˜ l . . . ( u r − ˜ l r ( v − ˜ k . . . ( v r − ˜ l s F λ G λ ∆( u )∆( v ) with some constant factors depending on d, p, q, where F λ = ( u r +1 − m − r − . . . ( u m − , G λ = ( v s +1 − m − s − . . . ( v n − and T = (˜ l , . . . , ˜ l r , ˜ k , . . . ˜ k s ) satisfy the conditions 0 ≤ ˜ l i ≤ l i , ≤ i ≤ i ( λ )and 0 ≤ ˜ k j ≤ k j , ≤ j ≤ j ( λ ). Since π λ is symmetric with respect to u , . . . , u r and v , . . . , v s we may assume that both ˜ l i and ˜ k j are pairwisedifferent. Now the proposition follows from theorem 2.1 because of thefollowing Lemma 4.5. Let a > · · · > a n sequence of nonnegative integers and b , . . . , b n sequence of pairwise different nonnegative integers such that a i ≥ b i , i = 1 , . . . , n . Let us reorder sequence { b i } in decreasing order b ′ > b ′ > · · · > b ′ n . Then for any ≤ i ≤ n we have b ′ i ≤ a i .Proof. We will prove lemma by induction on n . The case n = 1 is obvi-ous. Assume that lemma is true for some n and consider the sequences a > · · · > a n > a n +1 , b , . . . , b n , b n +1 , which satisfy the conditions of thelemma and the corresponding sequence b ′ > b ′ > · · · > b ′ n > b ′ n +1 . Let c = min { b , . . . , b n +1 } . If c = b n +1 . We can apply inductive assumption to a , . . . , a n , b , . . . , b n . If c = b i = b n +1 then we can apply inductive assump-tion to a , . . . , a n and b , . . . , b i − , b n +1 , b i +1 , . . . , b n . Lemma is proved. (cid:3) Let us finish the proof of the theorem 4.1. The proof is by induction on m + n. First note that if λ contains the m × n rectangle then the theoremfollows from Theorem 4.2 and Proposition 7.1 from Okounkov-Olshanski [9].In particular, this is always true when either m or n is zero.Assume now that both m and n are positive and λ does not contain the m × n rectangle. Make the substitution u m = v n = t in both sides ofthe formula (20). The result is independent on t and reduces to the casewith smaller number of variables u , . . . , u m − and v , . . . v n − : for the lefthand side this follows from Okounkov’s formula, for the right hand side fromproposition 4.4. Thus by induction we have that the difference between theright hand side and left hand side of the formula (20) is divisible by theproduct Q mi =1 Q nj =1 ( u i − v j ) . Note that both sides are linear combinations ofthe super Schur polynomials SP ˜ λ with ˜ λ ⊆ λ , which follows from Okounkov’sformula and proposition 4.4. However the ideal generated by Q mi =1 Q nj =1 ( u i − v j ) is known to be linearly spanned by the super Schur polynomials S P µ with µ containing the m × n rectangle. This means that the difference is actually ero since by assumption λ does not contain it. This completes the proof ofTheorem 4.1.As a corollary we have one of our main results. Theorem 4.6. The limit of the super Jacobi polynomials SJ λ ( u, v, − , p, q ) as ( p, q ) → ( − , is well defined and coincides up to a constant factor withthe Euler supercharacter for the Lie superalgebra osp (2 m + 1 , n ) . : S J λ ( u, v ; − , − , 0) = 2 i ( λ ) − m E λ ( u, v ) . (29)The proof follows from comparison of formulas (18) and (20).5. Pieri formula One of the problems with the special value k = − k = − k bytaking the limit k → − 1. However the corresponding calculations are quitelong, so we will do this in a different way.Let us introduce some notations. Let P m,n be the set of all partitions λ from the fat ( m, n ) hook H ( m, n ). Let us call a box (cid:3) = ( i, j ) of Youngdiagram λ special if i − j = d, where d = m − n as before is superdimension. We will write µ ∼ λ if Youngdiagram µ can obtained from λ by removing or adding one box.Introduce the following functions: a d ( µ, λ ) = ( , µ = λ \ (cid:3) for special (cid:3) , otherwise , (30) b d ( λ ) = − , if λ has a removable special box1 , if there is a special box which can be added to λ , otherwise. (31) Theorem 5.1. Let S J λ ( u, v ; − , − , 0) = SJ λ ( u, v ) be specialised super Ja-cobi polynomials, then the following Pieri formula holds m X i =1 u i − n X j =1 v j + 1 S J λ ( u, v ) = X µ ∼ λ, µ ∈P m,n a d ( µ, λ ) S J µ ( u, v ) + b d ( λ ) S J λ ( u, v ) (32) roof. We need the following Pieri formula for the Jacobi symmetric func-tions J λ ( u, k, p, q, h ) ∈ Λ , where Λ is the algebra of symmetric functions[18], h is an additional parameter (see [3]). First note that the limit of J λ ( u, k, p, q, h ) when k → − p, q is well defined as it followsfrom Okounkov’s formula. We denote this limit J λ . By λ ± ε i we denote thesets ( λ , . . . , λ i − , λ i ± , λ i +1 , . . . ) respectively. Lemma 5.2. For k = − and generic p, q the Jacobi symmetric functionssatisfy the following Pieri formula p J λ = X i : λ + ε i ∈P m,n J λ + ε i + l ( λ ) X i =1 a ( λ i + d − i ) J λ (33)+ (cid:18) p + p ( p + 2 q + 1)2 h − l ( λ ) + 1 (cid:19) J λ + X i : λ − ε i ∈P m,n b ( λ i + d − i ) J λ − ε i where p = u + u + . . . , d = h + p + q and a ( l ) = − p ( p + 2 q + 1)(2 l − p − q − l − p − q + 1) , (34) b ( l ) = 2 l (2 l − q − l − p − q − l − p − q − l − p − q )(2 l − p − q − (2 l − p − q − . (35)To prove the lemma we use the following formula for the Jacobi polynomi-als with k = − m variables (see Proposition 7.1 in Okounkov-Olshanski[9]): J λ ( u, − , p, q ) = 1∆( u ) X w ∈ S m ε ( w ) w [ f λ + m − ( u ) f λ + m − ( u ) . . . f λ m ( u m )] , where as before f l ( z ) = f l ( z, p, q ) are the classical normalized Jacobi poly-nomials in one variable with parameters p, q . They satisfy the followingthree-term recurrence relation (see e.g. [21]): zf l ( z ) = f l +1 ( z ) + a ( l ) f l ( z ) + b ( l ) f l − ( z )with a ( l ), b ( l ) given by (34),(35). Therefore we have m X i =1 u i ! J λ ( u ) = 1∆( u ) m X i =1 X w ∈ S m ε ( w ) w [ u i f λ + m − ( u ) . . . f λ m ( u m )]= X i : λ + ε i ∈P m,n J λ + ε i ( u ) + l ( λ ) X i =1 a ( λ i + m − i ) + m X i = l ( λ )+1 a ( m − i ) J λ ( u )+ X i : λ − ε i ∈P m,n b ( λ i + m − i ) J λ − ε i ( u ) , here J λ ( u ) = J λ ( u, − , p, q ) and l ( λ ) is the number of non-zero parts inpartition λ. Since a ( x ) = p ( p + 2 q + 1)2 x − p − q + 1 − p ( p + 2 q + 1)2 x − p − q − m X i = l ( λ )+1 a ( m − i ) = p + p ( p + 2 q + 1)2 m − l ( λ ) − p − q − . Comparing this with (33) we see that this formula is true after the naturalhomomorphism Λ → Λ m , Λ m is the algebra of symmetric polynomials of m variables, if we specialize d to m. Since this is valid for all m, the lemmafollows.The super Jacobi polynomials SJ λ ( u, v, − , p, q ) are defined as the imageof Jacobi symmetric functions J λ under the homomorphism ϕ : Λ → Λ m,n , ϕ ( p l ) = m X i =1 u li − n X j =1 v lj , where d is specialized to m − n and Λ m,n is the algebra of supersymmetricpolynomials (see e.g. [18]). Computing the limits when ( p, q ) → ( − , ( p,q ) → ( − , a ( l ) = δ ( l + 1) − δ ( l ) , lim ( p,q ) → ( − , b ( l ) = 1 − δ ( l )lim ( p,q ) → ( − , (cid:18) p + p ( p + 2 q + 1)2 d − l ( λ ) − p − q − (cid:19) = − δ ( d − l ( λ )) , we have the following formula m X i =1 u i − n X j =1 v j S J λ ( u, v ) = X i : λ + ε i ∈P m,n S J λ + ε i ( u, v ) + X i : λ − ε i ∈P m,n [1 − δ ( λ i − i + d )] S J λ − ε i ( u, v ) (36)+ l ( λ ) X i =1 [ δ ( λ i − i + d + 1) − δ ( λ i − i + d )] S J λ ( u, v )+[ δ ( d − l ( λ )) − SJ λ ( u, v ) , where d = m − n and δ ( x ) = ( , x = 00 , x = 0 . . One can check that it is equivalent to the Pieri formula (32). (cid:3) Remark 5.3. The form of the factor on the left hand side of Pieri formula(32), which was chosen by convenience, has a clear representation-theoreticmeaning: P mi =1 u i − P nj =1 v j + 1 is the supercharacter of the standard rep-resentation of osp (2 m + 1 , n ) . emark 5.4. One can possibly use this for the alternative proof of themain theorem. For this one should only prove that the Euler supercharacterssatisfy the corresponding version of the Pieri formula. This is related to thetranslation functors in representation theory (see e.g. [20] ). Jacobi–Trudy formula for Euler supercharacters In this section we give one more formula for specialized super Jacobipolynomials (and hence for the Euler supercharacters) of Jacobi–Trudy type.We start with the Jacobi–Trudy formula for Jacobi symmetric functionsfollowing [22]. Let J λ ∈ Λ be the Jacobi symmetric functions with k = − p and q (see [3]). They depend also on the additional parameter d replacing the dimension of the space. Let h i = J λ for the partition λ = ( i )consisting of one part for positive i and h i ≡ i < . Define recursivelythe sequence h ( r ) i ∈ Λ , i ∈ Z by the relation h ( r +1) i = h ( r ) i +1 + a ( i + d − h ( r ) i + b ( i + d − h ( r ) i − (37)for r = 0 , , . . . with initial data h (0) i = h i and a ( x ) = − p ( p + 2 q + 1)(2 x − p − q − x − p − q + 1) ,b ( x ) = 2 x (2 x − q − x − p − q − x − p − q − x − p − q )(2 x − p − q − (2 x − p − q − . Theorem 6.1. [22] The Jacobi symmetric functions with k = − have thefollowing Jacobi-Trudy representation: J λ = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) h λ h (1) λ . . . h ( l − λ h λ − h (1) λ − . . . h ( l − λ − ... ... . . . ... h λ l − l +1 h (1) λ l − l +1 . . . h ( l − λ l − l +1 (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (38) where l = l ( λ ) . Taking the limit p → − , q → φ m,n : Λ → Λ m,n we have the following Corollary 6.2. The specialized super Jacobi polynomials S J λ satisfy thefollowing Jacobi–Trudy formula S J λ ( u, v ; − , − , 0) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) h λ h (1) λ . . . h ( l − λ h λ − h (1) λ − . . . h ( l − λ − ... ... . . . ... h λ l − l +1 h (1) λ l − l +1 . . . h ( l − λ l − l +1 (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (39) where λ ∈ H m,n , d = m − n and h ( r ) i are defined recursively by (37) with a ( x ) = δ ( x + 1) − δ ( x ) , b ( x ) = 1 − δ ( x ) nd h (0) i = h i = S J λ ( u, v ; − , − , for λ = ( i ) for positive i and h (0) i = h i ≡ if i < . Euler supercharacters for different choice of Borelsubalgebra It turns out that the relation with super Jacobi polynomials can be mademore direct if we choose, following to Gruson and Serganova [10], a differentBorel subalgebra and suitable parabolic subalgebras.We consider again the case g = osp (2 m + 1 , n ), assuming for convenienceat the beginning that m ≥ n . Choose the following set of simple roots B = { ε − ε , . . . , ε m − n +1 − δ , δ − ε m − n +2 , ε m − n +2 − δ , . . . , ε m − δ n , δ n } This choice is special since this set contains the maximal possible numberof isotropic roots. The corresponding set of even positive roots is R +0 = { ε i ± ε j , i < j, δ p ± δ q , p < q, δ p } with the half-sum ρ = 12 X α ∈ R +0 α = ( m − 12 ) ε + ( m − 32 ) ε · · · + 12 ε m + nδ + ( n − δ + · · · + δ n . The set of positive odd roots is R +1 = { ε i ± δ j , i − j ≤ m − n, δ j ± ε i , i − j > m − n, δ j , j = 1 , . . . , n } with the half-sum ρ = n ( ε + · · · + ε m − n +1 ) + ( n − ε m − n +2 + · · · + ε m + ( n − 12 ) δ + · · · + 12 δ n , so ρ = ρ − ρ = m − n X i =1 ( m − n − i + 12 ) ε i − m X i = m − n +1 ε i + 12 n X j =1 δ j . The following lemma gives a description of the highest weights with re-spect to our choice of simple roots B in terms of partitions. We need thisto establish the relation with super Jacobi polynomials. Lemma 7.1. The weight χ = a ε + a ε + · · · + a m ε m + b δ + b δ + · · · + b n δ n is a highest weight of an irreducible finite dimensional osp (2 m + 1 , n ) -module if and only if there exists a partition λ ∈ H ( m, n ) from the fathook such that χ + ρ = i ( λ ) X i =1 ( λ i + d − i + 12 ) ε i − X i>i ( λ ) ε i + j ( λ ) X j =1 ( λ ′ j − d − j + 12 ) δ j + 12 X j>j ( λ ) δ j . (40) he proof is a geometric reformulation of the conditions on the highestweights given in [10], Corollary 3.Let λ be a partition from H ( m, n ) and χ be the corresponding highestweight. Consider the maximal parabolic subalgebra p = p ( λ ) such that χ can be extended to p as a one dimensional representation. One can checkthat in that case the corresponding roots are R p = {± ε i ± ε j , ± ε i , ± δ p ± δ q , ± δ p , ± δ p , ± ε i ± δ p } with i ( λ ) ≤ i, j ≤ m, i = j, j ( λ ) ≤ p, q ≤ n, p = q . They correspond tothe subalgebra osp (2 l + 1 , l ) , l = m − i ( λ ) = n − j ( λ ) . Let E p ( χ ) be the corresponding Euler supercharacter given by (11). Let π λ be the same as in section 2 and define t ( λ ) as the number of pairs ( i, j ) ∈ π ( λ ) with i − j > d = m − n. Similarly let s ( λ ) be the number of pairs( i, j ) ∈ λ with i − j > d. Theorem 7.2. Let p and χ be the parabolic subalgebra and its one-dimensionalrepresentation determined by a partition λ ∈ H ( m, n ) , then the specializedJacobi polynomial SP λ ( u, v, − , − , coincides up to a sign with the cor-responding Euler supercharacter: SP λ ( u, v, − , − , 0) = ( − s ( λ ) E p ( χ ) . (41) Proof. According to general formula (11) E p ( χ ) = X w ∈ W w Q α ∈ R +1 ( e α/ − e − α/ ) e χ + ρ Q α ∈ R +0 ( e α/ − e − α/ ) Q α ∈ R p ∩ R +1 (1 − e − α ) ! = X w ∈ W w Q α/ ∈ R p ∩ R +1 ( e α/ − e − α/ ) e χ + ρ + τ Q α ∈ R +0 ( e α/ − e − α/ ) ! , where τ = 12 X α ∈ R p ∩ R +1 α. A simple calculation shows that Y α/ ∈ R p ∩ R +1 ( e α/ − e − α/ ) = ( − t ( λ ) Y ( i,j ) ∈ π λ ( u i − v j ) Y j ≤ j ( λ ) ( y j − y − j ) , where u i = x i + x − i , x i = e ε i , i = 1 , . . . , m , v j = y j + y − j , y j = e δ j , j =1 , . . . , n. One can check also that χ + ρ + τ = m X i =1 ( l i + 12 ) ε i + j ( λ ) X j =1 ( k j + 12 ) δ j + n X j>j ( λ ) ( n − j + 1) δ j , where l i , k j are defined by (15), (16). Therefore we have E p ( χ ) = ( − t ( λ ) X w ∈ W w Q ( i,j ) ∈ π λ ( u i − v j ) Q j ≤ j ( λ ) ( y j − y − j ) e χ + ρ + τ ∆( u )∆( v ) Q mi =1 ( x i − x − i ) Q nj =1 ( y j − y − j ) = − t ( λ ) X w ∈ W w Y ( i,j ) ∈ π λ ( u i − v j ) e χ + ρ + τ ∆( u )∆( v ) m Y i =1 ( x i − x − i ) Y j ≤ j ( λ ) ( y j + y − j ) n Y j>j ( λ ) ( y j − y − j ) . Now we use the identities X w ∈ W l w y l y l − . . . y l ∆( v ) Q lj =1 ( y j − y − j ) ! = 1 = X w ∈ W l w y l − y l − . . . y l ∆( v ) Q lj =1 ( y j + y − j ) , where W l is the Weyl group of type C l ≈ BC l , which is a semi-direct prod-uct of permutation group S l and Z l . These identities follow from the Weyl(super)character formula for trivial representations of sp (2 l ) and osp (1 , l ) . This leads to E p ( χ ) = ( − t ( λ ) X w ∈ W w Y ( i,j ) ∈ π λ ( u i − v j ) x l + . . . x l m + m y k + . . . y k n + n ∆( u )∆( v ) m Y i =1 ( x i − x − i ) n Y j =1 ( y j + y − j ) . Comparing this with Theorem 4.1 and using the obvious relation s ( λ ) = t ( λ ) + b ( λ ) , b ( λ ) = X i>m λ i we have the claim. (cid:3) The case of osp (2 m, n )In this section we present the results in the even orthosymplectic case g = osp (2 m, n ) . It would be instructive to start with the special case n = 0, i.e. with theusual orthogonal Lie algebra o (2 m ) . It has the root system of type D m withsimple roots { ε − ε , . . . , ε m − − ε m , ε m − + ε m } . We have a symmetry ε m → − ε m , corresponding to the (outer) automor-phism θ of this Lie algebra. This automorphism appears in the descriptionof the representations of the corresponding Lie group O (2 m ) , which consistsof two connected components.Recall (see [6]) that the highest weights µ = µ ε + · · · + µ m ε m of ir-reducible finite-dimensional representations of Lie algebra o (2 m ) have thefollowing form: all µ i are either integer or half-integer and satisfy the in-equalities: µ ≥ µ ≥ · · · ≥ µ m − ≥ | µ m | . The half-integer µ correspond to the spinor representations and can notbe extended to the representations of the orthogonal group SO (2 m ) , so we estrict ourselves by integer µ. Corresponding representation V µ of o (2 m )can be extended to the full orthogonal group O (2 m ) if and only if it isinvariant under the automorphism θ, which is equivalent to µ m = 0 . If µ m = 0 then one should consider the direct sum W µ = V µ ⊕ V θ ( µ ) , which gives an irreducible representation of O (2 m ) . It is interesting that the sum of the corresponding Euler supercharactersappears also in the general orthosymplectic case g = osp (2 m, n ) as the limitof super Jacobi polynomials (see below), so these limits are natural to linkwith supergroup OSP (2 m, n ) rather than Lie superalgebra osp (2 m, n ) . Now let us give precise formulation of the results. We have g = g ⊕ g , where g = so (2 m ) ⊕ sp (2 n ) and g = V ⊗ V where V and V are the identical representations of so (2 m ) and sp (2 n ) respectively. Let ± ε , . . . , ± ε m , ± δ , . . . , ± δ n be the non-zero weights of the identical repre-sentation of g . The root system of osp (2 m, n ) consists of R = {± ε i ± ε j , ± δ p ± δ q , ± δ p , i = j, ≤ i, j ≤ m , p = q, ≤ p, q ≤ n } ,R = R iso = {± ε i ± δ p } , where R , R and R iso are even, odd and isotropic parts respectively. TheWeyl group W = (cid:16) S m ⋉ Z ( m − (cid:17) × ( S n ⋉ Z n ) acts on the weights by sep-arately permuting ε i , j = 1 , . . . , m and δ p , p = 1 , . . . , n and changing theirsigns such that the total number of signs of ε i is even. A distinguishedsystem of simple roots can be chosen as B = { δ − δ , . . . , δ n − − δ n , δ n − ε , ε − ε , . . . , ε m − − ε m , ε m − + ε m } . We have again a symmetry ε m → − ε m , corresponding to the automorphismof Lie superalgebra osp (2 m, n ), which also denote θ. It acts also in a naturalway on the Grothendieck ring and supercharacters.Consider the parabolic subalgebra p with R p = { ε i − ε j , δ p − δ q , ± ( ε i − δ p ) } , which is isomorphic to the sum of the Lie superalgebra gl ( m, n ) and somenilpotent Lie superalgebra. For any finite-dimensional representation M of p the corresponding Euler supercharacter E p ( M ) is given by (11), which inthis case has the form E p ( M ) = X w ∈ W w Y ( i,j ) ∈ Π m,n (1 − x − i y − j ) x m − . . . x m y n . . . y n schM ∆( u )∆( v ) Q nj =1 ( y j − y − j ) . (42)Here Π m,n is the rectangle of the size m × n , ∆( u ) = Q mi The proof is similar to Proposition 3.1 and is based on the formula Y i,j (1 − x − i y − j ) = X ( − | λ | S λ ( x − , . . . , x − m ) S λ ′ ( y − , . . . , y − n ) , where S λ is the Schur polynomial and the sum is over all the Young diagrams λ , which are contained in the ( m × n ) rectangle. Replacing in the formula(42) as before the product Q i,j (1 − x − i y − j ) by the sum X λ ⊆ Π m,n ( − | λ | x − λ m + m − . . . x − λ m y − λ ′ n + n . . . y − λ ′ +1 n , one can check that X w ∈ W w x − λ m + m − . . . x − λ m y − λ ′ n + n . . . y − λ ′ +1 n ∆( u )∆( v ) Q nj =1 ( y j − y − j ) ! = 0unless λ is either empty or partition of the form λ = ( a , . . . , a r | a + 1 , . . . , a r + 1)in Frobenius notations (see e.g. [18]), in which case it is equal to ( − | λ | . Now the proposition follows from the fact that the number of such diagramscontained in the ( m × n ) rectangle is equal to 2 min( m − ,n ) , which can beproved by induction or reduced to the odd case. (cid:3) Let M = V λ be the polynomial representation of gl ( m, n ) determinedby a Young diagram λ ∈ H ( m, n ) and define the Euler supercharacters as E λ = E ( V λ ) . Introduce the following polynomials ϕ a ( z ) = w a + w − a , a > , ϕ = 1 , ψ a ( z ) = w a +1 − w − a − w − w − , (44)where z = w + w − . They are particular case of the Jacobi polynomialsknown as Chebyshev polynomials of the first and second kind respectively.Let i ( λ ) , j ( λ ) , k i , l j be defined by as before by (1),(2),(15),(16). Definethe following modification of i ( λ ): i ∗ ( λ ) = max { i | λ i + d − i > , ≤ i ≤ m } . It is clear that that i ∗ ( λ ) is equal to i ( λ ) − i ( λ ) depending whether l i ( λ ) = 0 or not.Similarly to Theorem 3.2 one can prove that Theorem 8.2. If l m = 0 then the Euler supercharacters for osp (2 m, n ) can be given by the following formula E λ = C ( λ ) X w ∈ S m × S n w (cid:20) Π λ ( u, v ) ϕ l ( u ) . . . ϕ l m ( u m ) ψ k ( v ) . . . ψ k n ( v n )∆( u )∆( v ) (cid:21) , (45) here C ( λ ) = ( − b m − i ∗ ( λ ) − , b = P j>m λ j . If l m > then we have a similar formula for the sum of Euler superchar-acters E λ + θ ( E λ ) = ( − b X w ∈ S m × S n w (cid:20) Π λ ( u, v ) ϕ l ( u ) . . . ϕ l m ( u m ) ψ k ( v ) . . . ψ k n ( v n )∆( u )∆( v ) (cid:21) . (46)As a corollary we have Theorem 8.3. The limit of the super Jacobi polynomials SJ λ ( u, v, − , p, q ) as ( p, q ) → (0 , is well defined and coincides up to a constant factor withthe Euler supercharacter or sum of two Euler supercharacters: S J λ ( u, v ; − , , 0) = 2 i ∗ ( λ ) − m +1 E λ ( u, v ) (47) if l m = 0 , and S J λ ( u, v ; − , , 0) = E λ + θ ( E λ ) (48) if l m > . The proof follows from comparison of formulas (45),(46) with (20).The Pieri formula for the Euler supercharacters of osp (2 m, n ) followsfrom Pieri formula for super Jacobi polynomials with p = q = 0. Introducethe following functions: a d ( µ, λ ) = , µ = λ \ (cid:3) and j − i = − d , µ = λ \ (cid:3) and j − i = 1 − d , otherwise , (49)As before µ ∼ λ means that the Young diagram µ can obtained from λ byremoving or adding one box. Theorem 8.4. Let S J λ ( u, v ; − , , 0) = SJ λ ( u, v ) be specialized super Ja-cobi polynomials, then the following Pieri formula holds m X i =1 u i − n X j =1 v j S J λ ( u, v ) = X µ ∼ λ, µ ∈P m,n a d ( µ, λ ) S J µ ( u, v ) (50)The Jacobi-Trudy formula in this case follows directly from (38). Proposition 8.5. The specialized super Jacobi polynomials satisfy the fol-lowing Jacobi–Trudy formula S J λ ( u, v ; − , , 0) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) h λ h (1) λ . . . h ( l − λ h λ − h (1) λ − . . . h ( l − λ − ... ... . . . ... h λ l − l +1 h (1) λ l − l +1 . . . h ( l − λ l − l +1 (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (51) where λ ∈ H m,n , d = m − n and h ( r ) i are defined recursively by (37) with a ( x ) = 0 , b ( x ) = 1 + δ ( x − − δ ( x ) nd h (0) i = h i = S J λ ( u, v ; − , , for λ = ( i ) for positive i and h (0) i = h i ≡ if i < . Consider now a different choice of Borel subalgebra and suitable parabolicsubalgebras, following Gruson and Serganova [10].We assume for convenience that m > n . Choose the set of simple rootswith maximal number of isotropic roots: B = { ε − ε , . . . , ε m − n − δ , δ − ε m − n +1 , ε m − n +1 − δ , . . . , δ n − ε m , δ n + ε m } . The corresponding set of even positive roots is R +0 = { ε i ± ε j , i < j, δ p ± δ q , p < q, δ p } with the half-sum ρ = 12 X α ∈ R +0 α = ( m − ε + ( m − ε · · · + ε m − + nδ + ( n − δ + · · · + δ n . The set of positive odd roots is R +1 = { ε i ± δ j , i − j < m − n, δ j ± ε i , i − j ≥ m − n } with the half-sum ρ = n ( ε + · · · + ε m − n ) + ( n − ε m − n +1 + · · · + ε m − + nδ + · · · + δ n , so ρ = ρ − ρ = m − n X i =1 ( m − n − i ) ε i Lemma 8.6. The weight χ = a ε + a ε + · · · + a m ε m + b δ + b δ + · · · + b n δ n with a m ≥ is a highest weight of an irreducible finite dimensional osp (2 m, n ) -module if and only if there exists a partition λ ∈ H ( m, n ) from the fat hooksuch that χ + ρ = i ( λ ) X i =1 ( λ i + d − i ) ε i + j ( λ ) X j =1 ( λ ′ j − d − j + 1) δ j (52)The proof again follows from comparison with Corollary 3 from [10].Let λ be a partition from H ( m, n ) and χ be the corresponding highestweight. Consider the parabolic subalgebra p = p ( λ ) with R p = {± ε i ± ε j , ± δ p ± δ q , ± δ p , ± ε i ± δ p } with i ( λ ) ≤ i, j ≤ m, i = j, j ( λ ) ≤ p, q ≤ n, p = q , corresponding to thesubalgebra osp (2 l, l ) , l = m − i ( λ ) = n − j ( λ ) . Let E p ( χ ) be the corresponding Euler supercharacter given by (11). Let π λ be the same as in section 2 and define t ( λ ) as the number of pairs ( i, j ) ∈ π ( λ ) with i − j ≥ d = m − n. Similarly let s ( λ ) be the number of pairs( i, j ) ∈ λ with i − j ≥ d. heorem 8.7. Let p and χ be the parabolic subalgebra and its one-dimensionalrepresentation determined by a partition λ ∈ H ( m, n ) , then S J λ ( u, v ; − , , 0) = ( − s ( λ ) i ∗ ( λ ) − i ( λ ) E p ( χ ) (53) if λ m ≤ n , and S J λ ( u, v ; − , , 0) = ( − s ( λ ) ( E p ( χ ) + θ ( E p ( χ ))) (54) if λ m > n. Proof. According to general formula (11) E p ( χ ) = X w ∈ W w Q α/ ∈ R p ∩ R +1 ( e α/ − e − α/ ) e χ + ρ + τ Q α ∈ R +0 ( e α/ − e − α/ ) ! , where τ = 12 X α ∈ R p ∩ R +1 α = X i>i ( λ ) ( m − i ) ε i + X j>j ( λ ) ( n − j + 1) δ j . One can check that Y α/ ∈ R p ∩ R +1 ( e α/ − e − α/ ) = ( − t ( λ ) Y ( i,j ) ∈ π λ ( u i − v j )and that χ + ρ + τ = m X i =1 l i ε i + n X j =1 ( k j + 1) δ j , where l i , k j are defined by (15), (16). Therefore we have E p ( χ ) = ( − t ( λ ) X w ∈ W w Π λ ( u, v ) e χ + ρ + τ ∆( u )∆( v ) Q nj =1 ( y j − y − j ) ! =( − t ( λ ) X w ∈ W w Π λ ( u, v ) x l . . . x l m m y k +11 . . . y k n +1 n ∆( u )∆( v ) ! . If l m = 0 then it is easy to see that the average over Z m − × Z n gives E p ( χ ) = D ( λ ) X w ∈ S m × S n w (cid:18) Π λ ( u, v ) ϕ l ( u ) . . . ϕ l m ( u m ) ψ k +1 ( v ) . . . ψ k n +1 ( v n )∆( u )∆( v ) (cid:19) with D ( λ ) = ( − t ( λ ) i ( λ ) − i ∗ ( λ ) . If l m > E p ( χ ) + θ ( E p ( χ ) =( − t ( λ ) X w ∈ S m × S n w (cid:18) Π λ ( u, v ) ϕ l ( u ) . . . ϕ l m ( u m ) ψ k +1 ( v ) . . . ψ k n +1 ( v n )∆( u )∆( v ) (cid:19) . Now the claim follows from Theorem 4.1 and the relation t ( λ ) = s ( λ ) + b ( λ ) . (cid:3) emark 8.8. The coefficient i ( λ ) − i ∗ ( λ ) (which can be either 1 or 2) can beeliminated by a different choice of parabolic subalgebra. Indeed, in the casewhen i ∗ ( λ ) = i ( λ ) − our choice of parabolic subalgebra p is not the maximalone, which in that case corresponds to osp (2 l + 2 , l ) . Concluding remarks We have shown that Euler supercharacters for orthosymplectic Lie super-algebras coincide with specialized super Jacobi polynomials. This fact seemsto be very important and should have more conceptual proof. One possibil-ity is to show that the Euler supercharacters can be uniquely characterizedby Pieri formula and the fact that they are the common eigenfunctions ofthe corresponding algebra of quantum integrals of deformed Calogero–Moserproblem. From representation theory point of view this gives the action ofthe translation functors on Euler supercharacters. The results of Brundan[20] show that one might be able to characterize in a similar way the irre-ducible characters.Another interesting question is about geometry of the genuine parameterspace of the (super) Jacobi polynomials. We have seen that the specializa-tion process requires some blowing-up procedure. The calculations in thespecial case of osp (3 , 2) indicate that this may lead to a description of thecharacters of important classes of finite-dimensional representations.10. Acknowledgements We are very grateful to Vera Serganova for useful and stimulating discus-sions of the results of [10] during the Algebraic Lie Theory programme atINI, Cambridge in March 2009.This work has been partially supported by the EPSRC and by the Euro-pean Union through the FP6 Marie Curie RTN ENIGMA (contract numberMRTN-CT-2004-5652) and through ESF programme MISGAM. References [1] A.N. Sergeev, A.P. Veselov Deformed quantum Calogero-Moser problems and Lie su-peralgebras. Comm. Math. Phys. 245 (2004), no. 2, 249–278.[2] A.N. Sergeev, A.P. Veselov Grothendieck rings of basic classical Lie superalgebras. arXiv:0704.2250 (2007). Accepted for publication in Annals of Math.[3] A.N. Sergeev, A.P. Veselov BC ∞ Calogero-Moser operator and super Jacobi polyno-mials. arXiv: 0807.3858. Adv. Math. (2009) doi: 10.1016/j.aim.2009.06.014.[4] I. Penkov, V. Serganova Cohomology of G/P for classical complex Lie supergroups G and characters of some atypical G -modules. Ann. Inst. Fourier (Grenoble) (1989),no.4, 845-873.[5] V. Serganova Characters of irreducible representations of simple Lie superalgebras. Proceedings of the Int. Congress of Math., Vol. II (Berlin, 1998). Doc. Math. 1998, ExtraVol. II, 583–593.[6] W. Fulton, J. Harris Representation theory. A first course. Graduate Texts in Mathe-matics 129. Readings in Mathematics. Springer-Verlag, New York, 1991.[7] R. Bott Homogeneous vector bundles. Ann. of Math. (2) 66 (1957), 203–248. 8] V. Kac Representations of classical Lie superalgebras. In ”Differential geometricalmethods in mathematical physics, II ”, pp. 597–626. Lecture Notes in Math., 676,Springer, Berlin, 1978.[9] A. Okounkov, G. Olshanski Limits of BC -type orthogonal polynomials as the numberof variables goes to infinity. Jack, Hall-Littlewood and Macdonald polynomials, 281–318,Contemp. Math., 417, Amer. Math. Soc., Providence, RI, 2006.[10] C. Gruson, V. Serganova Cohomology of generalized super grassmannians and char-acter formula for basic classical Lie superalgebras. arXiv:0906.0918 (2009).[11] V.G. Kac, M. Wakimoto Integrable highest weight modules over affine superalgebrasand number theory. Lie theory and geometry, 415–456, Progr. Math., 123, BirkhuserBoston, Boston, MA, 1994.[12] P. Pragacz, A. Thorup On a Jacobi-Trudi identity for supersymmetric polynomials. Adv. Math. (1992), no. 1, 8–17.[13] A.N. Sergeev Tensor algebra of the identity representation as a module over the Liesuperalgebras Gl( n, m ) and Q ( n ) . Mat. Sb. (N.S.) 123(165) (1984), no. 3, 422–430.[14] A.N. Sergeev, A.P. Veselov Generalised discriminants, deformed Calogero-Moser-Sutherland operators and super-Jack polynomials. Adv. Math. 192 (2005), no. 2, 341–375.[15] A. Berele, A. Regev Hook Young diagrams with applications to combinatorics and torepresentations of Lie superalgebras. Adv. in Math. (1987), no. 2, 118–175.[16] A. Molev Factorial supersymmetric Schur functions and super Capelli identities. Kir-illov’s seminar on representation theory, 109–137, Amer. Math. Soc. Transl. Ser. 2, 181,Amer. Math. Soc., Providence, RI, 1998.[17] S. Cheng, W. Wang Remarks on modules of the ortho-symplectic Lie superalgebras. Bull. Inst. Math. Acad. Sin. (N.S.) 3 (2008), no. 3, 353–372.[18] I. Macdonald Symmetric functions and Hall polynomials . 2nd edition, Oxford Univ.Press, 1995.[19] E.M. Moens, J. van der Jeugt A determinantal formula for supersymmetric Schurpolynomials. J. Algebraic Combin. 17 (2003), no. 3, 283–307.[20] J. Brundan Kazhdan-Lusztig polynomials and character formulae for the Lie superal-gebra q ( n ) . Adv. Math. 182 (2004), no. 1, 28–77.[21] A.Erdelyi (Editor) Higher Transcendental Functions. , Vol.2. McGraw-Hill Book Com-pany, 1953.[22] A.N. Sergeev, A.P. Veselov Jacobi–Trudy formula for generalised Schur polynomials. arXiv: 0905.2603. Department of Mathematical Sciences, Loughborough University, Lough-borough LE11 3TU, UK E-mail address : [email protected] Department of Mathematical Sciences, Loughborough University, Lough-borough LE11 3TU, UK and Moscow State University, Moscow, 119899, Russia