Different faces of the shearlet group
Stefan Dahlke, Filippo De Mari, Ernesto De Vito, Sören Häuser, Gabriele Steidl, Gerd Teschke
aa r X i v : . [ m a t h . G R ] J a n Different faces of the shearlet group
Stephan Dahlke ∗ , Filippo De Mari † , Ernesto De Vito † ,Sören Häuser ‡ , Gabriele Steidl ‡ , Gerd Teschke § September 11, 2018
Abstract
Recently, shearlet groups have received much attention in connection with shearlet trans-forms applied for orientation sensitive image analysis and restoration. The square integrablerepresentations of the shearlet groups provide not only the basis for the shearlet transformsbut also for a very natural definition of scales of smoothness spaces, called shearlet coorbitspaces. The aim of this paper is twofold: first we discover isomorphisms between shearletgroups and other well-known groups, namely extended Heisenberg groups and subgroupsof the symplectic group. Interestingly, the connected shearlet group with positive dilationshas an isomorphic copy in the symplectic group, while this is not true for the full shearletgroup with all nonzero dilations. Indeed we prove the general result that there exist, upto adjoint action of the symplectic group, only one embedding of the extended Heisenbergalgebra into the Lie algebra of the symplectic group.Having understood the various group isomorphisms it is natural to ask for the relationsbetween coorbit spaces of isomorphic groups with equivalent representations. These con-nections are examined in the second part of the paper. We describe how isomorphic groupswith equivalent representations lead to isomorphic coorbit spaces. In particular we applythis result to square integrable representations of the connected shearlet groups and meta-plectic representations of subgroups of the symplectic group. This implies the definition ofmetaplectic coorbit spaces.Besides the usual full and connected shearlet groups we also deal with Toeplitz shearletgroups.
Keywords:
Shearlet group, Heisenberg group, Symplectic group, Coorbit space theory
Mathematics Subject Classifiation ∗ FB12 Mathematik und Informatik, Philipps-Universität Marburg, Hans-Meerwein Straße, Lahnberge, 35032Marburg, Germany, [email protected] † Dipartimento di Matematica, Università degli Studi di Genova, Via Dodecaneso 35, 16146 Genova, Italy,{demari,devito}@dima.unige.it ‡ Fachbereich für Mathematik, Technische Universität Kaiserslautern, Paul-Ehrlich-Str. 31, 67663 Kaiser-slautern, Germany, {haeuser,steidl}@mathematik.uni-kl.de § Institute for Computational Mathematics in Science and Technology, Hochschule Neubrandenburg, Universityof Applied Sciences, Brodaer Str. 2, 17033 Neubrandenburg, Germany, [email protected] Introduction
The shearlet transform was originally developed in the inaugural paper [31]. Among othertransforms applied in the analysis of directional information, the continuous shearlet transformstands out because it is related to group theory, i.e., it can be derived from square integrablerepresentations of the so-called shearlet group [9]. Recently, the shearlet transform and itsmodifications have found wide applications in the analysis and restoration of images, see, e.g.,[21, 22, 23, 25, 28]. Further, the shearlet group and its square integrable representations giverise to a natural scale of smoothness spaces, so-called shearlet coorbit spaces [7, 10, 11] whichcan be considered as a special example of general coorbit spaces introduced by Feichtinger andGröchenig [14, 15, 16]. An overview of recent developments on shearlets can be found in thebook [30]. On the other hand, other groups such as the Heisenberg group or the symplecticgroup have been playing an important role in harmonic analysis, linear algebra and signal andimage processing for a long time. In particular, the Heisenberg group is one of the basic toolsfor the mathematical foundation of the short-time Fourier transform, see [15, 19] for details.For the symplectic group we refer to [2, 3, 4] and references therein.In this paper we ask for relations between the different groups and the corresponding coorbitspaces. In particular, we discover isometries of the connected shearlet group and its relatives toextended Heisenberg groups and subgroups of the symplectic group. Interestingly such resultsdo not hold true for the full shearlet group. We show that isomorphic groups with equivalentrepresentations give rise to equivalent coorbit spaces.The first observation concerning the relationship between the extended Heisenberg group andthe shearlet group is found in [32], where square integrability is shown, and it is recalled in thebook [26]. The first embedding of the two-dimensional continuous shearlet group into
Sp(2 , R ) is found in [27], and, with γ = 1 , implicitly appears in [2]. Similarly, the first embedding ofshearlet-like groups with isotropic dilations in arbitrary dimensions (albeit with non-Toeplitzshearing matrices) into Sp(2 , R ) with matrices of the form Σ ⋊ H is found in [27]. The isotropicshearlet-like groups were further explored in the paper [5], while the anisotropic case (includingthe embedding into Sp(2 , R ) ) was generalized to higher dimensions in [6]. Organization of the paper : In Section 2 we discover isomorphisms between shearlet and Toeplitzshearlet groups and other well-known groups. We show that the full and connected shearletgroups are isomorphic to full and connected extended Heisenberg groups, respectively. Further,we prove that the connected shearlet group is isomorphic to a subgroup of the symplectic group,which holds also true for the connected Toeplitz shearlet group. Section 3 deals with the fullshearlet group. We show that it is not possible to embed this group into the symplectic groupor in any of its coverings. The proof is based on the general result that the Lie algebra ofthe extended Heisenberg group can only be embedded in one way into the Lie algebra of thesymplectic group. This result stands also for its own and is presented in Section 4. Finally, inSection 5 the very natural relations between coorbit spaces of isomorphic groups with equivalentrepresentations are presented. Naturally, these coorbit spaces are also isomorphic. At the endof Section 5 we use our findings to introduce metaplectic coorbit spaces.2
Shearlet groups and their isomorphic relatives
In this section we show that the shearlet groups are isomorphic to extended Heisenberg groupsand that the connected shearlet group and Toeplitz shearlet group have an isomorphic subgroupwithin the symplectic group.
For fixed γ ∈ R (usually < γ < , e.g., γ = d , to ensure directional selectivity) and a ∈ R ∗ := R \ { } we introduce the dilation matrices A a,γ := (cid:18) a
00 sgn( a ) | a | γ I d − (cid:19) and A a := aI d (1)and for s = ( s , . . . , s d − ) T ∈ R d the shear and Toeplitz shear matrices S s := (cid:18) s T I d − (cid:19) and T s := s s . . . s d − s s ...... . . . . . . . . . ...... . . . s . . . . . . . (2)Note that the product of two upper triangular Toeplitz matrices T s and T s ′ is again an uppertriangular Toeplitz matrix T s♯s ′ with ( s ♯ s ′ ) i := s i + s ′ i + X j + k = i s ′ j s k , i = 1 , . . . , d − . The shearlet and Toeplitz shearlet groups are defined as follows: • The (full) shearlet group S is the set R ∗ × R d − × ( R × R d − ) with the group operation ( a, s, t ) ◦ S ( a ′ , s ′ , t ′ ) := ( aa ′ , s + | a | − γ s ′ , t + S s A a,γ t ′ ) . Using the notation t = ( t , ˜ t ) T ∈ R d the group law can be rewritten as ( a, s, t , ˜ t ) ◦ S ( a ′ , s ′ , t ′ , ˜ t ′ ) = ( aa ′ , s + | a | − γ s ′ , t + at ′ +sgn( a ) | a | γ s T ˜ t ′ , ˜ t +sgn( a ) | a | γ ˜ t ′ ) . (3) • The connected shearlet group S + is the set R + × R d − × ( R × R d − ) with group law (3)reduced to positive a = | a | = sgn( a ) | a | . • The (full)
Toeplitz shearlet group S T is the set R ∗ × R d − × R d with the group operation ( a, s, t ) ◦ S + T ( a ′ , s ′ , t ′ ) = ( aa ′ , s ♯ s ′ , t + A a T s t ′ ) . (4) • The connected Toeplitz shearlet group S + T is the set R + × R d − × R d with group law (4)restricted to positive a . 3he four groups are locally compact groups, where the left and right Haar measures of theshearlet groups are given by dµ S ,l ( a, s, t ) := 1 | a | d +1 da ds dt and dµ S ,r ( a, s, t ) := 1 | a | da ds dt. see [9, 11] and of the Toeplitz shearlet groups by dµ S T ,l ( a, s, t ) = 1 | a | d +1 da ds dt and dµ S T ,r ( a, s, t ) = 1 | a | da ds dt, see [8, 12] with restriction to positive dilations a for the connected groups. The connected(Toeplitz) shearlet group is a subgroup of the the full (Toeplitz) shearlet group. The Heisenberg group and its polarized version are defined as follows: • The
Heisenberg group H is the set R d − × R × R d − endowed with the group operation ( p, τ, q ) ◦ H ( p ′ , τ ′ , q ′ ) := (cid:0) p + p ′ , τ + τ ′ + ( p T q ′ − q T p ′ ) , q + q ′ (cid:1) . • The polarized Heisenberg group H pol is the same set R d − × R × R d − but with the groupoperation ( p, τ, q ) ◦ H pol ( p ′ , τ ′ , q ′ ) := ( p + p ′ , τ + τ ′ + p T q ′ , q + q ′ ) . (5)These Heisenberg groups are isomorphic with isomorphism given by φ : H → H pol , ( p, τ, q ) ( p, τ + p T q, q ) . This is why we usually write the
Heisenberg group. If we set a = a ′ = 1 in (3) we obtain (1 , s, t , ˜ t ) ◦ S (1 , s ′ , t ′ , ˜ t ′ ) = (1 , s + s ′ , t + t ′ + s T ˜ t ′ , ˜ t + ˜ t ′ ) which looks very similar to the group law of the Heisenberg group in (5). We will show that theshearlet group is isomorphic to an extended Heisenberg group which is equipped with a dilation.For the general concept of group extensions we refer to [29]. We briefly recall the notion ofa semi-direct product. Given a group H and a group G acting on H by automorphisms, i.e.,a smooth map δ : G × H → H is defined such that δ ( g, · ) is an automorphism of H , we canextend H by G by forming the semi-direct product H ⋊ G . The multiplication and inversionare determined by ( h, g )( h ′ , g ′ ) = ( h ◦ H δ ( g, h ′ ) , g ◦ G g ′ ) and ( h, g ) − = ( δ ( g − , h − ) , g − ) . The extended Heisenberg group is the semi-direct product of the Heisenberg group H and R ∗ ,where R ∗ acts on H via the automorphism δ γa ( p, τ, q ) := ( | a | − γ p, aτ, sgn( a ) | a | γ q ) , γ > , for details see [24]. In other words, 4 the extended Heisenberg group is defined by H e := H ⋊ R ∗ with the group operation ( p, τ, q, a ) ◦ H e ( p ′ , τ ′ , q ′ , a ′ ) := (cid:0) p + | a | − γ p ′ , τ + aτ ′ + (cid:0) sgn( a ) | a | γ p T q ′ − | a | − γ q T p ′ (cid:1) , q + sgn( a ) | a | γ q ′ , aa ′ (cid:1) • and the extended polarized Heisenberg group by H pol e := H pol ⋊ R ∗ with group operation ( p, τ, q, a ) ◦ H pol e ( p ′ , τ ′ , q ′ , a ′ ) := (cid:0) p + | a | − γ p ′ , τ + aτ ′ + sgn( a ) | a | γ p T q ′ , q + sgn( a ) | a | γ q ′ , aa ′ (cid:1) . (6)These groups are again isomorphic with φ e : H e → H pol e , ( p, τ, q, a ) ( p, τ + p T q, q, a ) . Using only positive dilations we obtain the connected versions of the extended (polarized)Heisenberg group, whose composition laws we do not write explicitly. For γ = the dilation issymmetric in p and q . Comparing the definition of S and H pol e we see that both groups coincideup to a permutation of the variables. The same holds true for the connected group variants.Taking further the isomorphism between the Heisenberg groups and its polarized versions intoaccount we can summarize: Lemma 2.1. (Relation between extended Heisenberg groups and shearlet groups)
The following relations hold true: H e ∼ = H pol e = S and H + e ∼ = H pol , + e = S + . Let
GL( d, R ) denote the general linear group of real, invertible d × d matrices. The symplecticgroup Sp( d, R ) is the group of all matrices B ∈ GL(2 d, R ) fulfilling B T J B = J for J = (cid:16) I d − I d (cid:17) ,i.e., Sp( d, R ) := { B ∈ R d × d : B T J B = J } . Let H be a closed subgroup of GL( d, R ) and Σ an additive subspace of the symmetric matrices Sym( d, R ) that is invariant under the H -action given by M − T σM − ∈ Σ for all M ∈ H and σ ∈ Σ . Then we know by [13, Example 3] that the semi-direct product Σ ⋊ H := (cid:26)(cid:18) M σM M − T (cid:19) : M ∈ H, σ ∈ Σ (cid:27) (7)is a subgroup of Sp( d, R ) . We are interested in two special groups of the form (7). The firstone is the group TDS( d ) of translations, dilations and shears defined by TDS( d ) := (cid:26)(cid:18) M ( s, a ) 0 σ ( t ) M ( s, a ) M ( s, a ) − T (cid:19) : a ∈ R + , s ∈ R d − , t ∈ R d (cid:27) , where e A a,γ := a − a − γ I d − ! , e S s := (cid:18) − s I d − (cid:19) , M ( s, a ) := e S s e A a,γ , (8)5nd σ ( t ) belongs to the subspace of the symmetric matrices { σ ( t ) = σ ( t , ˜ t ) := (cid:18) t ˜ t T ˜ t d − ,d − (cid:19) : t ∈ R d } . (9)Straightforward computation shows that the subspace (9) is indeed invariant under H -actionwith matrices M ( s, a ) .The relation between this subgroup TDS( d ) of the symplectic group and the connected shearletgroup S + is stated in the following lemma. For a proof we refer to [24, 13]. Lemma 2.2. (Relation between S + and TDS( d ) ) The groups S + and TDS( d ) are isomorphic and the isomorphism κ + is given by κ + : S + → TDS( d ) , ( a, s, t , ˜ t ) (cid:18) M ( s, a ) 0 σ ( t , ˜ t ) M ( s, a ) M ( s, a ) − T (cid:19) . (10)The second interesting group of the form (7) is the group of translations, dilations and Toeplitzshears given by TDS T ( d ) = (cid:26)(cid:18) a − / T − T s a − / σ ( t ) T − T s a / T s (cid:19) : a ∈ R + , s ∈ R d − , t ∈ R d (cid:27) . Clearly, the subspace (9) is invariant under the H -action with matrices a − / T − s . This groupis related to the connected Toeplitz shearlet group as follows: Lemma 2.3. (Relation between S + T and TDS T ( d ) ) The groups S + T and TDS T ( d ) are isomorphic and the isomorphism κ + T is given by κ + T : S + T → TDS T ( d ) , ( a, s, t , ˜ t ) (cid:18) a − / T − T s a − / σ ( t , ˜ t ) T − T s a / T s (cid:19) . (11) Proof.
From the group law in the Toeplitz shearlet group we know that ( a, s, t ) ◦ S + T ( a ′ , s ′ , t ′ ) = ( aa ′ , s ♯ s ′ , t + aT s t ′ ) , so that we have to show κ + T ( a, s, t ) ◦ κ + T ( a ′ , s ′ , t ′ ) = κ + T ( aa ′ , s ♯ s ′ , t + aT s t ′ ) . The right hand side can be rewritten as κ + T ( aa ′ , s ♯ s ′ , t + aT s t ′ ) = ( aa ′ ) − T − T s♯s ′ aa ′ ) − σ (cid:0) t + a ( t ′ + s T ˜ t ′ ) , ˜ t + aT [ s ] ˜ t ′ (cid:1) T − T s♯s ′ ( aa ′ ) T s♯s ′ ! where [ s ] = ( s i ) d − i =1 ∈ R d − . For the left hand side we have (cid:18) a − / T − T s a − / σ ( t , ˜ t ) T − T s a / T s (cid:19) (cid:18) ( a ′ ) − / T − T s ′ a ′ ) − / σ ( t ′ , ˜ t ′ ) T − T s ′ ( a ′ ) / T s ′ (cid:19) = ( aa ′ ) − / T − T s♯s ′ aa ′ ) − / (cid:0) σ ( t , ˜ t ) + aT s σ ( t ′ , ˜ t ′ ) T T s (cid:1) T − T s♯s ′ ( aa ′ ) / T s♯s ′ ! (cid:18) t ′ + s T ˜ t ′ ( T [ s ] ˜ t ′ ) T T [ s ] ˜ t ′ (cid:19) = T s (cid:18) t ′ (˜ t ′ ) T ˜ t ′ (cid:19) T T s . (12)The right hand side of (12) is s · · · s d − . . . .... . . s (cid:18) t ′ (˜ t ′ ) T ˜ t ′ (cid:19) s . . .... . . . s d − · · · s = (cid:18) t ′ + s T ˜ t ′ (˜ t ′ ) T T [ s ] ˜ t ′ (cid:19) s . . .... . . . s d − · · · s = (cid:18) t ′ + s T ˜ t ′ + s T ˜ t ′ ( T [ s ] ˜ t ′ ) T T [ s ] ˜ t ′ (cid:19) which coincides with the left hand side of (12) and we are done. In the following we want to prove that it is not possible to embed the full shearlet group intothe symplectic group
Sp(2 , R ) or one of its covers. To show this result we pursuit the followingpath: a) establish a necessary property for those continuous, injective group homomorphismsof S + to Sp( d, R ) which can be extended to S ; b) show that this property is not fulfilled bythe special homomorphism κ + defined in (10) nor by its conjugation or concatenations withisomorphisms of S + ; c) prove that in dimension d = 2 actually any continuous, injective grouphomomorphism is given up to conjugation or concatenations with isomorphisms of S + by themap κ + .In the following S is regarded as the semi-direct product of its closed normal subgroup S + andits finite subgroup { ( ± , , , } ≃ Z . Indeed, ( − , , , ◦ S ( a, s, t , ˜ t ) ◦ S ( − , , ,
0) = ( a, s, − t , − ˜ t ) and, clearly, S + ◦ S Z = S and S + ∩ Z = { (1 , , , } . With slight abuse of notation, we write an element of S as a pair ( x, ε ) where x = ( a, s, t , ˜ t ) ∈ S + and ε ∈ Z . The group operation in S becomes ( x, ε ) ◦ S ( x ′ , ε ′ ) = ( x ◦ S R ε x ′ , εε ′ ) , where R ε is the group isomorphism of S + given by R ε x = R ε ( a, s, t , ˜ t ) = ( a, s, εt , ε ˜ t ) . (13)Let e := (1 , , , denote the identity of S + . Then, in particular, ( e, − ◦ S ( x,
1) = ( R − x, ◦ S ( e, − (14)and ( e, is the identity of S . 7 emma 3.1. An injective group homomorphism g + : S + → Sp( d, R ) extends to a group homo-morphism g : S → Sp( d, R ) if and only if there exists A ∈ Sp( d, R ) such that A = I d and Ag + ( x ) = g + ( R − x ) A (15) for all x ∈ S + . Under these assumptions we have for all x ∈ S + that g ( x,
1) = g + ( x ) and g ( x, −
1) = g + ( x ) A. (16) The extended group homomorphism g is injective. Note that by injectivity of g + we have A = I for any A fulfilling (15). Proof. ⇒ : Assume that g + : S + → Sp( d, R ) extends to a homomorphism g : S → Sp( d, R ) .Then we have in particular g ( x,
1) = g + ( x ) for all x ∈ S + . We show that A := g ( e, − fulfills(15). Since ( e, − ◦ S ( e, −
1) = ( e ◦ S + R − e,
1) = ( e, and g is a homomorphism we obtain A = I d . By ( e, − ◦ S ( x,
1) = ( e ◦ S + R − x, −
1) = ( R − x, ◦ S ( e, − and the fact that g is a homomorphism we get the second equality in (15). Finally, we concludesince ( x, − ◦ S ( e, −
1) = ( x ◦ S + R − e,
1) = ( x, , g is a homomorphism and A = I d that g ( x, −
1) = g + ( x ) A . ⇐ : Conversely, assume that there exists A ∈ Sp( d, R ) satisfying (15). Set g ( x, ε ) := ( g + ( x ) for ε = 1 ,g + ( x ) A for ε = − . Then (16) is fulfilled by definition. Direct computation shows that g is a group homomorphismfrom S into Sp( d, R ) .It remains to prove the injectivity of g . By (16) and the invertibility of A we see that g ( x, ε ) = g ( x ′ , ε ) implies g + ( x ) = g + ( x ′ ) and since g + is injective further x = x ′ . If g ( x ′ ,
1) = g ( x, − we obtain g + ( x ′ ) = g + ( x ) A and since g + is a homomorphism that g + ( x − x ′ ) = A . Set y := x − x ′ ∈ S + . Since g + is a homomorphism we conclude g + ( y ) = ( g + ( y )) = A = I d and with the injectivity of g + that y = e . But this is only possible if y = e and consequently g + ( y ) = I d = A which is a contradiction. Hence g is injective.We recall that a covering group of Sp( d, R ) is a connected Lie group G with a surjectivecontinuous group homomorphism p : G → Sp( d, R ) whose kernel is discrete. Lemma 3.2.
Let ( G, p ) be a covering group of Sp( d, R ) with covering homomorphism p andan injective continuous group homomorphism i : S → G . Then p ◦ i is an injective grouphomomorphism of S into Sp( d, R ) .Proof. By definition it is clear that p ◦ i is a homomorphism. Next we have that p ◦ i restrictedto S + is injective by the following argument. Recall that a continuous group homomorphismof a Lie group is always smooth. Since S + is a connected, simply connected Lie group itis enough to prove that its tangent map at the identity ( p ◦ i ) ∗ (cid:12)(cid:12) e is injective. Observe that ( p ◦ i ) ∗ (cid:12)(cid:12) e = p ∗ (cid:12)(cid:12) e G i ∗ (cid:12)(cid:12) e . Since i is injective, the same holds true for i ∗ (cid:12)(cid:12) e and since p is a cov-ering homomorphism p ∗ (cid:12)(cid:12) e G is injective. Thus their concatenation is injective. Now applyingLemma 3.1 with g + := p ◦ i | S + yields the assertion.8e have shown that a special injective homomorphism from S + into Sp(2 , R ) is given by g + := κ + defined in (10). Lemma 3.3.
For κ + : S + → Sp( d, R ) defined by (10) there does not exist A ∈ Sp( d, R ) satis-fying (15) . The same holds true for(i) any conjugation of κ + , i.e., for any map κ + B : S + → Sp( d, R ) with κ + B ( x ) := B κ + ( x ) B − , B ∈ Sp( d, R ) ,(ii) any map κ + ϕ : S + → Sp( d, R ) of the form κ + ϕ ( x ) := κ + ( ϕ ( x )) , where ϕ is a group auto-morphism of S + such that ϕ ( R − x ) = R − ϕ ( x ) (17) for all x ∈ S + .Proof.
1. Assume that there exists A := (cid:16) α βγ δ (cid:17) ∈ Sp( d, R ) satisfying (15) for κ + .Since A is symplectic it holds A T J A = J and since A − = A further A T J = J A . Hence (cid:18) α T γ T β T δ T (cid:19) (cid:18) I d − I d (cid:19) = (cid:18) − γ T α T − δ T β T (cid:19) = (cid:18) I d − I d (cid:19) (cid:18) α βγ δ (cid:19) = (cid:18) γ δ − α − β (cid:19) which implies β = − β T , γ = − γ T and δ = α T . Thus, A = (cid:16) α βγ α T (cid:17) with skew-symmetric β, γ . In particular, β and γ have zero diagonal elements. The second condition in (15) with x := (1 , , t , ˜ t ) results by definition of κ + in A (cid:18) I d σ ( t , ˜ t ) I d (cid:19) = (cid:18) I d − σ ( t , ˜ t ) I d (cid:19) A and straightforward computation shows that this implies βσ ( t , ˜ t ) = 0 = σ ( t , ˜ t ) β, (18) α T σ ( t , ˜ t ) = − σ ( t , ˜ t ) α (19)for all t ∈ R d . Since β = − β T , it has the form β = (cid:16) − u T u M (cid:17) with u ∈ R d − and M T = − M ∈ R d − × d − . Choosing t := (1 , , . . . , in (18) we see immediately by definition (9) of σ that u = 0 . Let α = (cid:0) a w T v N (cid:1) with v, w ∈ R d and N ∈ R d − × d − . Evaluating (19) for t := (1 , , . . . , we obtain a = 0 and w = 0 . In summary, the matrix A is of the form A = T v N T Mγ v T N T . Evidently, A is not invertible because it has a zero column.2. (i) Assume that there exists A ∈ Sp( d, R ) satisfying (15) for some conjugation map κ + B . Butthen ˜ A := B − AB ∈ Sp( d, R ) fulfills (15) for κ + . This contradicts the first part of the proof.(ii) Finally, assume there exists A ∈ Sp( d, R ) such that Aκ + ϕ ( y ) A − = κ + ϕ ( R − y ) for all y ∈ S + .With y = ϕ − ( x ) we get Aκ + ( y ) A − = κ + ( R − y ) , which is again a contradiction.9emma 3.1 and Lemma 3.3 imply that the special group homomorphism κ + : S + → Sp( d, R ) in(10) as well as its conjugations or concatenations with group automorphisms of S + satisfying(17) cannot be extended to a group homomorphism of S to Sp( d, R ) . For d = 2 we have thesharper result that this holds true for all injective continuous group homomorphisms g + : S + → Sp(2 , R ) . The following theorem will be proved in the next section. Theorem 3.4.
Let γ ∈ (0 , \ { , } . For any injective continuous group homomorphism g + : S + → Sp(2 , R ) there exists B ∈ Sp(2 , R ) and a continuous group isomorphism ϕ of S + satisfying (17) such that g + ( x ) = Bκ + ( ϕ ( x )) B − . As immediate consequence of the theorem, Lemma 3.1 and Lemma 3.3 we obtain our mainresult (for γ = and γ = see Remark 4.3). Theorem 3.5.
Let γ ∈ (0 , . There does not exist an injective continuous homomorphismfrom S into Sp(2 , R ) and into any of its coverings. We state the above result for γ ∈ (0 , since this is the range of interest in the applications.However, a simple inspection of the proof of Theorem 3.5 shows that Theorem 3.4 holds truefor any γ ∈ R \ { , } .It is not clear if the theorem can be generalized to higher dimensions d > . However, weconjecture that the result holds true in arbitrary dimensions. We start by examining the Lie algebra of the symplectic group in the next subsection and usethe findings for our embedding result in Subsection 4.2.
The Lie algebra sp (2 , R ) of the symplectic group Sp(2 , R ) consists of the real × matrices,called Hamiltonians , which satisfy the equation X T J + J X = 0 . It is the -dimensional Liealgebra sp (2 , R ) = (cid:26)(cid:18) M M M − M T (cid:19) : M ∈ R × , M , M ∈ Sym(2 , R ) (cid:27) . Root space decomposition.
To prove our main embedding result we need a representa-tion of Hamiltonians with respect to a certain basis of sp (2 , R ) which we provide next. Themaximally non compact Cartan subalgebra of sp (2 , R ) is given by a := n H a,b := a b − a
00 0 0 − b : a, b ∈ R o { H , , H , } . We define the linear functionals α and β on a by α ( H a,b ) := a − b, β ( H a,b ) := 2 b. The functionals in △ := △ + ∪ △ − , △ + := { α, β, α + β, α + β } , △ − := {− ν : ν ∈ △ + } form a so-called root system . The root system is meaningful since for any non-zero functional ν not contained in △ the vector space g ν := { X ∈ sp (2 , R ) : [ H, X ] = ν ( H ) X for all H ∈ a } is trivial. The root spaces g ν , ν ∈ △ , are one-dimensional and the linear space associated withthe zero functional g = a is two-dimensional. The four root vectors X ν spanning the space g ν , ν ∈ △ + , are X α := − , X β := , X α + β := , X α + β := . The root vectors X − ν spanning g − ν , − ν ∈ △ − , are given by the Cartan involution X − ν := − X T ν . The Lie algebra sp (2 , R ) has the following vector space direct sum decomposition, known as root space decomposition : sp (2 , R ) = a + X ν ∈△ g ν . To show our embedding result we will label Hamiltonians with respect to the basis B = n X α , X β , X α + β , X α + β , X − α , X − β , X − α − β , X − α − β , H , , H , o (20) = { B k : k = 1 , . . . , } , where the enumeration is with respect to the above ordering of the elements. The followingtable contains the commutator rules of the basis elements: [ · , · ] X α X β X α + β X α + β X − α X − β X − α − β X − α − β H , H , X α X α + β X α + β H − , − X − β − X − α − β − X α X α X β − X α + β − H , X − α − X β X α + β − X α + β X β − X α − H , X − α − X α + β − X α + β X α + β X α + β − X α − H , − X α + β X − α − H − , − X β − X α + β X − α − β X − α − β X − α − X − α X − β H , X α − X − α − β X − β X − α − β X − β − X − α H , X α − X − α − β X − α − β X − α − β X − α − β X − α − β − X − α H , X − α − β H , X α X α + β X α + β − X − α − X − α − β − X − α − β H , − X α X β X α + β X − α − X − β − X − α − β Table 1: Commutator relations [ B i , B j ] for B i , B j ∈ B , i, j = 1 , . . . , .11 anonical normal forms. Next we give the complete list of canonical normal forms towhich we can reduce real × Hamiltonians by means of real symplectic conjugations, i.e.,by applying
Ad( B ) X := BXB − with B ∈ Sp(2 , R ) . For arbitrary space dimensions andsymplectic spaces over any field the result is due to Williamson [33]. For real Hamiltonians thecharacterization can be found in a synthetic form in [1, Appendix 6] which we briefly recall for sp (2 , R ) below.The canonical normal forms are closely related to the Jordan normal forms of Hamiltonians.The eigenvalues of Hamiltonians are of four types , namely (i) real pairs ( + a, − a ), a > , (ii)purely imaginary pairs ( + bi, − bi ), b > , (iii) quadruples ( ± a ± ib ), a > , b > , and (iv)zeros. The Jordan blocks for the two members of a pair have the same structure, and there isan even number of blocks of odd order with zero eigenvalue.In Arnol’d’s book [1] the canonical normal forms are nicely determined by the help of a quadraticform (Hamiltonian function). To this end, note that any X ∈ sp (2 , R ) is related to a symmetricmatrix A ∈ Sym(4 , R ) by J X = A . Now, A ∈ Sym(4 , R ) and hence X = − J A is completelydetermined by the quadratic form H A ( x ) := 12 h Ax, x i . Using the notation x := ( p , . . . , p k , q , . . . , q k ) T , k ∈ { , } , the list of canonical normal formsfor the irreducible cases and their relation to their Jordan normal forms read as follows (orderas in [1]):(A) If X ∈ sp (1 , R ) has a pair of Jordan blocks of order one with real eigenvalues ± a , a ≥ ,then it has the normal form H A ( p , q ) = − ap q and − J A = (cid:18) a − a (cid:19) . (B) If X ∈ sp (2 , R ) has Jordan blocks of order two with real eigenvalues ± a , a ≥ , then H A ( p , p , q , q ) = − a ( p q + p q ) + p q and − J A = D = a − a − a
10 0 0 − a . (C) If X ∈ sp (2 , R ) has a quadruple of Jordan blocks of order one with complex eigenvalues ± a ± ib , a, b > , then H A ( p , p , q , q ) = − a ( p q + p q )+ b ( p q − p q ) and − J A = D = a b − b a − a b − b − a . (D) If X ∈ sp (2 , R ) has a single Jordan block of order four with eigenvalue zero, then, for ε = ± , H A ( p , p , q , q ) = ε p − q q ) − p q and − J A = D = ε ε ε −
10 0 0 0 . X ∈ sp (1 , R ) has a pair of Jordan blocks of order one with purely imaginary eigenvalues ± ib , b > , then, for ε = ± , H A ( p , q ) = − ε b p + q ) and − J A = (cid:18) ε − εb (cid:19) . If X ∈ sp (1 , R ) has a single Jordan block of order two with eigenvalue zero, then it hasthe canonical normal H A ( p , q ) = − ε q , which coincides with the above form for b = 0 .(F) If X ∈ sp (2 , R ) has a pair of Jordan blocks of order two with purely imaginary eigenvalues ± ib , b > , then, for ε = ± , H A ( p , p , q , q ) = − ε b q + q ) − b p q + p q and − J A = D = − εb b ε − b . Combining the irreducible × cases (A) and (E) we obtain the remaining three canonicalnormal forms. We will denote by (X) ⊕ (Y) the canonical form that corresponds to the directsum of the quadratic forms. We then multiply on the left by − J and obtain the correspondingmatrix in sp (2 , R ) . We obtain three further cases, namely:for (A) ⊕ (A): D = a a − a
00 0 0 − a , a ≥ a ≥ , for (E) ⊕ (A): D = ε a − b ε − a , a ≥ , b ≥ , ε = ± , and for (E) ⊕ (E): D = ε
00 0 0 η − b ε − b η , b ≥ b ≥ , ( ε, η ) ∈ { (1 , , (1 , − , ( − , − } . We summarize our specifications of the results in [1, 33] for d = 2 : Corollary 4.1.
For any X ∈ sp (2 , R ) , there exists B ∈ Sp(2 , R ) such that Ad( B ) X ∈ N ,where N := { D k : k = 1 , . . . , } . To prove Theorem 3.4, we first observe that S + is simply connected so that we can pass to itsLie algebra denoted by h + e , see also Lemma 2.1. Let g + ∗ , κ + ∗ and ( R − ) ∗ be the tangent maps13orresponding to g + , κ + and R − in Theorem 3.4. Note that g + ∗ , κ + ∗ are Lie algebra embeddingsof h + e into the sp (2 , R ) , whereas ( R − ) ∗ is a Lie algebra isomorphism of h + e . We have to provethat there exists B ∈ Sp(2 , R ) and a Lie algebra isomorphism Φ : h + e → h + e satisfying ( R − ) ∗ Φ = Φ( R − ) ∗ (21)such that g + ∗ = B ( κ + ∗ Φ) B − . (22)The image of h + e under κ + ∗ is the Lie algebra of TDS(2) and, with slight abuse of notation, weidentify it with h + e . By taking the derivative of the following four one-parameters subgroups, η κ + (exp(2 η ) , , , , η κ + (1 , η, , , η κ + (1 , , η, , η κ + (1 , , , η ) , we get a basis of h + e , namely D + = − H , + (1 − γ ) H , P + = X − α , Q + = − X − α − β , T + = − X − α − β . (23)By Table 1, the non-zero brackets of these generators are [ D + , P + ] = 2(1 − γ ) P + , [ D + , Q + ] = 2 γQ + , [ P + , Q + ] = T + , [ D + , T + ] = 2 T + . (24)The action of the Lie algebra isomorphism ( R − ) ∗ is given by ( R − ) ∗ D + = D + , ( R − ) ∗ P + = P + , ( R − ) ∗ Q + = − Q + , ( R − ) ∗ T + = − T + . Observe that, for any fixed u, z ∈ R with uz = 0 , the linear map Φ : h + e → h + e defined by Φ D + := D + , Φ P + := uP + , Φ Q + := zQ + , Φ T + := uzT + (25)is a Lie algebra isomorphism satisfying (21).An arbitrary Lie algebra embedding g + ∗ : h + e → sp (2 , R ) is in one-to-one correspondence withfour linearly independent generators D, P, Q, T of sp (2 , R ) whose Lie brackets are given by [ D, P ] = 2(1 − γ ) P, [ D, Q ] = 2 γQ, [ P, Q ] = T, [ D, T ] = 2 T, [ P, T ] = [
Q, T ] = 0 (26)where
D, P, Q, T are the images of D + , P + , Q + , T + and hence determine g + ∗ . Then, by (22), itremains to prove the following theorem. Theorem 4.2.
For γ ∈ (0 , \ { , } , let D + , P + , Q + , T + be given by (23) . Then, for anyfixed generators D, P, Q, T ∈ sp (2 , R ) fulfilling (26) , there exists B ∈ Sp(2 , R ) and a Lie algebraisomorphism Φ : h + e → h + e satisfying (21) so that Ad( B )Φ D + = D, Ad( B )Φ P + = P, Ad( B )Φ Q + = Q, Ad( B )Φ T + = T. Note that, the choice of a map Φ as in (25) allows to change P and Q up to a multiplicativeconstant. Proof.
First we obtain by straightforward computation14i) for < γ ≤ , Φ in (25) with u = z = − and B := (cid:18) − (cid:19) ∈ Sp(2 , R ) that Ad( B )Φ D + = H , + (1 − γ ) H , , Ad( B )Φ P + = X α + β , Ad( B )Φ Q + = X α , Ad( B )Φ T + = − X α + β . (27)(ii) for ≤ γ < , Φ in (25) with u = 1 , z = − and B := − J that Ad( B )Φ D + = H , − (1 − γ ) H , , Ad( B )Φ P + = X α , Ad( B )Φ Q + = X α + β , Ad( B )Φ T + = X α + β . (28)We show that, up to conjugation and change of sign in the definition of P and Q , the matricesin (27) and (28) are the unique ones which fulfill (26).Let D, P, Q, T ∈ sp (2 , R ) be arbitrarily fixed, linearly independent matrices with property (26).By Corollary 4.1, up to conjugation, D is one of the canonical forms in N . For each D ∈ N we have to find P, Q ∈ sp (2 , R ) fulfilling in particular [ D, P ] = 2(1 − γ ) P and [ D, Q ] = 2 γQ .For this purpose we consider for all γ ∈ (0 , the pairs of vector spaces V Γ := { X ∈ sp (2 , R ) : [ D, X ] = Γ X } , Γ ∈ { − γ ) , γ } , which coincide in the case γ = . We compute the Lie brackets of every D ∈ N with the basiselements in (20), i.e., we use Table 1 to find [ D, B k ] = P j =1 d kj B j , k = 1 , . . . , . Then weobtain for any X := P k =1 x k B k ∈ sp (2 , R ) that [ D, X ] − Γ X = X k =1 x k [ D, B k ] − Γ X j =1 x j B j = X k =1 10 X j =1 x k d kj B j − Γ X j =1 x j B j = X j =1 X k =1 ( d kj − Γ δ kj ) x k ! B j where δ kj = 1 for k = j and δ kj = 0 otherwise (Kronecker delta). Hence [ D, X ] = Γ X isequivalent to M Γ x = 0 , where x := ( x k ) k =1 and M Γ := ( d kj − Γ δ kj ) j,k =1 . (29)To have non-trivial linearly independent solutions X (for P and Q ) we need that M Γ has rank ≤ for each Γ ∈ { − γ ) , γ } if γ = and rank ≤ if γ = . For the seven matrices D ∈ N the brackets [ D, B k ] , k = 1 , . . . , , the corresponding matrices M Γ and their determinants arelisted in the appendix. Using these computations we have the following cases:1. For D ∈ { D , D , D } we see immediately that det M Γ = 0 for Γ ∈ { − γ ) , γ } , γ ∈ (0 , so that the matrices have full rank.2. For D ∈ { D , D } only a = 2(1 − γ ) = 2 γ leads to det M Γ = 0 , Γ ∈ { − γ ) , γ } . Thisimplies γ = and M γ = M − γ ) = M . But M has rank in both cases D ∈ { D , D } .15. For D = D we obtain det M Γ = 0 for Γ ∈ { − γ ) , γ } in the following cases:3.1 a = 2(1 − γ ) = 2 γ or a = 2(1 − γ ) = 2 γ and b = 0 , which implies γ = . However, asin the previous case, M has rank .3.2 For b = 0 : a = 2(1 − γ ) and a = 2 γ or a = 2 γ and a = 2(1 − γ ) which is only possibleif γ = or γ = . But for these cases the second or third column of M Γ is zero so that P would be a multiple of X β and Q a multiple of X α + β (or vice versa) but these basiselements commute.4. Finally, for D = D , the matrix M Γ is a diagonal matrix with diagonal entries ( a − a − Γ , a − Γ , a + a − Γ , a − Γ , a − a − Γ , − a − Γ , − a − a − Γ , − a − Γ , − Γ , − Γ) . Since a ≥ a ≥ and Γ > , the last six elements are less than zero so that M Γ x = 0[ · , · ] X α X β X α + β X α + β X α X α + β X α + β X β − X α + β X α + β − X α + β X α + β Table 2: Commutator relations [ B i , B j ] for B i , B j ∈ B , i, j = 1 , . . . , .implies x = x = . . . = x = 0 . Hence any solution X is a linear combination of at mostthe first four basis elements. To obtain solutions X for P and Q such that [ P, Q ] = T = 0 we see from Table 2 that at least one solution X must be a nontrivial combination with B = X α , i.e., x = 0 . Consequently, we need a − a − Γ = 0 for one Γ ∈ { − γ ) , γ } .Let γ = . Then, for the other choice of Γ , another diagonal element has to be zero. Table 3shows the corresponding six cases. Note that by setting the first and another diagonalelement to zero, the values a and a are uniquely determined by γ .The pairs ( a , a ) ∈ { (1 − γ, − γ ) , ( γ, γ − } lead to a solution X which is a multiple of X α + β and consequently commutes with all four basis elements, see Table 2. This contradictsthe requirement [ P, Q ] = T = 0 .For ( a , a ) ∈ { ( γ + 1 , − γ ) , (2 − γ, γ ) } the solutions X (for P and Q ) are multiples of X α and X β , so that by Table 2, the matrix T becomes a multiple of X α + β . But this T cannotcommute with P and Q as required by (24).For ( a , a ) = (1 , − γ ) with γ < , γ = we obtain (up to multiplication with scalars) thesetting (27), and for ( a , a ) = (1 , γ − with γ > , γ = the result (28). For γ ∈ (cid:8) , (cid:9) see Remark 4.3.Let γ = which implies Γ = 1 . Then, regarding that x = 0 , and consequently a − a − , i.e., a = a − , the first four diagonal elements of M must read as (0 , a − , a − , a − . The matrix M must have rank ≤ and a ≥ . This is only possible if ( a , a ) ∈{ ( , ) , (1 , } . For ( a , a ) = ( , ) the solutions X for P and Q are multiples of X α a Γ a − a − Γ 2 a − Γ a + a − Γ 2 a − Γ γ + 1 1 − γ − γ ) 2(2 γ −
1) 0 2 γ γ γ − γ − − γ −
1) 21 1 − γ − γ ) 2(2 γ − − γ γ ( γ < ) γ − γ − − γ − − γ − − γ − γ − γ ) 2(2 γ − − γ − γ ( γ < ) γ − γ − − γ − − γ − − γ γ − γ ) 0 2(2 γ −
1) 2 γ γ − γ −
1) 0 − γ − − γ − γ − − γ ) 0 2(3 γ −
2) 2(2 γ −
1) 2 γ ( γ > ) γ − γ −
1) 2( γ −
1) 0 − γ − γ γ − − γ ) 0 2(4 γ −
3) 2(3 γ −
2) 2(2 γ − ( γ > ) γ − γ −
1) 4( γ −
1) 2( γ −
1) 0
Table 3: Possible solutions for a and a such that a − a − Γ = 0 (the first entry of M Γ ).and X β . But then T = [ P, Q ] is a multiple of X α + β which does not commute with both X α and X β as required by (24).For ( a , a ) = (1 , we obtain the solution D = H , P = uX α + vX α + β , Q = wX α + zX α + β , T = ( uz − vw ) X α + β , where uz − wv = 0 must be fulfilled. Possibly changing P into − P , we can assume that uz − wv > . Now straightforward computation shows that C := 1 m m z − v − w u , m := √ uz − vw is a symplectic matrix which fulfills Ad( C ) H , = H , , Ad( C ) X α = uX α + vX α + β , Ad( C ) X α + β = wX α + zX α + β . This finishes the proof.
Remark 4.3.
For γ = (and γ = ) there are ’non-standard’ embeddings e κ of S + into Sp(2 , R ) , which are not conjugated with (10) unless w = 0 . At the Lie algebra level, e κ ∗ : h + e → sp (2 , R ) acts as e κ ∗ D + = H , + 13 H , e κ ∗ P + = uX α + β e κ ∗ Q + = vX α + wX β e κ ∗ Z + = − uvX α + β . However, it is easy to check that for such embeddings there does not exist a symplectic matrix A satisfying (15) . For further explanations we refer to [24]. Coorbit spaces for equivalent representations
In this section we show the relation between the coorbit spaces of isomorphic groups with equiv-alent representations and apply it to our setting. We briefly introduce the general coorbit spacetheory and describe how isomorphic groups with equivalent representations lead to isomorphicscales of coorbit spaces. Then we specify the results for our connected shearlet and shearletToeplitz groups and their isomorphic subgroups of the symplectic group. Since the latter onesare equipped with a metaplectic representation this leads finally to metaplectic coorbit spaces.
Let G be a locally compact group with left Haar measure dµ . A unitary representation of G ona Hilbert space H is a homomorphism π from G into the group U ( H ) of unitary operators on H that is continuous with respect to the strong operator topology, see [17]. A representationis called irreducible if there does not exist a nontrivial π -invariant subspace of H . A unitary,irreducible representation π fulfilling Z G |h ψ, π ( g ) ψ i| dµ ( g ) < ∞ (30)for some ψ ∈ H is called square integrable and a function fulfilling (30) admissible . Assumethat there exists a square integrable representation π of G . For an admissible function ψ ∈ H the mapping V ψ : H → L ( G ) with V ψ ( f )( g ) := h f, π ( g ) ψ i is known as voice transform of f (with respect to ψ ). The admissibility condition (30) is important since it yields to a resolutionof the identity that allows the reconstruction of a function f ∈ H from its voice transform ( h f, π ( g ) ψ i ) g ∈ G . Using the voice transform we can reformulate the admissibility condition (30)as V ψ ( ψ ) ∈ L ( G ) .For a general real-valued weight w and ≤ p ≤ ∞ we define the weighted L p space on G as L p, w ( G ) := { F measurable : F w ∈ L p ( G ) } . Further we will need the weighted sequence spaces ℓ p, w := { ( c i ) i ∈I : ( c i w i ) i ∈I ∈ ℓ p } . The voice transform can be extended from the Hilbert space H to weighted Banach spacesof distributions using coorbit space theory. This theory was developed by Feichtinger andGröchenig in a series of papers [14, 15, 16, 18, 20] and we collect the basic ideas in the following.Let now w be a real-valued, continuous and submultiplicative weight on G , i.e., w ( gh ) ≤ w ( g ) w ( h ) for all g, h ∈ G fulfilling in addition the conditions stated in [18, Section 2.2]. Weassume that the so-called set of analyzing vectors A w := { ψ ∈ H : V ψ ( ψ ) ∈ L ,w ( G ) } . (31)is nonempty and fix a nontrivial function ψ ∈ A w . We can define a set of test functions andequip it with a norm such that it becomes a Banach space by H ,w := { f ∈ H : V ψ ( f ) ∈ L ,w ( G ) } , k f k H ,w := k V ψ ( f ) k L ,w ( G ) . H ,w , alsocalled space of distributions , is denoted by H ∼ ,w . The definitions of H ,w and H ∼ ,w are in-dependent of the choice of the analyzing vector ψ ∈ A w , see [14, Lemma 4.2], in particular H ,w = A w as sets. The spaces H ,w and H ∼ ,w are π -invariant Banach spaces with continuousembeddings H ,w ֒ → H ֒ → H ∼ ,w . The inner product on H × H extends to a sesquilinear formon H ∼ ,w × H ,w : for ψ ∈ H ,w and f ∈ H ∼ ,w the extended representation coefficients V ψ ( f )( g ) := h f, π ( g ) ψ i H ∼ ,w ×H ,w are well-defined and provide the desired generalization of the voice transform on H ∼ ,w .Let m be a w -moderate weight on G which means that m ( xyz ) ≤ w ( x ) m ( y ) w ( z ) for all x, y, z ∈ G . The coorbit space of L p,m ( G ) is given by Co( L p,m ( G )) := H p,m := { f ∈ H ∼ ,w : V ψ ( f ) = h f, π ( · ) ψ i H ∼ ,w ×H ,w ∈ L p,m ( G ) } with norm k f k H p,m = kh f, π ( · ) ψ i H ∼ ,w ×H ,w k L p,m ( G ) . It is a π -invariant Banach space whichdoes not depend on the choice of the analyzing vector ψ ∈ A w , see [15, Theorem 4.2]. Inparticular, the spaces H , H ,w and H ∼ ,w can be identified with the following coorbit spaces H = Co( L ( G )) , H ,w = Co( L ,w ( G )) , and H ∼ ,w = Co( L ∞ , w ( G )) . To establish atomic decompositions and Banach frames for coorbit spaces we need (i) a strongerintegrability condition for the analyzing functions than (31), and (ii) a reasonable discretizationof our group G . Concerning (i) we require that the following better subset (or set of basic atoms )is nontrivial B w := { ψ ∈ H : V ψ ( ψ ) ∈ W L ( L ∞ ( G ) , L ,w ( G )) } where W L ( L ∞ ( G ) , L ,w ( G )) := { F ∈ L ∞ , loc : H F ∈ L ,w ( G ) } and H F : G → R is given by H F ( x ) := k ( L x χ Q ) F k ∞ = sup y ∈ xQ | F ( y ) | , with Q being a relativelycompact neighborhood of the identity element e ∈ G . Then we choose = ψ ∈ B w . Withrespect to (ii) we assume that G can be discretized on a so-called well-spread set . A (countable)family X = { g i : i ∈ I} in G is called well-spread if S i ∈I g i U = G for some compact set U withnon-void interior, and if for all compact sets K ⊂ G there exists a constant C K such that sup j ∈I { i ∈ I : g i K ∩ g j K = ∅} ≤ C K . The following theorem collects results about the existence of atomic decompositions and Banachframes from [18, 10, 7].
Theorem 5.1.
Let ≤ p ≤ ∞ and ψ ∈ B w , ψ = 0 . Then there exists a (sufficiently small)neighborhood U ⊂ G of e such that for any well-spread set X = { g i : i ∈ I} in G the set { π ( g i ) ψ : i ∈ I} provides an atomic decomposition and a Banach frame for H p,m . Atomic decomposition.
Every f ∈ H p,m possesses an expansion f = X i ∈I c i ( f ) π ( g i ) ψ, here the sequence of coefficients ( c i ( f )) i ∈I depends linearly on f and satisfies k ( c i ( f )) i ∈I k ℓ p,m ≤ C k f k H p,m with a constant C only depending on ψ . Conversely, if ( c i ) i ∈I ∈ ℓ p,m , then f = P i ∈I c i π ( g i ) ψ is in H p,m and k f k H p,m ≤ C ′ k ( c i ) i ∈I k ℓ p,m . Banach frames.
The set { π ( g i ) ψ : i ∈ I} is a Banach frame for H p,m which means that thereexist two constants C , C > depending only on ψ such that C k f k H p,m ≤ k ( h f, π ( g i ) ψ i H ∼ ,w ×H ,w ) i ∈I k ℓ p,m ≤ C k f k H p,m , and there exists a bounded, linear reconstruction operator R from ℓ p,m to H p,m such that R (cid:0) ( h f, π ( g i ) ψ i H ∼ ,w ×H ,w ) i ∈I (cid:1) = f. Let G and e G be locally compact groups with left Haar measures dµ and d e µ , respectively,which are isomorphic with isomorphism ι : G → e G . Further, let H and e H be Hilbert spaceswith isometric isomorphism Ψ :
H → e H . Let π : G → U ( H ) and e π : e G → U ( e H ) be unitaryrepresentations of G on H and of e G on e H , respectively, so that for all g ∈ G and all f ∈ H Ψ( π ( g ) f ) = e π ( ι ( g )) (Ψ f ) . (32)We will refer to such π and e π as equivalent representations. Setting e f := Ψ f , i.e., f = Ψ − e f this can be rewritten as Ψ (cid:16) π ( g )Ψ − e f (cid:17) = e π ( ι ( g )) e f . The following diagram illustrates therelations. U ( H ) U ( e H ) G e Gπ Ψ( · )Ψ − ι e π Assume that G , H and π give rise to a sequence of coorbit spaces including atomic decompo-sitions and Banach frames as described in the previous Subsection 5.1. For the correspondingweights and function spaces we use the notation from Subsection 5.1. We are interested in therelation to the coorbit spaces based on e G , e H and e π in terms of ι and Ψ .First, we define e w := w ◦ ι − which is clearly a real-valued, continuous, submultiplicative weighton e G satisfying the conditions in [18, Section 2.2], since w does. Let e A e w := n e ψ ∈ e H : h e ψ, e π ( e g ) e ψ i e H ∈ L , e w ( e G ) o . e ψ := Ψ ψ for ψ ∈ A w we obtain by (32) and since Ψ is an isometry h ψ, π ( ι − ( e g )) ψ i H = h Ψ ψ, Ψ (cid:0) π ( ι − ( e g )) ψ (cid:1) i e H = h e ψ, e π ( e g ) e ψ i e H . Thus, A w and e A e w are isomorphic. For an analyzing vector e ψ = Ψ ψ ∈ e A e w we introduce the setof test functions e H , e w := n e f ∈ H : h e f , e π ( e g ) e ψ i ∈ L , e w ( e G ) o . Since k f k H ,w = kh f, π ( g ) ψ ik L ,w ( G ) = kh Ψ f, e π ( ι ( g ))Ψ ψ ik L ,w ( G ) we see with e g = ι ( g ) and e ψ = Ψ ψ that k f k H ,w = kh f, π ( g ) ψ ik L ,w ( G ) = kh e f , e π ( e g ) e ψ ik L , e w ( e G ) = k e f k e H , e w . As for the set of analyzing vectors Ψ induces an isomorphism from H ,w onto e H , e w , where e f = Ψ f . Let ( e H , e w ) ∼ denote the anti-dual space of e H , e w . This space is related to H ∼ ,w by Ψ ∗ e f = f for all e f ∈ ( e H , e w ) ∼ , where Ψ ∗ : ( e H , e w ) ∼ → H ∼ ,w denotes the adjoint of theisomorphism Ψ : H ,w → e H , e w . The relations are illustrated in the following diagram. H ,w H H ∼ ,w e H , e w e H ( e H , e w ) ∼ Ψ Ψ Ψ ∗ The inner product on e H extends to a dual pairing on ( e H , e w ) ∼ × e H , e w by h e f , e π ( e g ) e ψ i ( e H ,w ) ∼ × e H ,w = h e f , Ψ ( π ( g ) ψ ) i ( e H ,w ) ∼ × e H ,w = h Ψ ∗ e f , π ( g ) ψ i H ∼ ,w ×H ,w = h f, π ( g ) ψ i H ∼ ,w ×H ,w . (33)It follows that both lines of the diagram are isomorphic Gelfand triples and the extended voicetransforms coincide in the sense of (33). To the w -moderate weight function m on G weassociate the e w -moderate function e m on e G by e m ( e g ) := m ( ι − ( e g )) . Now we are ready to definethe coorbit spaces of L p, e m ( e G ) by e H p, e m := n e f ∈ ( e H ,w ) ∼ : h e f , e π ( · ) e ψ i ( e H ,w ) ∼ × e H ,w ∈ L p, e m ( e G ) o which are isomorphic to H p,m with isomorphism e f = Ψ f .Finally, we want to analyze the relation between atomic decompositions and Banach framesof the isomorphic coorbit spaces H p,m and e H p, e m . Clearly, the set e B e w is just given by Ψ( B w ) .Further, given a well-spread set X := { g i : i ∈ I} of G we can check that e X = ι ( X ) = { e g i = ι ( g i ) : i ∈ I} is a well-spread set of e G . Then, we obtain from the atomic decomposition f = X i ∈I c i ( f ) π ( g i ) ψ f ∈ H p,m and the atomic decomposition of e f = Ψ f by e f = Ψ f = Ψ X i ∈I c i ( f ) π ( g i ) ψ ! = X i ∈I c i ( f ) e π ( e g i ) e ψ, i.e., e f can be decomposed using the same sequence of coefficients. Concerning the Banach frameproperty of { e π ( e g i ) e ψ : i ∈ I} it remains to deduce the reconstruction operators e R : ℓ p, e m → e H p, e m given a reconstruction operator R : ℓ p,m → H p,m . Let the sequence of moments of e f be givenas {h e f , e π ( g i ) e ψ i ( e H ,w ) ∼ × e H ,w } ∈ ℓ p, e m . Using f = Ψ ∗ e f we get h e f , e π ( e g i ) e ψ i ( e H ,w ) ∼ × e H ,w = h e f , Ψ ( π ( g i ) ψ ) i ( e H ,w ) ∼ × e H ,w = h Ψ ∗ e f , π ( g i ) ψ i H ∼ ,w ×H ,w and since R ( {h Ψ ∗ e f , π ( g i ) ψ i} i ∈I ) = Ψ ∗ e f that e R = (Ψ ∗ ) − ◦ R . In this section we want to use the results from the previous section to establish coorbit spacesfor the groups
TDS( d, R ) and TDS T ( d, R ) using the coorbit spaces for S + and S + T . First, wehave to determine the latter ones. Square integrable representations of S + and S + T . The coorbit spaces for the full shearletgroup S and H = L ( R d ) were introduced in [10, 11] and for the full Toeplitz shearlet group S T in [8]. However, we cannot use these results directly since the respective representations arenot irreducible if restricted to the connected groups S + and S + T . Therefore, we consider insteadof L ( R d ) the Hilbert space L (Θ L ) := { f ∈ L ( R d ) : supp ˆ f ⊆ Θ L } , where Θ L denotes the halfspace Θ L := { ξ ∈ R d : ξ ≤ } and ˆ f = F f is the Fourier transform of f .To shorten the notation we write in the following S +( T ) to address both S + and S + T . Further, wejust use A for the dilations in (1) and S for the shears in (2) for both groups and denote by µ their left Haar measures. We define a representation π : S +( T ) → U ( L (Θ L )) by π ( a, s, t ) f ( x ) = f a,s,t ( x ) := (det( A )) − f ( A − S − ( x − t )) (34)for all ( a, s, t ) ∈ S +( T ) and all f ∈ L (Θ L ) . The Fourier transform ˆ f a,s,t of f a,s,t is given by ˆ f a,s,t ( ω ) := ˆ π ( a, s, t ) ˆ f ( ω ) = (det( A )) ˆ f ( A T S T ω ) e − πi h t,ω i . (35)Note that the representations π and ˆ π are equivalent in the sense that F π F − = ˆ π. π is indeed a unitary representation of S +( T ) on L (Θ L ) .The following lemma shows that the unitary representation π defined in (34) is also squareintegrable. Lemma 5.2.
A function ψ ∈ L (Θ L ) is admissible if and only if it fulfills the admissibilitycondition < C ψ := Z Θ L | ˆ ψ ( ω ) | | ω | d dω < ∞ . (36) Then, for any f ∈ L (Θ L ) the following equality holds true: Z S +( T ) |h f, ψ a,s,t i| dµ ( a, s, t ) = C ψ k f k L (Θ L ) . (37) In particular, the unitary representation π is irreducible and hence square integrable.Proof. Observe that SH f ( a, s, t ) := h f, ψ a,s,t i = h f, det( A ) − ψ ( A − S − ( · − t )) i = f ∗ ψ ∗ a,s, ( t ) , (38)where ψ ∗ a,s,t := ψ a,s,t ( − · ) . For fixed a and s , since f, ψ a,s, ∈ L ( R d ) , then by standard facts f ∗ ψ a,s, = F − ( ˆ f c ψ ∗ a,s, ) and by Plancherel’s theorem we obtain Z R d | f ∗ ψ ∗ a,s, ( t ) | dt = Z b R d | ˆ f ( ω ) | | c ψ ∗ a,s, ( ω ) | dω, where the left hand side is finite if and only if the right hand side is such. Hence, by Fubini,(35), and (38) we obtain Z S +( T ) |h f, ψ a,s,t i| daa d +1 ds dt = Z S +( T ) | f ∗ ψ ∗ a,s, ( t ) | dt ds daa d +1 = Z R + Z R d − Z Θ L | ˆ f ( ω ) | | c ψ ∗ a,s, ( ω ) | dω ds daa d +1 = Z R + Z R d − Z Θ L | ˆ f ( ω ) | det( A ) a d +1 | ˆ ψ ( S T A T ω ) | dω ds da = Z Θ L Z R + Z R d − | ˆ f ( ω ) | det( A ) a d +1 | ˆ ψ ( S T A T ω ) | ds da dω. To continue we need the concrete matrices A and S for the connected shearlet and Toeplitzshearlet groups. We restrict our attention to S + T . The conclusions for S + can be drawn in thesame way with slightly simpler substitutions. Then T T s A T a ω = · · · s ... s s ... . . . . . . . . . s d − s d − . . . s aω aω ...... aω d = aω s aω + aω ...... s d − aω + as d − ω + . . . + aω d ξ d := s d − aω + . . . + aω d , ξ d − := s d − aω + . . . + aω d − , . . . , ξ := s aω + aω we obtain with d e ξ := dξ . . . dξ d and d e ξ = ( a | ω | ) d − ds the equality Z S +( T ) |h f, ψ a,s,t i| daa d +1 ds dt = Z Θ L Z R + Z R d − | ˆ f ( ω ) | a − d | ω | − ( d − | ˆ ψ ( aω , ξ , . . . , ξ d ) | d e ξ da dω = Z Θ L Z R − Z R d − | ˆ f ( ω ) | | ξ | − d | ˆ ψ ( ξ , ξ , . . . , ξ d ) | d e ξ dξ dω = Z Θ L | ˆ f ( ω ) | dω Z Θ L | ˆ ψ ( ξ ) | | ξ | d dξ = k f k L (Θ L ) Z Θ L | ˆ ψ ( ξ ) | | ξ | d dξ where again the left hand side is finite if and only if the right hand side is such. This yields(37) and since there exist functions ψ ∈ L (Θ L ) for which (36) is finite we have verified (30).Next, we show how (37) implies the irreducibility of π . Suppose by contradiction that it is notirreducible. Then there exist two non-zero functions f, ψ ∈ L ( R d ) for which h f, ψ a,s,t i = 0 asa function of ( a, s, t ) . But then the previous string of equalities yields that the right hand sidevanishes. But this in turn implies that either f = 0 or ˆ ψ = 0 , a contradiction.The whole coorbit space setting worked out in [10, 11, 8] for the full shearlet and Toeplitzshearlet groups can now be modified in a straightforward way to the connected groups. Equivalent representations of
TDS( d, R ) and TDS T ( d, R ) . We want to exploit the iso-morphisms κ +( T ) between S +( T ) and the subgroups TDS ( T ) ( d, R ) of the symplectic group to definecoorbit spaces for the latter subgroups with respect to their metaplectic representations. Forthis purpose, we define the diffeomorphism Q : Θ L → Θ L with ξ Q ( ξ ) := −
12 ( ξ , ξ ξ , . . . , ξ ξ d ) T . The inverse of Q is given for ξ < by Q − ( ξ ) = √ −√− ξ , ξ √− ξ , . . . , ξ d √− ξ ) T . Further, theabsolute value of the determinants of the Jacobian J Q of Q and its inverse read | det( J Q ( ξ )) | = 2 − d | ξ | d and (cid:12)(cid:12) det( J Q − ( ξ )) (cid:12)(cid:12) = ( √ d − | ξ | − d . (39)Based on Q we can define the isomorphism Ψ : L (Θ L ) → L (Θ L ) as Ψ = F − ˆΨ F , where ˆΨ ˆ f ( ξ ) = | det( J Q − ( ξ )) | ˆ f ( Q − ( ξ )) . (40)Its inverse is given by ˆΨ − F ( ξ ) = | det( J Q ( ξ )) | F ( Q ( ξ )) . The semi-direct product Σ ⋊ H in (7) possesses a (mock) metaplectic representation π m on L (Θ L ) defined for all g ∈ Σ ⋊ H by ˆ π m ( g ) ˆ f ( ξ ) := (det M ) − e − πi h t,Q ( ξ ) i ˆ f ( M − ξ ) , f ∈ L (Θ L ) . For
TDS( d, R ) this metaplectic representation becomes ˆ π m ( a, s, t ) ˆ f ( ξ ) = a − d + γ ( d − e − πi h t,Q ( ξ ) i ˆ f ( e A − a,γ e S − s ξ ) (41)24ith the matrices e A a,γ and e S s in (8), and for TDS T ( d, R ) just ˆ π m ( a, s, t ) ˆ f ( ξ ) = a d e − πi h t,Q ( ξ ) i ˆ f ( √ a T T s ξ ) . (42)We now show that these representations are equivalent to the representations (34) of the con-nected (Toeplitz) shearlet groups. Lemma 5.3.
Let the isomorphism κ + : S + → TDS( d, R ) be given by (10) . Then the represen-tation π m of TDS( d, R ) in (41) is equivalent to the representation π of S + defined by (34) inthe sense ˆΨˆ π m ( κ + ( · )) ˆΨ − = F π F − = ˆ π. (43) Proof.
First we verify that Q ( e A − a,γ e S − s ξ ) = Q a sa − + γ a − + γ I d − ! (cid:18) ξ e ξ (cid:19)! = Q ( a ξ , sξ a − + γ + a − + γ e ξ ) = − (cid:16) aξ , sa γ ξ + a γ ξ e ξ (cid:17) = − A a,γ (cid:18) ξ sξ + ξ e ξ (cid:19) = A a,γ S T s Q ( ξ ) . (44)Then we conclude by (40), the definition of ˆ π m and (44) that ( ˆΨˆ π m ( a, s, t ) ˆΨ − ˆ ψ )( ξ )= | det( J Q − ( ξ )) | ˆ π m ( a, s, t )( ˆΨ − ˆ ψ )( Q − ( ξ ))= | det( J Q − ( ξ )) | a − d + γ ( d − ( ˆΨ − ˆ ψ )( e A − a,γ e S − s Q − ( ξ )) e − πi h t,Q ( Q − ( ξ )) i = | det( J Q − ( ξ )) | a − d + γ ( d − | det( J Q ( e A − a,γ e S − s Q − ( ξ ))) | ˆ ψ ( Q ( e A − a,γ e S − s Q − ( ξ ))) e − πi h t,ξ i = | det( J Q − ( ξ )) | a − d + γ ( d − | det( J Q ( e A − a,γ e S − s Q − ( ξ ))) | ˆ ψ ( A a,γ S T s ξ ) e − πi h t,ξ i . By (39) we have | det( J Q − ( ξ )) | = ( √ d − ( p − ξ ) − d . (45)Simplifying | det( J Q ( e A − a,γ e S − s Q − ( ξ ))) | = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) det J Q a sa − + γ a − + γ I d − ! −√− ξ √ ξ √− ξ ... √ ξ d √− ξ (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (2 − d ( p − aξ ) d ) = ( √ (1 − d )+ d a d ( p − ξ ) d we obtain further | det( J Q − ( ξ )) | | det( J Q ( e A − a,γ e S − s Q − ( ξ ))) | = ( √ d − ( p − ξ ) − d ( √ (1 − d )+ d a d ( p − ξ ) d = a d ( ˆΨˆ π m ( a, s, t ) ˆΨ − ˆ ψ )( ξ ) = a + γ ( d − ˆ ψ ( A a,γ S T s ξ ) e − πi h t,ξ i = | det( A a,γ ) | ˆ ψ ( A a,γ S T s ξ ) e − πi h t,ξ i = ˆ π ( a, s, t ) ˆ ψ ( ξ ) . Equation (43) can be illustrated by the following diagram. U ( L (Θ L )) U ( L (Θ L )) S + TDS( d, R )ˆ π ˆΨ − ( · ) ˆΨ κ + ˆ π m Lemma 5.4.
Let the isomorphism κ + T : S + T → TDS T ( d, R ) be given by (11) . Then the repre-sentations π m of TDS T ( d, R ) in (42) and π of S + T given by (34) are equivalent in the sensethat ˆΨˆ π m ( κ + T ( · )) ˆΨ − = F π F − = ˆ π. (46) Proof.
By definition of σ ( t ) in (9) we have h σ ( t , ˜ t ) ξ, ξ i = (cid:28)(cid:18) t ˜ t T ˜ t (cid:19) (cid:18) ξ e ξ (cid:19) , (cid:18) ξ e ξ (cid:19)(cid:29) = (cid:28)(cid:18) t ξ + ˜ t T e ξ ξ ˜ t (cid:19) , (cid:18) ξ e ξ (cid:19)(cid:29) = t ξ + 12 ξ ˜ t T e ξ + 12 ξ ˜ t T e ξ = t ξ + ξ ˜ t T e ξ = − h t, Q ( ξ ) i . Using (12) we obtain for all t ∈ R d that − h t, Q ( T T s ξ ) i = h σ ( t , ˜ t ) T T s ξ, T T s ξ i = h T s σ ( t , ˜ t ) T T s ξ, ξ i = h σ ( T s t ) ξ, ξ i and by the above relation h σ ( T s t ) ξ, ξ i = − h T s t, Q ( ξ ) i = − h t, T T s Q ( ξ ) i such that Q ( T T s ξ ) = T T s Q ( ξ ) and by definition of Q further Q ( √ aT T s ξ ) = aT T s Q ( ξ ) . (47)Now we can compute ( ˆΨˆ π m ( a, s, t ) ˆΨ − ˆ ψ )( ξ )= | det( J Q − ( ξ )) | ˆ π m ( a, s, t )( ˆΨ − ˆ ψ )( Q − ( ξ ))= | det( J Q − ( ξ )) | a d ( ˆΨ − ˆ ψ )( √ aT T s Q − ( ξ )) e − πi h t,Q ( Q − ( ξ )) i = | det( J Q − ( ξ )) | a d | det( J Q ( √ aT T s Q − ( ξ ))) | ˆ ψ ( Q ( √ aT T s Q − ( ξ ))) e − πi h t,ξ i = | det( J Q − ( ξ )) | a d | det( J Q ( √ aT T s Q − ( ξ ))) | ˆ ψ ( aT T s ξ ) e − πi h t,ξ i . | det( J Q ( √ aT T s Q − ( ξ ))) | = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) det J Q √ a s ... . . . s d − −√− ξ √ ξ √− ξ ... √ ξ d √− ξ (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12) det (cid:16) J Q (cid:16) − p − aξ , . . . (cid:17)(cid:17)(cid:12)(cid:12)(cid:12) = (2 − d ( p − aξ ) d ) = ( √ (1 − d )+ d a d ( p − ξ ) d and (45) we get | det( J Q − ( ξ )) | | det( J Q ( √ aS T s Q − ( ξ ))) | = a d and finally ( ˆΨˆ π m ( a, s, t ) ˆΨ − ˆ ψ )( ξ ) = a d ˆ ψ ( aT T s ξ ) e − πi h t,ξ i = ˆ π ( a, s, t ) ˆ ψ ( ξ ) . Observe that, in light of (46), a vector ˆ ψ ∈ L (Θ L ) , is admissible for ˆ π m if and only if Z Θ L | ˆΨ ˆ ψ ( ξ ) | | ξ | d dξ < + ∞ . Metaplectic coorbit spaces.
Based on the equivalence of the metaplectic representationsof the subgroups
TDS ( T ) ( d, R ) to square integrable representations of S +( T ) we can apply theresults from Subsection 5.2 to define coorbit spaces with respect to TDS ( T ) ( d, R ) .Let ψ ∈ L (Θ L ) be an admissible shearlet. Then the transform SH ψ : L (Θ L ) → L ( S +( T ) ) defined by SH ψ ( f )( a, s, t ) = h f, π ( a, s, t ) ψ i L (Θ L ) is called the continuous (Toeplitz) shearlet transform , whereas the transform SH mψ : L (Θ L ) → L (TDS ( T ) ( d )) defined by SH mψ f ( κ +( T ) ( a, s, t )) = h f, π m ( κ +( T ) ( a, s, t )) ψ i L (Θ L ) is called the metaplectic continuous shearlet transform .Using the above equivalences the (Toeplitz) shearlet coorbit spaces SC p,m := { f ∈ H ∼ ,w : SH ψ ( f ) ∈ L p,m ( S +( T ) ) } and the metaplectic shearlet coorbit spaces SC mp,m := { f ∈ ( e H , e w ) ∼ : SH mψ ( f ) ∈ L p, e m (TDS ( T ) ( d )) } are diffeomorphic. 27 cknowledgements This work has been supported by Deutsche Forschungsgemeinschaft (DFG), Grants DA 360/19–1 and STE 571/11–1. Some parts of the paper have been written during a stay at the Erwin-Schrödinger Institute (ESI), Vienna, Workshop on “Time-Frequency Analysis”, January 13-172014. Therefore the support of ESI is also acknowledged.F. De Mari and E. De Vito were partially supported by Progetto PRIN 2010-2011 “Varietàreali e complesse: geometria, topologia e analisi armonica”. They are members of the GruppoNazionale per l’Analisi Matematica, la Probabilità e le loro Applicazioni (GNAMPA) of theIstituto Nazionale di Alta Matematica (INdAM).S. Dahlke, S. Häuser, G. Steidl and G. Teschke were partially supported by DAAD Project57056121, ”Hochschuldialog mit Südeuropa 2013”.
Appendix
In the following we list the Lie brackets of the canonical matrices D ∈ N with the basis matrices X ν , ν ∈ △ , and H , , H , from the root space decomposition of sp (2 , R ) and the matrices M Γ defined by (29). Case 1.
For D = a H , + a H , we obtain [ D , X α ] = ( a − a ) X α , [ D , X − α ] = − ( a − a ) X − α , [ D , X β ] = 2 a X β , [ D , X − β ] = − a X − β , [ D , X α + β ] = ( a + a ) X α + β , [ D , X − α − β ] = − ( a + a ) X − α − β , [ D , X α + β ] = 2 a X α + β , [ D , X − α − β ] = − a X − α − β , [ D , H , ] = 0 , [ D , H , ] = 0 and M Γ is a diagonal matrix with entries ( a − a − Γ , a − Γ , a + a − Γ , a − Γ , a − a − Γ , − a − Γ , − a − a − Γ , − a − Γ , − Γ , − Γ) , where a ≥ a ≥ . Case 2.
For D = X − α + aH , + aH , we obtain [ D , X α ] = H , − H , , [ D , X − α ] = 0 , [ D , X β ] = 2 aX β , [ D , X − β ] = − aX − β + X − α − β , [ D , X α + β ] = − X β + 2 aX α + β , [ D , X − α − β ] = − aX − α − β + X − α − β , [ D , X α + β ] = − X α + β + 2 aX α + β , [ D , X − α − β ] = − aX − α − β , [ D , H , ] = X − α , [ D , H , ] = − X − α M Γ = − Γ 0 0 0 0 0 0 0 0 00 2 a − Γ − a − Γ − a − Γ 0 0 0 0 0 00 0 0 0 − Γ 0 0 0 1 −
10 0 0 0 0 − a − Γ 0 0 0 00 0 0 0 0 1 − a − Γ 0 0 00 0 0 0 0 0 1 − a − Γ 0 01 0 0 0 0 0 0 0 − Γ 0 − − Γ with det M Γ = Γ (Γ − a ) (Γ + 2 a ) , a ≥ . Case 3.
For D = bX α + bX − α + aH , + aH , we obtain [ D , X α ] = bH , − bH , , [ D , X − α ] = − bH , + bH , , [ D , X β ] = 2 aX β + bX α + β , [ D , X − β ] = − aX − β + bX − α − β , [ D , X α + β ] = − bX β + 2 aX α + β + bX α + β , [ D , X − α − β ] = − bX − β − aX − α − β + bX − α − β , [ D , X α + β ] = − bX α + β + 2 aX α + β , [ D , X − α − β ] = − bX − α − β − aX − α − β , [ D , H , ] = − bX α + bX − α , [ D , H , ] = bX α − bX − α and M Γ = − Γ 0 0 0 0 0 0 0 − b b a − Γ − b b a − Γ − b b a − Γ 0 0 0 0 0 00 0 0 0 − Γ 0 0 0 b − b − a − Γ − b b − a − Γ − b b − a − Γ 0 0 b − b − Γ 0 − b b − Γ with det M Γ = Γ (Γ + 4 b )(Γ − a )(Γ + 2 a ) (cid:0) (Γ − a ) + 4 b (cid:1) (cid:0) (Γ + +2 a ) + 4 b (cid:1) , a, b > . Case 4.
For D = εX α + β − X − α − ε X − α − β we obtain [ D , X α ] = − εX α + β − εX − α − β − H , + H , , [ D , X − α ] = 2 εX β , [ D , X β ] = 0 , [ D , X − β ] = − εX α − X − α − β , [ D , X α + β ] = 2 X β + εX − α , [ D , X − α − β ] = − X − α − β − εH , − εH , , [ D , X α + β ] = 2 X α + β − εH , , [ D , X − α − β ] = 2 εX − α , [ D , H , ] = − εX α + β − X − α − εX − α − β , [ D , H , ] = − εX α + β + X − α M Γ = − Γ 0 0 0 0 − ε − Γ 2 0 2 ε − Γ 2 0 0 0 0 − ε − ε − ε − Γ 0 0 0 0 0 00 0 ε − Γ 0 0 2 ε − − Γ 0 0 0 0 − ε − − Γ 0 0 00 0 0 0 0 0 − − Γ − ε − − ε − ε − Γ 01 0 0 0 0 0 − ε − Γ with det M Γ = Γ . Case 5.
For D = ε X α + β + b ε X − α − β + aH , we obtain [ D , X α ] = − aX α + b εX − α − β , [ D , X − α ] = εX α + β , [ D , X β ] = 2 aX β , [ D , X − β ] = − aX − β , [ D , X α + β ] = aX α + β − b εX − α , [ D , X − α − β ] = − εX α − aX − α − β , [ D , X α + β ] = 2 b εH , , [ D , X − α − β ] = − εH , , [ D , H , ] = − εX α + β + b εX − α − β , [ D , H , ] , = 0 and M Γ = − a − Γ 0 0 0 0 0 − ε a − Γ 0 0 0 0 0 0 0 00 0 a − Γ 0 ε − Γ 0 0 0 0 − ε
00 0 − b ε a − Γ 0 0 0 0 00 0 0 0 0 − a − Γ 0 0 0 0 b ε − a − Γ 0 0 00 0 0 0 0 0 0 − Γ b ε
00 0 0 2 b ε − ε − Γ 00 0 0 0 0 0 0 0 0 − Γ with det M Γ = Γ (Γ − a )(Γ + 2 a )(Γ + 4 b ) (cid:0) (Γ − a ) + b (cid:1) (cid:0) (Γ + a ) + b (cid:1) , a, b ≥ . Case 6.
For D = ηX β + ε X α + β + b ηX − β + b ε X − α − β we obtain [ D , X α ] = − ηX α + β + b εX − α − β , [ D , X − α ] = εX α + β − b ηX − α − β , [ D , X β ] = b ηH , , [ D , X − β ] = − ηH , , [ D , X α + β ] = b ηX α − b εX − α , [ D , X − α − β ] = − εX α + ηX − α , [ D , X α + β ] = 2 b εH , , [ D , X − α − β ] = − εH , , [ D , H , ] = − εX α + β + b εX − α − β , [ D , H , ] = − ηX β + 2 b ηX − β M Γ = − Γ 0 b η − ε − Γ 0 0 0 0 0 0 0 − η − η − Γ 0 ε − Γ 0 0 0 0 − ε
00 0 − b ε − Γ 0 η − Γ 0 0 0 2 b ηb ε − b η − Γ 0 0 00 0 0 0 0 0 0 − Γ b ε
00 0 0 2 b ε − ε − Γ 00 b η − η − Γ with det M Γ = Γ (Γ + 4 b )(Γ + 4 b ) (cid:0) Γ + ( b − b ) (cid:1) (cid:0) Γ + ( b + b ) (cid:1) , b ≥ b ≥ . Case 7.
For D = − X α − εX β + ε b X α + β − b X − α and [ D , X α ] = − εX α + β − b H , + b H , , [ D , X − α ] = εb X α + β + H , − H , , [ D , X β ] = − X α + β , [ D , X − β ] = − b X − α − β − εH , , [ D , X α + β ] = 2 b X β − X α + β , [ D , X − α − β ] = − εb X α + εX − α + 2 X − β − b X − α − β , [ D , X α + β ] = 2 b X α + β , [ D , X − α − β ] = 2 X − α − β − εb H , , [ D , H , ] = X α − εb X α + β − b X − α , [ D , H , ] = − X α − εX β + b X − α and M Γ = − Γ 0 0 0 0 0 − εb − − Γ 2 b − ε − ε − − Γ 2 b εb − − Γ 0 0 0 0 − εb
00 0 0 0 − Γ 0 ε − b b − Γ 2 0 0 00 0 0 0 0 − b − Γ 2 0 00 0 0 0 0 0 − b − Γ 0 0 − b − εb − Γ 0 b − − ε − Γ with det M Γ = Γ (Γ + 4 b ) , b > . References [1] V. I. Arnol’d.
Mathematical Methods of Classical Mechanics , volume 60 of
Graduate Textsin Mathematics . Springer, 1989.[2] E. Cordero, F. De Mari, K. Nowak, and A. Tabacco. Analytic features of reproducinggroups for the metaplectic representation.
Journal of Fourier Analysis and Applications ,12(2):157–180, 2006.[3] E. Cordero, F. De Mari, K. Nowak, and A. Tabacco. Reproducing groups for the meta-plectic representation. In P. Boggiatto, L. Rodino, J. Toft, and M. W. Wong, editors,
Pseudo-Differential Operators and Related Topics , volume 12 of
Operator Theory: Ad-vances and Applications , pages 227–244. Birkhäuser Basel, 2006.314] E. Cordero, F. De Mari, K. Nowak, and A. Tabacco. Dimensional upper bounds for admissi-ble subgroups for the metaplectic representation.
Mathematische Nachrichten , 283(7):982–993, 2010.[5] W. Czaja and E. J. King. Isotropic shearlet analogs for L ( R k ) and localization operators. Numerical Functional Analysis and Optimization , 33(7-9):872–905, 2012.[6] W. Czaja and E. J. King. Anisotropic shearlet transforms for L ( R k ) . MathematischeNachrichten , 287(8-9):903–916, 2014.[7] S. Dahlke, S. Häuser, G. Steidl, and G. Teschke. Shearlet coorbit spaces: traces andembeddings in higher dimensions.
Monatshefte für Mathematik , 169(1):15–32, 2013.[8] S. Dahlke, S. Häuser, and G. Teschke. Coorbit space theory for the Toeplitz shearlettransform.
International Journal of Wavelets, Multiresolution and Information Processing ,10(4), 2012.[9] S. Dahlke, G. Kutyniok, P. Maass, C. Sagiv, H.-G. Stark, and G. Teschke. The uncertaintyprinciple associated with the continuous shearlet transform.
International Journal onWavelets Multiresolution and Information Processing , 6:157–181, 2008.[10] S. Dahlke, G. Kutyniok, G. Steidl, and G. Teschke. Shearlet coorbit spaces and associatedBanach frames.
Applied and Computational Harmonic Analysis , 27(2):195–214, 2009.[11] S. Dahlke, G. Steidl, and G. Teschke. The continuous shearlet transform in arbitrary spacedimensions.
Journal of Fourier Analysis and Applications , 16(3):340–364, 2010.[12] S. Dahlke and G. Teschke. The continuous shearlet transform in higher dimensions: vari-ations of a theme. In C. W. Danellis, editor,
Group Theory: Classes, Representations andConnections, and Applications , pages 167–175. Nova Science Pub Inc, 2010.[13] F. De Mari and E. De Vito. Admissible vectors for mock metaplectic representations.
Applied Computational Harmonic Analysis , 34(2):163–200, 2013.[14] H. G. Feichtinger and K. Gröchenig. A unified approach to atomic decompositions viaintegrable group representations. In
Function spaces and applications, (Lund, 1986) , Vol.1302 of Lecture Notes in Mathematics, pages 52–73. Springer, 1988.[15] H. G. Feichtinger and K. Gröchenig. Banach spaces related to integrable group rep-resentations and their atomic decompositions, Part I.
Journal of Functional Analysis ,86(2):307–340, 1989.[16] H. G. Feichtinger and K. Gröchenig. Banach spaces related to integrable group representa-tions and their atomic decompositions. Part II.
Monatshefte für Mathematik , 108(2):129–148, 1989.[17] G. B. Folland.
A Course in Abstract Harmonic Analysis . CRC Press, 1995.[18] K. Gröchenig. Describing functions: atomic decompositions versus frames.
Monatsheftefür Mathematik , 112(1):1–42, 1991.[19] K. Gröchenig.
Foundations of Time-Frequency Analysis . Birkhäuser, 2001.[20] K. Gröchenig and M. Piotrowski. Molecules in coorbit spaces and boundedness of opera-tors.
Studia Mathematica , 192(1):61–77, 2009.3221] P. Grohs. Continuous shearlet frames and resolution of the wavefront set.
Monatshefte fürMathematik , 164(4):393–426, 2011.[22] K. Guo and D. Labate. Characterization and analysis of edges using the continuous shearlettransform.
SIAM Journal on Imaging Sciences , 2(3):959–986, 2009.[23] K. Guo, D. Labate, and W. Lim. Edge analysis and identification using the continuousshearlet transform.
Applied and Computational Harmonic Analysis , 27(1):24–46, 2009.[24] S. Häuser.
Shearlet Coorbit Spaces, Shearlet Transforms and Applications in Imaging .Dissertation, TU Kaiserslautern, 2014.[25] S. Häuser and G. Steidl. Convex multiclass segmentation with shearlet regularization.
International Journal of Computer Mathematics , 90(1):62–81, 2013.[26] E. Kaniuth and K. F. Taylor.
Induced Representations of Locally Compact Groups , volume197 of
Cambridge Tracts in Mathematics . Cambridge University Press, 2013.[27] E. J. King.
Wavelet and frame theory: frame bound gaps, generalized shearlets, Grassman-nian fusion frames, and p -adic wavelets . Dissertation, University of Maryland, CollegePark, 2009.[28] E. J. King, G. Kutyniok, and W.-Q. Lim. Image inpainting: theoretical analysis andcomparison of algorithms. In D. Van De Ville, V. K. Goyal, and M. Papadakis, editors, SPIE Optical Engineering + Applications , page 885802. International Society for Opticsand Photonics, Sept. 2013.[29] A. W. Knapp.
Lie Groups beyond an Introduction , volume 140. Birkhäuser, 2002.[30] G. Kutyniok and D. Labate, editors.
Shearlets: Multiscale Analysis for Multivariate Data .Springer, 2012.[31] R. S. S. Laugesen, N. Weaver, G. L. Weiss, and E. N. Wilson. A characterization ofthe higher dimensional groups associated with continuous wavelets.
Journal of GeometricAnalysis , 12(1):89–102, 2002.[32] E. Schulz and K. F. Taylor. Extensions of the Heisenberg group and wavelet analysis in theplane. In
Spline Functions and the Theory of Wavelets (Montreal, PQ, 1996) , volume 18of
CRM Proceedings and Lecture Notes , pages 217–225, American Mathematical Society.[33] J. Williamson. On the algebraic problem concerning the normal forms of linear dynamicalsystems.