Nematic order on the surface of a three-dimensional topological insulator
NNematic order on the surface of a three-dimensional topological insulator
Rex Lundgren,
1, 2
Hennadii Yerzhakov, and Joseph Maciejko
3, 4, 5 Department of Physics, The University of Texas at Austin, Austin, Texas 78712, USA Joint Quantum Institute, NIST/The University of Maryland, College Park, Maryland 20742, USA Department of Physics, University of Alberta, Edmonton, Alberta T6G 2E1, Canada Theoretical Physics Institute, University of Alberta, Edmonton, Alberta T6G 2E1, Canada Canadian Institute for Advanced Research, Toronto, Ontario M5G 1Z8, Canada (Dated: October 11, 2018)We study the spontaneous breaking of rotational symmetry in the helical surface state of three-dimensional topological insulators due to strong electron-electron interactions, focusing on time-reversal invariant nematic order. Owing to the strongly spin-orbit coupled nature of the surfacestate, the nematic order parameter is linear in the electron momentum and necessarily involves theelectron spin, in contrast with spin-degenerate nematic Fermi liquids. For a chemical potential atthe Dirac point (zero doping), we find a first-order phase transition at zero temperature betweenisotropic and nematic Dirac semimetals. This extends to a thermal phase transition that changesfrom first to second order at a finite-temperature tricritical point. At finite doping, we find atransition between isotropic and nematic helical Fermi liquids that is second order even at zerotemperature. Focusing on finite doping, we discuss various observable consequences of nematicorder, such as anisotropies in transport and the spin susceptibility, the partial breakdown of spin-momentum locking, collective modes and induced spin fluctuations, and non-Fermi liquid behaviorat the quantum critical point and in the nematic phase.
I. INTRODUCTION
Rotationally invariant Fermi liquids can spontaneouslydevelop spatial anisotropy as a result of strong electron-electron interactions, a possibility first considered byPomeranchuk [1]. In the simplest scenario, for suffi-ciently strong attractive interactions in the l = 2 an-gular momentum channel the ground state energy ofthe Fermi liquid is lowered by a spontaneous quadrupo-lar distortion of the Fermi surface, leading to transportanisotropies and non-Fermi liquid behavior [2]. Alter-natively, the resulting time-reversal and translationallyinvariant form of order, nematic order, can arise viathermal or quantum melting of translational symmetry-breaking stripe/smectic orders [3]. There is strong ex-perimental evidence for the existence of a nematic phasein quantum Hall states [4–11], high-temperature super-conductors [12–14], and Sr Ru O [15, 16]. On the the-ory side, nematic order has been studied in a wide va-riety of systems including quantum Hall states [17–29],graphene [30, 31], two- and three-dimensional systemswith quadratic band crossing [32, 33], three-dimensionalDirac semimetals [34], dipolar Fermi gases [35–37], high-temperature superconductors [38] and doped Mott insu-lators [39].The surface of 3D topological insulators offers a newtype of gapless matter, the 2D helical Dirac fermion,which differs qualitatively from conventional Fermi sys-tems due to the phenomenon of spin-momentum lock-ing [40, 41]. This begs the question whether criteria forelectronic instabilities and the nature of possible broken-symmetry states on the surface of a 3D topological insu-lator differ from those of conventional 2D Fermi systems.While previous work has focused largely on supercon-ducting [42–51] and time-reversal breaking [48, 52–61] instabilities, little attention has been devoted to nematicinstabilities, with the exception of Ref. [62] which studiesthe spontaneous breaking of a discrete rotation symme-try on the surface of a topological Kondo insulator withmultiple Dirac cones.Our focus here is the isotropic-nematic phase transi-tion on the surface of a 3D topological insulator witha single rotationally invariant Dirac cone. For an un-doped system (chemical potential at the Dirac point)one always has continuous rotational invariance in thelow-energy limit; for a doped system our theory couldapply to a number of experimentally realized topologi-cal insulators with very nearly circular Fermi surfaces,such as Bi Se [63], Bi Te Se [64], Sb x Bi − x Se Te [64],Bi . Sb . Te . Se . [65], Tl − x Bi x Se − δ [66], strained α -Sn on InSb(001) [67], and strained HgTe [68]. A phe-nomenological Landau Fermi liquid theory of the topo-logical surface state developed earlier by two of us showedthat an isotropic-nematic quantum phase transition canoccur in the doped system for a sufficiently negative valueof the l = 2 “projected” Landau parameter ¯ f [69], in fullanalogy with the standard Pomeranchuk instability. Inthis work we construct a field theory of the transition,investigate both the doped and undoped limits, and ex-tend our analysis to nonzero temperatures. In the dopedlimit we find a continuous transition already at zero tem-perature, with a breakdown of helical Fermi liquid be-havior at the quantum critical point and in the nematicphase, in analogy with the spin-degenerate problem [2].The nematic phase exhibits a partial breakdown of spin-momentum locking, in the sense that spin and momen-tum are no longer orthogonal to each other except atcertain discrete points on the Fermi surface. Other un-usual observable consequences of the spin-orbit couplednature of nematic order in this system include anisotropy a r X i v : . [ c ond - m a t . s t r- e l ] J a n in the in-plane spin susceptibility in the absence of time-reversal symmetry breaking and the generation of spinfluctuations from nematic fluctuations at finite frequency.At zero doping the isotropic-nematic transition is first-order at zero temperature and becomes continuous at afinite-temperature tricritical point.The paper is organized as follows. In Sec. II, we intro-duce our model and argue that strong spin-orbit couplingon the topological insulator surface warrants a novel typeof nematic order parameter that mixes charge and spindegrees of freedom. In Sec. III, we construct a mean-field theory of the isotropic-nematic transition at bothzero and finite temperature and discuss the consequencesof nematic order for electronic properties at the mean-field level. Sec. IV discusses fluctuation effects beyondmean-field theory, namely, collective modes and their ef-fect on electronic properties. A brief conclusion is givenin Sec. V. II. MODEL AND NEMATIC ORDERPARAMETER
In this section, we introduce our field-theoretic modelfor the isotropic-nematic transition on the surface of a3D topological insulator. We follow largely the approachof Ref. [2], with important caveats due to the presence ofstrong spin-orbit coupling, as will be seen below. Whilenematic order in 2D electron gases with Rashba spin-orbit coupling has been studied before [70, 71], such sys-tems have two degenerate concentric Fermi surfaces andare thus qualitatively distinct from the single, nondegen-erate helical Fermi surface considered here.The Hamiltonian that describes the noninteractinggapless surface state of a topological insulator with asingle Dirac cone is given by [40, 41] (in units where (cid:126) = k B = 1) H = (cid:90) d k (2 π ) ψ † k ( h ( k ) − µ ) ψ k , (1)where ψ k = ( ψ k ↑ , ψ k ↓ ) is a two-component Dirac spinor, v F is the Fermi velocity, µ is the chemical potential, and h ( k ) = v F ˆ z · ( σ × k ) = v F (cid:18) ike − iθ k − ike iθ k (cid:19) , (2)where σ is a vector of Pauli matrices, θ k = tan − ( k y /k x )and k = (cid:113) k x + k y . The Hamiltonian (1) has a continu-ous spatial SO (2) rotation symmetry, [ J z , h ( k ) − µ ] = 0,where J z = − i ∂∂θ k + 12 σ z , (3)is the z component of total angular momentum.In order to study the isotropic-nematic transition weneed a suitable microscopic definition of the nematic or-der parameter in terms of the fermionic fields ψ, ψ † . In general, nematic order is described by a quadrupolar or-der parameter Q ab which transforms as a real, tracelesssymmetric rank-two tensor under rotations [72]. Becauseof spin-orbit coupling, here the relevant rotations are si-multaneous rotations in real space and spin space, gen-erated by the total angular momentum (3). Therefore,unlike for spin rotationally invariant Fermi liquids [2] thenematic order parameter can involve both the spatial(charge) and spin degrees of freedom of the electron. Tolowest order in the electron momentum, the appropriategeneralization of the nematic order parameter consideredin Ref. [2] for spin rotationally invariant Fermi liquids tothe surface state of 3D topological insulators isˆ Q ab ( r ) = − ik A ψ † ( r )( σ a ↔ ∂ b + σ b ↔ ∂ a − δ ab σ · ↔ ∂ ) ψ ( r ) , (4)where a, b = 1 ,
2, and ↔ ∂ = ( ↔ ∂ x , ↔ ∂ y ) is a vector of sym-metrized derivatives whose action is defined as ψ † ↔ ∂ a ψ ≡ ( ψ † ∂ a ψ + ( ∂ a ψ † ) ψ ). This ensures ˆ Q ab ( r ) is a Hermi-tian operator. Finally, the parameter k A is defined dif-ferently depending on whether one is in the doped orundoped limit. We consider that four-fermion interac-tions, to be written out explicitly below, only act withina high-energy cutoff that can be converted to a momen-tum cutoff Λ by dividing by v F . In the undoped limit µ = 0, we define k A ≡ Λ and the order parameter is lo-cal in space. This order parameter was first introducedby one of us in the context of nematic instabilities ofthe Majorana surface state of superfluid He- B [73], andits 3D analog was proposed as an order parameter forparity-breaking phases of spin-orbit coupled bulk met-als [74, 75]. In the doped limit, defined as µ (cid:29) v F Λ,only (angular) degrees of freedom on the Fermi surfaceare relevant and we define k A ≡ | ∂ | [76].In the spirit of Ref. [2], we consider an attractive four-fermion interaction in the quadrupolar ( l = 2) channel, H int = − f (cid:90) d r Tr (cid:16) ˆ Q ( r ) (cid:17) , (5)where Tr denotes a trace over the spatial (nematic) in-dices a, b . The action in imaginary time is then S [ ψ † , ψ ] = (cid:90) /T dτ (cid:90) d r (cid:20) ψ † ( ∂ τ − iv F ˆ z · ( σ × ∂ ) − µ ) ψ − f (cid:16) ˆ Q ( r ) (cid:17) (cid:21) . (6)As our focus is the vicinity of the isotropic-nematic tran-sition, interactions in other angular momentum channelshave been ignored. Indeed, in the doped limit, as longas such interactions are less than the critical value for a l (cid:54) = 2 Pomeranchuk instability, they will simply lead to afinite renormalization of physical quantities such as theFermi velocity [69]. While the phenomenological LandauFermi liquid description does not strictly apply to theundoped case, we will assume in this case that interac-tions in l (cid:54) = 2 channels are sufficiently weak so there areno competing instabilities. III. MEAN-FIELD THEORY
To investigate a possible isotropic-nematic phase tran-sition in the action (6), we analyze it in the mean-field approximation. Introducing a real auxiliary scalarfield Q ab ( τ, r ) to decouple the four-fermion term via theHubbard-Stratonovich transformation, we have S [ ψ † , ψ, Q ab ] = (cid:90) /T dτ (cid:90) d r (cid:20) ψ † ( ∂ τ − iv F ˆ z · ( σ × ∂ ) − µ ) ψ − iQ ab k A ψ † ( σ a ↔ ∂ b + σ b ↔ ∂ a − δ ab σ · ↔ ∂ ) ψ + 1 f Tr( Q ) (cid:21) . (7)Assuming a uniform and static order parameter Q ab ( τ, r ) = ¯ Q ab , and integrating out the fermions, weobtain the following saddle-point free energy density, F ( ¯ Q ) = 2 f ¯ Q − TV (cid:88) ik n (cid:88) k ln (cid:2) ( k n − iµ ) + (cid:15) k ( ¯ Q ) (cid:3) , (8)where k n = (2 n + 1) πT , n ∈ Z is a fermionic Matsub-ara frequency. We have rotated the order parametersuch that ¯ Q = − ¯ Q = 0, ¯ Q = ¯ Q = ¯ Q withoutloss of generality (corresponding to the principal axes ofthe distorted Fermi surface being parallel to the x and y axes [77]), and (cid:15) k ( ¯ Q ) = (cid:115) ( (cid:15) k ) − Q(cid:15) k kk A cos 2 θ k + 4 ¯ Q (cid:18) kk A (cid:19) , (9)is the mean-field dispersion relation of fermionic quasi-particles in the nematic phase (for ¯ Q (cid:54) = 0), where (cid:15) k = v F k is the dispersion relation in the isotropicphase. This corresponds to an anisotropic Dirac cone(in the doped limit, (cid:15) k ( ¯ Q ) is only meant to model thedispersion of quasiparticles on the Fermi surface, with k ≈ k F ≡ µ/v F ). Here k A is to be understood in mo-mentum space, i.e., k A = Λ in the undoped limit and k A = k in the doped limit. Performing the sum overMatsubara frequencies, and ignoring constant terms, weobtain F ( ¯ Q ) = 2 f ¯ Q − T (cid:88) s (cid:90) d k (2 π ) ln (cid:16) e − ( s(cid:15) k ( ¯ Q ) − µ ) /T (cid:17) , (10)where s = ± V → ∞ . At zero temper-ature, Eq. (10) becomes the ground state energy density, E ( ¯ Q ) = 2 f ¯ Q − (cid:88) s (cid:90) d k (2 π ) | s(cid:15) k ( ¯ Q ) − µ | . (11)In the following our analysis is performed at constant µ . A. Undoped limit
We first evaluate the free energy density in the un-doped limit ( µ = 0). At zero temperature, we have E ( ¯ Q ) = 2 f ¯ Q − (cid:90) | k | < Λ d k (2 π ) (cid:15) k ( ¯ Q ) , (12)where we have imposed the momentum cutoff Λ. Theintegral over momentum can be performed exactly, andwe obtain E (∆) = v F Λ π (cid:20) ∆ λ − | ∆ − | E (cid:18) − − (cid:19)(cid:21) , (13)where E ( m ) is the complete elliptic integral of the sec-ond kind, and we define a dimensionless nematic orderparameter ∆ = 2 ¯ Q/v F Λ and a dimensionless interac-tion strength λ = 2 f Λ / π v F . A strongly first-orderisotropic-nematic transition is found at a critical value λ c ≈ .
13, with a jump of order one in the order param-eter ∆ at the transition, corresponding to a value of ¯ Q on the order of the high-energy cutoff v F Λ. This is tobe expected since ¯ Q has units of energy, and in the un-doped limit the only energy scale in the problem is thecutoff (the critical value of the interaction strength f isalso determined by the cutoff, since the interaction (5) isperturbatively irrelevant at the Dirac point). Expanding(13) in powers of ∆ in the limit | ∆ | (cid:28)
1, we find E (∆) − E (0) = v F Λ π (cid:20)(cid:18) λ − π (cid:19) ∆ + . . . (cid:21) , (14)hence the limit of metastability of the isotropic phase(corresponding to the divergence of the nematic suscep-tibility) is λ ∗ = 8 /π ≈ .
55, but this is preempted by thefirst-order transition at λ c ≈ .
13. The limit of metasta-bility of the nematic phase can be found numerically, andis λ ∗∗ ≈ . - ���������������� ℰ ( Δ ) - ℰ ( � ) ��� ��� ��� ��� ��� ��� Δ FIG. 1. First-order isotropic-nematic quantum phase transi-tion in the undoped limit ( µ = 0). Plots of the mean-fieldground state energy density E (∆) in units of v F Λ / π aregiven as a function of the dimensionless nematic order param-eter ∆, for λ < λ c (blue curve), λ = λ c (black curve), and λ > λ c (red curve), where λ is the dimensionless interactionstrength with critical value λ c ≈ .
31 at the transition. Theleading correction to linear dispersion is given by α = − . deviations from a perfectly linear dispersion (which arepresent anyway in real topological insulator materials).In other words, we replace v F in the noninteracting dis-persion (cid:15) k by a k -dependent Fermi velocity v F ( k ) = v F (cid:34) α (cid:18) k Λ (cid:19) + . . . (cid:35) , (15)with the dimensionless parameter α representing theleading correction. Such corrections are formally irrel-evant in the low-energy limit k (cid:28) Λ but affect the freeenergy [2, 27], which depends on the noninteracting dis-persion at all wavevectors up to the cutoff. In the pres-ence of such terms the energy density cannot be evaluatedanalytically and one must resort to numerical integration.A typical plot of the ground state energy density in thevicinity of the transition for nonzero α is given in Fig. 1.We have found that negative values of α reduce both thecritical interaction strength and order parameter jumpat the transition below their values for a strictly lineardispersion.The apperance of a first-order transition is somewhatsurprising, since Landau theory predicts a continuousisotropic-nematic transition in 2D (unlike in 3D, thereare no cubic invariants). Expanding the quasiparticledispersion relation (cid:15) k ( ¯ Q ) in powers of ¯ Q in Eq. (12), andperforming the integral over k , we obtain the Landau theory E (∆) − E (0) ? = v F Λ π (cid:34)(cid:18) λ − π (cid:19) ∆ + ∞ (cid:88) n =2 c n ∆ n (cid:35) , (16)where c n < n ≥
2. We have checked that theonly way to get a quartic term ∝ ∆ with positive coef-ficient is to consider a k -dependent Fermi velocity v F ( k )that becomes negative at a certain value of k below thecutoff Λ, in clear contradiction with the assumption ofa single Dirac point in the low-energy spectrum. There-fore, the Landau theory (16) is unbounded from belowfor sufficiently large ∆, in disagreement with the exactenergy density (13) which behaves qualitatively like inFig. 1. As a result, there must be nonanalytic terms inEq. (13), but missed by the Landau expansion around∆ = 0, that stabilize the energy density. Such nonana-lytic terms are ultimately responsible for the first-ordercharacter of the phase transition. In fact, for | ∆ | (cid:29) E (∆) − E (0) ≈ v F Λ π (cid:18) ∆ λ − π | ∆ | (cid:19) , | ∆ | (cid:29) . (17)Thus the energy density is stabilized at large ∆ by the“bare” (tree-level) mass term ∆ /λ , which grows fasterthan the negative | ∆ | term coming from the one-loopfermion determinant, i.e., the integral over quasiparticleenergies in Eq. (12). The latter is in fact negative for all∆. We note that a first-order Ising nematic transition atzero temperature was also found for a model of interact-ing electrons on the square lattice [78]. In this case vanHove singularities in the quasiparticle density of states,corresponding to Lifshitz transitions tuned by the valueof ¯ Q , are responsible for nonanalyticities in the energydensity and the first-order character of the transition.At finite temperature the free energy density in theundoped limit is given by F ( ¯ Q ) = 2 f ¯ Q − T (cid:88) s (cid:90) | k | < Λ d k (2 π ) ln (cid:16) e − s(cid:15) k ( ¯ Q ) /T (cid:17) . (18)In the remainder of this section we focus on the limit ofstrict linear dispersion v F ( k ) = v F . The integral over themagnitude of k can be evaluated analytically in terms ofdilogarithms Li ( x ) and trilogarithms Li ( x ); the remain-ing angular integral must be performed numerically. InFig. 2a we plot the jump ∆ c in the order parameter atthe transition as a function of temperature T . The jumpdecreases smoothly from its value at zero temperature,eventually vanishing above a certain temperature T TCP corresponding to a tricritical point; for
T > T
TCP thetransition is continuous (a similar behavior was foundin Ref. [78]). Since ∆ vanishes at the tricritical point,to find T TCP we expand the free energy density (10) inpowers of ∆. To describe the tricritical point we must ������������������������ Δ � ��� ��� ��� ��� ��� � / � � Λ (a) ������� �������������� ��������������� ���� ** � * ������������������ � / � � Λ ��� ��� ��� ��� ��� ��� ��� ��� λ (b) FIG. 2. (color online) Finite-temperature isotropic-nematic transition in the undoped limit ( µ = 0): (a) Jump in dimensionlessnematic order parameter at the first-order phase transition as a function of temperature; (b) Mean-field phase diagram in theplane of of temperature T and dimensionless interaction strength λ . A first-order transition (red line) at low temperature turnsinto a continuous transition (blue line) above a tricritical point (black dot). Dotted lines correspond to limits of metastabilityof the isotropic ( T ∗ ) and nematic ( T ∗∗ ) phases. expand to sixth order, F (∆ , T ) − F (0 , T ) = v F Λ π (cid:0) a ∆ + a ∆ + a ∆ (cid:1) , (19)where a , a , a are functions of T . We find that a > . (cid:46) T /v F Λ (cid:46) .
6, which comprises the tricriti-cal point (Fig. 2a). The tricritical point ( T TCP , λ
TCP )is found from the condition a = a = 0, from whichwe find T TCP /v F Λ ≈ .
35 and λ TCP ≈ .
23. The finite-temperature phase diagram is shown in Fig. 2b, in whichwe also plot the limits of metastability of the isotropic( T ∗ ) and nematic ( T ∗∗ ) phases. Note that the first-orderphase boundary and limits of metastability are obtainedfrom the numerically evaluated, exact free energy density(10) rather than from the Landau expansion (19), whichis accurate only in the vicinity of the continuous tran-sition. Strictly speaking, the finite-temperature phasetransition for T > T
TCP is a Kosterlitz-Thouless tran-sition and the nematic phase only exhibits quasi-long-range order at finite T (but is truly long-range orderedat T = 0).At the mean-field level, the nematic phase is a theoryof noninteracting Dirac quasiparticles with anisotropicdispersion, with Hamiltonian H MF = (cid:80) k ψ † k H k ψ k where H k = v F ˆ z · ( σ × k ) + ¯ Q ab Λ ( σ a k b + σ b k a − δ ab σ · k ) . (20)Without loss of generality we choose ¯ Q = ¯ Q = ¯ Q ,¯ Q = − ¯ Q = 0, and thus H k = v F ˆ z · ( σ × k ) + 2 ¯ Q Λ ( σ x k y + σ y k x ) . (21) The velocities in the x and y directions (i.e., parallel tothe principal axes of the nematic order parameter) at theDirac point are v x = v F | − ∆ | , v y = v F | | . (22)Away from ∆ = ±
1, the density of states remains linearnear the Dirac point, N ( (cid:15) ) ∝ | (cid:15) | . In the limit of strictlinear dispersion v F ( k ) = v F , the value ∆ = 1 (∆ = − x ( y ) and degeneratesinto the intersection of two planes, i.e., a quasi-1D Diracdispersion with formally infinite density of states. In thepresence of nonzero band curvature however [Eq. (15)],this degeneracy is lifted, and the flat direction acquires acubic dispersion at small momenta, (cid:15) k (∆ = 1) ≈ v F (cid:114) k y + α Λ k x , k → , (23)with k x and k y interchanged for ∆ = −
1. This corre-sponds to a density of states of the form N ( (cid:15) ) ∝ | (cid:15) | / near the Dirac point (cid:15) = 0.An interesting signature of the unusual type of nematicorder described here is anisotropy in the in-plane spinsusceptibility in the absence of any time-reversal sym-metry breaking. To compute the spin susceptibility weaugment the mean-field Hamiltonian matrix (33) with aZeeman term, δ H Z k = − gµ B B · σ , (24)where g is the g -factor, µ B is the Bohr magneton, and B is an in-plane magnetic field. To linear order in ∆, we F ( x ) FIG. 3. Plot of the dimensionless function F ( x ) defined inEq. (26). find χ xx ( T ) − χ yy ( T ) = g µ B Λ8 πv F F (cid:18) Tv F Λ (cid:19) ∆( T ) , (25)where χ ij ( T ) is the spin susceptibility tensor at tempera-ture T , ∆( T ) is the dimensionless nematic order param-eter at temperature T , and F is a smooth function oftemperature (Fig. 3) defined as F ( x ) = x (cid:90) /x dy (cid:104) sinh y + y (cid:16) y tanh y − (cid:17)(cid:105) sech y . (26)Thus anisotropy in the in-plane susceptibility is a directmeasure of nematic order. For T > T
TCP , the transitionis continuous (blue curve in Fig. 2b) thus ∆( T ) is smallnear T c and the expression (25) can be used in the vicinityof the transition. We thus expect χ xx ( T ) − χ yy ( T ) ∝ F (cid:18) T c v F Λ (cid:19) ∆( T ) ∝ ( T c − T ) β , (27)on the nematic side of the transition, for ( T c − T ) /T c (cid:28) β , which is1 / B. Doped limit
In the doped limit µ (cid:29) v F Λ, the cutoff is imposedaround the Fermi surface, (cid:90) | k − k F | < Λ d k (2 π ) ≡ (cid:90) k F +Λ k F − Λ dk k π (cid:90) π dθ k π , (28)where k F ≡ µ/v F is the (isotropic) Fermi momentumof noninteracting electrons. We obtain the ground stateenergy density (11) to leading order in Λ /k F as E ( ¯ Q ) − E (0) = (cid:18) f − N ( µ ) (cid:19) ¯ Q + N ( µ )4 µ ¯ Q + O ( ¯ Q ) , (29)where N ( µ ) = µ/ (2 πv F ) is the noninteracting density ofstates at the Fermi surface. Since the coefficient of the¯ Q term is positive, we therefore find a continuous quan-tum phase transition at a critical value of the interactionstrength f given by N ( µ ) f = 2 . (30)From general considerations we expect a line of finite- T Kosterlitz-Thouless phase transitions that terminates atthis quantum critical point. We note also that Eq. (30)corresponds precisely to the l = 2 Pomeranchuk criterion¯ F = − , (31)derived from a phenomenological Landau theory for thehelical Fermi liquid on the surface of a 3D topolog-ical insulator [69]. In this context the dimensionless“projected” Landau parameters ¯ F l are defined as ¯ F l = N ( µ ) f l for l ≥
1, where f l is the quasiparticle interac-tion strength in angular momentum channel l . The dif-ference in sign arises simply from the fact that in Eq. (5)an attractive interaction corresponds to f >
0, while inRef. [69] it corresponds to f < H MF = (cid:80) k ψ † k H k ψ k where H k = v F ˆ z · ( σ × k ) − µ + ¯ Q ab ( σ a ˆ k b + σ b ˆ k a − δ ab σ · ˆ k ) , (32)and ˆ k a = k a /k . Without loss of generality we choose¯ Q = ¯ Q = ¯ Q , ¯ Q = − ¯ Q = 0, and thus H k = v F ˆ z · ( σ × k ) − µ + 2 ¯ Q ( σ x ˆ k y + σ y ˆ k x ) . (33)Eq. (33) describes an anisotropic Fermi surface. Nearthe Fermi surface, the eigenstates have positive helicity(assuming µ >
0, thus above the Dirac point) and aregiven by | ψ + ( k ) (cid:105) = 1 √ (cid:32) ie iθ k f ( θ k , ∆ F ) e iθ k − ∆ F (cid:33) , (34)where we define f ( θ k , ∆ F ) ≡ (cid:113) F − F cos 2 θ k . (35)We introduce a new dimensionless order parameter ∆ F ≡ Q/µ for the doped limit. The expectation value s k ≡(cid:104) ψ + ( k ) | σ | ψ + ( k ) (cid:105) of the spin operator on the Fermi sur-face is in plane, with components s x k = (1 + ∆ F ) sin θ k f ( θ k , ∆ F ) , s y k = − (1 − ∆ F ) cos θ k f ( θ k , ∆ F ) , (36)thus nematic order affects the spin polarization on theFermi surface. To leading order in ∆ F , the angle δ ( θ k )between the spin vectors in the presence and absence ofnematic order is δ ( θ k ) ≈ ∆ F | sin 2 θ k | . (37)Thus except for four points on the Fermi surface θ k =0 , π/ , π, π/
2, spin and momentum are no longer or-thogonal (Fig. 4). However, one might naively think thatspin-momentum locking is preserved in the sense that thespin vector remains tangent to the Fermi surface even ifthe latter is distorted. This is not true: defining a unitvector ˆ t k tangent to the distorted Fermi surface (thatwinds around the Fermi surface clockwise), we haveˆ z · ( s k × ˆ t k ) ≈ ∆ F sin 2 θ k , (38)to leading order in ∆ F , thus the spin vector is tangent tothe Fermi surface only at four points, θ k = 0 , π/ , π, π/ χ xx − χ yy = 14 g µ B N ( µ ) Λ k F ∆ F , (39)to leading order in ∆ F . More conventional measures ofnematicity, such as anisotropy in the in-plane resistiv-ity [2, 79], apply here as well. Considering scattering onnonmagnetic impurities modelled by a collision time τ , acalculation of the conductivity using the Kubo formulaand impurity-averaged Green’s functions in the first Bornapproximation gives ρ xx − ρ yy ρ xx + ρ yy ≈ ∆ F , (40)to leading order in ∆ F and assuming weak disorder1 / ( µτ ) (cid:28)
1. By symmetry we anticipate an analogousresult in the undoped case.
FIG. 4. (color online) Partial breakdown of spin-momentumlocking in the nematic phase. Blue dashed line: Fermi surfacein the isotropic phase (∆ F = 0); orange dashed line: Fermisurface in the nematic phase (here shown for ∆ F = 0 . IV. FLUCTUATION EFFECTS
We now go beyond the mean-field level and investi-gate the effect of fluctuations in the vicinity of the quan-tum critical point in the doped limit k F (cid:29) Λ. FollowingRef. [2], we rewrite the order parameter in terms of thePauli matrices τ z and τ x ,ˆ Q = ψ † ∆ ψτ z + ψ † ∆ ψτ x , (41)where∆ = − i ( σ x ˆ ∂ x − σ y ˆ ∂ y ) , ∆ = − i ( σ x ˆ ∂ y + σ y ˆ ∂ x ) , (42)and we define ˆ ∂ ≡ ↔ ∂ / | ∂ | in the sense of Fourier trans-forms (see Eq. (4)). We can now rewrite the imaginary-time action in a vectorial form, S [ ψ † , ψ ] = (cid:90) /T dτ (cid:90) d r (cid:20) ψ † ˆ G − ψ − f ψ † ∆ ψ ) (cid:21) , (43)where ∆ = (∆ , ∆ ) andˆ G − = ∂ τ − iv F ˆ z · ( σ × ∂ ) − µ, (44)is the noninteracting Green’s operator. Introducing abosonic auxiliary field n = ( n , n ) to decouple the four-fermion term, we have S [ ψ † , ψ, n ] = (cid:90) /T dτ (cid:90) d r (cid:20) ψ † ( ˆ G − − n · ∆ ) ψ + 12 f n (cid:21) . (45)After integrating out the fermions to second order in n ,we find the effective action S eff [ n ] = 12 (cid:88) iq n , q n ( q , iq n ) T χ − ( q , iq n ) n ( − q , − iq n ) , (46)where the inverse propagator for the auxiliary field isgiven to lowest order in momentum q and Matsubarafrequency q n by χ − ij ( q , iq n ) = δ ij ( r + κq ) + M ij ( q , iq n ) . (47)Here r = f − − N ( µ ) / κ = N ( µ ) / (8 k F )gives it a finite velocity, and M ( q , iq n ) = is N ( µ ) (cid:90) π dφ π is − cos( φ − θ q ) × (cid:18) sin φ − sin 2 φ cos 2 φ − sin 2 φ cos 2 φ cos φ (cid:19) , (48)is a dynamical term where s ≡ q n / ( v F q ) and θ q is theangle between q and the x axis. Performing the integralover φ , we have M ( q , iq n ) = N ( µ )2 | s |√ s + 1 × (cid:20) − (cid:16)(cid:112) s + 1 − | s | (cid:17) ( σ z cos 4 θ q + σ x sin 4 θ q ) (cid:21) , (49)which, after a rotation of θ q by π/
4, gives the sameinverse propagator as for the spinless nematic Fermifluid [2]. The effective action (46) can be diagonalizedby a rotation n → n (cid:48) , χ − → χ (cid:48)− , where n (cid:48) ( q , iq n ) = R (4 θ q ) T n ( q , iq n )= (cid:32) ˆ d q · n ( q , iq n )ˆ z · (cid:16) ˆ d q × n ( q , iq n ) (cid:17) (cid:33) . (50)Here R ( φ ) = e − iσ y φ/ is an orthogonal rotation matrixand ˆ d q ≡ (cos 2 θ q , sin 2 θ q ). Thus n (cid:48) and n (cid:48) correspondto the longitudinal and transverse components of n , re-spectively. The transformed inverse propagator is χ (cid:48)− ( q , iq n ) = R (4 θ q ) T χ − ( q , iq n ) R (4 θ q )= (cid:18) χ (cid:48)− ( q , iq n ) 00 χ (cid:48)− ( q , iq n ) (cid:19) . (51)For small s , we have χ (cid:48)− ( q , iq n ) = r + κq + 2 N ( µ ) s + . . . , (52) χ (cid:48)− ( q , iq n ) = r + κq + N ( µ ) | s | + . . . (53) A. Collective modes
Since the inverse propagator of nematic fluctuations isthe same as in the spinless case, the number and disper-sion of collective modes, given by the conditiondet χ − ( q , iq n ) = 0 , (54)is also the same. Analytically continuing Eq. (52)-(53)to real frequencies iq n → ω + iδ , we find χ (cid:48)− ( q , ω ) = r + κq − N ( µ ) (cid:18) ωv F q (cid:19) , (55) χ (cid:48)− ( q , ω ) = r + κq − N ( µ ) iωv F q , (56)to leading order in ω/ ( v F q ). At criticality r → + , thecollective mode dispersions are ω ( q ) ≈ (cid:114) κ N ( µ ) v F q , ω ( q ) ≈ − iv F κ N ( µ ) q , (57)thus ω is an undamped z = 2 mode and ω is anoverdamped z = 3 mode. Since ω (cid:28) ω in the long-wavelength limit q →
0, the overdamped mode dominatesthe long-wavelength response and the dynamical criticalexponent at the transition is z = 3 [2].We note that although ω corresponds to longitudinalfluctuations of n , when projecting to the Fermi surfacethe longitudinal (11 and 22) components of the orderparameter (4) map to the transverse (12 and 21) compo-nents of the usual spinless nematic order parameter ψ † ( ∂ a ∂ b − δ ab ∂ ) ψ, (58)where the effectively spinless field ψ † creates electrons ofthe appropriate helicity on the Fermi surface (see Sec.S4 of the Supplemental Material in Ref. [69]). Likewise,under projection the transverse components of (4) aremapped to the longitudinal components of (58). Thus inthis sense ω ( ω ) is the transverse (longitudinal) mode,in accordance with the terminology of Ref. [2].In the nematic phase ( r < n = (¯ n,
0) without loss of generality. Near thecritical point where ¯ n is small, the leading change in theeffective action for fluctuations compared to the isotropicphase is to the uniform and static part ( q = iq n = 0) ofthe inverse propagator [2], χ − ( q , iq n )= (cid:18) | r | + κq + M ( q , iq n ) M ( q , iq n ) M ( q , iq n ) κq + M ( q , iq n ) (cid:19) , (59)i.e., the longitudinal (amplitude) mode δn acquires amass 2 | r | and the transverse (Goldstone) mode δn ismassless. Deep in the nematic phase (i.e., ¯ n not small),the q part and the dynamical part M ij will be modifiedfrom their form at ¯ n = 0, but our conclusions drawn fromthe small ¯ n limit will not be affected in a major way (forinstance, a finite ¯ n would lead to a difference κ ⊥ (cid:54) = κ (cid:107) instiffness for the amplitude and Goldstone modes). Thetwo eigenvalues χ − ⊥ and χ − (cid:107) of the inverse propagator(59) give the spectrum of collective modes in the nematicphase. The inverse transverse propagator, given by χ − ⊥ ( q , iq n ) = κq + N ( µ ) | s | cos θ q − N ( µ ) (cid:18) cos 4 θ q + N ( µ )16 | r | sin θ q (cid:19) s + O ( s ) , (60)corresponds to the gapless nematic Goldstone mode,which is overdamped due to Landau damping exceptalong the principal axes of the distorted Fermi surface( θ q = ± π/ , ± π/ Q = − ¯ Q (cid:54) = 0). Along those directionsthe inverse transverse propagator reduces to Eq. (52) andthe Goldstone mode disperses quadratically accordingto ω ( q ) in Eq. (57). Those undamped directions alsocorrespond to the Fermi surface momenta where spin-momentum locking is preserved (green dots in Fig. 4).The inverse longitudinal propagator is given by χ − || ( q , iq n ) = 2 | r | + κq + N ( µ ) | s | sin θ q + N ( µ ) (cid:18) cos 4 θ q + N ( µ )16 | r | sin θ q (cid:19) s + O ( s ) , (61)and describes gapped amplitude fluctuations, as ex-pected.Despite the number and dispersion of collective modesbeing formally the same as in the spinless nematic Fermifluid, their physical nature is very different: in the lat-ter case only charge degrees of freedom fluctuate, whilefluctuations of the spin-orbit-coupled nematic order pa-rameter (4) strongly mix charge and spin. An importantobservable consequence of this difference is that nematicfluctuations in the helical liquid considered here shouldstrongly couple to the spin sector. While static nematicorder does not break time-reversal symmetry and thuscannot induce a static spin polarization, nematic fluctua-tions can in principle induce spin fluctuations. To quan-tify this effect, one can use linear response: a nematicfluctuation δ n ( q , ω ) with momentum q and frequency ω should induce a spin fluctuation δ (cid:104) s ( q , ω ) (cid:105) with the samemomentum and frequency, δ (cid:104) s i ( q , ω ) (cid:105) ∝ Π Rij ( q , ω ) δn j ( q , ω ) , (62)if a suitably defined retarded spin-nematic susceptibilityΠ Rij ( q , ω ) is nonzero. An appropriate definition isΠ Rij ( r , t ) = − iθ ( t ) (cid:10)(cid:2) ( ψ † σ i ψ ) ( r ,t ) , ( ψ † ∆ j ψ ) ( , (cid:3)(cid:11) , (63)in real space and time, where ψ † σ ψ is the spin operatorand ψ † ∆ ψ is the operator that couples to nematic fluc-tuations in Eq. (45). Eq. (63) will differ in the isotropic p + q , ip n + iq n p , ip n q , iq n q , iq n i j j i k q ,ik n iq n q , iq n k , ik n k , ik n (b)(a) FIG. 5. One-loop diagrams for (a) the spin-nematic suscepti-bility [Eq. (64)]; (b) the electron self-energy [Eq. (70)]. and nematic phases; here we compute Π
Rij in the isotropicphase and find a nonzero result, but we expect a nonzeroresult in the nematic phase as well.In the Matsubara frequency domain, the spin-nematicsusceptibility is given by the bubble diagram in Fig. 5(a),Π ij ( q , iq n ) = TV (cid:88) p ,ip n Tr σ i G ( p + q , ip n + iq n ) × ∆ j ( p , p + q ) G ( p , ip n ) , (64)where∆ ( k , k (cid:48) ) ≡ σ x (cid:32) ˆ k x + ˆ k (cid:48) x (cid:33) − σ y (cid:32) ˆ k y + ˆ k (cid:48) y (cid:33) , (65)∆ ( k , k (cid:48) ) ≡ σ x (cid:32) ˆ k y + ˆ k (cid:48) y (cid:33) + σ y (cid:32) ˆ k x + ˆ k (cid:48) x (cid:33) , (66)are the Fourier transform of the nematic vertices (42) and G is the unperturbed electron Green’s function, given by G ( p , ip n ) = ip n + µ + v F ˆ z · ( σ × p )( ip n + µ ) − v F p . (67)The retarded spin-nematic susceptibility Π Rij ( q , ω ) is ob-tained from (64) by analytic continuation iq n → ω + iδ .We evaluate its imaginary part Π (cid:48)(cid:48) ij ( q , ω ) at zero temper-ature and in the long-wavelength q (cid:28) k F , low-energy | ω | (cid:28) µ limits. To leading order in ω/v F q , we findΠ (cid:48)(cid:48) ( q , ω ) = − Π (cid:48)(cid:48) ( q , ω ) ∼ ωv F cos 3 θ q , (68)Π (cid:48)(cid:48) ( q , ω ) = Π (cid:48)(cid:48) ( q , ω ) ∼ ωv F sin 3 θ q , (69)ignoring constant prefactors (we are only interested inshowing that the response does not vanish). Fromtime-reversal symmetry one can show that Π (cid:48)(cid:48) ij ( q , ω ) =Π (cid:48)(cid:48) ij ( − q , − ω ), which is obeyed since Eq. (68)-(69) are oddin both q and ω . Kramers-Kronig relations imply thatthe real part Π (cid:48) ij ( q , ω ) approaches a constant at low fre-quencies and has the same structure in momentum space.By virtue of Eq. (62), nematic fluctuations can thus in-duce spin fluctuations, by contrast with the spinless (orspin degenerate) nematic Fermi fluid.0 B. Helical non-Fermi liquid behavior
We now turn to the fermion self-energy on the Fermisurface. In the random phase approximation (RPA), i.e.,at the one-loop level, the self-energy is given by the dia-gram in Fig. 5(b),Σ( k , ik n ) = TV (cid:88) q ,iq n (cid:88) ij ∆ i ( k , k − q ) G ( k − q , ik n − iq n ) × ∆ j ( k − q , k ) χ ij ( q , iq n ) , (70)where χ ij is the propagator of nematic fluctuations givenin Eq. (47). Here we only consider the effect of longitudi-nal fluctuations (i.e., the z = 3 overdamped mode) whichare expected to dominate at low energies. At the criticalpoint r = 0, we findΣ( k , ik n ) = (cid:16) z · ( σ × ˆ k ) (cid:17) Σ ( k , ik n ) , (71)for | k − k F | (cid:28) k F and | k n | (cid:28) µ , whereΣ ( k , ik n ) = − iω / | k n | / sgn k n , (72)and ω ∼ N ( µ ) − ( v F κ ) − , ignoring factors of order one.Near the Fermi surface, we can ignore the lower helicitybranch (assuming µ >
0) and the electron Green’s func-tion G ( k , ik n ) = (cid:2) G ( k , ik n ) − − Σ( k , ik n ) (cid:3) − is givenapproximately by G ( k , ik n ) ≈
12 1 + ˆ z · ( σ × ˆ k )2 iω / | k n | / sgn k n − ξ k , (73)where ξ k = v F | k | − µ . Thus to a first approximation thecritical Green’s function retains the same helicity struc-ture as in the noninteracting limit, G ( k , ik n ) ≈
12 1 + ˆ z · ( σ × ˆ k ) ik n − ξ k , (74)but exhibits non-Fermi liquid behavior with vanishingquasiparticle residue as ω →
0. The spectral function isof the form A ( k , ω ) ∼ (cid:16) z · ( σ × ˆ k ) (cid:17) ω / | ω | / ξ k , (75)in the limit ω / | ω | / (cid:28) | ξ k | (cid:28) µ . Apart from thehelicity structure, this is fully analogous to the spinlesscase [2]. In analogy with Ref. [80], we conjecture that thetransverse ( z = 2) fluctuations will give a finite anoma-lous dimension η to the electron propagator, replacingthe denominator ξ k in Eq. (75) by | ξ k | − η .In the nematic phase, the longitudinal modes aregapped [see Eq. (61)] and one must look at the effectof the transverse Goldstone modes described by the in-verse propagator (60). Because the symmetry generator J z that is broken in the nematic phase does not commute with translations, on general grounds one expects non-Fermi liquid behavior in the nematic phase as well [81].By contrast with the electron self-energy at the criticalpoint (71)-(72) however, we expect the self-energy in thenematic phase to reflect the broken rotational symmetry.To estimate the self-energy in the nematic phase, weobserve that on the Fermi surface | k | = k F , the electronGreen’s function appearing in Eq. (70) can be approxi-mated by G ( k − q , ik n − iq n ) ≈
12 1 + ˆ z · ( σ × ˆ k ) ik n − iq n + v F ˆ k · q , (76)since the momentum q of the collective mode is muchsmaller than the Fermi momentum. Here we assume weare close to the quantum critical point such that the dis-tortion of the Fermi surface is small and can be neglectedin the calculation of the self-energy; this is an O (¯ n ) effect,and can be understood in mean-field theory (Sec. III B),whereas the breakdown of Fermi liquid theory in the ne-matic phase appears at “zeroth” order in ¯ n as will beseen. In the low-energy limit (i.e., on the Fermi surface k n →
0) Eq. (76) is peaked at θ q = θ k ± π/
2, thus inEq. (70) one can replace θ q in the Goldstone mode prop-agator (60) by θ k ± π/ χ − ⊥ ( q , iq n ) ≈ κq + N ( µ ) | s | cos θ k . (77)We obtainΣ( k , ik n ) = (1 − σ y cos 3 θ k − σ x sin 3 θ k ) | cos 2 θ k | − / × Σ ( k , ik n ) , (78)where Σ is defined in Eq. (72). Ignoring the lower he-licity branch, we obtain the Green’s function G ( k , ik n ) ≈
12 1 + ˆ z · ( σ × ˆ k )2 iω / | cos 2 θ k | / | k n | / sgn k n − ξ k , (79)and the spectral function A ( k , ω ) ∼ (cid:16) z · ( σ × ˆ k ) (cid:17) ω / | cos 2 θ k | / | ω | / ξ k , (80)which are analogous to the spinless results [2] apartfrom the helicity structure. Equations (78)-(80) hold forgeneric angles θ k (cid:54) = ± π/ , ± π/ θ k = ± π/ , ± π/
4, we find that after pro-jection to the upper helicity branch the self-energy scalesas ∼ | ω | / , as in Ref. [2], corresponding to long-livedquasiparticles along those directions. Equations (75) and(80) correspond to a “helical non-Fermi liquid” in whichthe destruction of long-lived quasiparticles over most (inthe nematic phase) or all (at the quantum critical point)of the Fermi surface coexists with a Berry phase of π inspin space.1 V. CONCLUSION
In this work, we have developed a field-theoretic de-scription of nematic order for a single Dirac cone on thesurface of a 3D topological insulator. Due to spin-orbitcoupling present in topological insulators, the nematicorder parameter for helical Fermi liquids involves bothspin and momentum, in contrast to the case of regularFermi liquids which just involves momentum. In the un-doped limit at zero temperature, we found a first-orderisotropic-nematic transition at the mean-field level, incontrast with the expectation of a continuous transitionbased on Landau theory. The transition becomes con-tinuous at a finite-temperature tricritical point. In thedoped limit the transition was found to be continuouseven at zero temperature. The spin-orbit coupled natureof nematic order was shown to lead to the partial break-down of spin-momentum locking on the distorted Fermisurface and anisotropy in the in-plane spin susceptibil-ity in both the doped and undoped limits. The numberand dispersion of collective modes in the doped limit, as well as the prediction of non-Fermi liquid behavior at thequantum critical point and in the nematic phase, wereseen to be the same as for spin rotationally invariant ne-matic Fermi fluids. However, in the helical case it wasshown that nematic fuctuations can induce spin fluctua-tions, owing once again to the spin-orbit coupled natureof nematic order in these systems.
ACKNOWLEDGMENTS
R.L. thanks D. Lorshbough, L. Janssen, V. Chua, P.Nyugen, L. Savary and G. Fiete for fruitful discussionsand is particularly indebted to M. Edalati for providingan introduction to nematic phases. R.L. is supported byNational Science Foundation Graduate Fellowship awardnumber 2012115499 and a NIST NRC Research Postdoc-toral Associateship Award. J.M. is supported by NSERCgrant number RGPIN-2014-4608, the CRC Program, CI-FAR, and the University of Alberta. [1] I. Ya. Pomeranchuk, Zh. Eksp. Teor. Fiz. , 524 (1958)[Sov. Phys. JETP , 361 (1959)].[2] V. Oganesyan, S. A. Kivelson, and E. Fradkin, Phys.Rev. B , 195109 (2001).[3] E. Fradkin, S. A. Kivelson, M. J. Lawler, J. P. Eisenstein,and A. P. MacKenzie, Annu. Rev. Condens. Matter Phys. , 153 (2010).[4] M. P. Lilly, K. B. Cooper, J. P. Eisenstein, L. N. Pfeiffer,and K. W. West, Phys. Rev. Lett. , 394 (1999).[5] R. Du, D. Tsui, H. Stormer, L. Pfeiffer, K. Baldwin, andK. West, Solid State Commun. , 389 (1999).[6] W. Pan, R. R. Du, H. L. Stormer, D. C. Tsui, L. N.Pfeiffer, K. W. Baldwin, and K. W. West, Phys. Rev.Lett. , 820 (1999).[7] M. P. Lilly, K. B. Cooper, J. P. Eisenstein, L. N. Pfeiffer,and K. W. West, Phys. Rev. Lett. , 824 (1999).[8] K. B. Cooper, M. P. Lilly, J. P. Eisenstein, L. N. Pfeiffer,and K. W. West, Phys. Rev. B , 241313 (2002).[9] J. Xia, V. Cvicek, J. P. Eisenstein, L. N. Pfeiffer, andK. W. West, Phys. Rev. Lett. , 176807 (2010).[10] J. Xia, J. P. Eisenstein, L. N. Pfeiffer, and K. W. West,Nat. Phys. , 845 (2011).[11] N. Samkharadze, K. A. Schreiber, G. C. Gardner, M. J.Manfra, E. Fradkin, and G. A. Cs´athy, Nat. Phys. ,191 (2016).[12] M. Vojta, Adv. Phys. , 699 (2009).[13] R. M. Fernandes, A. V. Chubukov, and J. Schmalian,Nat. Phys. , 97 (2014).[14] H.-H. Kuo, J.-H. Chu, J. C. Palmstrom, S. A. Kivelson,and I. R. Fisher, Science , 958 (2016).[15] R. A. Borzi, S. A. Grigera, J. Farrell, R. S. Perry, S. J. S.Lister, S. L. Lee, D. A. Tennant, Y. Maeno, and A. P.Mackenzie, Science , 214 (2007).[16] C. M. Puetter, J. G. Rau, and H.-Y. Kee, Phys. Rev. B , 081105 (2010). [17] E. Fradkin and S. A. Kivelson, Phys. Rev. B , 8065(1999).[18] E. Fradkin, S. A. Kivelson, E. Manousakis, and K. Nho,Phys. Rev. Lett. , 1982 (2000).[19] C. Wexler and A. T. Dorsey, Phys. Rev. B , 115312(2001).[20] C. Wexler and O. Ciftja, J. Phys. Condens. Matter ,3705 (2002).[21] L. Radzihovsky and A. T. Dorsey, Phys. Rev. Lett. ,216802 (2002).[22] Q. M. Doan and E. Manousakis, Phys. Rev. B , 195433(2007).[23] M. Mulligan, C. Nayak, and S. Kachru, Phys. Rev. B , 085102 (2010).[24] M. Mulligan, C. Nayak, and S. Kachru, Phys. Rev. B , 195124 (2011).[25] J. Maciejko, B. Hsu, S. A. Kivelson, Y. Park, and S. L.Sondhi, Phys. Rev. B , 125137 (2013).[26] Y. You and E. Fradkin, Phys. Rev. B , 235124 (2013).[27] Y. You, G. Y. Cho, and E. Fradkin, Phys. Rev. X ,041050 (2014).[28] Y. You, G. Y. Cho, and E. Fradkin, Phys. Rev. B ,205401 (2016).[29] N. Regnault, J. Maciejko, S. A. Kivelson, and S. L.Sondhi, arXiv:1607.02178.[30] C. A. Lamas, D. C. Cabra, and N. Grandi, Phys. Rev.B , 075108 (2009).[31] B. Valenzuela and M. A. H. Vozmediano, New J. Phys. , 113009 (2008).[32] K. Sun, H. Yao, E. Fradkin, and S. A. Kivelson, Phys.Rev. Lett. , 046811 (2009).[33] L. Janssen and I. F. Herbut, Phys. Rev. B , 045117(2015).[34] R.-X. Zhang, J. A. Hutasoit, Y. Sun, B. Yan, C. Xu, andC.-X. Liu, Phys. Rev. B , 041108 (2016). [35] B. M. Fregoso and E. Fradkin, Phys. Rev. Lett. ,205301 (2009).[36] B. M. Fregoso, K. Sun, E. Fradkin, and B. L. Lev, NewJ. Phys. , 103003 (2009).[37] C.-K. Chan, C. Wu, W.-C. Lee, and S. Das Sarma, Phys.Rev. A , 023602 (2010).[38] S. A. Kivelson, E. Fradkin, and T. H. Geballe, Phys.Rev. B , 144505 (2004).[39] S. A. Kivelson, E. Fradkin, and V. J. Emery, Nature , 550 (1998).[40] M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. , 3045(2010).[41] X.-L. Qi and S.-C. Zhang, Rev. Mod. Phys. , 1057(2011).[42] L. Santos, T. Neupert, C. Chamon, and C. Mudry, Phys.Rev. B , 184502 (2010).[43] B. Roy, V. Juriˇci´c, and I. F. Herbut, Phys. Rev. B ,041401 (2013).[44] R. Nandkishore, J. Maciejko, D. A. Huse, and S. L.Sondhi, Phys. Rev. B , 174511 (2013).[45] S. Das Sarma and Q. Li, Phys. Rev. B , 081404 (2013).[46] T. Grover, D. N. Sheng, and A. Vishwanath, Science , 280 (2014).[47] P. Ponte and S.-S. Lee, New J. Phys. , 013044 (2014).[48] T. Neupert, S. Rachel, R. Thomale, and M. Greiter,Phys. Rev. Lett. , 017001 (2015).[49] W. Witczak-Krempa and J. Maciejko, Phys. Rev. Lett. , 100402 (2016).[50] N. Zerf, C.-H. Lin, and J. Maciejko, Phys. Rev. B ,205106 (2016).[51] S.-K. Jian, C.-H. Lin, J. Maciejko, and H. Yao,arXiv:1609.02146 (2016).[52] C. Xu, Phys. Rev. B , 020411 (2010).[53] C. Xu, Phys. Rev. B , 054403 (2010).[54] K.-S. Kim and T. Takimoto, Phys. Rev. B , 245138(2011).[55] J.-H. Jiang and S. Wu, Phys. Rev. B , 205124 (2011).[56] P. Ghaemi and S. Ryu, Phys. Rev. B , 075111 (2012).[57] F. S. Nogueira and I. Eremin, Phys. Rev. B , 085126(2013).[58] M. Sitte, A. Rosch, and L. Fritz, Phys. Rev. B , 205107(2013).[59] Y. Bahri and A. C. Potter, Phys. Rev. B , 035131(2015).[60] D. Mendler, P. Kotetes, and G. Sch¨on, Phys. Rev. B ,155405 (2015).[61] E. V. Gorbar, V. A. Miransky, I. A. Shovkovy, and P. O.Sukhachov, Phys. Rev. B , 115429 (2016).[62] B. Roy, J. Hofmann, V. Stanev, J. D. Sau, and V. Gal-itski, Phys. Rev. B , 245431 (2015).[63] Y. Xia, D. Qian, D. Hsieh, L. Wray, A. Pal, H. Lin,A. Bansil, D. Grauer, Y. S. Hor, R. J. Cava, and M. Z.Hasan, Nat. Phys. , 398 (2009); D. Hsieh, Y. Xia,D. Qian, L. Wray, J. H. Dil, F. Meier, J. Osterwalder,L. Patthey, J. G. Checkelsky, N. P. Ong, A. V. Fedorov,H. Lin, A. Bansil, D. Grauer, Y. S. Hor, R. J. Cava,and M. Z. Hasan, Nature , 1101 (2009); Z.-H. Pan,E. Vescovo, A. V. Fedorov, D. Gardner, Y. S. Lee, S. Chu,G. D. Gu, and T. Valla, Phys. Rev. Lett. , 257004(2011).[64] M. Neupane, S.-Y. Xu, L. A. Wray, A. Petersen,R. Shankar, N. Alidoust, C. Liu, A. Fedorov, H. Ji, J. M.Allred, Y. S. Hor, T.-R. Chang, H.-T. Jeng, H. Lin,A. Bansil, R. J. Cava, and M. Z. Hasan, Phys. Rev. B , 235406 (2012); M. Neupane, S. Basak, N. Alidoust,S.-Y. Xu, C. Liu, I. Belopolski, G. Bian, J. Xiong, H. Ji,S. Jia, S.-K. Mo, M. Bissen, M. Severson, H. Lin, N. P.Ong, T. Durakiewicz, R. J. Cava, A. Bansil, and M. Z.Hasan, Phys. Rev. B , 165129 (2013).[65] S. Kim, S. Yoshizawa, Y. Ishida, K. Eto, K. Segawa,Y. Ando, S. Shin, and F. Komori, Phys. Rev. Lett. ,136802 (2014).[66] K. Kuroda, G. Eguchi, K. Shirai, M. Shiraishi, M. Ye,K. Miyamoto, T. Okuda, S. Ueda, M. Arita, H. Na-matame, M. Taniguchi, Y. Ueda, and A. Kimura, Phys.Rev. B , 205306 (2015).[67] A. Barfuss, L. Dudy, M. R. Scholz, H. Roth, P. H¨opfner,C. Blumenstein, G. Landolt, J. H. Dil, N. C. Plumb,M. Radovic, A. Bostwick, E. Rotenberg, A. Fleszar,G. Bihlmayer, D. Wortmann, G. Li, W. Hanke,R. Claessen, and J. Sch¨afer, Phys. Rev. Lett. ,157205 (2013).[68] O. Crauste, Y. Ohtsubo, P. Ballet, P. A. L. Delplace,D. Carpentier, C. Bouvier, T. Meunier, A. Taleb-Ibrahimi, and L. L´evy, arXiv:1307.2008.[69] R. Lundgren and J. Maciejko, Phys. Rev. Lett. ,066401 (2015).[70] E. Berg, M. S. Rudner, and S. A. Kivelson, Phys. Rev.B , 035116 (2012).[71] J. Ruhman and E. Berg, Phys. Rev. B , 235119 (2014).[72] P. G. de Gennes and J. Prost, The Physics of LiquidCrystals , International Series of Monographs on Physics(Clarendon Press, 1995).[73] Y. Park, S. B. Chung, and J. Maciejko, Phys. Rev. B , 054507 (2015).[74] L. Fu, Phys. Rev. Lett. , 026401 (2015).[75] M. R. Norman, Phys. Rev. B , 075113 (2015).[76] C. Wu, K. Sun, E. Fradkin, and S.-C. Zhang, Phys. Rev.B , 115103 (2007).[77] There is a π/ Q ab in Eq. (4) and the principal axes of thedistorted Fermi surface, or equivalently, the orientationof the effective spinless nematic order parameter that re-sults from projection to the Fermi surface [69].[78] I. Khavkine, C.-H. Chung, V. Oganesyan, and H.-Y.Kee, Phys. Rev. B , 155110 (2004).[79] H.-Y. Kee, E. H. Kim, and C.-H. Chung, Phys. Rev. B , 245109 (2003).[80] M. Garst and A. V. Chubukov, Phys. Rev. B , 235105(2010).[81] H. Watanabe and A. Vishwanath, Proc. Natl. Acad. Sci. , 16314 (2014).[82] D. L. Maslov and A. V. Chubukov, Phys. Rev. B ,045110 (2010).[83] M. J. Lawler, D. G. Barci, V. Fern´andez, E. Fradkin,and L. Oxman, Phys. Rev. B73