Probabilistic models of genetic variation in structured populations applied to global human studies
PProbabilistic models of genetic variation in structuredpopulations applied to global human studies
Wei Hao ∗ , Minsun Song ∗ + , and John D. Storey , †
1. Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, NJ 085442. Department of Molecular Biology, Princeton University, Princeton, NJ 08544 ∗ These authors contributed equally to this work + Present address: Division of Cancer Epidemiology and Genetics, National Cancer Institute,National Institutes of Health, Rockville, MD 20850 † To whom correspondence should be addressed: [email protected]
Contents A BSTRACT I NTRODUCTION M ETHODS R ESULTS D ISCUSSION F IGURES AND T ABLES S UPPLEMENTARY M ATERIAL S UPPLEMENTARY F IGURES AND T ABLES R EFERENCES a r X i v : . [ q - b i o . P E ] M a r BSTRACT
Modern population genetics studies typically involve genome-wide genotyping of individuals from a di-verse network of ancestries. An important problem is how to formulate and estimate probabilistic mod-els of observed genotypes that account for complex population structure. The most prominent workon this problem has focused on estimating a model of admixture proportions of ancestral populationsfor each individual. Here, we instead focus on modeling variation of the genotypes without requiring ahigher-level admixture interpretation. We formulate two general probabilistic models, and we proposecomputationally efficient algorithms to estimate them. First, we show how principal component analysis(PCA) can be utilized to estimate a general model that includes the well-known Pritchard-Stephens-Donnelly admixture model as a special case. Noting some drawbacks of this approach, we introducea new “logistic factor analysis” (LFA) framework that seeks to directly model the logit transformation ofprobabilities underlying observed genotypes in terms of latent variables that capture population struc-ture. We demonstrate these advances on data from the Human Genome Diversity Panel and 1000Genomes Project, where we are able to identify SNPs that are highly differentiated with respect tostructure while making minimal modeling assumptions. I NTRODUCTION
Understanding genome-wide genetic variation among individuals is one of the primary goals of mod-ern human genetics. Genome-wide association studies aim to identify genetic variants throughout theentire genome that are associated with a complex trait [1–3]. One of the major challenges in analyzingthese studies is the problem of spurious associations due to population structure [4], and methods todeal with this are still in development [5–7]. A related effort is underway to provide a comprehensive,genome-wide understanding of how genetic variation among humans is driven by evolutionary and de-mographic forces [8]. A rigorous characterization of this variation will lead to a better understanding ofthe history of migration, expand our ability to identify signatures of natural selection, and provide impor-tant insights into the mechanisms of human disease [9,10]. For example, the Human Genome DiversityProject (HGDP) is an international project that has genotyped a large collection of DNA samples fromindividuals distributed around the world, aiming to assess worldwide genetic diversity at the genomiclevel [10–12]. The 1000 Genomes Project (TGP) is comprehensively cataloging human genetic varia-tion by producing complete genome sequences of well over 1000 individuals of diverse ancestries [13].Systematically characterizing genome-wide patterns of genetic variation is difficult due to the nu-merous and complex forces driving variation. There is a fundamental need to provide probabilisticmodels of observed genotypes in the presence of complex population structure. A series of influen-2ial publications have proposed methods to estimate a model of admixture, where the primary focusis on the admixture proportions themselves [14–16], which in turn may produce estimates of the al-lele frequencies of every genetic marker for each individual. Here, we instead focus directly on theseindividual-specific allele frequencies, which gives us potential advantages in terms of accuracy andcomputational efficiency.We propose two flexible genome-wide models of individual-specific allele frequencies as well asmethods to estimate them. First, we develop a model that includes as special cases the aforementionedmodels; specifically, the Balding-Nichols (BN) model [17] and its extension to the Pritchard-Stephens-Donnelly (PSD) model [14]. However, we identify some limitations of our method to estimate this model.We therefore propose an alternative model based on the log-likelihood of the data that allows for rapidestimation of allele frequencies while maintaining a valid probabilistic model of genotypes.The estimate of the first model is based on principal component analysis (PCA), which is a tooloften applied to genome-wide data of genetic variation in order to uncover structure. One of the earliestapplications of PCA to population genetic data was carried out by Menozzi et al. [18]. Exploratoryanalysis of complex population structure with PCA has been thoroughly studied [18–22]. We showthat a particular application of PCA can also be used to estimate allele frequencies in highly structuredpopulations, although we have to deal with the fact that PCA is a real-valued operation and is notguaranteed to produce allele frequency estimates that lie in the unit interval [0,1].The estimate of the second model is based on a generalized factor analysis approaches that di-rectly model latent structure in observed data, including categorical data [23] in which genotypes areincluded. We utilize a factor model of population structure [24] in terms of nonparametric latent vari-ables, and we propose a method called “logistic factor analysis” (LFA) that extends the PCA perspectivetowards likelihood-based probabilistic models and statistical inference. LFA is shown to provide accu-rate and interpretable estimates of individual-specific allele frequencies for a wide range of populationstructures. At the same time, this proposed approach provides visualizations and numerical summariesof structure similar to that of PCA, building a convenient bridge from exploratory data analysis to prob-abilistic modeling.We compare our proposed methods to existing algorithms (ADMIXTURE [16] and fastStructure [25])and show that when the goal is to estimate all individual-specific allele frequencies, our proposed ap-proaches are conclusively superior in both accuracy and computational speed. We apply the proposedmethods to the HGDP and TGP data sets, which allows us to estimate allele frequencies of every SNPin an individual-specific manner. Using LFA, we are also able to rank SNPs for differentiation accordingto population structure based on the likelihoods of the fitted models. In both data sets, the most differ-entiated SNP is proximal to
SLC24A5 , and the second most differentiated SNP is proximal to
EDAR .Variation in both of these genes has been hypothesized to be under positive selection in humans. In3he TGP data set, the second most different SNP is rs3827760, which confers a missense mutation in
EDAR and has been recently experimentally validated as having a functional role in determining a phe-notype [26]. We also identify several SNPs that are highly differentiated in these global human studiesthat have recently been associated with diseases such as cancer, obesity, and asthma. M ETHODS
It is often the case that human and other outbred populations are “structured” in the sense that the geno-type frequencies at a particular locus are not homogeneous throughout the population [5]. Geographiccharacterizations of ancestry often explain differing genotype frequencies among subpopulations. Forexample, an individual of European ancestry may receive a particular genotype according to a probabil-ity different than an individual of Asian ancestry. This phenomenon has been observed not only acrosscontinents, but on very fine scales of geographic characterizations of ancestry. Recent studies haveshown that population structure in human populations is quite complex, occurring more on a continuousrather than a discrete basis [10]. We can illustrate the spectrum of structural complexity with Figure 1,which shows dendrograms of hierarchically clustered individuals from the HapMap (phase II), HGDP,and TGP data sets. The HapMap samples strongly indicate explicit membership of each individual toone of three discrete subpopulations (due to the intended sampling scheme). On the other hand, theclusterings of the HGDP and TGP individuals show a very complex configuration, more representativeof random sampling of global human populations.Let us introduce Z as an unobserved variable capturing an individual’s structure. Let x ij be theobserved genotype for SNP i and individual j ( i = 1 , . . . , m , j = 1 , . . . , n ), and assume that x ij iscoded to take the values , , . We will call the observed m × n genotype matrix X . For SNP i ,the allele frequency can viewed as a function of Z , i.e. π i ( Z ) . For a sampled individual j from anoverall population, we have “individual-specific allele frequencies” [27] defined as π ij ≡ π i ( z j ) at SNP i . Each value of π ij informs us as to the expectation of that particular SNP/individual pair, supposingwe observed a new individual at that locus with the same structure; i.e. E[ x ij ] / π ij . If an observedSNP genotype x ij is treated as a random variable, then under Hardy-Weinberg Equilibrium π ij servesto model x ij as a Binomial parameter: x ij ∼ Binomial (2 , π ij ) . The focus of this paper is on thesimultaneously estimation of all m × n π ij values.The flexible, accurate, and computationally efficient estimation of individual-specific allele frequen-cies is important for population genetic analyses. For example, Corona et al. (2013) [28] recentlyshowed that considering the worldwide distribution of allele frequencies of SNPs known to be asso-4iated with human diseases may be a fundamental component to understanding the relationship be-tween ancestry and disease. Testing for Hardy-Weinberg equilibrium reduces to testing whether thegenotype frequencies for SNP i follow probabilities π ij , π ij (1 − π ij ) , and (1 − π ij ) for all individuals j = 1 , . . . , n . It can be shown that the well-known F ST measure can be characterized for SNP i usingvalues of π ij , j = 1 , , . . . , n (Section 6.5). Finally, we have recently developed a test of associationthat corrects for population structure and involves the estimation of log (cid:16) π ij − π ij (cid:17) [29]. Therefore, flexibleand well-behaved estimates of the individual-specific allele frequencies π ij are needed for downstreampopulation genetic analyses.It is straightforward to write other models of population structure in terms of Z . For the Balding-Nichols model, each individual is assigned to a population, thus z j indicates individual j ’s populationassignment. For the Pritchard-Stephens-Donnelly (PSD) model, each individual is considered to be anadmixture of a finite set of ancestral populations. Following the notation of [14], we can write z j as avector with elements q kj , where k indexes the ancestral populations, and we constrain q kj to be between0 and 1 subject to (cid:80) k q kj = 1 . Assuming the PSD model allows us to write each π ij = (cid:80) k p ik q kj andleads to a matrix form: F = PQ , where F is the m × n matrix of allele frequencies with ( i, j ) entry π ij , P is the m × d matrix of ancestral population allele frequencies p ik , and Q is the d × n matrix ofadmixture proportions. The elements of P and Q are explicitly restricted to the range [0 , .The PSD model is focused on the matrices P and Q , which have standalone interpretations, but weaim instead to estimate all π ij with a high level of accuracy and computational efficiency. Writing thestructure of the allele frequency matrix F as a linear basis, we have: Model 1: F = ΓS (1)where Γ is m × d and S is d × n with d ≤ n . The d × n matrix S encapsulates the genetic populationstructure for these individuals since S is not SNP-specific. The m × d matrix Γ maps how the structure S is manifested in the allele frequencies. Operationally, each SNP’s allele frequency are a linear com-bination of the rows of S , where the linear weights for SNP i are contained in row i of Γ . We define thedimension d so that d = 1 corresponds to the case of no structure: when d = 1 , S = (1 , , . . . , and Γ is the column vector of marginal allele frequencies.This model is not necessarily the most effective way to estimate π ij when working in the context of aprobabilistic model or with the likelihood function given the data. Model 1 resembles linear regression,where the allele frequencies are treated as a real-valued response variable that is linearly dependent onthe structure. A version of regression for the case of categorical response variables (e.g., genotypes)with underlying probability parameters is logistic regression. We developed an approach we call “logisticfactor analysis”, which is essentially an extension of nonparametric factor analysis to { , , } valuedgenotype data. Much of the justification for LFA is similar to that of generalized linear models [30].5he log-likelihood is the preferred mathematical framework for representing the information thedata contain about unknown parameters [31]. Suppose that Hardy-Weinberg equilibrium holds suchthat x ij ∼ Binomial (2 , π ij ) . We can write the log-likelihood of the data for SNP i and individual j as: (cid:96) ( π ij | x ij ) = log (Pr( x ij | π ij )) ∝ log (cid:16) π x ij ij (1 − π ij ) − x ij (cid:17) = x ij log (cid:18) π ij − π ij (cid:19) + 2 log(1 − π ij ) . The log-likelihood of SNP i for all unrelated individuals is the sum: (cid:80) nj =1 (cid:96) ( π ij | x ij ) . The term log (cid:16) π ij − π ij (cid:17) is the logit function and is written as logit( π ij ) . logit( π ij ) is called the “natural parameter” or “canonicalparameter” of the Binomial distribution and is the key component of logistic regression. An immediatebenefit of working with logit( π ij ) is that it is real valued, which allows us to directly model logit( π ij ) witha linear basis.Let L be the m × n matrix with ( i, j ) entry equal to logit( π ij ) . We formed the following parameteri-zation of L : Model 2: L = AH (2)where A is m × d and H is d × n with d ≤ n . In this case we can write logit( π ij ) = d (cid:88) k =1 a ik h kj , where all parameters are free to span the real numbers R .We call the rows of H “logistic latent factors” or just “logistic factors” as they represent unobservedvariables that explain the inter-individual differences in allele frequencies. In other words, the logit ofthe vector of individual-specific allele frequencies for SNP i can be written as a linear combination ofthe rows of H : [logit( π i ) , . . . , logit( π in )] = logit( π i ) = d (cid:88) k =1 a ik h k , where h k is the k th row of H . Likewise, we can write: ( π i , . . . , π in ) = π i = exp (cid:104)(cid:80) dk =1 a ik h k (cid:105) (cid:104)(cid:80) dk =1 a ik h k (cid:105) . The relationship between our proposed LFA approach and existing approaches of estimating latentvariables in categorical data is detailed in Section 6.6. Specifically, it should be noted that even thoughwe propose calling the approach “logistic factor analysis”, we do not make any assumptions about thedistribution of the factors (which are often assumed to be Normal). A technically more detailed name ofthe method is a “logistic nonparametric linear latent variable model for Binomial data.”6 .2 Estimation and Algorithms
The two models presented earlier make minimal assumptions as to the nature of the structure. Forexample, in Model 1, both Γ nor S are real valued. This allows us to apply an efficient PCA-based algo-rithm directly to the genotype matrix X , obtaining estimates of (cid:101) F , (cid:101) Γ , and (cid:101) S . In essence, (cid:101) F is estimatedby forming the projection of X / onto the top d principal components of X with an explicit interceptfor the d = 1 case. One drawback of this approach is that because PCA is designed for continuousdata, we have to artificially constrain (cid:101) F to be in the range [0 , . However, we show below that (cid:101) F is stillan extremely accurate estimate of the allele frequencies F for all formulations of F considered here,including the PSD model. Algorithm 1 : Estimating F from PCA1. Let (cid:101) µ i be the sample mean of row i of X . Set x ∗ ij = x ij − (cid:101) µ i and let X ∗ be the m × n matrix with ( i, j ) entry x ∗ ij .2. Perform singular value decomposition (SVD) on X ∗ which decomposes X ∗ = U∆V T . Notethat the rows of ∆V T are the n row-wise principal components of X ∗ and U are the principalcomponent loadings.3. Let (cid:101) X ∗ d − be the projection of X ∗ on the top d − eigen-vectors of this SVD, (cid:101) X ∗ d − = U d − ∆ d − V T d − .4. Construct (cid:101) F ∗ by adding (cid:101) µ i to row i of (cid:101) X ∗ d − (for i = 1 , . . . , n ) and multiplying the resulting matrixby / . In mathematical terms, (cid:101) F ∗ = (cid:101) Γ (cid:101) S where7 Γ = (cid:101) µ U d − ∆ d − ... (cid:101) µ m = u δ · · · u ,d − δ d − (cid:101) µ u δ · · · u ,d − δ d − (cid:101) µ ... ... ... u m δ · · · u m,d − δ d − (cid:101) µ m , (cid:101) S = (cid:32) V T d − . . . (cid:33) = v v · · · v n v v · · · v n ... ... ... v ,d − v ,d − · · · v n,d − · · · , and δ i is the i th diagonal entry of ∆ . Let (cid:101) π ∗ ij to be the ( i, j ) entry of (cid:101) F ∗ .5. Since it may be the case that some (cid:101) π ∗ ij are such that (cid:101) π ∗ ij < or (cid:101) π ∗ ij > , we truncate these. Thefinal PCA based estimate of F is formed as (cid:101) F where the ( i, j ) entry (cid:101) π ij is defined to be (cid:101) π ij = C if (cid:101) π ∗ ij ≤ C (cid:101) π ∗ ij if C < (cid:101) π ∗ ij < − C − C if (cid:101) π ∗ ij ≥ − C for some C (cid:38) . An estimate of L can be formed as (cid:101) L = logit( (cid:101) F ) .Here we used C = n . In summary, (cid:101) F is a projection of X into its top principal components, scaled by / , and truncated so that all values lie in the interval (0 , .For Model 2, we propose a method for estimating the latent variables H . Starting from the (cid:101) F foundby Algorithm 1, we apply the logit transformation to the subset of rows where we did not have toadjust the values that were < or > , and then extract the right singular vectors of this transformedsubset. As long as the subset is large enough to span the same space as the row space of L , thisapproach accurately estimates the basis of H . Next, we calculate the maximum likelihood estimationof A parametrized by (cid:98) H to yield (cid:98) A and then (cid:98) L = (cid:98) A (cid:98) H . This involves performing a logistic regressionof each SNP’s data on (cid:98) H . In order to estimate the individual-specific allele frequency matrix F , we8alculate (cid:98) F = logit − ( (cid:98) L ) . An important property to note is that all (cid:98) π ij ∈ [0 , due to the fact that weare modeling the natural parameter. Algorithm 2 : Estimating Logistic Factors1. Apply Algorithm 1 to obtain the estimate (cid:101) F ∗ from Step 4.2. Recalling that (cid:101) π ∗ ij is the ( i, j ) entry of (cid:101) F ∗ , we choose some C (cid:38) and form S = { i : C < (cid:101) π ∗ ij < − C, ∀ j = 1 , ..., n } . S identifies the rows of (cid:101) F ∗ where the logit function can be applied stably. Here we use C = n .3. Define (cid:101) F S to be the corresponding subset of rows of (cid:101) F ∗ , and calculate (cid:101) L S = logit (cid:16)(cid:101) F S (cid:17) . Let (cid:101) L (cid:48)S be the row-wise mean centered and standard deviation scaled matrix (cid:101) L S .4. Perform SVD on (cid:101) L (cid:48)S resulting in (cid:101) L (cid:48)S = TΛW T . Set (cid:98) H to be the d × n matrix composed of thetop d − right singular vectors of the SVD of (cid:98) L (cid:48)S stacked on the row n -vector (1 , , · · · , : (cid:98) H = (cid:32) W T d − · · · (cid:33) = w w · · · w n w w · · · w n ... ... ... w ,d − w ,d − · · · w n,d − · · · . Algorithm 3 : Estimating F and L from LFA1. Apply Algorithm 2 to X to obtain (cid:98) H .2. For each SNP i , perform a logistic regression of the SNP genotypes x i = ( x i , x i , . . . , x in ) onthe rows of (cid:98) H , specifically by maximizing the log-likelihood (cid:96) ( π i | x i , (cid:98) H ) = n (cid:88) j =1 x ij log (cid:18) π ij − π ij (cid:19) + 2 log(1 − π ij ) under the constraint that logit( π ij ) = (cid:80) dk =1 a ik (cid:98) h kj . It should be noted that an intercept is includedbecause (cid:98) h dj = 1 ∀ j by construction. 9. Set (cid:98) a ij ( j = 1 , . . . , n ) to be equal to the maximum likelihood estimates from the above model fit,for each of i = 1 , . . . , m . Let (cid:98) L = (cid:98) A (cid:98) H , (cid:98) F = logit − ( (cid:98) L ) , and (cid:98) π ij be the ( i, j ) entry of (cid:98) F : (cid:98) π ij = exp (cid:110)(cid:80) dk =1 (cid:98) a ik (cid:98) h kj (cid:111) (cid:110)(cid:80) dk =1 (cid:98) a ik (cid:98) h kj (cid:111) . PCA-based estimation of Model 1 requires one application of singular value decomposition (SVD)and LFA requires two applications of SVD. We leverage the fact that n (cid:29) d to utilize Lanczos bidiago-nalization which is an iterative method for computing the singular value decomposition of a matrix [32].Lanczos bidiagonalization excels at computing a few of the largest singular values and correspondingsingular vectors of a sparse matrix. While the sparsity of genotype matrices is fairly low, we find thatin practice using this method to perform the above estimation algorithms is more effective than usingmethods that require the calculation of all the singular values and vectors. This results in a dramaticreduction of the computational time needed for the implementation of our methods. R ESULTS
We applied our methods to a comprehensive set of simulation studies and to the HGDP and TGP datasets.
To directly evaluate the performance of the estimation methods (Section 2.2), we devised a simulationstudy where we generated synthetic genotype data with varying levels of complexity in population struc-ture. Genotypes were simulated based on allele frequencies subject to structure from the BN model,the PSD model, spatially structure populations, and real data sets. For the first three types of simu-lations, the allele frequencies were parameterized by Model 1, while for the real data simulations, theallele frequencies were taken from model fits on the data themselves.A key property to assess is how well the estimation methods capture the overall structure. Oneway to evaluate this is to determine how well (cid:101) S from the PCA based method (Algorithm 1) estimatesthe true underlying S , and likewise how well (cid:98) H from LFA estimates the true H . Note that even thoughthe genotype data was generated from the F of Model 1, we can evaluate (cid:98) H by converting with L =logit( F ) . To evaluate PCA, we regressed each row of F on (cid:101) S and calculated the average R ; similarly,for LFA we regressed each row of L on (cid:98) H and calculated the average R value. The results arepresented in Table 1. Both methods estimate the true latent structure well.10e specifically note that when the PSD model was utilized to simulate structure, we were able torecover the structure S very well (Supplementary Figure 6) without needing to employ the computa-tionally intensive and assumption-heavy Bayesian model fitting techniques from ref. [14]. Additionally, itseems that the (cid:101) S largely captures the geometry of S where it may be the case that S can be recoveredwith a high degree of accuracy by transforming (cid:101) S back into the simplex. By comparing the results on thereal data (Figures 2-3) with the simulated data (Supplementary Figure 6), one is able to visually assesshow closely the assumptions of the PSD model resemble real data sets. When structure was simulatedthat differed substantially from the assumptions of the PSD model, our estimation methods were ableto capture that structure just as well (Supplementary Figure 7). This demonstrates the flexibility of theproposed approaches.We also compared PCA and LFA to two methods of fitting the PSD model, ADMIXTURE [16] andfastStructure [25], by seeing how well the methods estimated the individual specific allele frequencies π ij (Table 3). For the real data scenarios, we generated synthetic genotypes based on estimates of F from the four different methods, thus giving each method an opportunity to fit its own simulation.The methods were compared by computing three different error metrics with respect to the oracle F :Kullback-Leibler divergence, absolute error, and root mean squared error. PCA and LFA significantlyoutperformed ADMIXTURE and fastStructure, which confirms the intuitive understanding of the differ-ences between the models: the goal of Model 1 and 2 is to estimate the allele frequencies π ij , whilethe PSD model provides a probabilistic interpretation of the structure by modeling them as admixtureproportions.The computational time required to perform the proposed methods was also significantly better thanADMIXTURE and fastStructure. Both proposed methods completed calculations on average over 10times faster than ADMIXTURE and fastStructure, with some scenarios as high as 150 times faster.This is notable in that both ADMIXTURE and fastStructure are described as computationally efficientimplementations of methods to estimate the PSD model [16, 25]. We analyzed the HGDP and TGP data using the proposed methods. The HGDP data consisted of n =940 individuals and m = 431 , SNPs, and the TGP data consisted of n = 1500 and m = 339 , (see Supplementary Section 6.1 for details). We first applied PCA and LFA to these data sets and madebi-plots of the top three PCs and top three LFs (Figures 2 and 3). It can be seen that PCA and LFAprovide similar visualizations of the structure present in these data. We next chose a dimension d forthe LFA model (Model 2) for each data set. This was done by identifying the value of d that providesthe best overall goodness of fit with Hardy-Weinberg equilibrium (Supplementary Section 6.2). Weidentified d = 15 for HGDP and d = 7 for TGP based on this criterion.11ne drawback of utilizing a PCA based approach (Algorithm 1) for estimating the individual-specificallele frequencies F is that we are not guaranteed that all values of the estimates lie in [0 , , so someform of truncation is necessary. We found that 65.4% of the SNPs in the HGDP data set and 26.5%in the TGP data set resulted in at least one estimated individual-specific allele frequency < or > before the truncation was applied. Therefore, the truncation in forming the estimate (cid:101) F is necessarywhen employing Algorithm 1 to estimate F from Model 1. On the other hand, due to the formulation ofModel 2, all estimated allele frequencies fall in the valid range when applying LFA (Algorithms 2 and 3).The LFA framework provides a natural computational method for ranking SNPs according to howdifferentiated they are with respect to structure. Note that existing methods typically require one tofirst assign each individual to one of K discrete subpopulations [33] which may make unnecessaryassumptions on modern data sets such as HGDP and TGP. In order to rank SNPs for differentiation,we calculate the deviance statistic when performing a logistic regression of the SNPs genotypes on thelogistic factors. Specifically we calculated the deviance by comparing the models logit( π i ) = a id h d vs. logit( π i ) = (cid:80) dk =1 a ik h k , where the former model is intercept only (i.e., d = 1 , no structure).Our application of LFA to identify SNPs with allele frequencies differentiated according to structurecan be developed further. First, the recently proposed “jackstraw” approach [34] provides a manner inwhich statistical significance can be assigned to these SNPs. Assigning statistical significance to thepopulation differentiation of SNPs has traditionally been a difficult problem [35]. Second, we found thedeviance measure tends to have more extreme values for SNPs with larger minor allele frequencies(MAFs). Therefore, the ranking of SNPs may be made more informative if MAF is taken into account.Third, although this ranking is identifying differentiation and not specifically selection, it may provide auseful starting point in understanding methods that attempt to detect selection.The most differentiated SNPs (Supplementary Tables 4 and 5) reveal some noteworthy results,especially considering the flexible approach to forming the ranking. SNPs located within or very closeto SLC24A5 were the top ranked in both HGDP and TGP. This gene is well known to be involved indetermining skin pigmentation in humans [36] and is hypothesized to have been subject to positiveselection [37]. The next most highly ranked SNPs in both studies are located in
EDAR , which playsa major role in distinguishing phenotypes (e.g., hair follicles) among Asians. SNP rs3827760 is thesecond most differentiated SNP in the TGP data, which has also been hypothesized to be under positiveselection in humans and whose causal role in the hair follicle phenotype has been verified in a mousemodel [26]. SNPs corresponding to these two genes for both studies are plotted in increasing order of (cid:98) π ij values, revealing subtle variation within each major ancestral group in addition to coarser differencesin allele frequency (Figure 4). Other noteworthy genes with highly differentiated proximal SNPs include: • FOXP1 , which is a candidate gene for involvement in tumor progression and plays an importantregulatory role with
FOXP2 [38, 39]; 12
TBC1D1 in which genetic variation has been shown to confer risk for severe obesity in females[40]; • KIF3C , a novel kinesin-like protein, which has been hypothesized to be involved in microtubule-based transport in neuronal cells [41]; • KCNMA1 , a recently identified susceptibility locus for obesity [42]; • CTNNA3 in which genetic variation has been shown to be associated with diisocyanate-inducedoccupational asthma [43]; • PTK6 , breast tumor kinase (Brk), which is known to function in cell-type and context-dependentprocesses governing normal differentiation [44].We have provided information on the 5000 most differentiated SNPs for both TGP and HGDP in sup-plementary files.
An R package called lfa is available at https://github.com/StoreyLab/lfa . D ISCUSSION
We have investigated two latent variable models of population structure to simultaneously estimate allindividual-specific allele frequencies from genome-wide genotyping data. Model 1, a direct model ofallele frequencies, can be estimated by using a modified PCA and Model 2, a model of the logit trans-formation of allele frequencies, is estimated through a new approach we called “logistic factor analysis”(LFA). For both models, the latent variables are estimated in a nonparametric fashion, meaning we donot make any assumptions about the underlying structure captured by the latent variables. These mod-els are general in that they allow for each individual’s genotype to be generated from an allele frequencyspecific to that individual, which includes discretely structured populations, admixed populations, andspatially structured populations. In LFA, we construct a model of the logit of these allele frequencies interms of underlying factors that capture the population structure. We have proposed a computationallyefficient method to estimate this model that requires only two applications of SVD. This approach buildson the success of PCA in that we are able to capture population structure in terms of a low-dimensionalbasis. It improves on PCA in that the latent variables we estimate can be straightforwardly incorpo-rated into downstream statistical inference procedures that require well-behaved estimates of allele13requencies. In particular, statistical inferences of Hardy-Weinberg equilibrium, F ST , and marker-traitassociations are amenable to complex population structures within our framework.We demonstrated our proposed approach on the HGDP and TGP data sets and several simulateddata sets motivated by the HapMap, HGDP, and TGP data sets as well as the PSD model and spatiallydistributed structures. It was shown that our method estimates the underlying logistic factors with ahigh degree of accuracy. We also showed that applying PCA to genotype data estimates a row basisof population structure on the original allele frequency scale to a high degree of accuracy. However,problems occur when trying to recover estimates of individual-specific allele frequencies because PCAis a real-valued model that does not always result in allele frequency estimates lying between 0 and 1.Although PCA has become very popular for genome-wide genotype data, it should be stressed thatPCA is fundamentally a method for characterizing variance and special care should be taken whenapplying it to estimate latent variables. The authoritative treatment of PCA [45] eloquently makes thispoint throughout the text and considers cases where factor analysis is more appropriate than PCAthrough examples reminiscent of the population structure problem. Here, we have shown that modelingand estimating population structure can be understood from the factor analysis perspective, leading toestimates of individual-specific allele frequencies through their natural parameter on the logit scale. Atthe same time, we have avoided some of the difficulties of traditional parameteric factor analysis bymaintaining the relevant nonparametric properties of PCA, specifically in making no assumptions aboutthe underlying probability distributions of the logistic factors that capture population structure.14 F IGURES AND T ABLES
HapMap lll
YRICEUJPT+CHB
Human Genome Diversity Project (HGDP) lllllll
AFRICAAMERICACENTRAL_SOUTH_ASIAEAST_ASIAEUROPEMIDDLE_EASTOCEANIA lllllll
AFRICA_IN_AFRICAAFRICA_IN_AMEAST_ASIA_IN_AM/ASIAEUROPE_IN_AM/EUMEXICO_IN_AMSOUTH_AMERICASOUTH_ASIA_IN_AM/ASIA
Figure 1: A hierarchical clustering of individuals from the HapMap, HGDP, and TGP data sets. A den-drogram was drawn from a hierarchical clustering using Ward distance based on SNP genotypes (MAF > ). Whereas the HapMap project shows a definitive discrete population structure (by samplingdesign), the HGDP and TGP data show the complex structure of human populations.15 lll lll ll l llllllllll lll lll ll ll lll l lll lll llllll l lll lllll llll llllllllllllll ll llll lll ll ll ll ll l ll llll ll ll ll lll lll llllll l ll ll ll ll ll llll l lll llllll ll lll l lll l ll ll l lll lll l lll llllllll ll lll lll lllllllll l llllll llllll llllllllll l lll l lllllllllllllllll llll l l lllll lllll ll l l lllllllllll lllll ll l lllll l lll lll lllll llll ll lllll ll lll lll llll llllllllll lllllllllll llll lll lllllllllll ll lll ll ll ll ll ll lll llll lll lll ll l llll lllllllll lll lllll lll l ll ll l lllllll llllllllll lll ll ll ll l l llllll lll l llll llllllllllllllllll llll lllll lll l lll ll lllll llll lll lllll l ll ll ll ll ll l lll l llllllll lllll lllllll ll llll lll ll llllllllllll ll ll ll llll ll l ll llllllllllll llllllll llll lll llll llll l l ll lll l ll ll ll lll ll lll ll lll llllllllllllll llllll ll lll ll llll lllll lllll llllllllllll l lll lll llll ll llllllllll lll lll ll l ll llllllll llll lllllll l llllll llllll llll llllll ll ll llllll ll l lll ll ll l lllll l ll ll ll ll lll ll ll ll llll llllllll lllllllllll ll l lllll lll l lllllll llll lll lll l llllll llll l llllll llll llll lllllllllll llll ll lll lllllllllll ll llll lll llll PC1 P C l lll lll ll l llllllllll lll lll ll ll lll l lll lll llllll l lll lllll llll llllllllllllll ll llll lll ll ll ll ll l ll llll ll ll ll lll lll llllll l ll ll ll ll ll llll l lll llllll ll lll l lll l ll ll l lll lll l lll llllllll ll lll lll lllllllll l llllll llllll llllllllll l lll l lllllllllllllllll llll l l lllll lllll ll l l lllllllllll lllll ll l lllll l lll lll lllll llll ll lllll ll lll lll llll llllllllll lllllllllll llll lll lllllllllll ll lll ll ll ll ll ll lll llll lll lll ll l llll lllllllll lll lllll lll l ll ll l lllllll llllllllll lll ll ll ll l l llllll lll l llll llllllllllllllllll llll lllll lll l lll ll lllll llll lll lllll l ll ll ll ll ll l lll l llllllll lllll lllllll ll llll lll ll llllllllllll ll ll ll llll ll l ll llllllllllll llllllll llll lll llll llll l l ll lll l ll ll ll lll ll lll ll lll llllllllllllll llllll ll lll ll llll lllll lllll llllllllllll l lll lll llll ll llllllllll lll lll ll l ll llllllll llll lllllll l llllll llllll llll llllll ll ll llllll ll l lll ll ll l lllll l ll ll ll ll lll ll ll ll llll llllllll lllllllllll ll l lllll lll l lllllll llll lll lll l llllll llll l llllll llll llll lllllllllll llll ll lll lllllllllll ll llll lll llll PC1 P C l lll lll l ll llllllllll lll llll l ll llll lll lll ll llllllll lll ll lll llllllllllll llll ll lllll ll ll ll lll l ll llll ll ll ll lll lll lllll ll ll llll l lllllll l ll lllllll ll lll l ll ll ll l llll ll ll l lll lllllll lll l lll llllllllllll llllll llllll llllllllll l lll l lllllllllllllllll lllll l llll llll lll ll l llllllllll lll ll lll l llllll lll lll lllll llll ll llll lll lll l lllll llllllllll lllllllllllll ll l lll lllll ll ll lll lll lll ll ll ll ll l ll lllllll lll ll l l lll lllllllll lll lllll ll ll ll lll lllllll llllllllll lll l lll l lll llllll lll l llll llllllllllllllll lll llllllll ll llll ll llll ll lll l ll llll l ll l lllll l lll ll ll l llllllll llll l lllllll ll llll ll l ll lllllllllll ll ll lll lll ll l lll lllllllllll lll lll ll lll lllll l ll llll ll l ll ll ll l l llll llllll lll ll l llllll llll llllllllll l lllll llllll llll llllll lllllllll l l ll lll llllll lll lllll llll llll lll l ll ll lll lllllll l llllll lllllllll llllll lllll l ll lll lll llllll lll ll ll ll ll lllll lllll ll l llll ll ll ll lllllll llllllllllllllll ll l lllll lll l llllll llll lll lll ll ll llllllll l l ll llllll llll l llllllllllll ll llllll l l lllllllllll llll lll llll PC2 P C l lll lll ll l llllllllll lll lll ll ll lll l lll lll llllll l lll lllll llll llllllllll ll ll ll llll lllllll ll ll l ll llll ll ll ll lll lll llllll l ll ll ll ll ll llll l l ll llllll ll lll l lll l ll ll l llllll l lll l lllllll ll lll lll lllllllll l llllll llllll lllllll lll l lll l llll lllllllllllll llll l l lllll lllll ll l l lllllllllll lllll ll l lllll l lll lll l llll llll ll lllll ll lll lll llll llllll llll lllllllllll llll l ll lllllllllll ll lll ll ll ll ll ll lll llll lll lll ll l l lll lllllllll lll lllll lll l ll ll l lllllll llllllllll lll ll ll ll l l llllll lll l llll llllllllllllllllll llll lllll lll l lll ll lllll llll lll lllll l ll ll ll ll ll ll ll l llllllll lllll lllllll ll llll lll ll llllllllllll ll ll ll llll l l l ll llllllllllll lll lllll llll lll llll lllll l ll lll l ll ll ll lll ll lll ll lll llllllllllllll llllll ll lll ll llll lllll lllll llllllllllll l lll l ll llll ll lllll lllll lll lll ll l ll l lllllllllll llllll l l llll ll llllll llll llll ll ll ll llllll ll l lll ll ll l lllll l ll ll ll ll lll ll ll ll llll llllllll llllllllll l ll l lllll lll l lllllll llll lll lll l llllll llll l l lllll llll llll lllllllllll llll ll lll lllllllllll ll llll lll llll LF1 L F l lll lll ll l llllllllll lll lll ll ll lll l lll lll llllll l lll lllll llll llllllllll ll ll ll llll lllllll ll ll l ll llll ll ll ll lll lll llllll l ll ll ll ll ll llll l l ll llllll ll lll l lll l ll ll l llllll l lll l lllllll ll lll lll lllllllll l llllll llllll lllllll lll l lll l llll lllllllllllll llll l l lllll lllll ll l l lllllllllll lllll ll l lllll l lll lll l llll llll ll lllll ll lll lll llll llllll llll lllllllllll llll l ll lllllllllll ll lll ll ll ll ll ll lll llll lll lll ll l l lll lllllllll lll lllll lll l ll ll l lllllll llllllllll lll ll ll ll l l llllll lll l llll llllllllllllllllll llll lllll lll l lll ll lllll llll lll lllll l ll ll ll ll ll ll ll l llllllll lllll lllllll ll llll lll ll llllllllllll ll ll ll llll l l l ll llllllllllll lll lllll llll lll llll lllll l ll lll l ll ll ll lll ll lll ll lll llllllllllllll llllll ll lll ll llll lllll lllll llllllllllll l lll l ll llll ll lllll lllll lll lll ll l ll l lllllllllll llllll l l llll ll llllll llll llll ll ll ll llllll ll l lll ll ll l lllll l ll ll ll ll lll ll ll ll llll llllllll llllllllll l ll l lllll lll l lllllll llll lll lll l llllll llll l l lllll llll llll lllllllllll llll ll lll lllllllllll ll llll lll llll LF1 L F l lll lll l ll llllllllll lll llll l ll llll lll lll ll llllllll lll ll lll llllllllllll ll ll ll lllll ll ll ll lll lllllll llll ll llllll lllll llll lllll lllllll l ll lllllll lllll l ll ll ll l llll ll ll l lll llllllllll l lll llllllllllll llllll llllll llllllllll l lll l lllllllllllllllll lllll l llll llll lll ll l llllllllll lll ll llll llllll lll lll lllll llll ll llll lll lll l lllll llllllllll lllllllllllll ll l lll lllll ll ll lll lll lll ll ll ll ll l ll lllllll lll ll l l lll lllllllll lll lllll ll llll lll lllllll llllllllll lll l lll l lll llllll lll l llll llllllllllllllll lll llllllll ll llll ll llll ll lll l ll llll l ll l lllll l lll ll ll l llllllll llll l lllllll ll llll ll l ll lllllllllll ll ll lll lll ll l lll lllllllllll lll lll ll lll lllll l ll llll ll l ll ll ll l lllll llllll lll ll l llllll llll llllllllll l lllll llllll llll llllll lllllllll l l ll lll llllll lll lllll llll llll lll l ll ll lll llll lll l lll lll lllllllllllllll lllll l lllll lll llllll lll ll ll ll ll lllll lllll ll l llll ll ll ll lllllll llllllllllllllll ll l lllll lll l llllll llll lll lll ll ll llllllll l l ll llllll llll l llllllllllll ll lllllll l lllllllllll lllllllllll LF2 L F lllllll AFRICAAMERICACENTRAL_SOUTH_ASIAEAST_ASIAEUROPEMIDDLE_EASTOCEANIA
Figure 2: Principal components versus logistic factors for the HGDP data set. The top three principalcomponents from the HGDP data are plotted in a pairwise fashion in the top panel. The top threelogistic factors are plotted analogously in the bottom panel. It can be seen that both approaches yieldsimilar visualizations of structure. 16 llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll l llllllllllllllllllllllllllllllllll lllllllllllllllllllllllllll llllll llllllllllllllllll ll llll lllllllll ll llllllll lll l lllll lll ll llll ll l lll lll llll lll lllllll lll lll ll lll ll llllll llll ll ll lll lll llllll lllllllll lllllllllllllllllllllllll ll llll llllllllllllllllll llllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllll l ll llll l lll lll ll lll l ll ll llll l l llll lll llll l lll llll l l lllllllllllllll lllllllllllllllllll l l lll llll lllllll lllllllllllllllllllllll llllllll lll lll ll lll lll ll llll llllll lllllllllllllllllllllllllllllllllllll ll l lllllll ll l llllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll llll llll lllll llll ll llllllll ll llll l ll llll lll ll lllll llllll lll lll lll ll ll lll llll ll lll lll ll l lll lll l lll lll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll
PC1 P C lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll l llllllllllllllllllllllllllllllllll lllllllllllllllllllllllllll llllll llllllllllllllllll ll llll lllllllll ll llllllll lll l lllll lll ll llll ll l lll lll llll lll lllllll lll lll ll lll ll llllll llll ll ll lll lll llllll lllllllll lllllllllllllllllllllllll ll llll llllllllllllllllll llllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllll l ll llll l lll lll ll lll l ll ll llll l l llll lll llll l lll llll l l lllllllllllllll lllllllllllllllllll l l lll llll lllllll lllllllllllllllllllllll llllllll lll lll ll lll lll ll llll llllll lllllllllllllllllllllllllllllllllllll ll l lllllll ll l llllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll llll llll lllll llll ll llllllll ll llll l ll llll lll ll lllll llllll lll lll lll ll ll lll llll ll lll lll ll l lll lll l lll lll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll PC1 P C lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllll lllllllll lll lllllllllllllllllllll llllllllll lllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllll ll lllllllllllllllllllllllllllllllll ll lllllllllllllllllllllllll lll ll lllllllllllllllllll lllll ll lllll ll llll lll llllll lll ll lll l llll lll l lll llllll l l ll lll l l ll lll ll ll ll l lll lll l lll l l ll llll ll ll ll ll l lllll l lllll ll llllllllllllllllllllllllll l llll llllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll ll ll lll l ll ll llll l ll l ll ll ll l ll ll ll l l ll ll ll lll ll llllllllllllllllll ll lllllllllllllllll llll lllll lllllll l llllllllllllllllllllll llllllll l l l lll lll lll l l lll llll ll ll ll llllllllllllllllllllllllllllllllllll l llll llll ll llllllll l llllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllll llllllllllllllllllllllllll llllllllll lllllllllllllllllllllll lll llllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll l lllll ll ll ll ll l lllll lll ll ll ll llllll ll llll l l l llll lll ll lll llll lllll ll l llllllll lll llll ll lll ll l ll lll l ll l lll l lll lll l l llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll ll ll lll ll lllll ll llllll lllllll ll lllll llll l lllllllllllllllllllllllll PC2 P C llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllll llll lllllllllllllllllllllllllll llllllllllllllllllllllll ll llll llllllll lll lllllllllll l lllll lllllllll lll lll lllllll lll lllllll lll lllll lll ll ll llll llll llll lll ll l ll llll l llllllll lllllllllllllllllllllllll ll llll llllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll l ll llll l lll lll ll ll l l ll ll llll l lllll lll ll ll l lll lll l l llllllllllllllll lllllllllllllllllll l l lll llll lllllll lllllllllllllllllllllll llllllll lll lll ll lll ll l ll llll llllll lllllllllllllllllllllllllllllllllllll ll l lllllll ll l llllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllll llllllllllllllll lllllllllllllllllllllll lll llllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll ll ll lll l lllll llll ll ll llllll ll llll l ll llll lll ll lll ll llllll lll l ll lllll ll lll llll ll lll lll ll l lll lll l lll lll llll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllll ll llllll llllllllllllll llllllllllllllllllllllllllllll LF1 L F llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllll llll lllllllllllllllllllllllllll llllllllllllllllllllllll ll llll llllllll lll lllllllllll l lllll lllllllll lll lll lllllll lll lllllll lll lllll lll ll ll llll llll llll lll ll l ll llll l llllllll lllllllllllllllllllllllll ll llll llllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll l ll llll l lll lll ll ll l l ll ll llll l lllll lll ll ll l lll lll l l llllllllllllllll lllllllllllllllllll l l lll llll lllllll lllllllllllllllllllllll llllllll lll lll ll lll ll l ll llll llllll lllllllllllllllllllllllllllllllllllll ll l lllllll ll l llllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllll llllllllllllllll lllllllllllllllllllllll lll llllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll ll ll lll l lllll llll ll ll llllll ll llll l ll llll lll ll lll ll llllll lll l ll lllll ll lll llll ll lll lll ll l lll lll l lll lll llll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll llllllllll ll llllll llllllllllllll llllllllllllllllllllllllllllll LF1 L F lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllll llllllll lll lllllllll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll lllllllllllllllllllllllllllllllll ll lllllllllllllllllllllllll lll ll lllllllllllllllllll lllll ll lllll ll ll ll lllllllll lll ll lll l llll llll lll llllll l l ll lll l l ll lll ll ll ll l lll lll l lll ll ll llll ll ll ll ll l lllll l lllll ll ll llllllllllllllllllllllll l llll llllllllllllllllllll llll lllllll llllllllllllllllllllllllllllllllllllllllllllllllllll ll ll lll l ll ll lll l l ll lll ll ll lll ll ll l l ll ll ll lll llllll llllllllllllllll l llllllllllllllll llll ll lllllllllll llllllllllllllllllllll lllllllll l l lll lll lll l l lll llll ll ll ll l llllllllllllllllllllllllllllllllllll llll llll ll lllll llll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll lllllllllllllllllllllllllll llllllllllllllllllllllllll lllllllllllllllllllllllllllllllll lll lllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllll llllllllllllllllllllllllllllllllllllllllll lllll lllllllllllllllllllllll lll llllllll lllllllll llllllllllllllll llllllll l lllll ll ll ll ll l lllll lll ll ll ll llllll ll llll l ll llll lll ll lll llll lllll ll l llllllll lll llll ll lll ll l ll lll l ll l lll l lll lll l l llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll ll ll ll lll ll lllll ll llllll lllllllll lllll lllll lllllllllllllllllllllllll LF2 L F lllllll AFRICA_IN_AFRICAAFRICA_IN_AMEAST_ASIA_IN_AM/ASIAEUROPE_IN_AM/EUMEXICO_IN_AMSOUTH_AMERICASOUTH_ASIA_IN_AM/ASIA
Figure 3: Principal components versus logistic factors for the TGP data set. The top three principalcomponents from the TGP data are plotted in a pairwise fashion in the top panel. The top three logisticfactors are plotted analogously in the bottom panel. It can be seen that both approaches yield similarvisualizations of structure. 17 s − S L C A r s − E D A R . . . . .
00 0 . . . . .
00 250500750 I nd i v i dua l s O r de r ed b y A ll e l e F r equen cy Estimated Individual−Specific Allele Frequencies g e o A F R I C A A M E R I C A C E N T R A L_ S O U T H _ AS I A EAS T _ AS I A E UR O PE M I DD L E _ EAS T O C EA N I A H G D P r s − S L C A r s − E D A R . . . . .
00 0 . . . . .
00 50010001500 I nd i v i dua l s O r de r ed b y A ll e l e F r equen cy Estimated Individual−Specific Allele Frequencies g e o A F R I C A _ I N _ A F R I C A A F R I C A _ I N _ A M EAS T _ AS I A _ I N _ A M / AS I A E UR O PE _ I N _ E U M EX I C O _ I N _ A M S O U T H _ A M E R I C A S O U T H _ AS I A _ I N _ A M / AS I A T G P F i gu r e4 : S N P s w i t hh i gh l y d i ff e r en t i a t eda ll e l e f r equen c i e s w i t h r e s pe c tt o s t r u c t u r e . T w oo ft he m o s t h i gh l y d i ff e r en t S N P s a cc o r d i ng t oL F A a r e s ho w n f o r t he H G D P and T G P da t a s e t s . F o r ea c h S N P ,t he (cid:98) π i j v a l ue s a r eo r de r edand t he y a r e c o l o r eda cc o r d i ng r epo r t ed an c e s t r y . T heho r i z on t a l ba r s on t he s i de s o ft hep l o t s deno t e t heu s ua l a ll e l e f r equen cy e s t i m a t e s f o r m ed w i t h i nea c han c e s t r a l g r oup . S . Column 1 shows the scenario from which the datawere simulated. Columns 2 and 3 display the estimation accuracy of the PCA based method (Column 2)and LFA (Column 3). Column 2 shows the mean R value when regressing the true ( π i , π i , . . . , π in ) on (cid:98) S from PCA, averaging across all SNPs. Column 3 shows the mean R value when regressing thetrue (logit( π i ) , logit( π i ) , . . . , logit( π in )) on (cid:98) H from LFA, averaging across all SNPs. All estimatedstandard errors fell between − and − so these are not shown. Note for each scenario, R valuesare higher for the method from which the true F matrix was generated. All but the two scenarios markedwith an asterisk (*) are from Model 1, while the two marked scenarios are from Model 2, where we took F = logit − L . Mean R Scenario F ∼ (cid:101) S logit( F ) ∼ (cid:98) H TGP fit by PCA 0.9998 0.9722TGP fit by LFA * 0.9912 0.9990HGDP fit by PCA 0.9996 0.9614HGDP fit by LFA * 0.9835 0.9983BN 0.9999 0.9999PSD α = 0 . α = 0 . α = 0 . α = 1 a = 0 . a = 0 . a = 0 . a = 1 ab l e2 : A cc u r a cy i ne s t i m a t i ng π i j pa r a m e t e r s b y t he P C A ba s ed m e t hodandL F A . E a c h r o w i s ad i ff e r en t s i m u l a t i on sc ena r i o . E a c h c o l u m n i s t hea cc u r a cy o f a m e t hod ’ sfi t s w i t h t heg i v en m e t r i c . S c ena r i o M ed i an K L M ean A b s . E rr . R M SE P C A L F AA D X F SP C A L F AA D X F SP C A L F AA D X F S BN . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - PSD α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - Spatial a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - TGPfit P C A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - A D X . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F S . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - HGDPfit P C A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - A D X . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F S . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - S UPPLEMENTARY M ATERIAL
The HGDP data set was constructed by intersecting the data available from the HGDP web site, , with the set of individuals “H952” identified by Rosenberg(2006) [46] with a high confidence as containing no first and second-degree relative pairs. This yieldedcomplete SNP genotype data on 431,345 SNPs for 940 individuals.In order to obtain data from the TGP we first obtained the genotype data that had been measuredthrough the Omni Platform, 2011-11-17, ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/working . We removed related individuals based on the TGP sample information. We then sorted in-dividuals according to least percentage of SNPs with missing data, and we selected the top 1500individuals. This yielded complete SNP genotype data on 339,100 SNPs for 1500 individuals.We utilized the HapMap data set in the simulated data described below. We obtained the HapMapdata release 23a, NCBI build 36 from consisting of unrelated individuals: 60 from Eu-ropean ancestry group (CEU), 60 from Yoruba, Africa (YRI) , and 90 from Japan and China (JPT+CHB).We identified all SNPs with observed minor allele frequency ≥ and with no missing data. Thetotal number of SNPs used after filtering in each population were CEU: 1,416,940, YRI: 1,539,314,JPT+CHB: 759,452. We then identified all SNPs common to all three populations resulting in a total of363,955. The model dimension d was determined for the HGDP and TGP data sets under the rationale that when d is large enough, then the great majority of SNPs should appear to be in HWE. When d is too small,then the structure which has not been accounted for will lead to spurious deviations from HWE. Values d = 1 , , . . . , were considered for each data set, and we ended up identifying d = 15 for HGDP and d = 7 for TGP. We note that these choices could also be interpreted as reasonable according to a screeplot when PCA was applied to the genotype data.For a given d value, we formed (cid:98) F using the LFA method. We calculated a HWE goodness of fitstatistic for each SNP i as follows: (cid:88) k =0 (cid:104)(cid:80) nj =1 x ij = k ) − (cid:80) nj =1 (cid:0) k (cid:1)(cid:98) π kij (1 − (cid:98) π ij ) − k (cid:105) (cid:80) nj =1 (cid:0) k (cid:1)(cid:98) π kij (1 − (cid:98) π ij ) − k , where (cid:80) nj =1 x ij = k ) is the observed number of genotypes equal to k and (cid:80) nj =1 (cid:0) k (cid:1)(cid:98) π kij (1 − (cid:98) π ij ) − k isthe expected number of genotypes equal to k under HWE. We then utilized (cid:98) F to simulate five instances21f a genotype matrix X under HWE where we simulated x ij ∼ Binomial (2 , (cid:98) π ij ) . On each simulatedgenotype matrix X , we again applied LFA to obtain (cid:98) F and calculate HWE goodness of fit statistics.These goodness of fit statistics were then pooled across all five simulated data sets and across all SNPsto form the null distribution, which then allowed us to calculate a HWE p-value for each observed SNP.(It should be noted that we also formed a separate null distribution according to minor allele frequencybins of length 0.05, and we arrived at the same conclusion.) We then compared these p-values to theUniform(0,1) distribution and also against the p-values from the d + 1 case. This allowed us to identifya value of d where the HWE p-values were both close to the Uniform(0,1) distribution and to the HWEp-values from the d + 1 case. For each simulation scenario, genotypes X were simulated such that x ij ∼ Binomial (2 , π ij ) , where π ij were elements of the allele frequency matrix F . The results from the simulated data are summarized inTables 1 and 2. Balding-Nichols (BN).
For each SNP in the HapMap data set, we estimated its marginal allele frequencyaccording to the observed frequency and estimated its F ST value using the Weir & Cockerham estimate[47]. We set the simulated data to have m = 100 , SNPs and n = 5000 individuals with d = 3 .Using Model 1, the S matrix was generated by sampling its columns s j i.i.d. from (1 , , T , (0 , , T ,and (0 , , T with respective probabilities / , / , and / to reflect the original data’ssubpopulation proportions. For each row i of Γ , we simulated i.i.d. draws from the Balding-Nicholsmodel: γ i , γ i , γ i i.i.d. ∼ BN ( p i , F i ) , where the pair ( p i , F i ) was randomly selected from among themarginal allele frequency and F ST pairs calculated on the HapMap data set. PSD.
We analyzed each SNP in the HGDP data set to estimate its marginal allele frequency accordingto the observed marginal frequency and F ST using the Weir & Cockerham estimate [47]. To estimate F ST , each individual in the HGDP data set was assigned to one of K = 5 subpopulations according tothe analysis in Rosenberg et al. (2002) [10]. We set m = 100 , SNPs and n = 5000 individuals with d = 3 . Again utilizing Model 1, each row i of Γ was simulated according to γ i , γ i , γ i i.i.d. ∼ BN ( p i , F i ) ,where the pair ( p i , F i ) was randomly selected from among the marginal allele frequency and F ST pairscalculated on the HGDP data set. To generate S , we simulated ( s j , s j , s j ) i.i.d. ∼ Dirichlet ( α ) for j = 1 , . . . , . We considered α = (0 . , . , . , α = (0 . , . , . , α = (0 . , . , . , and α = (1 , , . It should be noted that as α → , the draws from the Dirichlet distribution becomeincreasingly closer to assigning each individual to one of three discrete subpopulations with equalprobability. When α = (1 , , , the admixture proportions are distributed uniformly over the simplex. Spatial.
This scenario is meant to create population structure that is driven by spatial position of the22ndividual. We set the simulated data to have m = 100 , SNPs and n = 5000 individuals with d = 3 .Rows i = 1 , of S were simulated as s ij i.i.d. ∼ Beta ( a, a ) for j = 1 , . . . , , and row 3 of S containedthe intercept term, s j = 1 . We considered four values of a : 0.1, 0.25, 0.5, and 1. The first two rowsof S place each individual in a two-dimensional space (Figure 7), where the ancestry of individual j islocated at ( s j , s j ) in the unit square. When a = 1 , the Beta ( a, a ) distribution is Uniform (0 , , so thisscenario represents a uniform distribution of individuals in unit square. As a → , the Beta ( a, a ) placeseach individual with equal probabilities in one of the four corners of the unit square. The matrix Γ wascreated by sampling γ ij i.i.d. ∼ . × Uniform (0 , / for j = 1 , and γ i = 0 . . It should be noted thatall π ij ∈ [0 . , . by construction. Real Data.
For the HGDP and TGP scenarios, we estimated an allele frequency matrix F from the realdata via four different methods. For HGDP we had m = 431 , SNPs by n = 940 individuals with d = 15 , and for TGP we had m = 339 , and n = 1 , with d = 7 . The four methods are: • PCA : F was taken to be the matrix (cid:101) F estimated via Algorithm 1. • LFA : F = logit − ( (cid:98) L ) , where (cid:98) L was estimated via Algorithm 3. • ADX : F was taken to be the matrix formed by computing the marginal allele frequencies in thePritchard-Stephens-Donnelly model, i.e. F = PQ , and P and Q were estimated via the softwareADMIXTURE [16]. • FS : Same as above except P and Q are estimated via the software fastStructure [25]. F and L Estimates of π ij were evaluated with three different metrics. Let (cid:98) π ij be the estimate for any givenmethod.The Kullback-Leibler divergence for the binomial distribution allows us to measure the differencebetween the distribution from the estimated allele frequencies to the distribution from the oracle allelefrequencies: KL = π ij ln (cid:18) π ij (cid:98) π ij (cid:19) + (1 − π ij ) ln (cid:18) − π ij − (cid:98) π ij (cid:19) . Mean absolute error compares the allele frequencies directly:MAE = 1 m × n m (cid:88) i =1 n (cid:88) j =1 | π ij − (cid:98) π ij | . Root mean squared error : 23MSE = (cid:118)(cid:117)(cid:117)(cid:116) m × n m (cid:88) i =1 n (cid:88) j =1 (logit( π ij ) − logit( (cid:98) π ij )) . F ST for individual-specific allele frequencies By considering the derivation of F ST for K discrete populations as described in Weir (1984, 1996)[47, 48], it can be seen that a potential generalization of F ST to arbitrary population structure is F ST = 1 − E Z [Var( x | Z )]Var( x ) , where, as described in Section , Z is a latent variable capturing an individual’s population structureposition or membership. The allele frequency of a SNP conditional on Z can be viewed as beinga function of Z , which we have denoted by π ( Z ) . If n individuals are sampled independently andhomogeneously from the population such that z , . . . , z n are i.i.d. from the distribution on Z , then forSNP i in HWE, it follows that Var( x ij | z j ) = 2 π ij (1 − π ij ) and F ST a.s. = lim n →∞ − n (cid:80) nj =1 π ij (1 − π ij ) π i (1 − π i ) , where π i = (cid:80) nj =1 π ij /n is the marginal allele frequency among the n individuals. Thus, good estimatesof the π ij values may be useful for estimating F ST in this general setting. One example would be toform a plug-in estimate of F ST by replacing π ij with (cid:98) π ij from the proposed LFA method. The problem of modeling a genotype matrix X in order to uncover latent variables that explain crypticstructure is a special case of a much more general problem that has been considered for several yearsin the statistics literature [49, 50]. Under a latent variable model, it is assumed that the “manifest”(observed) variables are the result of the “latent” (unobserved) variables. Different types of the latentvariable models can be grouped according to whether the manifest and latent variables are categoricalor continuous. For example, factor analysis is a latent variable method for the case where both manifestvariable and latent variable are continuous. A proposed naming convention [23] is summarized asfollows: When the individuals are not sampled homogeneously throughout the population (e.g., in the HapMap data with 60, 60,and 90 observations from three discretely defined subpopulations), then it may be the case that the above quantity should bemodified to reflect the stratified or non-homogeneous sampling. x ij ∼ Bernoulli( π ij ) and the latent variables h kj are continuous variables, the GLLVM in this case is Model 2 , logit( π ij ) = (cid:80) dk =1 a ik h kj . While webegin with this model, there are some key differences. The number of manifest variables in the dataconsidered in Bartholomew (1980) and related work is notably smaller than genome-wide genotypedata, so the assumptions and estimation approach differ substantially. Model assumptions are typicallymade about the probability distributions of the latent variables; we consider these model assumptionstoo strong and also unnecessary for the genome-wide genotype data considered here, although theymay be quite reasonable for the problems considered in other contexts. Existing methods typicallyestimate Model 2 by calculating the joint posterior distribution of the h kj based on an assumed priordistribution of the latent variables.Our LFA approach for estimating the row basis of L is nonparametric since it does not require a priorassumption on the distribution of latent variables, H . The model fitting methods of ref. [23] are too com-putationally intensive for high-dimensional data, requiring many iterations and potential convergenceissues. Our proposed algorithm requires performing SVD twice, which leads to a dramatic reduction incomputational burden and difficulties. Engelhardt and Stephens (2010) [24] make an interesting con-nection between classical factor analysis models of F and other models of population structure, but thefactor analysis model runs into the difficulty that the latent factors are assumed to be Normal distributed,and the constraint that alleles frequencies are in [0 , is not easily accommodated by this continuous,real-valued model.Several extensions of PCA to categorical data have been proposed [52–54]. We found that thealgorithms perform very slowly on genome-wide genotyping data, and the estimation can be quite poor25hen d > . Also, PCA is essentially a method for characterizing variance in data [45], and thelatent variable approach is more directly aimed at uncovering latent population structure. Non-negativematrix factorization (NMF) [55] is another matrix factorization for count data (e.g., Poisson randomvariables). This identifies two non-negative matrices whose product approximates the original matrix.However, similarly to PCA, we do not find that this approach easily translates into interpretable modelsof population and it is computationally intensive. NMF has proven to be quite useful as a numerical toolfor decomposing images into parts humans recognize as distinct [56].26 S UPPLEMENTARY F IGURES AND T ABLES L = A H ≈ A H F = Γ S ≈ Γ S X € logit π ij ( ) € π ij Logis&c Factor Analysis (LFA)
Principal Components Analysis (PCA) SNP Data ⇒ values in (−∞,∞) ⇒ values in (0,1) ⌃ ⌃ ~ ~ Figure 5: A comparison of LFA model (2) and its estimate to model (1) and its PCA estimate. Theproposed LFA approach first models the logit of the individual-specific allele frequencies in terms ofthe product of two matrices, the left matrix establishing how population structure is present in allelefrequencies, and the right matrix giving the structure. Whereas the LFA approach preserves the scaleof the model through the estimate (all real-valued numbers), the same is not true to PCA. This leadsto issues in the estimation of individual-specific allele frequencies when utilizing PCA. We have shown,however, that PCA estimates very well a row basis for S from Model 1. This connects PCA to an explicitmodel of population structure. 27 lll lll ll llll lll llllllll lllllllll lllllllllllllllll lll ll llllllllllll lllllll lllll lllll lll lll llllll ll lll ll llllll llll ll l lll lll llllll ll lllllll lll l llll lll lll llllllll ll ll lllll llll ll llllllll lll lll llllll llll lllll ll lllll llll lllll llllll ll lllll lllllllll ll lllllll llll llllll lll llll lllllll lll ll llll lll lll lll lllll ll ll lllll llll llllllll ll lllllll lllllllll ll lllll llll ll lllll llll lllllll lllllll ll ll llllll ll ll l llllll lll lllll lll ll lll llll lllll llll lllll ll llll ll lllllllllll ll ll lll llll llll lllll llllll llll llll lllll lllll llllllll lllll ll lllllll lllll llll ll ll llll llllllllll ll llll lllllllllll lllllll ll lllll lll llllll lll ll lll lll lllllllll ll ll llllll llllllllllll ll ll ll ll lllllll lllllll lll ll lllll ll llllllll llllllllll llllllll llllllll llll lllll lll lll ll lllll ll ll lll llll lll lllll llll lll ll lll llll lll llllllll lll lllll ll ll llll ll ll lll l llllll l lll lll llll lllll llll llllllllll lll ll ll lllllll ll llllllll l lll llllll ll ll llll lllllll ll llll lllllllllllll lll ll l lll ll lllll ll lll lllll lllll ll llllllllll llllll llllllll ll lllll lllllll lllllll ll lllll llllll lllll ll llllll lll lllll ll ll ll llllll lll llll llll lllll ll ll ll llll llll lll lll lll l llllll lll lllll lll llll llllll ll lllll lll ll lll llllllll lllllll llll lll l llll llllll llllllll lll lll llll lllll lllll ll lll ll ll llllll lll llll lll lll llll lllllll lll lllllll lll lll llllllllll ll lll lll lllll ll lll llll lll ll lllllllllll lllllll l lll lll ll lll llll lll lllllllll lll lllll lll lllll l lllll llllll lll lll lllllll lllll ll ll lllll llllll llllll lllllll ll lllll lllll lllll llllllllll lll llllll lll llll lll llllll llll llllll lll llll llllll llll llllllll lll lllllllll llll lll lll llll lllllll lllllll lllll lll lll lll llll llllll ll lllll lllll llllllllllll llllll lll lll lllllll lll lll lllll lll lllll lll llllll llll lll llll llllll lll ll lll ll ll llll ll llll lllllllll ll ll ll lll ll llllllllll lll llll ll lllllll lll lllllll llllllllllllll lllll l lllll llll ll lll ll l lllllllllllll lll lll llll llll lllll ll lll lllllllllll lllll llllll llll lll ll lll lll ll lll ll lll lllll lll lllll lll ll lll lllllll lll l lll ll ll lll llllll llllllll ll lllll ll llllllllll lllllllllll ll lllllllllllll lllllllll ll lllllll ll ll lll lllllll ll lllllll ll llllll llll llllll lllll l lllllll llll ll ll lll ll lll lllllllllllll llllllll llllllll lll lllllllll lll ll lll ll lll lll llllllll llll lll ll lll lllllllll lllll ll ll llll llllll llll lllll lll lllllll llll lll llllllllll lllll lll ll llllllllllllll llllllllll lll lll llll llllllllll lllll llllllllll lll lllllll lllll lll lll lll ll llll lll lllll ll lllll lllll llllll llll llll lllll lllll lll lllll lllll lllllllll ll lllllll llll lll llll llllllllll ll lll ll llllllll llll llll lll lll lllll llllll lllll llll ll llllll llll ll ll l lllllll lllllll ll lllll llllll ll llll ll ll llllll ll lll llllllllll ll lll lllllll llll llllll l ll ll llll lllll ll llll lll llll lllllll lll lllll ll llllll lll llll lll l lll lll llll lll lllllll lllll lllllllll ll lll llll llllllll llll lll lllll l ll lllll lll lllll lllllll llllll lll lllllll lllll ll ll llllllllll ll lll lll llll lllll llllllll lllll llllll llll lll lllll lll lll llll lllll llllll ll llllllll llllllllll lll ll lllllll llllllll lll lll llllll lll llll ll llll llll lllllllllllllll lll lllll lllll ll ll lll lllll llllll lllll lllll lllllll lll llll lll lllll lllll lll lll ll lllll lll ll llll ll llll llll llll lll ll ll llll lllll llll lllll llll lll llll ll ll lll llll lllll llll ll lll ll llllll lll ll llll ll llll llll lllll llllllll ll lllll llllll l lll lll ll lllll ll llll llll lll ll lll ll lll lllllllll ll llllll lll llll ll llll llllllll lllll llll lllll l lllllll l lll lll ll llll ll lllllll lllll lll lllllllll lll lll llllll llllllll lll l ll llllll ll lll lllllllll lllll llllll lllll llll ll lllll ll llll lll l llllll lll ll ll llll ll llll llllll lllll lll lll lllll lllll lllll lll lll llll ll ll lllllllll lll ll llll lllllll lll lllll ll ll ll lllllllllllllllllll l ll lll ll ll lllll lllll llllll llll ll lllllll lllll l llll lllll llllll llllll ll ll llllll lll llllllll ll ll llll lll ll lllllll llll ll lll l lllll llll lll lllll lllllll lllllll llll lllll lll llllll lllll lll lll lll llllll lllll lll ll llllll ll lll l ll llll l lll lll lll llll lll l llllllll llll ll llll llll lll lllllllllllll llll lll llll llll lll llll lll llll llll ll lllllll llllllll lll lll lllll llllll lll ll lll ll lllll lllll llll lllllllll lllll lll lll llll ll lll lll llllll l ll lllllll lll lllll ll lllll llll lllll llll lll lllllll llll llll lll ll lllll lllllllll lllllllll lllllll lll lllll lllllll llll ll ll llllll lllll ll lll ll ll llll llll llllllll llllll ll llll ll llllllllll lllllllllllllll ll ll lllll ll llll lllllll lll l ll ll lllll ll lllll lll lllll llll lll lllll lllll lllll llllll llll lllll lllll ll ll lllllll ll llll llll ll lllll lllll llllll lllll lll lllllllll llllllll llll ll ll llll lllllll llll ll ll l lll lllll ll llll ll llll ll llll lll lll lll llllllll ll lll llllll llllllll ll lll ll lllll lll ll lllll l lll llll lllll llllll llllll lllllll ll lllll ll llll ll lll ll llllll ll lll llllllll llll lllll lllllllllllll lll llll llllll llllll lll l lll llll ll llllll llllll llllll lllllllll llll lll llll lllllllllll llllllllll llll llll lllll l ll lllllll lllllll llllll ll llll ll llll ll lll llll lll lll llll lll llllll lllll lll ll lll lll llllllllllllll ll ll lll lllll lllllllll llll ll lllll llll llll ll lllll llll llllllll ll lllll ll ll llllllll lll lll lllll llll lllllll ll lllllllllll ll llllll llll llll ll lllllllllllll lllll llll lllllllll llll ll ll ll lllll ll lllllllll lllll lllll llllll llllllll lllll lllll lll lllll ll lll l lll llll lllllllllllllll lllllllll l llllll ll lllllll llllll lllllllllll ll llllll ll llll ll l lllll ll ll lll llll ll lll l l lll lllllll llllll llll lll lll lllll lll lll lllll llllll lllll lllll lllllllll lll lllllll llll ll llll ll ll lll ll ll lll llll ll llllllllllll l ll ll lll ll lll lllll ll lll lll ll ll lll ll lllll ll lll llll lll llll lll l ll lllll ll lll lll l ll lll ll lll llll ll lll llll ll ll ll ll l ll ll ll ll ll lll l lll lll lllll lll l ll ll ll lllllll ll l llll lll l llll ll l lllll ll lllllllll ll lll llllll ll lll llll l ll llll l lll lll lll ll ll llll lll ll ll l ll l lllllll lll l ll llll llll l lllll lll ll ll lllll lll ll lll l ll l ll ll lll lll ll ll ll ll llll lll l l ll lll llllll lllll l llll lll lll ll ll llll llll ll lll llll ll ll l ll ll lll lll lllll lllllll lllllll l l ll ll l ll lll ll ll lllll ll lll llllll ll ll lll lll ll lllll llll ll llllll lllll ll llllll ll l ll lll l lll ll lll ll ll llll l lll lll ll ll ll ll ll lllll lll l lll l ll lllll lll l ll l l ll l llll ll ll llllllll llll l l lll ll l ll lll l ll llllll lllllllllllll ll lll llll l lll llll llll lll l llll ll llll lll llll ll l ll ll ll lll ll lllll l ll lll llll ll ll ll lllllll llll lll ll l ll ll lll lll ll lll l l lll ll ll ll l lll ll l l ll lllll l ll lll l llll l llll l llllll ll l lll lll llll lllll ll llll lll ll lllllll l lll l lll llll lll ll l ll lll llll l llllll l llll lll ll lll ll lllll lllll ll ll l llll ll l lllllllll ll ll ll ll llll l lll llll l l lllll lll ll l lll lll llll lll l lll l ll lll lll lllll lll lll llll ll l llllll l lll ll lll ll ll lll ll ll llll l ll lllll l l lllllll ll ll l l ll lll lll ll lll l l ll l l ll lll lll l ll ll ll lllll l ll lll lllll ll l ll lll ll lllll ll llll lll ll l llll l llll l l llll lll lll ll l lllll l ll ll ll lll lll l lllllll ll lll ll llll ll l llll l llllll l lllll llll lll lll ll l ll llllll llll lll l lll lll ll lll ll lll lll ll ll ll ll l lll ll l ll lllll lll l llll ll l ll llll lllll lll lll lll llll lll lll l l llll lllll ll ll ll l lll ll lllll l l lllll ll ll llll lll lllllllll lllllllll lll l lll llll ll lll lll lll lllll ll ll l lll ll ll llll lll ll lll ll ll ll l lllll l lll l lll ll l lll ll llllllllll ll l lll l l ll lllllll ll ll l llllll lll llllll ll lll lll l llll l ll ll lll l lllll lll l l ll l llll ll ll llll ll ll llll ll llllllll l llll lllll lll ll ll lll ll ll l l lll l llll ll ll ll ll l llll llllll l lllll ll l ll ll l lllll lll ll l llll ll lll lll lll lllll llll ll l llll llll ll lll lll llll ll lll ll ll ll ll lll llll ll ll ll ll ll llll llll l lllll ll ll ll ll llllll lll lllll ll llll ll llll lll lll l lllll ll ll lll lll l lllll l ll l lll lll ll l ll ll ll ll l llll llll ll ll lll ll l ll l lll l ll lll ll ll llllllll ll llll ll llll llll llll ll l ll ll l llllll ll ll lll ll ll ll lllll ll l llllll ll lll l lll ll lllll lll l lll llll ll lll ll lll llllll ll llll ll lll llllll lll ll lllll lll ll l l lll ll l ll ll llll l lll llll llll lll ll l l ll ll llllll lll lll l llll lll lll ll lll l lllll l ll lll ll lll ll ll lll l ll l lll ll lll llll lll l lllllll ll l llll l l lllll ll ll ll ll ll ll ll ll lll l lll ll ll llllll l lll ll l lll lll l llll ll ll lllll llll ll l ll ll l llllll l l lll ll lllll ll llll ll lllll ll ll lllll llll llll llll ll l ll ll lllll l lll lllll ll lll llll ll ll lll l lll ll l ll ll lll lllllll llll ll l llll l llll l ll ll lll lllll lll lllll l lll llllll ll ll ll ll l llllll ll llllll ll llllll ll l lll llll l ll ll ll ll ll ll lllll lll lll ll l ll llllll lll ll lllll ll l lllll lll ll llll lll lllll lll ll llllll ll ll l l lll llll l ll lllllll ll lll ll lll lll l l llll ll ll ll lll lllllll lll llll lllll ll ll lll llll lll ll lll l l ll lllll lll l l l llll lll ll ll l ll l l lll lllll ll llll l l llll lll lll ll lll lllllllll lll ll lll llllllll ll llll lll l ll lll l ll ll llll l llll lll lll llll lll llll llll llll ll ll lll l ll ll ll lllll lll lll lll l llll l lll lll lll l lll lll llll lll lll lll llll ll l llll llllllll lll ll llll ll l lllllllllll l ll llll ll ll llll llll ll l lll ll ll lll lll l l ll l lll ll lllll l lll llllll lllll lll ll llllll lll lll ll llll ll ll ll lllll lll ll ll lll ll lll llllll lllll ll l lllll lll lllll llll l ll lll lll l lll lllll ll ll lll llll ll l ll ll l ll l lll ll ll lllll ll llllll l l lll lll l ll ll ll l lll l ll l l lll lll lll ll ll lllll ll ll ll lll l ll ll ll l l ll lll l ll llll l ll llll lll ll l lll llll l ll llllllll lll ll llll l llll l ll ll l lll lll l lllll lll l ll lllll ll l l lllll l lllll llll ll ll l lll lll l l lll l lll ll l lll l ll l ll lll ll llll l lllll l l ll llll llll l llll l lll l l l ll llll ll ll lll l llll ll ll l ll l llllll ll l lll ll lll lllll ll lll llll ll lllll ll lll lll ll llll ll l ll ll lllll ll lllll ll ll ll lllllll ll llll l ll l llll lll l lll ll ll l lllllll lll ll lll llll llllll ll ll l lll l llll ll lllll llll l ll l ll llll lllllll lllll lll ll ll llll ll l ll ll ll l lll ll l l llll lll lllll llllllll l lll lll ll llllll llll lll ll lll l l lll llll lll lll lll l ll ll ll lll lllll lll ll lll ll ll ll ll lll ll ll lllll llll lll llll ll ll lll lll ll l lll llll llll lll lll lll lll lll l lllllll llllll llll l l lll l ll llll lllll lll ll llll ll l ll lll lll l l l lllll ll ll lll ll lll ll llllll lll ll ll l lll lll ll lll llllll ll lll lll llllllllll lllll lll lll l llll lll ll l l lll lllll llllll ll l llll ll ll lll ll lllll l ll llll ll ll ll ll lll ll llll ll ll ll lll l ll ll l l lll lll lll l lllll ll ll lll l ll l l ll lll ll ll l llllll ll ll ll lll l l llll l ll lll llll lll llllllll llllll lll ll llll lll l l lll lllll lllll l l ll lll lll ll ll ll lll lllll llll ll lll llll ll lll ll lll lll lll l ll ll ll ll llll lll l ll ll ll lll llllll l llll lll l lll ll l ll l ll llll lllll l llll ll ll ll llll llll llll ll lll lll ll llllll lllll ll lll ll ll l lllll ll llllll lll ll ll llll l lll l ll ll llll ll lllllll l ll ll lllllll l lll lllllllll llllll ll ll ll l ll ll l lll l lll l ll lll l lllll ll ll ll ll llll l lllll lll ll ll lll l llll l lll lllll l llll l llll l l lll l lll ll lll l l ll ll ll llll l llll lll llll ll lll ll lll lll lll ll l lllll l ll l llll lllll ll ll ll ll l ll l llllllll ll ll ll l lll l ll l l ll llll l ll lll ll lll lll llll lll lll l lllll llllll l ll l ll lll lllll l l lll ll l llll l ll lll lll ll lllllll l lll l l lll lll llll l lll ll llll llll l ll l ll l lllll lll lll l lllll lllllll l llllll ll lll ll lll ll lllllll llllll lll lllll ll l l lll lllll lll l lllll ll ll lll llll ll lll lll lll ll llll ll llllll l lll ll ll lllll ll lll l l lllll llllllll lll lll ll ll lll ll ll llllllll lllllll l l l llll ll llll lllll l llllllll ll lll ll lll llllll lll lll llllllll l lllll l lll ll ll ll lll llll ll lll lll lllll l llll lll ll l lll ll lll ll llll l l lll ll l lll llllll ll l lll ll l lll lll lllll lll lll ll ll lll llllll l llll lllllll lllll lll ll ll llllllll lll ll ll l lll ll ll l llll lllllll llll ll l ll lll ll lll ll ll lll lllll llll ll ll lll l ll l lll lllll lll llll ll ll l ll llll llllll llll llll l llllll lll llll ll ll l lll ll ll lll l l l lll lll lll lllll ll ll l lll l ll ll lll l ll ll ll ll lll llll l l ll ll l ll ll ll l lll ll l lll l lll ll l l l ll l lll ll l ll lll ll ll ll l ll ll lllll llll l lll ll ll llll l lll ll lll lll lll ll l ll l ll lll l lll ll ll l lll ll ll ll ll l ll ll l l ll l ll ll lll llll llll ll llll l l ll l ll ll ll ll l lll ll ll l ll ll l ll ll l lll llll l lll lll lll ll lll lll l lll l llll ll l l lll lll ll lll l ll l llll lll l lll ll l lll l l l lll l ll lll lllll l l ll lll l lll llll ll l lll lll ll lll lll lll lll lll lll ll l lll ll ll lll l ll ll l l ll l lll llll l l lll l ll lll l llll ll l l llllll lll llll llll lll ll l lll l ll ll l ll ll lll llll llll l lll ll ll llll l llll llll ll lll lll ll ll ll ll l ll ll ll l lll l ll ll lll ll ll ll ll ll ll lll llll l ll l l lll lll l l lll lll l ll l l ll l l lll lll l l l lll ll l l ll lll l lll ll ll ll ll lll ll l ll ll l ll l ll ll lll l ll ll l lll l lll lll llll ll ll ll ll ll ll l ll lll l llll l ll l l l ll lllll llll ll l ll ll l lllll ll ll ll lll l ll lll ll ll ll lll ll ll ll l lll lll lll lll l lll l ll ll lll lll ll l l ll ll ll ll lll lll ll ll ll lll lll llll l lll l lll ll l lll l ll ll ll l llllll ll l lll llll llll lll l ll ll l lll l l ll ll llll l l ll ll llll l l ll llll ll lll ll llll ll ll l ll ll lll l ll llll llll l ll lll lll ll l l llll ll ll ll l ll l ll l lll l l llll l lll l lll lll l l llll lll ll lll ll ll ll lll llll ll lll ll lll l ll llll l ll ll lll l lll l l lll ll ll ll l lll ll lll ll lll lll l lll lll l ll llll l ll ll ll lll ll ll l ll ll l ll l ll ll lll ll lll l l lll l llll l ll lll l lll ll lll llll l ll l ll ll lll ll l l ll lllll ll ll ll ll l ll llll l lllll l l ll lll lll l llll ll ll l ll l llll llllll ll lll ll l lll l ll ll l ll l ll ll ll ll ll ll ll ll ll ll l lll l llll l ll l ll ll l lll ll ll llll lll ll ll ll l lll l ll ll ll ll lll l ll ll ll lll l llll lll ll ll lll ll ll ll l l ll l l l ll l llll ll lll l l l ll ll l lll ll lll lll ll l ll l ll l llll l ll lll l ll l l llll ll ll l lll l llll l llll lll l ll l ll ll ll l lllll ll ll l ll llll l llll l ll llll ll llll l lll l ll ll l ll lll l l l ll l ll ll ll ll l ll l lll l l ll llll ll ll ll llll ll lll llll lll ll l l ll lllll l l llll ll lll l ll llll lll l lllll ll lll ll l l l l lllll l l lll l l ll lll l ll lll ll lllll ll ll l ll ll ll l ll lll ll ll lll ll l ll lll ll llll l llll lll lll l ll lllll l ll l l lll l l ll l lllllll ll l ll l ll ll ll lll ll ll ll lll ll ll ll ll l lll lll lll l ll ll ll ll llll l lll lll l ll lll llll lllll ll ll ll lll ll l lll ll l llll l lll ll lll lll lll lllll l lll l ll ll ll llll l lll llllll lll ll ll l lll lll ll ll ll l lll lll lll l llll l ll llll l ll ll l lllll l ll ll ll ll ll l ll ll ll ll lllll ll l ll l lll l llll l ll lll ll l l lll ll lll lll ll ll ll l ll lll llll l ll l lll l l ll ll ll l ll llll l ll ll lllll ll llll lll ll l l ll ll l lll ll ll ll l lll ll ll l lll llll l l ll ll lll l ll ll lll ll llllll ll lll ll ll l lll ll l ll ll lll ll lll l ll ll ll lllll l llll l lll l ll ll l ll llll l ll ll lll l ll llll l llll lllll ll lll l llll llll ll lllll llll ll ll ll ll l lll lll ll ll ll ll lll l l lll ll ll ll ll l l ll ll llll ll ll ll lll ll lll ll ll ll lll l l ll ll ll ll lll lll l ll l ll ll l ll llll ll lll lll ll lll lll ll llll ll l l llll ll l lll lll l lll llll ll lllll ll l lll llll l lll l l llllll llll lll ll lll ll l ll lll ll ll lll lll ll ll ll l ll llll l ll ll l lll l ll lll l ll ll lll ll l ll ll l llll ll ll ll lll ll llll l ll ll ll l ll ll l ll lll llll lll l lllll l l l ll ll llll ll llll lll ll ll ll llll l l lll ll l ll ll ll ll l l ll ll lll l ll lll l lllll lll ll llll l lll ll ll l l lll lll lll ll lll ll l ll ll lll l lll ll l ll lll l l ll lll lll lll l ll l ll ll l ll ll ll lll ll llll ll l ll lll ll ll lllll l lll l lll lll ll llll ll lll ll ll ll lll ll ll lll lll l lll lll ll lll ll l lll ll ll llll llll ll llll ll l ll ll ll l ll ll llll ll ll ll ll ll ll l ll l l lllll l lll lll ll lllll l l ll l lll ll l ll llllll lll l ll l ll ll ll ll llll l llll l lll llllll lll lll l lll lll ll lll llll ll l lll l lll l lll l ll l lllll l ll ll ll lllll ll l ll lllll l ll lll l llll ll ll l ll lll l ll lll ll lll l l ll llll lll l ll llll ll lll l lllll l l lll lll lll llll lll ll llll lllll lll lll l lll l l ll l lll lll ll ll lll ll lll ll ll ll l ll llll lll lllll l ll l lll l llllll l l l ll lll lll ll l l lll lll lll l ll l ll l l llll ll lll ll l ll lll lll ll ll lll ll lll ll ll ll ll ll l ll l lll ll llll ll ll l lll lll l l lll l ll llllll ll l lll ll lll ll ll l llll ll lll ll l ll ll lll l ll ll ll ll l lllll ll l l ll l llll llll ll lll lll l ll lll l l ll ll lll ll ll llll ll lll lll ll llll lll lll ll lll ll lll l l lll lll ll lll ll llll llll lll llll l l ll lllll ll lll l ll lll ll lll ll l ll lllll lll ll lllll lll l llll llll llll l ll lll l ll lll l l ll lll ll lll l l lll lll l ll ll lll l l llll l ll l l ll llll l lll l lll ll ll ll lll l ll ll l ll ll lll lll ll lll l ll l lll l ll lll ll l ll l ll l ll lll l ll lll ll ll lll ll ll l llll l ll lll ll l ll l lll l ll l lll ll llll ll lllll l ll l lll ll l ll lll ll l ll llll ll llll l ll ll ll lll ll ll ll ll l lll lll lll l l l ll lll ll ll llllll ll ll ll l ll ll ll lll ll lll l ll llll llll ll l lll llll ll ll l lll ll ll l lll lll lll lll l llll lll ll l lll l ll l l ll llll ll ll l ll l llll ll ll ll lll l lll ll ll l l llll l ll lll ll l ll l ll l llll ll l ll ll llllll llll ll llll lll l lll lll lll lllll l ll ll ll llll ll l ll l l ll ll lll ll ll ll l ll ll ll ll lll ll lll ll lll l llll l l ll ll ll l l llll l l lll l lll ll ll ll lll ll ll lll ll llll ll l ll llll llll lll ll l ll l ll llll ll ll lll l l lll ll llll ll lll l lll ll l ll lll l l l lll l ll l ll ll ll l l lll l ll l ll l ll llll l lll l lll l lll llll l ll ll l lll l ll ll l ll l llll l ll l ll llllll ll ll ll l ll llll ll lll l ll llll ll ll l l ll l ll llll ll l ll lll l llll l l ll ll ll ll ll ll ll ll l lll l lll l l lll lllll lll l lllll l ll l llll l ll l l ll lll ll lllll ll ll lll ll ll l ll l lll l ll l ll ll l ll lll llllll ll ll ll l l ll ll llll l ll ll l l lll ll l l ll lll ll ll lll l ll lll l l ll l l ll lll l lll lll ll l ll ll ll ll llll lll lll ll lllll l lll ll llll ll lll ll ll l ll ll l llll lll lll ll l ll lllll l lll l lll llll l l ll lll l ll lll l l ll l lll ll l ll lll l ll ll lll llll lll ll ll lll l l ll lll ll l l lll ll l ll l ll ll ll lll l llll ll ll l llllll ll ll lll l ll llll ll ll lll lll l ll l ll lll lll ll lll l ll l lll l llll l ll l ll lll l ll ll lllll llll lll lll ll lll l ll lll ll lll l lll lll lll lll l llll llll l llll lll ll llll lll llll lll llll ll lll lll llll l llll ll ll l lll lll llll ll ll l ll ll l lll lll ll ll ll ll lll ll ll lll ll l lll ll lll ll ll lll ll lllll l l lll ll l ll l lll l lll ll ll l l lll lll l ll lll ll lll ll ll l l ll ll ll ll ll ll llll ll l ll ll llll ll ll lll l ll ll lll l ll lll ll ll ll lllll ll l l ll lllll l lll lll l l lll ll l ll ll ll l lll lll l lll ll lll ll lll lllll ll lll ll ll lll ll ll l ll lll lll lll ll ll ll ll ll llll ll lll l lll lll ll ll llll l ll ll l ll llll l ll l ll ll l ll l ll l ll ll l ll l ll l ll ll ll llll ll ll ll ll ll l lll ll ll ll ll l lll llll llll ll llllll ll ll lll l llll l ll ll l lll ll ll ll lllll l l lll ll ll l ll ll llll l ll ll lll ll lll l ll ll l ll ll l lll l ll ll lll ll ll lll ll ll ll lll ll l l ll llll ll l lll l l ll lll ll lll ll l ll llll l l l ll l llll l lll l ll llll ll ll ll llll ll l ll ll l l lllll ll l ll ll l ll lll ll ll ll l ll lll ll l ll llll l l lll lll l l lll ll l l lll ll lll lll ll llllll l llll ll lll l l l llll ll llll l llll llll ll ll l ll l llll l ll ll ll ll ll ll l ll ll ll ll ll llll l lll ll ll ll l ll llll ll ll lllll ll l ll l lll ll ll ll lll lll llll lll ll l llll l ll l ll ll ll llll l ll lll l ll ll ll ll l ll ll l llll llll llll ll ll ll l l l llll l lll l l lll l ll lll ll l ll ll llll ll ll l ll l lll l lll l llll l ll l llll ll l llll ll ll ll l ll l llll ll llll l l lll ll llll ll ll ll llll lll lll ll l ll ll ll lll l l lll lll ll l l l lll l l ll l l ll l l lll ll l l lll l l lll lll ll lll ll ll ll ll l lll lll lll ll ll lll ll l ll ll l ll l ll lll l ll ll l lll l llll lllll l llll llll ll lll ll ll l lllll l lll ll l ll ll ll ll lll ll l ll ll l l l lll l ll ll ll ll ll ll l l lll l l lll ll lll l ll llll lll l lll ll lll lll ll l llll ll ll lll lll l l l lll lll l lll l l ll l ll ll ll lll ll lll lllllll ll lll ll ll lll lll l lll lll l ll l llll lll lll lll l ll ll ll l ll l lllll ll l l ll l lll l lll ll lll ll lll l lllll ll llll l ll l ll ll ll l ll lll l ll lllll l ll ll l ll l llll l ll ll llll l lllll ll l ll l lllllll l ll l ll ll l llll lll l ll ll ll ll l lll l lll l ll lll ll lll ll ll l lllll ll ll llll l lll ll l ll ll ll lll llll lll ll ll llllll l ll lll l ll ll ll ll lllll ll llll ll l ll ll l l ll lll l llll l l ll l ll lll lll lll ll ll llll l ll l llll l l lll ll ll lllll lllll l l lll l ll ll ll lll ll l l ll ll ll l ll l lll ll ll l l ll l lllll ll ll lll ll lll l ll ll lll ll ll ll l l ll ll lll ll l ll l lll l ll ll ll l ll l ll l lll l ll lll ll ll ll lll l l l ll ll ll ll l lll ll l lll lll lll lll ll lll ll l ll l llll lll l l llll l ll llll l ll l l llll ll ll ll l ll lllll l ll ll lll ll lll ll lll ll lll l ll ll lll ll l lllll lll ll lll llll l ll l lll ll l ll lll ll lll l llllll ll ll lll l lll ll lllll ll l l ll ll ll ll llll ll ll l l lll ll lll lll ll llll ll l ll ll l l lll l ll l ll l lll ll ll l ll lllll ll ll lll ll ll ll lll lllll lllll l ll l llll llll ll ll lll lll ll ll llll l l ll ll ll ll l ll ll l l l lll ll llllll lll l lll l lll lll ll ll lll ll lll ll ll ll lll ll l ll l l ll llll ll ll ll l ll lll l lll l ll lll l ll l lll ll l l ll lllll l lll ll lll l ll l llll l lll ll l llll l lll ll llll lll ll l l ll l ll lll lll ll l lllll ll lll ll l ll ll lll lll l ll l l ll llll l ll lll l l llll ll l ll l ll llll ll ll l ll l ll l lll ll ll ll ll llll ll l l lll l llll l l ll ll ll l l l ll lll lll ll lll l ll lll l ll ll ll ll l l ll ll ll lll l l lll l lll lll ll lll l ll l ll llll ll ll l lll ll ll llll l llllll l lll ll ll ll lll lll ll l llll l lll lll lll l l lll lll l llll l ll ll ll llll ll ll lll ll lll ll l llll l ll l lll ll ll l ll ll lll lll lll l l ll ll l l ll lll l llll lll ll l ll ll l l lll l l llll l lll l llll ll ll llll lll l l lllll ll ll llll l ll lll ll l lll ll l ll l l lll ll ll l lll l ll l llll lll lll l ll ll lll llllll l ll ll l ll l ll lll l ll ll ll l llll l l lll lll l lll l llll l lll l llll l l lll ll lll lll ll ll lll lll ll l ll ll ll ll l lll ll lll llll lll lllll ll ll ll ll l ll llll l ll lll l l ll ll ll l ll l ll ll ll lll llll l l lll llll l ll l l ll l ll l llll l l llll l l ll ll llll lll ll lll llll l ll l l lll ll ll llll lll ll ll llll ll ll lll l lll llll l ll ll ll llllll lllll ll ll ll ll l lll l l l ll lll ll lll l ll lll l lll ll l ll lll l ll l l ll lll llllllll ll l lll lllll l l llll l lll llll llll ll lll llll ll l lll ll lll ll l ll l ll l l ll ll l l lll llll ll ll l lll ll ll l llll ll lll l ll l ll lll ll l ll l ll l ll ll ll ll lllll llll lll ll l lll ll ll ll lll ll lll lll l ll ll ll l l l ll ll ll ll l ll l lll ll ll l ll ll l ll l ll lll lll ll ll lll ll ll ll lll lll l ll lll l lll ll ll llll ll lll l ll ll ll ll ll l lll l ll lll l ll ll l lll l ll ll l ll ll ll ll ll l lll lll l lll ll l lll lll l ll ll lll l ll lll ll ll l ll l llll ll lll ll ll ll ll ll ll ll ll l llll ll lll l ll ll lllllll lll ll l ll l ll lll l lll lll ll l ll lll ll lll ll lll ll ll ll ll lllll l ll lll ll ll ll l lll l lll l ll l ll ll l ll ll l llll l ll ll ll ll l ll l lll llllll ll ll llll l lll l ll l ll ll lll lll lll l l ll lll ll l ll ll l ll l ll ll ll ll lllll l l lll ll ll lll ll lll lll l ll ll ll lll ll ll l ll ll ll ll l lll l ll l lll lll l l ll ll ll lll ll ll l ll l lllll ll ll l lll ll lll ll ll ll ll ll l lll ll lll lll lll ll lll l lll l lll llll l l ll l llll l lll l ll l lll ll ll l ll ll l l llll lll l llll l lll lll lll ll l l l ll l lll l lllll lll ll ll lll l lll l llll ll lll l l lll lll lll l lll lll lll llll ll ll ll ll ll l llll l lll ll lll lll lll lll lll l lll lll l lll l lll ll l ll ll ll llll ll ll llll ll ll lll lll ll lll l ll ll l lll lll l ll ll ll llllll l lll ll ll l llll ll l ll ll l l llll l l ll l lll l l ll lll lllll l l ll l l ll ll lll llll ll lllll lll lllll l l l l ll ll l ll lll lll l ll l llll l ll lll ll l llll ll ll llll ll lll ll l ll ll lll llll llll ll lll l l ll lll ll l ll ll l l lll lll l llll ll lll ll lll l llll ll llll lll ll l ll lll l ll ll ll l llll lll ll llll ll ll l ll l ll ll lll l ll l lll ll ll l l l lll l l llll ll l ll ll ll l llll l l ll lll ll ll ll lllll l ll lll ll ll llll ll ll l ll ll ll ll llll l ll l lll l lll lll ll llll l lllll lllll ll ll l l llll ll l lll ll ll ll l ll l lll l l ll ll lll l llll lll l l ll ll llll lll lll ll l llll ll ll lll lll ll l ll l lll l llll ll ll l l ll lll lll ll llll ll ll lll l ll ll lll l lll ll l l ll l ll ll l lll l lll l lll ll lll ll ll lll lll ll llll ll ll lll l l l llll ll l llll ll lll l ll lll ll ll llll llllll l lll llll l ll ll ll ll llll ll ll lll ll ll ll ll lll l ll l llllll ll ll lll ll l lll l l lll llll ll ll l lll l l llll l ll ll ll l l l lll ll l lll lll l ll l llll l lll ll ll ll ll ll lll llll lll lll ll lll ll ll l l ll lll ll ll l lll ll l ll ll ll ll ll ll lll l ll l ll ll ll llll ll l ll l l lll lll l l l llll l l lll ll l l ll ll ll l ll ll ll lll ll l l llll lll ll l lll ll ll ll lll ll llll l lll ll lll ll lll l lll ll ll lll llll l ll lll l lll ll lll ll ll ll lll ll l llllll ll l ll lll l l lll l ll llll lll ll lll l lll l ll ll l lll lll ll ll ll ll ll l lllll ll ll llll lll ll llll lll ll lll ll l lll ll llllll ll ll l l lll l ll ll l llll ll ll l ll lll ll l lll ll llll l lllll ll ll l lll ll llll l ll lll l ll l ll ll l l ll ll l ll ll l ll llll lll lll l ll lll l lll ll l ll ll ll ll ll ll lll ll llll ll l l lll ll ll ll ll ll lllll llll l ll l ll l ll l ll ll l ll lll l ll ll l ll l lll l ll ll l ll ll l lll l lll llll l ll ll lll ll ll llll l l ll lll ll l ll ll lll lll ll l ll l lll l lll ll lll ll l l ll llll ll l ll l ll ll l ll ll l ll l lll l ll llll ll ll ll lll ll ll lllll l lll llll lllll l ll ll lllllll lll l ll l ll lll l ll l ll l l llll ll llll ll l lll l lll ll lll ll l lll l ll ll ll llll l lll ll ll ll l lll llll llll ll ll l ll l l lll l lll lll l l lll ll ll l ll ll ll ll ll alpha=0.01alpha=0.1alpha=0.5alpha=10.000.250.500.751.000.000.250.500.751.000.000.250.500.751.000.000.250.500.751.00 0.00 0.25 0.50 0.75 1.00 S S S l ll l lll ll llll lll lllll lll lllll ll ll l llll ll ll llll llll lll ll lll lll lll lll lllll ll lll ll llll l lll lll llll ll ll ll l ll llllll ll ll lll ll l lll lll lll ll lll llll llll llll lll lll llllll ll ll ll lllll ll ll ll llll llll lll lll llllll llll lll ll ll lllll llll llll l llllll ll lllll lllll lll l ll lll llll ll ll llllll lll ll ll ll lllll lll ll llll ll ll ll lll lllll ll ll llll l lll l ll ll llll ll lllllll llllll lll ll lllll ll ll ll lllll lll l ll lllll lllll ll ll ll llllll ll llll ll lll lll lllll lll lllll llll lll ll lll l lll ll ll lll l ll ll ll lll llll ll ll lll llll llll ll lll llll ll ll ll lll l llll l lllll llll ll ll lll ll lll lllll l llll lllll ll llll ll lllll lll ll ll lll l lllll llllll llll lll ll lllll lll ll llll lll ll lll lll lllll lll l ll ll llllll ll llllll ll ll ll llll ll llllll l ll lll ll lll ll lllll ll llllllll lllll lllll ll lllll l lllll ll l lll l lll ll lll ll l ll lll llll ll lll llll lll lll ll lll l lllll lll ll ll lll lllll lll lll lll ll ll ll llll ll ll lll l llll lll lll ll l llll lllll llll llll lllll l lll ll ll lllllll ll lllll llll lll llll ll ll ll llll ll lll ll ll ll ll lll ll llll llll lll ll llll ll llll l ll lll lllll lllll ll llll ll l lll lll lll lll ll lll ll lll ll lllll ll ll ll lll ll lllll lll lll llll lll llllll lll ll lll ll ll ll llll llll l lll l llll lllll ll ll lllll l llll lll lll llll lllll l lll lllll lll llll llllll ll lllll lll ll lll llllllll llllll l lll l lll lll ll lllll l llllllll lll lll lll l llll l lll ll lllll ll ll llllll lll lll l lll lll lll l lll ll ll ll l llll lll lll lll llllllll ll lllll lll lll ll ll ll l llll lll ll llll ll lllll ll llll ll lll ll l ll lll llll lll llllll lll lll ll lll lll llll ll ll lll llll ll lll lll lllllll llll l ll ll lllll ll lll l llll ll lll llll ll lllll lllll llll l lll lllll ll ll l lll ll llll llll ll l lll lll llll llll ll lll lll l llllll lllll llll lll ll l lllll l ll l ll ll lll lll lll l ll lllll llll lll llll l lll lll lll llll llll ll ll lllll lllll llllll ll llll ll lll l lll ll l llllll l ll l lll lllll lll lllll ll l llllll llll lll llll llllll lll ll lll ll ll llll ll llll lll llll ll ll ll ll lll ll lll ll ll lll ll l llll ll lllllll lll lll lll l lll llllll llll l lll ll llllll lll l ll lll lll lll ll llllll ll ll l ll l llll lll l lllll ll lll llll llllll l lll ll llllll llll lll ll lll lll ll lll ll lll llll l ll l lllll lll ll lll llllll l llll ll l ll ll lll llll ll lllll lll ll llll l ll llll lllll l llllllll ll l lllllll lllll ll l lll lll lll ll lllll ll ll ll lll lll ll ll ll lll ll ll ll lllll l lll l llllll llllll lllllll lll l ll ll ll l ll lll lll lll lllllll ll llllll llllllll lll llll llll l ll l ll lll l lll l ll l lll lllll lll ll ll ll lll llllll lll llll l ll ll llll lllll l lll l lllll lll llll lll llll lll lllllll lll lllll lll ll lllll llll ll ll l llll ll lll l lll lllllll llllll llll llll l lllll lllll lll lllllll lll ll lll lll lll ll llll lll lllll ll llll l ll ll l llll ll lll l lll l lll ll llll l ll l ll lll lllll ll llll lll ll ll lllll l lll lll llll llll ll llll ll lll ll llllll ll llll llll lll lll lllll llllll lll ll llll ll lll ll l ll ll ll ll l llll lll lllllll ll lllll llllll ll lll l ll ll llllll ll lll lll llllll l ll ll l lllllll llll llllll lll ll llll llll l ll lll l lll llll lll llll lll llll l ll lll lll lll ll ll lll l ll l lll llll lll lll llll lllll lllllll ll ll lll lll l llllllll llll lll llllll ll lllll ll l ll lll llll ll l lllll l lll llllll l lllll ll ll lll llllll l ll lll lll llll ll lll llllll ll llll l lll lll lll l lll ll lll lll lll lll l lll ll lllll l ll llll llll lllll llll l lll ll lllll ll llllll ll lll lll lllll l lll llll ll llll lll l lll llll llll llll ll l ll lll llll l ll ll lll lllll lll lll llll l llll l lll lll l lll llll lll llllllllll lll lll ll lll ll lll ll ll ll ll llll llll lll l lll ll ll llll lll ll llll lllll llll lll ll ll ll ll lll lll l lllll ll ll ll lll ll ll llll lll ll lll ll l lll l llll lllll lll lllll ll ll lll llll lll lll lll lll llll ll ll l l llll lll ll ll l ll lll lllll lll lll llll ll lll lll l ll llll lllll lllll ll l llll lll ll l llllllll lll lll ll llll ll lllllll lll ll lll lllll ll ll llll ll llllll lll lllll lll l ll llll ll ll lll llllll lll lll ll ll llll lllll lll l ll ll lll ll llll llll ll ll ll lll ll ll llll lll llll ll lll lllll lll lll lllll lllll ll ll l ll l lll llll ll ll lll lll lll ll l ll llll lll ll l l lll lllllll ll ll lll lllll l lllllll l lll ll lll ll ll lllll llll l llll ll llll ll llll lll lll lllll ll lllll lllll l llllll ll ll lll lll ll l llll llll ll ll llll lll ll lll llll llll ll llll lll lll lll lll llll l llll lll ll llll l lll llll ll lll lll lll llll l lll lll lll llll ll lllll lll ll lllll l ll lll lll llll l lll lllll l llll llll llllllll llll ll lll l ll ll lll lllllll lll ll l ll ll lll llll llll lll llll ll l lll l lll l ll ll lllll lll llll l lll lll lllll llll ll ll l ll lll ll lll ll lll ll llll lll lllll l lll ll lll lll lll l ll ll l lll llllll l ll llll lll lll lllll ll lllll lll l lllll llll lll lll lll l lll l llll lll ll lllll lllllll l l llll lllll llll ll l lll lllll lllllll lll l llll lll lll lllll ll lll llll lll l llll lll llll l lllll l ll ll llll llll ll ll ll lll ll lllll lllll llll llll l ll llll llllll lll l lll ll llllll llllll lll llll l llll lll lllll llll llll ll llllll llll lll ll lllll lll l lll lll l ll llll ll ll ll lll ll lllll lll ll l lllll lll ll lll lll l lll lllll lll l ll ll llll lll l lll llll ll ll llll lllll ll lll l ll llll ll lll l lll lll lll lllllll l ll lll llllll lllll ll l ll lll ll ll lll lll ll lll ll llll llll lllll lllll l lll ll l lllllll ll ll lll ll llll ll ll l ll llll ll ll lll ll llllll ll ll lll ll lllllllllll ll ll l llll ll llll lllll l ll ll lllllll ll llll ll llll ll llll ll llll lll ll llll lll llll lllll lllll l llll llllll llll ll ll lllll lll lllllll lllll ll llllll lllll l ll lll l ll lll lll l llllll ll ll lll llllll lllll lll ll lll lll llll llll llllll ll ll lll lll ll llll lll ll ll ll l l lllll ll ll lll l ll llll l llll lll ll ll l ll ll ll l ll ll llll ll ll ll l lll lll ll llll llllll l ll lllll lllll l ll lll lll ll ll llll ll llllllll lll ll lllll ll ll llllllll l lll l ll lll l ll lll ll lllll llll lllll lllll lll lll llllll ll lllll lllll llllllll ll lll l ll l ll ll lllll llllll llll llll ll llll lll lll ll ll ll lll llllll llllll lllll ll ll llll ll llll ll llllll lll l lll llll ll lll l l lll lllll ll llll llll ll lll lll lllll ll l ll l llll l llllll lllll lll ll lllll lllllll llllll l llll ll llll ll ll lll ll ll lll llll ll ll llllll ll ll l ll ll lll ll lll ll lll ll lll l ll l lll ll lll l ll ll ll l llll lllll l l llll lll llll llll lll l lllll l l lll l l llllllll llll llll lll l l lll ll lll l ll llllll lll llll lll l ll ll ll llll lll l lll ll lll l lll lll ll ll lll ll llll ll llll llll ll ll ll ll ll l ll ll llll l ll ll lll ll l lllll l ll ll ll ll lll lll ll llll lll ll lll lll lll llll ll l ll ll ll lllll ll ll ll llll ll lll llll l ll llll ll ll ll lll lll ll l ll ll lll llllll lll lll llll llll l ll l lll lll lll lll ll l lllllll lllll ll ll ll llll l lll lllllll l l ll lllllll llll l lll lll lllll l ll l lllll llllll l l lllll llll ll lll llll lll l ll lll lll l lll lll l lll l l lllll l lll lll l lll llll ll l ll ll lll ll l lllll ll ll ll ll llll l llll ll ll l ll ll ll ll llll l ll ll lllll l llll lll ll lll ll lll l l lll lllll lll ll l ll l l ll lll ll lll llll l l llllll ll llll llll l ll ll ll ll ll l ll lllll ll lll ll ll lll l llll ll lll ll ll llll l lll l llll ll lll ll ll lll ll l ll l ll l ll lll lll l llll ll ll ll lll ll ll ll l l lll llll ll l lll llll lll l lllll ll ll lll ll lll l ll lll ll llll lll ll l lllll lll ll ll ll lll llll lllll lllll llll lllll lll lllll l lll l llll ll llllll lll lll lll ll llll l ll llll l llll llllll llll l ll ll ll lllll ll lll lll llllll ll ll ll lll llll ll lll lllll lll lllll lllll ll ll l ll lll lll l lll llll l l ll l lll llllll ll lll l ll lll l llll ll lllll lll llll l l llll ll lll llll ll lll ll l llllll lll ll l llll ll l ll l lllllll ll ll ll ll llll ll lll l ll ll ll ll lllll lllll l lll l lll lllll llll ll llll ll l lllll ll ll lll ll ll ll l llll llll ll ll lllll ll llll llll lll ll llll lll l ll lll lll ll ll l ll lll lll ll llll ll llllllll llll ll llll lll ll l l l ll l lll ll lllll ll ll l ll ll llll lll l ll lllll llll l llll l l lllllll l ll ll l lll lll llll ll lll lll llll llll ll l ll l llll ll ll lll l lll ll lll lll lll l llll lll l lll lll lll ll l lll llll llll lll l lll ll ll ll ll l lll ll ll lll l ll l l llll lll lll lllll ll lll lll llll ll l lll ll l ll llll l ll lll l ll lllll l lllll llllll l l ll l lll l ll ll llll l l llll ll lll lll ll lll lll ll llll l lll lllll ll lll ll llll l ll ll ll l l lll lll l lllllll ll lllllll l ll l ll l l ll ll l ll ll lll ll ll ll ll ll ll lll l ll lll ll l llll ll lll l lll ll ll ll ll ll lll l ll l ll lll ll l ll lll l ll lllll llll ll ll llll lll ll l lll ll l ll ll l lll lll ll l lll ll l l ll ll ll lll ll ll lll ll ll l ll l l lll lll ll ll lll l lll l lll llll lll lllll lll lll lll lll ll lll l lll lll ll lll lll lllll l lllll l lll ll llll ll llll ll l ll ll ll ll ll ll l lll lllll ll ll llll llllll ll llllll ll ll l l llll lll l ll l lll lll ll lll llll lll l ll lll ll ll llll ll ll lll lll lll lll ll llllll l llll llll lllll ll llllll lll ll lllll l ll ll l llll ll ll llll ll ll llll lll lll ll ll lllll llll lll ll ll lll l ll l ll ll ll ll lll llll l ll ll l l ll l lll ll llll lll ll llll lll lll l ll ll l lll l llll ll lll llll ll ll lllllll llll ll lll ll lll llll lll l ll l lll ll llll ll llll l ll ll ll l lll ll llll l lll l lll l llll l ll llllll lll l lll lll l ll lll ll l ll ll lll ll l ll ll ll llll lll ll l ll lll lll l ll ll l llll ll llll ll lll ll ll llllll llll ll ll lll ll l lll l l lll llllll l lll l l llll ll ll ll ll lllll ll lll lll ll l lll ll ll l lll ll ll ll l l lll llll llll ll llll lll ll lll lll lll lll l ll lll llllll lll lll lll ll l lllll llll ll lll ll ll ll ll llllll l ll ll ll lll ll ll llll lll lll ll lll ll ll ll ll ll l llll ll l ll l lll ll ll lllll ll lll l llll lllll lll lll ll ll l llllll lll lll llll lll lll ll ll l lllll ll lll ll l ll l lllll llll ll ll llllll l lllllll ll ll ll ll ll lll lll l lll ll llll ll lll l ll ll l lll llll l l l ll ll l ll lllll ll lllll l ll llll lll lll ll ll ll ll ll l lllll lllll llll ll l llll ll ll llllll ll ll ll llll l ll l llllll llll ll lll ll l ll ll l l lll ll l ll ll lll l lllll llll l ll llll ll ll ll l ll ll ll lll lll llll lll l ll lll l ll ll llll lll l llll ll ll lll ll ll ll ll l llll lll llll lllll llll lllll ll ll ll l llll lll llll ll l l lll l ll lll ll ll ll l llllll ll ll ll l ll ll lll ll l llll lll llll ll ll l llll lll ll ll ll l llll ll lll l lll llll ll llllll ll ll llll l ll lll ll llll lll ll ll llll lll l llll llll ll ll lll lll llll llll ll l lll l ll lll l ll ll llll llllll l lll l ll l lll l llllll l l llll ll l lll ll lllll llll ll l lll l ll llll lll llll l ll l lllll lll llll l llll lllllll ll l llllll ll ll lll l ll l lll l llll lllll lll lll ll lll ll l lll llll lll lll lll llll l l lll ll l l ll l ll lll l lll ll ll lll l lllllllll l ll ll l ll ll ll ll lll lllll lll lll ll lll llll l ll llll ll l lll ll l lllll l lllllll lll ll lll l llll l ll llll lll llll ll l ll l llllllll lll ll lll ll llll ll ll ll ll ll l lll lllllll lll l l lllllll llll ll llll ll lll lll llllll lll ll ll lll l ll ll ll lll lll llll ll lll l ll llll ll l llll l ll lllll ll l llll ll l l ll ll ll ll lll l ll ll l lll ll lll lll ll l ll lllll lll l l ll ll ll lll lll ll lll ll ll ll ll lll lll lll lllll l l lllll llll l lllllll ll ll lll lll ll lll lllll ll ll ll lll ll ll llll lll l lllll ll llll ll l llll ll lll ll l lll ll ll ll ll ll llll llllll l l lll l ll ll llll ll lll lll lll ll ll lllll l ll ll ll ll ll ll ll ll l ll ll ll ll llll ll l ll llll ll ll ll ll llll ll ll ll lll ll l llll ll l lllll lll ll ll lll lll lll l ll ll lll ll l l llllll ll lll lll lll lll l lllll lll ll lll l ll ll l lllll ll ll ll l lll lll l ll lllllll l l ll lll l lll lll ll lll l lll ll ll l lll llllll l lllll lll l l lll l ll ll l ll l lll lll ll llll ll ll l lll lll l ll ll llll ll llll ll lll l lll ll lll l lll l lll lllll ll ll lll llll l l ll llll lllll lll lll lllll lllll ll ll ll ll lll ll l ll l l lllll ll ll ll l lll l ll llll ll ll lll ll l lll ll l ll l l lll ll lllll ll lll l ll lll llll ll lll l llll l l ll ll lll llll ll l lll l llll lll l ll ll ll lllll llllll lll lll lll lll ll l ll lll l ll l ll ll l ll lll ll l lll ll lll l ll ll lll ll lll ll ll l llll l llllllll lll l ll l l llll ll ll ll ll lll lll ll lll ll lll ll llll ll ll lll ll ll l lllll ll ll ll l ll lll l lll ll ll lllll l lllll ll l llll ll ll ll ll lll ll ll ll l l llll ll ll ll ll lll ll lll ll l ll lll ll lll lll lll ll lllll l ll lll llll ll ll lll lll llllll llllll lll ll ll l llll ll lll l lllll ll llll lllll lll lllll ll ll lll llll lll lll ll l llll l lll llll lllll lll lll ll llll ll llll lll lll ll ll ll lll llll l llllll ll ll llll ll llll l lll lll llll ll ll lll lll llll lllllll l lll ll lllll ll llll lllll llll lll ll l ll l lll ll ll l l ll llllll ll l l lll lll llll ll llll ll lllll lll l lll lll llll ll ll l ll lll ll ll ll lll lll ll lll llll lll l ll lll lll lll ll ll l ll lll ll lll ll lll l lll lll llll lll llll lll lllll llll ll lllll ll lll lll lll lll l ll l ll lllll lllll lll l ll ll ll l llll ll lll lll ll lll ll l lll lll llll llll llll llll l lll ll lll ll lll l llll llllll lllll ll ll l l lll l lll ll lll ll ll l lll ll l l lll l ll lll ll ll ll ll lll l l lll ll ll llll lll l ll ll ll ll lll ll ll l l llll l ll ll lll ll l ll l lll l lll lllll ll llll l l l ll lll ll ll ll l llll lll l l llll l ll l ll ll ll ll l lll ll ll l ll l lllll l lll ll lll l lll lll ll lll ll ll ll lll ll ll ll ll l ll l l lll lll l llll lll ll ll l ll l llll ll ll l lll lll ll ll lll ll lll lll llll llllll l lll ll ll l llll lll ll ll l ll ll lll lll ll llll l l l lll l ll l l lll lll l lll ll lll l ll l ll ll l l ll l ll lll llll llll lllll lll l ll lll ll l ll l lll lllll lll l lll ll ll ll ll ll ll l lll l lll llll l l lll l ll lll lll llll l l lll ll ll ll lll l lll lll l ll l lll lll ll l ll ll ll l llll lll l l lll ll l l lll l l lll l llll ll lll ll l ll ll ll ll l ll ll ll llll l ll ll lll llll ll ll ll ll lll ll ll l l lll lll lll l l ll l llll ll l l ll l l lll lllll l llll ll l ll ll ll lll ll l l ll ll ll l ll l ll ll l ll llll l lll l l l lll ll l l lll lll ll ll ll ll ll ll ll ll l lll l l l ll ll l ll l ll lllll ll llll ll l llll l llll lll ll ll ll l l ll lll ll ll ll lll ll llll l ll lll l lll llll lll l ll ll lll llll ll l ll ll ll ll lll ll l ll l lll llllll lll l l llll lll ll l lll l ll ll lll lll ll l lll lll llll lll lll l l lllll lll l lll ll llll l l ll ll lll ll lll llll ll ll ll l lll l ll lll ll ll lll l ll llllll ll l ll lll lll ll lll lll ll ll ll l ll l ll lll ll lllll l ll l ll ll lll l l llll lllll lll llll ll lll llll l l ll l ll lll lll llll lll ll lll l lll ll lll l l ll ll l lll l l lll ll l ll lll l ll l llll ll lll l l ll ll ll ll l ll l l l lllll ll lll ll ll lll llll lll ll llll l ll l lll ll lll lll ll ll l ll l ll ll llll ll l ll lll l lll ll ll ll l ll ll lll lll ll l l ll lll lll l lllll l ll l ll l lll l lll ll l ll lll l ll l lll ll ll l ll l ll ll ll ll lll ll l ll ll ll l ll ll llll l ll l ll ll l lll lll l ll ll llll l l lll l lll lll ll ll ll llll ll ll ll lll l llll lll ll ll lllll ll ll l lll ll lll l llll ll ll lll l ll lll l ll l l lll lll l ll ll l lllllll l ll lll llll l llll l l lll l ll l ll ll l lll l lll l ll l l l ll ll lllll l ll lll ll llll l lll ll ll lll lll ll l l llll lll lll ll ll l ll l ll l ll llllll l ll l lll l lll lll l ll ll ll lll l ll lll ll ll lll lll l ll lll lll l llll ll lll l ll lll l lll l lll ll ll lll ll ll l l lll lll l lll ll ll llll llll l ll l l lll ll ll l ll ll lll ll ll l llll ll llll ll ll l ll llll l llll lll lll l ll llll l l ll l l lll ll ll l lll llll ll l lll ll ll ll lll llllll lll ll ll ll ll l ll l lll llll ll llll l lll ll l lll llll l l lll llll lllll lll ll ll l l ll l ll ll ll llll l ll l ll lll lll lll llll l l llll llll ll lllll lll lll lll ll lll ll l lll ll l ll ll lll lll lll ll l lllll l ll lll l lll lll lll ll l ll llllll ll ll lll ll llllll l lll ll l lll l lll l l ll ll l llll ll l ll lll lll ll ll ll l ll lll llll l lll lll ll ll ll ll l ll lll ll ll ll ll lll ll l lll lll ll l l ll lll lll ll lll l l lll ll ll lll l llll ll ll ll lll l ll ll lll ll llll ll ll l ll ll l l l lll ll l ll ll lll ll lll l ll ll ll lllll l lll l l lll l ll llll l lllll ll ll lll l ll lllll llll ll ll lll l lll llll lll l l l llll l lll ll l ll ll ll lll l lll ll ll ll ll lll l l lll ll ll ll lll l ll ll ll ll ll ll ll ll l ll ll l ll ll ll llll l ll ll ll ll ll ll ll l ll l ll ll l ll llll ll lll lllll ll l ll l ll l l ll ll l l llll ll l lll ll ll lll l lll ll llll ll l lll l llll l lll ll llllll llll ll lll l ll lll l lll ll l ll l llll l ll ll lll ll lllll ll ll l lll l ll llll ll ll lll ll l ll ll l ll llll ll ll lll llll l l l ll ll ll lll ll l ll lll llll ll l l lll ll ll l ll ll llll l ll lll lll l l ll ll lll l l l lll ll l ll ll ll ll l l ll ll llll lll lll ll lll lll ll lll l l lllll ll ll llll ll lll ll lllll l ll lll ll l lll ll l ll lll l l ll lll lll llll lll lll ll ll ll ll lll ll lll l lll ll ll l ll ll ll ll ll llll ll ll ll ll lll lll lll l l l lll lll ll ll l ll lll l ll l ll l ll ll llll l ll ll ll lll l llll llll llll l ll lll ll ll ll llllll ll ll ll l l ll llll l ll ll ll ll llll ll lll ll l l lll lll ll l ll lll lll lll l ll l ll llll ll llll l llll l lll lllllll lllll l lll ll lll lll l lll ll l lll lllll lll l ll llllll l llll ll ll lll lll ll ll lll llll ll l ll ll ll lll ll lll l ll lll llll ll l ll ll ll llll l l ll lll l l ll lllll ll l lll lll lll llll ll l ll lll l ll lll lll ll l l ll ll l ll l ll ll ll ll ll lll ll ll ll l ll ll l ll llll lll lll ll lll l llll l llll ll l lll lll lll lll l lll l ll lll l ll l ll l l lll l ll lll lll llll l lll ll ll lll ll lll ll ll ll ll l l lll l ll l ll lll l llll l lll lll ll llll l l lll ll llll l ll ll lll ll lll lll l ll lll ll l ll ll lll l ll ll ll ll l lll ll ll ll ll l llll llll ll lll llll ll lll l l ll ll llll l ll llllll ll l lll llllll l llll l ll ll lll ll l l l lll lll ll lll lllll l l lll lll ll lll ll lll l ll ll lll lll ll ll l lll ll l ll lllll lll l l lllll ll l l llll lll l ll lll ll lll lll lll ll ll lllll ll ll l lll lll ll l ll lll l l llll lll ll ll llllll l l l lll ll ll ll ll l l ll ll ll l ll lll ll l ll lll l ll l lll l ll lll ll l ll l lllll lll l ll lll lll l lll ll ll l ll lll ll l lll l l lll ll ll l l ll ll lll lll ll lll ll l ll l lll lll ll ll lll l ll ll ll llllll l ll ll ll lll ll ll ll ll l lll lll llll ll ll lll ll ll llll ll llll lllll ll ll ll l ll lll lll ll ll llll lll lll l lll ll lll lll ll ll l l ll lll lll ll l l llll lll ll l lll l lll l ll lll ll lll l ll l llll ll ll ll llll l ll ll l ll l lll ll llll lll l ll lll l lll l ll l ll ll lll lll lll l ll llllll l l lll lll lll llll lll ll l ll lll l ll l lll l ll ll lll l l ll ll l ll ll ll ll lll lll ll ll ll l ll lll ll ll ll ll l l lllll llll l lll ll ll ll ll lll ll lll ll lllll l l ll ll ll llll lll ll ll ll ll llll ll ll lll l lll l ll ll ll ll lll l lll lll ll lll l l l lll l ll lll ll ll l llll l ll l lll l lllll l llll lll ll ll llll lll lll lll l ll ll l ll l llll lll lll llll l lll ll lll ll ll llll lll l ll lll l l llll l ll l llllll lll ll llll llll l lll l lll ll ll ll ll ll l lll llll l l ll l ll lll l lll lllll ll ll llll l ll l lll lll ll ll lll ll ll lll ll l l l lll lll l ll l ll ll l llll l ll llll ll ll ll l l ll llllll l ll ll l l lll lll l ll lll ll ll ll l l lll l l l l ll l l ll lll ll ll lll ll l ll ll ll ll ll ll lll lll ll llllll lll ll l lll lll lll l ll l ll ll lll ll ll ll ll ll lll llll lll ll l ll l ll lll l ll ll l l ll lll l l ll l llll ll ll lll l ll ll lll l lll l ll l lll llll l ll ll l ll l l lll ll l ll ll l ll ll lll l lll l ll ll l lll ll l ll lllll l ll lll l ll ll l ll lll l lll ll lll l l lll ll l l ll l llll ll ll lll l ll ll l l ll ll llll l llll lll l ll ll lll l ll lll ll llll lll ll ll lllll lllll lllll ll ll lll ll llll ll l ll ll ll l lll l lllll l ll ll ll l ll ll ll ll l lll lll llll ll lll lll ll lll lll llll ll ll lll ll ll lll ll l lll ll lll ll ll l ll ll llllll l lllll l lll l ll l lll ll ll l l lll ll ll lllll l l ll l ll ll ll ll ll ll ll ll ll llll lll ll ll lll l ll ll lll l llll llll ll lll ll ll ll llll l ll ll ll lll ll l ll l l ll l l ll l lllll l l ll l lll lll l ll l ll ll l ll ll l ll lll ll ll l ll ll ll l ll ll l ll lll lll lll ll ll l l ll ll llll l l lll l lll lll ll ll lll ll ll ll l ll lll l lll l ll ll l ll l ll l ll ll l ll l ll l l l ll ll ll l ll ll ll ll ll lll l ll ll ll ll lll l ll l l ll l l ll ll l l l lll ll ll l lll l l lll ll lll l ll ll ll lll ll llll l ll ll lll llll ll lllll ll lll ll l lll ll lll ll lll l lll ll ll lll ll lll ll ll ll lll ll llll ll l l ll lll l llll ll l ll ll l ll lll ll lllllll lll l llll l lll ll l l ll ll ll ll l l ll lll ll llll l llll lll ll lll ll l ll ll ll lll ll l ll lll ll l l llll lll l llll l ll lllll ll ll l ll l ll ll ll l l llll l ll ll l lllll l l ll ll l llll llllll ll ll lll llll l lll ll ll ll ll ll lll ll ll ll ll ll l llll l ll ll ll lll ll l l ll ll ll l ll ll lll lll lll ll ll ll l ll l ll ll ll lll lll l l lll lll ll ll ll l l lll ll l lll ll ll ll lll ll lll l l lll l ll l lll ll ll llllll llll lllll llll ll l ll lll ll ll ll ll ll lll lll l lll llll l l lll llll l ll lll l l ll llll lll lll l l ll ll ll llll l ll ll l l ll ll ll ll l l ll l ll l ll lll ll ll ll l llll l ll l ll lllll l llll llll llll lll llll lllll l ll l ll ll lll ll ll lllll l ll l ll l ll ll ll l ll lll ll lll lll ll l lll ll lll llll l l ll l l llll l l ll l l llll lll ll llll l l lll l ll lll ll ll ll ll l ll lllll lllll l lll ll ll ll ll llllll lllll l llll llll ll l lll l lll l ll ll lll lll lll l lll llll lll l lllll l ll llll l llll lll ll ll ll l ll ll l ll l lll llllll ll ll ll l ll l lll l ll l lll lll l l ll l ll l ll llll ll ll lll lll l l lll llll lll l lll l ll ll l ll ll llll l l l ll ll ll lll lll ll ll lll ll l lll llll l lll ll lll llll l lll ll ll l l lll l ll ll lll llll ll l l lll lll ll lll l l ll l lll ll ll ll lll l lll l lll ll l ll lll ll ll lll l llll ll llll lll l ll lll ll llll l lll lll l ll ll ll l llllll ll llll ll ll ll ll ll l ll lll l ll lll ll llll ll l lll l lllll lll ll l ll l ll l ll ll ll ll lll llll l lllll ll ll ll ll l lll l l llll llll ll ll ll l ll llll ll ll lll lll l ll ll llll lll l ll ll ll ll lll ll l lllll ll lll llll llll ll ll lll lll lll l lll ll ll lll lll llll lll ll l ll ll ll ll l lllll ll ll ll llll ll lll l ll l ll l ll l ll ll lll lll llll l ll l llll ll lll ll lllll llll l l ll ll ll lll ll l l l lll ll ll l ll ll l ll ll l ll ll l lll ll ll l ll lll ll lll lll lll ll ll lll lll lll lll ll l ll ll l lll ll ll ll llll l lll l ll ll ll lll llll ll ll ll ll l l ll ll llll l ll lllll lll ll ll ll lll ll llll l lll lll lll l ll ll lll ll ll lll ll ll l ll ll ll ll lll l l l lll l llll lll l l ll l l ll llll l ll l ll ll ll llllll llll ll lll ll lllll l ll ll lll l ll l lll lllll ll lllll ll l ll ll l ll ll ll ll l ll lll llll ll l lll ll ll lll ll l lll l lll ll l lll lll lll llll ll l l llll l llll l lll lll l llll l ll lll l llll lll ll l l ll l ll llll lll ll l ll l ll lll l l lll ll lll lll ll ll l ll l lll llll ll lllll ll l llll ll ll lll lll ll ll ll lllll lll llll ll ll ll llll ll ll llll l lll l lllll ll ll lllll ll l ll l ll ll l lll ll l lll ll ll ll llll ll ll ll l llll l lll l lll ll ll l lll lll ll ll ll ll lll l ll ll ll ll lll l l l l lll l ll ll ll ll l lll ll lll l llll l lll ll l llll lll l lll l l lll ll ll ll l l ll ll ll l ll ll l ll llll l lll llll ll ll lll ll ll lll llll llll ll llll ll l lll l llll ll lll ll llll l llll l l lll l lll l l ll ll ll l l ll l llll l l l ll ll ll l l lll ll l ll lll l ll lll lllll ll ll lll l lll lll l lll l lll lll ll ll l ll l lll lll lllll lll lll lll ll ll lll l l llll l lll lll llll l l lll l lll l l llll l ll ll l ll l ll ll ll l ll l ll lll ll ll ll lll l ll ll l ll l lll lll lll ll llll ll lll ll lllllll lllllll ll lll lll ll ll ll l ll ll llll l ll l l lll llll lll lll ll lllll l llll ll ll l l ll l ll ll lll l llll llll l ll ll ll l lll lll ll lll l ll ll ll llll l ll l llll ll ll ll l lllll l l lll ll ll ll lll l lllll ll lll ll l lll ll l lll l ll lll ll l lll llll ll l ll l l lll l ll lll l ll l llllll ll lll l ll ll ll ll ll ll lll llll lll l ll ll lll lll lll llll ll lllll lll l ll ll lll l llll lll l lll ll llll lll ll l ll lll lll lll ll ll ll ll lll ll llll l lllll lll ll ll ll l ll ll l ll l lll ll ll lllll ll ll ll lll lll l ll ll lll ll lll lll ll l ll l ll llll l ll ll ll ll l ll l lllll l lll l ll ll ll l l ll ll llll ll llll ll lll l lll ll l lll ll lll l lll ll lll ll ll ll ll lll l ll l lll l ll lll l ll l lll ll ll l lll ll l ll ll lll llll lll ll lll llll ll ll ll ll ll llll l ll ll l lll ll ll l l l ll ll l lllll lll ll llll l lll ll lll ll l ll ll l ll ll l ll ll ll ll ll l ll lll ll l ll ll ll lll l llll lll lll ll lll ll lll l l lll ll ll ll lll lll l ll l l l lll ll ll l l lll l lll lll ll lll ll l ll l llll ll l ll lll ll lll lll ll ll ll ll ll l llll lll ll ll l ll ll l ll l lllll ll lll ll ll lllll llll lll l lll lll l ll lllll ll ll ll l ll ll lll llll ll ll ll lll l ll ll lll ll ll ll ll lll lll ll lll l lll ll ll llll l lll l ll l l llll lll ll lll llll lll lll ll lll ll lllll lll llll l l lll lll l ll lll lllll lll l lll l l l ll lll ll ll l lll llll l l ll ll l llll l ll l lll llll ll l ll l lll lll llll ll ll lll l l lll l ll ll l ll l ll l lll ll l lll l ll l lll l lll lll lll ll llll l l ll ll ll l l ll ll ll lll l ll lll lll ll lll l ll l lll ll ll ll l l l l lll l ll ll lll ll ll lll ll llll l l llll lll l llll ll l ll lllllll llll ll ll l llll ll ll l l l ll l ll l l l llllll lllll ll l ll l lll lll ll lll lll ll lll l lll ll lll lll ll l ll lll ll ll l ll ll ll l lll ll l llll ll l ll lll ll llll l ll l lll l lll lll ll ll llll ll ll ll l lll l ll lllll l lll ll ll lll l llll ll ll ll ll ll lll lll ll ll llll lll l ll ll lllll l lllll l ll lll ll ll lll l l llll ll lll ll ll lll l l lll ll l ll ll ll l l ll ll lll llll ll llll lll lll l lll lll lll ll ll lll l l l ll l l l ll ll llll l l ll llll ll ll ll lll lll l llll ll ll l lll l l ll l llll ll ll llll lll l ll lll ll ll ll ll lll l ll lll lll l lll ll ll ll llll ll l ll l ll ll ll ll ll lll lll ll ll l lll l ll llll lll ll lll llll l lll l ll ll l ll llll lll l ll ll l l llll ll l lllll ll ll lll l lll ll l lll ll l ll ll ll ll ll l l l llll l ll ll lll ll ll ll ll l l ll ll ll l ll ll ll ll ll l lll llll l ll ll ll ll lll lll l llll l llll ll ll lll lllll l l lll ll lllllll lll lll l ll l lll lll l l lll l ll ll ll ll ll ll l ll ll ll lll l ll ll l ll ll llll ll l ll ll llll ll lll ll ll ll ll ll ll llll lll ll ll ll l l ll lll llll l ll lllllll l llll l ll llll ll ll lll ll ll ll lll llll l l ll l ll lll lll ll ll ll l ll ll lllll lll ll lll ll llll l ll ll ll l ll l l lll ll l lll l ll ll l llll ll ll l ll lll l l llll lll ll l llll l lllll l l ll l ll ll l llll lll ll lll l ll l ll ll ll ll ll lll lllll ll ll l llll llll l l lll ll ll l ll llll ll ll ll l l ll ll llll l lll ll lll l l ll ll lll ll lll lll l ll ll l l lll l l lll ll lll l llll l l lll ll l lll lll ll llll ll lll ll lll ll l l ll l lll lll ll l lll l ll lll llll ll ll ll ll lll ll ll ll lllll ll ll ll ll ll ll ll l lll l lll lll lll lll ll lll ll l lll ll lll lll l lll ll lll ll lll l lll l ll l l lll ll ll l ll ll ll l l llll ll l ll lll ll lll ll l ll lll lll l lll l ll ll l ll llll ll l l ll lll lll ll lll ll lll lll llll ll l l ll ll ll ll l ll ll lll l l lll lll l l ll l l llll ll ll lllll lll lll lll ll l lll lll llll l lll ll l lll lll l lll l ll ll lll lll llll ll ll ll l llll lll ll ll lll l ll l lll ll ll ll lll llll l lll l ll lllll lll ll lll ll ll ll ll l alpha=0.01alpha=0.1alpha=0.5alpha=1−0.020.000.02−0.020.000.02−0.020.000.02−0.020.000.02 −0.02 0.00 0.02 S S \tilde{S} S3 Figure 6: A mapping from S to (cid:101) S for four simulated S matrices under the PSD model. The left col-umn shows the simulated structure S for each of four scenarios (a–d) and the right column shows theresulting estimated row basis of S produced from PCA. It can be seen that the scale on which S wasgenerated, all values in (0,1), is lost in the principal components, values in R .28 l lll l llll lllllll ll llll lll ll l lll l llll llll l llll lll ll ll lllll lll ll llllllll llll ll ll ll lll llllll lll ll llll ll ll ll llll l ll l ll ll ll l l lll llllll l lll lll lll lll l llll lll lll lll lll l ll lll lll lll lll ll lll ll l l lll l ll llllllllll l ll l lll l lllll lllll ll lll ll ll ll l llllll l lll ll llll l l lll ll llll llll llll ll ll l llllll ll l ll lll lll lllll ll llll ll lllllll ll ll lll ll ll llll lll l ll lll lll lll l ll ll llll ll lll ll ll ll llll ll ll ll ll ll lll ll l lll lll ll l lll llll lll ll ll lll lll llll lll llllll lll ll ll lll ll lll ll llll ll ll llll lll ll lllll ll lll ll ll ll lllll lll ll lll ll lll ll l ll lll l ll lll llll llll ll lll l lll ll ll ll lll lllll l lll lll l ll ll lllll lll l ll lll ll ll lll ll llll llll l ll ll lll l llll ll llll lll lllll lllll l llll ll llll ll l lll llll l llll l lll ll ll ll lll ll l ll ll l l lll l lll ll ll ll ll l l lll ll lll ll lllll ll lll llll l lll ll lllll lllll lllll llll l ll ll ll l ll ll ll l ll lll lllll ll llll llll lllll llll lll ll lllllllll ll ll lllll ll ll lll l ll ll l ll lllll l lllll ll lll l ll l ll lllll lll llll llll l llll ll ll llll lllll ll lll lllll l l llllllll llll l llll ll ll ll ll lllll lll lll lllll lll ll lll l lll ll l l llllll l llll lllll lllll ll llll llll ll l l lll llll ll l ll lll llll l ll lllll l ll lll lll lll llll ll lll llll ll ll ll ll l llllll ll lll lllll lll lll ll l lll ll ll ll llll l lll l l lll l l llll llll ll lllll lll ll llll l ll lll lll l llll ll l ll lllll lll llll llll ll lllll ll lllll lll ll lll ll lll llllll l ll ll l ll l lll lll lll lll lll ll ll ll ll l llll lllll l ll l ll llllll ll llll ll ll lllll l lll l ll ll llllll ll llllll ll ll l llll lll l llll lll ll lll l ll lll llll llll llllll lllll ll lll l ll lll lll lll lll lll llll ll llll ll lll ll l l llll l lll ll ll l lll llll ll ll ll lllll lll lll l ll ll llll llll llllll ll llll ll ll lllll ll ll l lllll l lll l ll ll ll lllll ll lll lll ll lllll llll ll ll lll ll ll ll lll ll ll l l ll lll lllllllll lll l lll ll lll l ll llll lll llllll lll ll l lll ll llll llll lll ll l lll ll l ll lllll ll ll l ll lll ll llll llllll llll ll ll llllll lll l lll lll llll ll lll llllll lll lll ll lll llll l ll l ll l llllllllll ll ll llll ll l ll l ll llll lllll llll lll lll ll llll l lll l lllll l l lllllll llll ll lll ll llll l lllll ll lll ll lllll lll ll ll llll lll lll ll ll llll ll ll ll lll llll l lll ll llll l lllll ll ll ll ll llll lll lll ll ll l ll lllll lllllllll ll l lllll l ll l lllllll l lll ll ll ll lll ll ll llll l llll ll ll llll ll ll ll l l llll ll lll ll lll l ll l lllll ll ll lll ll lll l ll ll ll l ll l lll l lllll ll ll ll lllll ll lllll lll lllll l ll ll l ll lll ll ll l ll l llll l lll l ll lllll ll lll llll lll ll ll llll l lll ll lll llll ll ll lllll l lll llllll llll l llll ll lll lll ll llll ll ll ll l lllllllllll lllll l ll llll lll ll llll lllll lll lll lll lll l lll ll ll l ll ll l lll l lllll l ll ll ll llll l lll lllllll llllll ll l lll ll ll lllll ll llll ll l ll ll lll lll ll llll ll l lll llllll lll lll l lll ll ll lll ll lllll ll lll ll lllll lllll lllll ll llll ll lll lll lll lllllllll ll lll ll ll ll l ll ll l lllll lll ll ll lll l l l lll lllll ll lll lll ll lll llll ll llll ll l lll lll l ll l l lll ll ll ll ll l llll ll llll l ll lll ll lll l ll l lll l lll l lll llll llll ll llll ll lll llll l lll llllll l llllll ll l ll lll lll ll ll llll lllll lll lll l l ll lll ll l lllll llll ll l lll lll lll ll lll llllllll l lllll lll ll ll l ll llll lll lll lll ll ll ll l lllll lll l lllll l llll l ll lll llllllll lll l ll ll ll ll lllll lllll ll ll llllll lll l llll lll lll lllllll ll ll llll l ll ll l ll lll l lllll llllll lll lll l ll lll lll ll lllll l lll ll lll l l ll ll lllll l ll ll lll llll l ll lll lll ll ll ll llll llll llll ll llllll llll ll lllllll llll ll lll lll lll l lll ll llll lll lll ll ll ll lll llllllll llll ll lllll l ll lllll ll ll lll lllllll ll ll lll l ll lllll llll l ll ll llll ll lll l lll ll lll ll llll lll ll ll lll l ll lllll ll llll l lll l l llll ll l lll llll llll lll llll ll ll llllll ll lll llll l ll lll lll l lll lll ll lll ll l lllll ll ll l lll l ll lll l llll lll ll l lll lllll ll ll l ll ll l l ll l lll llllll ll llllll ll lll ll lll ll llll lll lll ll lll lll l ll l lll ll ll lllll lll ll l l llllll ll lll ll l lll l l llll ll l lll l llll ll lll lll ll l lllllll lllll llll ll lll l llllll l ll ll l lll ll lll llllll ll l ll l lll ll l lll llll lllll llll ll ll ll ll ll ll l llll ll ll ll ll l ll l l ll llll llll llllll ll l l llllll lll ll ll ll lll ll lll lllll ll lllll llllllll llll lll l ll lll lll llll lll l ll lll l ll lll ll lll ll lllll ll lll ll ll ll ll l lll llll lllll ll ll ll lllllllll lll ll l lll lll ll llll lll ll llll ll llll ll lll ll ll ll l l l ll ll llll l ll ll ll lll llll ll lll ll lll ll ll lll lll lll ll l llll ll lll ll l l ll lll ll l ll lll lll lll ll ll ll ll l ll lll lll ll llllll llll llll ll ll ll lll llll l lll lll ll lllll lll ll ll l lll ll llll llll ll ll l l llll lllll l ll ll ll lll ll ll l ll ll lllll llll lllll lllll llll ll l lll l l llll llllll lll ll ll ll llll lll l lll lll lll lll lll ll ll lll ll ll ll l lll lll ll lll l llll l lll lllllll lllll ll lll ll ll ll ll lll ll lll ll lll ll ll ll l ll ll lll l llll lll llllll ll lll l llll lll ll lll ll llllll l ll l lll llll ll llll llllll ll l ll lll l llll ll ll ll lll l llll lll ll ll lll llll ll lllll ll l l ll ll l ll ll ll ll ll llll ll l ll lll lll l llll ll lllll ll lll ll ll llll lll llll lll ll lll llll ll ll lll ll l ll ll ll llll ll ll l lll lll lll ll llll ll l l llll lll lllll l lll lll l ll ll llll lll l lll llllll lll lll ll ll ll ll lllll llll lll lll l ll lllllll l ll l llllll lll ll l ll lll lll lll lll lll ll lll lll llll lllllllll lll lll lll ll lllll lll lllll ll lll ll ll llllll ll ll l lllll l l lllll ll lll l lll llll ll ll lll lll l llll llll ll lllll lll l llll l l lllll ll lll ll ll l lll ll l ll llll ll ll llll ll lll lll llllll llll l lllll ll l lll lll ll ll ll lll lll l ll ll ll lll ll lll ll llllll ll ll ll l ll ll llll ll l lllll lll llllllll ll ll lll ll lll lll l lll llll ll llll ll ll llll ll ll lll llll lll lll lll l lll lll llll lllllll llll llllll lll llll ll llll l lll ll lll lll l ll lll l lll l ll l ll ll l ll lll ll ll ll lll ll llll lll l ll ll l lll ll l llll lll llll llllll lll ll lll ll llllll lllll l l lllll ll lll llll l lll lllll ll llll l ll ll lll llll ll llll l llll ll ll ll lll ll lll lll ll ll lll l lll ll ll l llll ll l ll lll ll ll ll ll lll lll ll ll ll lll llll ll ll ll ll ll l lllll ll ll lll lll ll ll lll l ll ll ll lll ll llll lll ll lll lll ll ll l lllll lll lll ll llll llll l lll lllll lll lll ll lll llllll lll ll ll llll ll lll lll ll ll lll l ll lll ll lll llll lll l llllllllll lll llllll ll lllllll ll l lll ll lllll ll l lll ll ll ll lll l ll l llll ll l ll ll l lllllll ll lll lll l llll l lll llll ll lll l llllll lll lll ll ll ll ll lllll l l ll lll ll llll ll ll lll l lllll lll ll l llll l l lll lll ll l l lll lll lll ll lll ll l ll lllll lll lllllll l l llllll ll ll l l ll l llll l ll lllll ll ll lll l l ll lll l lll l l l ll ll ll l llll lll ll llll ll ll l ll lll lll ll l llll l l llll l ll l llll ll ll l lll ll ll llll l lll ll lll lllllll ll ll llll l llll l llllll lll lll l llll ll l lll l l llll ll ll lll ll ll ll lll llll ll ll llll lllll lll llllll lll l lll l lllll l lll lll ll ll lll lll l ll ll l ll ll lll l ll lll ll lll llll l ll ll ll ll lll ll ll l l ll ll lllll llll llll ll l lll llll ll lll l l lll lll lllll l lll ll lll l ll lll l ll llll llll ll ll l lll lllll ll l l ll ll l ll ll lllll llllll ll ll lll lll l lll lllll lll lll l ll l llll ll ll llll l ll l lll lll ll lll ll ll l ll ll llll l lll ll l ll l lll lll l ll ll ll lll llll ll l llll l lll lll lll ll l ll l ll l llll llll l lll ll ll l lll ll lll ll ll lll ll llllll l l l lllll lll ll lll ll ll l ll llll l ll ll l ll lll ll ll ll ll lllll l lllll lll ll ll lll ll l llll l l ll ll l llll ll lll l ll lllll l lll ll ll l lll ll ll l llll l lll ll l ll ll l lll ll ll ll ll l ll l ll ll l l ll lll ll lll l ll l l l lll ll lll ll l ll lll ll ll lll lllll ll llllll ll l l lllll lll l lll l lll ll ll ll ll lll l ll l llllll lll ll lll ll l ll lll llll l ll lll llll l lll ll ll llll l l ll ll l ll l lllll llll ll ll lll ll ll ll llll l llll ll llll ll ll lll lllll ll ll lll ll lll lll lll l lll l ll ll ll ll l ll ll ll ll lll lll ll lll l ll l ll lll ll ll ll ll l ll lll lll ll ll ll ll ll lll lll l ll llll lll l lll ll lllllllll lll lll l ll lll ll lll lll ll ll ll ll lll llllll ll l ll ll lll ll lll ll ll ll ll l l ll l lll ll lllll lll lll ll llll l llll lll lll lll ll l ll ll ll l ll lll lll l l ll ll ll lll llll ll ll lll l lll ll ll llll ll lllll lll ll ll l llll lll ll ll l ll lll l lll llll l l lll l llll l ll lll llll lll lll ll ll lll l lllll ll ll ll lllll ll lll l ll l lllll llll lll l lll ll lll ll ll l ll l ll l ll ll ll l ll llll lll lll lll ll l ll lll ll l ll ll lll ll ll llll lll lll l ll ll lll llll ll l ll llll lll lll l llll llll ll l l lll ll ll l l llll l ll lll l lll l lll ll ll ll ll l ll ll lll l ll llll ll l ll ll lllll ll l ll l lll l l ll ll ll llll ll ll llllll l l lll llll llll llll lll l ll l ll lll lllll ll l ll ll llll ll ll llll l ll l lll l ll l llll lllll llll ll l llll ll lll ll ll lll ll lll lll l lll ll lll l lll l lllll l ll lllll ll lllll l l lll ll l ll lll ll l lllll l l lll ll ll llll ll ll lll l lllll ll l lll lll l ll l ll lll llllll l llll l lll llll lll ll l l ll ll l lll ll l l lll ll ll ll llll l ll ll l lll ll llll ll ll l lll llll lll lll ll l lll l l ll ll llll ll llllll l ll ll lll lll lll llllll l ll ll ll lll ll lll ll lll lllll lll ll l lll ll llll lll ll lll llll ll l lll ll lll l l l lll ll lll ll ll l ll ll l l ll ll lll llll ll ll ll l ll lll lll lll ll llll lll l ll llll lllll ll ll l llll lll lll ll lll l ll llll l ll lll llll ll l lll ll ll l l lll ll l lll ll l ll ll lll lll ll ll llll llll ll lll ll lll l lll l lll lll l lll ll llll llllll lll l l ll ll ll ll l lll llll lll l ll ll lll lll l l lll llll lll ll lll ll lll l ll ll lll l ll lll lll l lllll l ll ll ll l lll l llll ll ll ll l llll lll lll lll ll ll ll ll ll lll ll l llll l l lll ll llll ll lll l lll l llll lll ll l llll l ll ll l lll ll lll ll l lll lll lllll lll l lll ll lll llll l l lll ll lll l ll ll l l lll ll ll lll ll ll llllll lll lllll l lll ll ll ll lll ll ll lll l ll l ll lll ll l ll l ll ll ll ll llll llll l ll lll ll lll l lll l ll lll l lllll l ll ll ll ll lll l l lll lllll lll lll ll ll ll ll l l lll ll l ll l lll l l ll l lllllll ll llll l ll l l ll lll l ll lll l llll ll ll lllll ll l lll l ll l ll l lll ll lll lllll lll lll ll ll ll ll l ll ll ll l ll ll lll l l l lll lll ll llll lll ll ll ll ll ll l l ll ll ll l l llll l ll ll lll l ll ll llll lll l l ll l l llll ll ll ll lllll l ll ll l ll ll l ll ll ll ll l ll ll l l ll llll ll ll lll l lll lll lll l lll l llll lllllll lll llll llll lll llll llll ll l lll l ll l ll l llll llll ll l lll ll l l ll llll lll llll l ll lll ll ll l ll l lll llll ll llll ll ll llll lll lll lll l ll lll llll ll l l llll ll ll ll llll l l lll lll ll ll lll ll ll lllll l lll l lll ll l ll lll ll l ll l ll l l lll ll l ll l lll l llll ll ll l lllll ll lll l lll ll ll ll ll l ll l l l lll ll l lll lll l lll l llll l ll l l lll l lll lll ll l lll ll llll l llll ll ll ll l ll lll llll ll lll ll ll lll lll l ll ll l ll l ll l llll ll ll lll llll l lllllll ll ll lll ll ll ll ll llll l lll llll ll l l llll l ll l llll llll ll llll l lll lll ll lll l lll llll l ll ll ll llll l l ll ll lll ll ll lllll ll l lll l ll lllll l llll ll lll llll l l lll l l lll l ll llll llll lll ll l ll ll l l l ll lll llll ll llll ll llll l lll ll lll l ll l ll l l l l ll ll l l ll ll lll llll l ll ll l ll lll lll l l ll l ll ll ll l ll llll ll ll ll lll l l lll ll l l lll l lllll ll l ll l llllll ll ll ll lll l lll lllll lll ll llll lll l lll lll ll ll lll ll lll ll l ll l lllll lllllll ll lll l ll ll l lllll l lll l ll ll ll ll ll ll l lll ll ll lll l ll llll ll l llll lll llllll ll llllll l llll l ll ll ll lllll l l l ll ll ll l ll l ll l ll ll lll l ll l ll l l ll lll lll ll lll ll lll ll lll l ll l lll ll l ll lll ll ll ll ll ll l ll l lll llll ll l l l ll ll ll lllllll ll l lll l llll l ll lll l ll ll ll ll l lll llll ll llll l ll l l ll ll ll ll ll ll ll lll llll ll ll l lll ll ll ll l ll lll ll llll ll l lll l l ll lll lll llll ll ll ll l lll ll l ll ll lllll ll ll ll ll lll l ll ll l ll l ll ll l lll ll ll llll llll l ll l ll l ll lll lll ll lll lll ll ll l ll l lll lll ll llll l ll l lll l ll lll ll llll l ll l lll ll ll lll l llllllll lll l ll llll ll ll ll llll ll l ll lll ll lll l ll ll ll l lllllll ll ll ll l lll lll lll ll lll lll ll lll ll lll ll llll ll llll ll l lll lll lll llll l l ll lllll l lllll llll lll llll ll ll l lllll ll ll ll ll ll lll l lll lll lllll l ll llll ll lllll l l llll lll lll l l llll ll lll l lll lllll l ll lll ll ll llll l lll ll lll l ll llllll ll ll l lll llll ll l lllll ll ll ll l lll l l ll lll ll l l ll ll lll lll ll llll l llll ll l lll l ll ll l lll lll l lll llll ll lll ll l ll lll l lll lll l lll ll ll ll llll lll llll lll lll lll llll ll ll lll ll ll l ll l ll l ll llll l lll llll l ll ll l ll l ll ll l ll l lll l ll ll ll l ll ll ll l ll llll ll l lll l ll ll l lllll lll ll l llll l lll llllll lll ll l llll ll ll l ll l lll ll ll ll ll l lll llll ll ll l ll l lllllll ll lll ll l lll ll l lll ll l ll lll llll ll ll lll l ll lll ll l ll lll llll ll lll l ll ll l llll ll lll l lll l lll ll lll l ll ll ll l lll l l llll ll ll l ll ll llll ll l l lll lll llll ll ll llllll l llll ll ll ll ll ll ll ll llll lll lll ll lll l lll l ll lll ll lll ll l l lll ll lll l l lll ll ll llll l lll ll ll lll lll ll ll lll ll l ll lll ll lll ll ll ll lll ll l lll ll lll ll ll lll ll lllll ll l ll ll lll ll l l ll l lllll ll l lllll llll lll ll l l ll lllll lll ll l lll ll ll ll l ll ll ll l lll ll lll l lllll ll ll llll lll ll l llll ll lll l llll ll ll ll ll l lll lll l ll ll lllll llll ll ll lll lllll l lll ll ll l lll ll ll lll lll llll l l l ll ll ll ll ll l lll ll lll l lll lll l ll ll ll ll l lll ll ll l l lll lll l llllll ll ll l l ll ll ll l ll l llll l l lll l ll lllll ll l llll ll l lll lll ll ll ll ll l l l llll l lll ll l lll l lll l lll llll lll llll llll ll lll ll ll l lll ll l lll ll ll ll l lll l lll ll l l lll ll l ll ll ll ll ll lll ll l llll ll l l ll ll l ll ll l llll lll lll l ll ll ll ll lll ll ll ll ll ll l l l lll ll lll llll lll ll l llll l l llll ll lll ll l lllllll l ll lll l l ll l lll l lll ll ll l lll ll lll l ll l ll l l lll l l ll ll l ll l llll ll l lll ll l ll llll ll l lll lll lll lll ll lllll lll lll llll l ll l lll ll l ll ll l ll ll l ll l ll l ll ll l ll lll l l ll ll l llll lll l llllll lll ll l lll ll l l lll l l l l lll lll l lllll ll l lll ll lll l lll ll ll ll l ll ll llll l l l l ll ll l ll ll l ll llllll l ll lll l ll ll l lll ll lll llll l lll l ll llll ll l ll ll ll l lll l ll ll l l lll ll ll ll lll l ll llll l lll l ll lll ll llll l ll l l ll ll ll ll ll ll ll lll ll l l ll ll lll lll l ll lll l ll lll ll ll l ll l lll ll l ll l llllll lll ll l ll ll ll ll l lll ll ll lll ll ll ll ll ll l lll lll l lll ll ll ll lll llllll lll ll lll ll ll l lll l lll l lll llll ll ll ll lllll ll ll ll lll lll lll l lll lll ll ll ll ll llll llll l lll l lll llll ll l llll lll ll ll l lll l l l lll lll lll ll l lll ll l ll ll lll lll l l lll l l ll ll l l llll ll llll ll ll lll l lll lll ll lll ll l lll lll ll l lll ll l ll lll ll llll llllll lllll lll lll l ll llll lll l lll llll ll ll llll lll ll l ll ll ll lll lll l ll l ll ll ll l ll lll lll l lll ll ll lllll ll ll l ll l lll l ll l lll llll lll l l lll lll ll l ll ll ll l l llll ll lll lll llll lll l lll llllll l l llll ll l ll ll ll ll ll l lll lll ll lll l l l llll ll lll ll ll l ll ll lll lll l l ll l lll l ll l l lll lll ll ll ll lll l ll ll lll ll llll lll ll ll lllll lll ll ll ll lll ll l ll ll ll ll l ll lll l lll lll ll lll l ll l lllll lll l ll lll lll ll ll ll l ll lll l llll ll l ll lll l ll l l ll l lll l ll ll ll ll ll ll ll ll ll l ll ll lll l ll lll lll lll ll lll lll lll lll ll ll ll ll lll l llll l lll lll lll lll l l ll ll l ll l ll ll llll l l lll l lll ll ll ll ll l lll lll l llll llll lll ll ll ll l ll ll ll llll l ll l llll l ll l l lll ll l lll l ll l lll lll l ll llll lll l ll ll lll lll ll l ll llll l lllll ll ll llll ll l lll lll lll ll llll llll ll ll l lll lllllll ll l ll lll l l ll lll ll ll ll llll ll l lll ll l llll ll l ll l l llll llll ll l ll ll l lllll lll l l lll lll l ll l ll l lll llll ll ll ll lll llll ll ll l ll lll llll l ll l l ll l ll l ll l ll ll lll lll l lll llllll ll l ll ll lll ll l l lll ll ll ll lll l ll llll lll lll ll ll ll l ll lll ll lll ll lll ll ll l l ll ll ll ll lll ll ll llll l ll l ll l lll ll ll ll l lll ll lll l llll lll ll lll l lllll l lll lll l lllll ll l ll lll llll l ll ll l lll lll ll l lll lllll l ll llll ll l llll ll l lllll lll ll lll ll ll l lllllll ll ll l l ll l l lll ll l ll l ll l lllll l ll llll ll lll lll ll llll l ll ll ll ll lll ll ll l ll ll ll l ll l ll llll ll ll ll l ll l ll ll ll l ll ll l lll ll ll lllll l lll l ll ll ll l l lll l ll l llll l ll ll ll lll lll l ll lllll ll lll lll l llll ll l lll ll l lll ll l ll l lll l l ll ll ll l lll ll l lll lll lllll l ll ll llll lll ll ll l ll ll ll l ll lll lll ll l ll lll l ll lll lllll ll lllll l l lllll lllll l ll llll l l lll ll ll lll l lll ll ll lll l ll ll l llll ll llll ll l l ll lll lllll l lll ll lll l ll ll lll l ll l ll ll ll l ll l llll lllll ll ll lll lll l ll lllll ll ll lll l ll l llll ll l llllll ll ll llllll lll llll ll ll ll lll ll ll l ll l l l ll l ll ll ll ll lll ll l l llll l llll ll l ll lll ll ll ll lll lll lllll lll ll l llll l ll ll l ll l llll l lll lll ll lll llll l llll ll ll ll lll l lll ll ll ll lll lll ll ll l llll ll lllll l llll ll l ll lllll l lll l ll l l llll llll l ll llll ll l l lll ll ll ll ll l lll l l ll l l ll l lll l l lll l lll ll ll lll l ll l ll ll lll lll ll l ll l ll l lll ll ll lll l ll lll ll l lll l lll ll ll ll l l ll llll l ll ll ll l ll ll l ll ll lll ll lll l lll l ll llllll ll ll ll l l ll lll ll l llll ll lll l ll l lll ll l lll ll lllll l ll l ll ll lll lll ll l ll lll ll l lll ll ll ll llll ll lll l l l ll lll ll l lll l ll lll lll ll ll l ll lll l ll ll ll l ll l ll lll l ll l ll ll lll lll ll ll l lll l l llll l l lll l ll ll lll ll l lll l l lll ll lll ll l l l llll ll l ll ll ll lll ll lll ll lll ll ll llll l ll ll ll ll l llll ll ll ll ll ll llllll ll ll ll ll lll ll ll llll lll l lll ll l l lll l ll ll lll l ll ll ll l ll ll ll ll l ll l ll llll ll lll llll lll l ll ll ll l lll l l ll l l ll lll ll l lll lll lll lll ll l l lll ll ll l ll ll ll l lll l lll l ll ll l lll ll lll ll lll l ll lll ll lll lll lll llll l lll ll l ll ll l llll l lll ll l ll l lllll l ll ll ll ll ll l ll lll lll ll lll lll lll ll l ll l ll lll ll ll l l ll l lllll l lll l ll lll ll ll ll lll ll l ll llll l lllll l ll ll l l lll lll ll lll l lll ll lll lll ll ll lll l l ll lll ll lll lll l ll ll llll ll lll ll ll lll ll ll ll l ll ll ll ll ll lllll ll ll llllll ll ll llll l l lll l ll ll l llllll ll ll l ll l l lll llll l ll ll l llll ll l ll l lll l lll llll llll ll ll llll l l l ll l ll ll lll ll ll l l ll ll ll l ll ll ll lll l llll ll l ll ll ll ll l lll lll l lll ll lll l lll ll l l ll ll ll lll l ll l ll ll lll l ll ll ll l lll lll l ll ll l l lll ll l lll l ll llll l l lll llll ll lll ll lll ll l ll llll l ll ll ll lll lll lll lll l ll ll ll l lllll ll ll l ll l ll ll llll ll ll lll ll ll lll l l lll lll ll ll ll lll l lll l ll ll ll lll l lll ll ll ll l ll ll lll l lll lll ll l llll llll ll l ll l lll ll l ll l lll l lll ll ll l ll ll llll l l ll ll ll l lll ll l l ll llll ll ll llll ll ll ll ll l ll lll lll lll lll l lll l ll lll lll l l ll l lll l ll llll l lll ll ll l ll l l ll ll l lll ll lll lll l lll ll ll ll llll lll ll ll lllll l llll l ll lll lll llll l llll ll ll ll ll lll lll lllll ll l l ll lll l lllll l llll l lll l ll lll lll ll ll lll ll l lll ll ll ll l lll ll l llll ll l lll l l ll ll ll lll l lll ll l lll l ll l llll ll l lll lll l lll ll ll l lllll ll ll ll lll l lll ll ll lll lllll ll lll lll l ll ll l ll ll ll l lll lll l llll lll l llll ll lll ll ll llll ll ll lll l l ll ll ll ll lllll lll lll ll ll l lll l ll lllll llll lll ll ll l ll l l ll l lll ll l ll lll l lll l ll lll ll lll lll l lll llll llll lll l ll ll lll ll l ll l lll lll l llll l ll ll l ll l ll lll lll lll llll l l ll ll llll lll l lll l ll ll lll l ll ll lllll l l llll l ll l l llll llllll ll ll ll llll l ll lll ll ll ll l ll l ll llll l ll l ll ll ll lll l ll ll ll lll l ll lllll l l ll l lll l ll ll ll ll l ll l l llll l ll ll llll l l lll lll l lll l l llll l llll ll ll lll l lll lll l llll ll ll l lll lll lll llll ll l l llll lll ll l l ll l ll ll l ll ll l ll ll ll l ll ll ll ll lll ll ll ll l l ll lllll ll l llll l l lll l ll ll l ll l ll ll l l lll ll lll l ll lll ll ll ll ll ll lll ll llll l l llll lll ll ll llll ll ll l ll l ll ll l ll l l ll llll ll l ll lll ll l ll ll ll lll ll l ll ll lll ll ll l l ll l l ll ll l ll lll l ll ll ll llll l lll ll lll ll l ll ll ll l lll l lll l l llll l l ll llll l ll llll lll l llll ll lll l ll ll ll ll llll l lll lll l ll ll ll l l llll ll l ll ll llll ll llll ll ll lll ll l ll l l ll l ll l ll ll lll lll lll ll ll l ll l ll ll lll ll ll l l lll lll lll ll ll l l lll ll ll l ll lll lll ll llll l ll ll ll llll lll l ll ll l ll l ll lll ll ll l ll l l lll ll lllll l lll l llll ll l lll l ll l ll l llll l l ll lllll l ll l ll ll l llll ll l l ll lll ll ll ll llll ll ll l l lll l l ll lllll ll l ll lll l l ll ll lll l l l lll ll l lll ll lll l lll ll ll lll lll lll ll ll ll l ll lll lll lll lll llll ll ll lll lll llll l ll ll ll llll ll lll ll ll ll l llll l lll l lll ll lll ll l ll l lll l ll l l ll llll ll ll l ll ll ll lll l ll l lll lll ll ll ll l l lll ll l lll lll ll ll l lll lll lllll l lll ll ll ll l ll ll ll ll llll ll l lll ll lllll ll lllll l ll l ll l lll ll ll l ll lll l ll llll lll l l l lll ll lllllll ll ll l ll ll ll ll lll l ll l ll l ll ll l lllll l l lll ll l lll lll l ll lllll llll ll ll ll ll ll ll ll lll l ll ll ll l ll l l lll l ll l lll ll lll l lll l lll ll ll lll l l l ll l l lll ll lll l ll l lll l ll ll l ll ll l l l lll l llll l llll ll lll ll l llllll ll l llll ll lll ll l l llll ll l l ll l l ll ll ll lll l llll l lll llll lll l l ll lll lll l lll ll ll ll lll ll l ll l l ll ll llll ll ll l lll ll ll ll ll ll l lll ll lll l ll ll lll l ll lll lll ll llll l ll l ll l l ll l ll ll l l l ll ll l ll lll lll ll l ll l lllll ll llll ll ll ll l lllll ll ll l lll ll ll lllll ll ll lll lll ll ll ll l llll l llll lll lll ll ll l lll l l lll ll lllll ll l ll l ll lll ll ll ll l l lll lll llll l ll l lll ll l ll llll lll lll l ll lll lll l ll ll l lll ll lllll l ll lllll l llll l ll ll l ll lll ll l ll ll ll l lll ll lll l lll l lll l ll lll l ll ll l lll lll lllll l l ll ll ll lllll l ll lll ll l ll ll lll l l lll l ll l l lllll ll l ll ll ll ll ll ll lll ll lll l l llll l ll lll ll l ll ll lll l l ll ll l lll ll l lll ll l l lll l lll l l ll l lll l ll lll ll l ll l ll lll ll l llll l lll l lll l ll l ll l ll ll l ll ll ll ll l l ll llll l l l lll l ll l l l lll l l ll lll ll l llllll l ll l l ll l ll lll l lllll ll ll llll lll ll ll lll llll l ll lll ll lll ll ll lll l l ll llll llll l lll l ll ll llll l lllll lll ll l lll l ll ll ll l l lll l ll l lll lll lll ll lll l l lll l lll ll ll l ll ll ll lll ll l lll ll ll lll l ll lll l ll ll ll lll ll l ll lll l lll ll ll l ll lll ll ll ll ll l ll ll ll l llll l llllll l ll ll lll l ll ll ll l ll lll lllll l lll ll l ll l lll lll ll ll ll ll ll lll l ll l ll llll ll l lll l ll l ll ll llll l l ll ll l l ll ll l ll l ll l ll l ll ll ll ll lll l l ll llll lll llll ll l l ll l lll ll l l l lll l ll lll ll l ll llll ll ll llll lll lll l l lll lll l l ll llll l lll ll llll ll l lll ll lll lll l lll ll ll l l lll ll l ll ll ll l lllll ll l lllll lll l lll ll lllll ll ll l l ll ll lll ll lll l l ll ll l ll ll ll l ll ll lll l ll lllll l ll l ll l lll ll ll ll ll lll ll l lll l llll ll ll ll l l ll l lll ll lll l ll llll ll l ll lll ll ll l ll l ll lll ll lll ll l l lll ll l ll llll l ll llll ll l lll l ll ll ll l ll l l lll ll l ll ll llll lll l ll ll ll ll l l l lll lll l ll l ll ll l ll ll l ll ll lll ll l lll l ll llll ll lll lll ll l lll ll llll ll ll l lll ll ll llll ll l lll ll l l ll ll ll llll ll ll ll ll ll l ll lll l l llll ll lll ll l ll lll ll ll l ll ll l ll llll l ll l ll lll ll lllll lll l ll llll lll l llll l ll lll ll ll l ll l l l ll lll l l ll l l ll ll lll ll ll lll llll l l ll l lll ll ll l lll l l l ll l ll l lll l ll lll ll ll ll l ll l lll llll l ll ll ll llll ll lll l ll l l ll lll llll l ll ll lll lll llll lll lll ll ll l lllll ll ll l ll l lll l lll ll ll l llll l llll l l ll ll l lll l ll ll ll lll ll ll l ll lll ll ll lll l llll l l ll ll l l lll ll ll ll l ll ll l ll l ll ll l ll l l lll llll ll l ll ll l l llll ll l l llll l l l l ll ll ll ll lll l ll ll l ll ll l lll l ll lllll llll ll ll l ll ll llll ll lll ll ll lll lllll ll l ll ll ll ll ll llll ll l l ll l ll ll ll lll l l l ll lll l ll lll l ll l lllll ll ll l ll lll l l lll ll ll lll lll ll ll lll l ll l lll ll ll ll ll lll l lll lll l ll lllll llll l llll l l ll lll lll l ll ll ll l ll ll llll lllll ll l ll llll ll l lll ll l l l ll l lll l l ll ll l ll ll ll ll lll lll l ll ll l l ll ll lll ll ll lll l l lll lll ll ll l ll l l l ll ll llll lll l l ll l ll ll l lll ll ll ll ll ll ll ll ll llll l ll l l lllll l lll ll ll ll lll l lll llll ll ll ll l ll llll l ll ll ll ll ll llllll ll lll ll l lll lll l lll ll ll ll l ll l ll l ll l ll ll l ll lll ll lll ll ll lll ll lll lll l l ll ll l lll lll ll ll l lll l lll ll l l lll lll l llll ll l ll ll llllll l l lll lll llll ll l llll lll lll ll l ll lll lll l ll l ll ll l lll lll l ll ll ll ll ll l l ll l ll ll lll ll l lll lll ll l lll ll lll llll l l lll ll l ll l l lll l lll l llll lll ll ll l lll lll l ll ll lll lll ll l lll l ll ll l ll ll ll ll lll l ll l ll l l lll lll llll l lll lllll ll ll l ll lllllll llll ll ll l ll ll ll lll l ll lll llll l l ll l llll l lll l lll lll l ll ll ll lll lllllll ll l l ll ll ll ll lllll l lll l l ll l ll lll ll l lll l lll ll l llll lll lll ll l ll lll l llll l l ll ll ll ll lll l lllll lll llll ll ll l lll l l lll llll l lll ll ll l ll l ll lll l lll lll ll ll l lll ll l l ll l lll lll l ll ll l ll l ll lll l lll l ll ll l lll l l ll ll l ll ll l lll l ll ll l ll ll lll ll lll l l llll l ll ll l ll llll ll l l ll l ll l l l l llll lll l l l ll l lll lll llll lll llll l llll l llll l ll ll l lll ll ll lll l l l ll lll ll ll ll lll lllll lll l ll ll ll ll ll lll l l llll l llll ll lll ll lll ll ll l lll l l ll ll ll l lll ll ll llll l l ll ll l ll lll lll ll l ll ll lll l ll ll ll ll l llll l llll l ll ll ll ll ll l ll ll l lll l ll l ll lllll lll ll l ll lll lll l ll ll ll l ll lll ll l lll l lllll ll lll ll l l ll lll ll lll ll ll l ll lll ll lll ll ll ll ll lll lll lll lll ll ll ll ll llll llll l lll lll lll ll lll lll llll ll lll ll llll l ll llll lll l l lll l llll lll l ll ll ll lll lll ll lll lll ll lll ll ll l lll ll l ll ll ll l l lll ll ll ll ll ll ll llll lll ll l ll lll llllll lll l l ll ll ll l ll ll lll ll ll l ll ll ll l llll ll ll l l ll ll lll ll l l l ll ll l ll lll ll ll l ll ll lll l ll lll ll ll l ll ll l ll l llll l ll lll l l ll l lll llll lllll lll ll ll l l ll lll ll lll lll ll lll ll ll ll ll lll l ll ll l lll l llll ll l ll ll ll ll ll ll l ll ll ll ll ll ll l ll lll ll l l lll l lll llll l llll llll ll ll ll l lll ll llll l ll llll l lllll lll ll ll l lll lll ll l lll lll ll ll l lll ll ll l l lll ll l ll l lll ll lll l llll ll l ll l lll l l ll l ll lll ll ll lll l l ll l ll ll l lll l ll ll lll ll ll ll lll l l ll ll llll ll ll l ll ll l lll ll l llllll ll l ll ll ll ll llll l lll ll ll ll ll l ll llll ll ll ll ll ll ll l lll ll ll lll l lll ll ll ll ll ll ll ll lll ll l l ll l ll lll l l lll lllll llll llll l ll ll l l llll l l ll ll l lll ll ll llll l ll ll l ll lll lll ll lll l ll ll lll ll l ll l l llll l llll lll ll ll llll lll ll ll ll l ll ll lllll ll llllll l l lll l ll ll ll ll lll ll ll l lll lll l llll llll l l lll ll ll lll l ll l ll lllll l l llll ll ll lll l lll l ll ll l l ll ll l ll ll ll ll llll ll lll l lll ll lll ll llll lll l lll lll l ll l ll lll ll ll lll l ll ll ll lll lll l l ll ll lllll l llll ll lll ll l ll llll ll ll lll l llll ll ll l llll l l l ll ll l lll l ll ll ll l ll ll ll ll lll l l ll ll l lll ll l lll ll l ll ll lll l lll l l lll lll ll l llll ll a=0.1a=0.25a=0.5a=10.000.250.500.751.000.000.250.500.751.000.000.250.500.751.000.000.250.500.751.00 0.00 0.25 0.50 0.75 1.00 S S S l lll ll l ll llllll lllllll ll l ll ll l llll llll l llllll lll ll llll llllll ll ll ll llllll ll lll l lllll ll ll ll ll lll lll llll l ll ll lllllll l lll ll ll ll lll l ll l ll lll lll l ll lll l l lll l lll ll lll l lll ll lll l lll ll lll ll l lllll ll l ll llll l ll l ll ll llll l lll lll ll ll lll lll llll ll lll l llllll ll l ll ll ll lll ll lll llll lll ll llll llll ll lll lll ll l lll ll l llll ll ll llllllll l ll ll lll ll ll ll l lll lll l ll ll ll llll ll ll lll l ll ll llll ll l ll llll l l ll ll ll llll l ll ll lll ll lll ll l l ll l ll l ll ll lll l lll l ll ll ll l ll ll lll ll ll lll ll lll lll l ll ll ll ll lll ll lll llll ll l ll ll llll l l lll l ll ll lll ll lll lll ll l lll ll lll llllll l llll llll ll ll ll lll ll ll llll l ll l ll l lll ll l lll l ll l ll lll ll ll l ll lll lllll lll ll ll lll ll lll lll l ll lll ll lll l llll ll lll ll ll ll lll llll llll ll lll l llll l l ll ll lll ll ll lll ll lll ll llll ll ll lll ll lll ll ll l l llll ll ll lllll l ll lll l lll l lllll ll lll lll lll lll ll llll ll lll l lll llll lll llll llll l ll lllll l llll l l l lll lllll lll l lll l l lll lll ll lllll l llllll l lll ll l ll ll l lll l l lll lll ll ll lll ll llll ll ll ll lll ll ll lllll ll lllll l ll l lllll lll ll lllll lll lll l l l ll lll lll lll ll l ll ll ll lll ll lllll ll l lll l lll llllll l l ll l l lll l llll l lll l ll llll lll l ll lll l lll ll lll l ll ll llllll lll l l ll l llllll ll ll ll ll lllllll ll ll lll l llll l ll lll l ll lll llll ll ll llll llll ll llll lll llll ll lll l l l ll ll lllll lll llllllll lllll lll ll lll ll l lll ll ll lll ll ll lllll ll l ll ll lll lll lll l ll lll ll ll ll ll lll l ll l ll l lll l lllll ll lll lllll l l llllll ll l l llll ll l lllll ll ll l lll l lll lll lll ll ll llll l lll ll l ll l lll llll l llllll ll l lll lllll l l ll lll ll lll lll l lll lll lll lll ll llll llllll lll lllll l lll ll l ll lll lll lll l ll ll ll ll ll ll ll ll ll ll l ll ll l ll llll ll llll ll llllll l lll ll ll ll ll llll l ll ll ll lll ll ll l l llll l lll l llll lll ll l ll ll l ll lll l llll lll ll ll ll lll llll ll lll llll l ll llllll l lll lll lll ll lll l l ll l llll ll ll l ll l lll ll ll l l ll ll lllll l ll l ll lll ll l ll ll ll ll l ll l llll lll ll ll lllll ll ll lll lll lll l ll l ll l ll ll ll l ll lll ll l lll ll l ll ll l ll ll ll ll llllll l l ll lll ll ll ll ll ll lllll l ll ll lll l lll l llll lll ll lll l ll ll lll ll l lll ll l lll ll ll ll ll l l l ll llll l lllll lll ll ll ll lllll lllll l l ll lll ll lll l lll llll ll ll llll llll lll lll llll ll lll ll ll ll ll llll l ll lllll lll llll lll l ll lll lll llll l l lll llll l ll ll lll ll ll ll lllll ll ll l lllll lll ll ll l lllll ll ll ll llll ll l llll lll lllll l ll l ll ll ll l lll lll ll ll lll ll l lllll ll ll ll ll ll lllll llll llll llll ll ll ll ll lll l ll lllll lll l ll ll l lll ll llllllll ll lllll ll llll l llll lllll l llll ll ll ll l l llllll ll ll lll ll lllll ll ll l ll lll lll l ll llll ll llllllll l lll llllll ll l lll lll ll lllll ll lll ll llll ll l lll l ll ll l ll lllll l lllll lll l llll lll l ll l lll ll ll llll l l lll ll ll ll llll lll ll lll l ll lll llll llllll lll l ll lll l ll lll lll llll llll ll ll lll ll lll l lll ll l lll l lllll ll llllllll lll l ll ll lll ll l l lll ll llll ll lll ll ll ll lll lll ll lll ll l llll lll lll ll ll l lll ll lll l l lll lll l l ll ll lll l ll ll ll ll l ll l llll l ll ll lll llll l lllll lllll ll l ll lll lll l ll llll lllll lll l lll lll ll ll ll ll lll llll lll llll l llll ll l ll l ll llllll ll ll ll l lll ll lll lll l ll ll ll lll lll l l lll ll ll ll ll lll ll ll l ll l lll llll llll l l llll ll llll l llll lll ll llll ll ll ll llll l llll ll llll l lll l ll lll llll l lll llll lll ll ll ll ll ll l ll l llll ll lllll l ll l lll lllll lll ll lll llll ll lll l lllll ll l ll l lll l llll lll lll llll l ll lll ll l lllll l llll l ll ll l llll l l lll ll lll ll lll l l l llll llllll ll ll ll lll l ll l lllll llll llll ll llll ll lll l llll ll llll llllll ll l lll ll ll ll l lll ll ll ll lll lll lll llll ll lllll l lll l llll lll ll lll ll l l lll ll ll ll llll llll l lll l lll ll lll l lll lll ll l ll ll l llll ll ll l ll ll lll ll l ll ll l ll l ll llll l llll l lll llll ll l ll ll ll lllll l ll l llll l ll ll ll l ll ll lll lll ll l lll lll l lllll l l llll l lll ll ll ll llll l l lll lllll ll l ll lll lllll ll lll ll l lll ll l l llll l l ll ll ll ll ll lllll ll l llllll ll l ll l ll ll lllll ll lll lllll l ll l llll ll lll ll lll ll ll l ll l lll llll llll ll l ll llll ll lll ll l ll ll lll l ll llll llllll lll l lll lll llll ll llll ll lll ll ll l l ll lll lll l ll l lll lll l ll ll ll lll ll ll llll l llllll ll llll ll ll l ll llll lll l ll llll l lll ll lllll lll ll lllll ll ll lll ll l ll l llll lll ll ll l lll l lll ll lll l ll ll l ll l llll l ll llll l ll llll l lllll ll l l lll llll l ll ll ll ll l lll lll l lll ll ll ll ll lll ll l l lllll ll ll lllll l l lll l lllll llll ll lll l llll lll ll ll l ll llllllll lll ll llll l ll llll lllll lllllll l ll ll lll ll l lll l lll ll l lll lll ll lll lll ll lll l ll lll ll ll ll lll ll l lllll l l lll l ll ll ll l ll lll ll ll lll llllll ll lll ll ll lll llll l lll l ll ll lll ll lll ll lll ll l ll l l llll ll ll lll llll lll ll ll l llll llllll llll llllll l llll ll lll l l llll llll lll ll ll ll l lll l ll ll ll lll l ll llllll ll l lllll lll lll l ll l ll ll ll ll ll llll ll l ll l lll llllllll ll ll ll ll l llll lll ll ll l ll ll llll l ll ll llll l ll l lll l ll ll l ll ll ll l l lll ll l llll ll ll l llll lll l ll l l ll ll ll ll ll l lll l ll ll l lll l lll ll ll ll l ll lll lll ll lllll ll l l ll lll lll l ll ll ll l ll l l llll ll lll llll ll ll lll ll ll ll ll ll l l lll ll l ll l lllll lll ll llll l llll l llll lllll ll ll l ll ll ll ll ll ll lllll l ll l lllllll llll llll l lll lllll l lll l llll ll ll ll ll lll lll l lll l lllllll ll lll ll ll l lll l lll lll lll l ll ll l llll l lll lll ll ll lll ll l ll lll ll lll lll lll l ll lll ll ll lll lll ll llll ll llllll l ll lll l llllll l l lll ll ll lll ll lll ll lll ll llll l llll llll ll ll lll l ll lll lll l l ll l llll lll l lll ll l l l lll llll l ll ll ll l ll ll lll llll l l ll lll ll ll ll ll lll l l l ll l ll l ll ll l llll ll ll l llll lll l ll lll ll ll ll lll lll lll ll lll ll ll llll ll lllll ll ll ll l ll lll lll llll l lll l lll lll l lll llll lll lll ll l lll l ll llll ll l lllll llll ll ll lll ll l l ll llll ll ll lll ll llll llll l l lllll l ll l llllllll llll lll l ll l l ll ll l lll l ll ll ll lll lll ll l lllll ll l ll lll l ll lllll l lll l lll lll lll ll ll l ll ll llll lllll l lll l ll lll l ll ll l lllll ll llll llll lll l ll ll lll llll lll lll ll lll llll l ll l lllllll l ll ll ll ll lll l lll ll llll ll ll lll llll llll lll l ll l llll ll llll ll ll ll l lll ll ll llll llll lll ll ll l llll lll l lll ll lll ll lllllll l l ll llll ll lll lll lll llll ll lll ll ll l lll l lll l ll l llll llllll llll lll llll lll l ll lllll lll l l ll ll ll ll ll l llllll lll ll ll ll ll ll lllll lllll ll l lll lll l lll lll l ll l ll lll llll l l lll lll ll l ll l ll ll lll l ll ll lll ll l ll lll lllll ll ll ll lll l l ll l ll l lll l ll l llll l lll l ll ll l l llll l llll ll lll ll ll ll ll l ll llllll llll ll l ll l l ll ll ll l lllll l ll llll l lll l lll lll llll lll ll l ll l lll llll lll llll l ll l ll lll lll lll lll lll lll lllll l ll ll l l lllll ll l llll lll l ll ll ll llll l lll l ll lll ll ll l l llll lll lll ll l ll l ll ll l llll l lll lll ll ll ll lll l ll ll ll l llll lll llll ll l l ll ll llll ll ll ll lll ll l ll ll llll ll ll l ll ll l ll lll llllll ll l ll ll ll ll ll ll l ll ll lll llll l llll ll ll l lll ll lll lll l lll ll lll lll ll ll llll lll ll lll llll llll lll ll ll l ll l l lll ll ll ll lll ll lll lll lll l l ll lll l ll ll ll ll l lll llll l ll l ll l lll l llll ll ll lll lll lll lll ll llll lll lllll l lll lll ll lll ll ll lllll ll lll l l lll llllll l lll lll ll l ll ll ll lll ll ll ll llll l ll ll ll l llll ll l llll ll ll l ll ll lll ll lll llll lll ll ll lll l lll ll lll l ll llll l ll ll ll lll ll ll llllll lll l l llll l lll l l llll lllll ll ll lll lll ll ll l ll lll ll l l llll l l ll l lll ll l ll ll lll lll llllll lllll lll ll ll l l ll l l lll ll l l lll llll ll l ll ll llll lll l l llll l lllll lll ll lll ll ll ll ll l ll l l lll lll l l llllll ll l lll ll l ll ll ll ll ll l lll l lllll ll lll lll ll ll ll ll l ll ll l ll ll l ll lll llll lll lll lll ll l lll l lll lll l llll lll ll l l ll ll llll ll l lll llll l ll ll l ll ll l ll lll ll llll llll ll llll lllll ll ll ll lll lllll ll l ll lll llll l ll l lll llll lll ll ll lll ll l llll l lll l l ll ll l ll ll ll ll l ll ll lll l lll llll l ll llll ll ll l ll llll l l lll ll l ll lll ll lll l lll ll ll ll lll l l lll l ll ll ll lll ll ll ll llll lll llll l ll l ll ll ll ll ll llll lll lll ll l ll ll lll l ll l l llllll l ll l ll llllll l l ll l ll ll lll ll llll ll ll l ll l ll ll lllll l lll lll l ll ll ll l lll lllll ll l l ll ll lll lll l l lll l lll ll l lll lll ll ll ll l ll l ll l ll lllll lll ll l llll l ll l ll ll l llll llll ll l ll llll l llll l ll ll ll ll l l ll ll l ll ll ll lll lll ll ll l ll l lll ll ll llll lllll ll ll ll lll ll lll l lll llll lll lll ll lll ll ll l lll ll l l lll llll ll l ll l ll l l l llll l lll lll ll ll lll ll ll ll lll l ll lllll ll lll ll ll lll lll l lll ll l l lll lll l lll ll ll lll ll l lll ll lll lll l lll lll l llll l l lllll lllll lll lll lll l ll ll ll llll l l lll ll llll ll ll l ll l lll ll lllll l lll ll lll l ll ll ll ll lll lll l ll ll l ll llll l ll lll ll ll ll ll lll lll lll l l lll ll l llll l l lll llll lll ll l l lll ll ll ll lll l ll ll llll ll lll l l ll lll l lll llll ll l llll ll l l l llll ll l lllll l ll lll l llll l l l lll l l ll ll ll ll l lll ll lll lll l ll lllll l l lll ll l l lllll ll l ll lll ll lll ll llll ll lll l ll llll l ll l l ll ll l lll lll l ll ll l ll ll ll lllll l lll l ll l ll l ll l ll l ll ll ll lll l ll llll llll l lll l llll lll ll l ll l lll l l ll ll ll l ll lll ll llll ll ll l ll lll ll lll ll ll l ll ll l ll ll ll lll lll ll l ll lll l ll ll l ll ll lll lll ll ll l lll ll l ll ll l ll ll lll lll l lll ll lll ll l l ll l ll ll ll ll llllllll l l ll l ll lll ll l ll lll lllll lll ll l l ll lll lll ll llll l ll ll ll ll lll l llll ll l ll ll l l l llll l ll lll ll lll l ll llll l ll l llll ll ll ll ll ll lllll llll ll ll ll l lll ll l l lll llll l lll ll l llll ll ll lll l ll lll ll lll ll l ll lll lll lll l l llll lll lll ll ll lll lll l l lllll ll l ll l l lll l ll lll ll llll ll l lll lll ll l ll l lll lll lll l l lll l l lll lll llll l ll lll ll ll l ll lll lll ll l ll l ll ll ll lll l lll ll ll l ll lll ll lll llll ll ll l lll ll lll l lll ll ll l ll ll ll ll l lll l lll l ll lll l l ll llll l l llll l llll ll l ll l ll ll lll l ll ll ll ll l l lll l lll lll l lll llll llll ll l ll ll ll lll ll llll ll lllll llll l lll l lll ll ll ll ll ll lllll llll lll llll lll ll l ll ll llll lll llll ll llllll l llll l ll ll ll l l ll lllll lll llll lll ll ll ll l l llll ll ll ll llll l ll llll ll lll ll ll lll lll lll l ll ll l l ll lll ll ll ll ll ll ll lll l llll ll ll l ll lll ll l ll l ll l lll ll ll lll ll lll lll llll l llll ll lll ll lll ll l l ll ll l l lll ll ll l l ll ll l l l lll l lll lll ll ll lll ll lll ll l llll ll ll llll ll lll l l lll lll llll ll l lll ll lllll l llll ll l ll llll ll l lll l ll l ll ll ll ll lll lll lll llll l lllll lll ll llll lllll llll l ll lll ll ll ll l llll lllll ll ll l lllll llll llll lll llll llll l l ll l llll ll lll l ll ll lll lll l lll l ll l llllll lll ll ll ll llll l l ll ll l ll ll ll l lll l l lll llll l lll lll ll l ll ll llll l l llll lll l ll l l ll lll lll l llll lllll l lll ll ll lll ll l ll ll ll ll l ll ll ll l lll lll ll ll llll lll ll l ll llllll ll lll llll l ll ll ll lll lll l ll ll l lll l lll ll ll lll ll l lll ll ll lll ll ll ll llll lllll lll ll lll l lll llll ll ll lll lllll ll ll lll l l lll l ll ll ll ll llll ll l lll ll ll lllll ll l lll l ll ll l l llll ll llll l l l ll ll l lll llll ll lllll ll lll lll l l lll ll l llll ll l ll lll l llll ll lll l ll ll l ll ll ll l ll lll lll lllllll lll ll l l lll lll l ll ll ll lll ll ll lll lll l ll ll l llll l ll llll l ll ll ll l l ll l lll ll ll ll ll l ll ll llllll ll l ll l ll ll ll lll ll l ll lll ll l ll l ll ll l llll lll lll ll lll l llll ll lll ll ll ll llll l ll l llll lll lll ll l lll lll ll ll lllll ll l l lll l l l lllll ll l ll ll llll llll ll ll ll l ll ll l lll ll ll l lll l ll l lll l ll lll ll ll lll ll l ll ll ll ll ll ll l l lll ll lll ll ll llll lll ll l l ll ll ll lllll ll lll lll l ll ll llll lll l ll llll llll ll ll ll lll ll l ll ll ll ll l ll lll l l lll ll ll ll ll l ll llll llll l lll llll ll lllll l ll lll l ll ll ll lll l l lllll ll ll llll ll ll ll lll llll ll lllll llll ll ll ll ll ll lll ll l ll lll l llll lll l llll lll lll lll l ll lllll lll l l ll ll l lll lll l ll l lllll l lll l ll l ll ll lll l lll l lll llll ll l l ll l l lllll lll ll l ll lll llll ll llllllll ll llll llll llll llll l ll l l ll l ll llll llll ll llllll llllll l llll l l ll lll ll ll l ll l l lll llll ll lll lll l ll lll l lll l lll ll lll lllll ll l ll l ll ll ll l lll lllll ll lll l ll ll ll l ll l lll ll ll l lll lll l ll lll ll ll lll l l lll ll l l ll llll l ll l l llll lll ll llll ll lll lll ll l lll llll ll ll ll llll ll llll ll ll l llll lll llll llll ll ll l llll ll ll l lll llll l llllll l l llll lll ll lll l lll llll l ll l ll ll ll l lll l ll lll ll l lllll ll ll ll lll ll ll lll l ll lll lll lll ll ll l l ll lll ll lll lll lll l lll l ll l ll l l lll l l llll ll ll lll l ll llll ll l l ll ll l ll ll lll llll lll lll l l llll ll lll lll ll ll l lll l llll ll ll lll lll llll l l llllll l ll llll ll lll l ll ll ll lll ll ll ll ll l lll llll lllll ll lll l ll lll ll l ll ll llll ll l lll lll ll lll ll lll l l ll ll llll ll l ll ll ll ll llll l llll ll ll l lll llll ll ll llll l l ll ll lll ll ll ll ll lll ll l ll ll ll llll ll llll lll l l ll ll ll ll ll ll ll llll ll l ll lll ll lll ll lll ll l ll ll ll l lll ll ll ll ll lll ll l ll ll l l ll ll ll llll lll l ll ll ll lll lll lll l l ll llll l lll ll ll l ll l ll lll llll ll l ll ll l l lll ll lll lll ll llll llll l l lll lll l l ll ll ll ll l llll ll lll ll lll ll l ll l llll lll l ll l lll l ll lll l l llll l ll ll lll l l ll ll l llll ll lll l llll ll ll ll ll l l ll l ll l lll l lllll l lll l l ll l ll ll lll llllll lll lll llll lll lll lll ll ll llll ll ll lll l ll ll llll lll ll l l l l ll ll lll lll ll lll l ll l lll l l llll l ll ll llll ll ll l ll l lll l l l lll lll lllll ll ll lll ll l ll l lll l lll ll llll ll ll ll l ll ll ll ll llll lll l ll ll l ll llll llll ll ll lll ll l ll l ll l lllll l l l lllll l ll ll ll l ll ll l l ll ll ll ll ll llll l l llll ll ll ll ll lll ll l llll ll llll lll ll l lll l ll ll ll ll ll l ll ll ll lll l ll l lll ll l ll l llll l lll ll ll ll l l ll ll ll llll ll llll l llll lll lll ll ll l ll l ll l ll l lll ll ll lll l l ll ll l ll llll l ll llll lll ll l ll l ll l lll ll lll l ll l llll ll llll ll l ll ll l ll lll l ll lll ll lll ll lll l lll ll ll l ll ll l ll ll ll l l llllll l ll ll ll l ll lll lllll ll llll ll l ll l lll llll ll l ll ll ll llll ll ll ll ll ll ll ll ll ll llll l ll ll lll ll l lll lll ll l ll lll l ll ll ll ll ll ll l l lll llll l ll l lll l lll lllll l lll lll ll ll ll l llll ll llll ll ll lll ll ll l ll ll ll l lll ll lll ll ll ll ll llll ll ll l l ll ll lll ll ll ll l ll ll lll l lll l llll ll l lll ll l ll l llll ll ll ll ll llll llll llll ll ll lllll ll ll lll lllll ll l llll ll ll ll l ll l lll ll ll l l llll ll l ll ll l l lll lll l ll ll ll ll ll l ll l l l ll l ll lll lll lll llll l llll l l ll ll ll lll l l ll l ll lll l llll ll lll ll lll ll ll l l lll ll l ll lll lll ll l lll ll l ll llll lllll l llll l ll lll llll ll l lll lll ll ll l l ll l lll ll llllll lllll lll l llll ll ll lll ll llll l ll ll l ll llll l l lll l llll ll lll llll ll ll ll ll l ll l lll lll llll lll l lll l ll l l llll ll llllll lll l lll ll ll ll lll ll ll llll ll l lll lllllll l ll l ll lll ll l ll l ll llll ll lll l ll lll lll l lll lll l lll l llll l ll l ll l ll lll ll ll lllll ll l ll ll l llll l ll lll lllll lll l l ll ll ll l ll l ll ll l llll ll l l lll lll llll ll ll ll ll ll lll ll lll lll l l lll lll ll lll l lll ll lll l ll llll ll ll ll ll ll l lll l ll ll lll l lll ll lll llll ll l ll lll ll ll lll ll ll ll l lll ll lll llll ll ll lll lll l ll llll lll lll l l ll lll ll lll ll l lll ll l llll lll l ll l ll ll lll ll ll l ll ll ll l l lllll lll ll l ll l ll l ll l lll l ll l lll lll ll l ll lll lll ll l l ll l l l lll l ll ll ll l l lll l ll ll ll ll ll l llll lllll l ll ll ll l ll l llll ll ll ll l lll l ll l ll l ll llll llll ll ll l ll lll ll l llllll l llll lll ll llll lll l ll lll llll l llll llll ll ll l lll ll llll l ll l ll llll lll lll ll lll l ll ll ll ll ll ll lllll lll ll l l llll lll l ll l ll ll ll llll l l ll l lllll ll ll llll lll l ll lll ll ll lll ll ll ll ll lll lll lll llll l lll l ll l ll l ll ll lll l ll l l ll lll lll ll l ll ll lll ll l llll l lll ll l ll lll l lll lll lll ll llll l ll lll ll ll l lllll ll ll ll llllll ll l ll ll ll l ll ll ll l lll l ll ll ll l l llllll l ll l lll l ll ll l lll l lll lll lll ll ll lllll ll lll lll ll ll lll l ll lll lll ll l l ll ll ll l l ll llll ll l ll ll ll lll ll l ll lll lll ll lll llll lll ll lll l ll l llll lll ll l ll l llll l llll ll l lll ll l ll ll ll lll ll ll ll ll ll ll l ll l ll ll ll l ll l ll ll l l ll l ll ll ll l ll lllll ll l ll lll ll ll l llll ll ll lll ll lll ll lll ll l lllll ll l lll l lll ll lll llll l ll lll ll ll l ll l ll lll l lll ll ll l lll lll ll l ll ll lll lll ll l lll lll lllll llll l lll l ll lll ll l ll ll l lll lll l lll lll ll lll l ll ll l l llllll ll lll ll l lll l ll ll l l l lll lll l lll ll ll lll l lll ll ll llll ll lll ll ll ll l ll l lll ll l lll l lll l l lll ll lll l ll ll lll lll l ll lllll l l llll l llllll l ll lll ll ll ll ll lll ll ll l ll lll l lll l ll l lll lll ll ll ll llll ll l llll lll l ll lll lll ll lll lllll ll ll ll l ll ll ll ll lll l lllll lll l l lll ll ll ll ll llll ll lll l llll ll lll lll ll lll lllll ll lllll ll l ll ll lll l ll l ll ll ll lll l ll l ll ll lllll lll ll l l ll lll ll l lllll l lll lll ll llll l l lll l ll l l llll l ll lll lllll ll l l ll lll l lll ll l lll l llll l lll ll l lll ll l ll ll llll l l lll l ll ll lll ll l lll lll ll l ll l ll l lll ll ll lll ll l lll l lll ll l l ll ll ll llll l lll l lll ll ll ll lll lll ll lll l lll lll llll ll l l lll lll lll ll lll lll l ll l ll ll ll l ll l l l lll ll lllll l l llll ll lll lll ll lll lll lll lll ll ll ll ll ll ll ll ll ll l ll llll l lll ll l ll l lll ll ll l ll lll l lll llll ll lll ll l ll l lll ll lll l ll ll ll lll l l l llll l l llll l l ll l ll l l lllll l ll lll lll l lll l lll l ll l ll lll l lll lll ll lll ll ll ll ll ll lll ll ll ll lll ll ll ll ll l l ll ll lll l ll l lll ll ll l ll ll l lll l ll llll lll l ll ll ll ll l ll l ll l lll ll l ll ll ll l ll ll l llll ll lll ll ll l ll ll ll l ll l llll lll l ll llll l l l llll l l l ll lll ll l lll ll llll l lll lll lll l lll l ll ll l l llll lll ll llll ll lll ll lll lll ll l l lll llll ll ll llll ll lll lll ll llll l ll ll l ll ll llll ll l llll llll lll ll l ll lll ll ll ll lll l ll l ll l ll l llll l l ll ll llll ll ll ll ll ll l lll ll llll l lll ll lllll l l ll l lll lllll l lll ll lll l ll ll ll ll ll l lll l l ll lll l l l lll ll lll l l lll l ll ll l ll ll ll ll l ll lll l ll ll lllll ll ll l llll l l l ll llll l l ll ll l ll l llll lll llll l ll lll ll lll ll l l ll l llll ll l lll lll l lll llll lll ll l ll ll ll l llll ll ll llllll ll l l ll ll ll lllll ll lll lll lll l l ll ll ll lll ll l ll ll lll l l lll l llll ll lll ll ll lll l ll l llll llll l l ll ll lll l l ll l ll l l l ll ll ll l lll l lll l ll l lll lll ll ll lll ll l ll ll l ll l ll l l llll ll lll l ll ll lll l lll ll ll l ll ll lll lll lll ll ll l ll l ll l l lll ll ll lll l l lll l ll lll lll ll l lll l l l lll ll llll l ll ll ll ll l ll ll llll lll lll ll lll ll llllll l ll l lll lll lll lll lll l ll ll l ll ll l lll l l ll llll l lllll ll lll llll l ll lll l ll ll lll ll ll lll l ll l ll ll ll lll ll l ll ll l ll l ll l lll lll ll ll llll l lll l lll llll llll lll lll ll l l lll ll ll l l llll lll ll ll lllll l l lll l l ll ll lll llll l ll ll l lll ll ll lll ll l ll lll ll ll ll ll ll ll lll l ll ll llll l ll ll l lll l llll ll lll lll ll lll l llll ll lll ll lll ll ll l ll ll ll l ll l l ll ll l ll ll ll l llll ll l lll ll l l lll lll ll l llll l l ll ll l ll l l ll llll lll lll ll ll ll l lll lll ll l ll ll ll ll ll l ll l l lll ll ll ll ll ll lllll ll ll llll ll lll l l llll l l ll lll ll lll lll llll llll lll lllll llll lll ll ll l lll l lll lll ll l ll l ll l lll ll ll ll ll llll l ll l ll l lll llll lll l ll ll l ll ll lll l lll lll l l lll ll l lll lllll l ll lll lll lll l ll lll l llll llll llll ll ll l ll l ll llll llll l llll l lll l lll lll ll ll l l ll ll l lll l ll ll l lll l ll l lll lll lll l lll lll l llll l lllll ll lll lll l lll ll llll llll ll ll ll lll lll l llll l l lll ll ll l llll l ll llll ll llll lll ll ll ll l ll llll l ll l l lll ll l l l lll l ll ll l lll l ll l lll ll l lll ll ll l l ll ll l llll l lll l ll lll ll llll lll llll lll l ll lllll ll l lll ll l lll lllll l l ll ll ll l l lll l ll llll l ll lll ll ll ll ll lll ll llll l l lll lll l llll ll llll ll lll ll llll ll l l ll ll ll l ll ll lllll l ll ll lll ll ll lll ll lll ll ll l ll ll l ll lll ll l ll l ll l l ll llll l l ll ll lll ll l lll l ll l l ll lll l l lll llll ll l lll lll ll ll lll l ll ll l ll ll ll ll ll ll l ll l lll lll lll l ll ll ll ll llll lll ll ll l ll l ll l l ll ll ll l llll lll ll lll lll ll ll lll l ll l ll ll lll ll l ll ll lllll ll ll llll l l lll ll l lll lll ll ll llll ll lll ll l l l llll l lll llll ll llll ll l lll ll l ll ll ll l lll l l ll llll l ll ll lll l l ll ll l l lll lll llll ll ll l ll l lllll lll lllll ll ll llll ll llll l llll l ll l ll llll l llll lll ll ll lll ll l llll l l ll ll lllllll lll l ll ll llll l ll ll ll l l l l ll l ll ll l llll l l l lll ll l ll l lll ll lll ll l ll l ll ll l ll ll l llll l ll ll l ll ll ll ll l l lll ll lll l ll ll ll l lll lll l lll ll ll ll ll ll ll ll l lll l ll l ll ll l l l ll lll l ll ll ll ll ll lll l ll l ll llll ll l ll ll lll l ll l llll lll ll l l ll ll ll llll llll ll ll ll lll l ll l l lll ll lll l l lll lll l ll l lllll l lll llll ll ll llll lll l ll l lll l ll llll ll l lll lll lll ll ll ll ll lll ll ll llll lll lll ll ll l ll l lll l llll l ll ll ll ll l lll l l ll lll l lll l lll l ll l lll l l ll l l lll l ll llll l llll l lll l l ll ll l lll llll l ll ll lll ll lll ll llll l l lll l l ll ll l l l l llll l lll l ll ll ll lll ll l ll ll lll llll l lll ll lll l llll ll ll l lll ll l ll ll l l ll ll l l lll lll ll ll ll lll lll l lll lll lll ll ll lll l lll ll l llll l ll l llll l l lllllll l ll lll l l lll ll lll ll ll l ll ll l ll ll ll lll ll lll lll ll l ll lll ll ll ll lll l l ll l ll lllll ll l ll ll lll lll ll ll l ll ll ll lll ll l ll l l ll lll l l ll ll llll l ll ll ll lll l lll llll lll ll ll ll ll llll l llll ll ll lll ll l ll l llll lll lll ll l lll llll ll l llllll ll ll ll ll llllll l l ll l lllllll l llll lll l ll lll lll lll ll ll ll lll l l l lll lll lllll l ll l ll lllll lll llll llll l ll ll ll ll ll ll llll ll llllll ll ll l llll l lll l llll ll ll lllll ll ll ll ll lll ll l ll llll ll lll ll ll llllll lll llll ll l ll lll lll l ll ll ll ll ll lll llll l ll l l ll lll ll l ll ll l lll l lll ll lll lll l lll lll ll lll l ll ll lll l lll l l lll ll lll llll lll ll l l lll lll ll lll ll ll lll l lll lll ll l llll l l llll ll l lll l lll ll l lll l l llllll ll l l lll lllll ll ll lll l lll ll lll lll ll ll l llllll lll lll lll l lll ll ll lll l ll ll lll l ll lll l ll ll l lllll lll l ll lllllll l l lll l llll llll l ll l lll ll ll lll lll lll llll l ll l ll ll lll l l ll ll l ll l l ll ll l ll lll l lll l lll lll ll ll l l llll lll ll ll lll lll ll l ll l llll l lll l ll l lll ll ll lll lllll l llllll l ll l lllll ll ll lll l l ll lll l ll ll llll l ll ll lll lllll l llll l l lll l lll ll l lll llll l ll l lll lll lll lllll ll llll ll lll l lll l ll ll ll ll l ll lll l ll l lll l llll ll ll ll ll ll ll ll ll lll l ll ll l ll llll lll l l ll ll ll llll ll ll l llll l lllll l l ll l l ll ll ll ll l lll l ll ll ll ll lll l l ll l lll l ll l ll ll ll ll l ll l l l lll ll lll l ll lll ll ll ll ll ll ll ll lll lll llllll ll ll ll llll ll ll ll ll l lll lll lll l ll ll lllll ll lll lll lll ll lll llll l ll l ll ll l lll l lll llll ll l lll ll llll l ll l l l l ll ll lll l ll ll l ll lllll ll l llll l ll l ll ll l lll ll l ll ll l ll ll l ll ll l l lll ll ll ll ll l lll ll lll l ll ll l l ll ll lll ll ll lll l lll ll ll ll l ll l ll ll lll l ll ll ll ll ll lll ll l ll lll lll lll ll lll ll lll ll l ll l ll llll ll ll l ll l ll ll ll lll lll l lll lll l ll llll ll ll llll ll ll lll lll ll ll l l lll ll ll llll lll ll l l lll ll llll lll l llll lll llllll l lllll lll ll l lllll ll llll ll l ll l llllll l l l lll ll ll llll lll l lll ll ll lll ll l ll ll l lll lll l ll l lll l l llll l lllll ll llll lll lll ll l lll l lll l lllll ll lll l l ll ll ll ll l llll l ll lll llll ll llllllll l l l ll ll ll ll lll llll ll lll lll ll lll l ll llll ll ll l ll ll lll ll l l l lllll ll l l lll ll ll ll ll l lll ll lll ll lll l ll ll l ll lllll ll l lll l lll ll lll l l l ll ll ll lll ll l ll l llll ll ll ll lll l ll l lll llll lll ll l l lll lll l ll l ll l lll llll ll ll ll lll ll l lll ll ll ll ll ll ll ll lll l l lll ll llll ll llll l ll ll ll lll llll lll lll ll lll ll ll lll ll ll l lll ll l l ll l ll lll l ll lll ll l ll ll l lll lll ll l lllll lll l ll l lll lllll ll ll ll ll ll lll ll ll ll ll ll lll ll ll l lll l lll l ll ll l ll lllll ll ll ll ll l llll lll ll ll l l ll ll ll llll l l ll ll ll ll ll l lll l l ll ll l lll ll ll ll l lll l lll l ll llll l llll l l llll l ll ll ll lll lll ll ll l l l lll l lll l ll l ll ll lllll ll l lll lll ll l ll l ll l llllll llll lll ll l lll lll l ll llll lll llll lll ll ll l ll l lll lll l llll ll ll ll ll ll l ll l ll llll l ll l ll ll lll l lll l lll ll lll ll lll l lll lll ll lll l ll ll ll lll llll ll lll lll ll lll ll ll lllll ll ll l lll lll ll l llllll ll lll ll ll l lll ll lll ll ll ll ll ll l llll l ll ll ll l ll l ll ll ll lll ll lll l llll ll l llll ll l ll ll l ll l ll l l lll llll llll ll l l lllll ll l ll ll ll ll ll lll ll lll lll ll l ll lll lll lll l l llll l ll l lll lll ll ll lll ll ll ll lllllll ll lllll l l lll ll ll lll ll ll l l ll ll l l lll l ll lll l lll lll l ll lll ll lllll l l lll ll ll ll ll l l lll l l ll l ll ll ll ll llll lll llll l llll lll ll lll l ll lllllll lll ll ll l ll lll llll lll l ll ll ll ll lll lll lll lll ll llll lll ll lll ll l ll ll lll l lll lllll lllll ll ll l ll ll l l lll ll ll l ll ll ll ll ll ll l ll llll ll l l ll llll llll ll llll l ll lll ll lll ll ll ll l lllll lll ll lll ll l llll l ll l ll ll ll ll l ll lll ll llllll ll l l ll lll ll ll l l lll ll lll l l ll lll l l ll ll l ll ll l ll ll lll lllll lll l lll lll ll l l ll lll ll l ll lll l l lllll ll l l lllll ll llll ll llll llll ll ll l ll l l ll l lll ll ll l ll lll ll l ll l ll lll l ll ll lll ll l ll ll ll lll l l lllll l ll lllll l l ll ll l ll ll ll l ll l ll lllll ll ll llll lll l ll lll l ll lll l llll l l ll ll lll lll ll l lll lll lll ll l l ll lll ll ll ll llll ll ll l l lll l l llll lll l lll llll lll ll l ll llll ll ll ll l lll l lll lll ll l llll ll ll lll ll l l ll ll ll ll l lll l ll lll ll llll l ll ll l ll l lll ll lllllll l llll lll ll ll llll l llll ll ll ll lll ll lllll ll llllll ll ll lll ll ll l lll ll lll ll lll ll llll ll ll lll lll l lllll lll lll ll ll llllll lllll l ll l ll ll l ll lll ll l ll llll ll ll lll l ll lllllll l l lllll ll l lll l lll ll l llll ll ll l lll llll ll ll ll l lll l l l l lllll l ll l lll ll ll llll lllll l lll ll l llll ll llll llll l ll l lll ll llll ll l llll lll l ll ll lll ll ll ll l ll l lll ll lll lll ll lll ll l ll ll l lll ll l llll ll l llll l ll ll lll l ll lll ll l ll ll l l ll l llll ll lll l ll lll llll lll l ll lll ll l l ll lll ll ll ll ll ll lll l lll l ll ll lll ll l lll ll ll ll ll ll llll ll llll lll ll llll ll llll lll lll lll lll l ll l lll llll l lll ll llll ll l ll llll lll l ll lll l ll l lll ll ll lll l lll l llll llll ll l l lllll lll ll l ll lll lll l ll ll ll l ll l ll ll ll ll ll l ll ll ll lll ll ll l ll ll ll lll llllll l ll ll ll ll ll lll ll llll ll l ll l ll l l ll ll ll lll ll ll lll l l ll lll lllll l l l ll ll l l l lll l ll lll l lll ll ll lll ll ll llllll ll l lll l ll l ll llll ll ll lllll lll lll l ll lll llll lll lll l ll ll l l lll l lll ll ll l l ll ll l l ll lll ll lll lll l l lll lll lll l ll l ll lll l l lllll lllll l lll ll llll l l lll ll ll lllll ll l ll l llll ll ll lll ll ll l ll ll llll l lll ll lll l llll a=0.1a=0.25a=0.5a=1−0.020.000.02−0.020.000.02−0.020.000.02−0.020.000.02 −0.02 0.00 0.02 S S \tilde{S} Figure 7: A mapping from S to (cid:101) S for four simulated S matrices under the Spatial model. The leftcolumn shows the simulated structure S for each of four scenarios (a–d) and the right column showsthe resulting estimated row basis of S produced from PCA. It can be seen that the scale on which S was generated, all values in (0,1), is lost in the principal components, values in R .29 ab l e3 : A cc u r a cy i ne s t i m a t i ng π i j pa r a m e t e r s b y t he P C A ba s ed m e t hodandL F A . E a c h r o w i s ad i ff e r en t s i m u l a t i on sc ena r i o . E a c h c o l u m n i s t hea cc u r a cy o f a m e t hod ’ sfi t s w i t h t heg i v en m e t r i c . S c ena r i o M ed i an K L M ean A b s . E rr . R M SE P C A L F AA D X F SP C A L F AA D X F SP C A L F AA D X F S BN . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - PSD α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - α = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - Spatial a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - a = . . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - TGPfit P C A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - A D X . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F S . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - HGDPfit P C A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F A . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - A D X . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - F S . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - . E - The top 50 SNPs most associated with structure in the HGDP data, identified by performing a logistic regression ofSNP genotypes on the logistic factors. Shown are the SNP ID and location, deviance measure of differentiation, gene closestto the SNP, distance to gene (rounded to nearest 10bp), and the variant type (if none shown, then intergenic). rsid chr position deviance genesymbol locusID distance variant type1 rs1834640 15 48392165 1605.28 SLC24A5 283652 210002 rs2250072 15 48384907 1313.82 SLC24A5 283652 282603 rs12440301 15 48389924 1263.83 SLC24A5 283652 232404 rs260690 2 109579738 1262.72 EDAR 10913 0 intron-variant5 rs9837708 3 71487582 1189.48 FOXP1 27086 0 intron-variant6 rs260714 2 109562495 1184.50 EDAR 10913 0 intron-variant7 rs4918664 10 94921065 1178.40 XRCC6P1 387703 453408 rs10882168 10 94929434 1160.99 XRCC6P1 387703 369709 rs300153 2 17986417 1143.48 MSGN1 343930 1136010 rs9809818 3 71480566 1135.58 FOXP1 27086 0 intron-variant11 rs6583859 10 94893473 1119.25 NIP7P1 389997 2629012 rs11187300 10 94920291 1114.22 XRCC6P1 387703 4612013 rs260698 2 109566759 1111.64 EDAR 10913 0 intron-variant14 rs1834619 2 17901485 1111.40 SMC6 79677 0 intron-variant15 rs11637235 15 48633153 1104.45 DUT 1854 0 intron-variant16 rs4497887 2 125859777 1097.13 RNA5SP102 100873373 16918017 rs7091054 10 95018444 1085.45 RPL17P34 643863 2528018 rs7090105 10 75131545 1075.50 ANXA7 310 364019 rs973787 4 38263893 1074.57 TBC1D1 23216 12309020 rs4279220 4 38254182 1070.43 TBC1D1 23216 11338021 rs7556886 2 17908130 1062.58 SMC6 79677 0 intron-variant22 rs12473565 2 175163335 1056.31 LOC644158 644158 139023 rs6500380 16 48375777 1051.10 LONP2 83752 0 intron-variant24 rs2384319 2 26206255 1033.88 KIF3C 3797 810 upstream-variant-2KB25 rs12220128 10 94975011 1023.79 XRCC6P1 387703 609026 rs17034770 2 109616376 1019.03 EDAR 10913 1054027 rs3792006 2 26498222 998.96 HADHB 3032 0 intron-variant28 rs4918924 10 94976956 994.79 XRCC6P1 387703 803029 rs1984996 10 95008745 990.92 RPL17P34 643863 3498030 rs3751631 15 52534344 987.33 MYO5C 55930 0 reference,synonymous-codon31 rs4578856 2 17853388 987.29 SMC6 79677 0 intron-variant32 rs13397666 2 109544052 986.80 EDAR 10913 0 intron-variant33 rs12619554 2 17352372 986.20 ZFYVE9P2 100420972 11318034 rs3736508 11 45975130 981.05 PHF21A 51317 0 missense,reference35 rs12472075 2 177691130 973.02 RPL29P8 100131991 1665036 rs9522149 13 111827167 965.50 ARHGEF7 8874 0 intron-variant37 rs2917454 10 78892415 964.40 KCNMA1 3778 0 intron-variant38 rs10882183 10 94974083 961.04 XRCC6P1 387703 516039 rs10079352 5 117494640 960.33 LOC100505811 100505811 12362040 rs10935320 3 139056584 958.33 MRPS22 56945 627041 rs9571407 13 34886039 957.04 LINC00457 100874179 12354042 rs6542787 2 109556365 955.56 EDAR 10913 0 intron-variant43 rs953035 1 36079508 954.67 PSMB2 5690 0 intron-variant44 rs4657449 1 165465281 951.72 LOC400794 400794 0 intron-variant45 rs9960403 18 13437993 949.43 LDLRAD4 753 0 intron-variant46 rs203150 18 38037221 944.32 RPL17P45 100271414 31275047 rs2823882 21 17934419 942.05 LINC00478 388815 0 intron-variant48 rs10886189 10 119753963 937.81 RAB11FIP2 22841 1046049 rs2441727 10 68224886 937.08 CTNNA3 29119 0 intron-variant50 rs310644 20 62159504 931.90 PTK6 5753 260 downstream-variant-500B able 5:
The top 50 SNPs most associated with structure in the TGP data, identified by performing a logistic regression ofSNP genotypes on the logistic factors. Shown are the SNP ID and location, deviance measure of differentiation, gene closestto the SNP, distance to gene (rounded to nearest 10bp), and the variant type (if none shown, then intergenic). rsid chr position deviance genesymbol locusID distance variant type1 rs1426654 15 48426484 3129.76 SLC24A5 283652 0 missense,reference2 rs3827760 2 109513601 2395.27 EDAR 10913 0 missense,reference3 rs922452 2 109543883 2338.38 EDAR 10913 0 intron-variant4 rs372985703 17 19172196 1975.16 EPN2 22905 0 intron-variant5 rs4924987 17 19247075 1949.03 B9D1 27077 0 intron-variant,missense,reference6 rs260687 2 109578855 1925.18 EDAR 10913 0 intron-variant7 rs7209202 17 58532239 1890.67 APPBP2 10513 08 rs7211872 17 58550725 1890.67 APPBP2 10513 09 rs67929453 3 139109825 1890.57 LOC100507291 100507291 0 intron-variant,upstream-variant-2KB10 rs260643 2 109539653 1850.71 EDAR 10913 0 intron-variant11 rs260707 2 109574150 1838.37 EDAR 10913 0 intron-variant12 rs1545071 18 67695505 1821.35 RTTN 25914 0 intron-variant13 rs12729599 1 1323078 1812.91 CCNL2 81669 0 intron-variant14 rs12347078 9 344508 1811.16 DOCK8 81704 0 intron-variant15 rs12142199 1 1249187 1779.28 CPSF3L 54973 0 reference,synonymous-codon16 rs12953952 18 67737927 1750.15 RTTN 25914 0 intron-variant17 rs9467091 6 10651772 1746.75 GCNT6 644378 427018 rs7165971 15 55921013 1736.83 PRTG 283659 0 intron-variant19 rs6132532 20 2315543 1730.64 TGM3 7053 0 intron-variant20 rs959071 17 19142226 1729.18 EPN2 22905 0 intron-variant21 rs10962599 9 16795286 1726.24 BNC2 54796 0 intron-variant22 rs967377 20 53222217 1724.93 DOK5 55816 0 intron-variant23 rs4891381 18 67595449 1723.79 CD226 10666 0 intron-variant24 rs377561427 15 63988357 1713.98 HERC1 8925 0 frameshift-variant,reference25 rs73889254 22 46762214 1711.40 CELSR1 9620 0 intron-variant26 rs4918664 10 94921065 1700.64 XRCC6P1 387703 4534027 rs2759281 1 204866365 1691.03 NFASC 23114 0 intron-variant28 rs12065033 1 173579034 1682.54 ANKRD45 339416 0 utr-variant-3-prime29 rs9796793 16 30495652 1681.28 ITGAL 3683 0 intron-variant30 rs1240708 1 1335790 1675.48 LOC148413 148413 0 intron-variant,upstream-variant-2KB31 rs2615876 10 117665860 1670.53 ATRNL1 26033 0 intron-variant32 rs2823882 21 17934419 1669.32 LINC00478 388815 0 intron-variant33 rs8097206 18 38024931 1663.29 RPL17P45 100271414 30046034 rs8071181 17 58508582 1662.44 C17orf64 124773 0 reference,synonymous-codon35 rs1075389 15 64174177 1661.21 MIR422A 494334 1095036 rs6875659 5 175158653 1657.54 HRH2 3274 2241037 rs7171940 15 64170986 1654.01 MIR422A 494334 776038 rs2148359 9 7385508 1652.16 RPL4P5 158345 9144039 rs7531501 1 234338303 1648.15 SLC35F3 148641 0 intron-variant40 rs57742857 15 93567352 1645.21 CHD2 1106 0 intron-variant41 rs931564 17 58631702 1636.86 LOC388406 388406 1020042 rs4738296 8 73857539 1632.70 LOC100288310 100288310 0 intron-variant43 rs4402785 2 104766351 1631.33 LOC100287010 100287010 22895044 rs12988506 2 33162854 1630.14 LOC100271832 100271832 0 intron-variant45 rs9410664 9 91196828 1625.48 NXNL2 158046 612046 rs2041564 2 72453847 1623.91 EXOC6B 23233 0 intron-variant47 rs6024103 20 54034601 1623.41 LOC101927796 101927796 227048 rs6583859 10 94893473 1619.79 NIP7P1 389997 2629049 rs12913832 15 28365618 1611.23 HERC2 8924 0 intron-variant50 rs632876 2 216572452 1610.26 LINC00607 646324 0 intron-variant eferences [1] McCarthy, M. I., Abecasis, G. R., Cardon, L. R., Goldstein, D. B., Little, J., Ioannidis, J. P. A., andHirschhorn, J. N. Genome-wide association studies for complex traits: consensus, uncertainty andchallenges.
Nature reviews. Genetics (5), 356–369 (2008).[2] Frazer, K. A., Murray, S. S., Schork, N. J., and Topol, E. J. Human genetic variation and itscontribution to complex traits. Nat Rev Genet (4), 241–251, Apr (2009).[3] Wellcome Trust Case Control Consortium. Genome-wide association study of 14,000 cases ofseven common diseases and 3,000 shared controls. Nature (7145), 661–678, Jun (2007).[4] Pritchard, J. K. and Rosenberg, N. A. Use of unlinked genetic markers to detect population strati-fication in association studies.
Am J Hum Genet (1), 220–228, Jul (1999).[5] Astle, W. and Balding, D. J. Population structure and cryptic relatedness in genetic associationstudies. Statistical Science , 451–471 (2009).[6] Price, A. L., Zaitlen, N. A., Reich, D., and Patterson, N. New approaches to population stratificationin genome-wide association studies. Nat Rev Genet (7), 459–463, Jun (2010).[7] Kang, H. M., Sul, J. H., Service, S. K., Zaitlen, N. A., Kong, S.-Y., Freimer, N. B., Sabatti, C., andEskin, E. Variance component model to account for sample structure in genome-wide associationstudies. Nature Genetics (4), 348–354 (2010).[8] Jorde, L. B., Watkins, W. S., and Bamshad, M. J. Population genomics: a bridge from evolutionaryhistory to genetic medicine. Hum Mol Genet (20), 2199–2207, Oct (2001).[9] Nielsen, R., Hellmann, I., Hubisz, M., Bustamante, C., and Clark, A. G. Recent and ongoingselection in the human genome. Nat Rev Genet (11), 857–868, Nov (2007).[10] Rosenberg, N. A., Pritchard, J. K., Weber, J. L., Cann, H. M., Kidd, K. K., Zhivotovsky, L. A., andFeldman, M. W. Genetic structure of human populations. Science , 2381–2385 (2002).[11] Cann, H. M., de Toma, C., Cazes, L., Legrand, M.-F., Morel, V., Piouffre, L., Bodmer, J., Bodmer,W. F., Bonne-Tamir, B., Cambon-Thomsen, A., Chen, Z., Chu, J., Carcassi, C., Contu, L., Du,R., Excoffier, L., Ferrara, G. B., Friedlaender, J. S., Groot, H., Gurwitz, D., Jenkins, T., Herrera,R. J., Huang, X., Kidd, J., Kidd, K. K., Langaney, A., Lin, A. A., Mehdi, S. Q., Parham, P., Piazza,A., Pistillo, M. P., Qian, Y., Shu, Q., Xu, J., Zhu, S., Weber, J. L., Greely, H. T., Feldman, M. W.,Thomas, G., Dausset, J., and Cavalli-Sforza, L. L. A human genome diversity cell line panel.
Science (5566), 261–262, Apr (2002).12] Rosenberg, N. A., Mahajan, S., Ramachandran, S., Zhao, C., Pritchard, J. K., and Feldman, M. W.Clines, clusters, and the effect of study design on the inference of human population structure.
PLoS Genet (6), e70, Dec (2005).[13] 1000 Genomes Project Consortium. A map of human genome variation from population-scalesequencing. Nature (7319), 1061–1073, Oct (2010).[14] Pritchard, J. K., Stephens, M., and Donnelly, P. Inference of population structure using multilocusgenotype data.
Genetics (2), 945–959, Jun (2000).[15] Tang, H., Peng, J., Wang, P., and Risch, N. J. Estimation of individual admixture: analytical andstudy design considerations.
Genetic epidemiology (4), 289–301, May (2005).[16] Alexander, D. H., Novembre, J., and Lange, K. Fast model-based estimation of ancestry in unre-lated individuals. Genome Research (9), 1655–1664 (2009).[17] Balding, D. J. and Nichols, R. A. A method for quantifying differentiation between populations atmulti-allelic loci and its implications for investigating identity and paternity. Genetica (1-2), 3–12(1995).[18] Menozzi, P., Piazza, A., and Cavalli-Sforza, L. Synthetic maps of human gene frequencies ineuropeans. Science (4358), 786–792, Sep (1978).[19] Sokal, Oden, and Thomson. A problem with synthetic maps.
Human Biology , 1–13 (1999).[20] Rendine, Piazza, and Cavailli-Sforza. A problem with synthetic maps: Reply to sokal et al. HumanBiology , 15–25 (1999).[21] Novembre, J. and Stephens, M. Interpreting principal component analyses of spatial populationgenetic variation. Nat Genet (5), 646–649, May (2008).[22] Manni. Interview with luigi luca cavalli-sforza: Past research and directions for future investigationsin human population genetics. Human Biology , 245–266 (2010).[23] Bartholomew, D. J., Knott, M., and Moustaki, I. Latent Variable Models and Factor Analysis: AUnified Approach . Wiley Series in Probability and Statistics, (2011).[24] Engelhardt, B. E. and Stephens, M. Analysis of population structure: a unifying framework andnovel methods based on sparse factor analysis.
PLoS Genet (9) (2010).[25] Raj, A., Stephens, M., and Pritchard, J. K. fastSTRUCTURE: Variational inference of populationstructure in large snp datasets. Genetics , 573–589 (2014).26] Kamberov, Y., Wang, S., Tan, J., Gerbault, P., Wark, A., Tan, L., Yang, Y., Li, S., Tang, K., andChen, H. Modeling recent human evolution in mice by expression of a selected edar variant.
Cell (4), 691–702 (2013).[27] Thornton, T., Tang, H., Hoffmann, T., Ochs-Balcom, H., Caan, B., and Risch, N. Estimating kinshipin admixed populations.
The American Journal of Human Genetics , 122–138 (2012).[28] Corona, E., Chen, R., Sikora, M., Morgan, A., Patel, C., Ramesh, A., Bustamante, C., and Butte,A. Analysis of the genetic basis of disease in the context of worldwide human relationships andmigration. PLoS Genetics (5), e1003447 (2013).[29] Song, M., Hao, W., and Storey, J. D. Testing for genetic associations under arbitrarily structuredpopulations. bioRxiv http://dx.doi.org/10.1101/012682 (2014).[30] McCullagh, P. and Nelder, J. A. Generalized Linear Models . Chapman and Hall, (1989).[31] Lehmann, E. L. and Casella, G.
Theory of Point Estimation . Springer Verlag, 2nd edition, (1998).[32] Baglama, J. and Reichel, L. Restarted block lanczos bidiagonalization methods.
Numerical Algo-rithms , 251–272 (2006).[33] Coop, G., Pickrell, J. K., Novembre, J., Kudaravalli, S., Li, J., Absher, D., Myers, R. M., Cavalli-Sforza, L. L., Feldman, M. W., and Pritchard, J. K. The role of geography in human adaptation. PLoS Genet (6), e1000500, Jun (2009).[34] Chung, N. C. and Storey, J. D. Statistical significance of variables driving systematic variation. arXiv [stat.ME] , 1308.6013 (2013). http://arxiv.org/abs/1308.6013.[35] Akey, J. M., Zhang, G., Zhang, K., Jin, L., and Shriver, M. D. Interrogating a high-density snp mapfor signatures of natural selection. Genome Res (12), 1805–1814, Dec (2002).[36] Lamason, R. L., Mohideen, M.-A. P. K., Mest, J. R., Wong, A. C., Norton, H. L., Aros, M. C., Ju-rynec, M. J., Mao, X., Humphreville, V. R., Humbert, J. E., Sinha, S., Moore, J. L., Jagadeeswaran,P., Zhao, W., Ning, G., Makalowska, I., McKeigue, P. M., O’donnell, D., Kittles, R., Parra, E. J.,Mangini, N. J., Grunwald, D. J., Shriver, M. D., Canfield, V. A., and Cheng, K. C. Slc24a5, a puta-tive cation exchanger, affects pigmentation in zebrafish and humans. Science (5755), 1782–6,Dec (2005).[37] Sabeti, P. C., Varilly, P., Fry, B., Lohmueller, J., Hostetter, E., Cotsapas, C., Xie, X., Byrne, E. H.,McCarroll, S. A., Gaudet, R., Schaffner, S. F., Lander, E. S., and International HapMap Consortium.enome-wide detection and characterization of positive selection in human populations.
Nature (7164), 913–918 (2007).[38] Banham, A. H., Beasley, N., Campo, E., Fernandez, P. L., Fidler, C., Gatter, K., Jones, M., Mason,D. Y., Prime, J. E., Trougouboff, P., Wood, K., and Cordell, J. L. The foxp1 winged helix transcriptionfactor is a novel candidate tumor suppressor gene on chromosome 3p.
Cancer research (24),8820–8829, Dec (2001).[39] Shigekawa, T., Ijichi, N., Ikeda, K., Horie-Inoue, K., Shimizu, C., Saji, S., Aogi, K., Tsuda, H.,Osaki, A., Saeki, T., and Inoue, S. Foxp1, an estrogen-inducible transcription factor, modulates cellproliferation in breast cancer cells and 5-year recurrence-free survival of patients with tamoxifen-treated breast cancer. Hormones & cancer (5), 286–297, Oct (2011).[40] Stone, S., Abkevich, V., Russell, D. L., Riley, R., Timms, K., Tran, T., Trem, D., Frank, D., Jam-mulapati, S., Neff, C. D., Iliev, D., Gress, R., He, G., Frech, G. C., Adams, T. D., Skolnick, M. H.,Lanchbury, J. S., Gutin, A., Hunt, S. C., and Shattuck, D. Tbc1d1 is a candidate for a severe obe-sity gene and evidence for a gene/gene interaction in obesity predisposition. Human moleculargenetics (18), 2709–2720, Sep (2006).[41] Sardella, M., Navone, F., Rocchi, M., Rubartelli, A., Viggiano, L., Vignali, G., Consalez, G. G., Sitia,R., and Cabibbo, A. Kif3c, a novel member of the kinesin superfamily: sequence, expression, andmapping to human chromosome 2 at 2p23. Genomics (3), 405–408, Feb (1998).[42] Jiao, H., Arner, P., Hoffstedt, J., Brodin, D., Dubern, B., Czernichow, S., van’t Hooft, F., Axelsson,T., Pedersen, O., Hansen, T., Sørensen, T. I. A., Hebebrand, J., Kere, J., Dahlman-Wright, K.,Hamsten, A., Clement, K., and Dahlman, I. Genome wide association study identifies kcnma1contributing to human obesity. BMC medical genomics , 51, Jun (2011).[43] Bernstein, D. I., Kashon, M., Lummus, Z. L., Johnson, V. J., Fluharty, K., Gautrin, D., Malo, J.-L.,Cartier, A., Boulet, L.-P., Sastre, J., Quirce, S., Germolec, D., Tarlo, S. M., Cruz, M.-J., Munoz, X.,Luster, M. I., and Yucesoy, B. Ctnna3 ( α -catenin) gene variants are associated with diisocyanateasthma: a replication study in a caucasian worker population. Toxicological sciences : an officialjournal of the Society of Toxicology (1), 242–246, Jan (2013).[44] Ostrander, J. H., Daniel, A. R., and Lange, C. A. Brk/ptk6 signaling in normal and cancer cellmodels.
Current opinion in pharmacology (6), 662–669, Dec (2010).[45] Jolliffe, I. T. Principal component analysis . New York: Springer, 2nd edition, (2010).46] Rosenberg, N. A. Standardized subsets of the hgdp-ceph human genome diversity cell line panel,accounting for atypical and duplicated samples and pairs of close relatives.
Annals of HumanGenetics , 841–847 (2006).[47] Weir, B. and Cockerham, C. Estimating F-statistics for the analysis of population structure. Evolu-tion , 1358–1370 (1984).[48] Weir, B. S. Genetic Data Analysis II: Methods for Discrete Population Genetic Data . Sunderland,MA: Sinauer Associates, (1996).[49] Bartholomew, D. J. The foundations of factor analysis.
Biometrika , 221–232 (1984).[50] Moustaki and Knott. Generalized latent trait models. Psychometrika , 391–411 (2000).[51] Bartholomew, D. J. Factor analysis for categorical data. J Roy Stat Soc B , 293–321 (1980).[52] Collins, M., Dasgupta, S., and Schapire, R. A generalization of principle component analysis tothe exponential family. In Proceedings of Advances in Neural Information Processiong Systems ,(2002).[53] Schein, A. I., Saul, L. K., and Ungar, L. H. A generalized linear model for principal componentanalysis of binary data. In
Proceedings of the 9 th International Workshop on Artificial Intelligenceand Statistics , (2003).[54] Guo, Y. and Schuurmans, D. Efficient global optimization for exponential family pca and low-rankmatrix factorization. In
In Allerton Conf. on Commun., Control, and Computing , (2008).[55] Paatero, P. and Tapper, U. Positive matrix factorization: A non-negative factor model with optimalutilization of error estimates of data values.
Environmetrics , 111–126 (1994).[56] Lee, D. D. and Seung, S. Learning the parts of objects by non-negative matrix factorization. Nature401