Identifiability and testability in GRT with Individual Differences
IIdentifiability and testability in GRT with IndividualDi ff erences Noah H. Silbert a , Robin D. Thomas b a Department of Communication Sciences and Disorders, University of Cincinnati b Department of Psychology, Miami University
Abstract
Silbert and Thomas (2013) showed that failures of decisional separability are not, ingeneral, identifiable in fully parameterized 2 × × universal perception , which consists of shared perceptual distri-butions modified by attentional and global scaling parameters (Soto et al., 2015). Ifuniversal perception is valid, GRTwIND solves both issues. In this paper, we show thatGRTwIND with universal perception and subject-specific failures of decisional separa-bility is mathematically, and thereby empirically, equivalent to a model with decisionalseparability and failure of universal perception. We then provide a formal proof of thefact that means and marginal variances are not, in general, simultaneously identifiablein 2 × Keywords:
General Recognition Theory, identifiability, testability, GRTwIND,decisional separability
1. Introduction
Recent work within the general recognition theory (GRT) indicates that failuresof decisional separability are not generally identifiable under common assumptions(Silbert and Thomas, 2013; Thomas and Silbert, 2014), and it has long been known thatthe latent perceptual means and marginal variances are not, in general, simultaneouslyidentifiable in GRT models (e.g., Wickens, 1992). A recently developed multilevelextension of GRT (GRT with Individual Di ff erences, or GRTwIND) has been pro ff eredas a solution to both of these problems (Soto et al., 2015; Soto and Ashby, 2015). In this Preprint submitted to Journal of Mathematical Psychology October 19, 2018 a r X i v : . [ s t a t . O T ] J u l heoretical note, we show that any GRTwIND model exhibiting failures of decisionalseparability or non-unit marginal variances is mathematically, and thereby empirically,equivalent to a model the exhibits neither trait. GRTwIND solves these two problemsonly conditionally , if the assumption of of universal perception is valid, but the validityof universal perception cannot be established within GRTwIND. The purpose of thisnote is, in no small part, to establish precisely, and mathematically, what the assumptionof universal perception entails.It is important at the outset to mention a subtle distinction relating to the notion ofidentifiability. Specifically, in this paper we will distinguish between identifiability and testability . For the present purposes, identifiability concerns the mapping between par-ticular parameter values and observable data given a particular model. A set of param-eters is identifiable if, given a particular model, distinct parameter values map uniquelyto corresponding data. On the other hand, testability concerns the relationship betweenthe assumptions underlying a model and the model’s empirical consequences. The un-derlying assumptions of a model are testable if relaxation of these assumptions leadsto distinct empirical predictions. Although both identifiability and testability concernthe assumptions and empirical consequences of a model, the two ideas are not coexten-sive. We show below that there are substantial testability problems in GRTwIND, andwe return to these issues periodically throughout the text as they relate to the materialat hand.We begin, in section 2, by briefly reviewing the structure of the 2 × n × m idenficationmodels, with n , m >
2; Ashby, 1988; Wickens and Olzak, 1989, 1992). In section3, we briefly recapitulate Silbert & Thomas’s (2013) proposition i , which describes(a subset of) the relationships between failures of decisional separability, perceptualseparability, and perceptual independence. We also recapitulate Soto et al.’s (2015)generalization of this proposition as it relates to models with multiple decision boundson each dimension.In section 4, we describe the GRTwIND model and discuss its relationship to theconcurrent ratings and n × m models. In section 5, we discuss the logic of testabil-ity with respect to Silbert & Thomas’s proposition i and the assumption of universalperception in GRTwIND. We then show, through a minor generalization of proposition i , that any GRTwIND model with subject-specific failures of decisional separability ismathematically and empirically equivalent to a model in which decisional separabil-ity holds and in which the assumption of universal perception does not. In section 6,we give a proof of a frequently stated, but, to the best of our knowledge, not formallyproven empirical equivalence between mean and marginal variance parameters in 2 × i in section 5 and the proofof mean-variance equivalence in section 6 together help delineate precisely what the That is, under the assumption that the the functional form of the model and the probabilistic assumptionsof the data and model parameters are true Our notion of testability is closely related to the notion of structural identifiability (e.g. Bellman andstrm, 1970; Eisenfeld, 1985). Henceforth, we will use n × m to refer exclusively to identification models with more than 2 levels oneach dimension. dimensional orthog-onality of GRT models must be fixed, as well. We argue, following Silbert and Thomas(2013), that this is typically best done by assuming decisional separability, though wealso discuss other possible approaches, noting that the scope of this assumption insingle-subject identification and concurrent ratings models is straightforward, whereasit is somewhat less so in multilevel models like GRTwIND and the models described bySilbert (2012, 2014). We conclude with a brief discussion of the relationship betweenuniversal perception and the concept of perceptual primacy.
2. General Recognition Theory
GRT is a two-stage model of perception and response selection (Ashby and Townsend,1986; Kadlec and Townsend, 1992; Thomas, 2001b; Silbert, 2014; Wickens, 1992).The first stage consists of noisy perception. The second stage consists of determin-istic response selection. Noisy perception is modeled with multivariate probabilitydistributions defined over an unobserved perceptual space. Response selection is mod-eled with decision bounds, i.e., curves that exhaustively partition the perceptual spaceinto response regions. Any given perceptual e ff ect is represented as a point in percep-tual space. The response to a perceptual e ff ect is determined by the response regionin which the perceptual e ff ect occurs. The probability of a particular response to aparticular stimulus is modeled as the multiple integral of the perceptual distributioncorresponding to the stimulus over an appropriate response region. × model The most common use of GRT is to analyze identification-confusion data in a 2 × A B = low frequency, low inten-sity; A B = low frequency, high intensity; A B = high frequency, low intensity; and A B = high frequency, high intensity.Figure 1 illustrates the equal likelihood contours and decision bounds of one possi-ble 2 × a i b j , with i , j ∈ { , } ; a i indicates the level on the x di-mension (e.g., low vs high frequency) and b j indicates the level on the y dimension Uppercase A i B j indicate the levels of the stimuli and corresponding perceptual distributions, while low-ercase a i b j indicate response levels. c x , partitions the x -axis, andthe horizontal bound, c y , partitions the y -axis. Together, they specify response regionscorresponding to the same factorial structure that defines the stimuli. Figure 1: 2 × This model illustrates the three dimensional interaction concepts defined in theGRT framework: perceptual independence (PI), perceptual separability (PS), and de-cisional separability (DS). With Gaussian perceptual distributions, PI is equivalent tozero correlation, and failure of PI is equivalent to non-zero correlation. The top twoperceptual distributions illustrate failure of PI, while the bottom two exhibit PI. PS isillustrated with respect to the y dimension. The perceptual distributions are perfectlyhorizontally aligned at each level of B j ; the marginal distributions of perceptual e ff ectson the y dimension do not vary as a function of the level on the x dimension. By way ofcontrast, PS fails with respect to the x dimension; the marginal perceptual distributionson this dimension vary across levels of the y dimension. Finally, because the decisionbounds are parallel to the coordinate axes, DS holds in this model. Decision boundsthat are not parallel to the coordinate axes would represent a failure of DS. Although the 2 × n × m model (with n , m >
2) were two of the first extensions of GRT, and both play an important role here.A concurrent ratings model is illustrated in Figure 2. The structure of this model isvery similar to the 2 × ff erence that theconcurrent ratings model has two or more decision bounds on each dimension.Concurrent rating models are used to analyze judgments given separately to eachcomponent (dimension) of the stimulus. Crucially for our purposes, the number ofresponse levels can be greater than those that define the stimuli. For example, exper-iment participants may give ratings on a k -point scale indicating, on each dimension,the degree to which a stimulus is judged to have been at a low or high value. The modelillustrated in Figure 2 could be used to analyze data in which subjects could respond,e.g., ‘low’, ’uncertain’, or ’high.’ Figure 2: A concurrent ratings model with two (parallel) decision bounds on each dimension.
We refer to a closely related extension of GRT as the n × m model. Like the 2 × n × m model is used to analyze identification data, but likethe concurrent ratings model, the n × m model has multiple decision bounds on each di-mension. The key di ff erence between the concurrent ratings and n × m model is that thelatter has perceptual distributions corresponding to each response region. For example, Strictly speaking, ‘concurrent’ refers to separate responses on each dimension, while ’ratings’ refers tomultiple response levels on each dimension. n × m model has been used to model data from participants’ identification of stimuliconsisting of the factorial combination of three levels on each of two dimensions (e.g.,Ashby and Lee, 1991; Thomas et al., 2015).The 2 × n × m , and concurrent ratings models were each originally designedto analyze a single subject’s data (Ashby and Lee, 1991; Thomas, 2001b,a; Wickensand Olzak, 1989). Although the concurrent ratings and n × m model are distinct, forthe present analyses we introduce the term multi-bound model , which we will use torefer to both types of model in order to distinguish them as a class distinct from thestandard 2 × × Two recent extensions of GRT have focused on the simultaneous analysis of multi-ple subjects’ data. One of these extensions is a Bayesian model in which each subject’sdata is fit to a standard 2 × mul-tilevel to distinguish them as a class distinct from models designed to analyze a singlesubject’s data.It is important to note that multi-bound and multilevel models are not mutually ex-clusive classes. Both the Bayesian multilevel model and GRTwIND could, in principle,be implemented as concurrent ratings or n × m models. As it happens, neither have beenso implemented thus far, so, in practice, no multi-bound GRT models are multilevel,and no multilevel GRT models are multi-bound.The distinction between multi-bound and multilevel models helps elucidate thescope of Silbert & Thomas’s proposition i and Soto et al.’s generalization thereof.Specifically, Silbert and Thomas (2013) show that DS, PS, and PI are not simulta-neously testable in single-subject 2 × i to show that this is also true of two-dimensional Gaussian GRT modelswith multiple bounds on each dimension if and only if the bounds on a given dimensionare parallel. Soto et al. infer that neither proposition i nor their generalization thereofapply to GRTwIND. We show below that this is incorrect.In the following section, we briefly recapitulate, for convenience, the proof of pro-postion i . We also recapitulate Soto et al.’s generalization of this proposition. Neither Silbert and Thomas (2013) nor Soto et al. (2015) distinguish between identifiability and testa-bility as we use the terms here, in both cases discussing these issues exclusively in terms of identifiability. We focus here on the non-testability of DS in 2 × . Recapitulation of Silbert and Thomas (2013), proposition i , and Soto et al.’sgeneralization thereof Silbert and Thomas (2013) showed that single-subject 2 × × × Mathematically, Silbert and Thomas (2013) showed that a model with angle φ be-tween c y and the x -axis and angle ω between c x and c y can be rotated and sheared byapplying the following two linear transformations : R = (cid:34) cos φ − sin φ sin φ cos φ (cid:35) (1) S = (cid:34) − ω (cid:35) (2)Figures 3, 4, and 5 illustrate how a model with linear failure of DS can be rotated(via R ) and sheared (via S ) to induce DS. Figure 3 illustrates a model exhibiting linearfailure of DS, wherein the horizontal decision bound c y deviates from the x axis by theangle φ , and the two decision bounds are separated by the angle ω .Application of the rotation R aligns c y with the x -axis, producing the model illus-trated in Figure 4. The angle ω between c x and c y is preserved by the rotation. Appli-cation of the shear transformation S preserves the alignment of c y with the x -axis andaligns c x with the y -axis, thereby inducing DS. Because these linear transformationsare invertible, the predicted response probabilities are preserved (Billingsley, 2012, pp.215-216).Silbert and Thomas (2013) provide a formal proof of their proposition i , whichstates that, with linear decision bounds and a single-subject 2 × / or shear transformations. reason to think that they would resolve this issue if implemented in GRTwIND. Silbert and Thomas (2013)show, via simulation, that the same basic issue exists in 2 × A special case in which PS may be induced by application of linear transformations (mean-shift inte-grality) is given in proposition ii of Silbert and Thomas (2013) and clarified in Thomas and Silbert (2014). Note that, without loss of generality, the location of the model is fixed by putting the intersection ofthe two decision bounds at the origin. The same rotation and shear transformations will induce DS in anon-centered model, e.g., one in which the A B perceptual distribution is fixed at the origin. igure 3: 2 × φ indicates the angle between the ‘horizontal’decision bound c y and the x -axis. ω indicates the angle between the two decision bounds c x and c y . igure 4: Rotated 2 × igure 5: Sheared 2 × i from Silbert and Thomas (2013) andprove that this result also holds for Gaussian GRT models with multiple, parallel deci-sion bounds on each dimension. More specifically, as stated above, Soto et al. showthat proposition i holds in two-dimensional Gaussian GRT models with more than onebound on each dimension if and only if the bounds on a given dimension are parallel. Based on this generalization, Soto et al. (2015) write that Silbert & Thomas’s propo-sition i “is not generally true in GRT-wIND or any other model with more than onebound per dimension. The non-identifiability of decisional separability arises in suchmodels only under very specific circumstances” (p. 108). They conclude, incorrectly,that failures of DS are, in general, testable in GRTwIND.Because GRTwIND is a multilevel 2 × n × m model do; using the terminologyintroduced above, GRTwIND is not a multi-bound model. Soto et al. are correct thatthe transformations at the heart of proposition i cannot induce DS simultaneously for allsubjects. However, proposition i applies in a subject-specific manner, such that subject-specific failures of DS in GRTwIND map one-to-one onto subject-specific rotation andshear transformations. Any GRTwIND model with universal perception and subject-specific failures of DS is mathematically and empirically equivalent to a transformedGRTwIND model with DS for all subjects and violation of universal perception.
4. The structure of GRTwIND
GRTwIND is a multilevel, 2 × Mathematically, in GRTwIND there is a shared group-level set of four bivariateGaussian perceptual distributions. Each individual subject’s modeled perceptual dis-tributions are modifications of the shared group-level distributions. In addition, eachindividual subject has a set of linear decision bounds, each specified by an interceptand a slope. To the best of our knowledge, it has not previously been noted that multi-bound models with non-parallelbounds on a given dimension produces uninterpretable response regions, making them incoherent models ofperception and response selection. Consider, for example, the model illustrated in Figure 2 if c y and c y were not parallel. Because non-parallel lines intersect, this would produce a region that is simultaneously above c y and below c y . Here ‘non-identifiability’ refers to non-testability, using the terminology established above. A i , B j has a mean vector andcovariance matrix: µ A i B j = (cid:34) µ x , A i B j µ y , A i B j (cid:35) (3) Σ A i B j = (cid:34) σ xx , A i B j σ xy , A i B j σ yx , A i B j σ yy , A i B j (cid:35) (4)As in the the standard 2 × , T to fix the location of the model, and, as in multi-bound models, the marginal variances in one distribution are set equal to one to fix thescale of the model. For individual subject k , the covariance matrix corresponding to stimulus A i , B j isgiven by the following equation, with κ k > < λ k < Σ k , A i B j = σ xx , AiBj κ k λ k σ xy , AiBj κ k √ λ k (1 − λ k ) σ yx , AiBj κ k √ λ k (1 − λ k ) σ yy , AiBj κ k (1 − λ k ) (5)Note that, because the (absolute and relative) scaling is applied only to marginalvariances, subject k ’s mean vector for stimulus A i B j is just the group-level mean vector: µ k , A i B j = µ A i B j (6)Note, too, that although Soto et al. (2015) state, in the quote above, that the covari-ance of each distribution is constant, the scaling and dimension-weighting parameters κ k and λ k ensure that this is not generally true. Rather, the assumption is that the cor-relation of each distribution is constant across subjects.More generally, it is clear that the assumption of universal perception allows forscaling of marginal variances, both with respect to the absolute scale of the space ( κ )and with respect to the relative importance of each dimension ( λ ), but it does not allowfor di ff erences in failures of PS or PI. Naturally enough, given that it is a constraint onperceptual representations, universal perception also allows for di ff erences with respectto failures of DS across subjects. × m, and concurrent ratings models In two-dimensional, Gaussian GRT models, the predicted probability of response a i b j to stimulus A k B l is given by the following equation, expressed with some abuse ofnotation in the interest of simplicity:Pr( a i b j | A k B l ) = ¨ R aibj N (2) (cid:16) µ A k B l , Σ A k B l (cid:17) d y d x (7) It is typical in the standard 2 × N (2) ( µ , Σ ) indicates a bivariate Gaussian (normal) probability density func-tion with mean vector µ and covariance matrix Σ , and the integration is taken overthe response region R a i b j .As discussed above, in a multi-bound model for a given subject’s data, a number ofthe response regions are determined both by decision bounds above and below (on the y -axis) and / or to the left and right (on the x axis) of the region. See, for example, theresponse regions at the intermediate levels a or b in Figure 2 above. Correspondingto this structure in the model, the data from a concurrent ratings or n × m identificationtask may contain responses at intermediate levels.By way of contrast, no response region in the 2 × × k , the task is identical to the standard 2 × i and Soto et al.’s general-ization of it. It follows directly from these results that failure of DS is not generallytestable in single-subject multi-bound models with parallel bounds on a given dimen-sion. The rotation and shear transformations described by Silbert and Thomas (2013)apply to the whole single-subject model. But it does not then follow from this fact thatfailures of DS are, in general, testable in GRTwIND. In the next section, we show thatthey are not.
5. Mathematical and empirical equivalence of GRTwIND with and without deci-sional separability
Before providing a formal demonstration of the fact that failures of DS are not, ingeneral, testable in GRTwIND models, we discuss some of the philosophical issuesunderlying identifiability and testability.As discussed above, Silbert and Thomas (2013) showed, in proposition i , that fail-ures of DS are not testable in 2 × × conditional on the assumption that DS holds .Now, consider the following logic: Suppose we assume that DS holds, and we fit a2 × + PS + PI that has been rejected, but we do notknow unconditionally which antecedents are false. If our assumption that DS holds isnot valid, then our conclusions regarding the failure of PS and PI are incorrect. The setof DS, PS, and PI together is not testable. In order to maintain consistenty with Silbert and Thomas (2013), we reserve φ to indicate the anglebetween c y and the x -axis. Hence, we use N to indicate the normal (Gaussian) probability density function. / or DS fail. What can we conclude? In thiscase, it is the joint hypothesis of universal perception + PS + PI + DS that has beenrejected, and, once again, we do not know unconditionally which antecedents are false.If it is universal perception, then GRTwIND provides no basis for concluding that anyof the GRT interaction constructs have failed. Because of this, logically, GRTwINDdoes not provide a general solution to the (identifiability and testability) problems dis-cussed by Silbert and Thomas (2013). One can cover exactly the same data space withGRTwIND or with a transformed version of GRTwIND in which DS holds and failuresof PS are allowed to vary across individuals.In the following two sections, we prove that universal perception is not testable inGRTwIND by virtue of the fact that proposition i implies that any GRTwIND modelwith subject-specific failures of DS maps one-to-one onto a model with subject-specificrotation and shear transformations in which DS holds across the board. This mathe-matical equivalence delineates, in part, what the assumption of universal perceptionconsists of, and shows that a general solution to the identifiability and testability is-sues in question will have to be non-mathematical and not dependent on the 2 × GRTwIND as a whole is, like any other GRT model, invariant to a ffi ne transfor-mations; the modeled perceptual and decisional space is not fixed with respect to anyabsolute frame of reference. So, for example, rotation and / or shear transformations of afull GRTwIND model (i.e., all shared and subject-specific parameters) would preservethe full set of predicted response probabilities.Soto et al. (2015) argue correctly that DS cannot, in general, be induced for allsubjects simultaneously in a GRTwIND model by the application of global rotationand / or shear transformations. Their Figure 2 illustrates this fact. The argument is that,although rotation and shear transformations applied to the full model can align sub-ject m ’s decision bounds with the coordinate axes, as long as other subjects’ decisionbounds are not parallel to subject m ’s bounds, these transformation will not also alignthe other subjects’ bounds with the coordinate axes.However, applying single rotation and shear transformations to the full GRTwINDmodel is not the only option at our disposal, nor is it a direct analog to rotation or sheartransformations of single-subject 2 × each subject m ’s decision bound slopes define subject-specific angles φ m and ω m (see Figure3), which define subject-specific rotation and shear matrices R m and S m . This im-plies that universal perception and failure of decisional separability are not testable inGRTwIND. That is, Silbert & Thomas’s proposition i applies directly to any GRTwINDmodel with respect to each subject’s decision bounds.14he rotated and sheared mean vector for stimulus A i B j for subject m is given by: ν m , A i B j = S m R m µ A i B j (8) = S m (cid:34) µ x , A i B j cos φ m − µ y , A i B j sin φ m µ x , A i B j sin φ m + µ y , A i B j cos φ m (cid:35) = (cid:34) µ x , A i B j cos φ m − µ y , A i B j sin φ m − µ x , A i B j cos φ m tan ω m + µ y , A i B j sin φ m tan ω m µ x , A i B j sin φ m + µ y , A i B j cos φ m (cid:35) And the rotated and sheared covariance matrix for stimulus A i B j for subject m isgiven by: Ψ m , A i B j = S m R m Σ m , A i B j R Tm S Tm (9) = S m Θ m , A i B j S Tm Here, Σ m , A i B j is subject m ’s scaled covariance matrix, defined in equation 5 above,and R m and S m are subject m ’s rotation and shear matrices, respectively.Keeping in mind that (cid:16) Θ m , A i B j (cid:17) = (cid:16) Θ m , A i B j (cid:17) (i.e., that Θ m , A i B j is symmetric), theelements of Θ m , A i B j are: (cid:16) Θ m , A i B j (cid:17) = σ xx , A i B j κ m λ m cos φ m − σ xy , A i B j κ m √ λ m (1 − λ m ) sin φ m cos φ m + σ yy , A i B j κ m (1 − λ m ) sin φ m (10) (cid:16) Θ m , A i B j (cid:17) = (cid:32) σ xx , A i B j κ m λ m − σ yy , A i B j κ m (1 − λ m ) (cid:33) cos φ m sin φ m + σ xy , A i B j κ m √ λ m (1 − λ m ) (cid:16) cos φ m − sin φ m (cid:17) (11) (cid:16) Θ m , A i B j (cid:17) = σ xx , A i B j κ m λ m sin φ m + σ xy , A i B j κ m √ λ m (1 − λ m ) sin φ m cos φ m + σ yy , A i B j κ m (1 − λ m ) cos φ m (12)And keeping in mind that (cid:16) Ψ m , A i B j (cid:17) = (cid:16) Ψ m , A i B j (cid:17) (i.e., that Ψ m , A i B j is symmetric),the elements of Ψ m , A i B j are: (cid:16) Ψ m , A i B j (cid:17) = (cid:16) Θ m , A i B j (cid:17) − (cid:16) Θ m , A i B j (cid:17) + tan ω m tan ω m + (cid:16) Θ m , A i B j (cid:17) tan ω m (13) (cid:16) Ψ m , A i B j (cid:17) = (cid:16) Θ m , A i B j (cid:17) − (cid:16) Θ m , A i B j (cid:17) tan ω m (14) (cid:16) Ψ m , A i B j (cid:17) = (cid:16) Θ m , A i B j (cid:17) (15)The formulas given in equations 8-15 are fairly cumbersome, but they are the resultof straightforward linear algebra operations. As noted above, subject m ’s decisionbound slopes are mathematically equivalent to the angles φ m and ω m , which in turndetermine R m and S m , so the rotated and sheared model has the same number of freeparameters as the specification of GRTwIND with non-zero decision bound slopes.Indeed, the rotated and sheared model is a straightforward reparameterization of the15RTwIND model, not a more general model restricted to mimic a GRTwIND model.Application of R m and S m to subject m ’s parameters merely induces DS and transformsthe shared mean and covariance parameters in a subject-specific manner.These invertible, linear transformations preserve the predicted response probabili-ties of the model, so the GRTwIND model transformed by subject-specific rotation andshear transformations is also empirically equivalent to the original model exhibitinglinear failures of DS.Soto et al. (2015, p. 93) state that “if violations of decisional separability are foundand individual decision bounds have slightly di ff erent slopes, then it is not possible tofind an equivalent model (i.e., producing the same response probabilities) in which de-cisional separability holds for all participants, unless the assumption of universal per-ception is violated [emphasis added].” Expressed slightly di ff erently, subject-specificDS and shared PS and PI are only testable conditional on the validity of the assumptionof universal perception. In general, the conjunction of universal perception and DSis not testable in GRTwIND, though, since relaxation of the assumption of universalperception renders the model’s perceptual and decisional parameteres non-identifiable.For every GRTwIND model, there is a mathematically and empirically equivalentmodel that relies on very di ff erent assumptions about the nature of the underlying per-ceptual and decisional interactions. We return to this issue again below.
6. Identifiability of means and marginal variances
The fact that means and marginal variances are not simultaneously identifiable inthe 2 × ff erence between the means (i.e., d (cid:48) ) and a response bias pa-rameter can both be estimated (Green and Swets, 1966). To the best of our knowledge,however, no formal proof of this fact has been published. We provide such a proof here,after which we discuss how this result further delineates the assumption of universalperception in GRTwIND. 16 .1. Proof of mean-variance equivalence in the standard × Gaussian GRT model
Let µ and Σ be the mean vector and covariance matrix of a bivarite Guassian den-sity, and let c be a vector containing the response criteria on each dimension: µ = (cid:34) µ x µ y (cid:35) (16) Σ = (cid:34) σ xx σ xy σ xy σ yy (cid:35) (17) c = (cid:34) c x c y (cid:35) (18)If we apply the a ffi ne transformation T + ∆ , defined below, the covariance matrixis transformed into a correlation matrix and the means are shifted with respect to theresponse criteria in order to preserve the distances between the means and responsecriteria in units of standard deviation. T + ∆ = √ σ xx √ σ yy + c x − c x √ σ xx c y − c y √ σ yy (19)Application of this transformation produces a new covariance matrix R = T Σ T T and new mean vector η = T µ + ∆ . The transformed covariance matrix is the correlationmatrix: T Σ T T = √ σ xx √ σ yy (cid:34) σ xx σ xy σ xy σ yy (cid:35) √ σ xx √ σ yy (20) = σ xx √ σ xx σ xx σ xy √ σ xx σ yy σ xy √ σ xx σ yy σ yy √ σ yy σ yy = (cid:34) ρ xy ρ xy (cid:35) And the transformed mean vector is the vector of response criteria added to the If DS holds, each decision bound is equivalent to a simple response criterion. η = T µ + ∆ (21) = √ σ xx √ σ yy (cid:34) µ x µ y (cid:35) + c x − c x √ σ xx c y − c y √ σ yy = µ x √ σ xx µ y √ σ yy + c x − c x √ σ xx c y − c y √ σ yy = c x + µ x − c x √ σ xx c y + µ y − c y √ σ yy Before applying the transformation, the signed distance between the means and theresponse criteria are µ x − c x √ σ xx and µ y − c y √ σ yy . After applying the transformation, the means arethese values added to the response criteria. (The transformation applied to the responsecriteria produces no shift; substitute c x for µ x and and c y for µ y in equation 21 to seethis.) Hence, the signed distances between the transformed means and the responsecriteria are: η x − c x = (cid:32) c x + µ x − c x √ σ xx (cid:33) − c x = µ x − c x √ σ xx (22) η y − c y = (cid:32) c y + µ y − c y √ σ yy (cid:33) − c y = µ y − c y √ σ yy (23)This guarantees that the integrals of the marginal densities are equivalent pre- andpost-transformation. More generally, because this transformation is invertible, it pre-serves the model’s predicted response probabilities (Billingsley, 2012, pp. 215-216).Therefore, for a pair of response criteria, there is a one-to-one mapping between em-pirically equivalent bivariate Gaussian GRT perceptual distributions, one of which mayhave arbitrary marginal variances and the other of which has unit marginal variancesand suitably shifted means.As discussed above, the non-identifiability of means and marginal variances in the2 × n × m identification tasks have more degreesof freedom than there are unknown variables (free parameters) in the correspondingmodels.More specifically, suppose there are n > m > nm −
1) degrees of freedom, while the model will have 16 parameters governing the perceptual distributions, and n + m − n = m =
3) has 32 degrees of freedom, while the corresponding model has 20 freeparameters. The degrees of freedom in the data grow multiplicatively with n and m ,while the number of free parameters in the model grows additively, so any more com-plex concurrent ratings data and model will have more degrees of freedom than freeparameters, respectively.Of course, a simple inequality between the degrees of freedom in the data and thenumber of free parameters in the model does not guarantee identifiability. We can seethat such models are identifiable in this case by considering that the concurrent ratingsdata and model may be expressed as a system of 4( nm −
1) equations with 14 + n + m unknowns of the following form:Pr( a i , b j | A k , B l ) = c yi ˆ c yi − c xj ˆ c xj − N (2) (cid:16) µ A k B l , Σ A k B l (cid:17) d y d x (24)With i , j , k , l ∈ { , } , c x = c y = −∞ , and c x m = c y n = ∞ . Crucially, everyfree parameter appears in more than one equation, since each parameter plays a rolein specifying more than one predicted response probability. For example, each of theestimated decision bound partially specifies predicted probabilities on either side ofthe bound for every perceptual distribution, and each perceptual distribution parameterpartially specifies the predicted probabilities for every response to the correspondingstimulus.The n × m identification task and model exhibits a similar relationship, with thesimplest data set having 72 degrees of freedom, while the model has 57 perceptualdistribution parameters and n + m − In general, the n × m identification data and model can be expressed as a system of nm ( nm −
1) equationswith 7( nm − + n + m − In GRTwIND, there is a similar, but not identical, relationship between the data andthe model. With N subjects producing data in the 2 × There are nm − nm − Three mean vectors with two free parameters each, one correlation parameter in the distribution withfixed marginal variances, three (co)variance parameters in each of the other three distributions The confusion matrix is 9 ×
9, so it has 9 × one correlation parameter in a distribution with fixed mean and marginal variances, and two mean andfive (co)variance parameters in each of the other eight distributions There may be additional decision bound parameters if failures of DS are modeled with piecewise linearbounds, as in, e.g., Ashby and Lee (1991), though see Silbert and Thomas (2013) for a discussion of someimportant ambiguities with the specification of piecewise failures of DS N degrees of freedom, while the model has 16 shared perceptual distributionparameters and 6 N scaling, dimension weighting, and decision bound parameters. Hence, there will be a system of 12 N equations with 16 + N unknowns, again takingthe same general form as the equation given above.The di ff erences between GRTwIND and multi-bound models are twofold. First,as described above, the way in which the parameters partially specify multiple pre-dicted response probabilities di ff er between the two types of model. A single-subjectmulti-bound model is designed to analyze a single subject’s data and predict interme-diate (and extreme) response levels therein, whereas GRTwIND is designed to analyzemultiple subjects’ data and cannot, by defintion, predict intermediate response levels.Second, whereas an appropriately specified single-subject multi-bound model can befit to a single subject’s data, if the number of subjects N ≤
2, the number of free pa-rameters in a GRTwIND model exceeds the degrees of freedom in the data. Hence,GRTwIND is over-parameterized with data from fewer than three subjects, as noted bySoto et al. (2015).It is also worth noting that, because GRTwIND is not a multi-bound model (i.e.,because it was designed to analyze multiple subjects’ 2 × R m and S m . Expressions for agiven subject’s mean, variance, and correlation parameters can be found by appropriatesubstitutions of terms from equations 8-15 into equations 20 and 21.We can conclude from this that, in order for the assumption of universal perceptionto enable the simultaneous identification of means and marginal variances, it must alsodisallow the subject- and stimulus-specific scaling of marginal variances and meansdescribed in equations 20 and 21. Given the mathematical and empirical equivalenceof the covariance matrices and mean vectors on either side of equations 20 and 21,it seems once again impossible that a purely mathematical justification can be foundfor disallowing these transformations while allowing the variance scaling described bySoto et al. (2015).
7. Conclusion
Silbert and Thomas (2013) showed, in their proposition i , that simultaneous DS,PS, and PI are not jointly testable in 2 × Each confusion matrix in the 2 × × One correlation parameter for the distribution with fixed location and scale, and two mean and three(co)variance parameters in each of the other three distributions One scaling, one dimension weighting, two decision bound intercepts and two decision bound slopesper subject
20t al. (2015) showed that Silbert & Thomas’s proposition i holds for models with multi-ple decision bounds on each dimension if and only if the bounds on a given dimensionare parallel. In addition, it has been known for more than 20 years that the means andmarginal variances in 2 × n × m identification models, which we here refer to asmulti-bound models (Ashby and Lee, 1991; Ashby, 1988; Wickens, 1992). A recentmultilevel extension of GRT called GRTwIND was developed in an attempt to solvethese problems in the 2 × λ m and κ m , m ∈ { , , . . . , N } , which modify the perceptual covariancematrices, and subject-specific decision bounds, each of which is specified by interceptand slope parameters.In section 5.2, we showed that each subject’s decision bound slopes map one-to-oneonto subject-specific angles φ m and ω m , which in turn define subject-specific rotationand shear matrices R m and S m (see equations 8-15). These one-to-one mappings provethat GRTwIND with subject-specific failures of decisional separability is mathemati-cally, and thereby empirically, equivalent to a model in which decisional separabilityholds for all subjects and in which universal perception is violated. Finally, we showedthat means and marginal variances are not, in general, simultaneously identifiable in2 × disallow the subject-specific rotationand shear transformations described in equations 8-15 and the subject- and stimulus-specific mean and marginal variance scaling transformations described in equations 20and 21.These results establish the complete mathematical and empirical equivalence ofGRTwIND and a model with subject-specific rotation and shear transformations. Hence,the pattern of allowed and disallowed transformations described above can only be jus-tified by non-mathematical means or by empirical means other than the 2 × .2. Dimensional orthogonality and perceptual primacy We conclude by proposing that the full suite of results concerning the (lack of)identifiability and testability of DS, PS, and PI in GRT models, and of universal per-ception in GRTwIND, points toward an important, and thus far incompletely addressedissue at the heart of the GRT framework, namely the orthogonality of the modeledperceptual dimensions. From the initial development of GRT, it was recognized thatorthogonality of perceptual dimensions is intimately intertwined with perceptual anddecisional dimensional interactions (Ashby and Townsend, 1986). Indeed, Ashby andTownsend (1986) discuss the di ffi culties related to testing dimensional orthogonalityin some detail. Nonetheless, the full import of this assumption seems only now, threedecades later, to be fully understood.As discussed above, in order for a GRT model’s parameters to be identifiable, thelocation and scale of the model must be fixed, and this is typically done by setting onemean vector equal to the origin and by setting one perceptual distribution’s marginalvariances equal to one. The recent mathematical developments in the GRT framework,including those discussed above, indicate that we must also fix the orthogonality of theperceptual dimensions.Without describing it explicitly in these terms, Silbert and Thomas (2013) recom-mend fixing the orthogonality of the perceptual dimensions by assuming that decisionalseparability holds in a single-subject 2 × µ x , A B = µ x , A B and µ y , A B = µ y , A B ). However, asnoted by Silbert and Thomas (2013), decisional separability can always be induced inthe 2 × × × ff ected by theneed to fix the orthogonality of the dimensions even more strongly than is GRTwIND.Silbert (2012, 2014) used a multilevel model in which each subject’s data is modeled22y a fully parameterized 2 × orthogonality and more with dimensional primacy .Establishing the perceptual primacy of a particular set of dimensions demands evi-dence that is not simple to come by. For example, Melara and Marks (1990) argue thatpatterns of change in the magnitude of Garner interference across levels of physical di-mension orientations provide evidence of perceptual primacy (or lack thereof), but theyanalyzed the perception of well-defined (orthogonal) physical dimensions (acousticfrequency and intensity). By way of contrast, assuming perceptual primacy, Soto et al.(2015) analyzed dimensions with no straightforward physical definitions (facial iden-tity and neutral vs sad emotional expressions), and Soto and Ashby (2015) analyzednovel dimensions based on morphed faces, stating that “there are no psychologically-meaningful directions in a space constructed this way” (p. 110). Similarly, Silbert(2012, 2014) used GRT to probe interactions between dimensions defined with respectto abstract linguistic categories.Evidence for perceptual primacy with respect to novel dimensions may be partic-ularly di ffi cult to find, as unsupervised learning seems to play a role in the creation ofad-hoc perceptual dimensions (Jones and Goldstone, 2013). When considering the pri-macy of particular dimensions and assumptions of universal perception, it is also worthkeeping in mind that holistic vs analytic cognition may vary across cultures (Nisbettet al., 2001).To the extent that dimensional primacy and / or universal perception requires sharedperceptual correlations across subjects, one could argue against the rotation and sheartransformations described above. However, it is not clear that the shared perceptualcorrelations of Soto et al.’s universal perception is a valid assumption in all cases. Forexample, multilevel 2 × ff erences in correlations across subjects have been reported in 2 × × . ReferencesReferences Ashby, F. G., 1988. Estimating the parameters of multidimensional signal detectiontheory from simultaneous ratings on separate stimulus components. Perception &Psychophysics 44 (3), 195–204.Ashby, F. G., Lee, W. W., 1991. Predicting similarity and categorization from identifi-cation. Journal of Experimental Psychology: General 120 (2), 150–172.Ashby, F. G., Townsend, J. T., 1986. Varieties of perceptual independence. Psycholog-ical Review 93 (2), 154–179.Bellman, R., strm, K. J., 1970. On structural identifiability. Mathematical Biosciences7 (3), 329–339.Billingsley, P., Jan. 2012. Probability and Measure. John Wiley & Sons.Eisenfeld, J., 1985. Remarks on Bellman’s structural identifiability. Mathematical Bio-sciences 77 (1), 229–243.Green, D. M., Swets, J. A., 1966. Signal detection theory and psychophysics. RobertE. Krieger.Jones, M., Goldstone, R. L., 2013. The structure of integral dimensions: Contrast-ing topological and Cartesian representations. Journal of Experimental Psychology:Human Perception and Performance 39 (1), 111–132.Kadlec, H., Townsend, J. T., 1992. Implications of marginal and conditional detectionparameters for the separabilities and independence of perceptual dimensions. Journalof Mathematical Psychology 36 (3), 325–374.Melara, R. D., Marks, L. E., 1990. Perceptual primacy of dimensions: Support fora model of dimensional interaction. Journal of Experimental Psychology: HumanPerception and Performance 16 (2), 398.Nisbett, R. E., Peng, K., Choi, I., Norenzayan, A., 2001. Culture and systems ofthought: Holistic versus analytic cognition. Psychological Review 108 (2), 291–310.Silbert, N. H., 2012. Syllable structure and integration of voicing and manner of artic-ulation information in labial consonant identification. The Journal of the AcousticalSociety of America 131 (5), 4076–4086.Silbert, N. H., 2014. Perception of voicing and place of articulation in labial and alve-olar English stop consonants. Laboratory Phonology 5 (2), 289–335.Silbert, N. H., Thomas, R. D., 2013. Decisional separability, model identification, andstatistical inference in the general recognition theory framework. Psychonomic Bul-letin & Review 20, 1–20. 24oto, F. A., Ashby, F. G., 2015. Categorization training increases the perceptual sepa-rability of novel dimensions. Cognition 139, 105–129.Soto, F. A., Vucovich, L., Musgrave, R., Ashby, F. G., 2015. General recognition theorywith individual di ffff