Bruce D. Spencer

Benefit-Cost Analysis of Data Used to Allocate Funds

0 Introduction.- 1 Loss Function and Benefit Measurement.- 1.0 Introduction.- 1.1 Utility and Social Welfare.- 1.2 Equity.- 1.3 A Simple Loss Function.- Errors in Allocation.- 1.4 Estimating the Parameters of the Loss Function L(f?,??) for GRS.- 1.5 Fisher-Consistency and Other Properties of the Loss Function L(f?,??).- 1.6 More General Loss Functions: LH(f?,??) , Lw(f?,??).- 1.7 Exponential Loss Functions.- 2 The Delta Method.- 2.0 Outline.- 2.1 Notation.- 2.2 Description of the Delta Method.- 2.3 Applicability of the Delta Method to GRS.- 2.4 Application of the Delta Method to Calculating Expected Loss.- 3 Data Used in General Revenue Sharing.- 3.1 Introduction.- 3.2 Population (state level).- 3.3 Urbanized Population.- 3.4 Population (substate level).- 3.5 Total Money Income, Per Capita Income (state level).- 3.6 Per Capita Income (substate level).- 3.7 Personal Income.- 3.8 Net State and Local Taxes.- 3.9 State Individual Income Taxes.- 3.10 Federal Individual Income Tax Liabilities.- 3.11 Income Tax Amount (state level).- 3.12 Adjusted Taxes (substate level).- 3.13 Intergovernmental Transfers (substate level).- 3.14 Interrelationships Among State- level Data Elements.- 3.15 Interrelationships Among Substate Data Elements.- 3.16 Outline-Symbol Glossary.- 4 Interstate Allocation in GRS 4.1 Introduction 151.- 4.2 Notation.- 4.3 Conclusions.- 4.4 Assumptions and Degrees of Approximation.- 4.5 Derivations and Proofs.- 5 Intrastate Allocations in GRS.- 5.1 Overview.- 5.2 Determination of Substate Allocations.- 5.3 Notation.- 5.4 Assumptions and Degrees of Approximations.- 5.5 Conclusions.- 6 Computations and Analyses.- 6.1 Introduction.- 6.2 Errors in Allocation to States.- 6.3 Substate Errors in Allocation.- 6.4 Benefit Analysis.- 6.5 Summary of Findings.- 7 Policy Perspectives and Recommendations.- 7.0 Introduction and Summary.- 7.1 Redundancy cf Allocation Formulas.- 7.2 Data Burdens Imposed by Legislation.- 7.3 Minimizing Large Errors in Allocation.- 7.4 Uniform Biases, Non-uniform Variances.- 7.5 Adjusting for Undercoverage or Other Suspected Biases.- 7.6 Improving the Coverage of the Decennial Census.- 7.7 State Errors vs. Substate Errors.- 7.8 Allocation Formulas as Approximations.- 7.9 Other Uses of Data.- Appendix A Tables of Biases in Data.- Appendix B Determination of General Revenue Sharing Allocations.- Technical Appendix.

Optimal data quality

Abstract Commonly accepted hypotheses guide practical determination of needed data quality; for example, as the probability that a decision uses data increases, the needed data quality increases, and the more rudimentary the uses of the data, the less data quality is needed. These hypotheses are formally defined and analyzed in some decision-theoretic models. Conditions under which the hypotheses hold and fail are examined. Particular attention is given to determining needed data quality when the users of the data behave nonoptimally.

ESTIMATING NETWORK DEGREE DISTRIBUTIONS UNDER SAMPLING: AN INVERSE PROBLEM, WITH APPLICATIONS TO MONITORING SOCIAL MEDIA NETWORKS

Networks are a popular tool for representing elements in a system and their interconnectedness. Many observed networks can be viewed as only samples of some true underlying network. Such is frequently the case, for example, in the monitoring and study of massive, online social networks. We study the problem of how to estimate the degree distribution - an object of fundamental interest - of a true underlying network from its sampled network. In particular, we show that this problem can be formulated as an inverse problem. Playing a key role in this formulation is a matrix relating the expectation of our sampled degree distribution to the true underlying degree distribution. Under many network sampling designs, this matrix can be defined entirely in terms of the design and is found to be ill-conditioned. As a result, our inverse problem frequently is ill-posed. Accordingly, we offer a constrained, penalized weighted least-squares approach to solving this problem. A Monte Carlo variant of Steins unbiased risk estimation (SURE) is used to select the penalization parameter. We explore the behavior of our resulting estimator of network degree distribution in simulation, using a variety of combinations of network models and sampling regimes. In addition, we demonstrate the ability of our method to accurately reconstruct the degree distributions of various sub-communities within online social networks corresponding to Friendster, Orkut and LiveJournal. Overall, our results show that the true degree distributions from both homogeneous and inhomogeneous networks can be recovered with substantially greater accuracy than reflected in the empirical degree distribution resulting from the original sampling.

Metrics for Assessing Earthquake-Hazard Map Performance

Recent large earthquakes that caused great damage in areas predicted to be relatively safe, illustrate the importance of criteria that assess how well earthquake hazard maps used to develop codes for earthquake‐resistant construction are actually performing. At present, there is no agreed‐upon way of assessing how well a map performed and thus determining whether one map performed better than another. The fractional site exceedance metric implicit in current maps, that during the chosen time interval the predicted ground motion will be exceeded only at a specific fraction of the sites, is useful but permits maps to be nominally successful even if they significantly underpredict or overpredict shaking, or permits them to be nominally unsuccessful but do well in terms of predicting shaking. We explore some possible metrics that better measure the effects of overprediction and underprediction and can be weighted to reflect the two differently and to reflect differences in populations and property at risk. Although no single metric alone fully characterizes map behavior, using several metrics can provide useful insight for comparing and improving hazard maps. For example, both probabilistic and deterministic hazard maps for Italy dramatically overpredict the recorded shaking in a 2200‐yr‐long historical intensity catalog, illustrating problems in the data (most likely), models, or both.

Test Scores as Social Statistics: Comparing Distributions

If test scores are considered ordinal but not cardinal attributes, the average test score may not be a meaningful way to summarize the scores of a group of individuals. Similarly, correlation coefficients may not form meaningful summary measures of association between test scores. Using the theory of stochastic ordering and quadrant dependence, new techniques that do not depend on any particular test scale are developed for reporting, interpreting, and comparing distributions of test scores. The use of a relevant test scale is considered also, as well as measures of distance between distributions.

When do latent class models overstate accuracy for diagnostic and other classifiers in the absence of a gold standard

Latent class models are increasingly used to assess the accuracy of medical diagnostic tests and other classifications when no gold standard is available and the true state is unknown. When the latent class is treated as the true class, the latent class models provide measures of components of accuracy including specificity and sensitivity and their complements, type I and type II error rates. The error rates according to the latent class model differ from the true error rates, however, and empirical comparisons with a gold standard suggest the true error rates often are larger. We investigate conditions under which the true type I and type II error rates are larger than those provided by the latent class models. Results from Uebersax (1988, Psychological Bulletin 104, 405-416) are extended to accommodate random effects and covariates affecting the responses. The results are important for interpreting the results of latent class analyses. An error decomposition is presented that incorporates an error component from invalidity of the latent class model.

Statistical aspects of equitable apportionment

Abstract Two problems that arise in apportioning the U.S. House of Representatives are: (a) fractional numbers of representatives cannot be allocated, so states receive different per capita representation, and (b) the state population sizes are known only with error. Both problems are addressed in a unified way with decision theory. Although the method currently in use, equal proportions, has poor properties when the populations are assumed perfectly known, it performs surprisingly well in the presence of modest errors in the data. The converse is true for the quota method. Previously developed qualitative notions of bias in apportionment methods are extended to provide a quantitative definition of bias. The new definition accounts both for bias in the apportionment method and for bias arising from imperfect population measurements. Illustrative estimates of the bias against states with large black populations are developed.

Feasibility of benefit-cost analysis of data programs

Explicit consideration of benefits and costs is needed to determine how much to spendfor public data. It is argued that benefit-cost analysis is also feasible. In support of feasibility, a benefit-cost analysis of the 1970 census is discussed in some detail, with emphasis on allocative uses of the data. Other precedents are discussed for evaluating the production, analysis, and dissemination of forecasts and projections, data used for determining allocations, and physical monitoring data. The greatest potential value of benefit-cost analysis of data may be for social monitoring data, such as social and economic indicators.

Sampling Probabilities for Aggregations, With Application to NELS:88 and Other Educational Longitudinal Surveys

This article considers surveys where one can observe, after the sample is selected, that each member of the sample belongs to one or more aggregations. The population of aggregations is of interest, and we consider the probability that a given aggregation contains at least one sample member. An expression for the probability is derived as a function of population parameters, many of which can be known only if additional, costly data collection is undertaken. A variety of model-based estimators of those parameters is discussed, and their relative advantages and disadvantages are noted. The estimators are applied to a particular case, the National Educational Longitudinal Study of 1988 (NELS:88), in which the aggregations are tenth-grade schools. Evaluations of the estimators are presented. In NELS:88, a sample of eighth-grade schools and eighth-grade students in those schools was surveyed and 2 years later, when most of the students were in tenth grade, the students were resurveyed. If the tenth-grade school sample (i.e., the set of tenth-grade schools enrolling one or more sampled students) is a probabiliy sample, then we can make inferences to the population of tenth-grade schools. A requirement for a sample to be a probability sample is that the selection probabilities be nonzero for all units and be known for the selected units. This article discusses how those probabilities can be estimated from complete and incomplete data. Various estimators based on incomplete data are derived and empirically evaluated with data from a special test sample of schools and data from the Houston Independent School District.

Sensitivity of benefit-cost analysis of data programs to monotone misspecification

Abstract The sensitivity of a benefit-cost analysis of a data program to incorrect specifications of the cost function C and benefit function B is an important consideration for producers and consumers of the analyses. We consider the effects of misspecification of B or C on the optimal data expenditure and optimal data quality. Geometric and analytical results show when the analysis will be sensitive and when order of magnitude estimates for costs and benefits will suffice in practice. A benefit-cost analysis of the 1970 census illustrates these results. We also show why misspecification of benefits typically has a larger effect than comparable misspecification of costs. If the benefits and the costs are both large but their difference small, the optimal expenditure for data (or optimal data quality) will be highly sensitive to changes in specifications of costs, benefits, data use, etc. In other cases, the sensitivity of the prescribed optimal expenditure to scalemisspecification of costs or benefits depends on the sharpness of the bend of the graph of B versus C near the optimum. Although the prescribed optimal expenditure is the same when B is misspecified as when C is comparably misspecified, the levels of data quality attained can differ. When B is misspecified, the data quality attained decreases if benefits are understated. When C is misspecified, the data quality attained usually decreases if costs are overstated, but for some unusual decision problems it increases and in rare, ‘self-correcting’ problems, it remains unchanged.