Insiders and Outsiders in Research on Machine Learning and Society
IInsiders and Outsiders in Research on Machine Learning and Society
Yu Tao and Kush R. Varshney Stevens Institute of Technology IBM Research – T. J. Watson Research [email protected], [email protected]
Abstract
A subset of machine learning research intersects with societalissues, including fairness, accountability and transparency, aswell as the use of machine learning for social good. In thiswork, we analyze the scholars contributing to this research atthe intersection of machine learning and society through thelens of the sociology of science. By analyzing the authorshipof all machine learning papers posted to arXiv, we show thatcompared to researchers from overrepresented backgrounds(defined by gender and race/ethnicity), researchers from un-derrepresented backgrounds are more likely to conduct re-search at this intersection than other kinds of machine learn-ing research. This state of affairs leads to contention betweentwo perspectives on insiders and outsiders in the scientific en-terprise: outsiders being those outside the group being stud-ied, and outsiders being those who have not participated asresearchers in an area historically. This contention manifestsas an epistemic question on the validity of knowledge derivedfrom lived experience in machine learning research, and pre-dicts boundary work that we see in a real-world example.
Research on the theory and methods of machine learning hasled to the ability of technological systems to grow by leapsand bounds in the last decade. With this increasing com-petence, machine learning is increasingly being employedin real-world sociotechnical contexts of high consequence.People and machines are now truly starting to become part-ners in various aspects of life, livelihood, and liberty.This intersection of machine learning with society has fu-eled a small segment of research effort devoted to it. Twosuch efforts include research on (1) fairness, accountabilityand transparency of machine learning (FAccT), and (2) ar-tificial intelligence (AI) for social good. The first of thesefocuses on the imperative ‘do no harm’ or nonmaleficence,with a special focus on preventing harms to marginalizedpeople and groups caused or exacerbated by the use of ma-chine learning in representation and decision making. Thesecond focuses on using machine learning technologies asan instrument of beneficence to uplift vulnerable people andgroups out of poverty, hunger, ill health, and other societalinequities.
Copyright © 2021, by the authors. All rights reserved.
In this paper, we focus on who is conducting this researchat the intersection of machine learning and society throughthe lens of the sociology of science. The theoretical foun-dation for our investigation is the concept of insiders and outsiders in the research enterprise (Merton 1972). In thesocial sciences and humanities, researchers are consideredinsiders if they are members of the community being studied(and thus have lived experience of that community) and out-siders otherwise. (Formal and natural sciences typically donot study communities of people, but the societal aspects ofresearch on machine learning and society does.) A differentperspective says that members of groups that have been his-torically underrepresented in a field of study are outsiders.These two notions, illustrated in Figure 1 and Figure 2 maybe at odds. Researchers being insiders from one perspec-tive and at the same time outsiders from the other perspec-tive raises contention in the production of knowledge, in-cluding in the epistemic validity of knowledge arising fromlived experience. The social construction of whether scien-tific knowledge arising from lived experience is valid or in-valid is an instance of boundary work (Gieryn 1983).To analyze researchers in machine learning and societyfrom the theory of insiders and outsiders, first we empiri-cally show that machine learning researchers from underrep-resented backgrounds, compared to researchers from over-represented backgrounds, are more likely to study the so-cietal aspects of machine learning than they are to studyaspects of machine learning that are more divorced fromsociety. Recognizing the inadequacy of binary gender cat-egories, we nevertheless take binary gender as one sensi-tive attribute. (Women are underrepresented and men areoverrepresented.) Recognizing the inadequacy of the socialconstructs of coarse race and ethnicity categories, we alsotake race/ethnicity as another sensitive attribute. (Blacks andHispanics are underrepresented, and whites and Asians areoverrepresented.) We also examine the intersection of gen-der with race/ethnicity.Next, we extrapolate beyond what the empirical analysisis able to tell us by critically examining the factors that mayhave led to the current state. We also predict the character of Asians may be disadvantaged in certain considerations like ca-reer mobility in a United States context, but are considered overrep-resented here in the worldwide machine learning research context. a r X i v : . [ c s . C Y ] F e b igure 1: Researchers with lived experience relevant for thetopic of inquiry have traditionally been seen as insiders. Wehypothesize that the topic of machine learning and society isbeing conducted at a greater rate by those with lived experi-ence of marginalization.Figure 2: Researchers from overrepresented groups havetraditionally been insiders. Machine learning and societyis part of a field with underrepresentation of women andracial/ethnic groups.the boundary work that may arise in machine learning andsociety. Finally, through a short case study, we confirm thatthere is at least one example in which the theorized epistemiccontention has arisen in real life.The remainder of the paper is organized as follows. Af-ter providing a brief recapitulation of research on machinelearning and society in Section 2, we dive into the theory ofinsiders and outsiders in knowledge production in Section3. In Section 4, we discuss the participation of underrep-resented groups in science and technology with a focus oncomputer science. Section 5 presents the empirical work; itis conducted on submissions to arXiv, a preprint server thathosts a large fraction of machine learning research papers.Section 6 analyzes the sociology of knowledge productionin the area of machine learning and society using the theoryof insiders and outsiders, and boundary work. We concretizethis analysis in Section 7 through a brief case study. Section Figure 3: A hierarchical representation of the different topicsthat constitute research on machine learning and society.8 summarizes and concludes. As discussed in the introduction, two movements with a so-cietal focus have arisen alongside the growth of research anddevelopment of machine learning technologies: FAccT andAI for social good. We briefly summarize these movementsin this section, and also in Figure 3.Ethical AI, responsible AI, trustworthy machine learn-ing, and FAccT all refer to the cross-disciplinary theory andmethods for understanding and mitigating the challenges as-sociated with unwanted discrimination, lack of comprehen-sibility, and lack of governance of machine learning systemsused in applications of consequence to people’s lives such asemployment, finance, and criminal justice (Varshney 2019).Broadly speaking, there have been three different kinds ofresearch in this area (Kind 2020), including (1) philosophi-cal contributions on ethical principles for AI (Jobin, Ienca,and Vayena 2019; Whittlestone et al. 2019); (2) technicalcontributions on bias metrics and mitigation (Menon andWilliamson 2018; Kearns et al. 2019), explainability and in-terpretability algorithms (Du, Liu, and Hu 2019; Bhatt et al.2020), and factsheets as transparent reporting mechanisms(Arnold et al. 2019; Mitchell et al. 2019); and (3) contribu-tions bringing forth a social justice angle by adapting theo-ries of feminism, decoloniality, and related traditions (Buo-lamwini and Gebru 2018; Mohamed, Png, and Isaac 2020).Research in the first category, ethical principles for AI, tendsnot to overlap with research on machine learning methodsand algorithms. The second and third categories, technicalcontributions and social justice perspectives, most certainlydo intersect with other machine learning research.Algorithmic fairness research has two main branches. Thefirst is concerned with allocation decisions like loan ap-proval, pretrial detention judgement, and hiring (Barocasand Selbst 2016). The program of research is to define math-matical notions of fairness, audit existing systems with re-spect to those notions, and develop bias mitigation algo-rithms that optimize for those notions while maintaining fi-delity to the learning task. The second branch of algorith-mic fairness is concerned with representational issues, forexample in information retrieval, natural language under-standing, and dialogue systems (Blodgett et al. 2020). Herethe program of research mainly revolves around definingthe problem itself, since there are many forms of unwantedrepresentational bias ranging from stereotypes encoded intopronouns and occupations, to slurs, offensive language andhate speech, to poorer understanding of dialects and accentsof marginalized groups. In both branches, reasons for ma-chine learning models to exhibit systematic disadvantage to-wards marginalized groups include prejudice of human an-notators who label training data, undersampling of marginal-ized group members in training data, and subjective biasesby data scientists in problem specification and data prepara-tion. Research in both branches can span the spectrum fromcompletely formal applied mathematics to wholly social sci-ence with calls for justice, i.e., from the second to the thirdkind of FAccT research. Regardless of where on the spec-trum it falls, algorithmic fairness research tends to always beconsidered part of the machine learning and society nexus.Explainable and interpretable machine learning, in whichthe goal is for a person to understand how a machine learn-ing model makes its decisions, has several methodologiesappropriate for different contexts and different personas con-suming the explanations (Hind 2019). One use for explain-ability is to reveal unwanted biases in machine learningmodels, but doing so is not reliable (Dimanov et al. 2020).To date, the majority of the research has leaned towards theformal and mathematical. Calls to ground explainability insocial psychology and cognitive science (Miller 2019) havestarted to bring a greater social science character to the topic.Nevertheless, many interpretability researchers do not con-sider their methodological work to have a societal aspect,and their papers are not abundant at FAccT-specific venues.On the other hand, the framing of efforts to increasetransparency of machine learning lifecycles does incorpo-rate a societal angle. For example in factsheets—a tool andmethodology for transparently reporting information about amachine learning model as it is specified, created, deployed,and monitored—the reported information can include the in-tended use of the model as well as quantitative test resultson accuracy, fairness, and other performance indicators. It isuseful to individuals impacted by the machine learning sys-tem (especially those from marginalized groups) and to reg-ulators charged with ensuring the system behaves accordingto laws and societal values.Whereas FAccT is concerned with preventing societalharms, AI for social good takes the opposite track and usesthe technology to benefit society, especially those at themargins (Chui et al. 2018; Varshney and Mojsilovi´c 2019).The working paradigm is to pair data scientists with socialchange organizations to work towards the 17 SustainableDevelopment Goals (SDGs) ratified by the member statesof the United Nations in 2015, which include: ‘no poverty,’‘zero hunger,’ ‘good health and well-being,’ ‘quality edu- cation,’ ‘gender equality,’ and twelve others. The form ofthis pairing may be data science competitions, weekendvolunteer events, longer term volunteer-based consultingprojects, fellowship programs, corporate philanthropy, spe-cialized non-governmental organizations, innovations unitswithin large development organizations, or data scientistsemployed directly by social change organizations. Someprojects require research and some are more application ori-ented. The ones that require research and whose results arepublished fall squarely within the intersection of machinelearning and society.
The insider/outsider discussion in social sciences and hu-manities addresses the role of the researcher as an insider(i.e., a member of the community being studied) as opposedto an outsider in affecting research, approach, relationshipwith participants, and/or findings. The insider doctrine ofMerton (1972) highlights the insider’s exclusive access (thestrong version) or privileged access (the weaker version) toknowledge and the outsider’s exclusion from it. Researchersare considered as insiders or outsiders based on their as-cribed status (e.g., gender, race, nationality, cultural or re-ligious background) or group membership. The strong ver-sion asserts that the insider and the outsider cannot arrive atthe same findings even when they examine the same prob-lems; the weaker version argues that insider and outsider re-searchers would focus on different research questions. Thecombined version argues that the researcher needs to be aninsider in order to understand the community and also toknow what is worth understanding or examining about thecommunity (Merton 1972).However, structurally speaking, it is hard to completelydistinguish the insider from the outsider because we all oc-cupy a combination of different statuses, including sex, age,class, race, occupation, and so on. The insider knowledgethat is accessible to only individuals who occupy a highlycomplex set of statuses is limited to a very small group, andthis way of knowledge production and sharing is not sustain-able. Similarly, social scientists like Karl Marx recognize thevalue of political, legal, and philosophical theories in eco-nomics. Another limitation of the insider doctrine is that ittakes a static perspective and does not recognize that our sta-tuses and life experience evolve over time, which shifts ourstatus as an insider or an outsider. In the meantime, the out-sider, while not being able to completely transcend existingbeliefs and social problems, has the advantages of using lessbias in examining social issues and bringing new perspec-tives to solving issues taken for granted by insiders. The in-teraction of the insiders and outsiders makes intellectual ex-change possible, and Merton argues that we could integrateboth sides in the process of seeking truth.Extending Merton’s and other scholars’ thoughts on theinsider/outsider debate, Griffith (1998) also believes thatthe researcher occupies a particular social location, and herknowledge is situated in particular sets of social relations.owever, the insider status is just the beginning but not theend of the research process. Reflecting on her own researchexperience in mothering work for schooling, she and her col-laborator who were both single mothers started as insiders(mothers). However, they had to cross the social and con-ceptual boundaries to include only mothers from two-parentfamilies (and thus become outsiders in the research process)as the two-parent family is the ideological norm perceivedby the schools and society. In other words, researchers arerarely insiders or outsiders but oftentimes insiders and out-siders at the same time, and research is constructed betweenthe researcher and many Others.Dwyer and Buckle (2009) argue that both the insider andthe outsider statuses have pros and cons, so what is impor-tant is not the insider or the outsider status but “an ability tobe open, authentic, honest, deeply interested in the experi-ence of one’s research participants, and committed to accu-rately and adequately representing their experience.” In fact,researchers occupy the ‘space between.’ Challenging the di-chotomy and the static nature of insider versus outsider sta-tus, the ‘space between’ recognizes the evolving nature ofthe researcher’s life experience and knowledge on the re-search topic as well as her relationship to participants.When there are insiders and outsiders in scientific re-search, there is also a boundary between them. Specifically,the boundary delineates what is considered ‘science’ andwhat is considered ‘non-science’ in a particular subfield.Boundary work attempts to shape or disrupt the boundary ofwhat is considered as valid knowledge (Gieryn 1983, 1999).Research reveals two types of boundary work: symbolic andsocial boundaries. Symbolic boundaries are formed whenmembers agree on meaning and definition of the field andobtain a collective identity. Social boundaries enable mem-bers’ access to material and non-material resources (e.g., sta-tus, legitimacy, and visibility) (Lamont and Moln´ar 2002;Grodal 2018).For example, Grodal (2018) details how core communi-ties who entered the nanotechnology field early expandedthe boundaries of the field by enlarging the definition of thefield and associating new members. Peripheral communities,including service providers, entrepreneurs, and universityscientists, self-claimed membership during the expansionphase due to newly available material and cultural resources.Later on, while some peripheral communities continued toassociate themselves to nanotechnology, the core communi-ties, realizing their collective identity being threatened andresources being restricted because of the enlarged symbolicboundaries, contracted boundaries by restricting the defini-tion and policing membership. Also, some peripheral com-munities, not identifying strongly with the more restrictivecollective identity, self-disassociated and focused on otherfields of interest. In this process, the insiders or the corecommunities entered the field earlier and had a vested inter-est to protect, while the outsiders or the peripheral commu-nities entered the field later and had a weak association withthe field. The insiders had more power than the outsiders indefining the boundaries of the field and making certain typesof work and research legitimate.
Participation of Women
In science, women have been at the “Outer Circle” for along time. Historically, women faced multiple barriers in en-tering a scientific career, and even those who were able tobecome a scientist were not allowed into the inner circlesof the emerging scientific community (Zuckerman, Cole,and Bruer 1991). While women’s representation, experi-ence, and advancement in science has increased over time,many of them continue to face barriers, especially at thecultural and structural levels (Zuckerman and Cole 1975;Rosser 2004; Hill, Corbett, and St Rose 2010; NationalResearch Council of the National Academies 2010; Ceciet al. 2014). This is especially true in computer science(CS), where, unlike other scientific fields, women’s partic-ipation has been consistently low, with some fluctuations.Hayes (2010) records the changes of women’s representa-tion in CS in multiple decades: women represented 11% ofall CS bachelor’s degree recipients in 1967; this percentagepeaked at 37% in 1984 and then declined to only 20% in2006. For comparison, women represented 44% of all bach-elor’s degree recipients in 1966 and 58% in 2006, and otherSTEM fields also witnessed steady increases in this period.Despite the rapid growth of the computer and mathemati-cal science workforce, women’s proportion declined from31% in 1993 to 27% in 2017. However, the silver lining isthat among workers with a doctoral degree in these occupa-tions, women’s share increased from 16% in 1993 to 31%in 2017 (National Academies of Sciences, Engineering, andMedicine 2020).Multiple factors that oftentimes reinforce each other con-tribute to women’s low representation relative to men’s inCS at different life stages. Earlier research reports individ-ual factors, such as a lack of early exposure to and experi-ence with computing, women students’ inaccurate percep-tions of their low quantitative abilities, and a lack of selfconfidence despite their good performance and computerknowledge level. Other research has also focused on so-cial, cultural, and structural factors which are much harder tochange. For instance, women’s perceptions of their abilitiesand the field of computing could be affected by the ‘chillyclassroom’ with male students’ unfriendly reactions and pro-fessors’ lack of attention to them; a lack of role models andmentoring; stereotypes against women and against the peo-ple, work involved, and values of CS; and the perceived mis-match of women’s career orientation to help people and so-ciety and what they think CS could offer. Combating thesebarriers could increase women’s representation in CS orlower their attrition from CS (G¨urer and Camp 2002; Beyer,Rynes, and Haller 2004; Beyer and DeKeuster 2006; Co-hoon 2006; Kim, Fann, and Misa-Escalante 2011; Cheryan,Master, and Meltzoff 2015; Lehman, Sax, and Zimmerman2016; Cheryan et al. 2017).Policy recommendations and college intervention pro-grams have been made and established to change the cul-tural and institutional environment in order to recruit and re-tain more women students and professionals in CS. Some ofhe recommendations were repeatedly made in different timeperiods, reflecting a reluctance of change over time. They in-clude involving women students in research at both the un-dergraduate level and early in their graduate study, activelycountering stereotypes and misperceptions of CS, and high-lighting and showing women students the positive social im-pact that scientists can make and the diverse group of scien-tists making social impacts in their fields (Cuny and Aspray2002; National Academies of Sciences, Engineering, andMedicine 2020). Successful college intervention programsin increasing the number and percentage of women CS stu-dents and their sense of belonging all tackled the culture ofCS and the institution instead of changing the (women) stu-dents. These efforts changed the stereotypes of CS by creat-ing introductory CS courses to be inclusive of a diverse stu-dent body, providing role models and mentoring to womenstudents, providing research experience, and exposing stu-dents to a wide range of applications of CS in solving so-cietal issues (Roberts, Kassianidou, and Irani 2002; Muller2003; Wright et al. 2019; Frieze and Quesenberry 2019; Na-tional Academies of Sciences, Engineering, and Medicine2020).
Participation of Racial and Ethnic Minorities
In addition to gender, race/ethnicity also shapes scientists’representation and experience in science as well as their out-sider and insider statuses. Among racial/ethnic minorities ina United States context, while Asians tend to be overrepre-sented in science, the other groups (blacks, Hispanics, andAmerican Indians or Alaska natives) are considered as un-derrepresented minorities (URMs) due to their low represen-tation in scientific fields, despite their growth over time. Forinstance, URMs made up 9% of workers in computer sci-ence and mathematics occupations in 2003, and this percent-age increased to 13% in 2017 (Khan, Robbins, and Okrent2020). While their participation increased over time, it wasstill lower than their representation in the general popu-lation, confirming their persistent “outsider” status in sci-ence. Similar trends hold in a world context with Asians andwhites overrepresented compared to black, Hispanic, and in-digenous people.Research on race and science finds that racial/ethnic mi-norities, especially URMs, tend to be less likely to pub-lish their research, receive research grants, get recognitionfor their work, and get promoted but more likely to workin institutions with less resources and more likely to bemarginalized in formal and informal scientific communitiesthan their white counterparts. Research also reveals someimprovement in their representation in scientific fields aswell as in their career experience and outcomes over time,but the progress is slow relative to the growth of the sci-entific workforce (Pearson 1985, 2005; Ginther et al. 2011;Ginther 2018; Tao and McNeely 2019). In the meantime,an increasing number of studies employ intersectionality asthe research framework that indicates power relations andsocial inequalities to examine the double disadvantages thatminority women scientists suffer from due to both their gen-der and race, e.g., (Malcom, Hall, and Brown 1976; Mal-com and Malcom 2011; Collins 2015; Metcalf, Russell, and Hill 2018). While minority women scientists of differentracial/ethnic groups differ from each other in their careerexperience and outcomes, they all tend to fare less wellthan comparable white women as well as men of the sameracial/ethnic group (Malcom, Hall, and Brown 1976; Mal-com and Malcom 2011; Pearson 1985; Ong et al. 2011; Tao2018; Tao and McNeely 2019), revealing the persistent in-tersectional effect of race and gender.
Status and Career/Research Focus
Broadly speaking, women tend to be more engaged thantheir male peers in relatively new, interdisciplinary scien-tific fields (e.g., environmental studies) that are oftentimesmore contextual and problem-based than traditional fields,may not have existing gender hierarchy, and are not well-embedded in the structure of academia or knowledge pro-duction, providing more opportunities for women to buildthe discipline (Rhoten and Pfirman 2007). While somewomen shy away from technical fields because they donot see the social engagement of these fields, e.g., (Carter2006), those who choose technical fields do so not onlybecause of the excitement of solving technical problems,but also the potential of addressing issues concerning themand positively impacting people’s lives, which is consistentwith their interpersonal and career orientations (Silbey 2016;Bossart and Bharti 2017). Women CS majors choose com-puting in the context of what they could do for the worldwith computing—they would like to use the computer in thebroader context of education, medicine, music, communi-cation, healthcare, environmental studies, crime prevention,etc. While they also enjoy exploring the computer, the mainfactor reported by men, women are more likely than theirmale peers to address the broader social context (Fisher,Margolis, and Miller 1997; Carter 2006; Hoffman and Fried-man 2018). While few women in AI were at the “outer cir-cle” in its initial stage, they were attracted to it when itstarted to develop in the 1980s and 1990s because it wasmore cognitive than other areas of CS and there were fewerexisting stereotypes to fight against (Strok 1992). The in-tersection of machine learning and society makes careers inmachine learning meaningful to them (Hoffman and Fried-man 2018). As discussed in Section 4, women and URMs tend to selectfields in which they perceive they can help people and so-ciety. The intersection of machine learning and society pro-vides exactly that opportunity to make social impact. There-fore, we hypothesize that women and URMs are more likelyto contribute to research in machine learning and societyrather than machine learning without a direct societal com-ponent.We performed the following analysis to test our hypoth-esis. On September 19, 2020, we downloaded the full col- In this part of the paper and later, we discuss women morethan Hispanics and blacks, not because of differing experiences,but because of a dearth of published literature on the analogousexperience (Spertus 1991). igure 4: Percentage of machine learning and society papersout of the overall set of machine learning papers posted toarXiv computed on a monthly basis.lection of arXiv paper metadata available on Kaggle. ThearXiv is a preprint server that has become a de facto stan-dard location for authors to upload their machine learningpapers, often irrespective of a paper’s publication status inpeer-reviewed conferences and journals. When authors up-load papers, they self-select one or more subject area cat-egories to tag their paper with. We extracted the subset ofmachine learning papers among all arXiv papers by filteringthose with the set of tagged categories containing stat.ml orcs.lg, the two tags indicating ‘machine learning.’ This gaveus 71,605 papers from March 1997 to present. We furthermarked the papers that also had the tag cs.cy for ‘comput-ers and society.’ We consider these papers to be the onesdescribing research on machine learning and society. Therewere 1,077 such papers from August 2004 to present. Thus,the machine learning and society papers represent 1.50% ofthe totality of machine learning papers across years. We alsoanalyzed this percentage as function of time, shown in Fig-ure 4. There has been growth over time and the most recent(partial) month, September 2020, had the highest percent-age of societally-oriented papers at 3.16% (except for a fewblips in early years when there were small sample sizes).The conclusion thus far is that machine learning and societyis a tiny sliver of the overall machine learning universe, butis growing in fraction.Now, let us move on to analyzing the authors. In contrastto analyses in other sociology of science research, we arenot focused on research productivity. Therefore, we did nothave to count papers per author or distribute credit amongauthors. We simply collected the unique set of authors whonumbered 103,094. Of these authors, 1,904 had papers onlyin machine learning and society, 99,460 had no papers inmachine learning and society, and 1,730 had papers of bothkinds: at least one tagged with cs.cy and at least one othernot tagged with cs.cy. Table 1: Average soft classification score of race/ethnicityand gender for three categories of authors.
Asian Hispanic Black White Maleno cs.cy 0.370 0.077 0.057 0.497 0.791both 0.367 0.073 0.055 0.504 0.777only cs.cy 0.266 0.097 0.071 0.566 0.726slope -0.0430 0.0077 0.0055 0.0298 -0.0293p-value 0 0.0062 0.0241 < < We used the names of authors to estimate their genderand their race/ethnicity. Such imputation based on observ-able proxies is fraught with potential biases, as analyzed anddiscussed at length by Chen et al. (2019). As recommendedin that work, we used a soft rather than hard-threshold clas-sifier to minimize the bias as much as possible. For gen-der classification from the first name, we used the gender-ize.io API, which has been used frequently for similar anal-yses (Topaz and Sen 2016; Hart and Perlis 2019; Patel et al.2020). If genderize.io was unable to produce a classification,we dropped that author from all subsequent gender analy-sis. In the analysis that follows, a soft classification scorein the range [0 , . corresponds to female and a soft clas-sification score in the range (0 . , corresponds to male.For race/ethnicity classification, we used two pre-trainedneural networks from ethnicolr (Sood and Laohaprapanon2018), which has also been used previously in similar anal-yses (Hofstra et al. 2020; Chang and Fu 2020; Parasurama,Ghose, and Ipeirotis 2020). We took the simple average ofthe soft classification score from a model trained on lastnames from the United States Census and the soft classifi-cation score from a model trained on first, middle, and lastnames from the 2017 Florida voter registrations. Both mod-els have four class labels: Asian, Hispanic, black, and white.While we acknowledge the limitations of such ways of iden-tifying authors’ gender and race/ethnicity, they do allow usto analyze the relationship between gender or race/ethnicityand research focus in the absence of better alternatives.The average soft classification score of race/ethnicity andof gender, broken down by authorship group membership isshown in Table 1. Recall that the three rows in the table,the author groups, correspond to authors having no machinelearning papers with a societal aspect, authors having ma-chine learning papers both with and without a societal as-pect, and authors having machine learning papers only witha societal aspect. First look at the gender column labeled‘Male.’ (It is labeled as such because the number indicates a(possibly uncalibrated) probability of being male.) The num-bers decrease as we go down the table, which indicates thatthe overrepresented male group is less likely to work on ma-chine learning and society and the underrepresented femalegroup is more likely to work on machine learning and soci-ety. The table also gives the p-value and slope for the trend inproportion obtained using the Cochran-Armitage trend test.The negative slope result is consistent with our hypothesis.Next focus on Asians; again, the slope is negative, andagain, an overrepresented group is less likely to work on ma-chine learning with societal impact. In the next two columns,able 2: Average soft classification score of race/ethnicityamong estimated males for three categories of authors. Asian Hispanic Black Whiteno cs.cy 0.335 0.078 0.059 0.528both 0.343 0.076 0.057 0.525only cs.cy 0.247 0.097 0.073 0.583slope -0.0345 0.0072 0.0051 0.0223p-value < < Table 3: Average soft classification score of race/ethnicityamong estimated females for three categories of authors.
Asian Hispanic Black Whiteno cs.cy 0.446 0.071 0.050 0.433both 0.420 0.067 0.051 0.463only cs.cy 0.287 0.093 0.068 0.551slope -0.0695 0.0081 0.0077 0.0537p-value 0 0.0029 0.0007 0
Hispanic and black, the trend is reversed. This indicates thatthese two underrepresented groups are more likely to workon machine learning with societal impact. These results arestatistically significant and consistent with our hypothesis.Finally, if we look at the white column, we see thenumbers increase (positive slope), which is the hypothe-sized trend of underrepresented rather than overrepresentedgroups. To understand this better, we conducted an intersec-tional analysis of race/ethnicity by hard-thresholded gender,presented in Table 2 and Table 3. Looking at the white col-umn in both tables, we see that the increase is much strongerfor white females than white males, which again is consis-tent with the hypothesis of underrepresented groups havingan affinity for making social impact. The Hispanic and blackcolumns in these two tables are similar to the aggregate re-sult in Table 1. The Asian columns for males and femaleshave differences, however. Asian females show a strongertrend than Asian males in the direction hypothesized foroverrepresented groups.To reconfirm this finding, we also did the intersectionalanalysis in the opposite direction: by taking the hard clas-sification of the predicted race/ethnicity and producing theTables 4–7. The average soft classification score for gen-der (probability of male) is much lower for Asians in theno societal impact category, shown in Table 4, than all otherrace/ethnicity groups, shown in Tables 5–7. Asian femaleswork on machine learning research with a societal anglethe least among the different female racial/ethnic groups.Broadly speaking, the results are consistent with the hypoth-esis that underrepresented groups defined by Hispanic orblack race/ethnicity or female gender focus more researchefforts towards machine learning and society than machinelearning without societal impact.
Limitations
The empirical methodology has limitations, but still pro-vides us with a unique opportunity to understand the rela-tionship between insider/outsider status and research focus. Table 4: Average soft classification score of gender amongestimated Asians for three categories of authors.
Maleno cs.cy 0.738both 0.732only cs.cy 0.696slope -0.0183p-value < Table 5: Average soft classification score of gender amongestimated Hispanics for three categories of authors.
Maleno cs.cy 0.830both 0.815only cs.cy 0.722slope -0.0473p-value 0
As mentioned before, using proxies for sensitive attributeshas limitations in general (Chen et al. 2019), but more sowhen the proxy is the person’s name. The name does not ac-curately identify the gender and/or race/ethnic backgroundequally well for all people, with a particular confusion be-tween Hispanics and white southern Europeans. Also, theuse of arXiv as a de facto standard repository for machinelearning research was not true in early years. We minimizethis concern by only comparing machine learning paperswith and without a societal aspect, rather than looking at ab-solute numbers. There may be some small bias in the analy-sis if authors of works with and without societal aspects be-have differently in arXiv posting behavior, which may hap-pen because of different publication practices in the socialsciences and humanities compared to CS.
The insider/outsider discussion is embedded in qualitativeresearch in social sciences and humanities, and it has ad-dressed the impacts of the researcher’s insider or outsiderstatus on the role they play in their research, participants’trust of and openness to the researcher, and the data be-ing collected, e.g., (Labaree 2002; Mercer 2007; Kerstet-ter 2012; Hayfield and Huxley 2015). However, this discus-sion has an implication on research on machine learning andsociety as well. On the one hand, while women and URMmachine learning researchers do not necessarily have face-to-face interactions with their subjects like some social sci-entists, they do have lived experience of unwanted bias insociety and the workplace. They care about redressing so-cial inequalities and can bring that insider experience intotheir research. This insider status is illustrated in Figure 1.On the other hand, women, Hispanics and blacks are histori-cally underrepresented in science and engineering fields, es-pecially CS, which makes them outsiders, illustrated in Fig-ure 2. It is worth noting that this outsider status does notrefer to a lack of technical competence, it only means thatthey are underrepresented in the ML community and, as aable 6: Average soft classification score of gender amongestimated blacks for three categories of authors.
Maleno cs.cy 0.825both 0.871only cs.cy 0.721slope -0.0345p-value 0
Table 7: Average soft classification score of gender amongestimated whites for three categories of authors.
Maleno cs.cy 0.822both 0.801only cs.cy 0.739slope -0.0375p-value 0 result, it is hard to get into the inner circle of the community.However through the empirical study of Section 5, we knowthat women and URMs are overrepresented in research onmachine learning and society as compared to plain machinelearning research.
Analysis of the Current State
Despite being at the center of building the field of machinelearning and society research, women’s (and URM’s) experi-ence in the workplace reflects their overall struggles in soci-ety. Similar to women in some other scientific fields, womencomputer scientists tend to be more subject to stereotyping,less likely to be full professors or in senior research andtechnical positions, less recognized for their work and paidless, more likely to be subject to overt discrimination andharassment, more likely to face pressures in balancing workand life, and more likely to be marginalized than their malepeers (Strok 1992; Simard and Gilmartin 2010; Rosser 2004;Tao 2016; Fox and Kline 2016; Khan, Robbins, and Okrent2020). These “outsider” disadvantages provide them withthe insider perspective when conducting algorithmic fair-ness and other socially-oriented machine learning research.In fact, women (and URM) scientists’ lived experience andconsequent insider status place them in a unique position toformulate questions and conduct research at the intersectionof machine learning and society.The finding that women and underrepresented minoritiesare more likely to work on machine learning and societyresearch should not be interpreted as that all insiders con-duct only machine learning research without the social as-pect and all outsiders conduct only machine learning andsociety research. However, this finding is consistent with lit-erature that reveals women’ and underrepresented minori-ties’ preference for conducting and applying research in abroader context—one that goes beyond the technical. As in-siders of social inequality, they bring their lived experienceand the new perspective into a field where they have beenoutsiders. Now in the late 2010s and early 2020s, machine learning and society represents an area of AI that is relativelynew, interdisciplinary, not well-embedded in the structureof academia, and without existing hierarchies, and thus onewith an opportunity for women and URMs to build, whichthey are doing.While our empirical analysis finds that women of differ-ent racial/ethnic groups tend to behave more similarly toeach other than to their male counterparts, we also find thatthe women’s groups differ from each other, confirming theintersectionality perspective and that some groups are notpurely insiders or purely outsiders. We would like to high-light Asian women, who are in a unique position in ma-chine learning (or science as a whole) because Asians areoverrepresented but women are underrepresented in science.Asian men tend to behave similarly to their white counter-parts in their career outcomes, but Asian women tend tobehave more like other women’s groups, making the gen-der gap among Asians greater than that among some otherracial/ethnic groups (Tao 2015, 2018; Tao and McNeely2019). Being insiders in machine learning on the one hand(Asians) and being outsiders on the other hand (women)could possibly constrain some of their choices because theymay receive inconsistent expectations and experience multi-level barriers. In the meantime, the Asian and Asian Amer-ican cultures tend to emphasize technical expertise and theinstrumental value of education to fight their marginal statusand to achieve upward social mobility in American society(Xie and Goyette 2003; Min and Jang 2015). The emphasison technical aspects and some structural barriers they experi-ence in their careers may suggest that Asian women pursuemachine learning occupations due to the technical and fi-nancial aspects more than the social impact of such occupa-tions that are more likely to be highlighted by other women’sgroups.In addition, the findings reveal complicated issues ofpower and inequality in the ML community, which reflectssocietal inequality. Both at the personal level, e.g., in termsof exposure to and experience with computing at an earlyage, and at the cultural and structural levels, e.g., in termsof experiences in computing classes and workplace, statuses(e.g., as women or racial/ethnic minorities) affect our livedexperience and opportunities to pursue a career in science.When entering science, our lived experience could impactour research focus. While women and URMs are not out-siders to ML in the sense of being less technically com-petent, they are outsiders as historically underrepresentedgroups that have not been successful in penetrating into theinner circle. As a result, they are not in a position of powerbut are disadvantaged in various ways. In the meantime, be-ing insiders to the experience of inequality, they use theirtechnical expertise to address and provide solutions to per-sistent social inequality. In this sense, they are empoweringnot only themselves as underrepresented groups but also theML community by raising awareness and impact of ML re-search with social implications.
Epistemic Conflict
Outsiders’ entrance into the field could be shaped by ex-isting barriers and policed by the insiders. Once outsidersnter a field, they have another challenge of making legiti-mate the research that they prefer but somehow diverge fromthe mainstream. Although research driven by lived experi-ence (including the third category of FAccT research thatbrings in feminist, post-colonial, and other related critical-theoretic thoughts) may be celebrated within the intersec-tion of machine learning and society, it is questioned out-side of the intersection on epistemic grounds. Accordingto Haraway (1988), knowledge is situated and embodied inspecific locations and bodies, and the multidimensional andmultifaceted views and voices, from both those in power andthose with limited voices, combine to make science. Never-theless, despite scholarship supporting lived experience notbeing in conflict with scientific objectivity, the common re-frain summarized by the feminist and postcolonial episte-mologist Sandra Harding is as follows: “‘Real sciences’ aresupposed to be transparent to the world they represent, tobe value neutral. They are supposed to add no political, so-cial, or cultural features to the representations of the worldthey produce.” In other words, ways of knowing that do notfollow the (Western) scientific method are not seen by prac-titioners as scientific (Harding 2006), despite scholarly crit-icisms to this perspective. The implication in the context ofmachine learning and society is that critical-theoretic workbased on lived experience as the source of knowledge willbe discounted in mainstream machine learning: insider re-search by outsiders is precarious.
Boundary Work Predictions
Based on the sociology theory, we may predict two possiblefutures, both involving boundary work . The first is a sever-ing of the connection between mainstream machine learn-ing research and societally-relevant applications and gover-nance, i.e., the expulsion of machine learning and societyfrom mainstream machine learning. The second is the ex-pansion of machine learning research to include knowledgefrom lived experience, while overcoming tendencies for ex-pulsion and the protection of autonomy that many insidersof machine learning research may have.The future that emerges among the two possibilities coulddepend on what insiders perceive as legitimate, as in thecase of nanotechnology. (The insiders hold epistemic au-thority both due to their entrenched status and the powerthat comes from their identity (race/ethnicity, gender, etc.)(Pereira 2012, 2019).) In addition, another factor may shapethe future of machine learning and society research: sus-tainability of the ML field as a thriving site of research tocontinuously attract the next generation of scholars, includ-ing women and underrepresented minorities. When the out-siders conduct their machine learning and society research,they raise awareness of the broader context of technical is-sues. While machine learning and society research is stilla small portion of machine learning research overall, it hasbeen growing. Led by “the outsiders,” this line of inquiry isincreasingly being addressed and published. Based on thistrend and considering that the field could benefit from bothinsiders and outsiders’ perspectives, we have reasons to be-lieve that machine learning and society research will trans-form and sustain ML knowledge and practice, even though it may not happen soon and there may be backlashes.
Let us see if our boundary work predictions from Section6 hold in a specific case study. In June 2020, the soft-ware for an image super-resolution algorithm (Menon et al.2020) was posted on GitHub and soon discovered to alterthe perceived race/ethnicity of individuals whose downsam-pled face images were presented as input (Johnson 2020a;Kurenkov 2020; Vincent 2020). Examples of input black,Hispanic, and Asian face images yielded white-looking re-sults. About these results, Facebook machine learning re-searcher Yann LeCun commented on Twitter: “ML systemsare biased when data is biased. This face upsampling systemmakes everyone look white because the network was pre-trained on FlickFaceHQ, which mainly contains white peo-ple pics. Train the *exact* same system on a dataset fromSenegal, and everyone will look African.”In response, Google machine learning researcher TimnitGebru pointed to the video of her recently completed tutorial(with Emily Denton)
Fairness Accountability Transparencyand Ethics in Computer Vision with the comment: “Yann,I suggest you watch me and Emily’s tutorial or a numberof scholars who are experts in this are. You can’t just re-duce harms to dataset bias. For once listen to us people frommarginalized communities and what we tell you. If not nowduring worldwide protests not sure when.” She also posted:“I’m sick of this framing. Tired of it. Many people have triedto explain, many scholars. Listen to us. You can’t just reduceharms caused by ML to dataset bias.” A back and forth de-bate ensued on Twitter with many interlocutors taking sidesand offering inputs.Let us analyze what happened using the insider/outsiderunderstanding of research on machine learning and societythat we have developed in this paper. LeCun is a white male,Chief AI Scientist at Facebook, and Turing award winner—a person likely without lived experience of marginalizationand a clear insider in mainstream machine learning research.Gebru is a black female, co-lead of the Ethical Artificial In-telligence Team at Google at the time—a person with livedexperience of marginalization and thus an insider in algo-rithmic fairness research, but an outsider in machine learn-ing overall.Although Gebru et al. (2019) say: “Of particular concernare recent examples showing that machine learning mod-els can reproduce or amplify unwanted societal biases re-flected in datasets,” which is consistent with LeCun’s argu-ment, Gebru’s comments in the debate point to her holding astance consistent with Merton’s insider doctrine of lived ex-perience providing privilege (bordering on exclusivity) forconducting research on machine learning bias. Additionally,her epistemic perspective appears to be that such lived ex-perience is a valid source of knowledge for “many schol-ars.” The repeated call to listen to scholars is an attempt atexpansion boundary work. On the other hand, LeCun’s per-spective epitomizes a boundary and epistemic authority thatleaves lived experience out of machine learning research;scientifically-derived knowledge is valid knowledge. At theend, some white males in positions of power also joinedhe debate and offered allyship, which may have expandedthe epistemic boundary of machine learning just a little bit.What may have appeared at first glance to be a personal warof words was in fact an example of boundary work in prac-tice, manifested as contention between two insiders of theirown respective domains.After we completed the first draft of this paper in Oc-tober 2020, there was further contention involving TimnitGebru in December 2020 (Johnson 2020b). She was dis-missed from her research position at Google by Jeff Deanand Megan Kacholia, who are both white and in positionsof power. The dismissal was widely argued in a public man-ner. Among others, one of the factors was Gebru’s reluctanceto remove Google authors from or withdraw the paper “Onthe Dangers of Stochastic Parrots: Can Language ModelsBe Too Big? a ” (Bender et al. 2021) from the ACM FAccTConference. One of the main parts of this paper conducts acritical analysis of large language models through the lensof decolonizing hegemonic views (Srigley and Sutherland2018), which is a prototypical example of the third, socialjustice, angle to FAccT research mentioned in Section 2.Dean and Kacholia’s criticism of the paper was the inclusionof the decolonial perspective at the expense of a technical-only analysis that would include a discussion of techniquesto mitigate representation bias, which correspond to the sec-ond kind of FAccT research mentioned in Section 2. The firstauthor of the paper, Emily Bender, posted on Twitter: “Theclaim that this kind of scholarship is ‘political’ and ‘non-scientific’ is precisely the kind of gate-keeping move set upto maintain ‘science’ as the domain of people of privilegeonly.” This case is also one of epistemology and further il-lustrates how boundary work can engender extreme conflictand the tendency for expulsion boundary work.Together, these cases illustrate a little bit of expansionboundary work and a healthy dose of expulsion boundarywork, both as predicted by the theory of insiders and out-siders. In this paper, we have studied the sociology of researcherscreating new knowledge in the area of machine learning. Wehave analyzed the intersection of general machine learningresearch with FAccT and AI for social good—collectivelymachine learning and society—using Merton’s concepts ofinsiders and outsiders. Although these concepts are usuallyonly applied in studying the social sciences and humanitiesrather than the natural sciences and formal sciences, there isa clear insider in terms of lived experience in machine learn-ing and society: a member of a marginalized group. Just asimportantly, these same marginalized groups have low his-torical participation and representation in computer science,the parent field of machine learning and are thus outsidersin a different way. Through an empirical study, we find thatresearchers from marginalized groups are overrepresented inconducting research on machine learning and society. There-fore, we have a situation in which the same group takes theinsider role in terms of lived experience and the outsider rolein terms of historical participation. This situation leads to tension over the boundaries of valid knowledge in machinelearning and society. Specifically, the epistemic question thatarises is whether lived experience is a valid source of knowl-edge. Instances of expansion and expulsion boundary workare predicted and verified in a case study.If one takes the normative stance that expansion bound-ary work is preferable to expulsion, then the resolution ofthe epistemic contention calls for facilitation by researcherswith ascribed status that is not marginalized and who leantowards including knowledge derived from lived experiencewithin the boundary of machine learning research.This paper illuminates a few avenues for future research.One such direction is to dive deeper into the topics of situ-ated knowledge, feminist epistemology, and boundary workto better understand how the field of machine learning andsociety may evolve and to understand strategies for directingthat evolution in a beneficial way. Another future direction isto study the impact of papers in machine learning and soci-ety produced by teams containing only members with livedexperience with marginalization, containing only memberswithout lived experience with marginalization, and contain-ing both types of researchers, to understand whether workthat bridges the epistemic divide of formal science and crit-ical theory is more valuable than other pieces of work.
The authors thank Delia R. Setola for her assistance and TinaM. Kim for her comments.
References
Arnold, M.; Bellamy, R. K. E.; Hind, M.; Houde, S.; Mehta,S.; Mojsilovi´c, A.; Nair, R.; Natesan Ramamurthy, K.;Olteanu, A.; Piorkowski, D.; Tsay, J.; and Varshney, K. R.2019. FactSheets: Increasing Trust in AI Services ThroughSupplier’s Declarations of Conformity.
IBM Journal of Re-search and Development
California Law Review a . In Proceedings of the ACMConference on Fairness, Accountability, and Transparency .Beyer, S.; and DeKeuster, M. 2006. Women in ComputerScience or Management Information Systems Courses: AComparative Analysis. In Cohoon, J.; and Aspray, W., eds.,
Women and Information Technology: Research on Under-representation . MIT Press.Beyer, S.; Rynes, K.; and Haller, S. 2004. Deterrents toWomen Taking Computer Science Courses.
IEEE Technol-ogy and Society Magazine
Proceed-ings of the ACM Conference on Fairness, Accountability,and Transparency , 648–657.lodgett, S. L.; Barocas, S.; Daum´e, III, H.; and Wallach,H. 2020. Language (Technology) is Power: A Critical Sur-vey of “Bias” in NLP. In
Proceedings of the 58th AnnualMeeting of the Association for Computational Linguistics ,5454–5476.Bossart, J.; and Bharti, N. 2017. Women in Engineering: In-sight into Why Some Engineering Departments Have MoreSuccess in Recruiting and Graduating Women.
AmericanJournal of Engineering Education
Proceedings of the Conference on Fairness,Accountability, and Transparency , 77–91.Carter, L. 2006. Why Students With An Apparent Aptitudefor Computer Science Don’t Choose to Major in ComputerScience.
ACM SIGCSE Bulletin
Psychological Science in the Public Interest arXiv preprint arXiv:2002.07789 .Chen, J.; Kallus, N.; Mao, X.; Svacha, G.; and Udell, M.2019. Fairness under Unawareness: Assessing DisparityWhen Protected Class Is Unobserved. In
Proceedings of theACM Conference on Fairness, Accountability, and Trans-parency , 339–348.Cheryan, S.; Master, A.; and Meltzoff, A. N. 2015. Cul-tural Stereotypes as Gatekeepers: Increasing Girls’ Inter-est in Computer Science and Engineering by DiversifyingStereotypes.
Frontiers in Psychology
6: 49.Cheryan, S.; Ziegler, S. A.; Montoya, A. K.; and Jiang, L.2017. Why Are Some STEM Fields More Gender Balancedthan Others?
Psychological Bulletin
Women and Information Technol-ogy: Research on Underrepresentation . MIT Press.Collins, P. H. 2015. Intersectionality’s Definitional Dilem-mas.
Annual Review of Sociology
41: 1–20.Cuny, J.; and Aspray, W. 2002. Recruitment and Retentionof Women Graduate Students in Computer Science and En-gineering: Results of a Workshop Organized by the Com-puting Research Association.
ACM SIGCSE Bulletin
AAAIWorkshop on Artificial Intelligence Safety , 63–73.Du, M.; Liu, N.; and Hu, X. 2019. Techniques for Inter-pretable Machine Learning.
Communications of the ACM
In-ternational Journal of Qualitative Methods
ACM SIGCSE Bulletin
Pathways, Potholes, and the Persistence of Women in Sci-ence: Reconsidering the Pipeline , 41–55. Lexington Books.Frieze, C.; and Quesenberry, J. L. 2019. How ComputerScience at CMU is Attracting and Retaining Women.
Com-munications of the ACM arXiv preprint arXiv:1803.09010v4 .Gieryn, T. F. 1983. Boundary-Work and the Demarcationof Science from Non-Science: Strains and Interests in Pro-fessional Ideologies of Scientists.
American SociologicalReview
Cultural Boundaries of Science: Credi-bility on the Line . University of Chicago Press.Ginther, D. K. 2018. Using Data to Inform the Scienceof Broadening Participation.
American Behavioral Scientist
Science
Human Studies
Ad-ministrative Science Quarterly
ACM SIGCSE Bulletin
Feminist Studies
Science and Social Inequality: Feministand Postcolonial Issues . University of Illinois Press.Hart, K. L.; and Perlis, R. H. 2019. Trends in Proportion ofWomen as Authors of Medical Journal Articles, 2008–2018.
JAMA Internal Medicine
Gender Codes: WhyWomen Are Leaving Computing , 25–49. John Wiley & Sons.Hayfield, N.; and Huxley, C. 2015. Insider and Outsider Per-spectives: Reflections on Researcher Identities in Researchwith Lesbian and Bisexual Women.
Qualitative Research inPsychology
Why So Few?Women in Science, Technology, Engineering, and Mathemat-ics.
AAUW.Hind, M. 2019. Explaining Explainable AI.
ACM XRDSMagazine
Journal of Research in Gender Studies
Proceedings of the NationalAcademy of Sciences
Nature Machine Intelligence
VentureBeat .Johnson, K. 2020b. Google AI Ethics Co-Lead Timnit Ge-bru Says She Was Fired over an Email.
VentureBeat .Kearns, M.; Neel, S.; Roth, A.; and Wu, Z. S. 2019. An Em-pirical Study of Rich Subgroup Fairness for Machine Learn-ing. In
Proceedings of the ACM Conference on Fairness,Accountability, and Transparency , 100–109.Kerstetter, K. 2012. Insider, Outsider, or SomewhereBetween: The Impact of Researchers’ Identities on theCommunity-Based Research Process.
Journal of Rural So-cial Sciences
National Science Foun-dation, January
ACM Transactions onComputing Education
VentureBeat .Kurenkov, A. 2020. Lessons from the PULSE Model andDiscussion.
The Gradient .Labaree, R. V. 2002. The Risk of ‘Going Observationalist’:Negotiating the Hidden Dilemmas of Being an Insider Par-ticipant Observer.
Qualitative Research
Annual Review of Sociology
Computer ScienceEducation
Harvard Educational Review
Proceedings of the Con-ference on Fairness, Accountability and Transparency , 107–118.Menon, S.; Damian, A.; Hu, S.; Ravi, N.; and Rudin, C.2020. PULSE: Self-Supervised Photo Upsampling via La-tent Space Exploration of Generative Models. In
Proceed-ings of the IEEE/CVF Conference on Computer Vision andPattern Recognition , 2437–2445.Mercer, J. 2007. The Challenges of Insider Research in Ed-ucational Institutions: Wielding a Double-Edged Sword andResolving Delicate Dilemmas.
Oxford Review of Education
American Journal of Sociology
American Behavioral Scientist
Artificial Intelligence
Ethnic and Racial Stud-ies
Proceedingsof the ACM Conference on Fairness, Accountability, andTransparency , 220–229.Mohamed, S.; Png, M.-T.; and Isaac, W. 2020. DecolonialAI: Decolonial Theory as Sociotechnical Foresight in Arti-ficial Intelligence.
Philosophy & Technology
33: 659–684.Muller, C. B. 2003. The Underrepresentation of Women inEngineering and Related Sciences: Pursuing Two Comple-mentary Paths to Parity. In
Pan-Organizational Summit onthe US Science and Engineering Workforce: Meeting Sum-mary , 119–126. National Academies Press.National Academies of Sciences, Engineering, andMedicine. 2020.
Promising Practices for Addressing theUnderrepresentation of Women in Science, Engineering,and Medicine: Opening Doors . National Academies Press.National Research Council of the National Academies.2010.
Gender Differences at Critical Transitions in theCareers of Science, Engineering, and Mathematics Faculty .National Academies Press.Ong, M.; Wright, C.; Espinosa, L.; and Orfield, G. 2011. In-side the Double Bind: A Synthesis of Empirical Researchn Undergraduate and Graduate Women of color in Science,Technology, Engineering, and Mathematics.
Harvard Edu-cational Review
SSRNpreprint 3672484 .Patel, A.; Gupta, R.; Harakere, M.; Krishna, S.; Alok, A.;and Liu, P. 2020. Towards Classification Parity Across Co-horts. arXiv preprint arXiv:2005.08033 .Pearson, Jr., W. 1985.
Black Scientists, White Society, andColorless Science: A Study of Universalism in American Sci-ence . Associated Faculty Press.Pearson, Jr., W. 2005.
Beyond Small Numbers: Voices ofAfrican American PhD Chemists . Emerald Group Publish-ing.Pereira, M. d. M. 2012. ‘Feminist Theory is Proper Knowl-edge, But ...’: The Status of Feminist Scholarship in theAcademy.
Feminist Theory
Science, Technology, & HumanValues
Research Policy
ACM SIGCSE Bulletin
The Science Glass Ceiling: AcademicWomen Scientists and the Struggle to Succeed . PsychologyPress.Silbey, S. S. 2016. Why Do So Many Women Who StudyEngineering Leave the Field.
Harvard Business Review .Simard, C.; and Gilmartin, S. 2010. Senior technicalwomen: A profile of success.
Anita Borg Institute for Womenand Technology, California .Sood, G.; and Laohaprapanon, S. 2018. Predicting Race andEthnicity from the Sequence of Characters in a Name. arXivpreprint arXiv:1805.02109 .Spertus, E. 1991. Why Are There So Few Female ComputerScientists? .Srigley, K.; and Sutherland, L. 2018. Decolonizing, Indige-nizing, and Learning Biskaaybiiyang in the Field: Our OralHistory Journey.
The Oral History Review
IEEE Expert
TheOpen Social Science Journal
Journal of Women and Mi-norities in Science and Engineering
American Behavioral Scientist
International Journal of Gender, Sci-ence and Technology
PLoS One
ACM XRDS Magazine
Proceedings of the ICMLWorkshop on AI for Social Good .Vincent, J. 2020. What a Machine Learning Tool that TurnsObama White Can (and Can’t) Tell Us About AI Bias.
TheVerge .Whittlestone, J.; Nyrup, R.; Alexandrova, A.; and Cave, S.2019. The Role and Limits of Principles in AI Ethics:Towards a Focus on Tensions. In
Proceedings of theAAAI/ACM Conference on AI, Ethics, and Society , 195–200.Wright, R. N.; Nadler, S. J.; Nguyen, T. D.; Sanchez Gomez,C. N.; and Wright, H. M. 2019. Living-Learning Commu-nity for Women in Computer Science at Rutgers. In
Pro-ceedings of the 50th ACM Technical Symposium on Com-puter Science Education , 286–292.Xie, Y.; and Goyette, K. 2003. Social Mobility and the Ed-ucational Choices of Asian Americans.
Social Science Re-search
Minerva