How Personal is Machine Learning Personalization?
HH OW P ERSONAL IS M ACHINE L EARNING P ERSONALIZATION ? Travis Greene
Institute of Service ScienceNational Tsing Hua UniversityHsinchu, Taiwan [email protected]
Galit Shmueli
Institute of Service ScienceNational Tsing Hua UniversityHsinchu, Taiwan [email protected]
December 25, 2019 A BSTRACT
Though used extensively, the concept and process of machine learning (ML) personalizationhave generally received little attention from academics, practitioners, and the general public. Wedescribe the ML approach as relying on the metaphor of the person as a feature vector and contrastthis with humanistic views of the person. In light of the recent calls by the IEEE to consider theeffects of ML on human well-being, we ask whether ML personalization can be reconciled withthese humanistic views of the person, which highlight the importance of moral and social identity.As human behavior increasingly becomes digitized, analyzed, and predicted, to what extent doour subsequent decisions about what to choose, buy, or do, made both by us and others, reflectwho we are as persons? This paper first explicates the term personalization by considering MLpersonalization and highlights its relation to humanistic conceptions of the person , then proposesseveral dimensions for evaluating the degree of personalization of ML personalized scores. Bydoing so, we hope to contribute to current debate on the issues of algorithmic bias, transparency,and fairness in machine learning. keywords:
Person, feature vector, user embedding, GDPR, moral identity, digital identity, humanistic view, AI
Explication is elimination: We start with a concept the expression for which is somehow trouble-some; but it serves certain ends that cannot be given up. - John RawlsPersonalized products and services are an inescapable fact of modern life. From personalized predictions (UberETA, recidivism risk scores, credit scores), to personalized recommendations (Netflix, Spotify, Amazon, Tinder),personalized treatments (medicine and psychiatry), personalized ads, offers, prices, and more, we are inundated withpersonalized scores based on data about our physical condition, genetic makeup, location, measurable behaviorsand interactions. Ideally, personalization reduces information overload and search costs, thereby improving decisionmaking and user experience. The decision maker and scored individual can be the same (self-directed) as inrecommender systems, or different (other-directed) as in decision support systems used in law, medicine, business,and finance. In both cases, personalization relies on machine learning (ML).Nevertheless, ML personalization remains an ambiguous and under-examined concept. A literature search reveals asurprising lack of consensus on its essential characteristics. It is either not explicitly defined, explained circularly,or used in relation to or interchangeably with customization , tailoring , or precision marketing . Cremonesi et al.(2010) say what personalization is not : giving "any user a pre-defined, fixed list of items, regardless of his/herpreferences." Likewise, in the paper "What Is Personalization?" Fan and Poole (2006) classify various approachesto personalization. Despite offering a useful taxonomy of ideal personalization types, the authors overlooked asimple, but fundamental point: the user is a person . Thus, critical questions arise: Does merely assigning a uniquescore to each user constitute a personalized score? Is a score personalized when it is based (wholly or in part) on thebehavior of other persons? Does it matter who these "others" are? Does personalization require personal data ?Such questions are critical because lawmakers, journalists, social scientists, managers, and the general publicencounter ML personalization everywhere, yet each group may have a very different understanding of the process.These differences can lead to confusion about the origin and accuracy of so-called "personalized" scores. This is We refer to recommendations, predictions, and treatments more generally as "scores." a r X i v : . [ s t a t . M L ] D ec specially relevant in light of recent regulations, such as the General Data Protection Regulation’s (GDPR) right toexplanation for cases of automated profiling and its inclusion of a right to be forgotten .This paper contrasts the process of ML personalization with the humanistic concept of a person and offers somenormative criteria for evaluating the level of personalization. A clearer understanding of ML personalization willalso contribute to current debates on algorithmic bias, transparency, and fairness in machine learning. Most commercial personalization engines (e.g. recommender systems) rely on a combination of explicitly-givenpreferences, demographic data, and observed behavior assumed to reflect one’s underlying preferences, (i.e.,"implicit ratings") such as purchases, repeated use, save/print, delete, reply, mark, glimpse, query (Nichols, 1998).Such data are derived only from what is actually observed when using an application or device and are a smallsubset of our possibly observed behaviors. Further, this narrow subset of recorded behaviors must be convertedinto a digital (database) representation supporting ML algorithms. Information is lost during the transformationof unstructured data (text, pictures, images, videos) into structured data which can be represented in matrix form.Lastly, the database representation is projected into feature space . Figure 1 illustrates the ML pipeline resulting inthe operationalization of a person as a feature vector.A key idea in ML personalization rests on the metaphor of the person as a feature vector . A feature vector (alsoknown as a user embedding ) is an array of numbers–each representing some descriptive feature or aspect of anobject–which can be thought of as the axes of a coordinate system (Kelleher et al., 2015). A 10-dimensional featurevector of a person, for instance, represents a person as an array of 10 numbers, derived from measurements of theirobserved behavior, and replaces the "person" with a single point in 10-dimensional feature space. Once converted toa point in feature space, the "similarity" of this person to others can be computed by measuring the distance betweenthis point and others in the feature space. Points closer to each other are deemed more similar. Depending on one’smodeling and predictive goals, this can indeed be a useful approach. But like any good metaphor, the feature vectorapproach emphasizes certain similarities at the expense of certain dissimilarities. When we represent a person as afeature vector, we tend to forget the long abstraction process that preceded it. As just one example of a potentialproblem in this process, Barocas et al. (2017) remark, "[numeric] features incorporate normative and subjective. . . assumptions about measurements, but all features are treated as numerical truth by ML systems." In short, threekey characteristics of ML personalization arise from the above process: ML personalization 1) is highly behavioristin its assumptions, 2) is extremely narrow in predictive scope, and 3) often relies on data of a "community" whoseselection is based on predictive goals.Figure 1: From humanistic concepts of the person to personalized scores: ML operationalization requires narrowinga person embedded in social and cultural space to a feature vector embedded in feature space.
Three Characteristics of ML Personalization: Behaviorism, Predictive Scope, and Community
First,
ML personalization is strongly behaviorist . In the behaviorist worldview, "reality" is only what can beobserved (Van Otterlo, 2013). Behaviorism seeks to eliminate the messy theoretical causal mechanisms of humanbehavior (beliefs, intentions, goals, etc.) and focus instead on what can be measured and recorded (Skinner, 1965).ML personalization takes this idea one step further by focusing on predictive accuracy rather than on causal factorsleading to the observed behavior (Rudin, 2019).In the representation of the person as a feature vector, "important" attributes are only those that contribute to thealgorithm’s predictive accuracy. The predictive goal of ML personalization leads to valuing certain aspects ofhuman behavior more than others; for example, behaviors most amenable to measurement tend to be recorded.This leads to a form of selection bias in the representation of the person. Unpredictable behavior is deemed noise,and measured behaviors which do not add to predictive accuracy are deemed redundant. Extraneous features can2ead to the "curse of dimensionality" and are removed in order to preserve model parsimony, storage space, andcomputational efficiency. One example of how ML personalization favors parsimony is the use of unsuperviseddimension reduction techniques, such as Singular Value Decomposition (SVD). These techniques rely on thediscovery that many of our measured behaviors are simply linear combinations of other behaviors and can thus bediscarded with little information loss. We will see how issues of parsimony, scale, and computational efficiencyconstrain the degree of personalization when we examine Instagram’s Explore, which must select relevant contentfor a focal user from hundreds of millions of other potential users’ contributions.As we discuss in Section 4, the ML approach is at odds with a humanistic view of the person conceived as having aninner and outer self. As persons, our self-conceptions are highly bound up with our reasons for action, and thesereasons may stem from our identity. If the reasons we give for our actions are indeed causes of our action, as manyphilosophers believe (see, e.g., Davidson, 1963), then we lose an important aspect of the human experience byfocusing solely on prediction. The representation of a person as a feature vector abstracts a "person" away fromher socio-cultural network of meanings, identities, and values and represents her as a precisely-defined digitalobject in order to predict a very specific behavior, limited to a specific application (see Figure 1). Human societiesevolve over time and vary in their interpretations of codes of conduct regulating behavior (i.e., morality); yet, dueto their abstract generality, mathematical objects and numbers do not. Further, the definitions and properties ofmathematical objects remain fixed as a formal public system of rules, whereas human morality is an informal publicsystem of rules of conduct in which the meaning of behavior is often unclear and evolving (Gert, 1998). This is onepossible source of the disconnect in the feature vector representation of the person used in ML personalization.Second,
ML personalization focuses on predicting a very narrow set of possible behaviors, often limited by thecontext of the application . This can make it seem more powerful and accurate than it really is, especially whenpredictive performance is evaluated. For example, Yeomans et al. (2019) compared the predictions by friends andspouses to those from a simple collaborative filtering (CF) system for predicting a focal user’s ratings of jokes. Thestudy found the basic CF system was able to predict more accurately than a friend or spouse. Yet, the experimentincluded a set of 12 jokes pre-selected by the researchers. The much more difficult problem of selecting 12 jokesfrom a nearly infinite set of possible jokes in all cultures and languages was left to the humans. In essence, theresearchers had already personalized a list of jokes to each subject in the study, given their linguistic background,country of origin, and current location. Once narrowed to such a small recommendation space, the algorithm’sperformance appears quite impressive, but nevertheless hides the fact the hardest task had already been done byhumans. A similar argument can be made for personalization on e-commerce sites: by going to a website, a personhas already self-selected into a group who would be interested in products offered by the website. Lastly, recidivismprediction in the judicial setting provides another example of the narrow predictive goal of ML personalization.Small changes in the definition of recidivism and its time-horizon– Is it one, two, or three years in the future? Doesit include misdemeanors or only felonies?–will drastically affect the algorithm’s predicted scores and performance.Third,
ML personalization uses data not only from the focal user, but also from other users . This is clear in thecase of recommendation systems using social network data, where the interests and preferences of a focal usercan be inferred from their direct and indirect connections to other users and groups. But this point is less obviouswhen when the data are from measured behaviors on an application or device. Though recommendations basedon data from other users could lead one to believe that one’s personalized scores are not, in fact, "personal," inthe sense of stemming from one and the same person, the real issue is whether the "community" chosen reflectsone’s chosen social identity. In other words, personalization can still occur when the personalized scores are basedon the behavior of others sharing the same social identity as the focal user. However, this is unlikely to occur inmany ML contexts because "nearest neighbor" threshold values are chosen in order to optimize a given error metric,such as RMSE or precision/recall/top-N, not on the basis of any shared social identity (Cremonesi et al., 2010).Recommendations based on the behavior of others who do not share a similar social and moral identity with thefocal user are less personalized under our conception of personalization–all things equal. This principle also impliesthat when dimension reduction techniques are used, such as PCA and SVD (which rely on calculations of totalvariance across the entire user-item matrix), the selection of a "community" of users making up the user-item matrixis important. Personalized scores using latent factors derived from users who do not share a similar social or moralidentity are less personalized in our view. Of course, determining who is truly "similar" to us is a difficult question,but it is one where the methods and concepts of the humanities and social sciences may play an important role.
ML Personalization Case Study: Instagram’s Explore
The personalization system of the popular social network Instagram shapes the desires, preferences, and self-conceptions of millions of people every day. By determining which content is displayed to which users, socialmedia personalization systems may play an important role in the formation and maintenance of one’s online andoffline identity (Helmond, 2010). When scaled to hundreds of millions of users, ML personalization can evenhave society-wide effects, shifting opinions towards political issues and candidates, as in the 2016 US presidentialelection. For this reason, we believe it is important to see how ML personalization relates to the "person" and itstacit assumptions regarding observed behavior. We therefore briefly describe Instagram’s Explore to illustrate theperson as a feature vector approach and the implications discussed in the previous section.3he complicated set of algorithms used for Instagram’s Explore recommendations essentially work by creating"account embeddings" in order to infer topical similarity. Roughly put, a feature vector (FV) representation of anInstagram user is derived from the sequence of account IDs a user has visited. Next, the FV is used to determine a"community" of similar accounts. Similarity in this space implies topical similarity, assuming users do not randomlyexplore different topics, but instead tend to view multiple accounts offering similar content. Once the candidateaccounts (the nearest neighbors) have been found, a random sample of 500 pieces of eligible content (videos, stories,pictures, etc.) is taken and a "first-pass" ranking is made to select the 150 "most relevant" pieces of content for afocal user. Then a neural network narrows the 150 to the top 50 most relevant. Notice that at each stage, possiblecontent is narrowed-down and that computational efficiency is a high priority. Finally, a deep neural network is usedto predict specific actions (narrowed only to those possible on the Instagram app) for 25 pieces of content, such as"like," "save," or "see fewer posts like this." Content is then ranked based on a weighted sum of these very specificbehaviors permitted in the app and their associated predicted probabilities. Weights are not determined by users;they are determined by the system engineers and reflect their assumptions about the users’ intentions. Assuming auser wishes to discover new topics and avoid seeing content from the same users, a simple heuristic rule is used todownrank certain content and diversify the results. After applying the downranking procedure, the content with thehighest weighted sum in the "value model" is displayed in decreasing order on the focal user’s Explore page.
As described and illustrated in Section 2, personalized scores are generated by combining information about yourobserved behavior with certain assumptions about you and your goals, beliefs, and preferences to predict a veryconstrained set of future actions. But which information should constitute me as a person, and from whose viewpoint ?These are difficult philosophical questions. To answer we might first ask, "What is a person?" But this question isproblematic because "person" is often defined in such a way as to justify a particular world-view. It can variouslymean "human animal,” "moral agent,” "rational, self-conscious subject”, "possessor of particular rights,” and "beingwith a defined personality or character" (Schechtman, 2018). Nevertheless, Western thought has focused on twogeneral aspects to the person, one of which is revealed through the etymology of the word itself."Person" comes from the Latin persona , which derives from the Ancient Greek word for a type of mask worn bydramatic actors. For the Ancient Greeks, the idea of a person was inherently connected to context and role. One’spersona was a specific kind of self-identity that was public, socially defined, and varied depending on context.In time, the outward-facing Greek conception was complemented by a Christian emphasis on self-reflection andawareness, resulting in a focus on the human capacity for rational introspection (Douglas and Ney, 1998). Laterthinkers expanded the idea of the person into two main parts: an outer personnage of public roles and masks, andan inner conscience, identity, and consciousness. In short, Western thought has generally viewed the person asconsisting in two co-existing inner and outer domains.We note that the word personalization contains the adjective personal , implying that personalized scores should, atleast in part, be based on personal data . For example, Liang et al. (2006) write that personalization is "a process ofcollecting and using personal information to uniquely tailor products, content and services to an individual." Yet,researchers in ML have largely overlooked the crucial connection between personalized scores and legal definitionsof personal data. A source of complexity is that different legal regimes define personal data differently. For instance,the most influential data protection law at the moment, the EU’s GDPR, defines personal data as "any informationrelating to an identified or identifiable natural person (‘data subject’)" (Article 4). Such a broad definition meanspersonal data could constitute anything from browser cookies, to location data, to even a combination of non-sensitive measurements, if there are sufficiently few or unique observations to single out individuals. Ultimately,context determines whether data are personal data (Greene et al., 2019).Consent is another legal issue in the generation of personalized scores, especially when sensitive data such asrace, gender, or political affiliation are involved. The GDPR’s reliance on explicit consent has important moralimplications. Consent implies choice in how one represents oneself publicly and also which inferences dataprocessors can draw about one’s identity (see, e.g., Kosinski et al., 2013). The GDPR’s right to rectification anderasure (the so-called "right to be forgotten") further entrenches the importance of autonomy in deciding the factualbasis on which one will be judged, classified, and potentially discriminated against, and the extent to which ouridentities as persons are reflected in observations of our public behavior. This notion has been called informationalprivacy (Moreau, 2010; Shoemaker, 2010). In short, the existing literature on ML personalization overlooks twokey points: first, that it depends on differing legal definitions of personal data; and second, informational privacyconcerns are generally not considered in the design of personalized systems. Example taken from ai.facebook.com/blog/powered-by-ai-instagrams-explore-recommender-system/ Identity, Self, and Data Regulation
ML researchers and practitioners normally deal with issues of “practical identity,” while philosophers and socialscientists deal with issues of "moral" and "social" identity (De Vries, 2010; Manders-Huits, 2010). Recently,humanistic scholars have focused on a new, third kind of identity arising from the difficulties in the formation of asingle, unitary digital identity in light of rapidly evolving technology. The nature of distributed storage and collectionof observable behavioral data related to living persons further complicates the process of identity formation (see"Digital reality" in Figure 1). These distributed digital representations of our practical, moral, and social identitieshave been variously termed data citizens , data doubles , data shadows and digital subjects (Goriunova, 2019).The narrowing from reality, to observed reality, to digital reality (Fig 1) raises questions about the relation betweenpersons, personal data, and personalized scores. While many of today’s ML algorithms generate different scoresgiven different contextual input data, these scores can fail to reflect our moral identities . For example, scores basedon categorical features may not reflect one’s social or personal identity. Similarly, explanations for personalizedscores at the algorithm or model level (e.g., variable importance scores) may fail to reflect the salience of beliefs,motivations, and intentions we have when explaining our behavior to others. An explanation of a personalizedscore that leaves out these important aspects of one’s moral identity surely cannot be said to relate to one asa person . Though we cannot control how society sees us, we can choose to (partly) accept or reject society’scategorization of us. This latter aspect of human experience remains largely unaccounted for in ML personalization.As Manders-Huits (2010) argues, "The challenge [of] justice to data subjects as moral persons is to take into accountthe self-informative perspective that is part of ‘identity management.’"We now provide a brief overview of identity theories and humanistic perspectives drawing mainly from social andpersonality psychology, consumer behavior, sociology, philosophy, economics, information systems, and recentdata protection law. We hope exposure to these ideas can influence future thinking on and design of ML systemsgenerating personalized scores. Table 1 summarizes key concepts related to the person from these disciplines. Identity from a Social Science Perspective
Theories and concepts of self and identity are fundamental topics in psychology, despite being difficult to preciselydefine. Personality psychologists Larsen and Buss (2009) define personality as a an organized set of relativelyenduring psychological traits that guide a person’s interactions with her "intrapsychic, physical, and social environ-ments." These traits can be expressed as the "Big Five" dimensions of personality: Openness to New Experience,Conscientiousness, Extraversion, Agreeableness, and Neuroticism (also known as OCEAN), and have been shownto have varying levels of predictive power. Personality psychology has also begun to study the formation of moralidentity . For example, Aquino et al. (2002) show how highly important moral identities–the collection of certainbeliefs, attitudes, and behaviors relating to what is right or wrong–can provide a basis for the construction of one’s"self-definition." Finally, social psychologists define the self as an interface between the biological processes ofthe body and the larger social and cultural context. Further, they highlight the power of the cultural context indefining identity: "Without society, the self would not exist in full" (Baumeister and Bushman, 2014, p. 74). Asthe summaries above illustrate, understanding the person requires description at multiple levels. McAdams (1995)presents an influential multi-dimensional theory of the person, focusing on traits, personal concerns, and life stories.According to identity theorists in sociology and psychology, the self is fluid and occupies multiple social roles(identity theory) or group identities (social identity theory) that coexist and vary over time. The dynamism of theself-concept is reflected in its numerous sub-components or self-representations: the past, present, and future self;the ideal, "ought," actual, possible, and undesired self (Markus and Wurf, 1987). Identity theorists highlight how aunified self-conception arises from the variety of meanings given to various social roles the self occupies (Strykerand Burke, 2000). Common types of social identities relate to one’s ethnicity, religion, political affiliation, job, andrelationships. People may also identify strongly with their gender, sexual orientation, and various other "stigmatized"identities, such as being homeless, an alcoholic, or overweight (Deux 2001). In short, social psychologists generallyagree one’s self-concept is one of the most important regulators of one’s behavior.The self-concept has also influenced postmodernist consumer behavior research. Consumers now take for grantedtheir sometimes paradoxical identities, beliefs, and behaviors in everyday life (Fuat Firat et al., 1995). In thepostmodern condition one "listens to reggae, watches a Western, eats McDonald’s food for lunch and local cuisinefor dinner, wears Paris perfume in Tokyo and retro clothes in Hong Kong" (Lyotard, 1984).
Identity, the Self, and Morality from a Philosophical Perspective
How do philosophers understand the link between the identity and the self? For some, the self arises from anidentification of structural sameness over time, experienced as a kind of narrative (Ricoeur, 1994). Though individualcells of our bodies are constantly renewed, a person’s essential structure nevertheless remains similar enough tobe identified as the "same" over time. For others, such as Taylor (1989), social and communal life link the twoconcepts. Taylor (1989) argues that one’s identity is tied to one’s moral values and a defining social community. He Mischel et al. (2007) show that knowing an individual’s Big Five trait profile allows for only weak prediction of a particularbehavior in a particular context
Philosophy & Humanities Psychology, Sociology & Con-sumer Behavior Economics & InformationSystems Data Regulation & Profes-sional Codes of Conduct
Moral identity Personality: Big 5 Identity explains irrational Right to be forgottenCommunity-defined identity Socially-defined identity behavior Right to rectificationPursuit of the good life Culturally-influenced narratives Digital identity Automated profilingThe Postmodern experience Emotions and moods Online identity affects Informational privacyNarrative identity Hermeneutic interpretation behavior Agency over digital identityInner and outer-facing self Meaning of symbols, texts, images Identity as economic choice Ethically-aligned design writes, "I define who I am by defining where I speak from, in the family tree, in social space, in the geography ofsocial statuses and functions, in my intimate relations to the ones I love, and also crucially in the space of moral andspiritual orientation within which my most important defining relations are lived out." Our identities therefore reston the constant interplay between how others view us and how we view ourselves. These "community" identitiesfurther entail various moral obligations to others that influence and motivate our behavior .Atkins (2010) draws on work by Ricoeur and Korsgaard to explain the connection between morality and identity.She claims that "one’s identity is the source of one’s moral agency, expressed in one’s normative reasons." One’sidentity constitutes "the condition of the possibility for having a perspective from which to perceive, deliberate, andact . . . it is a condition of the possibility of morality." For both Taylor and Atkins, the development of a self-identityprovides a moral scaffolding upon which possible actions can be evaluated; our identities are critical for embeddingus both in the physical and social worlds, and adding a moral dimension to our actions as persons.In Schechtman (2018)’s influential narrative self-constitution view, personhood and personal identity are fleshed outin terms of a unified and "unfolding developmental structure,” analogous to a complex musical sonata. Echoing theideas of Ricoeur, she writes that human life "is a structural whole that has, by its very nature, attributes that apply toit as a whole which do not necessarily apply to each individual portion." Through the unity of experience–whichtakes the form of a narrative–the same person can be an infant at one time and later in life suffer from dementia. Hernotion of personal identity is similar to McAdams’ psychological view in that it explains via a diachronic processhow identity can evolve over time. These views highlight the processual, dynamic, and emergent nature of the self.
Identity in Information Systems (IS) and Economics
As the World Wide Web grew in popularity, scholars in the field of IS began to examine its impact not juston the organization, but on the individual. Some were influenced by postmodern arguments and pushed for areconsideration of such fundamental concepts as space, time, the world, and the self (Introna and Whitley, 1996).Erickson (1996), for example, foreshadowed the rise of social media when he examined the "portrayal management"of online identities afforded by personal webpages. The anonymity of the Internet meant that individuals couldselectively represent themselves to others without the complexities of physical contact. Nearly two decades later,the study of online communities and identity formation has entered into the mainstream, leading to novel methodsof data collection and analysis. A growing stream of research in IS now draws on the constructs of identity andsocio-cultural norms to explain online behavior. Burtch et al. (2016), for example, explored how contributing tothe same crowdfunding campaign could foster a common identity in users and influence a campaign’s fundraisingsuccess. Finally, in the more traditional domain of economics, Akerlof and Kranton (2000) presented one of the firstand most influential economic analyses of the role of social and group identity on behavior. Their work illustrateshow decisions about identity can be used to explain apparently irrational behavior, the creation of behavioralexternalities in others, and preference evolution.
Digital Identity in Data Regulation and Professional Codes of Conduct
Around the world, the passage of new data protection laws reflects growing social concern over the process ofturning a person into a feature vector. The GDPR and the California Consumer Privacy Act (CCPA) both seek togive citizens more control over their personal data and their digital identities. The GDPR gives data subjects rightsto opt out of "automated profiling" with "legal" or other "significant" effects. The GDPR’s right to be forgotten canbe viewed as a potential legal remedy to the issue of one’s practical and moral identity diverging over time. Datasubjects have the right to destroy and recreate aspects of their digital identities when they no longer represent themas persons. At the same time, important ML societies, such as the IEEE and ACM, have also voiced concern aboutthe effects of ML on human well-being. For example, recent IEEE guidelines state that autonomous and intelligentsystems should give people access and control over their personal data and allow them "agency over their digitalidentity." The following section thus lays out a basic conceptual framework for use by regulators, professionalgroups, and practitioners to assess levels of personalization. The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems.
Ethically Aligned Design: A Vision forPrioritizing Human Well-being with Autonomous and Intelligent Systems , 1st ed, 2019 https://ethicsinaction.ieee.org Evaluating ML Personalization
In order to better align ML personalization with a humanistic view of the person and current trends in data protectionlaw, we propose six criteria to guide future discussions. We consider the degree of personalization to be on acontinuum and to depend on objective (outer-facing) and subjective (inner-facing) dimensions. As shown earlier inFigure 1, the inner aspects of the person are inaccessible through observational methods. First-person, subjectiveinput from the person herself is therefore crucial in measuring the level of personalization. The social sciences andhumanities are especially well-positioned to contribute to the interdisciplinary development of such measures usingquantitative and qualitative data collection and analysis methods. Lastly, legal experts will also be needed to relatethese measures to data subjects’ rights under current data protection laws, such as the GDPR.
Objective dimensions include (1) a personal data component - the extent to which legally-defined personal data areused, and (2) a uniqueness component - the percent of other users sharing the same input data, personalized score, ordiscretized recommendation. The personal data component implies that legally-allowable degrees of personalizationmay vary by country and legal regime. Regarding the uniqueness component, it is not clear how to weight thesethree aspects in determining the degree of personalization. Numeric measurements may uniquely identify youmore so than categorical measurements, but these categories may more closely reflect your personal identity andcommunity. Similarly, it is not obvious how to reconcile dimension reduction techniques with personalization. Forexample, when highly morally-salient measurements are correlated to other measures, should they be removed?Another objective dimension worth examining is (3) an accuracy component - the accuracy of predicted scores. Itseems reasonable to assume personalized predictions should be more accurate than non-personalized predictions.But how should accuracy be measured? As we noted above, the very narrow predictive goal of ML personalizationcan obscure the interpretation of some accuracy metrics. An easy way to improve accuracy is simply to narrow thesize of the "recommendation set." However, in some cases, accuracy may not be desired; some persons may preferto trade off predictive accuracy (in the narrow sense above) for informational privacy. As noted earlier, data subjectshave basic rights to informational privacy under the GDPR.
Subjective dimensions include (4) a self-determination component - the extent to which personalized scores respectthe data subject’s right to determine his public self, captured in the notion of informational privacy. This subjectivecomponent is relevant to debates on fair and transparent ML (FATML). Current legal and ML approaches to fairnessoperate on the level of the "database representation" of the person (i.e., practical identity), overlooking the person’smoral identity. Thus we propose a (5) right reasons component - fair decisions should capture the person’s intendedpublic self and associated goals, attitudes, meanings and motivations in that context. The relevance of reasonsgiven in algorithmic explanations can be assessed, for example, using relevance feedback techniques already inuse in information retrieval. Lastly, another subjective dimension is (6) a moral importance component - themoral importance of the personalization context. For example, how should the different subjective and objectivecomponents be weighted when assessing a system that recommends a movie versus a bail amount?Finally, our proposed list of evaluation components is by no means exhaustive. In fact, some of these componentsmight interact with or even contradict other aspects of personalization. For example, serendipitous recommendations (Ge et al., 2010) can stimulate evolution in one’s moral identity, yet might conflict with the accuracy component .Allowing users to decide for themselves how they would like to weight these aspects may be one solution. Going forward, we believe that as ML personalization becomes increasingly rooted in daily life, we should considernew approaches to data collection, such as those used in qualitative social science (e.g., hermeneutics, discourseanalysis, fuzzy-set theory). By incorporating more diverse forms of personal data into personalized scores, we maybe able to reduce the gap between one’s identity as a person embedded in social and cultural space and as a featurevector embedded in feature space. As we emphasize, however, the ML personalization process will never be perfect.Our presentation of the ML pipeline can also be expanded to consider further ML operations, such as how andwhich data are used for training algorithms. For example, personalized scores for a given person are computed usingmodels that were trained on data from other persons, typically excluding that person’s own data. Chronology of thetraining set is also important when considering the dynamism of a person’s self-concept.Lastly, these evaluation dimensions are merely an entry point into an interdisciplinary discussion about personaliza-tion. We hope to initiate and contribute to such a discussion by describing what the process of ML personalizationtypically looks like, particularly its defining metaphor of the person as feature vector. We argued that this conceptionis radically different from that found in the humanities, social sciences, and law. As the long-term effects of MLpersonalization on personal identity, politics, law and society are still unclear, it is important to critically examine theprocess of ML personalization. Perhaps it cannot be precisely defined, but an examination of its key characteristicsand assumptions may help foster new insights on the issues of algorithmic bias, transparency, and fairness. Serendipity is used in Instagram’s "downranking" procedure, under the assumption that one’s preferences should not remainstatic and so new topics should be presented in order to provide diverse personalized content. cknowledgements We thank Ching-Fu Lin, Jack Buchanan, Mariangela Guidolin, Kellan Nguyen, and Boaz Shmueli for their helpfulcomments on an earlier draft of this paper.
References
Akerlof, G. A. and Kranton, R. E. (2000). Economics and identity.
Quarterly Journal of Economics , 115(3):715–753.Aquino, K., Reed, I., et al. (2002). The self-importance of moral identity.
J of personality and soc psych , 83(6):1423.Atkins, K. (2010).
Narrative identity and moral identity . Taylor & Francis.Barocas, S., Hardt, M., and Narayanan, A. (2017). Fairness in machine learning.
NIPS Tutorial .Baumeister, R. F. and Bushman, B. (2014).
Social psychology and human nature . Belmont, CA: Cengage Learning.Burtch, G., Ghose, A., and Wattal, S. (2016). Secret admirers: An empirical examination of information hiding andcontribution dynamics in online crowdfunding.
Information Systems Research , 27(3):478–496.Cremonesi, P., Koren, Y., and Turrin, R. (2010). Performance of recommender algorithms on top-n recommendationtasks. In
Proceedings of the fourth ACM conference on Recommender systems . ACM.Davidson, D. (1963). Actions, reasons, and causes.
The journal of philosophy , 60(23):685–700.De Vries, K. (2010). Identity, profiling algorithms and a world of ambient intelligence.
Ethics and informationtechnology , 12(1):71–85.Douglas, M. and Ney, S. (1998).
Missing persons: A critique of the personhood in the social sciences , volume 1.Univ of California Press.Erickson, T. (1996). The world-wide-web as social hypertext.
Communications of the ACM , 39(1):15–17.Fan, H. and Poole, M. S. (2006). What is personalization? perspectives on the design and implementation ofpersonalization in information systems.
Journal of Org. Comp. and Elec. Comm. , 16(3-4):179–202.Fuat Firat, A., Dholakia, N., and Venkatesh, A. (1995). Marketing in a postmodern world.
European journal ofmarketing , 29(1):40–56.Ge, M., Delgado-Battenfeld, C., and Jannach, D. (2010). Beyond accuracy: evaluating recommender systems bycoverage and serendipity. In
Proceedings of the fourth ACM conference on Recommender systems . ACM.Gert, B. (1998).
Morality: Its nature and justification . Oxford University Press on Demand.Goriunova, O. (2019). The digital subject: People as data as persons.
Theory, Culture & Society , 36(6):125–145.Greene, T., Shmueli, G., Ray, S., and Fell, J. (2019). Adjusting to the gdpr: The impact on data scientists andbehavioral researchers.
Big data , 7.Helmond, A. (2010). Identity 2.0: Constructing identity with cultural software. In
Proceeding of Mini-conferenceinitiative, University of Amsterdam (Amsterdam, ND–Jan 20-22 2010) . Citeseer.Introna, L. D. and Whitley, E. A. (1996). Information systems as a social science? the individual perspective.
Information Systems , 8:16–1996.Kelleher, J. D., Mac Namee, B., and D’arcy, A. (2015).
Fundamentals of machine learning for predictive dataanalytics: algorithms, worked examples, and case studies . MIT Press.Kosinski, M., Stillwell, D., and Graepel, T. (2013). Private traits and attributes are predictable from digital recordsof human behavior.
Proceedings of the National Academy of Sciences , 110(15):5802–5805.Larsen, R. and Buss, D. M. (2009).
Personality psychology . McGraw-Hill Publishing.Liang, T.-P., Lai, H.-J., and Ku, Y.-C. (2006). Personalized content recommendation and user satisfaction: Theoreti-cal synthesis and empirical findings.
Journal of Management Information Systems , 23(3):45–70.Lyotard, J.-F. (1984).
The postmodern condition: A report on knowledge , volume 10. U of Minnesota Press.Manders-Huits, N. (2010). Practical versus moral identities in identity management.
Ethics and informationtechnology , 12(1):43–55.Markus, H. and Wurf, E. (1987). The dynamic self-concept: A social psychological perspective.
Annual review ofpsychology , 38(1):299–337.McAdams, D. P. (1995). What do we know when we know a person?
Journal of personality , 63(3):365–396.Mischel, W., Shoda, Y., and Ayduk, O. (2007).
Introduction to personality: Toward an integrative science of theperson . John Wiley & Sons.Moreau, S. (2010). What is discrimination?
Philosophy & Public Affairs , 38(2):143–179.Nichols, D. (1998). Implicit rating and filtering. In
Proc of 5th DELOS Workshop on Filtering and Collab Filtering .Ricoeur, P. (1994).
Oneself as another . University of Chicago Press.Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretablemodels instead.
Nature Machine Intelligence , 1(5):206.Schechtman, M. (2018).
The constitution of selves . Cornell university press.Shoemaker, D. W. (2010). Self-exposure and exposure of the self: informational privacy and the presentation ofidentity.
Ethics and Information Technology , 12(1):3–15.Skinner, B. F. (1965).
Science and human behavior . Simon and Schuster.Stryker, S. and Burke, P. J. (2000). The past, present, and future of an identity theory.
Social Psychology Quarterly ,63(4):284–297.Taylor, C. (1989).
Sources of the self: The making of the modern identity . Harvard University Press.8an Otterlo, M. (2013). A machine learning view on profiling.
Privacy, Due Process and the ComputationalTurn-Philosophers of Law Meet Philosophers of Technology. Abingdon: Routledge .Yeomans, M., Shah, A., Mullainathan, S., and Kleinberg, J. (2019). Making sense of recommendations.