Approaching Ethical Guidelines for Data Scientists
aa r X i v : . [ s t a t . O T ] J a n Approaching Ethical Guidelines for DataScientists
Ursula GarzcarekCytel Inc, Clinical Research Services ICCRoute de Pré-Bois, 20 C.P. 1839, 1215 Geneva 15, [email protected] SteuerHelmut-Schmidt-Universität, Universität der Bundeswehr HamburgHolstenhofweg 85, 22043 Hamburg, [email protected] updated January 16, 2019
The goal of this article is to inspire data scientists to participate in the debate onthe impact that their professional work has on society, and to become active in publicdebates on the digital world as data science professionals. How do ethical principles(e.g., fairness, justice, beneficence, and non-maleficence) relate to actual situationsin our professional lives? What lies in our responsibility as professionals by ourexpertise in the field? More specifically this article makes an appeal to statisticiansthat may consider themselves not as data scientists, nor what they do as data science,to join that debate, and to be part of the community that establishes data scienceas a proper profession in the sense of Airaksinen [28], a philosopher working onprofessional ethics. As we will argue, data science has one of its roots in statisticsand at the same time extends beyond it. To shape the future of statistics, and to takeresponsibility for the statistical contributions to data science, statisticians shouldactively engage in the discussions.In Section 1 the term data science is defined, and the technical changes that haveled to a strong influence of data science on society are outlined. In Section 2.1the systematic approach from [39] is introduced. Along the lines of that approachprominent examples are given for ethical issues arising from the work of datascientists. In Section 3 we provide reasons why data scientists should engage inshaping morality around data science and to formulate codes of conduct and codesof practice for data science professionals. In Section 4 we present established ethicalguidelines for the related fields of statistics and computing machinery. Section 51escribes necessary steps in the community to develop professional ethics for datascience. Finally in Section 6 we motivate our own engagement and give our startingstatement for the debate:
Data science is in the focal point of current societaldevelopment. Without becoming a profession with professional ethics, data sciencewill fail in building trust in its interaction with and its much needed contributions tosociety!
We start with the definition of data science as given by Donoho which we find very useful. Wewill describe how data science relates to statistics and machine learning and why the role of adata scientists in society is becoming increasingly important.
There is currently no generally agreed definition of data science. Here we use the definition ofDonoho [1] of greater data science:
Data science is the science of learning from data; it studies the methods involved in the analysisand processing of data and proposes technology to improve methods in an evidence-basedmanner. The scope and impact of this science will expand enormously in coming decades asscientific data and data about science itself become ubiquitously available.
Donoho also provides a classification of the related activities into six divisions:1. Data gathering, preparation, and exploration,2. data representation and transformation,3. computing with data,4. data modeling,5. data visualization and presentation,6. science about data science.Items 1 to 5 describe the work of a data scientist, item 6 differentiates what he calls greater datascience from data science.
The lack of an agreed definition of data science is a symptom of a larger problem: it is not (yet)a profession of its own. Some see it as subdivision of machine learning, and thus a subdivisionof artificial intelligence, others as subdivision of statistics, that is exploratory statistics, andmany see it as a collection of methods from both statistics and machine learning, used by peopleof different professional backgrounds, or people with no actual professional background onlytrained in the application of those methods, without the necessary formal scientific education.By starting with the definition of Donoho (sec. 1.1) we already make two statements:2. Data science should become a profession in the sense of Airaksinen [28], with a definition,a grounding in science, and a task and responsibility in society, and2. exploratory statistics is a historical predecessor of data science.With respect to the second point, we do not claim exploratory statistics to be the only predecessorof data science. With the same right, people from the artificial intelligence community can seemachine learning as a historical predecessor of data science. Therefore, we want the machinelearning and artificial intelligence community to work together with the statistics community onthe first point.
The biggest, relatively recent changes in practical data science are the availability of vast amountof data together with the increase in computational power. Technically speaking this enablesfast, low-cost processing of ever-changing large data bases by algorithms to derive continuouslyupdated highly condensed and aggregated data, i.e. results. These results can be fed into humandecision making, that is based on the interpretation and understanding of the results, or they canbe used in rules for automatic decision making. Whether or not, at least interim, the decisionsare made with human understanding of the results and how they were generated, distinguishesblack-box algorithms from other algorithms.Focus of this article are the consequences of processing and analysing vast amounts of dataabout humans and human behaviour. Todays possibilities in these respects change humaninteraction and thus society directly and fundamentally. Examples for this broad claim will begiven in subsequent sections.As data science is the focal point of these developments the role of data scientists in soci-ety becomes more influential and important. With increased influence and importance comesincreased responsibility.
The awareness that data science and its algorithms have an increased and fundamental impact onsociety is vivid around the world. There are ongoing or starting discussions in many countriesand organisations in legal and political context, actually too many to cite. Instead, we refer toany search in news portals, social media and internet with terms as algorithm, impact, society .Actually such considerations are not really new. To our knowledge, the first data scienceapplication recognised to have a large impact on societal processes are election forecasts andpolls on voting behaviour. Many countries have thus regulations on what is allowed to publishwhen in context of an upcoming or ongoing elections. An overview over such regulations isgiven in [18].A systematic approach to identify, describe and categorise those ethical issues was undertakenby CNIL (Commission nationale de l’informatique et des libertés) in 2017 [38, 39]. The reportis the result of a public debate organized by the french data protection authority. We will followits structure and give examples for each of the given categories of ethical issues to make themtangible. The main points relevant for consideration by data scientists are identified.3 .1 Six main ethical issues according to CNIL
In the debate six main issues were identified. Citations referencing [38] are given in front ofeach of the following sections. These ciatations are set in italics to be easily identifiable.
Delegation of complex and critical decisions and tasks to machines increases the human capacityto act and poses a threat to human autonomy and free will and may water down responsibilities.
The most widely discussed application of this type are autonomous vehicles. Autonomousvehicles have the potential to increase traffic safety, but who is responsible for remaining acci-dents? Will it be possible to overrule a machine’s decision on lowest or allowable risk, i.e. incase of an emergency.On a more abstract level any sufficiently complex system may be called an autonomousmachine.Already today many Kafkaesque situations arise due to complex semi-automatic regulations,i.e. the story of a man who was released from his job by an algorithm due to an error, and nohuman was able to stop that procedure [11] after the lay-off was triggered.It must be noted, that in these settings the data scientist is not involved directly. May be she orhe built some model in preparation to steer the machine, but the implementation generally wasnot her or his task. Algorithms and artificial intelligence can create biases, discrimination or even exclusion towardsindividuals and groups of people
General remarks
This issue is one where data science expertise is very important for under-standing the extent of the problem. We start stressing one point that is often overlooked, whenalgorithmic bias is discussed. The very nature of the most commonly applied algorithms, -calledpattern recognition or classification and clustering-, if applied to humans, is applying prejudice .In statistical language they form a prior belief on an individual generated by experience withother individuals assigned to the same group. Goal of these algorithms is the assignment of a newobject, in this case a person, according to some measured characteristics of this person into somegroup. Judgements and predictions on e.g. future behaviour or reactions to a medical treatmentfor the individual are then made according to previously observed behaviours or reactions of theother’s in the group. Obviously, if this leads to an improved medical decision making, this is tothe benefit to the individual and the society at large.In many examples, though, there is a possible benefit to some and a negative impact on others.In those cases, questions of fairness and justice are touched by the use of these algorithms forjudgement/prediction and decision making in general. Any of their use constitute bias , if themeasured characteristics, that lead to the assignment into the group, are only correlated but notcausally related to the features that are judged about. Formally the reason is, that the relationshipbetween what is predicted or judged about for the individual and the measured characteristic of4he individual is conditionally independent given the individual. Note that this bias is createdindependently on whether or not the underlying database is representative for the larger populationfor the measured characteristics. The bias is created by applying an approach (= data + method)that is suitable for correlational analyses only for judgements that require causal reasoning onindividual level.Practically this is not different from humans basing their judgement on a person, on experiences(= data) they have made with other people that are alike based on some arbitrary (that is bearingno causal relationship) assessment on similarity. If this is implemented by an algorithm theimpact can be more severe, as the identical bias is applied to more people and forms a moresystematic bias towards certain groups. Combined with monopolies on data ownership, - likecurrently for social media or search data -, and with the scalability of computing power such asystematic bias can easily become a universal norm. Where the algorithm uses characteristicsthat include or are related to protected characteristics by anti-discrimination laws (mostly race,sexual orientation, religion or belief, age and disability) any judgement and any decision basedon the algorithm constitute instances of discrimination , when they result in one person beingtreated less favourably than another in a comparable situation.This does not happen only in badly designed or malfunctioning systems. It is in the core of allclassification applied to people.Another, -practically incurable-, drawback of those algorithms is that they infer from data ofthe past, - on the members of the group and/or the individual on which one wants to judge -, andhuman behaviour on an individual level and their patterns do change over time.
Examples
The probably most famous example is COMPAS (Correctional Offender Manage-ment Profiling for Alternative Sanctions) a software used in the US judicial system to classifythe probability of defendants’ recidivism. A good discussion of the approach can be found in[27]. It was shown in a detailed analysis [5, 6] that the privately owned algorithm used in thejuridical system gave far better prognoses for white than for black people, thus it discriminatedimplicitly based on color. The machine generated prognosis was intended just to help the judges,but in interviews it could be seen, that it played a crucial rule in the judgements. Especiallydecisions by the judges whether defendants could get out on parole or had to go to jail werestrongly influenced by the algorithm’s output and discriminated against black people.It must be stressed, that this bias in application was not intentional as far as it is known.The bias most probably was introduced through available data on prisoners in conjunction withthe above described fundamental misunderstanding that observed correlations would be goodenough to make decisions that require causal reasoning.Examples of the application of algorithms are not restricted to the US. In Europe for examplethere is a recent initiative in Austria to classify unemployed people in one of the three possiblegroups: bad (<= 25%) , mediocre or good chances (>=66%) to be employed for at least 6 monthsin 24 months from now [19]. The idea is to spend money to bring people back into the workforcemore on target. Controversial is the stated goal to spend less money on those in the lowest group.It is reported that age and nationality increase one’s probability to be put in the lowest group.Both points seem to be openly discriminating. The official stance is, that the algorithm does notdecide, but only helps a human to decide and therefore no discrimination would happen. This is5gnoring to the large influence that those supportive systems have, when there is a shortage ofmoney: decision makers typically need to justify, if they deviate from the algorithmic choices,but not if they follow the machine’s decision. The default mode of operation may change throughthe use of such a simple helper algorithm.A very similar system is already in use in Poland [20].In the examples given, in addition to generating bias, the automatic classifiers act like self-fulfilling prophecies. The automatic, even secret, classification of an individual will influence hisor her future life, in the direction the chosen algorithm determines. At the same time it becomesimpossible to assess the algorithms performance in the future, as the future of the individual’slife is changed based on the algorithms outcome and there is no control group.Also the algorithms act very similar to ancient oracles. For an outsider it is impossible tofind out which characteristics of a person exactly have led to the given classification. They areblack-box algorithms, a feature shared by many of the algorithms from the artificial intelligencecommunity. There only is the saying of the oracle, no reasoning, and no possible recourse.Black-box algorithms therefore will always be problematic for usage in any juridical system orfor any scoring implying a value judgement of an individual, i.e. credit scoring.These applications are examples for applications where some people have a benefit and othersnegative consequences from the application of the algorithm. It is accepted, that the applicationmay be not in the interest of the individual that is judged.Of course this is not a drawback inherent in using algorithmic decision making. It is possibleto set up procedures with no intention to inflict negative consequences on some to the benefitof others, if care is given to transparency and possible discriminating behaviours. For examplein Germany there exists a program RADAR-iTE (Regelbasierte Analyse potentiell destruktiverTäter zur Einschätzung des akuten Risikos - islamistischer Terrorismus) [2] where an algorithm isused to try identifying the more dangerous people in a group of people already under investigationby law enforcement.Decisions are based on a set of 72 questions which are transparent for anybody involved. Be-cause those under inspection by RADAR-iTE already are under investigation, the most importantaspect of its application is resource allocation by law enforcement. There is no additional nega-tive effect on those individuals that are judged to be high risk beyond being under investigationalready. Publicized numbers [3] give around half (96 of 205) of the suspects are considered lowrisk after classification by RADAR-iTE, only around 40% (82 of 205) are considered high risk .Transparency of all steps seems guaranteed throughout all decisions performed with respect toalgorithmic classifications.In this case those applying the algorithms and those being judged share in some sense the goalto reduce the number of individuals that are observed. The application of the algorithm has thepotential to help an individual by being removed from the group of high risk people.The implications of a similar algorithm if it was applied to screen the overall population wouldlead to a completely different asessment. Technically, there is no barrier to such a use. It canonly be prevented by morality and law . 6 .1.3 Algorithmic profiling Personalizing versus collective benefits: Individuals have gained a great deal from profiling andever finer segmentation. This mindset of personalising can affect the key collective principleslike democratic and cultural pluralism and risk-sharing in the realm of insurance.
The most discussed form of personalizing in the age of the internet is the so-called filter bubble [36]. The scandal around Cambridge Analytica using Facebook data for micro-targeting a veryspecific subset of the public with the aim to influence the US elections in 2016 made the dangersof highly personal news and marketing feeds obvious [7, 8].As a reaction the legislative started to formulate laws to reduce the risks of such personalizedtargeting with fabricated news, i.e. in Germany the “Netzwerkdurchsetzungsgesetz” [9]. Face-book restricted the admission to personal data for third parties in the aftermath of that scandal[10].A data scientists role, if implementing schemes for targeting specific sub-population identifiedby profiling with the help of the vast amount of information available on each active person in theinternet, should at least be to warn of possible misuse. She or he should understand the dangersfor society and only help to implement lawful or ethical algorithms.A nice example for the second point on risk-sharing are telemetry data collected by so-calledsmart devices and transmitted to insurance companies. Since the beginning of 2018 each newautomobile in the EU has to record telemetry data in a system called eCall [21]. While thatsystem will only transfer data in case of an emergency, there are systems that collect lots ofinformation about all aspects of car usage, down to location and the music the driver listens to[4]. First there are obvious problems with privacy, if there can be unlawful information sharing.The second problem here are insurance companies who try to give personalized policy premiumsbased on level of data sharing a car owner accepts. Probably even more problematic are healthdata, which can be accessed by insurance companies [15].While at first nothing seems at stake if an unhealthy living style is punished with higher policycosts, a second look reveals that the fundamental principle of an insurance, namely risk sharingamong a large group, is eroded. In addition there is a direct conflict of personalized insurancepolicies and personal freedom. Big monetary pressure on customers to live a good live in thesense of the insurance companies must be expected.
Artificial intelligence by being based on advanced techniques of machine learning requires asignificant amount of data. Still, data protection laws are rooted in the belief that individuals’rights regarding their personal data must be protected and thus prevent the creation of massivefiles. AI brings up many hopes: to what extent the balance chosen by the lawmaker and applieduntil now should be renegotiated?
A field of research that is already very experienced and advanced in using large databases onhumans and trying to find ways to make that balance is the medical field. Thus the followingtwo examples are able to illustrate the benefits of the availability of collected personal data andhow the risks for individuals regarding their privacy or for the society regarding fair access toinformation were mitigated. 7n July 2018 some valsartan products were discovered to have been contaminated with N-nitrosodimethylamine (NDMA). In September 2018 an expedited assessment of cancer riskassociated with exposure to NDMA through contaminated valsartan products could be published[30], providing reassuring interim evidence that the short term overall risk of cancer in usersof valsartan contaminated with NDMA was not markedly increased. This fast assessmentin a relatively large cohort (5150 Danish patients) was possible by linking data from fourofficial Danish registries on individual level thus collecting information on prescriptions, cancerdiagnosis hospital admissions, mortality and migration. Privacy was implemented by a processwhere officials from the registries perform the linking, derive the important information, andthen de-identify the data before it is sent to the scientists.In 2018 the German health insurance company DAK Gesundheit in cooperation with scientistsfrom the University of Bielefeld published a report on the health status and the health costs ofchildren and adolescents based on the claims database from the people insured with the DAKGesundheit [33]. Next to some general overview on the health status, a key topic was theinvestigation of the influence of socioeconomic status and education of the parents on the healthand induced health costs of the children. The main conclusion is that education is a strongerinfluencing factor than socioeconomic status and that important preventive measures consist ofgiving children good health education. In the same report, and by guest authors [34], also theresults from the KiGGS study [35] are discussed. That study puts its emphasis more on theprinciple of equal opportunity and the influence of socioeconomic status on general health andspecifically mental health. Publishing this together shows sensitivity of the topic in the politicaldebate and the role that an open scientific environment has to play.Both, the valsartan case and the DAK study show that there are true benefits for public healththat can be generated from using large medical databases. When balancing these benefits withthe risk for privacy violations for the people whose data is used, in the valsartan case, we want tohighlight the high trust from the citizens that is given to officials: if data on any medical problemone encounters in life can be linked to the home address, citizens need to trust the governmentthat this data is not accessible or made accessible to anyone that uses this information with otherthan the best intentions. With the DAK study we want to highlight another important aspectof balancing benefit-risk: the ownership of data, and fair access to data. Data is the new oil ,and evidence generation shapes how benefit is defined and how it is implemented. Thus, if riskis shared by people of all political opinions, then fairness requires that evidence generation ispossible for people from different political opinions.In general, an important measure for respecting privacy is to de-identify data in the databases,and making them non-identifiable. Guidelines exist for de-identification processes (e.g. theSafe Harbor method [32]), yet, with growing databases through social media use and geneticand biomarker research, non-identifiability is a moving target. A good counter-measure isimplemented in the process for requesting access in the so called MIMIC-III database [31] oncritical care unit patients. In addition to a required training on data privacy, and a strict de-identification of the data, all scientists accessing the data have to submit a data use agreementwith 10 points, among which there is one requiring the scientists take immediate action shouldthey realize that there is a way to de-identify data. This is acknowledging the fact that de-identification is no guarantee to de-identifiability at all times by installing a process to monitorde-identifiability by those who have the expertise and knowledge, namely the data scientists,8olding them responsible for it and giving them, as a community, a general credit of trust.
The acceptance of the existence of potential bias in datasets curated to train algorithms is ofparamount importance.
Even if implemented in best of mind, there may be unexpected bias in the training data goingbeyond what has already been said about bias in Section 2.1.2. There are many examples to find,we want to give two.One famous example of algorithmic training going wrong was Microsoft’s twitter bot
Tay [13]. Tay was implemented to act on Twitter as a regular user. The bot should learn from thecomments by others how to perform common twitter conversations. In less than a day the humanshad learned how to manipulate the learning algorithm in such a way that
Tay started to speak outfascistic and racist paroles. Microsoft decided to take
Tay offline less than a day after it startedlearning.A recent example for a similar event is an AI system at amazon. That system should helpto find the most qualified applicants in their huge stream of applications. The experiment hadto be stopped, when it was noted that the algorithm systematically downgraded applications ofwomen. In [12] some probable causes for that behaviour are given. The training data containedmostly applications of men, so most of the successful applicants were men. There are not toomany details, but as a consequence any appearance of the word woman reduced the chances ofthat applications.Finally the whole project was stopped, even after the developing team tried to correct forknown shortcomings, because there was no guarantee the machine would not devise ways todiscriminate in other ways [12].The important observation in both cases is, that these black-box algorithms couldn’t be im-proved. They had to be taken offline and completely replaced. As an obvious consequence suchalgorithms should not be used, where such a replacement is complicated or dangerous.
Hybridisation between humans and machines challenges the notion of our human uniqueness.How should we view the new class of objects, humanoid robots, which are likely to arouseemotional responses and attachment in humans?
This point from the debate in France run by CNIL is given only for the sake of completeness.At the moment, we do not believe that this is an ethical issue where data scientists have a specialresponsibility due to their expertise.
The given examples show the multitude of complex ethical issues that arise from a data scientist’swork. In the next section we argue that ethical guidelines for data scientists are one mean to helpthem taking their responsibility. 9
Guidance for data science
The call for more guidance for digital technologies in media in general is loud and all acrossthe globe, leading to various initiatives and groups engaging in discussions around ethical rulesfor developing and implementing those technologies. For an overview on initiatives and ethicalvalues in the tech field visit the website of the think tank doteveryone [16] or the blog ofErickson [17]. There is a long history of computer scientists discussing the ethics of algorithms.A good starting point is the website fatml.org . Here fatml is an acronym for
Fairness,Accountability, and Transparency in Machine Learning and stands for a series of conferences. Forthe german speaking communities, we recommend the slides to the one day workshop
EthischeLeitlinien wissenschaftlicher Fachgesellschaften of the Deutschen Gesellschaft für MedizinischeInformatik, Biometrie und Epidemiologie (GMDS) [14] or the Algorithmic Accountability Lab(AAL) at the University of Kaiserslautern aalab.informatik.uni-kl.de . AAL provides agood source for current discussions not specific for data scientists but about the use of algorithmsin general with some hints toward data science.This article is in that sense, one contribution among many. Its main purpose is to broaden theaudience and increase the number of participants in the discussions, and to foster the developmentof morality , a set of deeply held, widely shared, and relatively stable values [37] on data sciencewithin and around the data science community. As any ethical guidance, be it in form of codes,oaths, and even law, only has the intended impact, if people are willing to follow it, and thechance for that is high, if the underlying norms and values are in accordance with, in this case,the data science community’s own morality.
Not everyone would agree that data scientists need more guidance how to make moral decisionsin their professional life: many do work in companies with codes of conduct, work for institutionsthat require some oath, or are members of scientific societies that give ethical guidelines to theirmembers, or have religious beliefs that give guidance to wrong or right in their life, and there isthe fundamentally skeptical view that paper does not blush. Also, we are all obliged to obey tolaw. So what does a special set of ethical rules for the profession of data science add?Four rationales:1. For the individual data scientist, the translation from very general ethical principles fromcommon morality, law or religion, to an ethical issue at work can be quite difficult.Especially since most issues are not about intentions, but about the consequences of one’swork. Those consequences are often not very easy to judge upon. Having some referenceto well-thought through and well-reasoned guidelines in that sense is not more nor lessthan having publications on specific methods: it helps to avoid re-inventing the wheelever so often. In addition, it can be very helpful to have such a reference along with thereasoning for justification, if the consequences of an ethical decision increase the workloadfor a colleague or costs for an employer or client.2. For data scientists as a community, having formulated codes of conduct or some serviceideal makes the difference of acting as professionals or merely having a job that does data10runching. In sociology, a profession is defined by means of professionalism. This impliesthat a profession has a certain degree of autonomy in society, its members’ expertise isbased on science, and the professional work exemplifies a service ideal [28]. In otherwords: without a service ideal, there is no professionalism and without professionalism,there is no profession.3. For data scientists as members of society, for their clients, employers and colleagues,written rules of conduct for data science services can help to establish a relationship oftrust. If they are written clearly, they give lay people some mean to know what to expectfrom a data scientist, to compare what they are getting against that standard, and finallygain trust if the expectations are met. Being trusted as a professional increases socialstatus, reputation and possibly the money that is paid for the service.A code of conduct or ethical guidelines may even be the start of a well defined job definitionfor data scientist!4. In case of conflicts of interests an ethical guideline under the maintainership of someprofessional society may offer an arbitration process between different interests.
In the previous section, we provided references to ongoing efforts to develop ethical guidelinesto data science itself and connected scientific or technical fields. Here, we want to give moredetails on the three main guidelines from the fields of statistics and computer sciences from someof the largest and oldest established associations for those communities. If one could establishadditional sub-guidelines that filled the gaps with respect to data science aspects, the audiencewould immediately be very large, and there would be no need to establish a new association.Both, ACM and ASA, acknowledge data science as an important field in their domains.
The American Statistical Association was founded in Boston in 1839 and has more than 19000members worldwide. The current Ethical Guidelines [23] have been updated and approvedby the ASA Board in April 2018. The guideline has eight sections, six of which describethe responsibilities towards individuals and groups of people to which the statistical work maymatter:• Professional integrity and accountability,• integrity of data and methods,• responsibilities to science/public/funder/client,• responsibilities to research subjects,• responsibilities to research team colleagues,11 responsibilities to other statisticians or statistics practitioners,• responsibilities regarding allegations of misconduct,• responsibilities of employers, including organizations, individuals, attorneys, or otherclients employing statistical practitioners.Checking which of the ethical issues discussed in Section 2.1 are covered, one recognises, thatimplicitly, it is a clear call for human responsibility addressing the issue raised on autonomousmachines (Section 2.1.1). It only touches very briefly on the risk, that information presentedas aggregates on groups may lead to bias, discrimination and exclusion (Section 2.1.2). It setshigh standards for privacy and respecting data confidentiality (Section 2.1.4). With the integrityof data and methods section and throughout almost any other point, it gives clear guidance onquality, quantity, and relevance of data, and to a general notion of scientific honesty. It alsoaddresses ethical issues specific to human studies, not covered in section 2.1, but very relevantto all scientists working in that field. The guidelines have gaps concerning those ethical issuesthat result from the implementation of statistical procedures into daily practice. Missing arediscussions on all ethical issues that can arise from implementing algorithmic results withoutfurther human interaction into automatic decision making.
The Association of Computing Machinery (ACM) was founded in 1947 and has more than100.000 members worldwide. The ACM has ethical guidelines for a long time.
The Code [24]as it is named, has just been updated and adopted by ACM in June 2018. It has a preamble, andfour sections:1. General ethical principles,2. professional responsibilities,3. professional leadership responsibilities and4. compliance with the code.On a general level
The Code addresses all ethical issues that we present in Section 2.1. Yet,the Code is not a code for data science, and it is not providing the constructive guidance ASAgives on the integrity of data and methods related to scientific honesty and on responsibilities toresearch subjects.
The German Informatics Society (GI) has a long history of its ethical guidelines [25]. The latestupdate was in June 2018. These guidelines are concise and consist of a preamble and 12 veryshort sections.• Sections 1 to 4 concentrate on aspects of the professional competence of computer scien-tists, 12 sections 5 and 6 are about individual working conditions,• sections 7 and 8 are about teaching and researching in the field of computer science.• Very interesting are sections 9, 10, and 11 which clearly state the societal responsibilitiesof computer scientists. We see some intercept with the work of data scientists there.• Finally section 13 defines a mediating role of the German Informatics Society in case ofconflicts stemming from these guidelines.There are no data science specific sections in these guidelines, nevertheless many importantaspects are touched. We think the structure of the ethical guidelines of the GI can be a goodskeleton to develop ethical guidelines for data science.
The ethical guidelines for statisticians from the ASA are constructive and detailed for the ethicalissues of statisticians and data scientists in the sense of Donoho (Section 1) that work in researchand the special responsibilities towards participants in human studies.
The Code of the ACMcovers the area of using data from and about humans outside from human studies and issues thatarise from implementing algorithms from data science for repeated use and that have impact onindividuals and communities. What we have in mind is a combination of those aspects, maybestructured as in the guidelines of the GI, as data scientists work on data from all sources andacross all those areas.
There are hurdles to overcome before a meaningful guideline can be established. In our view themain ones are the lack of a sense of community and a lack of communication on ethics.
At the moment the term data scientist in not a protected professional title. Data scientists can havean academic training in statistics, or computer science, as their main fields of professional training,but also engineering, psychology, business management, or they can be trained programmers oronly have been following a three-month course on data science learning Python, Julia, or R. Inthat sense, data science today is not a profession but only an occupation. [28]. Between the datascientists from statistics and computer science, on the ground, there is not much tension, butthere are many turf battles on academic levels. So the first step would be to realize that ethicalguidelines are a shared interest and to then start discussing the content within data science relatedsocieties, at conferences, in University courses, at work with colleagues.Being a community does not mean that there is a need for a new association. A good optionwould be to add data science specific guidelines to those of the ACM, the ASA, and the GI.Such an approach would have the big advantage, that it would not require to first establish a newdata science association. Of course the authors would like to see the european statistics societiesembracing ethical issues in their agenda. 13 .2 Data scientists have to overcome shyness or ignorance to discuss ethics andown moral views related to data science
In the perception of the authors it is very uncommon for data scientists to express any moral viewon the work they do or on the impact their work may have for fellow people and the society atlarge. That might be, because only recently society and data scientists themselves have realizedhow much impact data science services have on individuals and communities. Maybe that isbecause the very nature of this impact is, to be de-personalized and it is easy to overlook one’s ownresponsibility. Maybe it is because most people in data sciences are coming from a mathematical,technical, or computer science background and are in general less vocal on anything outside hardscience. The places to change such culture fundamentally should be universities and collegeswhere data science is taught. Ethics and professional ethics should be part of the curriculum, justas inspiring critical thinking and expressing one’s views. In the meantime every data scientistcan work towards that goal within her or his environment. Crucial is taking part in discussionsat work in critical projects or within any community when there are e.g. discussions on theso-called digital revolution, the influence of social media, or algorithms in health care or thecriminal justice system.Talking about ethical questions must become natural for any data scientist.
We wrote this article for most parts without assuming that our views are generally shared views,or that anyone has to agree that any given specific application is good or bad. Underlying, thereis an understanding that the morality of the data science community is evolving and that it isa shared task to develop it, which in turn needs open discussions. Yet, there is at least onefundamental basic moral conviction of the authors, which we have taken as a generally agreedmoral principle: as a human being one has to think about possible consequences of one’s actions.That responsibility for the consequences grows with the knowledge and the potential one has tothink about consequences.Finally we want to start the the debate with a first statement:Data science is in the focal point of current societal development. To build trust in datascience and its interaction with society and to empower data science to take its resposibility forits contributions to society, data science must develop professional ethics and become a clearlydefined profession!
References [1] David Donoho (2017), 50 Years of Data Science, Journal of Computational and GraphicalStatistics, 26:4, 745-766, doi: 10.1080/10618600.2017.1384734[2] Bundeskriminalamt, Presseinformation: Neues Instrument zur Risikobewertung vonpotentiellen Gewaltstraftätern, RADAR-iTE (Regelbasierte Analyse potentiell destruk-tiver Täter zur Einschätzung des akuten Risikos - islamistischer Terrorismus), 2 Feb14017,
Cited 6 Nov 2018[3] FAZ, Jeder zweite Gefährder hat das Potential zum Terroristen, 18.12.2017, . Cited 6 Nov 2018.[4] Jürgen Seeger, ADAC-Untersuchung: Autohersteller sammeln Daten in großem Stil, 4 June2016, . Cited 6 Nov 2018.[5] Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, ProPublica: Machine Bias,May 23, 2016, . Cited 24 Oct 2018[6] Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, ProPublica: How We An-alyzed the COMPAS Recidivism Algorithm, May 23, 2016, . Cited 24Oct 2018[7] Nicholas Confessore, New York Times, Cambridge Analytica and Facebook: The Scandaland the Fallout So Far, 4 Apr 2018, . Cited 6 Nov 2018.[8] FAZ, Wir dachten, wir tun etwas völlig Normales, 21 Mar 2018, [9] Bundesministerium der Justiz und für Verbraucherschutz, Gesetz zur Verbesserung derRechtsdurchsetzung in sozialen Netzwerken (Netzwerkdurchsetzungsgesetz - NetzDG),1.9.2017, .Cited 6 Nov 2018.[10] Mike Schroepfer, CTO Facebook, An Update on Our Plans to Restrict Data Access onFacebook, 4 Apr 2018, https://newsroom.fb.com/news/2018/04/restricting-data-access/ . Cited 6 Nov 2018.[11] Ibrahin Diallo, The machine fired me, 17 June 2018, https://idiallo.com/blog/when-a-machine-fired-me . Cited 2 Nov 2018[12] Jeffrey Dastin, Amazon scraps secret AI recruiting tool that showed bias against women,Reuters News, 22 Oct 2018, 5:12 am, . Cited 1 Nov 2018[13] Sarah Perez, Microsoft silences its new A.I. bot Tay, after Twitter users teach it racism,2016, https://techcrunch.com/2016/03/24/microsoft-silences-its-new-a-i-bot-tay-after-twitter-users-teach-it-racism/ . Cited 1 Nov 20181514] Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie(GMDS), Arne Manzeschke, Alfred Winter, Christoph Isele, Thomas Deserno, Frank Pal-las, Karsten Weber and W. Niederlag , Workshop: Ethische Leitlinien wissenschaftlicherFachgesellschaften, 4 May 2017, https://gmds.de/ueber-uns/organisation/praesidiumskommissionen/ethische-fragen-in-der-medizinischen-informatik-biometrie-und-epidemiologie/ . Cited 6 Nov 2018.[15] Bernard Marr, Forbes, How Big Data Is Changing Insurance Forever, 16 Dec2015, [16] Laura James, Oaths, pledges and manifestos: a master list of ethical tech values,doteveryone https://doteveryone.org.uk , 7 Mar 2018, https://medium.com/doteveryone/oaths-pledges-and-manifestos-a-master-list-of-ethical-tech-values-26e2672e161c . Cited 6 Nov 2018.[17] Lucy C. Erickson, Natalie Evans Harris, and Meredith M. Lee, It’s Time to Talk AboutData Ethics, 26 Mar 2018, . Cited 6 Nov 2018.[18] Wissenschaftlicher Dienst des Bundestags Fachbereich WD 3: Verfassung undVerwaltung, Veröffentlichung der Ergebnisse von Umfragen vor Wahlen (Deutsch-land und Mitgliedstaaten der EU), Aktenzeichen WD 3 - 3000 - 058/18,2018, . Cited 2 Nov 2018.[19] Alexander Fanta, Österreichs Jobcenter richten künftig mit Hilfe von Software über Arbeit-slose, 13 Oct 2018, https://netzpolitik.org/2018/oesterreichs-jobcenter-richten-kuenftig-mit-hilfe-von-software-ueber-arbeitslose/ . Cited 2 Nov2018.[20] Panoptykon Foundation, J ˛edrzej Niklas, Karolina Sztandar-Sztanderska and KatarzynaSzymielewicz, 2015, Warsaw, https://panoptykon.org/sites/default/files/leadimage-biblioteka/panoptykon_profiling_report_final.pdf . Cited 2 Nov2018.[21] European Commission, Cybersecurity & Digital Privacy Policy (Unit H.2), eCall: Timesaved = lives saved, 14 Feb 2018, https://ec.europa.eu/digital-single-market/en/ecall-time-saved-lives-saved . Cited 7 Nov 2018.[22] Katharina Zweig, Wo Maschinen irren können, Impuls Algorithmenethik https://doi.org/10.11586/2018006 [23] American Statistical Association,Ethical Guidelines for Statistical Practice, ApprovedApril 2018, . Cited 9 Nov 2018.1624] Association for Computing Machinery (ACM), ACM Code of Ethics and ProfessionalConduct, Approved June 2018, . Cited 9 Nov2018[25] Gesellschaft für Informatik, Ethical Guidelines of the German Informatics Society, 29 June2018, https://gi.de/ethicalguidelines/ . Cited 6 Nov 2018.[26] Futurezone, AMS-Chef: "Mitarbeiter schätzen Jobchancen pessimistischer ein als derAlgorithmus", 12 Oct 2018, https://futurezone.at/netzpolitik/ams-chef-mitarbeiter-schaetzen-jobchancen-pessimistischer-ein-als-der-algorithmus/400143839 . Cited 6 Nov 2018.[27] Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality andThreatens Democracy, 2016, Crown Publishing Group, New York, NY, USA[28] Timo Airaksinen, The Philosophy of Professional Ethics, 2009, in INSTITUTIONALISSUES INVOLVING ETHICS AND JUSTICE, edited Robert Charles Elliot, Vol 1, PageNumber (201), in Encyclopedia of Life Support Systems (EOLSS), Developed under theAuspices of the UNESCO, Eolss Publishers, Paris, France, [29] Pollard, T. J. and Johnson, A. E. W., The MIMIC-III Clinical Database http://dx.doi.org/10.13026/C2XW26 (2016).[30] Pottegård, Anton and Kristensen, Kasper Bruun and Ernst, Martin Thomsen and Johansen,Nanna Borup and Quartarolo, Pierre and Hallas, Jesper, Use of N-nitrosodimethylamine(NDMA) contaminated valsartan products and risk of cancer: Danish nationwide cohortstudy, vol. 362 BMJ, 2018, doi: 10.1136/bmj.k3851, BMJ Publishing Group Ltd, [31] Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PCh, Mark RG, MietusJE, Moody GB, Peng C-K and Stanley HE. PhysioBank, PhysioToolkit, and PhysioNet:Components of a New Research Resource Complex Physiologic Signals. Circulation101(23):e215-e220 [Circulation Electronic Pages; http://circ.ahajournals.org/content/101/23/e215.full ]; 2000 (June 13).[32] U.S. Department of Health and Human Services. Standards for privacy of individuallyidentifiable health information, final rule. Federal Register. 2002;45 CFR:160–164. [Reflist][33] Prof. Dr. Wolfgang Greiner, Manuel Batram, Oliver Damm, Stefan Scholz, and JulianWitte, 2018, Kinder- und Jugendreport 2018, Beiträge zur Gesundheitsökonomie undVersorgungsforschung (Band 23), Andreas Storm (Herausgeber), DAK-Gesundheit[34] Dr. Benjamin Kuntz, Elvira Mauz und PD Dr. Thomas Lampert, Die KiGGS-Studiedes Robert Koch-Instituts: Studiendesign, Erhebungsinhalte und Ergebnisse zur gesund-heitlichen Ungleichheit im Kindes- und Jugendalter – Robert Koch-Institut, Berlin, Kinder-und Jugendreport 2018, Beiträge zur Gesundheitsökonomie und Versorgungsforschung(Band 23), Andreas Storm (Herausgeber), DAK-Gesundheit1735] Bärbel-Maria Kurth, Panagiotis Kamtsiuris, Heike Hölling, Martin Schlaud, Rüdiger Dölle,Ute Ellert, Heidrun Kahl, Hiltraud Knopf, Michael Lange, Gert BM Mensink, HanneloreNeuhauser, Angelika Schaffrath Rosario, Christa Scheidt-Nave, Liane Schenk, RobertSchlack, Heribert Stolzenberg, Michael Thamm, Wulf Thierfelder and Ute Wolf, Thechallenge of comprehensively mapping children’s health in a nation-wide health survey:Design of the German KiGGS-Study BMC Public Health20088:196 https://doi.org/10.1186/1471-2458-8-196
Kurth et al; licensee BioMed Central Ltd. 2008[36] Eli, Pariser, The filter bubble: how the new personalized web is changing what we readand how we think, Penguin Books, 2012, New York, N.Y., ISBN 0143121235[37] Jennifer Horner, Ph.D., J.D, 2003. Morality, Ethics, and Law: Introductory Concepts.SEMINARS IN SPEECH AND LANGUAGE. Vol 24 (4) 263-274.[38] Commission Nationale Informatique & Liberte, HOW CAN HUMANS KEEP THE UP-PER HAND? The ethical matters raised by algorithms and artificial intelligence, Dec2017, . Cited 2Nov 2018.[39] Commission Nationale Informatique & Liberte, Algorithms and artificial intelligence:CNIL’s report on the ethical issues, 25 May 2018,