Reuben Binns

Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data

Decisions based on algorithmic, machine learning models can be unfair, reproducing biases in historical data used to train them. While computational techniques are emerging to address aspects of these concerns through communities such as discrimination-aware data mining (DADM) and fairness, accountability and transparency machine learning (FATML), their practical implementation faces real-world challenges. For legal, institutional or commercial reasons, organisations might not hold the data on sensitive attributes such as gender, ethnicity, sexuality or disability needed to diagnose and mitigate emergent indirect discrimination-by-proxy, such as redlining. Such organisations might also lack the knowledge and capacity to identify and manage fairness issues that are emergent properties of complex sociotechnical systems. This paper presents and discusses three potential approaches to deal with such knowledge and information deficits in the context of fairer machine learning. Trusted third parties could selectively store data necessary for performing discrimination discovery and incorporating fairness constraints into model-building in a privacy-preserving manner. Collaborative online platforms would allow diverse organisations to record, share and access contextual and experiential knowledge to promote fairness in machine learning systems. Finally, unsupervised learning and pedagogically interpretable algorithms might allow fairness hypotheses to be built for further selective testing and exploration. Real-world fairness challenges in machine learning are not abstract, constrained optimisation problems, but are institutionally and contextually grounded. Computational fairness tools are useful, but must be researched and developed in and with the messy contexts that will shape their deployment, rather than just for imagined situations. Not doing so risks real, near-term algorithmic harm.

Privacy Languages: Are we there yet to enable user controls?

Privacy protection is one of the most prominent concerns for web users. Despite numerous efforts, users remain powerless in controlling how their personal information should be used and by whom, and find limited options to actually opt-out of dominating service providers, who often process users information with limited transparency or respect for their privacy preferences. Privacy languages are designed to express the privacy-related preferences of users and the practices of organisations, in order to establish a privacy-preserved data handling protocol. However, in practice there has been limited adoption of these languages, by either users or data controllers. This survey paper attempts to understand the strengths and limitations of existing policy languages, focusing on their capacity of enabling users to express their privacy preferences. Our preliminary results show a lack of focus on normal web users, in both language design and their tooling design. This systematic survey lays the ground work for future privacy protection designs that aim to be centred around web users for empowering their control of data privacy.

Algorithmic Accountability and Public Reason

The ever-increasing application of algorithms to decision-making in a range of social contexts has prompted demands for algorithmic accountability. Accountable decision-makers must provide their decision-subjects with justifications for their automated system’s outputs, but what kinds of broader principles should we expect such justifications to appeal to? Drawing from political philosophy, I present an account of algorithmic accountability in terms of the democratic ideal of ‘public reason’. I argue that situating demands for algorithmic accountability within this justificatory framework enables us to better articulate their purpose and assess the adequacy of efforts toward them.

Third Party Tracking in the Mobile Ecosystem

Third party tracking allows companies to identify users and track their behaviour across multiple digital services. This paper presents an empirical study of the prevalence of third-party trackers on 959,000 apps from the US and UK Google Play stores. We find that most apps contain third party tracking, and the distribution of trackers is long-tailed with several highly dominant trackers accounting for a large portion of the coverage. The extent of tracking also differs between categories of apps; in particular, news apps and apps targeted at children appear to be amongst the worst in terms of the number of third party trackers associated with them. Third party tracking is also revealed to be a highly trans-national phenomenon, with many trackers operating in jurisdictions outside the EU. Based on these findings, we draw out some significant legal compliance challenges facing the tracking industry.

Algorithms that remember: model inversion attacks and data protection law

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EUs recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around ‘model inversion’ and ‘membership inference’ attacks, which indicates that the process of turning training data into machine-learned systems is not one way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation. This article is part of the theme issue ‘Governing artificial intelligence: ethical, legal, and technical opportunities and challenges’.

When Data Protection by Design and Data Subject Rights Clash

Data protection law has historically faced significant enforcement challenges. Data protection authorities (DPAs) have classically been underfunded and outgunned, possessing limited ability to scrutinize the onthe-ground practices of data controllers and restricted capacity to meaningfully act when transgressions are suspected. In response to these governance challenges, concerned communities have advocated a range of technological approaches that allow effective but noninvasive use of data, or ‘DIY’ protections which data subjects can adopt unilaterally. These approaches, often called ‘privacy-enhancing technologies’ (PETs), are commonly discussed in regulatory circles within the context of ‘privacy by design’ (PbD). PbD emphasizes that issues of privacy should be considered from the start and throughout the design process through creative social and technical means. Most point to its intellectual home in a report undertaken by the Dutch Data Protection Authority and TNO, with support of the then Information and Privacy Commissioner for Ontario, Tom Wright, although its heritage can be traced further back to the considerations given to ‘technical and organizational measures’ in the Key Points

Measuring Third-party Tracker Power across Web and Mobile

Third-party networks collect vast amounts of data about users via websites and mobile applications. Consolidations among tracker companies can significantly increase their individual tracking capabilities, prompting scrutiny by competition regulators. Traditional measures of market share, based on revenue or sales, fail to represent the tracking capability of a tracker, especially if it spans both web and mobile. This article proposes a new approach to measure the concentration of tracking capability, based on the reach of a tracker on popular websites and apps. Our results reveal that tracker prominence and parent–subsidiary relationships have significant impact on accurately measuring concentration.

X-Ray Refine: Supporting the Exploration and Refinement of Information Exposure Resulting from Smartphone Apps

Most smartphone apps collect and share information with various first and third parties; yet, such data collection practices remain largely unbeknownst to, and outside the control of, end-users. In this paper, we seek to understand the potential for tools to help people refine their exposure to third parties, resulting from their app usage. We designed an interactive, focus-plus-context display called X-Ray Refine (Refine) that uses models of over 1 million Android apps to visualise a persons exposure profile based on their durations of app use. To support exploration of mitigation strategies, emphRefine can simulate actions such as app usage reduction, removal, and substitution. A lab study of emphRefine found participants achieved a high-level understanding of their exposure, and identified data collection behaviours that violated both their expectations and privacy preferences. Participants also devised bespoke strategies to achieve privacy goals, identifying the key barriers to achieving them.

Self-authored interest profiles for personalised recommendations

A large portion of the content, recommendations and advertisements shown on the web are targeted, based on a profile of an individual user. This paper explores two ways of creating and using such profiles. Behavioural profiling - a commonly used technique which makes inferences based on an individuals previous activity - is compared to what I call Self-Authored Interest (SAI) profiling, which is based on information explicitly volunteered and controlled by the individual. I present the results of an experimental study comparing the effectiveness of the two systems in generating targeted product recommendations. I find that (a) people respond more positively to product recommendations when they are derived from SAI profiles, and (b) the mere belief that a recommendation comes from an SAI profile is also associated with more positive responses.