Читать книгу Data protection for the prevention of algorithmic discrimination - Alba Soriano Arnanz - Страница 6

An algorithm is a set of instructions aimed towards solving a problem step by step. In the context of this research, the algorithms studied are executed by computers. For example, an algorithm contained in the computer system of a university may be used in order to count the number of students enrolled in a course.1 The algorithm will incorporate the following instructions: considering that, initially, N=0, perform the action N + 1 each time a new name appears on the list. Each time, the result of that mathematical operation will become the new value of N. That is to say:

Initially, N=0

For each new name in the list, perform the following operation:N+1=New value of N

These instructions are translated and transformed into code (computer language). Algorithms are used to achieve a purpose within the framework of a representation of reality. This representation of reality, built through the instructions contained in algorithms, is called a model. A model is constructed using an algorithm (series of instructions) to count enrolled students. We have a computer model (a representation) of a human being counting students.

Algorithms are usually executed in groups of programmes. That is why references to algorithms made throughout the book should be understood as references made both to algorithms individually considered and grouped in programmes.

The algorithm explained above is very simple and can be grouped with other algorithms to form simple automated systems or programmes. Simple automated data processing and decision-making systems have existed and been used by both the public and private sector for decades.2 Earlier systems, some of which are still useful for a number of purposes, worked by matching different databases, each of which contained a very specific set of data, such as information on taxes or social security. For example, in Norway, the housing aid system has been automatised since 1972 in order to make aid assignment more efficient3 and in the Netherlands, a system that matches two simple databases in order to fine drivers has been put in place for quite some time.4 In addition, simple forms of automated recruitment systems were also developed and deployed since the 1970s.5

However, the rate of technological developments and increasing computational capacity of the technologies involved in these systems for processing and generating information,6 that is, the computational power that is now available, has largely extended the use of these tools. Today, these systems can quickly analyse huge amounts of data, unassumable for any human being, and autonomously improve the way they process the data they are fed, in what is clearly a process similar to learning. The increase in efficiency that these systems bring about is the main reason for their growing implementation in all kinds of decision-making processes. What, then, do we mean by efficiency in the context of this book? According to the Cambridge English Dictionary, efficiency entails “the good use of time and energy in a way that does not waste any”.7 As Okun puts it, “to the economist, as to the engineer, efficiency means getting the most out of a given input”.8 Applied to the use of automated systems, efficiency entails “all technology intended to reduce human time needed for a task”.9

Algorithms are thus used with the aim of classifying or predicting situations or individuals’ behaviours, recommending courses of action, or a combination of all these functions in a much more efficient manner than if only humans were involved in these processes. Automated systems are used in a wide variety of contexts, from predicting recidivism risk to targeting ads to certain groups or individuals.10 Their heavy penetration in all areas of Western societies has increased the number of ways in which they can directly and indirectly affect individuals’ fundamental rights as well as many aspects of their lives. In addition, these systems are becoming more and more complex and, sometimes, almost impossible to understand and control by humans.11

The growing use of automated decision-making systems in many types of decision-making processes has led to the realisation that, far from solving problems and processes in an objective manner, algorithms reproduce and perpetuate structures of discrimination that harm the members of groups that have historically been in a position of social subordination and disadvantage.12

From a legal perspective, discrimination can be defined as the action of treating a physical or legal person or group of people in a manner that is worse than the way in which other or others in a comparable situation are treated. Discrimination can therefore occur in a wide variety of contexts and situations. However, there are a series of discriminatory actions that are considered especially harmful by democratic states and international human rights instruments. These are instances of discrimination that occur based on categories or grounds that are a priori “suspect”, such as race or sex. The rationale behind establishing these especially protected categories is that it is unacceptable to base decisions on grounds (characteristics) that are, in principle, immutable (such as race or sex) or that belong to the sphere of autonomy of the individual (such as religion and political opinions), especially when said characteristics are not relevant for the purposes for which the decision is made.13

Within each protected category there are certain sub-categories that identify the members of groups that are considered especially vulnerable or have historically suffered oppression and disadvantage, such as non-white populations, women or individuals that come from lower socioeconomic backgrounds. Members of these groups are still subjected to discriminatory treatment as a result of stereotypes and prejudices held against them and of social norms and institutions having been built from the perspective of and for those who have traditionally held positions of power. Hence, the special protection offered in cases of discrimination based on suspect categories is also largely aimed to protect these disadvantaged groups. That is, it is obviously inherently wrong to discriminate an individual because he is a man. However, due to the influence of historical power constructions in society, it is not men but women that are generally the victims of structural and specific cases of discrimination. This means that when analysing both particular cases and the general discrimination suffered by members of vulnerable or disadvantaged groups, it is necessary to consider the historical oppression and current existence and pervasiveness of structures of discrimination that affect members of said groups. This historical oppression also lies at the origin of instances of algorithmic discrimination and the perpetuation of inequality mediated by algorithms.

The specific importance and pervasiveness of the discrimination suffered by members of disadvantaged groups is the main focus of this work. Hence, in order to clarify the terminology used, it is important to note that when references to protected or suspect grounds, categories, attributes or characteristics are made, they will refer to the general characteristic or ground (sex, race, age, etc.). However, when concepts such as “disadvantaged group”, “protected group”, “especially protected group” and “oppressed group” are used, they aim to encompass only the sub-categories of protected characteristics which are especially vulnerable to both structural and specific instances of discrimination.

Over the past few years, a large number of cases in which the use of algorithmic systems has resulted in discriminatory effects for the members of disadvantaged groups or has helped to perpetuate inequality in other ways have been brought forward. Some of these examples will be referred to through the course of this work. However, algorithms and new data processing technologies also generate a wide variety of problems that go beyond possible violations to the rights to equality and non-discrimination of the members of disadvantaged groups. As will be seen throughout the book, automated systems present problems of opacity, difficulty in assigning responsibility for decisions, as well as important risks for the autonomy, freedom and dignity of individuals and to their rights to privacy and data protection. Likewise, when these systems are used by the public sector, they may lead to significant breakdowns in the chain of legitimacy of the decisions made by public authorities.

All the problems that arise from the growing use of algorithms are, moreover, closely linked to each other. For example, the possibility of proving cases of direct algorithmic discrimination (making a discriminatory decision on the basis of a suspect ground) will be severely hampered if mechanisms for system transparency or explainability are not put in place. While indirect discrimination attends to the effects of the measure on the group as a whole, direct discrimination takes place when the protected characteristic (race, sex, etc.) is specifically considered in the decision-making process in a way that negatively affects members of the protected group. Since algorithms take into consideration many variables, it is possible for an algorithm to discriminate against women because they are women and yet for the final decision to not affect women on average because the algorithm takes other elements into account. This would still be a case of direct discrimination, however, it may not be easily proven unless transparency is provided.

All of these issues, their increasing capacity, widespread use, the risks they generate, their lack of transparency and difficulties regarding their control, bring one of the key questions to which this work aims to answer: are existing regulatory instruments appropriate to address the problems generated by these newly developed technologies, or should new regulations that specifically address algorithmic decision-making be developed?

At present, regulatory frameworks based on privacy are the main tools for specifically regulating data processing. It should be noted that the object of study of the book is not any kind of data processing, but only data processing carried out either in a totally or partially automated manner. It is important to delimit this issue as the regulations on data protection analysed throughout this work also address non-automated data processing. It should therefore be borne in mind that any reference made to the regulatory provisions contained in that legal framework and to the problems arising from data processing must be understood to be referred to automated (or semi-automated) processing and decision-making.

The analysis carried out in this book mainly focuses on the European Union’s data protection legal framework but also briefly addresses the US legal system with the objective of providing an element of comparison to the EU data protection system. In the European context, data protection rules, namely the General Data Protection Regulation (GDPR)14 and the Directive for data protection in law enforcement and the criminal justice system,15 aim to provide a broad framework that covers as many of the problems that arise from the increased automation of data processing as possible. Similarly, in the US context, privacy rules and regulatory instruments are being addressed as the main solution for the different problems generated by algorithmic systems. There are, however, significant differences between the EU and US systems that will be pointed out throughout the course of this work.

With regard to the EU system, the reason why a supranational scope of analysis is chosen is, firstly, the fact that the widespread use of automated systems leads to very similar problems arising from the growing use of these systems in all EU Member States.

Secondly, the scope of action of algorithmic systems is not governed by national borders, since one system can affect citizens of different states. Furthermore, a system created in one state can be sold to companies belonging to many others, which makes it all the more necessary to provide a legal framework that goes beyond the strictly national sphere. This is especially the case with large transnational technological corporations, which are increasingly present and dominant in many areas of society, leading the problems generated by the algorithms that these companies use to have a global impact.

Finally, the regulatory responses that have been thus far provided to the issues generated by algorithms, which are mainly built from the perspective of data protection, have taken place at a European level. Furthermore, recent proposals for a new framework for the regulation and control of algorithms also envisage future legal responses to the risks and damages generated by algorithms to be developed by the European Union.

It should also be noted that, in order to make this work as practical as possible, many examples of algorithmic systems used not only in the European Union, but also in other countries, especially the United States, are provided. It is particularly important to refer to the systems used in the US because the technological innovations, and the social changes they entail, that take place in the United States tend to spread to the rest of the world and, in the case of the increasing use of automated systems by both the public and private sectors, this reality has been particularly evident.

This work aims to determine how the different risks generated by algorithmic decision-making justify public intervention in the private development and use of algorithmic systems. The book mainly focuses on how regulators have aimed to solve the harms and problems generated by automatisation through the privacy framework and how this approach has, to a certain degree, failed. Based on said shortcomings, a series of proposals are made regarding the regulatory solutions that could be provided in order to address algorithmically-generated problems. Amongst the different problems generated by the use of algorithms, the main issue that is addressed in this work is algorithmic discrimination.

The first chapter briefly explains different relevant concepts related to the technologies analysed in this work, demonstrates their growing use both in the public and private sectors and analyses some of their possible applications. Chapter two maps the harms and risks generated by algorithmic decision-making and, building from these issues, aims to justify public intervention in the private development and use of algorithms. Chapters three through to six explain the reasons why, to date, the legal framework in the field of data protection has been chosen as the main tool for tackling some of the problems arising from the growing use of algorithms and analyses the European legal framework in the field of data protection, comparing it to the privacy framework and legal tradition in the United States. The seventh chapter establishes the limitations and insufficiencies of this legal framework when it comes to tackling the different problems generated by the growing use of automated systems and, in particular, algorithmic discrimination. The final chapter briefly develops a series of proposals, based on available legal techniques and already existing legal rules, aimed towards achieving better regulation and control of algorithmic systems.

1. This example is inspired by the explanations of what models and algorithms constitute provided by David Malan and Cathy O’Neil in Malan, D., “What is an algorithm?”, May 2013. Available on 27th August 2020 at: https://www.ted.com/ and O’Neil, C., Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, London, Penguin Books, 2017.

2. Parloff, R., “Why deep learning is suddenly changing your life”, Fortune, 28th September 2016. Available on 28th April 2019 at: http://fortune.com/

3. Bing, J., “Code, access and control” in Murray, A., & Klang, M., Human Rights in the Digital Age, London, Glasshouse Press, 2005, p. 204.

4. Van Eck, M., “Algorithms in public administration”, 31st January 2017. Available on 17th July 2019 at: https://marliesvaneck.wordpress.com/

5. Lowry, S. & Macpherson, G., “A blot on the profession”, British Medical Journal, 5th March 1988, pp. 657-658.

6. Parloff, R., “Why deep learning is suddenly changing your life”, cit., 2019.

7. Cambridge English Dictionary, “Efficiency”, 2021. Available on 27th January 2021 at: https://dictionary.cambridge.org/dictionary/english/efficiency

8. OKUN, A. M., Equality and Efficiency Equality and Efficiency: The Big Tradeoff, Washington DC, Brookings Institution Press, 2015 (1st ed. 1975), p. 2.

9. Tenner, E., The Efficiency Paradox: What Big Data Can’t Do, New York, Vintage Books, 2019, p. xii.

10. O’Neil, C., Weapons of Math Destruction…, cit., 2017.

11. Bhatia, R., “How do machine learning algorithms differ from traditional algorithms?”, Analytics India Magazine, 10th September 2018. Available on 13th June 2019 at: https://analyticsindiamag.com/.

12. Whenever references to “algorithmic discrimination” or “discrimination” in general are made throughout the course of this book, they specifically refer to the phenomenon by which members of historically disadvantaged groups (women, racial and religious minorities, individuals with disabilities, etc.) are directly or indirectly treated less favourably than non-members of said groups.

13. Gerards, J., “The discrimination grounds of article 14 of the European Convention on Human Rights”, Human Rights Law Review, vol. 13, No. 1, 2013, p. 114.

14. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation).

15. Directive (EU) 2016/680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and repealing Council Framework Decision 2008/977/JHA.