How artificial intelligence could be violating our human rights

The use of artificial intelligence in predictive policing and other areas can lead to indirect discrimination. Is it time for software engineers to study human rights laws?

Monday 08 October 2018 13:26
Comments
Alexa, what are my human rights?
Alexa, what are my human rights?

Artificial intelligence (AI) is finding its way into more and more aspects of our daily lives. It powers the smart assistants on our mobile phones and virtual “home assistants”. It is in the algorithms designed to improve our health diagnostics. And it is used in the predictive policing tools used by the police to fight crime.

Each of these examples throws up potential problems when it comes to the protection of our human rights. Predictive policing, if not correctly designed, can lead to discrimination based on race, gender or ethnicity.

Privacy and data protection rules apply to information related to our health. Similarly, systematic recording and use of our smartphones’ geographical location may breach privacy and data protection rules and it could lead to concerns over digital surveillance by public authorities.

Software engineers are responsible for the design of the algorithms behind all of these systems. It is the software engineers who enable smart assistants to answer our questions more accurately, help doctors to improve the detection of health risks, and allow police officers to better identify pockets of rising crime risks.

Software engineers do not usually receive training in human rights law. Yet with each line of code, they may well be interpreting, applying and even breaching key human rights law concepts – without even knowing it.

This is why it is crucial that we teach human rights law to software engineers. Earlier this year, new EU regulation forced businesses to become more open with consumers about the information they hold. Known as GDPR, you may remember it as the subject of numerous desperate emails begging you to opt in to remain on various databases.

GDPR increased restrictions on what organisations can do with your data, and extends the rights of individuals to access and control data about them. These moves towards privacy-by-design and data-protection-by-design are great opportunities to integrate legal frameworks into technology. On their own, however, they are not enough.

For example, a better knowledge of human rights law can help software developers understand what indirect discrimination is and why it is prohibited by law. (Any discrimination based on race, colour, sex, language, religion, political or other opinion, national or social origin, property, association with a national minority, birth or other status is prohibited under article 14 of the European Convention on Human Rights.)

Direct discrimination occurs when an individual is treated less favourably based on one or more of these protected grounds. Indirect discrimination occurs when a rule that is neutral in appearance leads to less favourable treatment of an individual (or a group of individuals).

Similarly, understanding the intricacies of the right to a fair trial and its corollary, presumption of innocence, may lead to better informed choices in algorithm design. That could help avoid the possibility that algorithms would presume that the number of police arrests in a multi-ethnic neighbourhood correlates with the number of effective criminal convictions.

Even more importantly, it would assist them in developing unbiased choices of datasets that are not proxies for discrimination based on ethnicity or race. For example, wealth and income data combined with geographic location data may be used as a proxy for the identification of populations from a certain ethnic background if they tend to concentrate in a particular neighbourhood.

Legal code

Likewise, a better understanding of how legal frameworks on human rights operate may stimulate the creation of solutions for enhancing compliance with legal rules. For instance, there is a great need for technological due process solutions, by which individuals could easily challenge AI-based decisions made by public authorities that directly affect them. This could be the case of parents who would be wrongly identified as potential child abusers by opaque algorithms used by local authorities.

Such solutions could also be relevant to the private sector. For example, decisions on insurance premiums and loans are often determined by profiling and scoring algorithms hidden behind black boxes. Full transparency and disclosure of these algorithms may not be possible or desirable due to the nature of these business models.

Thus, a due process-by-design solution could allow individuals to easily challenge such decisions before accepting an offer. As our contemporary societies inexorably evolve towards intensive AI applications, we need to bear in mind that the humans behind the AI curtain have the power to make (mis)informed decisions that affect us all.

It is high time that resources and energy are reverted towards educating them not only in cutting edge technology – but also on the relevant human rights rules.

Ana Beduschi is a senior lecturer in Law at the University of Exeter. This article first appeared on The Conversation (theconversation.com)

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Please enter a valid email
Please enter a valid email
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Please enter your first name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
Please enter your last name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
You must be over 18 years old to register
You must be over 18 years old to register
Opt-out-policy
You can opt-out at any time by signing in to your account to manage your preferences. Each email has a link to unsubscribe.

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Join our new commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged in