notes/docs/lectures/ethics/08_autonomous_systems.md

Autonomous Systems

Autonomous systems include robots and cyber-physical systems that actuate or perform actions in the world, and algorithmic systems particularly machine learning systems or AI.

The UK robotics and autonomous systems or RAS network identifies 7 key ethical challenges that confront autonomous systems. These include

1. bias
2. opacity
3. privacy
4. safety
5. deception
6. employment
7. oversight.

“The race between job creation through new products and job destruction from new technologies has in the past been won by the job-creating effects of innovation. There is no guarantee for a happy end this time; however, an important lesson from the past is that we tend to under-estimate the job-creating potential of fundamental technological transformations, because we lack sufficient knowledge and imagination about the types of jobs that will be created under the new technological paradigm.

https://www.europarl.europa.eu/RegData/etudes/STUD/2018/614539/EPRS_STU(2018)614539_EN.pdf

Alan Winfield and Marina Jirotka in their Royal Society paper on building societal trust in autonomous systems: http://dx.doi.org/10.1098/rsta.2018.0085. They thus propose 5 pillars of good governance, which include establishing a machine intelligence commission to address public fears, including the impact of autonomous systems on jobs. The UK Government established an AI Council in 2019. Regulation is seen as the second pillar of good governance, as are standards, such as those established by professional bodies including the BCS, ACM and IEEE.

The Third Pillar

Recommends we take particular care about the use of AI in safety-critical systems. Of particular concern, as we will take a closer look at later in this lecture, are artificial neural networks, whose decision-making cannot easily be verified. Neural networks learn for themselves and how they arrive at particular decisions is extremely difficult if not impossible to determine.

Fourth Pillar

Good governance, transparency not only of product, i.e., how an autonomous system arrived at a decision, but also of process and how such machines are developed. The concern with process involves

• developing ethical codes
• ensuring ethical training for everyone involved in development
• being transparent about how development is governed
• taking the need for good governance seriously.

Fifth Pillar

Build ethical governors into autonomous systems which would enable a robot or AI system to evaluate the consequences of its actions and modify its actions according to a set of ethical rules.

This is a longstanding ideal in AI, which must address the fundamental problem of encoding and implementing ethics, all of which begs the question of whose ethics get encoded and implemented? Pillar five is then the most idealistic, problematic and challenging of Winfield and Jirotka’s proposals.

Deception

Another key ethical challenge of autonomous systems is posed by humanoid or animal-like robots, which create significant risks of emotional attachment and dependency issues, especially for naive or vulnerable users, that we need to be particularly attentive to.

For example, Babyclon’s animatronic babies and the strong emotions they evoke in those who ‘care’ for them and those who don’t. https://www.theguardian.com/lifeandstyle/video/2020/feb/26/reborn-baby-dolls-women-collectors-video

The issue of deception is part of a broader set of ethical principles governing the development of robots advocated by the UK’s Engineering and Physical Sciences Research Council or EPSRC

• Principle 1 states that robots should not be designed solely or primarily to kill or harm humans, except in the interests of national security.
• Principle 2 states that humans, not robots, are responsible agents and that robots should therefore be designed and operated in compliance with existing laws and respect the fundamental rights and freedoms of human beings, including privacy.
• Principle 3 states that robots should be designed to be safe and secure.
• Principle 4 states that robots are manufactured artefacts and their machine nature should therefore be transparent so as to avoid deception.
• Principle 5 states that the party with legal responsibility for a robot should always be attributed, which is to say that it should always be possible to find out who is responsible for any robot.
  • This of course is not a straightforward matter as the disruption of flights at airports by drones demonstrates.

Algorithmic Bias

Bias is a concern with the validity of outputs or decisions made by autonomous systems, particularly with whether or not those outputs or decisions discriminate against individuals and/or social groups and thus treat them unfairly.

Discrimination is rife in computing today:

• webcams that fail to track black people’s faces
• auto-tagging of black people and women as animals or gorillas
• systematic targeting of racial minorities by the police and undue sentencing of black people
• systematic discrimination against female job candidates and black patients in need of healthcare
• the A-Level debacle in the UK

Discrimination is a specific form of harm based on personal characteristics including gender identity, marital status, sexual orientation, colour, race, ethnic origin, nationality, religion, age, union membership, political affiliation, military status, and disability.

These characteristics are otherwise called “special categories of personal data” or “protected characteristics” and are regulated by GDPR and equality legislation, which would appear to provide a relatively straightforward way of tackling algorithmic bias.

Sources of Algorithmic Bias

Selena Silva and Martin Kenney identify 9 sources of algorithmic bias within the ML life cycle. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3246252

1. Training bias
   • The data used to train the algorithm may be unrepresentative or prejudiced
   • If a facial recognition algorithm is trained on data which primarily consists of white faces, it will be worse at recognising black faces and may even categorise them wrongly.
2. Algorithmic focus bias
   • The attributes it takes into account and either includes or excludes
   • The exclusion of gender or race in a health diagnostic algorithm can lead to inaccurate and harmful outcomes.
   • Whereas the inclusion of gender or race in a sentencing algorithm can lead to discrimination against protected groups.
3. Algorithmic processing bias
   • Thomas Guskey and Lee Ann Jung found, for example, that when an ML algorithm processed student grades across a learning module, it scored students based on the average marks for their assignments, but when teachers were given the same data, they adjusted the students’ scores according to their progress and understanding of the material and provided a fairer assessment of students’ learning. https://core.ac.uk/download/pdf/232576892.pdf
4. Non-transparency bias
   • The lack of transparency about algorithmic decision-making.
   • This is not only to do with how decisions were arrived at, but also concerns IPR and trade secrets and what developers are willing and expected to divulge about their ML systems and AI
5. Transfer context bias
   • The use of ML systems in inappropriate or unintended contexts is also a source of bias. The use of credit scores as a variable in employment provides a ready example of what is called “transfer context bias”
   • Employers request credit checks on job candidates, which effectively means that bad credit is being equated with bad job performance.
6. Automation bias
   • A human bias which involves the users of algorithmic systems treating outputs as objectively true, rather than as statistical probabilities.
   • The COMPAS system used by judges in sentencing criminals in the US provides a good example, where a judge might take the output at face value and apply it uncritically, without reference to other information
   • Automation bias is very much a case of “computer says so …”
7. Consumer bias
   • Is bias expressed by the users of digital platforms
   • Great care needs to be taken with ML systems trained on such data, as they will reflect consumer bias and be inherently prejudiced in one way or another.
8. Feedback loop bias
   • Where ML systems learn from user behaviour, including discriminatory behaviour.
   • So even though an ML system may have been developed without bias in its training, focus and initial processing of data, over time bias may be introduced through use.
   • Twitter taught Microsoft’s AI chatbot Tay to be a racist in less than a day.
9. Interpretation bias
   • Occurs when users interpret outputs according to their own prejudices. For example, it is ultimately up to a judge to interpret the score provided by a recidivism prediction system such as COMPAS, and to decide what action to take. However, a judge may interpret a risk score of 6 as high in a particular case, while they may treat it as an indicator of medium or even low risk in another.