Add the rest of university notes
This commit is contained in:
366 files changed
+9844
-110
No files matched your search
@@ -0,0 +1,64 @@
|
||||
# Why do we need Professional Ethics
|
||||
Computers enable social harm:
|
||||
|
||||
|
||||
## Illegal content and activity
|
||||
- Terrorism
|
||||
- Crypto-currencies can finance this
|
||||
- Organised crime
|
||||
- Phishing and fraud
|
||||
- Information stealing malware
|
||||
- Ransomware and DDoS extortion
|
||||
- Domestic Abuse
|
||||
- Abusers can look at devices connected to the internet to control their partner even after they have left the house to establish control
|
||||
- Cyber-Bullying
|
||||
- Sending harmful messages/photos to people
|
||||
- Promoting hate
|
||||
- Impersonating another person
|
||||
- *No legal definition of cyber-bullying but still prosecutable*
|
||||
- Child sexual exploitation and Abuse
|
||||
- Solicitation, Grooming, Distribution of images & videos
|
||||
- Trafficking
|
||||
|
||||
## Impact on health and well being
|
||||
|
||||
- Computers can affect physical, social and mental health
|
||||
- Lower physical activity
|
||||
- Increases loneliness
|
||||
- Designed for addiction
|
||||
- Click bait
|
||||
- Infinite scroll
|
||||
- Short term dopamine-driven feedback loops - Chamath Palihapitya (ex Facebook VP)
|
||||
- Self-harm
|
||||
- Enables people to research self harm methods
|
||||
- Validates negative feelings
|
||||
- Legitimise suicide as an acceptable course of action
|
||||
|
||||
## Threats to our way of life
|
||||
- Manipulating public opinion
|
||||
- Can be state sanctioned
|
||||
- Distribution of inaccurate information, disinformation and fake news
|
||||
- The Oxford internet institute found 26 countries including China, Turkey and Russia were using computational propaganda to suppress human rights and discredit political opposition
|
||||
|
||||
### Risk to critical national infrastructure
|
||||
- Cyber attacks on nuclear power stations, electricity grids, banking communications
|
||||
- WannaCry targeting the NHS
|
||||
|
||||
## Environmental Impact
|
||||
- Data centres consume huge amounts of energy
|
||||
- Consumed 416.2 TWH of electricity - more than the total UK’s power consumption
|
||||
- 3% of global electricity supply
|
||||
- 2% of greenhouse gas emissions
|
||||
|
||||
## GDPR
|
||||
|
||||
- Data protection
|
||||
- Data is the oil of the digital economy
|
||||
- **GDPR** applies to the processing of personal data by automated means, regardless of whether the processing takes place in the EU or not relating to:
|
||||
- The offering of goods or services to EU citizens
|
||||
- The monitoring of their behaviour
|
||||
- There are stiff fines for those who break GDPR
|
||||
- £20,000,000 or 4% of total annual turnover - whichever is greater.
|
||||
|
||||
# A world under attack
|
||||
It’s not computer scientists who do harm, but the way the technology is designed, who designed it and the outcomes it is trying to achieve influence how it impacts its users and wider society.
|
||||
@@ -0,0 +1,165 @@
|
||||
# Professional Codes of Conduct
|
||||
|
||||
**Professional Ethics** - A set of morally permissible standards of a group that each member of the group wants every other member of the group to follow even if their doing so would mean that he/she must do the same `Micheal Davis, Professional Code and Ethics Burlington: Ashgate 2001`
|
||||
|
||||
## Morally permissible
|
||||
|
||||
- Morality is ubiquitous, as moral standards apply to everyone
|
||||
- Professional ethics only apply to the members of particular groups (such as lawyers, doctors etc)
|
||||
|
||||
**Ethical does not equal moral**
|
||||
|
||||
>For example it is against ethical standards in the USA for doctors to advertise prices for their services, but there is nothing inherently immoral about advertising prices for services.
|
||||
- An action may be morally permissible but unethical
|
||||
- It is also possible to behave ethically but apparently immorally
|
||||
- Professional ethics requires that one behaves consistently with the standards of the group.
|
||||
|
||||
**Professional ethics is a subset of moral concerns**
|
||||
- Morality encompasses societal reasoning and norms of conduct as to what constitutes right and wrong
|
||||
- Professional ethics govern professional practice with respect to particular moral issues or challenges like *algorithmic decisions*
|
||||
- As the broader social-moral order evolves so do professional ethics, like ACM Code of Ethics
|
||||
|
||||
## Standards
|
||||
Govern professional practice
|
||||
Standards consist of:
|
||||
- Principles
|
||||
- Rules of Conduct
|
||||
- Embedded in code of conduct or code of ethics
|
||||
|
||||
>A professional puts profession first. When a conflict arises between the professional's code and the policy of an employer or the law, the professional's code must take precedence - Brinkman & Sanders, *Ethics in Computing Culture.* Boston: Cengage Learning, 2013.
|
||||
|
||||
### Shared by a Group
|
||||
Standards are shared by a cohort of people engaged in professional activity
|
||||
|
||||
**What constitutes professional activity?**
|
||||
|
||||
- Provides an important service to soceity
|
||||
- Requires extensive training
|
||||
- Involves significant intellectual effort
|
||||
- Organisation of members
|
||||
- Individual autonomy
|
||||
- Certification or Licensing
|
||||
|
||||
#### Is computing a profession?
|
||||
The problematic static of computing
|
||||
- Lack of accreditation, certification or licensing
|
||||
+ No single organisation of members for the computing profession
|
||||
Question is immaterial:
|
||||
The harms enabled by computing mean that computing professionals still have important ethical obligations
|
||||
|
||||
>Programmers need ethics when designing the technologies that influence people's lives - President of the ACM
|
||||
|
||||
We still need professional ethics in computing even if computings professional status is dubitable.
|
||||
- We need ethics if we are to be considered professionals
|
||||
|
||||
> It is impossible to satisfy the definition of profession without a code of ethics, impossible to teach 'professionalism' without teaching the code, and indeed impossible to understand professions without understanding them as bound by such a code. Without a code of ethics, there are only honest occupations, trade associations, and the like - Micheal Davis
|
||||
|
||||
## The Different Codes
|
||||
|
||||
#### British Computer Society (BCS)
|
||||
|
||||
##### Public Interest
|
||||
|
||||
These standards require:
|
||||
|
||||
- You have due regard for public health, privacy, security and the wellbeing of others and the environment in your work
|
||||
- Your work has due regard for the legitimate rights of third parties
|
||||
- You conduct your professional activities without discrimination
|
||||
- You promote equal access to the benefits of IT
|
||||
|
||||
##### Professional competence and integrity
|
||||
|
||||
- Only undertake to do work or provide a service that is within your professional competence
|
||||
- Do not claim a level of competence that you do not possess
|
||||
- Continue to develop professional knowledge relevant to your field
|
||||
- Ensure that you have the knowledge and understanding of relevent legislation
|
||||
- Respect and value alternate viewpoints
|
||||
- Avoid injuring others
|
||||
- Reject and will not make any offer of bribery or unethical inducement
|
||||
|
||||
##### Duty to relevant authority
|
||||
|
||||
- Carry out your professional responsiblities with due care and diligence
|
||||
- Avoid situations that conflict with the interests of relevant authorities
|
||||
- Accept professioal responsibilities for your work
|
||||
- Do not disclose confidential information
|
||||
- Do not misrepresent or withhold information on the performance of products, system or services
|
||||
|
||||
##### Duty to Profession
|
||||
|
||||
- Accept your personal duty to uphold the reputation of the profession
|
||||
- Seek to improve professional standards
|
||||
- Uphold the reputation and good standing of BCS
|
||||
- Act with integrity and respect in your professional relationships
|
||||
- Notify the BCS if convicted of a criminal offence
|
||||
- Support fellow members in their professional development
|
||||
|
||||
#### Institute of Electrical and Electronics Engineers (IEEE)
|
||||
|
||||
|
||||
|
||||
Covers about half of what the BCS covers, little attention to duty to relevant authority which undermines its commitment to the highest ethical and professional conduct.
|
||||
|
||||
#### Association of Computing Machinery (ACM)
|
||||
|
||||
25 principles governing professional conduct
|
||||
|
||||
- 7 general ethical principles
|
||||
- 9 principles governing professional responsiblities
|
||||
- 7 principles of professional leadership
|
||||
- 2 principles of compliance
|
||||
|
||||
##### General ethical principles
|
||||
|
||||
- Contribute to society and human well-being
|
||||
- Avoid harm
|
||||
- Be honest and trustworthy
|
||||
- Be fair and take action not to discriminate
|
||||
- Respect the work of others
|
||||
- Respect privacy
|
||||
- Honor confidentiality
|
||||
- Unless in cases in which it is evidence of the violation of law or the code itself
|
||||
|
||||
This links to the BCS public interest requirement
|
||||
|
||||
##### Professional responsibilities
|
||||
|
||||
- Strive to achieve high quality work
|
||||
- Maintain high standards to professional competence
|
||||
- Know and respect rules pertaining to professional work
|
||||
- Accept and provide appropriate professional review
|
||||
- Evaluate computer systems and possible risks
|
||||
- Providing objective evaluations for employers or clients
|
||||
- Perform work only in areas of competence
|
||||
- Foster public awareness and understanding of computing
|
||||
- Access computing only when authorised or for public good
|
||||
- Basically **do not hack**, unless it is to disrupt or inhibit malicious systems
|
||||
- Design and implement robust and secure systems
|
||||
- Does not link to BCS code however important
|
||||
|
||||
##### Professional leadership Principles
|
||||
|
||||
- Ensure centrality of public good
|
||||
- Promote social responsibility
|
||||
- Enhance quality of working life
|
||||
- Support the principles of the code
|
||||
- Create oppotunities for professional development
|
||||
- User care when modifying or retiring systems
|
||||
- Take special care of systems integrated in societal infrastructure
|
||||
|
||||
##### Compliance with the Code
|
||||
|
||||
- Uphold, promote and respect the principles of the code
|
||||
- Treat violations as inconsistent with ACM membership
|
||||
|
||||
## ACM & BCS Code of Ethics
|
||||
|
||||
#### Mapping
|
||||
|
||||
- More to the ACM code
|
||||
- But a strong relationship between the two exists, although it is not always direct
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# Coursework Issue
|
||||
|
||||
The coursework issue is about a class action lawsuit against Ring.
|
||||
|
||||
file: <studentID>_Surname
|
||||
|
||||
## Example of applying Codes
|
||||
|
||||
The example is taken from the ACM code of ethics - case study 5
|
||||
|
||||
> ###### Malicious Input to Content Filters
|
||||
>
|
||||
> **The US. Children’s Internet Protection Act (CIPA) mandates that public schools and libraries employ mechanisms to block inappropriate material that is deemed harmful to minors.**
|
||||
>
|
||||
> Blocker Plus is an automated Internet content filter designed to help these institutions comply with CIPA’s requirements. To accomplish this task, Blocker Plus has a centrally controlled blacklist maintained by the software maker. In addition, Blocker Plus provides a user-friendly interface that makes it a popular product for home use by parents.
|
||||
>
|
||||
> Due to the challenge of continually updating the blacklist, the makers of Blocker Plus began to explore machine learning techniques to automate the identification of inappropriate content. During the development of these changes, Blocker Plus combined input from both home and library users to aid in the classification of content. Pleased with their initial results, Blocker Plus deployed these techniques in their production system. Furthermore, Blocker Plus continued to collect input from users to refine their learned models.
|
||||
>
|
||||
> During a recent review session, the development team reviewed several recent complaints about content being blocked inappropriately. An increasing amount of content regarding gay and lesbian marriage, vaccination, climate change, and other topics not covered by CIPA, had been added to the blacklist. Initial investigations into these incidents suggested that some activist groups had exploited Blocker Plus’s feedback mechanism to provide input that corrupted the classification model.
|
||||
>
|
||||
> **ANALYSIS SUMMARY:**
|
||||
>
|
||||
> Blocker Plus is a system designed to block content legally designated as harmful to children. While this filtering constitutes a form of censorship, children are considered a protected vulnerable class. To reduce the impact on adults, CIPA also mandates that these filters must be disabled on request. Given that Blocker Plus is complying with US. federal regulations to facilitate socially responsible uses of computers, the system is consistent with Principles 1.1 and 2.3. Given the complexity and risk involved in Blocker Plus’s use of machine learning techniques, Principle 2.5 calls for extraordinary care. Principle 2.9 suggests that Blocker Plus should have included better protections against the intentional misuse by the activist groups. Blocker Plus’s deployment of machine learning causes harm by suppressing information of legitimate public interest and safety, as well as by discriminating based on sexual orientation, raising concerns for both Principles 1.2 and 1.4. In addition, Blocker Plus provides an example of a system becoming integrated into the educational infrastructure of society. Principle 3.7 emphasises that the developers of such systems have an added responsibility to provide good stewardship and Blocker Plus must correct these issues.
|
||||
|
||||
#### Which principles apply to Blocker Plus?
|
||||
|
||||
- `1.1` Contribute to society and human well-being
|
||||
- Socially responsible uses of computing
|
||||
- `2.3` Know and respect rules pertaining to professional work
|
||||
- This is broken as a federal law is being broken
|
||||
- `2.5` Evaluate computer systems and their impacts, including risks
|
||||
- Extraordinary care be taken to identify and mitigate potential risks. Blocker Plus violates this principle by allowing its feedback algorithm to be manipulated by activists to corrupt the classification model.
|
||||
- `2.9` Design and implement robustly and usably secure systems
|
||||
- 2.9 requires that computing professionals should perform due diligence to ensure systems function as intended, and take appropriate action to secure resources against accidental and intentional misuse, modification or denial of service. That the activists were able to intentionally misuse Blocker Plus means that the system violates this principle
|
||||
- `1.2` Avoid harm
|
||||
- Avoid harm applies as the corruption of the machine learning model means that information of legitimate public interest (gay & lesbian marriage) and safety (vaccinations and climate change) is suppressed by the activists' intentional misuse of the system
|
||||
- `1.4` Be fair and do not discriminate
|
||||
- This applies in the respect of suppression of information of legitimate public interest enables discrimination of the basis of sex and sexual orientation
|
||||
- `3.7` Take special care of systems integrated into societal infrastructure
|
||||
- Applies as Blocker Plus is designed for educational purposes. In failing to prevent intentional misuse of the system, the leadership of Blocker Plus have failed in their responsibility to be good stewards of the system and enabling fair access.
|
||||
|
||||
Codes for the coursework only apply in negative reasons, e.g. 1.1 may apply as amazon wished to contribute to society and human well being. However this will not be marked.
|
||||
|
||||
There is one code in the amazon ring that there is no evidence of, however it is inferred by a *lack* of action.
|
||||
|
||||
## The ACM CARE Framework
|
||||
|
||||
For determining whether a case is consistent with the code
|
||||
|
||||
##### Consider
|
||||
|
||||
What were the observable effects of Amazon's actions or decisions for Ring users
|
||||
|
||||
> Who are the relevant actors and stakeholders? What were the anticipated and/or observable effects of the actions or decisions for those stakeholders? What additional details would provide a greater understanding of the situational context?
|
||||
|
||||
##### Analyse
|
||||
|
||||
What stakeholder rights (legal, natural or social) were impacted and to what extent, and ask what principles of the code are relevent here.
|
||||
|
||||
> What stakeholder rights (legal, natural, or social) were impacted and to what extent? What technical facts are most relevant to the actors’ decision? What principles of the Code were most relevant? What personal, institutional, or legal values should be considered?
|
||||
|
||||
##### Review
|
||||
|
||||
What potential actions could changed the outcomes
|
||||
|
||||
> What responsibilities, authority, practices, or policies shaped the actors’ choices? What potential actions could have changed the outcomes?
|
||||
|
||||
##### Evaluate
|
||||
|
||||
What actions (or lack of actions) supported or violated the Code. Are the actions taken in this case justified, particularly when considering the rights of and impact on all stakeholders.
|
||||
|
||||
> How might the decision in this case be used as a foundation for similar future cases? What actions (or lack of action) supported or violated the Code? Are the actions taken in this case justified, particularly when considering the rights of and impact on all stakeholders?
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
# Professional Responsibilities
|
||||
|
||||
You are generally expected to **uphold the profession**
|
||||
|
||||
Unethical conduct includes:
|
||||
|
||||
- Exaggerating skills and competences
|
||||
- Withholding or misrepresenting technical information
|
||||
- Conflicts of interest
|
||||
- Divulging confidential information
|
||||
- Dishonest conduct
|
||||
|
||||
### Due Diligence
|
||||
|
||||
It is imperative that you **take reasonable care** in conducting your professional work
|
||||
|
||||
This means:
|
||||
|
||||
- You are competent to do the work required of you and your team
|
||||
- Appropriate steps are taken to avoid harm
|
||||
- Systems are robust, secure and respect privacy
|
||||
- Rules are followed
|
||||
- Special care is taken when modifying or retiring systems or systems are integrated in societal infrastructure
|
||||
|
||||
### Public Good
|
||||
|
||||
> "Computing professionals actions change the world. To act responsibly, they should reflect upon the wider impacts of their work, consistently supporting the public good" - ACM Code of Ethics
|
||||
|
||||
### Unconscious Bias
|
||||
|
||||
- Subtle and built into all of us
|
||||
- Entirely natural
|
||||
- Can be mitigated
|
||||
- Draws our attention to micro-issues
|
||||
- for example discriminate against people of tattoos, or people with piercings
|
||||
- Can have an squally detrimental effect as the big issues
|
||||
- Design to minimise unconscious bias
|
||||
|
||||
### Respect the Work of Others
|
||||
|
||||
- Do no harm
|
||||
- Do not hack
|
||||
- Unless public good requires it or you are authorised to do so
|
||||
- Respect intellectual property rights (IPR)
|
||||
- Relevant types of IPR: trade marks, industrial designs, patents, trade secrets, databases & domain names
|
||||
|
||||
#### IPR
|
||||
|
||||
###### Trademarks
|
||||
|
||||
A distinctive sign, symbol or logo
|
||||
|
||||
- Owner holds exclusive rights of use
|
||||
- Registered nationally, regionally or globally
|
||||
- Protection lasts 10 years, with the option to renew indefinitely
|
||||
|
||||
###### Industrial Designs
|
||||
|
||||
Distinctive elements of a product
|
||||
|
||||
Used where products have a short design life e.g. fashion
|
||||
|
||||
- Two types of protection
|
||||
- Registered Community designs (RCD)
|
||||
- Protection lasts **5** years, renewed up to **25** years
|
||||
- Unregistered Community designs (UCD)
|
||||
- Protection lasts for **3** years
|
||||
|
||||
###### Patent
|
||||
|
||||
- An exclusive right granted to protect an invention
|
||||
- Prevents others from making, using, offering for sale, selling or importing invention without owner's permission
|
||||
- Lasts for **20** years from date of filed
|
||||
- Costs between $3,000 and \$6,000
|
||||
- Can't patent a computer program only a "computer-implemented invention"
|
||||
|
||||
###### Utility Models
|
||||
|
||||
- Prevents others from making, using, offering for sale, selling or importing invention without owner's permission
|
||||
- Cheaper than a patent (not available in UK)
|
||||
- Lasts 7-10 years
|
||||
- Registered nationally
|
||||
|
||||
###### Trade Secrets
|
||||
|
||||
- Confidential business information that provides a competitive advantage
|
||||
- Must put reasonable measures in place to keep it a secret
|
||||
- Store safely, implement NDAs
|
||||
- Do not confer proprietary rights
|
||||
- Protected by law for an unlimited time period
|
||||
|
||||
###### Copyright
|
||||
|
||||
- Author's or creator's right to protection over uses of their work
|
||||
- Ideas cannot be copyrighted, only the concrete implementation of the idea
|
||||
- Obtained automatically
|
||||
- Includes economic rights (renumeration for use by others)]
|
||||
- Fair use allowed
|
||||
- Covers life-time of owners plus **50-70** years
|
||||
|
||||
###### Databases
|
||||
|
||||
- A systematic arrangement of data, works or materials
|
||||
- Two forms:
|
||||
- Original
|
||||
- Protection lasts lifetime + 50-70 years
|
||||
- Non-original (like a phone directory)
|
||||
- Protected by *sui generis* database right which lasts for **15** years
|
||||
|
||||
###### Domain Names
|
||||
|
||||
- Registered by ICANN registars
|
||||
- Not protected by copyright
|
||||
- May be protected by a registered trade mark
|
||||
- Last up to **10** years, renewed indefinitely
|
||||
|
||||
### Ethical Limits of IPR
|
||||
|
||||
Don't go too far in protecting your own works
|
||||
|
||||
###### Sony Rookit
|
||||
|
||||
They produced CDs that when entered into a computer downloaded a rootkit which gained administrator control on the victims computer.
|
||||
|
||||
Rookit modified the victims OS, limiting the users ability to use the CD.
|
||||
|
||||
**Profoundly unethical and illegal**
|
||||
|
||||
### Whistleblowing
|
||||
|
||||
**Negligence by inaction is not ethical**
|
||||
|
||||
Whistleblowing is the final recourse, is the last resort.
|
||||
@@ -0,0 +1,173 @@
|
||||
# Dependable Computing
|
||||
|
||||
### What is a dependable System
|
||||
|
||||
Another way of putting it is that computing systems, especially systems built into societal infrastructure, and which are otherwise safety-critical as London ambulance system was, are **dependable**.
|
||||
|
||||
**Dependability** is defined by Brian Randell as the **trustworthiness** of a computer system such that reliance can justifiably be placed on the service it delivers. Dependability thus includes such properties as:
|
||||
|
||||
- Reliability
|
||||
- Integrity
|
||||
- Privacy
|
||||
- Safety
|
||||
- Security
|
||||
- Maintainability
|
||||
|
||||
And provides a convenient means of subsuming these various concerns within a single conceptual framework.
|
||||
|
||||
**Reliability** means that a system provides continuity of correct service during its useful lifetime, from commisioning, through operation, to decomissioning.
|
||||
|
||||
**Safety** means that a system is engineered to avoid catastrophic consequences for user and the environment and that the life-critical system behaves as needed, even if components fail.
|
||||
|
||||
**Integrity** means that a system’s source code or state cannot be altered improperly, i.e., it is secure, or its data be corrupted.
|
||||
|
||||
**Maintainability** means that a system is engineered to permit adaptive maintenance, ease of modification and repair of defects.
|
||||
|
||||
#### Dependability
|
||||
|
||||
##### Uber’s self-driving car accident
|
||||
|
||||
- Back up drivber charged with negligent homicide
|
||||
- However the National Transport Safety Board finds ubers system to be at fault
|
||||
- While Uber’s radar and Lidar detected Elaine 6 seconds before the impact, their system did not have the capacity to **classify** the object as a pedestrian unless they were near a crosswalk
|
||||
- It classified Elaine as a vehicle, bicycle and an unknown object
|
||||
- It assumed Elaine would be travelling in the same direction as the car and therefore did not slow down
|
||||
- Furthermore, the car had its own in-built automatic braking system which was capable of detecting and stopping for Elaine, but it was disabled by Uber engineers as they thought it would interfere with Uber’s self driving sensors
|
||||
- When the car was just a second away from Elaine, Uber’s system finally recognised that the object could not be avoided
|
||||
- Now at this point, Uber’s system could have slammed on the brakes to migate the imapact, instead an *action supression* component kicked in.
|
||||
- This was implemented to avoid extreme manoeuvers in response to false alarms.
|
||||
- Uber couldn’t supply documents showing checks performed on the backup driver
|
||||
|
||||
Computing failures are not restricted to 1 car and 2 plane crashes
|
||||
|
||||
The FDA reports, that medical device recalls are at an all time high and that defective software is a major cause. One in every three medical devices that use software for operations have been **recalled** because of **failures in their software**.
|
||||
|
||||
As the Uber and Boeing cases clearly demonstrate, dependability is still a critical issue in computing today.
|
||||
|
||||
- Apart from the direct human cost, the failure of computing systems costs a great deal of money.
|
||||
- The 5th edition of the Software Fail Watch identified 606 recorded software failures, impacting half of the world’s population (3.7 billion people) and 314 companies to the cost of 1.7 trillion dollars, and noted that “this is just scratching the surface – there are far more software defects in the world than we will likely ever know about.”
|
||||
|
||||
We have an ethical duty to the public to minimise these harms. I purposefully say minimise and not eradicate, as it is inevitable that things will go wrong some-times due to unforeseen circumstances, but if we exercise due diligence in our work then we should be able to significantly reduce the harms caused through what are euphemistically called “software bugs”.
|
||||
|
||||
#### Software Bugs
|
||||
|
||||
A software bug is defined as an error, flaw or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways.
|
||||
|
||||
##### Debugging and Testing
|
||||
|
||||
###### Waterfall Model
|
||||
|
||||
The waterfall model places testing after requirements, analysis and specification, software design and implementation.
|
||||
|
||||
- Placing testing here is problematic as it means testing only takes place during the later stages of development
|
||||
- The waterfall model is inflexible and has been widely blamed for a great many large-scale projects running over budget, over time and failing to deliver on requirements
|
||||
|
||||
###### V Model
|
||||
|
||||
The V Model adapts the waterfall by placing an emphasis on early testing
|
||||
|
||||
- V model is often criticised for squeezing testing into tight windows at the end of development phases when earlier stages have overrun but implementation dates remain fixed.
|
||||
|
||||
###### Spiral Model
|
||||
|
||||
Spiral model provides a major alternative and places testing, in iterative requirements, design, implement and test sequences that spiral out from one another and are marked by the development of increasingly high fidelity prototypes
|
||||
|
||||
##### Testing Methodologies
|
||||
|
||||
###### Static Testing
|
||||
|
||||
- Static testing takes place early in a software system’s development and examines source code and accompanying documentation but doesn’t execute the program.
|
||||
- It may be done manually, though increasingly relies on automated analysis tools.
|
||||
|
||||
###### Dynamic Testing
|
||||
|
||||
- Dynamic testing checks the behaviour of software code when it is executed.
|
||||
|
||||
- Testers compare outputs with expected behaviour to determine whether or not the software works as intended.
|
||||
|
||||
###### White Box Testing
|
||||
|
||||
White box testing digs into the inner workings of the software.
|
||||
|
||||
- It tests each statement, object, and function on an individual basis
|
||||
- Identifies broken or poorly structured paths in coding processes
|
||||
- Internal security holes
|
||||
- It also verifies the flow of specific inputs through the code and expected outputs.
|
||||
|
||||
###### Black Box Testing
|
||||
|
||||
Black box testing on the other hand examines the outer workings of the software and that the software does what it’s supposed to do.
|
||||
|
||||
- Knowledge of coding isn’t necessary, and testers work at the user-interface level checking inputs and outputs.
|
||||
|
||||
###### GUI Testing
|
||||
|
||||
Graphical user interface or GUI testing
|
||||
|
||||
- Checks user interface works as per the GUI specification.
|
||||
- It tests the software control dialogues, including:
|
||||
- screen layouts
|
||||
- menus
|
||||
- buttons
|
||||
- icons, pop-up windows, text boxes, text formatting, colours, fonts, font sizes, etc.
|
||||
|
||||
##### Testing Levels
|
||||
|
||||
###### Unit Testing
|
||||
|
||||
###### Component or Module Testing
|
||||
|
||||
###### Integration Testing
|
||||
|
||||
###### System Testing
|
||||
|
||||
###### Alpha, Beta and acceptance Testing
|
||||
|
||||
### Testing and Dependability
|
||||
|
||||
As Linda Rosenberg and her colleagues told us in their award-winning 1998 IEEE paper on software reliability,
|
||||
|
||||
> “Metrics to measure software reliability exist and can be used starting in the requirements phase. At each phase of the development life cycle, metrics can identify potential areas of problems that may lead to problems or errors. Finding these areas in the phase they are developed decreases the cost and prevents potential ripple effects from the changes, later in the development life cycle by at least a factor of 14.”
|
||||
|
||||
#### Limits of Testing
|
||||
|
||||
Brian Randell tells us that
|
||||
|
||||
> “a system **failure** occurs when the delivered service no longer complies with the **specification**, the latter being an agreed description of the system's expected function and/or service.”
|
||||
|
||||
Daniel Jackson and colleagues elaborate the point, saying that,
|
||||
|
||||
> “Software, according to a popular view, fails because of bugs: errors in the code that cause the software to fail to meet its specification. In fact, only a tiny proportion of failures due to the mistakes of software developers can be attributed to bugs – **3%** in one study that focused on fatal accidents. As is well known to software engineers (but not to the general public), by far the largest class of problems arises from errors made in the eliciting, recording, and analysis of requirements.
|
||||
|
||||
### Boeing 737 MAX 8
|
||||
|
||||
The bug at work here was a **faulty** angle of attack or AOA **sensor**, which indicated the angle at which the aircraft was positioned in flight.
|
||||
|
||||
The Ethiopian accident investigation report says that Boeing’s engineers determined that no piloted simulation, was required for take-off or low speed flight. This meant that specific failures that could lead to MCAS activation, such as false AOA input, were not simulated as part of the aircraft’s functional hazard assessment and validation tests.
|
||||
|
||||
Boeing assumed that the worse that could happen would be single fault-driven MCAS activation that flight crew would correct as per “trained memory procedures” acquired during flight training for previous 737 models. As the graph showing the plane going up and down in the Vox video makes painfully visible, the MAX 8 crashes involved multiple MCAS activations, caused by the faulty AOA sensor.
|
||||
|
||||
Poor specification requirements: Input was only required from one AOA sensor to activate MCAS, depsite two sensors being fitted.
|
||||
|
||||
- This means the faulty sensor constantly triggered MCAS
|
||||
- No information about MCAS was given in the flight crew manuals and MCAS was not included in flight crew training.
|
||||
- Boeing assumed that pilots certified to fly on earlier versions of the 737 didn’t need any extra training.
|
||||
- The lack of documentation and training meant that flight crews were unaware of MCAS and its effects
|
||||
- The lack of information about MCAS in the flight crew manual meant that there were no procedures for mitigating erroneous input from the AOA sensors
|
||||
- An AOA disagree warning light would flash if the two sensors were at odds with each other
|
||||
- These indicators were sold as optional extras
|
||||
- These extras were not found on either aircraft
|
||||
- The Indonesian crash report finds that the flight crew were **not aware** that the AOA DISAGREE warning would not appear if AOA DISAGREE conditions were met, and that in failing to install the warning lights **Boeing denied the flight crew valid information** about the abnormal conditions they faced
|
||||
|
||||
It becomes apparent then that the **AOA sensor bug wasn’t really the problem**. It could well have been handled
|
||||
|
||||
- Had MCAS not been designed to activate off input from a single sensor
|
||||
- If flight crew had been informed about MCAS
|
||||
- Its effects built into difference training and the flight crew manual,
|
||||
- Had the planes been fitted (like their predecessors) with the AOA warning lights.
|
||||
|
||||
The crashes are as much, if not more, a failure of poor requirements, including both poor technical and usability specifications, and inadequate, indeed non-existent, documentation and training, which are also key parts of the user interface to and usability of a system.
|
||||
|
||||
There are limits to software testing.
|
||||
|
||||
Dependability relies as much on **sound requirements specifications** as it does **good code** and **rigorous testing**.
|
||||
@@ -0,0 +1,166 @@
|
||||
# Secure By Design
|
||||
|
||||
> “Security vulnerabilities are to some extent an exception; the overwhelming majority of security vulnerabilities reported in software products – and exploited to attack the users of such products – occur at the implementation level.” - Daniel Jackson
|
||||
|
||||
**Integrity**: Ensuring the security of both a system and its data, including unauthorised disclosure of data.
|
||||
|
||||
Security is legally required for systems that process personal data.
|
||||
|
||||
> GDPR Requires that personal data be secured through the implementation of technical **and** organisational measures. Technical measures include the pseudonymisation and encryption of personal data; the ability to ensure the ongoing confidentiality, integrity, availability and resilience of processing systems and services; and the ability to restore the availability of and access to personal data in a timely manner in the event of a physical or technical incident.
|
||||
|
||||
#### Why is Security so Important
|
||||
|
||||
In the UK 46% of businesses and 26% of charities have delt with cyber attacks
|
||||
|
||||
Ransomware is the fastest growing type of cybercrime and costs are predicted to reach 20 billion dollars by 2021, which is 57 times greater than it was in 2015.
|
||||
|
||||
Cyber security breaches have increased globally by 67% since 2014. They essentially operate in 2 ways:
|
||||
|
||||
1. Through bad actors, particularly people who try to phish for and otherwise elicit usernames and passwords to access systems
|
||||
2. Through bad computing, particularly the use of viruses, malware and denial of service attacks that compromise systems.
|
||||
|
||||
It is broadly acknowledged that IoT devices, which typically exploit low cost sensors, suffer from extremely poor and indeed non-existent security.
|
||||
|
||||
#### Causes of poor Security
|
||||
|
||||
In addition to internal reasons to do with poor coding and testing, and poor specification of technical and usability requirements, poor security has also been attributed to the law and limits of liability.
|
||||
|
||||
In the US, for example, the courts have consistently interpreted software licenses in a way that allows vendors to disclaim almost all liability for software defects.
|
||||
|
||||
**The economic loss**: rule states that if a product causes no personal injury or property damage, other than to the product itself, then such damages are determined by contract law and limited to a breach of contract claim.
|
||||
|
||||
- This prevents customers from suing as most often claims consist of
|
||||
- Loss of sensitive & personal data
|
||||
|
||||
Then there is the fact that any data entered into a computer system by the user is **not considered part of the software**, and hence **not part of the product**. The data and the software are separate. The data can be read and manipulated by the software, but it is created by the user or a third party, not the software vendor. Therefore, destruction of data due to insecure software is not deemed damage to or destruction of the software itself.
|
||||
|
||||
Now GDPR, the EU’s updated data protection regulation, goes some way towards incentivising secure treatment of personal data with its 20 million euro fines for anyone who **fails to put adequate technical and organisational safeguards in place**, but that of course only **applies to the parties who process such data**, and **not to those who build**, **distribute**, **sell**, or **maintain** the software they use.
|
||||
|
||||
#### National Cyber Security Strategy
|
||||
|
||||
UK Govement invested £1.9 bn in its National Cyber Security strategy in 2016.
|
||||
|
||||
The UK’s National Cyber Security Strategy stands on 3 pillars:
|
||||
|
||||
1. **DEFEND**: the country against evolving cyber threats, which involves responding effectively to incidents, ensuring UK networks, systems and data are protected and resilient, and providing UK citizens and businesses with the knowledge needed to defend themselves.
|
||||
2. **DETER**, which involves detecting, investigating and disrupting hostile action, and pursuing and prosecuting offenders.
|
||||
3. **DEVELOP** a self-sustaining pipeline of talent providing the skills to meet national needs across the public and private sectors.
|
||||
|
||||
#### Secure By Design
|
||||
|
||||
Cyber-physical systems include software systems that not only compute but also act in the world, e.g., IoT devices such as smart thermostats or smart door locks or autonomous systems such as self-driving cars.
|
||||
|
||||
**Secure by design:** software has been designed from its foundations up to be secure.
|
||||
|
||||
NCSC articulates **5 core secure by design principles**. These include:
|
||||
|
||||
1. Establishing the context before designing a system
|
||||
- Risk analysis is **critical**
|
||||
- Component-driven analysis and system-driven analysis (see below)
|
||||
2. Making compromise difficult
|
||||
- External data inputs cannot be trusted
|
||||
- Data inputs must be sanitised, validated
|
||||
- Attack surfaces should be minimised, exposing as few components as possible
|
||||
- Read-only views should be enforced where ever possible
|
||||
- All privileged actions should be accessed through control functions and must be attributed to individuals
|
||||
3. Making disruption difficult
|
||||
- Identify system bottlenecks
|
||||
- Test systems with unreasonably high loads and Ddos attacks
|
||||
- Understanding how the system responds to failure
|
||||
- Monkey testing
|
||||
4. Making compromise detection easier
|
||||
- Monitoring system behaviour
|
||||
- Logging security events
|
||||
- Like a log of all logins and logouts
|
||||
- Ensuring the monitoring is independent of the software itself
|
||||
5. Reducing the impact of compromise.
|
||||
- Removing unnecessary functionality such as debug or test functionality
|
||||
- Segmenting assets on networks to contain breaches to particular segments
|
||||
- Designing systems so that they can be quickly rebuilt to a known clean state
|
||||
|
||||
###### Component-driven Analysis
|
||||
|
||||
Focuses on the technical components a system is composed of, the threats and vulnerabilities that may effect those components, and the impact caused if any of the components was compromised.
|
||||
|
||||
This type of analysis allows the specific risks faced by specific components within a system to be identified and prioritised
|
||||
|
||||
1. According to the **ease** with which a vulnerablity could be exploited and a component comprimised.
|
||||
2. According to the **severity** of impact.
|
||||
|
||||
The purpose of prioritising risks in this way is to mitigate the worst risks first.
|
||||
|
||||
###### System-driven Analysis
|
||||
|
||||
Focuses on understanding the purposes of the system, i.e., what it is being built to do, its functionality or the services it offers. System-driven analysis should not only identify what a system should do but also what it should not do.
|
||||
|
||||
NCSC suggests we rarely consider what a system should not do at the beginning of the project’s lifecycle.
|
||||
|
||||
### Securing the IoT
|
||||
|
||||
There are more the 10 billion IoT devices as of 2021. This inevitably creates an exponential increase in the attack surface and opens up society to cyber attack on an unprecedented scale, especially as IoT devices are broadly recognised to have very poor cyber security.
|
||||
|
||||
#### Guidelines
|
||||
|
||||
1. **No longer set default passwords**
|
||||
|
||||
- Many IoT devices are compromised by the Mirai botnet, which exploits default passwords set by manufacturers.
|
||||
|
||||
- All IoT device passwords should be unique and should not reset to a universal factory default.
|
||||
|
||||
2. **Vulnerability disclosure policy**
|
||||
|
||||
- Provide a public point of contact to enable security researchers and users to report issues.
|
||||
- This enables the continual monitoring, identification and rectification of security vulnerabilities as part of a device’s security lifecycle.
|
||||
|
||||
3. **Keep their software updated**
|
||||
|
||||
- Security patches should be delivered over a secure channel and their provenance be assured.
|
||||
|
||||
4. **Secure data storage**
|
||||
|
||||
- Sensitive data, including cryptographic keys, device identifiers and initialisation vectors, should be **stored securely** using mechanisms provided by a Trusted Execution Environment.
|
||||
|
||||
5. **Secure Communications**
|
||||
|
||||
- All data should be encrypted in transit to ensure **secure communications**.
|
||||
|
||||
6. **Minimise the attack surface of devices**
|
||||
|
||||
- Device manufacturers and service providers should ensure hardware does not unnecessarily expose access points
|
||||
- Unused ports should be closed, services should not be available if they are not used, and code should be minimised to the functionality necessary for the service to operate.
|
||||
- All devices should operate on the principle of least **privilege**
|
||||
- Giving users or processes only those privileges essential to the performance of their intended function.
|
||||
|
||||
7. **Ensure software integrity**
|
||||
|
||||
- Using secure boot mechanisms to verify software.
|
||||
- If an unauthorised change is detected, the device should alert the consumer and not connect to wider networks, other than those necessary to perform the alerting function.
|
||||
|
||||
8. **Resilient to outages**
|
||||
|
||||
- Whenever possible, IoT systems should remain operating and be **locally functional** in the case of a loss of network connectivity and should recover cleanly in the case of restoration of a loss of power.
|
||||
|
||||
9. **Easy to install and maintain**
|
||||
|
||||
- User interfaces should be easy to use and clear guidance should be provided to users to set up devices securely and reduce their exposure to threats.
|
||||
|
||||
10. **Monitor telemetry data**
|
||||
|
||||
- Telemetry data (such as usage and measurement data) allows for unusual circumstances to be identified and dealt with, minimising security risks and allowing quick mitigation of problems.
|
||||
|
||||
11. **Sanitise Inputs**
|
||||
|
||||
- Manufacturers, service providers and mobile app developers should ensure that **data input** via user interfaces, and any transferred via APIs or between networks, is **validated**.
|
||||
|
||||
12. **Protect personal data**
|
||||
|
||||
- Device manufacturers, service providers, mobile app developers and retailers should also ensure that any **personal data** collected by IoT devices is **protected**
|
||||
|
||||
- Users are provided with means to preserve their privacy through configuring device and service functionality.
|
||||
|
||||
13. **Delete personal data**
|
||||
|
||||
- Users should be able to **delete personal data** easily if they wish to, when there is a transfer of ownership, or when they dispose of a device.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
# Data Protection by Design and Default
|
||||
|
||||
**DPbDD** - Data Protection by Design and Default
|
||||
|
||||
##### What is Privacy
|
||||
|
||||
- It is a state in which one is not observed or disturbed by other people
|
||||
- Or the state of being free from public attention
|
||||
- Or someone’s right to keep their personal matters and relationships secret
|
||||
- Or freedom from unauthorised intrusion
|
||||
- Or the right to make personal decisions regarding intimate matters
|
||||
- Or the right to lead one’s life in a manner that is reasonably secluded from public scrutiny
|
||||
|
||||
And so on
|
||||
|
||||
> Privacy allows us to negotiate who we are and how we want to interact with the world around us, and is essential to who we are as human beings. It gives us a space to be ourselves without judgement, allows us to think freely without discrimination, and is essential to individual autonomy and the protection of human dignity.
|
||||
|
||||
https://privacyinternational.org/explainer/56/what-privacy
|
||||
|
||||
> “No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks.” **Article 12 of the UN declaration**
|
||||
|
||||
> **Article 8.1 of the EU convention – the right to respect for private and family life**
|
||||
>
|
||||
> 1. Everyone has the right to respect for his private and family life, his home and his correspondence;
|
||||
> 2. There shall be no interference by a public authority with the exercise of this right except such as is in accordance with the law and is necessary in a democratic society in the interests of national security, public safety or the economic well-being of the country, for the prevention of disorder or crime, for the protection of health or morals, or for the protection of the rights and freedoms of others.
|
||||
|
||||
Privacy is a fundamental human right and underpins many other human rights including freedom of association and free speech.
|
||||
|
||||
It’s politically contentious status makes it an ethical imperative in professional computing and key to ensuring public confidence and trust.
|
||||
|
||||
> That’s why the BCS and ACM include “respect for privacy” as a requirement in their ethics codes, and the IEEE has a separate Data Access and Use policy to align it with industry best practice and ensure compliance with international regulations including the European Union’s General Data Protection Regulation or GDPR
|
||||
|
||||
### Informational Privacy
|
||||
|
||||
Informational privacy is a subset of general privacy concerns.
|
||||
|
||||
Warren and Brandeis argued that technology enabled harms to privacy including intrusion into one’s private life and affairs
|
||||
|
||||
- public disclosure of embarrassing private facts
|
||||
- unwanted publicity
|
||||
- misuse of a person’s name or likeness for financial advantage.
|
||||
|
||||
Informational privacy is thus a concern with the protection of personal or private information from unauthorised disclosure and misuse. https://plato.stanford.edu/entries/privacy
|
||||
|
||||
### Relevant Authority
|
||||
|
||||
From a UK perspective, GDPR still applies because it was adopted into UK law by the Data Protection Act 2018 and is enforced by the Information Commissioner’s Office or ICO, so it’s clearly relevant to BCS accreditation.
|
||||
|
||||
> GDPR has global reach and violations may result in fines of up to 20 million euros or 20 up to 4 % of the total worldwide annual turnover of the preceding financial year, whichever is higher (Article 83)
|
||||
|
||||
###### Definitions
|
||||
|
||||
The **data subject** is a natural person, an individual who can be identified, directly or indirectly, by the personal data.
|
||||
|
||||
**Personal data** is **any** information relating to an identified **or** identifiable person (i.e., the ‘data subject’), **either directly or indirectly**. Personal data includes a bunch of technical information including such things as account handles, IP or MAC addresses, cookies, RFID frequencies, device fingerprints, etc.
|
||||
|
||||
- The key point here is that personal data may not directly link to a *data subject* as say a passport might
|
||||
- But may relate indirectly to a person once the data has been procesed
|
||||
|
||||
**Processing** means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means.
|
||||
|
||||
Processing includes:
|
||||
|
||||
- collection
|
||||
- structuring
|
||||
- storage
|
||||
- alteration
|
||||
- retrieval
|
||||
- combination
|
||||
- adaptation
|
||||
- consultation
|
||||
- use, disclosure, dissemination, making available, restriction, erasure or destruction of personal data.
|
||||
|
||||
Basically, if you touch someone’s personal data in any way you are involved in processing it. Processing does not just mean the data is run through a computer in some way.
|
||||
|
||||
Similarly, **processor** does not refer to a CPU on a computer, but to the person, legal entity, public authority, agency or other body which processes personal data on behalf of the controller and may use computing to do so.
|
||||
|
||||
**Controller** means the person, legal entity, public authority, agency or other body which, alone or jointly with others, determines the purposes for which personal data will be processed and the means of processing them.
|
||||
|
||||
**Data protection** officer or **DPO**, who may be an employee of the controller or processor or an independent contractor who has expert knowledge of data protection law and must be consulted by the controller or processor in a timely manner in all issues which relate to the protection of personal data. A DPO must be appointed if a controller or processor’s core activities involve the processing of personal data on a large scale or involve large scale, regular and systematic monitoring of individuals.
|
||||
|
||||
#### GDPR
|
||||
|
||||
GDPR places specific legal requirements on controllers, which directly impact processors.
|
||||
|
||||
> **Article 23 of GDPR** says that, “1. Taking into account the state of the art, the cost of implementation and the nature, scope, context and purposes of processing as well as the risks … posed by the processing, the controller shall, both at the time of the determination of the means for processing and at the time of the processing itself, implement appropriate technical and organisational measures … in an effective manner and … integrate the necessary safeguards into the processing in order to meet the requirements of this Regulation and protect the rights of data subjects. 2. The controller shall **implement** appropriate technical and organisational measures … **by default** …”
|
||||
|
||||
> The European Data Protection Board or EDPD, which furnishes guidance on GDPR tells us that, “a ‘default’, as commonly defined in computer science, refers to the pre-existing or preselected value of a configurable setting that is assigned to a software application, computer program or device. Such settings are also called ‘presets’ or ‘factory presets’.” EDPB Guidelines
|
||||
|
||||
So the term **implement by default** in GDPR refers to the design of preset technical and organisational measures to ensure that data processing operations meet the requirements of GDPR and thus protects the legal rights of data subjects. We’ll take a look at what those presets are about shortly.
|
||||
|
||||
The controller is legally **accountable** for the choice of presets and implementing data protection by design and default. (Article 5 GDPR)
|
||||
|
||||
This means that the controller must be able to **demonstrate** to themselves, to data subjects and to supervisory authorities alike that the technical and organisational measures they have put in place are a) appropriate and b) effective in ensuring data protection by design and default.
|
||||
|
||||
### Data Protection by Design and default
|
||||
|
||||
By default controllers must be **transparent** about how they collect, use and share personal data and how data subjects may exercise their legal rights over data processing.
|
||||
|
||||
These include:
|
||||
|
||||
- the right to access any personal data held by the controller that relates to the data subject (Article 15)
|
||||
- to object to the processing of personal data (Article 21)
|
||||
- obtain human intervention when querying automated decisions (Article 22)
|
||||
- to restrict processing (Article 18)
|
||||
- to rectify inaccuracies (Article 16)
|
||||
- to export data in a commonly used and machine-readable format (Article 20)
|
||||
- to have data erased and be forgotten (Article 17).
|
||||
|
||||
> **Recital 63** which says, “Where possible, the controller should be able to provide remote access to a secure system which would provide the data subject with direct access to his or her personal data.”
|
||||
|
||||
So transparency is something that needs to built into systems in the long term and not simply be seen as a matter of appending documentation to their use.
|
||||
|
||||
The controller must also by default identify and declare a **valid legal basis** for the processing. Six legal grounds exist including:
|
||||
|
||||
1. consent
|
||||
2. performance of a contract
|
||||
3. compliance with a legal obligation
|
||||
4. protecting vital interests
|
||||
5. carrying out a task in the public interest or official duty
|
||||
6. pursuing legitimate interests.
|
||||
|
||||
Fairness is an overarching principle of data protection, which requires that personal data should not be processed in ways that are unjustifiably detrimental, unexpected or misleading to the data subject.
|
||||
|
||||
Fairness is especially important with respect to data processing operations that rely on machine learning and AI, for as we saw in lecture 2 these technologies are responsible for widespread discrimination.
|
||||
|
||||
The controller must also ensure that data is only collected for **specific, explicitly stated purposes** and that data is not further processed in a manner that is incompatible with the purposes for which they were initially collected.
|
||||
|
||||
This is called **purpose limitation**. It means a controller cannot simply collect as much data as they like and do with it what they want. Data collection must be limited by default to specific purposes which are transparent to the data subject.
|
||||
|
||||
**Data minimisation**: the controller must ensure that data collection is limited to what is necessary to meet the purposes for which they are being processed.
|
||||
|
||||
Data minimisation requires that the controller verify whether the purposes can be achieved by processing less personal data, or having less detailed or aggregated personal data or without having to process personal data at all. Such verification should take place before any processing takes place, and be carried out at any during the processing lifecycle.
|
||||
|
||||
Data minimisation also refers to the degree of identification. If the purpose does not require the final set of data to refer to an individual (such as statistics) - then the controller should delete or anonymise personal data as soon as possible. If continued identification is needed for other processing activities, personal data should be pseudonymized to mitigate risks for the data subjects’ rights.
|
||||
|
||||
By default, the controller must **limit** the period for which personal data kept in a form which permits identification of data subjects are **stored** and retain data in such a form for no longer than is necessary to meet the purposes for which it has been collected.
|
||||
|
||||
No time periods are specified by GDPR, it all depends on the purposes for which the data was collected and the risks that attach to keeping the data in identifiable form. Anonymised data can be stored indefinitely, though risks of reverse engineering attach to pseudonymised data, which need to be mitigated if the data is to be retained for long periods.
|
||||
|
||||
By default, the controller must put technical and organisational measures in place to protect personal data against unauthorised access, accidental loss, destruction or damage, and to manage data breaches.
|
||||
|
||||
- Regular reviews should be conducted to make sure it is being stored securely
|
||||
|
||||
### Data Protection Impact Assessment
|
||||
|
||||
DPIA - **D**ata **P**rotection **I**mpact **A**ssessments
|
||||
|
||||
> DPIAs are mandated by **Article 35 GDPR**, which says that “Where a type of processing in particular using new technologies … is likely to result in a high risk to the rights and freedoms of natural persons, the controller shall, prior to the processing, carry out an assessment of the impact of the envisaged processing operations on the protection of personal data.”
|
||||
>
|
||||
> Article 35 goes on to say that a DPIA “shall in particular be required” in the case of automated processing, including profiling, and systems that produce decisions that have legal effects (e.g., which effect a person’s right to claim state benefits) or similarly significantly affect the natural person (e.g., by assessing their creditworthiness). This applies especially to machine learning and AI systems.
|
||||
|
||||
A DPIA is also required by law where large amounts of special category data are processed.
|
||||
|
||||
Special category data is data that reveal racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, bio-metric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person's sex life or sexual orientation.
|
||||
|
||||
DPIAs are legally required for these areas of personal data processing, but they are generally recommended as “good practice” for any processing of personal data. https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/accountability-and-governance/data-protection-impact-assessments/
|
||||
|
||||
### How to know when processing is high risk
|
||||
|
||||
There are 4 critieria specified in GDPR article 35
|
||||
|
||||
1. The use of new technologies to process personal data
|
||||
2. Automated-decision making with legal or significant effect
|
||||
3. Processing of special category data
|
||||
4. Systematic monitoring of public spaces
|
||||
|
||||
There are additional criteria
|
||||
|
||||
5. **Evaluation or scoring, including profiling and predicting**
|
||||
- especially of data concerning the data subject's performance at work, economic situation, health, personal preferences or interests, reliability or behavior, location or movements.
|
||||
- Examples of this are financial institutions that screen customers against a credit reference database
|
||||
6. **The processing of sensitive data or data of a highly personal nature**
|
||||
- Not only special categories of personal data, but also any data considered as sensitive as the term is commonly understood
|
||||
- e.g., data linked to household and private activities (such as electronic communications), or data that impact the exercise of a fundamental right (such as location data whose collection may impact freedom of movement), financial data, personal documents, personal information contained in life-logging applications, etc.
|
||||
7. **The processing of personal data on a large scale**
|
||||
- which is determined by the number of data subjects concerned
|
||||
- the volume of data and/or the range of different data items being processed
|
||||
- the duration or permanence of the data processing activity
|
||||
- the geographical extent of the processing activity
|
||||
8. **Matching or combining datasets**
|
||||
- data originating from two or more data processing operations performed for different purposes and/or by different data controllers in a way that would exceed the reasonable expectations of the data subject.
|
||||
9. **Data is processed that relates to vulnerable data subjects**
|
||||
- For example, children, employees, and vulnerable persons requiring special protection such as mentally ill persons, asylum seekers, the elderly, patients, etc.
|
||||
- Indeed any personal data where an imbalance in the relationship between the data subject and the controller can be identified and processing increases the power imbalance between them.
|
||||
10. **Data processing that prevents data subjects from exercising a right, using a service or entering into a contract**
|
||||
- This includes processing operations that permit, modify or refuse data subjects’ access to a service or entry into a contract.
|
||||
- An example of this is where a bank screens its customers against a credit reference database in order to decide whether to offer them a loan.
|
||||
|
||||
**If a processing operation meets 2 of these criteria, then a DPIA is required by law.**
|
||||
|
||||
#### Whats involved in carrying out a DPIA?
|
||||
|
||||
###### Step 1
|
||||
|
||||
Identify the need for a DPIA, which is done by applying the criteria we have just discussed.
|
||||
|
||||
Must be done before processing takes place
|
||||
|
||||
###### Step 2
|
||||
|
||||
Specify the nature of the processing including the source of the data
|
||||
|
||||
- the nature of the data and its status (e.g., special category, sensitive, vulnerable, etc.)
|
||||
- how it will be collected, used, stored and deleted
|
||||
- the amount of data to be collected
|
||||
- the frequency and duration of collection and storage, and the geographical area covered
|
||||
- the flow of data and if it will be shared, how and with who
|
||||
- any types of processing that are identified as high risk.
|
||||
|
||||
Also involves specifying the purpose or purposes of the processing and what the controller wants to achieve by processing the data, including the intended effect on data subjects (if any), the benefits of the processing to the controller and more broadly.
|
||||
|
||||
###### Step 3
|
||||
|
||||
Is consider the need for consultation
|
||||
|
||||
1. when and how the views of data subjects will be sought
|
||||
2. justifying why it is not appropriate to do so
|
||||
|
||||
Third & external parties need to be consulted to ensure data protection by design and default
|
||||
|
||||
###### Step 4
|
||||
|
||||
Accessing necessity and proportionality, which involves specifying how the processing will actually achieve the purpose and that there is no other way to achieve the same outcome.
|
||||
|
||||
- the lawful basis for processing
|
||||
- how data minimisation and data quality will be ensured
|
||||
- how function creep will be prevented; what information will be given to data subjects and their rights will be supported
|
||||
- measures that will be taken to ensure processors are in compliance with DPbDD
|
||||
- how any international data transfers will be safeguarded.
|
||||
|
||||
###### Step 5
|
||||
|
||||
Identify and assess risks and involves identifying sources of risk and specifying
|
||||
|
||||
1. risks to data subjects
|
||||
2. corporate risks
|
||||
3. compliance risks
|
||||
|
||||
and the potential impact of each.
|
||||
|
||||
###### Step 6
|
||||
|
||||
Identify and specify measures to mitigate the risks, including the options available to
|
||||
|
||||
1. reduce risk
|
||||
2. eliminate risk
|
||||
|
||||
###### Step 7
|
||||
|
||||
Have the DPAI signed off and outcomes recorded. If the DPO’s advice is overruled, justification must be provided, as must the reasons for not abiding by consultation outcomes. A **review date must also be specified** for the DPIA and done so over the lifetime of a processing operation.
|
||||
|
||||
You cannot do a DPIA on your own. IBM’s Dave Whitelegg says you must have the following invovled
|
||||
|
||||
> - The developer lead or project manager, who is responsible for managing the DPIA process.
|
||||
> - A data protection officer who must be consulted about and sign off on the DPIA process*.*
|
||||
> - A security specialist who must verify that best practises are adopted throughout development.
|
||||
> - A risk manager to advise on privacy risk management.
|
||||
> - Project sponsors and business directors, who are accountable for privacy risks.
|
||||
> - And where processing operations are developed for external organisations, who must be able to verify that the processing is compliant with GDPR.
|
||||
|
||||
### Relevance of DPbDD and DPIA to computing
|
||||
|
||||
Now you might be tempted to think that data protection by design and default and DPIAs have little if anything to do with the actual business of developing computing systems.
|
||||
|
||||
However, we should not forget that documentation is a key part of the software engineering process – particularly requirements engineering – and that poor requirements specification is a primary source of computing failure.
|
||||
|
||||
> As IBM’s Dave Whitelegg puts it, “GDPR privacy obligations should be documented as requirements within the requirements analysis phase of the Software Development Lifecycle or SDLC. The DPIA should be performed within the design phase of the SDLC. Then, further privacy risk verification should be conducted throughout the latter phases of the SDLC, to assure the privacy requirements are all achieved, and the design mitigates or eliminates privacy risks as intended.” *Dave Whitelegg (2018) Application privacy by design*
|
||||
|
||||
#### Privacy Engineering
|
||||
|
||||
> Dave Whitelegg also tells us that while GDPR does not prescribe technical solutions, indeed it is in its own words “technologically neutral” (GDPR, recital 15), that nonetheless a “number of technical solutions that can be utilized to significantly enhance the protection of personal data” *Dave Whitelegg (2018) Minimising application privacy risk.*
|
||||
|
||||
###### OWASP’s Security Principles
|
||||
|
||||
1. Data anonymisation methods include: nulling, deletion and redaction, which involves removing all direct and indirect identifier fields in a dataset,
|
||||
- removing names or postcodes.
|
||||
2. Substitution, which involves overwriting personal data identifier fields with fake personal data.
|
||||
3. Data masking, which involves substituting identifier field characters with a ‘mask’ character,
|
||||
- e.g., inserting X’s instead numbers on a credit card field.
|
||||
4. Scrambling / shuffling, which involves moving the contents of identifier fields around
|
||||
- e.g. moving surnames up or down.
|
||||
5. Aggregation / generalisation, which involves rendering data in statistical form.
|
||||
6. Hashing provides a method of pseudonymisation and involves using an algorithm to transform personal data fields into alphanumeric strings.
|
||||
7. Penetration testing is recommended to verify whether these methods enable reidentification in any actual case.
|
||||
|
||||
https://owasp.org/www-project-top-ten/
|
||||
|
||||
Privacy engineering may help you implement the presets and meet the requirements, but it is your **ethical responsibility** to know and respect the rules that pertain to professional work. You now know what rules you need to follow to respect people’s privacy and protect their data.
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
# Automonous Systems
|
||||
|
||||
Autonomous systems include robots and cyber physical systems that actuate or perform actions in the world, and algorithmic systems particularly machine learning systems or AI.
|
||||
|
||||
The UK robotics and autonomous systems or RAS network identifies 7 key ethical challenges that confront autonomous systems. These include
|
||||
|
||||
1. bias
|
||||
2. opacity
|
||||
3. privacy
|
||||
4. safety
|
||||
5. deception
|
||||
6. employment
|
||||
7. oversight.
|
||||
|
||||
> “The race between job creation through new products and job destruction from new technologies has in the past been won by the job-creating effects of innovation. There is no guarantee for a happy end this time; however, an important lesson from the past is that we tend to under-estimate the job-creating potential of fundamental technological transformations, because we lack sufficient knowledge and imagination about the types of jobs that will be created under the new technological paradigm.
|
||||
>
|
||||
> https://www.europarl.europa.eu/RegData/etudes/STUD/2018/614539/EPRS_STU(2018)614539_EN.pdf
|
||||
|
||||
Alan Winfield and Marina Jirotka in their Royal Society paper on building societal trust in autonomous systems: http://dx.doi.org/10.1098/rsta.2018.0085. They thus propose 5 pillars of good governance, which include establishing a machine intelligence commission to address public fears, including the impact of autonomous systems on jobs. The UK Government established an AI Council in 2019. Regulation is seen as the second pillar of good governance, as are standards, such as those established by professional bodies including the BCS, ACM and IEEE.
|
||||
|
||||
###### The Third Pillar
|
||||
|
||||
Recommends we take particular care about the use of AI in safety critical systems. Of particular concern, as we will take a closer look at later in this lecture, are artificial neural networks, whose decision-making cannot easily be verified. Neural networks learn for themselves and how they arrive at particular decisions is extremely difficult if not impossible to determine.
|
||||
|
||||
###### Fourth Pillar
|
||||
|
||||
Good governance, transparency not only of product, i.e., how an autonomous system arrived at a decision, but also of process and how such machines are developed. The concern with process involves
|
||||
|
||||
- developing ethical codes
|
||||
- ensuring ethical training for everyone involved in development
|
||||
- being transparent about how development is governed
|
||||
- taking the need for good governance seriously.
|
||||
|
||||
###### Fifth Pillar
|
||||
|
||||
Build ethical governors into autonomous systems which would enable a robot or AI system to evaluate the consequences of its actions and modify its actions according to a set of ethical rules.
|
||||
|
||||
This is a longstanding ideal in AI, which must address the fundamental problem of encoding and implementing ethics, all of which begs the question of who’s ethics get encoded and implemented? Pillar five is then the most idealistic, problematic and challenging of Winfield and Jirotka’s proposals.
|
||||
|
||||
### Deception
|
||||
|
||||
Another key ethical challenge of autonomous systems is posed by humanoid or animal-like robots, which create significant risks of emotional attachment and dependency issues, especially for naive or vulnerable users, that we need to be particularly attentive to.
|
||||
|
||||
For example, Babyclon’s animatronic babies and the strong emotions they evoke in those who ‘care’ for them and those who don’t. https://www.theguardian.com/lifeandstyle/video/2020/feb/26/reborn-baby-dolls-women-collectors-video
|
||||
|
||||
The issue of deception is part of a broader set of ethical principles governing the development of robots advocated by the UK’s Engineering and Physical Sciences Research Council or EPSRC
|
||||
|
||||
- **Principle 1** states that robots should not be designed solely or primarily to kill or harm humans, except in the interests of national security.
|
||||
- **Principe 2** states that humans, not robots, are responsible agents and that robots should therefore be designed and operated in compliance with existing laws and respect the fundamental rights and freedoms of human beings, including privacy.
|
||||
- **Principle 3** states that robots should be designed to be safe and secure.
|
||||
- **Principle 4** states that robots are manufactured artefacts and their machine nature should therefore be transparent so as to avoid deception.
|
||||
- **Principe 5** states that the party with legal responsibility for a robot should always be attributed, which is to say that it should always be possible to find out who is responsible for any robot.
|
||||
- This of course is not a straightforward matter as the disruption of flights at airports by drones demonstrates.
|
||||
|
||||
### Algorithmic Bias
|
||||
|
||||
Bias is a concern with the validity of outputs or decisions made by autonomous systems, particularly with whether or not those outputs or decisions **discriminate** against individuals and/or social groups and thus treat them unfairly.
|
||||
|
||||
Discrimination is rife in computing today:
|
||||
|
||||
- webcams that fail to track black people’s faces
|
||||
- auto-tagging of black people and women as animals or gorillas
|
||||
- systematic targeting of racial minorities by the police and undue sentencing of black people
|
||||
- systematic discrimination against female job candidates and black patients in need of healthcare
|
||||
- the A-Level debacle in the UK
|
||||
|
||||
Discrimination, is a specific form of harm based on a personal characteristics including gender identity, marital status, sexual orientation, colour, race, ethnic origin, nationality, religion, age, union membership, political affiliation, military status, and disability.
|
||||
|
||||
These characteristics are otherwise called **“special categories of personal data”** or **“protected characteristics”** and are regulated by GDPR and equality legislation, which would appear to provide a relatively straightforward way of tackling algorithmic bias.
|
||||
|
||||
#### Sources of Algorithmic Bias
|
||||
|
||||
Selena Silva and Martin Kenney identify 9 sources of algorithmic bias within the ML life cycle. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3246252
|
||||
|
||||
1. **Training bias**
|
||||
- The data used to train the algorithm may be unrepresentive or prejudiced
|
||||
- A facial recognition algorithm is trained on data which primarily consists of white faces, it will be worse at recognising black faces and may even categorise them wrongly.
|
||||
2. **Algorithmic focus bias**
|
||||
- The attributes it takes into account and either includes or excludes
|
||||
- The exclusion of gender or race in a health diagnostic algorithm can lead to inaccurate and harmful outcomes.
|
||||
- Whereas the inclusion of gender or race in a sentencing algorithm can lead to discrimination against protected groups.
|
||||
3. **Algorithmic processing bias**
|
||||
- Thomas Guskey and Lee Ann Jung found, for example, that when an ML algorithm processed student grades across a learning module, it scored students based on the average marks for their assignments, but when teachers were given the same data, they adjusted the students’ score according to their progress and understanding of the material and provided a fairer assessment of students learning. https://core.ac.uk/download/pdf/232576892.pdf
|
||||
4. **Non-transparency bias**
|
||||
- The lack of transparency about algorithmic decision-making.
|
||||
- This is not only to do with how decisions were arrived, but also concerns IPR and trade secrets and what developers are willing and expected to divulge about their ML systems and AI
|
||||
5. **Transfer context bias**
|
||||
- The use of ML systems in inappropriate or unintended contexts is also a source of bias. The use of credit scores as a variable in employment provides a ready example of what is called “**transfer context bias**”
|
||||
- Employer’s request credit checks on job candidates, which effectively means that bad credit is being equated with bad job performance.
|
||||
6. **Automation bias**
|
||||
- A human bias which involves the users of algorithmic systems treating outputs as objectively true, rather than as statistical probabilities.
|
||||
- Such as the COMPAS system used by judges in sentencing criminals in the US, provides a good example, where a judge might take the output at face value and apply it uncritically, without reference to other information
|
||||
- Automation bias is very much a case of “computer says so …”
|
||||
7. **Consumer bias**
|
||||
- Is bias expressed by the users of digital platforms
|
||||
- Great care needs to be taken with ML systems trained on such data, as they will reflect consumer bias and be inherently prejudiced in one way or another.
|
||||
8. **Feedback loop bias**
|
||||
- Where ML systems learn from user behaviour, including discriminatory behaviour.
|
||||
- So even though an ML system may have been developed without bias in its training, focus and initial processing of data, over time bias may be introduced through use.
|
||||
- Twitter taught Microsoft’s AI chatbot Tay to be a racist in less than a day.
|
||||
9. **Interpretation bias**
|
||||
- Occurs when users interpret outputs according to their own prejudices. For example, it is ultimately up to a judge to interpret the score provided by a recidivism prediction system such as COMPAS, and to decide what action to take. However, a judge may interpret a risk score of 6 as high in a particular case, while they may treat it as indicator of medium or even low risk in another.
|
||||
@@ -0,0 +1,197 @@
|
||||
# Responsible Research and Innovation
|
||||
|
||||
RRI - **R**esponsible **R**esearch and **I**nnovation
|
||||
|
||||
#### What is RRI
|
||||
|
||||
An 80 billion euro programme to tackle:
|
||||
|
||||
- Health, demographic change and the well-being of citizens
|
||||
- Food security and sustainable agriculture
|
||||
- Sustainable and efficient energy
|
||||
- Smart, green transport
|
||||
- Climate action, resource efficiency and raw materials
|
||||
- Inclusive and innovative society
|
||||
- Secure society protecting the rights and freedoms of citizens
|
||||
|
||||
What RRI seeks to achieve with respect to these grand challenges is **situate** science and technology development in its **social context**. Fundamentally, RRI aims to drive high quality innovations in science and technology that are in the public interest and create a society in which research and innovation practices work towards **ethically acceptable, socially desirable and sustainable outcomes**.
|
||||
|
||||
#### Responsible Innovation
|
||||
|
||||
Now Stilgoe et al. posit 4 ‘dimensions’ or aspects of responsible innovation that together provide **a heuristic framework supporting** **ethical governance** **of research and innovation**. These include anticipation, reflexivity, inclusion and responsiveness.
|
||||
|
||||
**Anticipation**
|
||||
|
||||
Thinking through potential social risks of future visions of technology
|
||||
|
||||
**Anticipation** recognises that the detrimental effects of new technologies are often unforeseen – take, for example, the range of social harms we considered in lecture 1 – and that existing approaches to R&I have commonly failed to provide early warnings of future effects.
|
||||
|
||||
Anticipation seeks to remedy this situation by encouraging organisations and individuals involved in research and innovation to ask “what if” questions about their work and in doing so to consider what is known, what is possible, what is likely, what is plausible, and what contingencies might well impact their visions of science and technology.
|
||||
|
||||
Anticipating and mitigating risk is also a key plank of the proposed EU regulation of AI, which means that anticipation is not a conceptual abstraction but something that many of you will be legally required to actively engage in and document in your professional work.
|
||||
|
||||
**Reflexivity**
|
||||
|
||||
**Reflexivity** requires that researchers and innovators hold a mirror up to themselves and their activities, including the assumptions that underpin our work. It requires that we develop awareness of the limits of our knowledge, and are mindful that the ways in which we think about and frame problems and challenges may not be universally held.
|
||||
|
||||
Reflexivity is particularly important at an institutional or organisational level to ensure that the value systems, theories and practices that shape research and innovation and its governance are subject to scrutiny.
|
||||
|
||||
This is called “second-order reflexivity” and contrasts with “first-order reflexivity”, where individuals reflect on and scrutinise themselves privately. Second-order reflexivity seeks to make reflexivity a public matter and leads to kinds of consideration of ethical governance proposed by Alan Winfield and Marina Jirotka we discussed in lecture 7.
|
||||
|
||||
Reflexivity is key to the development of ethically acceptable and socially desirable innovations. It requires researchers and innovators see beyond organisational boundaries and responsibilities and consider their wider, moral responsibilities.
|
||||
|
||||
**Inclusion**
|
||||
|
||||
**Inclusion** recognises the need to open up anticipatory visions of future social worlds to public dialogue in ways that critically interrogate the social, political and ethical viewpoints implicated in technology development.
|
||||
|
||||
Inclusion requires that we are sensitive to
|
||||
|
||||
- a) the ‘intensity’ of public engagement – i.e., how early members of the public and other stakeholders are consulted in the innovation process
|
||||
- b) ‘openness’ – i.e., how diverse the sample is and who is represented
|
||||
- c) the ‘quality of engagement’, including the gravity or seriousness of public and stakeholder involvement and the continuity of engagement and discussion throughout the research and innovation process.
|
||||
|
||||
**Responsiveness**
|
||||
|
||||
**Responsiveness** recognises that responsible innovation must be able to change shape or direction in response to public and stakeholder viewpoints and values, and to changing circumstances. Responsiveness involves building new knowledge into the research and innovation process as it emerges.
|
||||
|
||||
Stilgoe et al. also place emphasis on the role of governance approaches in R&I, including research funding, intellectual property regimes and technological standards, which may act to close down responsiveness, along with other norms and expectations that reinforce particular dependencies and lock-ins. In short, if RRI is to be effective we also need to be attentive to the environment which enables R&I and the structures that organise it, in order to counter the “logic of unresponsiveness” that underpins findings like Birhane’s.
|
||||
|
||||
### AREA Framework
|
||||
|
||||
https://www.epsrc.ac.uk/research/framework
|
||||
|
||||
The framework is called **AREA** and reflects the 4 dimensions of Stilgoe et als responsible innovation framework, reframed as Anticipate, Engage, Reflect and Act.
|
||||
|
||||
**Anticipate** asks researchers to describe and analyse any economic, social and / or environmental impacts, intended or otherwise, that might arise from the proposed research. The aim is not to predict the actual impact of the proposed research, but to explore potential impacts and implications of the research that may otherwise remain ignored during the research the process.
|
||||
|
||||
**Reflect** asks researchers to reflect on the purposes, motivations, and potential implications of their research, and the associated uncertainties, areas of ignorance, assumptions, framings, questions, dilemmas and social transformations these may occasion.
|
||||
|
||||
**Engage** asks researchers to open up their research visions and their potential impacts to broader deliberation, dialogue, engagement and debate with stakeholders and the public in an inclusive way.
|
||||
|
||||
**Act** asks researchers to using the processes of Anticipation, Reflection and Engagement to influence the direction and trajectory of the research and innovation process itself.
|
||||
|
||||
So RRI is an important part of the EU and UK research and innovation pipeline and will become much more so now that the UK research councils have been brought together under the umbrella of UK Research and Innovation or UKRI.
|
||||
|
||||
## How does RRI work?
|
||||
|
||||
The focus of RRI is not only on achieving ethically acceptable, socially desirable and sustainable outcomes. It also and fundamentally concerned with *how* research and innovation is conducted and the parties involved in the process. RRI can thus be broken down into four key elements: **policy**, **stakeholders**, **outcomes**, **process**.
|
||||
|
||||
###### Policy
|
||||
|
||||
The EU sets out six key policies to shape responsible research and innovation processes, which are target at governments, funding agencies and R&I organisations.
|
||||
|
||||
1. Robust goverence
|
||||
- RRI principles should, as a matter of policy, be **embedded in robust** **governance** frameworks. These frameworks should be flexible and adapt to change so as to be capable of responding to the unpredictable nature of research and innovation.
|
||||
2. Gender equality
|
||||
- It is also a matter of policy that research and innovation take the perspectives of both men and women into account to ensure outcomes are relevant to the whole population.
|
||||
- Decision-making bodies and R&I organisations should have balanced gender representation and strive to ensure **gender equality** in research and innovation.
|
||||
3. Integrity
|
||||
- Honesty, accountability, fairness and good stewardship should be core principles of research and innovation and are key to ensuring the **integrity** of R&I.
|
||||
4. Public and stakeholder engagment
|
||||
- The **public and other stakeholders** should, as a matter of policy, be **engaged in research** and innovation processes as early as possible to avoid tokenism, ensure outcomes align with the values, needs and expectations of society and to avert societal backlash
|
||||
- as, for example, happened with the attempted introduction of GM crops into the UK
|
||||
5. Open Access (FAIR)
|
||||
- publicly funded research should be **open access** in order to catalyse broader innovation, encourage collaboration and improve the quality of research
|
||||
- Scientific results and data should follow the FAIR principle
|
||||
- results and data should be **F**indable, **A**ccessible, **I**nteroperable, and **R**eusable
|
||||
6. Science and technology education
|
||||
- The demand for highly qualified people continues to rise globally and there is also need as a matter of policy for improved **science and technology education** to build the necessary capacity to enable R&I at scale and to provide citizens with the knowledge they need to engage with research and innovation.
|
||||
|
||||
###### Stakeholders
|
||||
|
||||
RRI involves a range of stakeholders, who should in one way or another be involved in permanent and ongoing dialogue with one another. These stakeholders include:
|
||||
|
||||
**Policymakers**, who have the ability to bring stakeholders to the table and foster debate. This not only includes government but funding agencies, the directors R&I organisations and anyone else involved in making decisions that shape research and innovation locally, nationally and internationally.
|
||||
|
||||
The **research community** is obviously a key stakeholder in research and innovation and includes everyone in the research and innovation pipeline from science advocates and communicators, to research managers, researchers, technicians and support staff.
|
||||
|
||||
**Business and industry**, from start ups to SMEs to large corporates and transnational companies, are all key to research and bringing innovations to bear on social life.
|
||||
|
||||
**The education community**, from primary school to university, science centres and museums, and including teachers, students and their families, play a key role in building capacity and promoting public understanding of science and technology.
|
||||
|
||||
**Civil society organisations**, such as trade unions, NGOs and the media, also play important roles in shaping research and innovation.
|
||||
|
||||
RRI seeks to involve these stakeholders in shaping ethically acceptable, socially desirable and sustainable outcomes. Indeed, in recognising that research and innovation reaches beyond the lab, RRI seeks to foster **shared** **responsibility** for research and innovation and ensure that it that serves the public good.
|
||||
|
||||
###### Process
|
||||
|
||||
The emphasis placed on shared responsibility is reflected in the RRI process, which seeks to put the responsible innovation framework into action and is characterised by diversity and inclusion, anticipation and reflection, openness and transparency, and responsiveness and adaptive change.
|
||||
|
||||
**Diversity and inclusion** emphasise that the RRI process should involve a wide range of stakeholders early in research and innovation to produce outcomes that align with the values and expectations of the groups involved in and affected by R&I. Voices across a diversity of communities should be involved from the beginning of R&I through to its commercialisation, as different perspectives and expertise generate higher quality science and ensure all points of view are taken into account.
|
||||
|
||||
**Anticipation and reflection** emphasise that the RRI process should look beyond the immediate impact of research and innovation and reflect on possible unintended consequences that may arise further down the line. Researchers and innovators should explore potential impacts with stakeholders to generate insights that enable the negative consequences to be avoided.
|
||||
|
||||
**Openness and transparency** emphasise that the RRI process should be accountable to stakeholders and requires that they are provided with meaningful information during all stages of the process to empower them, encourage engagement, foster debate, scrutinise research and innovation, and enable them to make informed decisions.
|
||||
|
||||
**Responsiveness and adaptive change** emphasise that the RRI process should take account of societal need and thus respond to the views expressed by the public and other stakeholders. If necessary, the goals of the research or methods should be adapted and changed.
|
||||
|
||||
###### Outcomes
|
||||
|
||||
The RRI process is essentially concerned to deliver the right outcomes, where right means that:
|
||||
|
||||
1. The process has engaged and empowered stakeholders and the public;
|
||||
2. The process has produced outcomes that are ethically acceptable, socially desirable and sustainable;
|
||||
3. Outcomes provide solutions to ‘grand’ societal challenges.
|
||||
|
||||
## Putting RRI into practice
|
||||
|
||||
In order to foster uptake of RRI, the EU funded the RRI Tools project, involving over 25 different institutions across 30 countries. The RRI Toolkit is available online https://rri-tools.eu and is free to access and use. It aims to drive a new culture of research that puts citizens at the centre of innovation.
|
||||
|
||||
Abma Tineke and Jacqueline Broerse’s ‘dialogue model’ of participatory research developed in the healthcare sector. The dialogue model has 5 discrete phases, including exploration, consultation, prioritisation, integration, programming and implementation.
|
||||
|
||||
**Exploration** is the first phase of the dialogue model and aims to identify and make contact with the different stakeholder organisations, groups, and individuals that should be involved in the research.
|
||||
|
||||
**Consultation** does at it suggests and engages stakeholders separately in a dialogue about the research to ensure their voices are heard. Tineke and Broerse emphasize the importance of paying attention to diversity (age, gender, ethnicity, etc.) and being sensitive to asymmetries in power in doing this.
|
||||
|
||||
- They underscore the need to empower stakeholders who are not used to actively participating in research to enable “more equal interaction with professionals” and that researchers should pay particular attention to the issues that matter to specific stakeholders.
|
||||
- Consultation also involves determining appropriate methods of conducting research dialogues with stakeholders, e.g., interviews, focus groups, questionnaires, observations, etc.
|
||||
|
||||
**Prioritisation** as the name suggests is about identifying which research themes that emerge from the consultation process should be take priority.
|
||||
|
||||
- This often an iterative process involving further consultation with stakeholders to ensure the right themes are being prioritised appropriately.
|
||||
- Importantly it involves consideration of what can reasonably be expected to be achieved within the lifetime of project, which means that while a theme may have high priority for stakeholders, it may not be technically achievable in the available timeframes, which may lead to it being de-prioritised.
|
||||
- Prioritisation is a matter of compromise between what stakeholders want and what can be technically delivered.
|
||||
|
||||
**Integration** seeks to combine the prioritised research themes into a coherent research agenda.
|
||||
|
||||
- It involves bringing the different stakeholders together to discuss the research agenda and to agree upon outcomes.
|
||||
- Tineke and Broerse emphasise the need for this dialogue to be fair and for stakeholder groups to be proportionally represented to ensure that no single group dominates agenda setting and that all have an equal say.
|
||||
|
||||
The **programming** phase involves specifying a research plan to enable the research agenda to be implemented.
|
||||
|
||||
- It involves setting a programming committee involving stakeholder representatives to ensure the research addresses the concerns of all stakeholders as it proceeds into implementation.
|
||||
|
||||
And **implementation** obviously involves putting the plan into practice.
|
||||
|
||||
- This may involve identifying parties to carry out the research plan, or parts of it, but it also requires oversight and again involves stakeholder representatives on management committees or advisory boards to ensure the research stays on track.
|
||||
|
||||
#### Participatory methods in computing
|
||||
|
||||
The collective resources approach led to action-based and experience-based design methods that leveraged prototypes as vehicles for participatory research.
|
||||
|
||||
Prototyping was established as an alternative approach to requirements specification in the 1970s, replacing a written document subject to the vagaries of interpretation with a functioning version of a computing system.
|
||||
|
||||
The **problem** with prototyping is that it is by its very nature a technical exercise, all too often preoccupied with demonstrating technical features to stakeholders and having them sign-off on them.
|
||||
|
||||
The challenge that Cooperative Design set out tackle was how to *involve* ordinary people – users and other non-technical stakeholders – in the actual development of prototypes.
|
||||
|
||||
Prototyping is a common feature of many design models today, from the spiral model to agile. The contribution of Cooperative Design is to use it as a vehicle for put stakeholder viewpoints and experience at the centre of the design process, not technical specifications and feature demonstrations, and it provides us with a tried and tested way of doing participatory research in computing.
|
||||
|
||||
### RRI self-reflection tool
|
||||
|
||||
Perhaps the most useful tool in the RRI Toolkit is the self-reflection tool: https://rri-tools.eu/self-reflection-tool
|
||||
|
||||
- It helps you determine whether or not your research is responsible.
|
||||
|
||||
The public engagement section asks you 10 questions about stakeholder involvement.
|
||||
|
||||
1. How do you involve stakeholders and the public in your work?
|
||||
2. What channels do you use to enable stakeholder participation in the R&I process?
|
||||
3. At which stage of the R&I process is it most effective for you to engage stakeholders, and why?
|
||||
4. What does public engagement in the decision-making process mean in your work or organisation?
|
||||
5. What dimensions are usually discussed during your engagement activities?
|
||||
6. How do you tailor R&I processes to include stakeholders with different genders, ethnicities, classes, ages, routines, experience, or levels of power?
|
||||
7. How do you ensure that stakeholders understand and accept their roles and the objectives of their engagement?
|
||||
8. What measures would have a direct impact on your multi-stakeholder engagement activities?
|
||||
9. What effects do your engagement activities have on public participants and on your R&I processes?
|
||||
10. How do you address critical aspects of public engagement activities?
|
||||
@@ -0,0 +1,44 @@
|
||||
# COMP3020 PROFESSIONAL ETHICS IN COMPUTING
|
||||
|
||||
Student ID: `20153544`
|
||||
|
||||
Student Code: `psyjg11`
|
||||
|
||||
## Question 2
|
||||
|
||||
#### a)
|
||||
|
||||
According to code 1.4 from the ACM code of ethics, computing professionals should “be fair and take action not to discriminate”. All humans possess some biases whether they be conscious or not, which is why care should be taken to ensure systems developed do not discriminate against social groups regardless of the social groups of the developers.
|
||||
|
||||
#### b)
|
||||
|
||||
Algorithmic bias is a series of systematic and repeatable errors, that over the course of the systems runtime, produces output that dis-proportionally discriminates against individuals and/or social groups. Selena Silva and Martin Kenny found 9 sources of algorithmic bias in their research paper, all of which capable of discriminating and producing bias
|
||||
|
||||
Bias can be introduced in the development of a machine learning system. Training bias is where data used to train the algorithm may be unrepresentive or prejudiced, this can cause the system to unfairly associate one trait to another even though they have no effect on one another. This can be through the developers own bias by only including data sets representative to their own socitak group or through systemic bias where minority groups are under represented in national and global data sets. Developers can also introduce bias by including or excluding certain attributes. This is called algorithmic focus bias and developers must take variables supplied to the algorithm into careful consideration, evaluating why each variable needs to be included in the system. Similarly bias can arise from the way data is processed, for example this can be from weighting quantitative attributes higher than qualitative ones simply as quantitative data is easier to manipulate, this is called algorithmic processing bias. Non-transparency bias is where companies do not divulge or explain how they came to certain decisions, what their rationale was for different design choices. In the best case this can introduce bias in an unforeseen way as all the developers may come from similar social groups and in the worse case scenario developers can obstruct reviews of the algorithm, allowing discrimination to take place.
|
||||
|
||||
Bias can also arise in the use of computing systems. Transfer context bias is where machine learning systems are used inappropriately. This can happen in job applications where credit checks are required or in justice systems where race needs to be explicitly stated. The assumption job performance correlates to wealth or criminal charges correlates to race is unfair and biased. Therefore the use of computer systems particularly in subjective use cases should be scrutinised to ensure the potential benefits outweigh the increased chance of discriminating or additional steps are taken after the system outputs to mitigate any potential harms. Similarly automation bias is where humans hold the output of a system in high regard and don’t question or apply additional thought. Computer systems used in subjective context such as justice systems should be treated as a second opinion or a statistical model and disregarded readily when an unsuitable result is returned. Consumer bias is where bias is introduced to the system via the training data. Humans are inherently flawed and biased and therefore extra care and additional review steps should be added to check the neutrality of the training data. Likewise feedback loop bias affects systems that learn from user behaviour, which again is prone to being discriminatory. This requires special attention has even when a system has been developed without bias, bias is introduced through the systems use lifetime. Lastly interpretation bias is where humans introduce bias from interpreting results from the algorithm. For example if the algorithm agrees with someones own bias, they might be more likely to give a more extreme verdict however if it opposes their own opinion, the result may be completely disregarded.
|
||||
|
||||
## Question 3
|
||||
|
||||
#### a)
|
||||
|
||||
The territorial scope of GDPR is the European Union and the UK. The EU passed GDPR into law in 2018 and the UK adopted it into the Data Protection Act the same year. Individuals living within the European Union or in the UK are protected by the act.
|
||||
|
||||
The application scope is worldwide, the regulation states “This Regulation applies … whether the processing takes place in the Union or not”, which means any company that stores the data of EU citizens must adhere to GDPR.
|
||||
|
||||
#### b)
|
||||
|
||||
To enable proper data protection by design and default, a number of presets must be implemented.
|
||||
|
||||
Firstly controllers must be transparent about how and why they are collecting and using data, how they use and share personal data and how data subjects can exercise their legal rights over data processing. This includes the right to: access, object, intervene, restrict, rectify, export and erase. This allows for data subjects to have full knowledge and control over their data and on top of this, recital 63 of GDPR states “where possible controller[s] should … provide remote access … with direct access to his or her personal data”. Controllers must also by default declare a valid legal basis for the processing. This ensures transparency as there is full disclosure of how data subjects legal rights are being maintained.
|
||||
|
||||
Controllers must ensure their data processing operations are fair. This principle requires personal data should not be processed in ways that are unjustifiably detrimental, unexpected or misleading to the data subject. Fairness is especially prevalent in dealing with AI systems since these do not operate on predefined instructions written by humans, therefore controllers should be able to demonstrate fairness through the inputs and outputs of the system.
|
||||
|
||||
Controllers must explicitly state what the data collected on data subjects will be used for. These must be specific tasks and cannot be processed in way that doesn’t align with the initial reason given. This is called purpose limitation and prevents controllers from collecting as much data as possible for monetary gain or nefarious purposes. This allows data subjects to only give their data to controllers who’s vision aligns with their own.
|
||||
|
||||
Controllers must practise data minimisation, this is a practice where the controller must review the data being asked and verifying all pieces of data are needed to meet the purposes for which they are being processed. This can also include the degree of identification, if the purpose is statistical this likely does not require any immediate identifying attributes. If continued identification is needed, data should be pseudonoymised to migrate damages caused from a data breach. Similarly data must be deleted once it has fulfilled it’s purpose. GDPR places no time limit on data storage of anonymised data however this can be reversed engineered and this data should be treated analogous to raw personal data.
|
||||
|
||||
Controllers must also ensure data is accurate, and if not it is the controllers duty to rectify or erase mistakes immediately. This is important as data subjects could be relying on this data for employment, housing or other civic needs and not being able to obtain this could cause harm to the data subject and family.
|
||||
|
||||
Lastly controllers must put substantial measures in place to prevent unauthorised access, accidental loss and destruction or damage. Regular reviews should be conducted, testing security and inviting professional hackers to further test how the system stands up to new hacking methods.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 266 KiB |
Reference in new issue
Block a user