Content Warning

Dear computer vision researchers, students and practitioners 🔇 🔇🔇

Remi Denton and I have written what I consider to be a pretty comprehensive paper on the harms of computer vision systems reported to date, from different angles, and how people have proposed addressing them, from many angles.

As we say in the introduction, this isn’t a paper from the lens of fairness in machine learning, although we touch on those works a little bit.

▶️ Link for PDF: https://cdn.sanity.io/files/wc2kmxvk/revamp/79776912203edccc44f84d26abed846b9b23cb06.pdf

Screenshot of the table of contents, part 1

Contents
1 Introduction 217
2 Positionality 221
3 Overview of Risks and Harms Associated with Computer
Vision Systems and Proposed Mitigation Strategies 223
3.1 Representational Harms . . . . . . . . . . . . . . . . . . . 223
3.2 Quality-of-Service and Allocative Harms . . . . . . . . . . 229
3.3 Interpersonal Harms . . . . . . . . . . . . . . . . . . . . . 237
3.4 Societal Harms: System Destabilization and Exacerbating
Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 245
4 Frameworks and Principles for Computer Vision
Researchers 266
4.1 Guidelines for Responsible Data and Model Development . 267
4.2 Measurement Modeling . . . . . . . . . . . . . . . . . . . 271
4.3 Reflexivity . . . . . . . . . . . . . . . . . . . . . . . . . . 273
5 Reorientations of Computer Vision Research 276
5.1 Grounded in Historical Context and Considering
Power Dynamics . . . . . . . . . . . . . . . . . . . . . . . 276
5.2 Small, Task Specific . . . . . . . . . . . . . . . . . . . . . 279
5.3 Community-Rooted . . . . . . . . . . . . . . . . . . . . . 280
Screenshot of the table of contents, part 1 Contents 1 Introduction 217 2 Positionality 221 3 Overview of Risks and Harms Associated with Computer Vision Systems and Proposed Mitigation Strategies 223 3.1 Representational Harms . . . . . . . . . . . . . . . . . . . 223 3.2 Quality-of-Service and Allocative Harms . . . . . . . . . . 229 3.3 Interpersonal Harms . . . . . . . . . . . . . . . . . . . . . 237 3.4 Societal Harms: System Destabilization and Exacerbating Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 245 4 Frameworks and Principles for Computer Vision Researchers 266 4.1 Guidelines for Responsible Data and Model Development . 267 4.2 Measurement Modeling . . . . . . . . . . . . . . . . . . . 271 4.3 Reflexivity . . . . . . . . . . . . . . . . . . . . . . . . . . 273 5 Reorientations of Computer Vision Research 276 5.1 Grounded in Historical Context and Considering Power Dynamics . . . . . . . . . . . . . . . . . . . . . . . 276 5.2 Small, Task Specific . . . . . . . . . . . . . . . . . . . . . 279 5.3 Community-Rooted . . . . . . . . . . . . . . . . . . . . . 280
Screenshot of the table of contents, part 2. 

6 Systemic Change 285
6.1 Collective Action and Whistleblowing . . . . . . . . . . . . 285
6.2 Refusal/The Right not to Build Something . . . . . . . . . 287
6.3 Independent Funding Outside of Military and Multinational
Corporations . . . . . . . . . . . . . . . . . . . . . . . . . 289
7 Conclusion 291
References 293
Screenshot of the table of contents, part 2. 6 Systemic Change 285 6.1 Collective Action and Whistleblowing . . . . . . . . . . . . 285 6.2 Refusal/The Right not to Build Something . . . . . . . . . 287 6.3 Independent Funding Outside of Military and Multinational Corporations . . . . . . . . . . . . . . . . . . . . . . . . . 289 7 Conclusion 291 References 293

Content Warning

@timnitGebru Les récentes révélations sur l'utilisation de l'IA par l'armée israélienne, notamment avec le système "Lavender", soulèvent des inquiétudes majeures. Ce programme, qui identifie des cibles, entraîne des pertes civiles tragiques, avec un taux d'erreur de 10 %. La rapidité de décision au détriment de l'humanité est alarmante. Nous devons dénoncer cette automatisation de la violence. #IA#Palestine#DroitsHumains 🕊️💔

Content Warning

@timnitGebru Les récentes révélations sur l'utilisation de l'IA par l'armée israélienne, notamment avec le système "Lavender", soulèvent des inquiétudes majeures. Ce programme, qui identifie des cibles, entraîne des pertes civiles tragiques, avec un taux d'erreur de 10 %. La rapidité de décision au détriment de l'humanité est alarmante. Nous devons dénoncer cette automatisation de la violence. #IA#Palestine#DroitsHumains 🕊️💔
+ -

Content Warning

We discuss the many ways in which people, ranging from journalists to civil society to all sorts of disciplines outside of computer vision, have uncovered various harms that arise from it, and proposed many mitigation strategies. We cover everything from research from those within computer vision to the labor movement.

Believe it or not, I signed up to do this either when I was still at Google or right after I was fired. Started working on it with Remi who was in our team at the time.

+ -

Content Warning

We got it done like 3 years later, after writing a paper which I feel like is the length of a short book at this point.

I hope you find it helpful in your undergraduate/graduate student teachings. The paper is geared towards people within the computer vision discipline, but hopefully people who aren’t in the discipline can find it useful too.

+ -