The AI Bias Debate: A Discussion of Discussions
Uncovering the double standard in tech: why AI bias is often downplayed as a 'feature' rather than a flaw, and what this says about our perception of machine learning and artificial intelligence.
Over the past couple of years, stories about machine learning (ML) and artificial intelligence (AI) bias have gone viral, highlighting issues with large language models (LLMs) and generative AI. One interesting aspect of these discussions is how people react differently to AI bias compared to "classical" software bugs. When laypeople discuss AI bias, they often deny that a model producing output opposite to the user's request is a bug. For instance, when an Asian MIT graduate asked Playground AI (PAI) to generate a professional LinkedIn profile photo, the model converted her face to a white face with blue eyes.
On Reddit, a top comment argued that there's no bias because Asian women are overrepresented in images generated by Stable Diffusion models. The commenter suggested that this overrepresentation cancels out the bias in the PAI example. However, this argument is flawed, as it ignores the context of the prompt and the model's decision to change the user's ethnicity. Other comments echoed this sentiment, with some arguing that the model is simply reflecting the average LinkedIn profile photo, which is often white.
These responses are typical of the "AI isn't biased" reflex that many people exhibit when discussing AI bias. They often argue that the model is simply reflecting the data it was trained on, without considering the potential consequences of this reflection. This reaction is different from how people respond to classical software bugs, where they typically acknowledge the issue and expect it to be fixed.
To test the PAI model, I tried the same prompt with my own profile photo, substituting "man" for "girl." The model usually converted my Asian face to a white face, sometimes making me appear ethnically ambiguous. When I used a darker-skinned summer photo, the model often turned my face into a South Asian or African face. These results suggest that the model has preconceptions about ethnicity and profession that override the input photo.
Other users have reported similar results, with the model making assumptions about a person's ethnicity based on their profession. For example, when asked to generate a LinkedIn profile picture for a computer science professor, the model produced a white person in almost every case. When asked to generate a picture for a Chinese history professor, the model produced a stereotypical Chinese person. These results demonstrate that the model is perpetuating biases and stereotypes, rather than simply reflecting the data it was trained on.
The CEO of Playground AI responded to these concerns by asking rhetorical questions, such as whether rolling a dice once and getting a 1 means the dice is biased towards 1. However, this response ignores the fact that the model is making systematic errors that affect certain groups of people more than others.
When I searched for examples of autocorrect bugs on Reddit, I found that most users acknowledged that incorrect autocorrects were bugs. In contrast, discussions of AI bias often feature users denying that the model's output is a bug, even when it's clear that the model has made a mistake.
The issue of AI bias is not new, and it's not unique to ML models. However, the widespread use of ML has made these biases more legible to laypeople, making them more likely to make the news. The problem of bias in automation predates ML, and it's not limited to ML models. For example, compression algorithms like Brotli are biased towards the English language, which can make it harder for non-English speakers to compress their data.
The solution to AI bias is not simply to hire more diverse teams or to try harder to care about the issue. While diversity is important, it's not a panacea for bias. The problem of bias is complex and multifaceted, and it requires a more nuanced approach. One possible solution is to use more efficient testing techniques, such as fuzzing and randomized testing, to identify and fix biases in ML models. However, this approach requires a significant investment of time and resources, and it's not clear whether it will be effective in the long run.
Ultimately, the issue of AI bias is a symptom of a larger problem: the prioritization of velocity over quality in software development. As long as companies prioritize shipping features quickly over ensuring that their models are fair and unbiased, we can expect to see more examples of AI bias in the future. To address this issue, we need to rethink our approach to software development and prioritize quality and fairness over velocity and feature creep.
In the appendix, I've included comments from other experts in the field, including Yossi Kreinin, who argues that AI bias is not a result of clever scheming, but rather a result of the software maker's lack of effort to fix bugs. I've also included comments from an anonymous founder of an AI startup, who notes that ML code is often riddled with old-fashioned software bugs. Finally, I've included a reproduction of Rob Ricci's results, which demonstrate the bias in generative AI models.
The story of this post is also included in the appendix, where I explain how I wrote a draft of this post when the Playground AI story went viral, and then sat on it for a year to see if it seemed to hold up when the story was no longer breaking news. I've also included a prediction about the odds that this post will still be relevant a decade later, in 2033.