Last year, a group of interdisciplinary researchers were asked to “break” GPT-4, a language model developed by OpenAI, by getting it to output harmful or biased content. This process, known as AI red teaming, is critical to understanding how AI systems can fail and preventing unintended consequences. However, red teaming alone may not be enough to address the potential harms of AI systems.
To address this, the author proposes the concept of “violet teaming,” which involves identifying the ways in which AI systems can harm public goods and institutions, and then developing tools to defend against those harms using the same system. This approach, similar to judo, aims to redirect the power of AI systems in a positive direction to benefit society.
Red teaming and violet teaming should be part of a larger framework for responsible AI development. This includes involving external experts and public reports, as well as stressing the importance of a cross-company community of red-teamer pairs with a deep understanding of these issues. The AI industry should invest in such efforts to reduce the likelihood of critical blind spots and systemic impacts of AI systems.
