The title of this post is the title of this notable new paper available via SSRN authored by Brandon Garrett and Cynthia Rudin. Here is its abstract:
As we embrace data-driven technologies across a wide range of human activities, policymakers and researchers increasingly sound alarms regarding the dangers posed by “black box” uses of artificial intelligence (AI) to society, democracy, and individual rights. Such models are either too complex for people to understand or they are designed so that their functioning is inaccessible. This lack of transparency can have harmful consequences for the people affected. One central area of concern has been the criminal justice system, in which life, liberty, and public safety can be at stake. Judges have struggled with government claims that AI, such as that used in DNA mixture interpretation, risk assessments, facial recognition, and predictive policing, should remain a black box that is not disclosed to the defense and in court. Both the champions and critics of AI have argued we face a central trade-off: black box AI sacrifices interpretability for predictive accuracy.
We write to counter this black box myth. We describe a body of computer science research showing “glass box” AI that is interpretable can be more accurate. Indeed, criminal justice data is notoriously error prone, and unless AI is interpretable, those errors can have grave hidden consequences. Our intervention has implications for constitutional criminal procedure rights. Judges have been reluctant to impair perceived effectiveness of black box AI by insisting on the disclosures defendants should be constitutionally entitled to receive. Given the criminal procedure rights and public safety interests at stake, it is especially important that people can understand AI. More fundamentally, we argue that there is no necessary tradeoff between the benefits of AI and the vindication of constitutional rights. Indeed, glass box AI can better accomplish both fairness and public safety goals.