As part of our big data work, we’ve been thinking about the opacity of the algorithms we use. The pretty picture and tables that result from the research are persuasive, but if people wanted to know how they were derived, it would not be easy to explain.
But then, we have to always think about the alternative. A method may be familiar and may have been used for decades if not centuries. But that does not necessarily make it fair.
It’s not necessarily a bad idea to use risk assessment systems like COMPAS. In many cases, ADM systems can increase fairness. Human decision making is at times so incoherent that it needs oversight to bring it in line with our standards of justice. As one specifically unsettling study showed, parole boards were more likely to free convicts if the judges had just had a meal break. This probably had never occurred to the judges. An ADM system could discover such inconsistencies and improve the process.
The above quote is from a thoughtful piece on bias in algorithms in Technology Review.
Comments are closed.