It’s perfect software for criminal sentencing, except for a few bugs where it metes out harsher punishments when a local sports team loses, changes its leniency based on time of day and temperature outside, is kinder to people when they get sentenced on their birthday, and also has a lot of random noise thrown in so that different computers will vary in their judgments over several orders of magnitude.

Plus racism.

If humans were algorithms, they would be banned from use in courtrooms.

It can also acquire brand new forms of bigotry by using the internet as training data!

All examples from the book Noise, by Kahneman, Sibony, and Sunstein, which I'm enjoying right now.