Professor Arvind Narayanan of Princeton published a brief deck on AI snake oil. Helpfully, he divides applications into three (non-exclusive) domains:
- Perception (e.g., face recognition, speech to text): “genuine, rapid progress”
- Automating judgment (e.g., spam detection, essay grading) “imperfect but improving”; and
- Predicting social outcomes (e.g., recidivism, job success): “fundamentally dubious”
His point is that humans are terrible at predicting social outcomes, and AI’s are no better. And in fact manually sorting the data using just a few features is often the best we know how to do.
So AI’s predicting job success = snake oil.