AI snake oil

Professor Arvind Narayanan of Princeton published a brief deck on AI snake oil. Helpfully, he divides applications into three (non-exclusive) domains:

  1. Perception (e.g., face recognition, speech to text): “genuine, rapid progress”
  2. Automating judgment (e.g., spam detection, essay grading) “imperfect but improving”; and
  3. Predicting social outcomes (e.g., recidivism, job success): “fundamentally dubious”

His point is that humans are terrible at predicting social outcomes, and AI’s are no better. And in fact manually sorting the data using just a few features is often the best we know how to do.

So AI’s predicting job success = snake oil.