Frontier models can’t see, and if you think they can, you’ve probably been fooled by benchmarks that can totally be gamed. In the very short essay linked below I discuss a stunning new finding from Stanford that shows just how serious the problem is.
And why this means a lot of jobs are safe for a long while. open.substack.com/pub/garymarcus…
23 Replies
24 Retweets
115 Likes
9,037 Views 
One Sentence Summary
Gary Marcus argues that frontier models lack genuine visual understanding and that current benchmarks are easily gamed.
Summary
Gary Marcus challenges the prevailing narrative that frontier models possess true visual understanding. Citing a new study from Stanford, he argues that performance on current benchmarks is often misleading or gamed. He posits that these inherent limitations in visual reasoning suggest that the threat of AI-driven job displacement may be overstated in the near term.
AI Score
80
Influence Score 41
Published At Today
Language
English
Tags
Computer Vision
AI Benchmarks
Frontier Models
AI Limitations
Gary Marcus