New benchmark shows how AI models lack intuitive visual reasoning:…Humanity’s Sixth Sense highlights some shortcomings of current AI systems…AI startups Elorian and Scale AI have built and released Humanity’s Sixth Sense, a benchmark that “measures how a model interprets and reasons through information a visual scene implies but doesn’t explicitly show.” Some examples of the types of task in HSS are a picture of a bookcase and being asked whether it can fit more books, or a photo of horses pulling a plough and a question of whether the chains bear the plough’s drag weight. HSS contains 522 questions that cover 288 images and 234 short video clips (totalling 17.6 hours of video). The questions fall into four categories:
Temporal & causal dynamics: “What happened before the scene, and what happens next.”
Physical & spatial logic: “What fits, what’s in reach, and what someone else can see from where they are standing.”
Abstract & contextual inference: “The patterns and rules a scene implies.”
Source: Import AI — Read the full article on the original site ↗