TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Import AI 476 brings together three developments: a new benchmark where leading AI systems trail human performance on intuitive visual reasoning, an evaluation of AI systems’ research judgment, and analysis of how AI could accelerate activity in space. The newsletter reports benchmark results and discusses possible consequences; claims about recursive self-improvement and future space power remain questions, not established outcomes.
Import AI issue 476 reports on new efforts to measure whether AI systems can infer what a visual scene implies and make research choices associated with scientific judgment, while also examining how AI might accelerate space development. The benchmark results described in the newsletter show a substantial gap between people and tested AI models on intuitive visual reasoning, while the links between AI research taste and recursive self-improvement, and between AI and future space power, remain open questions.
The issue describes Humanity’s Sixth Sense (HSS), a benchmark created by AI startups Elorian and Scale AI. It tests whether a model can reason about information suggested by a scene but not explicitly stated. Its 522 questions draw on 288 images and 234 short video clips, which together run for 17.6 hours. Tasks include judging whether a bookcase has room for more books or whether chains attached to horses can bear the drag of a plough.
According to the results summarized by Import AI, people scored 93.1 pass@1 overall. The best-performing AI system listed, GPT 6 Astra, scored 53.6. The newsletter also reports scores of 26.8 for GLM-5.3-flash and 25.5 for Kimi K3. The benchmark authors identify missed decisive details and mistaken identification of objects, people or roles as common failure types. These scores describe performance on HSS; they do not establish how systems perform across visual reasoning generally.
The issue also covers TasteVal, a benchmark from P-Zero Research that evaluates eight tasks relevant to frontier AI research, including data curation, model training, preference modeling and robustness to adversarial prompts. Import AI connects research judgment to the possibility of recursive self-improvement, but the supplied report excerpt cuts off before giving TasteVal’s results or enough detail to assess that connection. A separate AI Frontiers article argues that autonomous spacecraft, space-based assembly and manufacturing could make AI a factor in a larger space economy.
What the Benchmarks Can Tell Us
The HSS results give readers a concrete measure of a capability often discussed broadly as visual reasoning. The reported difference between human and model scores suggests that current systems can struggle to infer unstated physical, temporal and social details, even when they can process images and video. For developers and researchers, the benchmark may help identify where models fail, rather than relying only on general claims about visual intelligence.
The newsletter also points to a more nuanced interpretation: models’ performance on these tasks may indicate that language-and-vision systems can handle some reasoning challenges previously expected to require separate, explicitly spatial world models. That is an interpretation, not a result the benchmark alone can settle. TasteVal raises a different question: if AI can reliably select promising research directions, it could contribute more directly to AI development. The source material does not establish that such ability is present at a level that would produce recursive improvement.
The space discussion matters because autonomous systems can operate where human presence and rapid communication are difficult. If space infrastructure expands, AI could support spacecraft operations and industrial work beyond Earth. The article presents possible economic and strategic effects, including advantages for early builders, as scenarios requiring attention—not as developments already underway at the scale described.
Products Worth Considering
LG UltraGear 34GX90SA-W 34-inch WQHD Curved OLED Gaming Monitor
Amazon Kindle Paperwhite Signature Edition, 32GB (2024), Metallic Black
Amazon Kindle Scribe (64GB) with Premium Pen, Metallic Jade
As an affiliate, we earn on qualifying purchases.
Three Threads in Issue 476
Import AI is a research-focused newsletter, and issue 476 brings together a benchmark report, an evaluation of research decision-making and an essay about space. The HSS benchmark focuses on inferences that depend on details outside a question’s literal wording: what may have happened before a clip, what is physically possible, or what a person can see from a particular position. Its authors divide questions into temporal and causal dynamics, physical and spatial logic, social understanding, and abstract or contextual inference.
Its reported pass@1 figures are the proportion of questions answered correctly on the first attempt, as presented in the newsletter. Human performance was not perfect, but was substantially higher than the leading system score reported. The scores for GLM-5.3-flash and Kimi K3 are also specific to this benchmark; the newsletter’s characterization of them as Chinese open-weight models should not be read as a broad comparison of all models from China or all open-weight systems.
The other two threads extend beyond immediate benchmark performance. TasteVal aims to assess choices across parts of the AI research process. The space analysis, published by AI Frontiers and summarized in Import AI, considers how autonomy and industrial capacity could change the role of AI in orbit and farther out. Its longer-range possibilities include compounding infrastructure advantages and expanded strategic control.
“Measures how a model interprets and reasons through information a visual scene implies but doesn’t explicitly show.”
— Elorian and Scale AI, describing Humanity’s Sixth Sense
Limits of the Reported Evidence
The source excerpt does not include the full TasteVal discussion: it ends while defining research taste and provides no scores, model comparisons or detailed findings. The strength of any link between research taste and recursive self-improvement therefore cannot be assessed from the material provided.
For HSS, the excerpt gives selected scores and error categories but does not provide enough information to judge how performance varies by model version, prompting method or benchmark design. It also does not establish whether the results generalize to everyday visual tasks. The source names GPT 6 Astra, GLM-5.3-flash and Kimi K3, but readers should treat those results as the reported evaluation, not as a complete ranking of current models.
The space article discusses potential future capabilities and strategic consequences. The timing, scale and likelihood of AI-led space industrialization are not established in the newsletter excerpt. It is also unclear what technical, economic or policy barriers would shape those scenarios.
Further Results and Space Decisions
The next evidence to watch for is the full TasteVal report, including its results, evaluation methods and comparisons across systems. For HSS, additional testing across models and versions, alongside fuller methodological details, would help show whether the reported gaps persist and what kinds of scene inference remain difficult.
On space, the immediate next steps are not specified in the source material. The issue points readers to the AI Frontiers analysis for its broader argument. Whether autonomous spacecraft and space-based manufacturing expand as described will depend on future technical and economic developments, as well as decisions by governments and other organizations. Those possibilities remain forecasts, not confirmed outcomes.
Key Questions
What is Humanity’s Sixth Sense?
Humanity’s Sixth Sense is a benchmark from Elorian and Scale AI that tests whether models can infer information implied by images and video, including physical, temporal and social details.
How did AI systems score against people on HSS?
Import AI reports a human aggregate pass@1 score of 93.1 and a score of 53.6 for the best-performing system listed, GPT 6 Astra. The figures apply to this benchmark and should not be treated as general measures of all visual reasoning.
What does TasteVal evaluate?
TasteVal, from P-Zero Research, evaluates eight tasks associated with AI research, including pretraining data curation, model training and fine-tuning, preference modeling and robustness to adversarial prompts. The supplied excerpt does not include its results.
Does the report show that AI can improve itself recursively?
No. The newsletter connects research taste with the possibility of recursive self-improvement, but the supplied material does not provide evidence that AI systems can independently drive that process.
Is AI already building a dominant presence in space?
The space discussion is a forward-looking analysis of how autonomy and future space infrastructure could affect economic and strategic power. The source does not report that AI has already established such a presence.
Source: rss
