Three AI models got 100 questions. After about 30, they barely improved.
Text
In a research guessing game, weaker models could select useful questions supplied by other models but struggled to invent them. More turns alone did not solve the problem.
An AI had 100 chances to ask its way toward a hidden research idea. Three tested models made almost no progress after roughly question 30. Yet those same systems improved when other models supplied questions for them to choose from. In Re-Search, released November 10, researchers at Texas A&M and Johns Hopkins isolated a gap between recognizing a useful question and coming up with one.
The team built a scientific guessing game from 200 ICML 2026 papers. Six models received a paper’s research problem, but not its solution, and questioned another model that could read the full paper. The answers were yes, no or unknown. Every 10 questions, the questioning model recorded its best explanation. GPT-OSS-120B, DeepSeek-V4-Flash and Qwen3.6 largely stalled late in the game. Gemma and Claude Sonnet improved slowly, while GPT-5.4 kept making substantial progress.
A separate experiment gave each questioner a pool of four questions, one from each participating model. After 30 rounds, GPT-OSS-120B scored 5.20 out of 10 for recovering the paper’s idea, compared with 4.16 when working alone. Generating four candidates itself reached only 4.57, while choosing randomly from the shared pool produced 4.43. Both the supply of informative questions and the ability to select among them mattered.
These are controlled preprint results, not a measure of independent scientific discovery. Three AI judges scored the ideas; their choices agreed with human majority judgments in 91.2% of 498 paired comparisons. The researchers excluded newer models to reduce the risk that they already knew the papers. A shared pool also requires multiple models and actually lowered GPT-5.4’s score, so it is not a universal, cost-free improvement. The practical suggestion is narrower: when an AI researcher gets stuck, bringing in different candidate questions may help more than repeatedly asking the same system to keep thinking.
Source time: Muhan Gao and colleagues / arXiv, November 11, 2026, 02:47:22 Beijing time / November 10, 2026, 1:47:22 PM EST.
Share this article
What do you think of this story?
Anonymous feedback, up to 5 reactions per article.
Anonymous comments
Share your thoughts without signing in.
Discuss the technology and the story. Hate, politics and war, threats, abuse, personal accusations, sexual content, ads, links and personal information are not allowed.
No comments yet. Be the first to share.