arXiv cs.AI paper: Yang and Wang find models can calculate a limit yet accept designs beyond it

arXiv cs.AI paper: Yang and Wang find models can calculate a limit yet accept designs beyond it

In their arXiv paper, Shaoliang Yang and Jun Wang report a split in language models’ responses to physically impossible engineering designs. A model may identify a design as impossible when asked only about feasibility, yet return a full answer marked “solved” for the same design. The authors tested 12 models in 4,860 calls, using beam loads and disk speeds set above or below plastic collapse or burst. Three models accepted beams 10 to 30% beyond collapse, even though they calculated the limit precisely in separate questions; many such replies stated the violation. In follow-up tests on new designs, Opus 5 and Gemini 3.8 Flash rejected every impossible design they were given when asked only about feasibility, without false alarms. The authors also found that rewording when to decline or changing answer format improved rejection on the original beams, but not consistently on new designs. Their practical concern is software that trusts a model’s “solved” status and passes an impossible design onward. They recommend testing feasibility judgment separately from rejection in full answers, and reporting the exact prompt. How well would these results carry over to other engineering tasks and prompts?

Publication: 2a399cf0-9881-4d3d-8278-876eac75a79e

attribution — pass: The findings and recommendations are attributed to Shaoliang Yang and Jun Wang in an arXiv paper, matching the listed authors. The title's 'cs.AI' label matches the feed publisher. The paper's primary category is cs.CE, but it is cross-listed in cs.AI, so the label is not misleading.

Authors:Shaoliang Yang, Jun Wang

summary — pass: The summary opens with the central finding: a model can judge a design infeasible yet mark the same design 'solved' in a full answer. It keeps the force of the paper, the per-model results, the mixed effect of prompt changes, and the authors' recommendation. The question about transfer to other tasks follows from the narrow test scope. 'Plastic collapse' and 'burst' could use term-library tooltips, but 'physically impossible' and 'limit' keep the meaning readable.

a model can recognize an impossible design when asked only whether it is feasible, yet accept the same design, reporting "solved", when asked for a full answer

date — pass: The originalPublished date of 2026-10-28 matches the arXiv v1 submission date. The later feed date of Oct 31 is not used.

[Submitted on 28 Oct 2026]

support — pass: Each claim matches the abstract: 12 models, 4,860 calls, three models accepting beams 10 to 30% beyond collapse, and Opus 5 and Gemini 3.8 Flash rejecting all impossible new designs without false alarms. The reading reports the inconsistent gains from rewording and the recommendation. 'Many such replies' closely follows the source's 'many accepted replies' in the same sentence. The limitations accurately name gaps in the abstract, and no benefits are overstated.

Three models accepted beams 10 to 30% beyond collapse despite precise limit calculations in separate questions, and many accepted replies stated the violation.

media — pass: The selected media is the verified direct link to the original arXiv abstract page, which is the most direct access to the source.