Charin Polpanumas presents Vendster, an arena for testing forecasts in LLM-agent negotiations

Charin Polpanumas presents Vendster, an arena for testing forecasts in LLM-agent negotiations

In an arXiv paper, Charin Polpanumas argues that forecast accuracy alone may not show whether a forecast helps an agent make a better decision. The paper introduces Vendster, an evaluation arena where two LLM agents negotiate a buyer-vendor discount, and tests how forecasts affect the result. In Vendster-M5, model-generated forecasts raised sales target attainment by up to 17 percentage points, while standard forecast metrics did not predict negotiation outcomes at a statistically significant level. In tests with synthetic forecasts, under-prediction improved outcomes for agents that reacted more to forecasts. The authors report no statistically significant effect from interval width when bias was zero. The paper’s central point is that a forecast’s value depends not only on its accuracy but also on its direction of bias and how strongly an agent responds to it. This matters when forecasts are tools used inside AI-driven decisions: a score for the forecast may not tell users what outcomes it will shape. The results are from a specific negotiation arena, not evidence that under-prediction benefits other decisions. How well do these patterns hold beyond Vendster-M5 and its tested agents?

Publication: 139f2dc3-a40a-4904-8a2e-63fa10f59648

attribution — pass: The paper is correctly attributed to its sole author, Charin Polpanumas, on arXiv. One small wording slip: the summary says 'The authors report' although the page lists one author. The smallest repair is 'The paper reports' or 'Polpanumas reports'. This does not misattribute the claim.

Authors:Charin Polpanumas

summary — pass: The reading faithfully states the thesis and the main results: the +17-point gain in Vendster-M5, the non-predictive metrics, the under-prediction effect for reactive agents and the null interval-width result. It keeps the claim scoped to the arena. 'Depends on' is slightly stronger than the source's 'more strongly associated', but it is defensible given the synthetic causal test. Prediction-interval width and under-prediction should get term-library tooltips. The question follows from the evidence.

These findings indicate that the direction of forecast bias and agent forecast-reactivity are more strongly associated with downstream outcomes than forecast accuracy.

date — pass: originalPublished 2026-10-28 matches the v1 submission date. The feed date of 2026-10-30 is later and correctly not used.

[Submitted on 28 Oct 2026]

support — uncertain: The claims match the abstract, but the provenance is incomplete. Claim 2's excerpt is cut off at '(p' and does not cover the second half of the claim, which says standard metrics do not predict outcomes. The repair is to extend the excerpt to include the relWQL/MASE sentence. The limitation saying no valid excerpt can be provided is false, because the text is not truncated; it should be removed. The significance line about forecast scores not revealing downstream outcomes is supported and appropriately hedged.

yet standard forecast evaluation metrics such as relWQL and MASE do not predict negotiation outcomes at a statistically significant level

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.