arXiv: In-Context Black-Box Optimization with Unreliable Feedback

arXiv: In-Context Black-Box Optimization with Unreliable Feedback

Nicolas Samuel Blumer, Julien Martinelli and Samuel Kaski study how an optimizer can use advice that may be useful, biased or misleading while searching for a good candidate. Their feedback-informed in-context black-box optimization (FICBO) method gives a pretrained optimizer both its past observations and auxiliary feedback: inexpensive signals from sources such as experts, simulators, pretrained predictors or heuristics. The authors train a feedback-aware transformer with structured priors covering different relationships between auxiliary feedback and the objective. At test time, it compares observed objective values with those signals to estimate how reliable each source is, then uses that estimate to select queries. The intended advantage is to benefit from informative advice without depending on advice that is weak or misleading. The paper reports experiments on synthetic and real-world tasks and says the method exploits informative feedback while remaining robust to weak or misleading sources. This addresses a tension in optimization: extra guidance can make a search more efficient, but treating that guidance as trustworthy can misdirect it. The authors also investigate how the model perceives sources at test time, as a way to examine its interpretability and decision-making. The arXiv abstract does not give performance measures or details of those investigations, so a

Publication: d14f07fd-e3f4-479f-854b-8e0795d8ef17

attribution — pass: The reading correctly credits Nicolas Samuel Blumer, Julien Martinelli and Samuel Kaski. It frames FICBO, the priors, the reliability estimation and the results as the authors' own work and reported findings.

Authors:Nicolas Samuel Blumer, Julien Martinelli, Samuel Kaski

summary — uncertain: The summary is cut off mid-sentence: its final paragraph ends with 'so a'. The closing statement about what remains unknown is therefore unreadable. Smallest repair: complete or remove that sentence. Separately, the central term 'black-box optimization' and the words 'objective' and 'query' are never explained in plain language, so a non-specialist may not grasp what is being searched for. A brief gloss would help, such as: searching for the best design by testing candidates without knowing the underlying formula. Otherwise the title and question are faithful to the abstract.

Black-box optimization in science and engineering often comes with side information

date — pass: originalPublished is 2026-05-07, which matches the v1 submission. The 4 Nov 2026 revision and the November feed date are correctly not used.

[Submitted on 7 May 2026 (v1), last revised 4 Nov 2026 (this version, v2)]

support — pass: Each claim matches the abstract and stays attributed to the authors. The tension between faster search and misdirection reflects the abstract's own framing. The reading notes that the abstract gives no performance measures and does not convert reported robustness into demonstrated real-world reliability. The question follows naturally from the evidence and does not assume an answer.

Such feedback can accelerate search, but can also be biased, input-dependent, or misleading.

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.