arXiv cs.AI: BEiTScore++: Reference-free Image Captioning Evaluation with BEiT-3 and Conformalized Quantile Regression

arXiv cs.AI: BEiTScore++: Reference-free Image Captioning Evaluation with BEiT-3 and Conformalized Quantile Regression

Authors Gonçalo Gomes, Bruno Martins and Chrysoula Zerva propose BEiTScore++, an uncertainty-aware extension of the BEiTScore image captioning metric. It produces conformally calibrated confidence intervals alongside alignment scores, and models predictive distributions over human judgments. The authors say the method scores captions while estimating uncertainty in a single forward pass. Across multiple benchmarks with human-annotated ratings, the authors report that BEiTScore++ preserves BEiTScore’s ranking correlation. They also report that its interval width tracks prediction error more closely than Monte Carlo dropout on most benchmarks. The abstract does not claim this advantage holds on every benchmark. The authors report that estimated uncertainty grows with image degradation and on out-of-domain artwork images. This makes the method a proposed way to signal when image-caption alignment scores may be less certain, rather than providing only a score. The abstract describes benchmark results, not evidence of improved decisions in deployed evaluation settings.

Publication: bff27c8f-32f5-4cab-8a79-ca67f5414a7e

attribution — pass: The authors Gonçalo Gomes, Bruno Martins and Chrysoula Zerva are named correctly. Each result is framed as something the authors propose or report, not as an established fact. The publisher label arXiv cs.AI matches the feed source.

Authors:Gonçalo Gomes, Bruno Martins, Chrysoula Zerva

summary — uncertain: The reading is faithful, but its central terms are left unexplained for a general reader. It never says plainly that BEiTScore is an automated score for how well a caption matches an image. It also leaves 'reference-free' (in the title), 'conformally calibrated confidence intervals', 'Monte Carlo dropout', 'single forward pass' and 'ranking correlation' unexplained. Without these, a reader cannot tell what improved or why it matters. Smallest repair: add one plain sentence saying what the metric does. Then gloss Monte Carlo dropout as the baseline uncertainty method, via tooltips or inline wording.

we propose an uncertainty-aware extension of the BEiTScore image captioning metric

date — pass: The originalPublished date, 2026-11-06, matches the v1 submission date on the arXiv page. The 2026-11-09 feed date is later and is correctly not used.

[Submitted on 6 Nov 2026]

support — pass: All three claims match the abstract word for word, including the 'most benchmarks' qualifier. The reading's interpretation stays within the authors' 'practical tool' framing, and it explicitly notes there is no evidence from deployed use. The question follows from the 'most benchmarks' wording and does not assert an answer.

making BEiTScore++ a practical tool for uncertainty-aware evaluation of image-caption alignment

media — pass: The selected media is the verified link to the arXiv abstract page. It is the direct original source.