# Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding [Canonical reading](https://escapemeridian.com/readings/does-explanation-correctness-matter-linking-computational-xai-60c716c2) The paper’s authors, Gregor Baer, Chao Zhang, Isel Grau and Pieter Van Gorp, test whether more functionally correct explanations help people understand an AI system. In a user study, 200 participants used explanations of a simulated AI’s decisions in a synthetic time series classification task. The authors varied explanation correctness across four levels and measured participants’ ability to predict the AI’s decisions, using that accuracy as a proxy for understanding. Accuracy was lower with explanations at 70% and 55% correctness than with fully correct explanations; the authors found no conclusive difference between 85% and 100%, or further reduction below 70%. Lower correctness meant fewer participants predicted accurately, rather than uniformly lowering everyone’s accuracy. Even with fully correct explanations, only a subset predicted well above chance. The authors also report exploratory findings: some participants predicted decisions accurately but described the wrong pattern the AI used, and self-reported ratings correlated with prediction accuracy only when explanations were fully correct. Their argument is that computational measures of explanation correctness do not reliably map onto human understanding at every level, so those measures need validation against human outcomes. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Mar 26, 2026 (2026-03-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T01:48:50.128Z. First observed: 2026-11-05T05:06:29.996Z. Retrieved: 2026-11-09T01:48:09.108Z. - [Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding](https://arxiv.org/abs/2603.25251) ## Limits The study used a synthetic time series classification task and a simulated AI; the reported results are specific to that study setup. The authors define understanding through participants’ forward simulation accuracy, which they use as a proxy. The explanations were constructed to resemble feature attributions, and correctness was measured against known ground truth rather than estimated from a trained model. The excerpt reports exploratory analyses but does not provide further detail on their methods or results. ## Open question Would the pattern hold when people use explanations for tasks beyond this synthetic time series classification task? ## Source-stated The authors tested how explanation correctness relates to participants’ ability to predict a simulated AI’s decisions in a synthetic time series classification task. > We conducted a user study (N=200) that manipulated explanation correctness at four levels (100%, 85%, 70%, 55%) in a synthetic time series classification task - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated Participants’ prediction accuracy was used as a proxy for understanding; it fell at 70% and 55% correctness compared with 100%, with no conclusive difference between 85% and 100%. > accuracy dropped at 70% and 55% relative to 100%, with no further reduction below 70% and no conclusive difference between 85% and 100%. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated Lower correctness reduced the number of participants who predicted accurately, and only a subset predicted well above chance even with fully correct explanations. > Lower correctness reduced how many participants predicted accurately rather than lowering accuracy uniformly, and even with fully correct explanations, only a subset of participants predicted well above chance. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated In exploratory analyses, some participants predicted decisions accurately but described the wrong pattern, and self-reported ratings correlated with prediction accuracy only when explanations were fully correct. > some participants predicted decisions accurately but described the wrong pattern when asked what the AI used, and self-reported ratings correlated with forward simulation accuracy only when explanations were fully correct. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated The authors argue that functional correctness differences do not always translate into differences in human understanding, and call for validation against human outcomes. > not all differences in functional correctness translate to differences in human understanding, highlighting the need to validate functional metrics against human outcomes. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source and contributor context Baer, Gregor, Zhang, Chao, Grau, Isel, Van Gorp, Pieter · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Time-series Observations arranged over time, such as monthly sales; forecasting uses their patterns to estimate later values. - [Definition reference](https://developers.google.com/machine-learning/glossary#time-series-analysis) ## xAI xAI develops the Grok family of AI models and publishes its own release claims and evaluations. - [Explore this context](/entities/xai) - [Reference](https://x.ai/) - [Reference](https://x.ai/news/grok-4-7) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 57e70792-3486-4cba-8fe5-40462e22aa64. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T01:48:50.128Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: d1c2bcb77f402e1037780365d709edfe3639aeec655edebeb82ccfde05443630. Source text hash: 8153a82c381c42f1f562766810c24e35c1387808752b5921c1433d09b5499151. ## Revision and source-check history ### Version 1 — 2026-11-09T01:48:50.128Z The paper’s authors, Gregor Baer, Chao Zhang, Isel Grau and Pieter Van Gorp, test whether more functionally correct explanations help people understand an AI system. In a user study, 200 participants used explanations of a simulated AI’s decisions in a synthetic time series classification task. The authors varied explanation correctness across four levels and measured participants’ ability to predict the AI’s decisions, using that accuracy as a proxy for understanding. Accuracy was lower with explanations at 70% and 55% correctness than with fully correct explanations; the authors found no conclusive difference between 85% and 100%, or further reduction below 70%. Lower correctness meant fewer participants predicted accurately, rather than uniformly lowering everyone’s accuracy. Even with fully correct explanations, only a subset predicted well above chance. The authors also report exploratory findings: some participants predicted decisions accurately but described the wrong pattern the AI used, and self-reported ratings correlated with prediction accuracy only when explanations were fully correct. Their argument is that computational measures of explanation correctness do not reliably map onto human understanding at every level, so those measures need validation against human outcomes. ## Original source arXiv cs.AI · Article Original source date: Mar 26, 2026 (2026-03-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T01:48:50.128Z. First observed: 2026-11-05T05:06:29.996Z. Retrieved: 2026-11-09T01:48:09.108Z. - [Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding](https://arxiv.org/abs/2603.25251) ## Limits The study used a synthetic time series classification task and a simulated AI; the reported results are specific to that study setup. The authors define understanding through participants’ forward simulation accuracy, which they use as a proxy. The explanations were constructed to resemble feature attributions, and correctness was measured against known ground truth rather than estimated from a trained model. The excerpt reports exploratory analyses but does not provide further detail on their methods or results. ## Open question Would the pattern hold when people use explanations for tasks beyond this synthetic time series classification task? ## Source-stated The authors tested how explanation correctness relates to participants’ ability to predict a simulated AI’s decisions in a synthetic time series classification task. > We conducted a user study (N=200) that manipulated explanation correctness at four levels (100%, 85%, 70%, 55%) in a synthetic time series classification task - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated Participants’ prediction accuracy was used as a proxy for understanding; it fell at 70% and 55% correctness compared with 100%, with no conclusive difference between 85% and 100%. > accuracy dropped at 70% and 55% relative to 100%, with no further reduction below 70% and no conclusive difference between 85% and 100%. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated Lower correctness reduced the number of participants who predicted accurately, and only a subset predicted well above chance even with fully correct explanations. > Lower correctness reduced how many participants predicted accurately rather than lowering accuracy uniformly, and even with fully correct explanations, only a subset of participants predicted well above chance. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated In exploratory analyses, some participants predicted decisions accurately but described the wrong pattern, and self-reported ratings correlated with prediction accuracy only when explanations were fully correct. > some participants predicted decisions accurately but described the wrong pattern when asked what the AI used, and self-reported ratings correlated with forward simulation accuracy only when explanations were fully correct. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source-stated The authors argue that functional correctness differences do not always translate into differences in human understanding, and call for validation against human outcomes. > not all differences in functional correctness translate to differences in human understanding, highlighting the need to validate functional metrics against human outcomes. - [Supporting source](https://arxiv.org/abs/2603.25251) ## Source and contributor context Baer, Gregor, Zhang, Chao, Grau, Isel, Van Gorp, Pieter · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Time-series Observations arranged over time, such as monthly sales; forecasting uses their patterns to estimate later values. - [Definition reference](https://developers.google.com/machine-learning/glossary#time-series-analysis) ## xAI xAI develops the Grok family of AI models and publishes its own release claims and evaluations. - [Explore this context](/entities/xai) - [Reference](https://x.ai/) - [Reference](https://x.ai/news/grok-4-7) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 57e70792-3486-4cba-8fe5-40462e22aa64. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T01:48:50.128Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: d1c2bcb77f402e1037780365d709edfe3639aeec655edebeb82ccfde05443630. Source text hash: 8153a82c381c42f1f562766810c24e35c1387808752b5921c1433d09b5499151. ### uncertain — 2026-11-09T01:48:50.128Z — anthropic/claude-opus-5.5 Publication: 57e70792-3486-4cba-8fe5-40462e22aa64 attribution — pass: The draft correctly attributes the study and its findings to the listed arXiv authors, Gregor Baer, Chao Zhang, Isel Grau and Pieter Van Gorp. It frames the conclusions as the authors' findings and argument, not as established fact. > Authors:Gregor Baer, Chao Zhang, Isel Grau, Pieter Van Gorp summary — uncertain: The numbers and findings are faithful, but central terms go unexplained for a general reader. The summary never says what functional correctness is (how closely an explanation reflects the model's reasoning). XAI (in the title) and feature attributions are also undefined. The closing sentence drops the source's scope, 'for feature-attribution-style explanations', and hardens 'suggest... not all differences' into 'do not reliably map'. Smallest repair: briefly define correctness and XAI, and restore the feature-attribution qualifier and the hedged wording. > These findings suggest that, for feature-attribution-style explanations, not all differences in functional correctness translate to differences in human understanding date — pass: The value in evidence.originalPublished, 2026-03-26, matches the v1 submission date. Later revisions (v2 to v4) and the November feed date are correctly not used. > \[Submitted on 26 Mar 2026 (v1), last revised 5 Nov 2026 (this version, v4)\] support — pass: Each claim is directly supported by its abstract excerpt. The limitations accurately reflect the study design. The question about generalizing beyond the synthetic task follows from the stated setup and does not imply an unsupported conclusion. No invented benefits or costs. > Correctness affected forward simulation accuracy, but not at every level media — pass: The selected media is the verified, direct arXiv abstract page for this paper. It is the most direct way to access the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/60c716c201efe7074bc00335706131adb1cc58e113a3bc034e6c18048a9aec56/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/does-explanation-correctness-matter-linking-computational-xai-60c716c2.json)