Ding, Jiayu, Huang, Yupan, Park, Jinyoung, Jia, Yilin, Wu, Wenshan, Lv, Tengchao, Huang, Shaohan, Zhao, Yuzhong, Zhou, Xiangyang, Ge, Tao, Mao, Sun, Wang, Xun, Jiao, Huitian, Chen, Si-Qing, Cui, Lei, Zheng, Nanning, Wei, Furu · Nov 11, 2026
The authors introduce a benchmark for context-based Word editing
Markdown · JSON · Original source
The authors introduce WordInferBench to test a gap they say existing Word-editing leave unmeasured: whether agents can infer formatting changes from a document, rather than follow instructions that spell out each change. Each of 280 base editing instances has descriptive, referential and inferential instruction conditions. In the inferential condition, an agent must use an in-document exemplar—a matching example elsewhere in the document—to determine the target formatting. The authors say the instances come from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes. Across six evaluated coding-agent systems, Code with Opus 4.6 has the highest reported task-level pass rate: 62.9% on descriptive variants and 40.7% on inferential variants. Escape Meridian’s reading: on this benchmark, success with explicit directions does not carry over as reliably when the needed formatting must be inferred from context. That distinction matters for editing tasks in which users expect consistency without listing every attribute. The evidence is limited to the benchmark and systems evaluated; the abstract does not establish how these results translate to editing work beyond those test instances.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 11, 2026 (2026-11-11).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T12:01:37.537Z. First observed: 2026-11-13T07:39:49.163Z. Retrieved: 2026-11-13T11:47:56.131Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported results concern the ’s constructed instances and six evaluated coding-agent systems; the abstract does not establish how they generalize to editing work beyond those tests. The inferential instances are constructed so the target formatting is recoverable from an in-document exemplar, which bounds what the benchmark tests.
Open question
How do the other five evaluated systems compare on the difference between descriptive and inferential variants?
Source-stated
The authors introduce a benchmark for evaluating whether agents can infer formatting changes from document context, a capability they say existing Word-editing benchmarks do not evaluate.
They do not evaluate agents' ability to infer the appropriate attributes from document context
Source-stated
Each of the 280 base editing instances is paired with descriptive, referential and inferential instruction conditions.
a benchmark comprising 280 base document-editing instances, each paired with three instruction conditions: descriptive
Source-stated
The instances come from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes.
derived from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes.
Source-stated
Among six evaluated coding-agent systems, Claude Code with Opus 4.6 has the highest reported pass rate, with a lower rate on inferential than descriptive variants.
the best-performing evaluated system, Claude Code with Opus 4.6, achieves a 62.9% task-level pass rate on descriptive variants but only 40.7% on inferential variants
Escape Meridian interpretation
The result suggests that, on this benchmark, performance with explicit editing directions may not carry over as reliably when agents must infer changes from context.
indicating that current systems complete tasks less reliably when the required formatting changes must be determined from document context.
Source and contributor context
Ding, Jiayu, Huang, Yupan, Park, Jinyoung, Jia, Yilin, Wu, Wenshan, Lv, Tengchao, Huang, Shaohan, Zhao, Yuzhong, Zhou, Xiangyang, Ge, Tao, Mao, Sun, Wang, Xun, Jiao, Huitian, Chen, Si-Qing, Cui, Lei, Zheng, Nanning, Wei, Furu · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: 8111c9fa-2f62-43b3-8c28-660da7bd90a0.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T12:01:37.537Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 30933e722ae30d2907f2336dba391d0c1e91d2262b10bc0146f96f9ddc4cf5be. Source text hash: fe6454854841ab2506be9b1a8cf16fd99c01d4cf0adf42d2fb6613f0a799ff08.
Revision and source-check history
Version 1 · 2026-11-13T12:01:37.537Z
The authors introduce a benchmark for context-based Word editing
The authors introduce WordInferBench to test a gap they say existing Word-editing leave unmeasured: whether agents can infer formatting changes from a document, rather than follow instructions that spell out each change. Each of 280 base editing instances has descriptive, referential and inferential instruction conditions. In the inferential condition, an agent must use an in-document exemplar—a matching example elsewhere in the document—to determine the target formatting. The authors say the instances come from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes. Across six evaluated coding-agent systems, Code with Opus 4.6 has the highest reported task-level pass rate: 62.9% on descriptive variants and 40.7% on inferential variants. Escape Meridian’s reading: on this benchmark, success with explicit directions does not carry over as reliably when the needed formatting must be inferred from context. That distinction matters for editing tasks in which users expect consistency without listing every attribute. The evidence is limited to the benchmark and systems evaluated; the abstract does not establish how these results translate to editing work beyond those test instances.
Original source
cs.AI · Article
Source details
Original source date: Nov 11, 2026 (2026-11-11).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-13T12:01:37.537Z. First observed: 2026-11-13T07:39:49.163Z. Retrieved: 2026-11-13T11:47:56.131Z.
Limits
The reported results concern the ’s constructed instances and six evaluated coding-agent systems; the abstract does not establish how they generalize to editing work beyond those tests. The inferential instances are constructed so the target formatting is recoverable from an in-document exemplar, which bounds what the benchmark tests.
Open question
How do the other five evaluated systems compare on the difference between descriptive and inferential variants?
Source-stated
The authors introduce a benchmark for evaluating whether agents can infer formatting changes from document context, a capability they say existing Word-editing benchmarks do not evaluate.
They do not evaluate agents' ability to infer the appropriate attributes from document context
Source-stated
Each of the 280 base editing instances is paired with descriptive, referential and inferential instruction conditions.
a benchmark comprising 280 base document-editing instances, each paired with three instruction conditions: descriptive
Source-stated
The instances come from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes.
derived from real Word documents collected from Common Crawl and cover 13 object types and 81 attributes.
Source-stated
Among six evaluated coding-agent systems, Claude Code with Opus 4.6 has the highest reported pass rate, with a lower rate on inferential than descriptive variants.
the best-performing evaluated system, Claude Code with Opus 4.6, achieves a 62.9% task-level pass rate on descriptive variants but only 40.7% on inferential variants
Escape Meridian interpretation
The result suggests that, on this benchmark, performance with explicit editing directions may not carry over as reliably when agents must infer changes from context.
indicating that current systems complete tasks less reliably when the required formatting changes must be determined from document context.
Source and contributor context
Ding, Jiayu, Huang, Yupan, Park, Jinyoung, Jia, Yilin, Wu, Wenshan, Lv, Tengchao, Huang, Shaohan, Zhao, Yuzhong, Zhou, Xiangyang, Ge, Tao, Mao, Sun, Wang, Xun, Jiao, Huitian, Chen, Si-Qing, Cui, Lei, Zheng, Nanning, Wei, Furu · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: 8111c9fa-2f62-43b3-8c28-660da7bd90a0.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T12:01:37.537Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 30933e722ae30d2907f2336dba391d0c1e91d2262b10bc0146f96f9ddc4cf5be. Source text hash: fe6454854841ab2506be9b1a8cf16fd99c01d4cf0adf42d2fb6613f0a799ff08.
passed · 2026-11-13T12:01:37.537Z · anthropic/claude-opus-5.5
Publication: 8111c9fa-2f62-43b3-8c28-660da7bd90a0
attribution — pass: The reading attributes the benchmark, its motivating gap, the dataset details and the results to the paper's authors. It labels the generalization point as Escape Meridian's reading.
We introduce WordInferBench, a benchmark comprising 280 base document-editing instances
summary — pass: The reading leads with the authors' new benchmark and the gap they claim it fills. It explains the in-document exemplar in plain language and reports the headline 62.9% vs 40.7% result with its scope. The title and question are accurate and non-leading; the abstract does not report the other five systems' scores, so that question is genuinely open. Referential and descriptive conditions are only named, but they are not central to the finding. A Common Crawl tooltip could help readers but is not required.
inferential (the agent must examine document context to infer what attributes to apply)
date — pass: originalPublished 2026-11-11 matches the arXiv v1 submission date. The later feed date of Nov 13 was correctly not used.
[Submitted on 11 Nov 2026]
support — pass: All factual claims match the abstract: 280 instances, three instruction conditions, Common Crawl sourcing, 13 object types, 81 attributes, six systems and the best system's two pass rates. The interpretation is labeled and grounded in the authors' own conclusion, and is narrowed to this benchmark. The limitations correctly note the exemplar-recoverable construction. No benefits or costs beyond the source are asserted.
indicating that current systems complete tasks less reliably when the required formatting changes must be determined from document context.
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the easiest route to the original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference