Sun, Yutao, Li, Haokun, Chen, Mingshuai, Li, Guanyan, Wang, Zhongyi, Qin, Shengchao, Yin, Jianwei · Nov 7, 2026
Claude Haiku 4.5 with an FSM improves procedural decisions in the authors’ test
Markdown · JSON · Original source
The authors argue that language-model agents can drift from a procedure during long unattended runs, and that repeating instructions or restarting an agent may simply shift the drift. They study an executable finite-state machine (FSM): workflow state is kept outside the model’s context, and the runtime checks tool calls, state transitions and the end of each turn. In a pre-registered experiment on 200-account SOP-Bench batches, Haiku 4.5 with the FSM averaged 139 accounts handled correctly and by the procedure, compared with 48 for the strongest baseline. The authors say the FSM maintained this rate to the end, and that gains across three models tracked how much the plain agent drifted. They also report that Haiku 4.5 with the FSM averaged more such decisions than plain Claude Sonnet 4.6 at 38% of the cost. The authors’ central implication is that control of execution, not only a more capable model, may support reliable procedural automation. The abstract also points to analyses on SWE-bench Verified, $\tau^2$-bench, and staged and cross-context workflows, but the reported account counts concern the SOP-Bench experiment.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 7, 2026 (2026-11-07).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T05:33:57.057Z. First observed: 2026-11-10T08:36:47.900Z. Retrieved: 2026-11-12T05:32:57.104Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported account counts are from the described SOP-Bench experiment; the abstract names other analyses but gives no corresponding results for them. The evidence is the paper’s abstract and does not provide the experimental procedures or further detail about the evaluated workflows.
Open question
How do the FSM’s results on SOP-Bench and the other named workflows vary across models and workflows?
Source-stated
The authors say language-model agents drift from procedures during long unattended runs, and that restating instructions or resuming a stopped agent shifts rather than removes the drift.
agents drift from such procedures over long runs, and that restating the instructions or resuming a stopped agent only moves the drift to another shortcut.
Source-stated
The studied executable finite-state machine keeps workflow state outside the model’s context and enforces it at tool calls, state transitions and the end of the agent’s turn.
keeps the workflow state outside the model's context and enforces it at three points: tool calls, state transitions and the end of the agent's turn.
Source-stated
In the pre-registered experiment on 200-account SOP-Bench batches, Claude Haiku 4.5 with the FSM averaged 139 accounts decided correctly and by the procedure, versus 48 for the strongest baseline.
Claude Haiku 4.5 with the FSM decided on average 139 accounts correctly and by the procedure, against 48 for the strongest baseline
Source-stated
The authors report that Haiku 4.5 with the FSM averaged more procedural decisions than plain Claude Sonnet 4.6 at 38% of the cost.
Haiku 4.5 with it averaged more such decisions than plain Claude Sonnet 4.6 at 38% of the cost.
Source-stated
The authors suggest that reliable long-horizon automation may come from stronger execution control, not only stronger models.
reliable long-horizon automation can come from stronger execution control, not only stronger models.
Source and contributor context
Sun, Yutao, Li, Haokun, Chen, Mingshuai, Li, Guanyan, Wang, Zhongyi, Qin, Shengchao, Yin, Jianwei · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 18263b2c-55ff-4094-8cf3-4c30f889cc5d.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T05:33:57.057Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 63ac384d3ab2159bb58ce516b31f86026324e31f5284b3649832dfec99eb33d8. Source text hash: 404871ccfc432aaa0204f8250528f0c0347b57bb910e45a7a5a8d4a47b2448ba.
Revision and source-check history
Version 1 · 2026-11-12T05:33:57.057Z
Claude Haiku 4.5 with an FSM improves procedural decisions in the authors’ test
The authors argue that language-model agents can drift from a procedure during long unattended runs, and that repeating instructions or restarting an agent may simply shift the drift. They study an executable finite-state machine (FSM): workflow state is kept outside the model’s context, and the runtime checks tool calls, state transitions and the end of each turn. In a pre-registered experiment on 200-account SOP-Bench batches, Haiku 4.5 with the FSM averaged 139 accounts handled correctly and by the procedure, compared with 48 for the strongest baseline. The authors say the FSM maintained this rate to the end, and that gains across three models tracked how much the plain agent drifted. They also report that Haiku 4.5 with the FSM averaged more such decisions than plain Claude Sonnet 4.6 at 38% of the cost. The authors’ central implication is that control of execution, not only a more capable model, may support reliable procedural automation. The abstract also points to analyses on SWE-bench Verified, $\tau^2$-bench, and staged and cross-context workflows, but the reported account counts concern the SOP-Bench experiment.
Original source
cs.AI · Article
Source details
Original source date: Nov 7, 2026 (2026-11-07).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-12T05:33:57.057Z. First observed: 2026-11-10T08:36:47.900Z. Retrieved: 2026-11-12T05:32:57.104Z.
Limits
The reported account counts are from the described SOP-Bench experiment; the abstract names other analyses but gives no corresponding results for them. The evidence is the paper’s abstract and does not provide the experimental procedures or further detail about the evaluated workflows.
Open question
How do the FSM’s results on SOP-Bench and the other named workflows vary across models and workflows?
Source-stated
The authors say language-model agents drift from procedures during long unattended runs, and that restating instructions or resuming a stopped agent shifts rather than removes the drift.
agents drift from such procedures over long runs, and that restating the instructions or resuming a stopped agent only moves the drift to another shortcut.
Source-stated
The studied executable finite-state machine keeps workflow state outside the model’s context and enforces it at tool calls, state transitions and the end of the agent’s turn.
keeps the workflow state outside the model's context and enforces it at three points: tool calls, state transitions and the end of the agent's turn.
Source-stated
In the pre-registered experiment on 200-account SOP-Bench batches, Claude Haiku 4.5 with the FSM averaged 139 accounts decided correctly and by the procedure, versus 48 for the strongest baseline.
Claude Haiku 4.5 with the FSM decided on average 139 accounts correctly and by the procedure, against 48 for the strongest baseline
Source-stated
The authors report that Haiku 4.5 with the FSM averaged more procedural decisions than plain Claude Sonnet 4.6 at 38% of the cost.
Haiku 4.5 with it averaged more such decisions than plain Claude Sonnet 4.6 at 38% of the cost.
Source-stated
The authors suggest that reliable long-horizon automation may come from stronger execution control, not only stronger models.
reliable long-horizon automation can come from stronger execution control, not only stronger models.
Source and contributor context
Sun, Yutao, Li, Haokun, Chen, Mingshuai, Li, Guanyan, Wang, Zhongyi, Qin, Shengchao, Yin, Jianwei · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 18263b2c-55ff-4094-8cf3-4c30f889cc5d.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T05:33:57.057Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 63ac384d3ab2159bb58ce516b31f86026324e31f5284b3649832dfec99eb33d8. Source text hash: 404871ccfc432aaa0204f8250528f0c0347b57bb910e45a7a5a8d4a47b2448ba.
uncertain · 2026-11-12T05:33:57.057Z · anthropic/claude-opus-5.5
Publication: 18263b2c-55ff-4094-8cf3-4c30f889cc5d
attribution — pass: The reading consistently attributes the findings and implications to the paper's authors, using phrases like 'the authors argue' and 'the authors report'. It does not present SOP-Bench results or cost claims as established fact or as Escape Meridian's view.
Authors:Yutao Sun, Haokun Li, Mingshuai Chen, Guanyan Li, Zhongyi Wang, Shengchao Qin, Jianwei Yin
summary — uncertain: The summary is faithful overall, but readers without specialist background are left with two gaps. First, 'SOP' and 'SOP-Bench' are never explained; the reading never says that an SOP is a standard operating procedure or that SOP-Bench is a benchmark of such procedures. Second, it drops the abstract's 95% CI for the difference, [46, 130]. That wide interval is the source's own signal of uncertainty about the size of the gain. It also omits the framing that each step 'needs little intelligence'. Smallest repair: add a plain gloss of SOP/SOP-Bench and the confidence interval.
against 48 for the strongest baseline (95% CI of the difference [46, 130])
date — pass: The originalPublished value, 2026-11-07, matches the v1 submission date on arXiv. The feed date of 2026-11-10 is correctly not used.
[Submitted on 7 Nov 2026]
support — pass: Every claim is backed by an exact quotation from the abstract. The implication is kept conditional ('may support'), and the limitations correctly say the account counts come only from SOP-Bench. The softening 'may simply shift' is weaker than the source's 'only moves', but it does not misstate the claim. The question does not presuppose any result. No unsupported beneficiaries or costs are introduced.
Because the runtime, not the model, supplies this long-horizon reliability, the FSM can let small models carry procedural work
media — pass: The selected media is the verified direct link to the arXiv abstract page for this paper, which is the most direct way to reach the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv