# Reda Heddad’s arXiv paper tests runtime contracts for coding agents

[Canonical reading](https://escapemeridian.com/readings/reda-heddad-s-arxiv-paper-tests-runtime-contracts-for-coding-f73f5819)

Reda Heddad’s paper, “Beyond Prompts: Runtime Contracts for Coding Agents,” tests whether executable contracts—rules enforced while an agent works—prevent coding-agent rule violations better than prompt text, retrieved guidance or checks after generation. Heddad argues that prompt text provides neither enforcement nor evidence that rules were followed, a gap for teams that expect both. Across three studies, enforcement during execution produced no oracle-detected violation in 104 paired runs, while 21 to 55% of runs without rules had violations. In a confirmatory study targeting known failure situations, enforcement reduced violating runs by 31 percentage points relative to no rules. The 17-point difference from prompted rules was not significant, and neither were contrasts in another study, where few tasks produced violations. The paper reports no loss of functional correctness in its two confirmatory studies. Heddad bounds the contribution: enforcement satisfies the rules it enforces by construction; the data instead address where prompting falls short, the absence of a functional cost, and limits in the rules and oracle. The studies used low-cost models and one small fixture, so the abstract does not establish how results generalize beyond those settings.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 14, 2026 (2026-11-14).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T09:18:38.216Z. First observed: 2026-11-17T08:31:37.880Z. Retrieved: 2026-11-17T09:17:49.748Z.

- [Beyond Prompts: Runtime Contracts for Coding Agents](https://arxiv.org/abs/2611.19549)

## Limits

The abstract reports that the studies used low-cost models and one small fixture; it does not establish how results generalize beyond those settings. The paper notes that violations are scored by an oracle and describes limits of the rules and oracle themselves.


## Open question

How would runtime contracts perform beyond the low-cost models and one small fixture used in these studies?


## Source-stated

Reda Heddad argues that prompt text alone does not enforce engineering rules or provide evidence of their application.

> a rule given as prompt text provides neither

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

Across the reported studies, enforcement during execution produced no oracle-detected violation in 104 paired runs; runs without rules had violations at rates of 21 to 55%.

> enforcement during execution produced no oracle-detected violation in 104 paired runs, against 21 to 55% of runs without rules

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

In Study 3, enforcement reduced the share of violating runs by 31 percentage points relative to no rules; its 17-point difference from prompted rules was not significant.

> enforcement lowered the share of violating runs by 31 percentage points relative to no rules (Holm-adjusted p = 0.014); the 17-point difference from prompted rules was not significant (p = 0.12)

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

The paper reports no loss of functional correctness in its two confirmatory studies.

> no loss of functional correctness in the two confirmatory studies

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

The authors characterize the evidence as bounded by the rules and oracle, and note that the studies used low-cost models and one small fixture.

> the limits of the rules and of the oracle themselves. The studies use low-cost models and one small fixture.

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source and contributor context

Heddad, Reda · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: ba2ae164-30ad-4edb-83d7-a8150292d6ca.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T09:18:38.216Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: dc8958fa77ecb5df729b302c60502cccdc217e71e5015693ddfb2fa15ebc0417. Source text hash: 3d70c798cf85c51ac78455c464e4eab3387d210f5639214eb21d7798b1e9643d.


## Revision and source-check history

### Version 1 — 2026-11-17T09:18:38.216Z

Reda Heddad’s paper, “Beyond Prompts: Runtime Contracts for Coding Agents,” tests whether executable contracts—rules enforced while an agent works—prevent coding-agent rule violations better than prompt text, retrieved guidance or checks after generation. Heddad argues that prompt text provides neither enforcement nor evidence that rules were followed, a gap for teams that expect both. Across three studies, enforcement during execution produced no oracle-detected violation in 104 paired runs, while 21 to 55% of runs without rules had violations. In a confirmatory study targeting known failure situations, enforcement reduced violating runs by 31 percentage points relative to no rules. The 17-point difference from prompted rules was not significant, and neither were contrasts in another study, where few tasks produced violations. The paper reports no loss of functional correctness in its two confirmatory studies. Heddad bounds the contribution: enforcement satisfies the rules it enforces by construction; the data instead address where prompting falls short, the absence of a functional cost, and limits in the rules and oracle. The studies used low-cost models and one small fixture, so the abstract does not establish how results generalize beyond those settings.

## Original source

arXiv cs.AI · Article

Original source date: Nov 14, 2026 (2026-11-14).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-17T09:18:38.216Z. First observed: 2026-11-17T08:31:37.880Z. Retrieved: 2026-11-17T09:17:49.748Z.

- [Beyond Prompts: Runtime Contracts for Coding Agents](https://arxiv.org/abs/2611.19549)

## Limits

The abstract reports that the studies used low-cost models and one small fixture; it does not establish how results generalize beyond those settings. The paper notes that violations are scored by an oracle and describes limits of the rules and oracle themselves.


## Open question

How would runtime contracts perform beyond the low-cost models and one small fixture used in these studies?


## Source-stated

Reda Heddad argues that prompt text alone does not enforce engineering rules or provide evidence of their application.

> a rule given as prompt text provides neither

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

Across the reported studies, enforcement during execution produced no oracle-detected violation in 104 paired runs; runs without rules had violations at rates of 21 to 55%.

> enforcement during execution produced no oracle-detected violation in 104 paired runs, against 21 to 55% of runs without rules

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

In Study 3, enforcement reduced the share of violating runs by 31 percentage points relative to no rules; its 17-point difference from prompted rules was not significant.

> enforcement lowered the share of violating runs by 31 percentage points relative to no rules (Holm-adjusted p = 0.014); the 17-point difference from prompted rules was not significant (p = 0.12)

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

The paper reports no loss of functional correctness in its two confirmatory studies.

> no loss of functional correctness in the two confirmatory studies

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source-stated

The authors characterize the evidence as bounded by the rules and oracle, and note that the studies used low-cost models and one small fixture.

> the limits of the rules and of the oracle themselves. The studies use low-cost models and one small fixture.

- [Supporting source](https://arxiv.org/abs/2611.19549)

## Source and contributor context

Heddad, Reda · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: ba2ae164-30ad-4edb-83d7-a8150292d6ca.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-17T09:18:38.216Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: dc8958fa77ecb5df729b302c60502cccdc217e71e5015693ddfb2fa15ebc0417. Source text hash: 3d70c798cf85c51ac78455c464e4eab3387d210f5639214eb21d7798b1e9643d.


### uncertain — 2026-11-17T09:18:38.216Z — anthropic/claude-opus-5.5

Publication: ba2ae164-30ad-4edb-83d7-a8150292d6ca

attribution — pass: The paper, title and sole author Reda Heddad are correctly identified, and findings are attributed to the paper. Claim 5 says 'The authors' for a single-author paper. This is minor wording, not a misattribution.

> Title:Beyond Prompts: Runtime Contracts for Coding Agents
Authors:Reda Heddad

summary — uncertain: The figures are faithful, but the reading has two gaps. First, 'oracle' and 'fixture' are central yet unexplained. A general reader needs to know that the oracle is a separate scoring method, distinct from the in-loop checks and validated by two human reviewers and a model reviewer. Second, the summary drops the paper's concrete account of where prompting falls short: prompted rules did not stop recursive deletes, and post-generation checks could not undo commands that had already run. Smallest repair: add a plain gloss for oracle, and add one sentence on these two failure examples.

> Prompted rules removed some violation types but not recursive deletes of scratch directories, and checks after generation repaired file edits but not commands that had already run.

date — pass: The originalPublished date of 2026-11-14 matches the arXiv v1 submission date. It does not reflect the later feed date.

> \[Submitted on 14 Nov 2026\]

support — pass: Each claim's excerpt is verbatim and supports it. The non-significant 17-point contrast and the null contrasts in Study 2 are preserved. The question and limitations follow from the stated low-cost models and one small fixture. No unsupported benefits are asserted.

> the 17-point difference from prompted rules was not significant (p = 0.12), and neither was any contrast in Study 2, where only 3 of 14 tasks produced a violation

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the original source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/f73f5819cc942e1aa62fd489cc7da9d6483c646814ddf7d786bb62502766ec09/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/reda-heddad-s-arxiv-paper-tests-runtime-contracts-for-coding-f73f5819.json)
