Escape Meridian

Shashank · Nov 15, 2026

FCP checks AI-generated artifacts against the original request

Markdown · JSON · Original source

In its abstract, the paper presents the Formal Context Protocol (FCP) as a way to check AI-generated artifacts against the requests they are meant to satisfy. Its concern is that a model may produce both an artifact and tests that accept it, even when both reflect the same misreading. FCP separates conformance—whether an artifact satisfies a formal reading of a requirement—from fidelity—whether that reading matches the prose. The paper says model checkers and SMT solvers decide conformance, while fidelity has no decision procedure and can instead be refuted by a concrete witness. FCP turns eight ways a proof can falsely certify a requirement into mechanical checks, and turns what machines cannot settle into questions for the request’s author. On four hardware designs, the paper reports, trusting AI-written assertions certified 23% of violated requirements, while FCP certified none. On ten code tasks, code from three language models passed its own tests while violating the request every ; trusting those tests certified all 33 violations, while FCP certified three. These reported results concern the designs and tasks tested.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

arXiv cs.AIArticle · arxiv.orgOpen original source

Original source

cs.AI · Article

Source details

Original source date: Nov 15, 2026 (2026-11-15).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T14:48:01.882Z. First observed: 2026-11-18T05:07:14.155Z. Retrieved: 2026-11-18T14:46:49.952Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

The reported results cover four hardware designs and ten code tasks; the abstract does not establish performance beyond those tested cases. The abstract says fidelity has no decision procedure, so its account does not present fidelity as something the system can decide automatically.

Open question

How would FCP perform on hardware designs and code tasks beyond the four designs and ten tasks reported in the abstract?

Source-stated

The paper argues that tests written by the same language model as an artifact may accept a faulty artifact when both reflect a misreading of the request.

one misreading of the request yields a faulty artifact and a check that accepts it

Source-stated

FCP separates conformance, which the paper says model checkers and SMT solvers decide, from fidelity, which has no decision procedure and can be refuted by a concrete witness.

conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness

Source-stated

The paper says FCP makes eight ways a proof can falsely certify a requirement into mechanical checks that return a witness, while unresolved machine judgments become questions for the request’s author.

Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author.

Source-stated

Across four hardware designs, the paper reports that trusting AI-written assertions certified 23% of violated requirements, while FCP certified none.

On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none.

Source-stated

On ten code tasks, the paper reports that code from three language models passed its own tests while violating the request every time; trusting the tests certified all 33 violations, and FCP certified three.

On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three.

Source-stated

The paper cautions that agreement among models, readings or tests does not imply correctness.

Agreement among models, readings or tests did not imply correctness.

Source and contributor context

Shashank · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

TIME

TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.

Editorial provenance

Version 1. Publication ID: 99dc249a-ad42-4d48-8cb4-366c14c393b0.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:01.882Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 5fbf47dce98c0e4fc7d02c1096e56335fe1977b902234c38cb600a5ae6dd7810. Source text hash: f47291577c132df20f6fceea38fce4ee44ea900de84139eca70accfd37bac8f9.

Revision and source-check history

Version 1 · 2026-11-18T14:48:01.882Z

FCP checks AI-generated artifacts against the original request

In its abstract, the paper presents the Formal Context Protocol (FCP) as a way to check AI-generated artifacts against the requests they are meant to satisfy. Its concern is that a model may produce both an artifact and tests that accept it, even when both reflect the same misreading. FCP separates conformance—whether an artifact satisfies a formal reading of a requirement—from fidelity—whether that reading matches the prose. The paper says model checkers and SMT solvers decide conformance, while fidelity has no decision procedure and can instead be refuted by a concrete witness. FCP turns eight ways a proof can falsely certify a requirement into mechanical checks, and turns what machines cannot settle into questions for the request’s author. On four hardware designs, the paper reports, trusting AI-written assertions certified 23% of violated requirements, while FCP certified none. On ten code tasks, code from three language models passed its own tests while violating the request every ; trusting those tests certified all 33 violations, while FCP certified three. These reported results concern the designs and tasks tested.

Original source

cs.AI · Article

Source details

Original source date: Nov 15, 2026 (2026-11-15).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-18T14:48:01.882Z. First observed: 2026-11-18T05:07:14.155Z. Retrieved: 2026-11-18T14:46:49.952Z.

Limits

The reported results cover four hardware designs and ten code tasks; the abstract does not establish performance beyond those tested cases. The abstract says fidelity has no decision procedure, so its account does not present fidelity as something the system can decide automatically.

Open question

How would FCP perform on hardware designs and code tasks beyond the four designs and ten tasks reported in the abstract?

Source-stated

The paper argues that tests written by the same language model as an artifact may accept a faulty artifact when both reflect a misreading of the request.

one misreading of the request yields a faulty artifact and a check that accepts it

Source-stated

FCP separates conformance, which the paper says model checkers and SMT solvers decide, from fidelity, which has no decision procedure and can be refuted by a concrete witness.

conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness

Source-stated

The paper says FCP makes eight ways a proof can falsely certify a requirement into mechanical checks that return a witness, while unresolved machine judgments become questions for the request’s author.

Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author.

Source-stated

Across four hardware designs, the paper reports that trusting AI-written assertions certified 23% of violated requirements, while FCP certified none.

On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none.

Source-stated

On ten code tasks, the paper reports that code from three language models passed its own tests while violating the request every time; trusting the tests certified all 33 violations, and FCP certified three.

On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three.

Source-stated

The paper cautions that agreement among models, readings or tests does not imply correctness.

Agreement among models, readings or tests did not imply correctness.

Source and contributor context

Shashank · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

TIME

TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.

Editorial provenance

Version 1. Publication ID: 99dc249a-ad42-4d48-8cb4-366c14c393b0.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:01.882Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 5fbf47dce98c0e4fc7d02c1096e56335fe1977b902234c38cb600a5ae6dd7810. Source text hash: f47291577c132df20f6fceea38fce4ee44ea900de84139eca70accfd37bac8f9.

passed · 2026-11-18T14:48:01.882Z · anthropic/claude-opus-5.5

Publication: 99dc249a-ad42-4d48-8cb4-366c14c393b0

attribution — pass: The reading consistently attributes the framing, method and results to the paper or its abstract ('the paper says', 'the paper reports') and does not present them as established fact. Not naming the author (Shashank) is acceptable because the paper is the identified source.

We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself.

summary — pass: The reading leads with the problem: when a model writes both an artifact and its tests, one misreading can produce a faulty artifact and a check that accepts it. It explains conformance and fidelity in plain language and reports the results with their scope. Technical terms such as SMT solvers, decision procedure and witness are kept exactly and are suitable for term-library tooltips; they do not block understanding of the central point. The question is a genuine open issue about generalization. Omitting the adapter sentence (hardware, text, agents and code) is acceptable.

When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence

date — pass: The originalPublished value of 2026-11-15 matches the arXiv v1 submission date. It is the original submission, not the later feed date of 2026-11-18.

[Submitted on 15 Nov 2026]

support — pass: Each claim matches the abstract, including the 23% figure, the 33 violations certified when the tests were trusted, and the three violations FCP certified. One minor scope point: the source reports in the past tense that agreement 'did not imply correctness', while claim 6 says the paper 'cautions' that it 'does not'. The smallest repair is 'did not imply correctness in the reported experiments'. This does not materially change the meaning. The limitations and question add no unsupported benefits or costs.

Agreement among models, readings or tests did not imply correctness.

media — pass: The selected media is the verified direct link to the arXiv abstract page. It is the original source, and no other media was found.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.