Shashank · Nov 15, 2026
FCP checks AI-generated artifacts against the original request
Markdown · JSON · Original source
In its abstract, the paper presents the Formal Context Protocol (FCP) as a way to check AI-generated artifacts against the requests they are meant to satisfy. Its concern is that a model may produce both an artifact and tests that accept it, even when both reflect the same misreading. FCP separates conformance—whether an artifact satisfies a formal reading of a requirement—from fidelity—whether that reading matches the prose. The paper says model checkers and SMT solvers decide conformance, while fidelity has no decision procedure and can instead be refuted by a concrete witness. FCP turns eight ways a proof can falsely certify a requirement into mechanical checks, and turns what machines cannot settle into questions for the request’s author. On four hardware designs, the paper reports, trusting AI-written assertions certified 23% of violated requirements, while FCP certified none. On ten code tasks, code from three language models passed its own tests while violating the request every ; trusting those tests certified all 33 violations, while FCP certified three. These reported results concern the designs and tasks tested.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 15, 2026 (2026-11-15).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-18T14:48:01.882Z. First observed: 2026-11-18T05:07:14.155Z. Retrieved: 2026-11-18T14:46:49.952Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported results cover four hardware designs and ten code tasks; the abstract does not establish performance beyond those tested cases. The abstract says fidelity has no decision procedure, so its account does not present fidelity as something the system can decide automatically.
Open question
How would FCP perform on hardware designs and code tasks beyond the four designs and ten tasks reported in the abstract?
Source-stated
The paper argues that tests written by the same language model as an artifact may accept a faulty artifact when both reflect a misreading of the request.
one misreading of the request yields a faulty artifact and a check that accepts it
Source-stated
FCP separates conformance, which the paper says model checkers and SMT solvers decide, from fidelity, which has no decision procedure and can be refuted by a concrete witness.
conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness
Source-stated
The paper says FCP makes eight ways a proof can falsely certify a requirement into mechanical checks that return a witness, while unresolved machine judgments become questions for the request’s author.
Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author.
Source-stated
Across four hardware designs, the paper reports that trusting AI-written assertions certified 23% of violated requirements, while FCP certified none.
On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none.
Source-stated
On ten code tasks, the paper reports that code from three language models passed its own tests while violating the request every time; trusting the tests certified all 33 violations, and FCP certified three.
On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three.
Source-stated
The paper cautions that agreement among models, readings or tests does not imply correctness.
Agreement among models, readings or tests did not imply correctness.
Source and contributor context
Shashank · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 99dc249a-ad42-4d48-8cb4-366c14c393b0.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:01.882Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 5fbf47dce98c0e4fc7d02c1096e56335fe1977b902234c38cb600a5ae6dd7810. Source text hash: f47291577c132df20f6fceea38fce4ee44ea900de84139eca70accfd37bac8f9.
Revision and source-check history
Version 1 · 2026-11-18T14:48:01.882Z
FCP checks AI-generated artifacts against the original request
In its abstract, the paper presents the Formal Context Protocol (FCP) as a way to check AI-generated artifacts against the requests they are meant to satisfy. Its concern is that a model may produce both an artifact and tests that accept it, even when both reflect the same misreading. FCP separates conformance—whether an artifact satisfies a formal reading of a requirement—from fidelity—whether that reading matches the prose. The paper says model checkers and SMT solvers decide conformance, while fidelity has no decision procedure and can instead be refuted by a concrete witness. FCP turns eight ways a proof can falsely certify a requirement into mechanical checks, and turns what machines cannot settle into questions for the request’s author. On four hardware designs, the paper reports, trusting AI-written assertions certified 23% of violated requirements, while FCP certified none. On ten code tasks, code from three language models passed its own tests while violating the request every ; trusting those tests certified all 33 violations, while FCP certified three. These reported results concern the designs and tasks tested.
Original source
cs.AI · Article
Source details
Original source date: Nov 15, 2026 (2026-11-15).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-18T14:48:01.882Z. First observed: 2026-11-18T05:07:14.155Z. Retrieved: 2026-11-18T14:46:49.952Z.
Limits
The reported results cover four hardware designs and ten code tasks; the abstract does not establish performance beyond those tested cases. The abstract says fidelity has no decision procedure, so its account does not present fidelity as something the system can decide automatically.
Open question
How would FCP perform on hardware designs and code tasks beyond the four designs and ten tasks reported in the abstract?
Source-stated
The paper argues that tests written by the same language model as an artifact may accept a faulty artifact when both reflect a misreading of the request.
one misreading of the request yields a faulty artifact and a check that accepts it
Source-stated
FCP separates conformance, which the paper says model checkers and SMT solvers decide, from fidelity, which has no decision procedure and can be refuted by a concrete witness.
conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness
Source-stated
The paper says FCP makes eight ways a proof can falsely certify a requirement into mechanical checks that return a witness, while unresolved machine judgments become questions for the request’s author.
Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author.
Source-stated
Across four hardware designs, the paper reports that trusting AI-written assertions certified 23% of violated requirements, while FCP certified none.
On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none.
Source-stated
On ten code tasks, the paper reports that code from three language models passed its own tests while violating the request every time; trusting the tests certified all 33 violations, and FCP certified three.
On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three.
Source-stated
The paper cautions that agreement among models, readings or tests does not imply correctness.
Agreement among models, readings or tests did not imply correctness.
Source and contributor context
Shashank · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 99dc249a-ad42-4d48-8cb4-366c14c393b0.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-18T14:48:01.882Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 5fbf47dce98c0e4fc7d02c1096e56335fe1977b902234c38cb600a5ae6dd7810. Source text hash: f47291577c132df20f6fceea38fce4ee44ea900de84139eca70accfd37bac8f9.
passed · 2026-11-18T14:48:01.882Z · anthropic/claude-opus-5.5
Publication: 99dc249a-ad42-4d48-8cb4-366c14c393b0
attribution — pass: The reading consistently attributes the framing, method and results to the paper or its abstract ('the paper says', 'the paper reports') and does not present them as established fact. Not naming the author (Shashank) is acceptable because the paper is the identified source.
We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself.
summary — pass: The reading leads with the problem: when a model writes both an artifact and its tests, one misreading can produce a faulty artifact and a check that accepts it. It explains conformance and fidelity in plain language and reports the results with their scope. Technical terms such as SMT solvers, decision procedure and witness are kept exactly and are suitable for term-library tooltips; they do not block understanding of the central point. The question is a genuine open issue about generalization. Omitting the adapter sentence (hardware, text, agents and code) is acceptable.
When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence
date — pass: The originalPublished value of 2026-11-15 matches the arXiv v1 submission date. It is the original submission, not the later feed date of 2026-11-18.
[Submitted on 15 Nov 2026]
support — pass: Each claim matches the abstract, including the 23% figure, the 33 violations certified when the tests were trusted, and the three violations FCP certified. One minor scope point: the source reports in the past tense that agreement 'did not imply correctness', while claim 6 says the paper 'cautions' that it 'does not'. The smallest repair is 'did not imply correctness in the reported experiments'. This does not materially change the meaning. The limitations and question add no unsupported benefits or costs.
Agreement among models, readings or tests did not imply correctness.
media — pass: The selected media is the verified direct link to the arXiv abstract page. It is the original source, and no other media was found.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv