Gupta, Sachin · Nov 4, 2026
AgentTrustCI: Build-Time Policy, Grounding, and Trajectory Tests for Java LLM Agents
Markdown · JSON · Original source
Sachin Gupta presents AgentTrustCI, a Java/JUnit 5 framework for testing whether agents follow application-level expectations that infrastructure security does not cover: selecting allowed tools, grounding answers in tool output, and following required workflows. It applies policy, grounding, and trajectory checks to recorded traces, with deterministic replay for repeatable build- tests and an optional judge interface for semantic evaluation. In 55 synthetic cases across seven enterprise domains, Gupta reports that the full pipeline classified all 30 injected faults and 25 passing controls correctly, and missed none of 12 designated critical cases. The single-layer configurations each classified 35 cases; the corpus deliberately isolates ten faults per layer. One example explains why trajectory checks add coverage: an answer can be grounded in tool output yet obtained through a disallowed sequence. These results cover the authored regression suite, not production accuracy. Gupta describes a protocol for validating production traces, optional LLM-judge integration, and maintaining expectations as workflows evolve. The paper’s practical idea is to make agent behavior part of the existing Java release pipeline, while its evidence leaves production performance to be tested.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T01:49:20.130Z. First observed: 2026-11-05T05:06:35.184Z. Retrieved: 2026-11-10T01:48:43.737Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evaluation uses 55 synthetic cases; the paper states that its results establish coverage of the authored regression suite, not production accuracy. The supplied evidence is the abstract and does not report outcomes from applying the production-trace validation protocol.
Open question
How do AgentTrustCI’s checks perform when applied to production traces, under the production-trace validation protocol Gupta describes?
Source-stated
Gupta presents AgentTrustCI as a Java/JUnit 5 framework that tests application-level expectations for LLM agents using policy, grounding, and trajectory checks on recorded traces.
AgentTrustCI is a Java/JUnit 5 framework that expresses these application-layer expectations as policy, grounding, and trajectory checks over recorded traces.
Source-stated
The framework uses deterministic replay for repeatable build-time tests and offers a judge interface for optional semantic evaluation.
A deterministic replay implementation supports repeatable build-time tests; a judge interface provides an extension point for optional semantic evaluation.
Source-stated
On 55 synthetic cases across seven enterprise domains, the full pipeline correctly classified 30 injected faults and 25 passing controls, and missed none of 12 designated critical cases.
On 55 synthetic cases spanning seven enterprise domains, the full pipeline correctly classifies all 30 injected faults and 25 passing controls, including ten non-trivial controls, and misses none of 12 designated critical cases.
Source-stated
The single-layer configurations each classified 35 cases because the corpus deliberately isolated ten faults per layer.
Each single-layer configuration correctly classifies 35 cases because the corpus deliberately isolates ten faults per layer.
Source-stated
The paper gives an example of a grounded answer produced through a disallowed sequence to illustrate the added coverage of trajectory checks.
A grounded answer obtained through a disallowed sequence illustrates the additional coverage of trajectory checks.
Source-stated
The reported results establish coverage of the authored regression suite, not production accuracy.
These results establish coverage of the authored regression suite, not production accuracy.
Source-stated
Gupta describes a production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve.
We describe a production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve.
Source and contributor context
Gupta, Sachin · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: a480e1a2-d06b-4edb-8764-fc9a909daafa.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-10T01:49:20.130Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: ac51e39e6a5737b92fd74ac9b950f8a0752ff45846dd8e342e2a17e5bfe65235. Source text hash: d883bb2bd739056b7510a6ef1b0237e08893024406e1b5379f442c1c2e43365f.
Revision and source-check history
Version 1 · 2026-11-10T01:49:20.130Z
AgentTrustCI: Build-Time Policy, Grounding, and Trajectory Tests for Java LLM Agents
Sachin Gupta presents AgentTrustCI, a Java/JUnit 5 framework for testing whether agents follow application-level expectations that infrastructure security does not cover: selecting allowed tools, grounding answers in tool output, and following required workflows. It applies policy, grounding, and trajectory checks to recorded traces, with deterministic replay for repeatable build- tests and an optional judge interface for semantic evaluation. In 55 synthetic cases across seven enterprise domains, Gupta reports that the full pipeline classified all 30 injected faults and 25 passing controls correctly, and missed none of 12 designated critical cases. The single-layer configurations each classified 35 cases; the corpus deliberately isolates ten faults per layer. One example explains why trajectory checks add coverage: an answer can be grounded in tool output yet obtained through a disallowed sequence. These results cover the authored regression suite, not production accuracy. Gupta describes a protocol for validating production traces, optional LLM-judge integration, and maintaining expectations as workflows evolve. The paper’s practical idea is to make agent behavior part of the existing Java release pipeline, while its evidence leaves production performance to be tested.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T01:49:20.130Z. First observed: 2026-11-05T05:06:35.184Z. Retrieved: 2026-11-10T01:48:43.737Z.
Limits
The evaluation uses 55 synthetic cases; the paper states that its results establish coverage of the authored regression suite, not production accuracy. The supplied evidence is the abstract and does not report outcomes from applying the production-trace validation protocol.
Open question
How do AgentTrustCI’s checks perform when applied to production traces, under the production-trace validation protocol Gupta describes?
Source-stated
Gupta presents AgentTrustCI as a Java/JUnit 5 framework that tests application-level expectations for LLM agents using policy, grounding, and trajectory checks on recorded traces.
AgentTrustCI is a Java/JUnit 5 framework that expresses these application-layer expectations as policy, grounding, and trajectory checks over recorded traces.
Source-stated
The framework uses deterministic replay for repeatable build-time tests and offers a judge interface for optional semantic evaluation.
A deterministic replay implementation supports repeatable build-time tests; a judge interface provides an extension point for optional semantic evaluation.
Source-stated
On 55 synthetic cases across seven enterprise domains, the full pipeline correctly classified 30 injected faults and 25 passing controls, and missed none of 12 designated critical cases.
On 55 synthetic cases spanning seven enterprise domains, the full pipeline correctly classifies all 30 injected faults and 25 passing controls, including ten non-trivial controls, and misses none of 12 designated critical cases.
Source-stated
The single-layer configurations each classified 35 cases because the corpus deliberately isolated ten faults per layer.
Each single-layer configuration correctly classifies 35 cases because the corpus deliberately isolates ten faults per layer.
Source-stated
The paper gives an example of a grounded answer produced through a disallowed sequence to illustrate the added coverage of trajectory checks.
A grounded answer obtained through a disallowed sequence illustrates the additional coverage of trajectory checks.
Source-stated
The reported results establish coverage of the authored regression suite, not production accuracy.
These results establish coverage of the authored regression suite, not production accuracy.
Source-stated
Gupta describes a production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve.
We describe a production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve.
Source and contributor context
Gupta, Sachin · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: a480e1a2-d06b-4edb-8764-fc9a909daafa.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-10T01:49:20.130Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: ac51e39e6a5737b92fd74ac9b950f8a0752ff45846dd8e342e2a17e5bfe65235. Source text hash: d883bb2bd739056b7510a6ef1b0237e08893024406e1b5379f442c1c2e43365f.
passed · 2026-11-10T01:49:20.130Z · anthropic/claude-opus-5.5
Publication: a480e1a2-d06b-4edb-8764-fc9a909daafa
attribution — pass: The reading attributes the framework, results and described protocol to Sachin Gupta, the sole listed author. Reported results are kept as the paper's claims and are not presented as independently verified.
Authors:Sachin Gupta
summary — pass: The reading leads with the actor and the development. It explains in plain language what the checks test: allowed tool choice, answers grounded in tool output, and required workflows. It keeps the author's own scoping, that the results reflect suite coverage rather than production accuracy. Technical terms such as JUnit 5, deterministic replay and regression suite are kept exactly, suitable for tooltips. The question follows directly from the protocol the paper describes but whose outcomes it does not report. Omitting the ten non-trivial controls is minor.
These results establish coverage of the authored regression suite, not production accuracy.
date — pass: The originalPublished value of 2026-11-04 matches the arXiv v1 submission date. The feed entry on November 5 is a later announcement.
[Submitted on 4 Nov 2026]
support — pass: Each claim matches a verbatim excerpt from the abstract. The closing interpretation, that production performance remains to be tested, is supported by the paper's own limitation. The reading does not inflate synthetic-suite results into real-world reliability. It makes no unsupported benefit or cost assertions.
On 55 synthetic cases spanning seven enterprise domains, the full pipeline correctly classifies all 30 injected faults and 25 passing controls
media — pass: The selected media is the verified arXiv abstract page, which is the direct original source. No better source media was found.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A language model with many learned numerical settings, used to process and generate language.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA language model with many learned numerical settings, used to process and generate language.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv