{"id":"13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa","title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","argument":"The paper’s authors test the reference agent shipped with the Vals Finance Agent Benchmark v2, without tuning it, asking whether its score is stable, failures can be diagnosed, and changes can be trusted. Seven identical runs on 18 questions passed between 105 and 123 of 149 rubric criteria, while the bill varied by up to 17%. In 20 failed trajectories, the authors assigned responsibility to the layer whose owner would remedy each failure: 13 to the agent’s instructions, 3 to its tools, 4 to the model, and none to its data. Four blind readers agreed with the authors’ LLM analyzer in 94% of cases, close to their 93% agreement with one another. An optimization loop kept a targeted change only when it cleared a fixed threshold. Under a cost objective, the final configuration cost 0.34x the baseline on held-out questions, with accuracy within noise. Under an accuracy objective, it gained 14 points on development questions and 1.5 on held-out questions, which the authors judge not to be a real gain against measured run-to-run noise. They emphasize that this is a case study of one agent and nine test questions, not a general result.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The authors describe this as a case study of one agent and nine test questions, so the evidence does not establish that the results generalize to other agents or tests. The evidence is the arXiv abstract; detailed methods and results beyond what it reports are not available here.","speaker":{"name":"Galeone, Cosimo, Ettorre, Anna, Mariani, Ginevra, Park, Minsu, Ettorre, Giuseppe, Ligorio, Daniele","roleAtSource":"Article author"},"source":{"title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.17939","publishedAt":"2026-11-13","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors tested the untuned reference agent from the Vals Finance Agent Benchmark v2 across repeated runs and found variation in both rubric performance and cost.","excerpt":"Seven identical runs on 18 questions pass between 105 and 123 of 149 rubric criteria, and the bill varies by up to 17%.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"The authors classified failures by the layer whose owner would remedy them; most were assigned to the agent’s instructions, with none assigned to its data.","excerpt":"13 belong to the agent's instructions, 3 to its tools, 4 to the model, none to its data.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"Blind readers given traces agreed with the authors’ LLM analyzer at a rate similar to the readers’ agreement with one another.","excerpt":"name the same fault as our LLM analyzer in 94% of cases, the rate at which they agree with one another (93%).","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"The authors report that a cost-focused optimization reduced cost on held-out questions while accuracy stayed within noise; an accuracy-focused optimization did not yield a meaningful held-out gain under their prewritten rule.","excerpt":"Under a cost objective its final configuration costs 0.34x the baseline on the held-out questions (95% CI [0.22, 0.58]) at accuracy within noise.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"The authors caution that their findings are limited to one agent and nine test questions.","excerpt":"This is a case study of one agent and nine test questions, not a general result.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.17939","checkedAt":"2026-11-16T07:48:11.289Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would the same pattern of run-to-run variation, failure ownership, and optimization results appear with other agents or test questions?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","summary":"The paper’s authors test the reference agent shipped with the Vals Finance Agent Benchmark v2, without tuning it, asking whether its score is stable, failures can be diagnosed, and changes can be trusted. Seven identical runs on 18 questions passed between 105 and 123 of 149 rubric criteria, while the bill varied by up to 17%. In 20 failed trajectories, the authors assigned responsibility to the layer whose owner would remedy each failure: 13 to the agent’s instructions, 3 to its tools, 4 to the model, and none to its data. Four blind readers agreed with the authors’ LLM analyzer in 94% of cases, close to their 93% agreement with one another. An optimization loop kept a targeted change only when it cleared a fixed threshold. Under a cost objective, the final configuration cost 0.34x the baseline on held-out questions, with accuracy within noise. Under an accuracy objective, it gained 14 points on development questions and 1.5 on held-out questions, which the authors judge not to be a real gain against measured run-to-run noise. They emphasize that this is a case study of one agent and nine test questions, not a general result.","question":"Would the same pattern of run-to-run variation, failure ownership, and optimization results appear with other agents or test questions?","claims":[{"text":"The authors tested the untuned reference agent from the Vals Finance Agent Benchmark v2 across repeated runs and found variation in both rubric performance and cost.","excerpt":"Seven identical runs on 18 questions pass between 105 and 123 of 149 rubric criteria, and the bill varies by up to 17%.","support":"source-stated"},{"text":"The authors classified failures by the layer whose owner would remedy them; most were assigned to the agent’s instructions, with none assigned to its data.","excerpt":"13 belong to the agent's instructions, 3 to its tools, 4 to the model, none to its data.","support":"source-stated"},{"text":"Blind readers given traces agreed with the authors’ LLM analyzer at a rate similar to the readers’ agreement with one another.","excerpt":"name the same fault as our LLM analyzer in 94% of cases, the rate at which they agree with one another (93%).","support":"source-stated"},{"text":"The authors report that a cost-focused optimization reduced cost on held-out questions while accuracy stayed within noise; an accuracy-focused optimization did not yield a meaningful held-out gain under their prewritten rule.","excerpt":"Under a cost objective its final configuration costs 0.34x the baseline on the held-out questions (95% CI [0.22, 0.58]) at accuracy within noise.","support":"source-stated"},{"text":"The authors caution that their findings are limited to one agent and nine test questions.","excerpt":"This is a case study of one agent and nine test questions, not a general result.","support":"source-stated"}],"limitations":["The authors describe this as a case study of one agent and nine test questions, so the evidence does not establish that the results generalize to other agents or tests.","The evidence is the arXiv abstract; detailed methods and results beyond what it reports are not available here."],"mediaId":"13cd08575d0b4428a576fe0d716eb8cc73438e0787d577b168b301909e4c8aea","correctionReason":null,"id":"13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa","publicationId":"f237075d-4904-45d5-84ce-31ee7e29ce0b","version":1,"sourceUrl":"https://arxiv.org/abs/2611.17939","sourceTitle":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-13","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-16T05:06:36.888Z","createdAt":"2026-11-16T07:16:19.524Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"13cd08575d0b4428a576fe0d716eb8cc73438e0787d577b168b301909e4c8aea","kind":"article","url":"https://arxiv.org/abs/2611.17939","sourceUrl":"https://arxiv.org/abs/2611.17939","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-16T07:03:20.572Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.17939","checkedAt":"2026-11-16T07:48:11.289Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"13cd08575d0b4428a576fe0d716eb8cc73438e0787d577b168b301909e4c8aea","confidence":0.79,"provider":"typesafe","model":"jev-1.13.0","inputHash":"df8fb5c220ac105ba282878019e0c87132ec789b593120b518cf5f78d23deb8a","decidedAt":"2026-11-16T07:03:22.369Z"},"provenance":{"textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observationId":"fad274f0488444fc0388812c94843bb6db10a7bd10d05408f392e406c97ccc11","retrievedAt":"2026-11-16T07:03:20.313Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"13a759b2-42f2-480e-87e3-775b3e4f9a8e"},"reviewer":{"model":"gpt-6-luna","runId":"dffd44cd-231f-4da9-85ee-5aa6ccfc0d4a"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-16T07:16:19.524Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Galeone, Cosimo","url":null,"kind":"unknown"},{"name":"Ettorre, Anna","url":null,"kind":"unknown"},{"name":"Mariani, Ginevra","url":null,"kind":"unknown"},{"name":"Park, Minsu","url":null,"kind":"unknown"},{"name":"Ettorre, Giuseppe","url":null,"kind":"unknown"},{"name":"Ligorio, Daniele","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Galeone, Cosimo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Galeone, Cosimo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Ettorre, Anna","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ettorre, Anna","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Mariani, Ginevra","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Mariani, Ginevra","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Park, Minsu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Park, Minsu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Ettorre, Giuseppe","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ettorre, Giuseppe","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Ligorio, Daniele","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ligorio, Daniele","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.17939","snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa/source"}},"canonicalUrl":"https://escapemeridian.com/readings/can-you-trust-your-finance-agent-drift-diagnosis-and-auditable-13119145","formats":{"html":"/readings/can-you-trust-your-finance-agent-drift-diagnosis-and-auditable-13119145","markdown":"/readings/can-you-trust-your-finance-agent-drift-diagnosis-and-auditable-13119145.md","json":"/readings/can-you-trust-your-finance-agent-drift-diagnosis-and-auditable-13119145.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa","revision":"f237075d-4904-45d5-84ce-31ee7e29ce0b","revisionKind":"publication-id","title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","url":"https://escapemeridian.com/readings/can-you-trust-your-finance-agent-drift-diagnosis-and-auditable-13119145"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.17939","revision":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","revisionKind":"sha256-source-text","title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","url":"https://arxiv.org/abs/2611.17939"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa","title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","argument":"The paper’s authors test the reference agent shipped with the Vals Finance Agent Benchmark v2, without tuning it, asking whether its score is stable, failures can be diagnosed, and changes can be trusted. Seven identical runs on 18 questions passed between 105 and 123 of 149 rubric criteria, while the bill varied by up to 17%. In 20 failed trajectories, the authors assigned responsibility to the layer whose owner would remedy each failure: 13 to the agent’s instructions, 3 to its tools, 4 to the model, and none to its data. Four blind readers agreed with the authors’ LLM analyzer in 94% of cases, close to their 93% agreement with one another. An optimization loop kept a targeted change only when it cleared a fixed threshold. Under a cost objective, the final configuration cost 0.34x the baseline on held-out questions, with accuracy within noise. Under an accuracy objective, it gained 14 points on development questions and 1.5 on held-out questions, which the authors judge not to be a real gain against measured run-to-run noise. They emphasize that this is a case study of one agent and nine test questions, not a general result.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The authors describe this as a case study of one agent and nine test questions, so the evidence does not establish that the results generalize to other agents or tests. The evidence is the arXiv abstract; detailed methods and results beyond what it reports are not available here.","speaker":{"name":"Galeone, Cosimo, Ettorre, Anna, Mariani, Ginevra, Park, Minsu, Ettorre, Giuseppe, Ligorio, Daniele","roleAtSource":"Article author"},"source":{"title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.17939","publishedAt":"2026-11-13","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors tested the untuned reference agent from the Vals Finance Agent Benchmark v2 across repeated runs and found variation in both rubric performance and cost.","excerpt":"Seven identical runs on 18 questions pass between 105 and 123 of 149 rubric criteria, and the bill varies by up to 17%.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"The authors classified failures by the layer whose owner would remedy them; most were assigned to the agent’s instructions, with none assigned to its data.","excerpt":"13 belong to the agent's instructions, 3 to its tools, 4 to the model, none to its data.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"Blind readers given traces agreed with the authors’ LLM analyzer at a rate similar to the readers’ agreement with one another.","excerpt":"name the same fault as our LLM analyzer in 94% of cases, the rate at which they agree with one another (93%).","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"The authors report that a cost-focused optimization reduced cost on held-out questions while accuracy stayed within noise; an accuracy-focused optimization did not yield a meaningful held-out gain under their prewritten rule.","excerpt":"Under a cost objective its final configuration costs 0.34x the baseline on the held-out questions (95% CI [0.22, 0.58]) at accuracy within noise.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"},{"text":"The authors caution that their findings are limited to one agent and nine test questions.","excerpt":"This is a case study of one agent and nine test questions, not a general result.","url":"https://arxiv.org/abs/2611.17939","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.17939","checkedAt":"2026-11-16T07:48:11.289Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would the same pattern of run-to-run variation, failure ownership, and optimization results appear with other agents or test questions?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","summary":"The paper’s authors test the reference agent shipped with the Vals Finance Agent Benchmark v2, without tuning it, asking whether its score is stable, failures can be diagnosed, and changes can be trusted. Seven identical runs on 18 questions passed between 105 and 123 of 149 rubric criteria, while the bill varied by up to 17%. In 20 failed trajectories, the authors assigned responsibility to the layer whose owner would remedy each failure: 13 to the agent’s instructions, 3 to its tools, 4 to the model, and none to its data. Four blind readers agreed with the authors’ LLM analyzer in 94% of cases, close to their 93% agreement with one another. An optimization loop kept a targeted change only when it cleared a fixed threshold. Under a cost objective, the final configuration cost 0.34x the baseline on held-out questions, with accuracy within noise. Under an accuracy objective, it gained 14 points on development questions and 1.5 on held-out questions, which the authors judge not to be a real gain against measured run-to-run noise. They emphasize that this is a case study of one agent and nine test questions, not a general result.","question":"Would the same pattern of run-to-run variation, failure ownership, and optimization results appear with other agents or test questions?","claims":[{"text":"The authors tested the untuned reference agent from the Vals Finance Agent Benchmark v2 across repeated runs and found variation in both rubric performance and cost.","excerpt":"Seven identical runs on 18 questions pass between 105 and 123 of 149 rubric criteria, and the bill varies by up to 17%.","support":"source-stated"},{"text":"The authors classified failures by the layer whose owner would remedy them; most were assigned to the agent’s instructions, with none assigned to its data.","excerpt":"13 belong to the agent's instructions, 3 to its tools, 4 to the model, none to its data.","support":"source-stated"},{"text":"Blind readers given traces agreed with the authors’ LLM analyzer at a rate similar to the readers’ agreement with one another.","excerpt":"name the same fault as our LLM analyzer in 94% of cases, the rate at which they agree with one another (93%).","support":"source-stated"},{"text":"The authors report that a cost-focused optimization reduced cost on held-out questions while accuracy stayed within noise; an accuracy-focused optimization did not yield a meaningful held-out gain under their prewritten rule.","excerpt":"Under a cost objective its final configuration costs 0.34x the baseline on the held-out questions (95% CI [0.22, 0.58]) at accuracy within noise.","support":"source-stated"},{"text":"The authors caution that their findings are limited to one agent and nine test questions.","excerpt":"This is a case study of one agent and nine test questions, not a general result.","support":"source-stated"}],"limitations":["The authors describe this as a case study of one agent and nine test questions, so the evidence does not establish that the results generalize to other agents or tests.","The evidence is the arXiv abstract; detailed methods and results beyond what it reports are not available here."],"mediaId":"13cd08575d0b4428a576fe0d716eb8cc73438e0787d577b168b301909e4c8aea","correctionReason":null,"id":"13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa","publicationId":"f237075d-4904-45d5-84ce-31ee7e29ce0b","version":1,"sourceUrl":"https://arxiv.org/abs/2611.17939","sourceTitle":"Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-13","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-16T05:06:36.888Z","createdAt":"2026-11-16T07:16:19.524Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"13cd08575d0b4428a576fe0d716eb8cc73438e0787d577b168b301909e4c8aea","kind":"article","url":"https://arxiv.org/abs/2611.17939","sourceUrl":"https://arxiv.org/abs/2611.17939","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-16T07:03:20.572Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.17939","checkedAt":"2026-11-16T07:48:11.289Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"13cd08575d0b4428a576fe0d716eb8cc73438e0787d577b168b301909e4c8aea","confidence":0.79,"provider":"typesafe","model":"jev-1.13.0","inputHash":"df8fb5c220ac105ba282878019e0c87132ec789b593120b518cf5f78d23deb8a","decidedAt":"2026-11-16T07:03:22.369Z"},"provenance":{"textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observationId":"fad274f0488444fc0388812c94843bb6db10a7bd10d05408f392e406c97ccc11","retrievedAt":"2026-11-16T07:03:20.313Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"13a759b2-42f2-480e-87e3-775b3e4f9a8e"},"reviewer":{"model":"gpt-6-luna","runId":"dffd44cd-231f-4da9-85ee-5aa6ccfc0d4a"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-16T07:16:19.524Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Galeone, Cosimo","url":null,"kind":"unknown"},{"name":"Ettorre, Anna","url":null,"kind":"unknown"},{"name":"Mariani, Ginevra","url":null,"kind":"unknown"},{"name":"Park, Minsu","url":null,"kind":"unknown"},{"name":"Ettorre, Giuseppe","url":null,"kind":"unknown"},{"name":"Ligorio, Daniele","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Galeone, Cosimo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Galeone, Cosimo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Ettorre, Anna","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ettorre, Anna","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Mariani, Ginevra","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Mariani, Ginevra","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Park, Minsu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Park, Minsu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Ettorre, Giuseppe","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ettorre, Giuseppe","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}},{"actor":{"name":"Ligorio, Daniele","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ligorio, Daniele","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.17939","locator":"head/meta[name=author|citation_author]"},"assertionId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-16T07:48:11.289Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","sourceUrl":"https://arxiv.org/abs/2611.17939","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.17939","snapshotId":"f1b71108ecf6c9c020e4d16361be1c71339e18868d14310153873c4264982cd1","contentSha256":"5ea8a72e65336c51938af39edda14667084efad8c97b77d287da26c493f04ee9","textHash":"59f25a1983404a763b42905337355611d1af35c9d9a7705186c4a4cf6d6d409d","observedAt":"2026-11-16T07:48:11.289Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/13119145e78f2eeb8c10899039bf2167a99264e51138ce052251126c28d7fdaa/source"}}}],"audits":[{"publicationId":"f237075d-4904-45d5-84ce-31ee7e29ce0b","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading consistently attributes the questions, results, and judgments to the paper's authors. It does not present the claims as established facts or as Escape Meridian's opinion.","excerpt":"Can you trust a finance agent you did not build?"},{"dimension":"summary","status":"uncertain","explanation":"Mostly faithful, but two problems affect meaning. First, the summary drops 'from three model families', so readers will likely assume the four blind readers were humans; they appear to be AI models. Smallest repair: 'four blind model-based readers from three model families'. Second, central jargon is unexplained for a general reader: finance agent, reference agent, Vals benchmark, rubric criteria, trajectories, LLM analyzer, and held-out versus development questions. Tooltips for these terms would close the gap. The question is a fair, open issue that follows from the source.","excerpt":"Four blind readers from three model families, given only the traces, name the same fault as our LLM analyzer in 94% of cases"},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-13 matches the arXiv v1 submission date. The later feed date is not used as the publication date.","excerpt":"[Submitted on 13 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim's excerpt is verbatim and matches the abstract: score and cost variation, fault ownership counts, 94% versus 93% agreement, the 0.34x cost result, and no real held-out accuracy gain under the prewritten rule. The case-study limitation is also stated in the source. No unsupported benefits or costs are implied.","excerpt":"Judged against the measured run-to-run noise, under a rule written before the test was opened, the first is a real saving and the second is not."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It is the most direct way for readers to access the source.","excerpt":null}],"createdAt":"2026-11-16T07:16:19.524Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}