# arXiv eess.SY: Metamorphic testing of learned AC optimal power flow surrogates [Canonical reading](https://escapemeridian.com/readings/arxiv-eess-sy-metamorphic-testing-of-learned-ac-optimal-power-2daea315) Authors Stephen Peng and Vineet Jagadeesan Nair argue that testing a learned AC optimal power flow (AC-OPF) surrogate on one grid serialization cannot show whether its accuracy depends on that encoding. They test 17 model and pipeline configurations against 13 metamorphic relations—rules for when transformations are valid and how outputs should compare. Every configuration violates at least six relations, even with reporting tolerances ten times tighter or looser. For GridSFM-Open v1.1, their inspection and interventions point to an indexing bug and three noncanonical or numerically unstable preprocessing choices. A repaired checkpoint remains untested. In a separate case500 training experiment, augmentation improves some measures, but effects depend on the training protocol, some relations worsen, and every evaluated prediction remains AC infeasible. The authors attribute most of the large raw-Laplacian-baseline gain to a GPU/CPU eigensolver mismatch. They recommend reporting representation consistency alongside accuracy, feasibility, and cost, while cautioning that the results establish neither deployment readiness nor universal invariance. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv eess.SY · Article Original source date: Nov 10, 2026 (2026-11-10). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T01:01:11.246Z. First observed: 2026-11-11T05:01:33.075Z. Retrieved: 2026-11-12T00:48:51.962Z. - [Metamorphic testing of learned AC optimal power flow surrogates reveals representation dependence](https://arxiv.org/abs/2611.13566) ## Limits The repaired GridSFM-Open v1.1 checkpoint remains untested, so the reported findings do not establish whether those proposed repairs resolve its violations. ## Open question When tested, will the repaired GridSFM-Open v1.1 checkpoint still violate metamorphic relations? ## Source-stated The authors say that testing an AC-OPF surrogate on one grid serialization cannot establish whether its accuracy depends on that encoding. > Evaluating a learned AC optimal power flow (AC-OPF) surrogate on one grid serialization cannot show whether its accuracy depends on that encoding. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated Across 17 model and pipeline configurations, each violated at least six of the 13 tested metamorphic relations, including at wider and narrower reporting tolerances. > Every configuration violates at least six relations, even at reporting tolerances ten times tighter or looser. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated For GridSFM-Open v1.1, the authors’ inspection and interventions point to an indexing bug and three preprocessing choices; they have not tested a repaired checkpoint. > source inspection and intermediate-tensor interventions point to an indexing bug and three noncanonical or numerically unstable preprocessing choices; a repaired checkpoint remains untested. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated In the case500 training experiment, augmentation improved some reported measures, but the benefits depended on training protocol; every evaluated prediction remained AC infeasible. > These benefits depend on the training protocol: some relations worsen, and every evaluated prediction remains AC infeasible. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated The authors recommend reporting representation consistency alongside accuracy, feasibility, and cost, and say the results establish neither deployment readiness nor universal invariance. > We recommend reporting representation consistency alongside accuracy, feasibility, and cost. These results establish neither deployment readiness nor universal invariance. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source and contributor context Peng, Stephen, Nair, Vineet Jagadeesan · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Graphics processing unit A chip that performs many calculations at once; the main hardware used to train and run AI models. - [Definition reference](https://en.wikipedia.org/wiki/Graphics_processing_unit) ## CPU The central processing unit: a general-purpose processor that carries out a computer's instructions. - [Definition reference](https://aws.amazon.com/what-is/cpu/) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: ea80bafd-db1e-4440-8665-0537f2a0a5a7. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T01:01:11.246Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 2e9e0d6512e0b6c26693ab44bd7349518db21ae9ad8eb45cf91be16130c8bb33. Source text hash: 8d28580f19a579250823243c0ab5ae86b089260dd29f386944cd408ebe2ffa2b. ## Revision and source-check history ### Version 1 — 2026-11-12T01:01:11.246Z Authors Stephen Peng and Vineet Jagadeesan Nair argue that testing a learned AC optimal power flow (AC-OPF) surrogate on one grid serialization cannot show whether its accuracy depends on that encoding. They test 17 model and pipeline configurations against 13 metamorphic relations—rules for when transformations are valid and how outputs should compare. Every configuration violates at least six relations, even with reporting tolerances ten times tighter or looser. For GridSFM-Open v1.1, their inspection and interventions point to an indexing bug and three noncanonical or numerically unstable preprocessing choices. A repaired checkpoint remains untested. In a separate case500 training experiment, augmentation improves some measures, but effects depend on the training protocol, some relations worsen, and every evaluated prediction remains AC infeasible. The authors attribute most of the large raw-Laplacian-baseline gain to a GPU/CPU eigensolver mismatch. They recommend reporting representation consistency alongside accuracy, feasibility, and cost, while cautioning that the results establish neither deployment readiness nor universal invariance. ## Original source arXiv eess.SY · Article Original source date: Nov 10, 2026 (2026-11-10). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-12T01:01:11.246Z. First observed: 2026-11-11T05:01:33.075Z. Retrieved: 2026-11-12T00:48:51.962Z. - [Metamorphic testing of learned AC optimal power flow surrogates reveals representation dependence](https://arxiv.org/abs/2611.13566) ## Limits The repaired GridSFM-Open v1.1 checkpoint remains untested, so the reported findings do not establish whether those proposed repairs resolve its violations. ## Open question When tested, will the repaired GridSFM-Open v1.1 checkpoint still violate metamorphic relations? ## Source-stated The authors say that testing an AC-OPF surrogate on one grid serialization cannot establish whether its accuracy depends on that encoding. > Evaluating a learned AC optimal power flow (AC-OPF) surrogate on one grid serialization cannot show whether its accuracy depends on that encoding. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated Across 17 model and pipeline configurations, each violated at least six of the 13 tested metamorphic relations, including at wider and narrower reporting tolerances. > Every configuration violates at least six relations, even at reporting tolerances ten times tighter or looser. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated For GridSFM-Open v1.1, the authors’ inspection and interventions point to an indexing bug and three preprocessing choices; they have not tested a repaired checkpoint. > source inspection and intermediate-tensor interventions point to an indexing bug and three noncanonical or numerically unstable preprocessing choices; a repaired checkpoint remains untested. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated In the case500 training experiment, augmentation improved some reported measures, but the benefits depended on training protocol; every evaluated prediction remained AC infeasible. > These benefits depend on the training protocol: some relations worsen, and every evaluated prediction remains AC infeasible. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source-stated The authors recommend reporting representation consistency alongside accuracy, feasibility, and cost, and say the results establish neither deployment readiness nor universal invariance. > We recommend reporting representation consistency alongside accuracy, feasibility, and cost. These results establish neither deployment readiness nor universal invariance. - [Supporting source](https://arxiv.org/abs/2611.13566) ## Source and contributor context Peng, Stephen, Nair, Vineet Jagadeesan · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Graphics processing unit A chip that performs many calculations at once; the main hardware used to train and run AI models. - [Definition reference](https://en.wikipedia.org/wiki/Graphics_processing_unit) ## CPU The central processing unit: a general-purpose processor that carries out a computer's instructions. - [Definition reference](https://aws.amazon.com/what-is/cpu/) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: ea80bafd-db1e-4440-8665-0537f2a0a5a7. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-12T01:01:11.246Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 2e9e0d6512e0b6c26693ab44bd7349518db21ae9ad8eb45cf91be16130c8bb33. Source text hash: 8d28580f19a579250823243c0ab5ae86b089260dd29f386944cd408ebe2ffa2b. ### uncertain — 2026-11-12T01:01:11.246Z — anthropic/claude-opus-5.5 Publication: ea80bafd-db1e-4440-8665-0537f2a0a5a7 attribution — pass: The reading correctly credits the paper and its findings to Stephen Peng and Vineet Jagadeesan Nair. Claims and recommendations stay attributed to the authors rather than being presented as established fact. > Authors:Stephen Peng, Vineet Jagadeesan Nair summary — uncertain: The summary is faithful but not readable without specialist background. It never explains what AC-OPF is (computing how to run a power grid within physical limits) or what a learned surrogate is (an ML model approximating that solver). Grid serialization, metamorphic relations, checkpoint, case500, augmentation, AC infeasible, raw-Laplacian baseline and eigensolver are also unexplained, and no supported significance is given. It also says 17 configurations failed without noting that only two are released trained checkpoints and 15 are random-weight controls. Smallest repair: add that scope and brief plain-language glosses or term tooltips. > Our audit covers 17 model and pipeline configurations: two released trained checkpoints and 15 random-weight architecture or mechanism controls. date — pass: The originalPublished value of 2026-11-10 matches the arXiv v1 submission date. The later feed date was correctly not used. > \[Submitted on 10 Nov 2026\] support — pass: Each claim matches the abstract, including its stated conditions: tolerance robustness, the untested repaired checkpoint, protocol-dependent benefits, AC infeasibility, and the authors' disclaimer. The question and limitation follow from the source, and the reading invents no benefits or costs. > These results establish neither deployment readiness nor universal invariance. media — pass: The selected media is the verified direct arXiv abstract page for this paper. That is the most direct route to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/2daea3155a42b7d18cc30a267dc34bbd3fd3b41536e686a4b53037e1aa91ff20/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-eess-sy-metamorphic-testing-of-learned-ac-optimal-power-2daea315.json)