{"id":"84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e","title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","argument":"In their paper, Tianyang Luo and coauthors propose MemReward, a graph-based experience memory for using unlabeled rollouts in reinforcement learning (RL) fine-tuning. A rollout is a model’s generated reasoning process and answer; the authors connect queries and rollouts in a heterogeneous graph, then use a graph neural network (GNN) to predict rewards for rollouts without labels. Their approach addresses a specific limitation: when reward labels cover only some rollouts, the others are left out of policy optimization, while existing semi-supervised methods either use fixed similarity weights or ignore relations among queries, reasoning and answers. The authors report evaluations across 13 benchmarks, including tasks with verifiable rewards and evaluations using LLM-judge, human-preference and continuous rewards. With ground-truth rewards on only 20% of rollouts, MemReward reaches 96.6% of fully supervised training performance on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B; the authors also report transfer to Llama backbones up to 8B and near-Oracle results on out-of-domain tasks. These results suggest a way to make more use of limited reward labels during training, but the abstract does not give benchmark-by-benchmark results or enough detail to assess how broadly the reported performance holds.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv page’s abstract; it does not include benchmark-by-benchmark results or details needed to assess the reported performance across tasks. The reported performance figures are the authors’ results; the supplied evidence does not independently verify them.","speaker":{"name":"Luo, Tianyang, Feng, Tao, Hua, Zhigang, Xie, Yan, Yang, Shuang, Liu, Ge, You, Jiaxuan","roleAtSource":"Article author"},"source":{"title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2603.19310","publishedAt":"2026-03-13","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose MemReward, which uses a graph-based experience memory to propagate rewards to unlabeled rollouts during online policy optimization.","excerpt":"We propose MemReward, a graph-based experience memory that propagates rewards to unlabeled rollouts during online policy optimization.","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"},{"text":"The graph connects queries and rollouts, and a GNN trained on labeled rollouts predicts rewards for unlabeled ones.","excerpt":"warms up a GNN on labeled rollouts to propagate rewards over this graph.","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"},{"text":"The authors report evaluation on 13 benchmarks spanning mathematics, question answering, and code generation, as well as additional reward settings.","excerpt":"We evaluate MemReward on 13 benchmarks in mathematics, question answering, and code generation with verifiable rewards","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"},{"text":"With ground-truth rewards on 20% of rollouts, the authors report 96.6% and 97.3% of fully supervised training performance on Qwen2.5-1.5B and Qwen2.5-3B, respectively.","excerpt":"With ground-truth rewards on only 20% of rollouts, MemReward achieves 96.6% of the performance of fully supervised training (Oracle) on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2603.19310","checkedAt":"2026-11-09T13:50:10.946Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do MemReward’s results vary across the 13 benchmarks, particularly on out-of-domain tasks?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","summary":"In their paper, Tianyang Luo and coauthors propose MemReward, a graph-based experience memory for using unlabeled rollouts in reinforcement learning (RL) fine-tuning. A rollout is a model’s generated reasoning process and answer; the authors connect queries and rollouts in a heterogeneous graph, then use a graph neural network (GNN) to predict rewards for rollouts without labels. Their approach addresses a specific limitation: when reward labels cover only some rollouts, the others are left out of policy optimization, while existing semi-supervised methods either use fixed similarity weights or ignore relations among queries, reasoning and answers. The authors report evaluations across 13 benchmarks, including tasks with verifiable rewards and evaluations using LLM-judge, human-preference and continuous rewards. With ground-truth rewards on only 20% of rollouts, MemReward reaches 96.6% of fully supervised training performance on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B; the authors also report transfer to Llama backbones up to 8B and near-Oracle results on out-of-domain tasks. These results suggest a way to make more use of limited reward labels during training, but the abstract does not give benchmark-by-benchmark results or enough detail to assess how broadly the reported performance holds.","question":"How do MemReward’s results vary across the 13 benchmarks, particularly on out-of-domain tasks?","claims":[{"text":"The authors propose MemReward, which uses a graph-based experience memory to propagate rewards to unlabeled rollouts during online policy optimization.","excerpt":"We propose MemReward, a graph-based experience memory that propagates rewards to unlabeled rollouts during online policy optimization.","support":"source-stated"},{"text":"The graph connects queries and rollouts, and a GNN trained on labeled rollouts predicts rewards for unlabeled ones.","excerpt":"warms up a GNN on labeled rollouts to propagate rewards over this graph.","support":"source-stated"},{"text":"The authors report evaluation on 13 benchmarks spanning mathematics, question answering, and code generation, as well as additional reward settings.","excerpt":"We evaluate MemReward on 13 benchmarks in mathematics, question answering, and code generation with verifiable rewards","support":"source-stated"},{"text":"With ground-truth rewards on 20% of rollouts, the authors report 96.6% and 97.3% of fully supervised training performance on Qwen2.5-1.5B and Qwen2.5-3B, respectively.","excerpt":"With ground-truth rewards on only 20% of rollouts, MemReward achieves 96.6% of the performance of fully supervised training (Oracle) on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv page’s abstract; it does not include benchmark-by-benchmark results or details needed to assess the reported performance across tasks.","The reported performance figures are the authors’ results; the supplied evidence does not independently verify them."],"mediaId":"64f4f258160aac730105071bdc7e4d97312663bf296c5e1b8891d7a9028f7e3b","correctionReason":null,"id":"84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e","publicationId":"9e22405f-142b-40b7-8987-b1cc283750b5","version":1,"sourceUrl":"https://arxiv.org/abs/2603.19310","sourceTitle":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-03-13","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:00:36.390Z","createdAt":"2026-11-09T12:46:42.118Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"64f4f258160aac730105071bdc7e4d97312663bf296c5e1b8891d7a9028f7e3b","kind":"article","url":"https://arxiv.org/abs/2603.19310","sourceUrl":"https://arxiv.org/abs/2603.19310","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T12:40:11.233Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2603.19310","checkedAt":"2026-11-09T13:50:10.946Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"64f4f258160aac730105071bdc7e4d97312663bf296c5e1b8891d7a9028f7e3b","confidence":0.81,"provider":"typesafe","model":"jev-1.13.0","inputHash":"348b75fccc2d590ad2b9d3b8bc8a653bcfb59bb2e6590616b19e8e3ad067658f","decidedAt":"2026-11-09T12:40:11.937Z"},"provenance":{"textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observationId":"3f54e9dac1e8715f98dce20e4f5ed436772d60dd09d02bd0ae42e2fba2be8128","retrievedAt":"2026-11-09T12:40:11.144Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"fb934db8-2e6a-4679-8c86-1782afa6a4b3"},"reviewer":{"model":"gpt-6-luna","runId":"28ca80b7-73c0-4b93-84dd-504896a212b2"}},"audit":{"status":"passed","reviewedAt":"2026-11-09T12:46:42.118Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Luo, Tianyang","url":null,"kind":"unknown"},{"name":"Feng, Tao","url":null,"kind":"unknown"},{"name":"Hua, Zhigang","url":null,"kind":"unknown"},{"name":"Xie, Yan","url":null,"kind":"unknown"},{"name":"Yang, Shuang","url":null,"kind":"unknown"},{"name":"Liu, Ge","url":null,"kind":"unknown"},{"name":"You, Jiaxuan","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Luo, Tianyang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luo, Tianyang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Feng, Tao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Feng, Tao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Hua, Zhigang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hua, Zhigang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Xie, Yan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xie, Yan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Yang, Shuang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Shuang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Liu, Ge","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Ge","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"You, Jiaxuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"You, Jiaxuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2603.19310","snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z","basis":"publisher-page-metadata","sourceLinks":52,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e/source"}},"canonicalUrl":"https://escapemeridian.com/readings/memreward-graph-based-experience-memory-for-llm-reward-84f2ac4f","formats":{"html":"/readings/memreward-graph-based-experience-memory-for-llm-reward-84f2ac4f","markdown":"/readings/memreward-graph-based-experience-memory-for-llm-reward-84f2ac4f.md","json":"/readings/memreward-graph-based-experience-memory-for-llm-reward-84f2ac4f.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-reinforcement-learning","name":"Reinforcement learning","explanation":"Training by trial and error, in which a system is rewarded for the outcomes its designers want.","referenceUrl":"https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl"},{"id":"term-fine-tuning","name":"Fine-tuning","explanation":"Further training of an existing model to adapt it to a particular task or use.","referenceUrl":"https://developers.google.com/machine-learning/glossary#fine-tuning"},{"id":"term-neural-network","name":"Neural network","explanation":"A system of connected simple calculations, loosely inspired by the brain, that learns patterns from data.","referenceUrl":"https://developers.google.com/machine-learning/glossary#neural-network"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e","revision":"9e22405f-142b-40b7-8987-b1cc283750b5","revisionKind":"publication-id","title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","url":"https://escapemeridian.com/readings/memreward-graph-based-experience-memory-for-llm-reward-84f2ac4f"},"source":{"type":"source","id":"https://arxiv.org/abs/2603.19310","revision":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","revisionKind":"sha256-source-text","title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","url":"https://arxiv.org/abs/2603.19310"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e","title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","argument":"In their paper, Tianyang Luo and coauthors propose MemReward, a graph-based experience memory for using unlabeled rollouts in reinforcement learning (RL) fine-tuning. A rollout is a model’s generated reasoning process and answer; the authors connect queries and rollouts in a heterogeneous graph, then use a graph neural network (GNN) to predict rewards for rollouts without labels. Their approach addresses a specific limitation: when reward labels cover only some rollouts, the others are left out of policy optimization, while existing semi-supervised methods either use fixed similarity weights or ignore relations among queries, reasoning and answers. The authors report evaluations across 13 benchmarks, including tasks with verifiable rewards and evaluations using LLM-judge, human-preference and continuous rewards. With ground-truth rewards on only 20% of rollouts, MemReward reaches 96.6% of fully supervised training performance on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B; the authors also report transfer to Llama backbones up to 8B and near-Oracle results on out-of-domain tasks. These results suggest a way to make more use of limited reward labels during training, but the abstract does not give benchmark-by-benchmark results or enough detail to assess how broadly the reported performance holds.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv page’s abstract; it does not include benchmark-by-benchmark results or details needed to assess the reported performance across tasks. The reported performance figures are the authors’ results; the supplied evidence does not independently verify them.","speaker":{"name":"Luo, Tianyang, Feng, Tao, Hua, Zhigang, Xie, Yan, Yang, Shuang, Liu, Ge, You, Jiaxuan","roleAtSource":"Article author"},"source":{"title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2603.19310","publishedAt":"2026-03-13","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose MemReward, which uses a graph-based experience memory to propagate rewards to unlabeled rollouts during online policy optimization.","excerpt":"We propose MemReward, a graph-based experience memory that propagates rewards to unlabeled rollouts during online policy optimization.","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"},{"text":"The graph connects queries and rollouts, and a GNN trained on labeled rollouts predicts rewards for unlabeled ones.","excerpt":"warms up a GNN on labeled rollouts to propagate rewards over this graph.","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"},{"text":"The authors report evaluation on 13 benchmarks spanning mathematics, question answering, and code generation, as well as additional reward settings.","excerpt":"We evaluate MemReward on 13 benchmarks in mathematics, question answering, and code generation with verifiable rewards","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"},{"text":"With ground-truth rewards on 20% of rollouts, the authors report 96.6% and 97.3% of fully supervised training performance on Qwen2.5-1.5B and Qwen2.5-3B, respectively.","excerpt":"With ground-truth rewards on only 20% of rollouts, MemReward achieves 96.6% of the performance of fully supervised training (Oracle) on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B","url":"https://arxiv.org/abs/2603.19310","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2603.19310","checkedAt":"2026-11-09T13:50:10.946Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do MemReward’s results vary across the 13 benchmarks, particularly on out-of-domain tasks?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","summary":"In their paper, Tianyang Luo and coauthors propose MemReward, a graph-based experience memory for using unlabeled rollouts in reinforcement learning (RL) fine-tuning. A rollout is a model’s generated reasoning process and answer; the authors connect queries and rollouts in a heterogeneous graph, then use a graph neural network (GNN) to predict rewards for rollouts without labels. Their approach addresses a specific limitation: when reward labels cover only some rollouts, the others are left out of policy optimization, while existing semi-supervised methods either use fixed similarity weights or ignore relations among queries, reasoning and answers. The authors report evaluations across 13 benchmarks, including tasks with verifiable rewards and evaluations using LLM-judge, human-preference and continuous rewards. With ground-truth rewards on only 20% of rollouts, MemReward reaches 96.6% of fully supervised training performance on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B; the authors also report transfer to Llama backbones up to 8B and near-Oracle results on out-of-domain tasks. These results suggest a way to make more use of limited reward labels during training, but the abstract does not give benchmark-by-benchmark results or enough detail to assess how broadly the reported performance holds.","question":"How do MemReward’s results vary across the 13 benchmarks, particularly on out-of-domain tasks?","claims":[{"text":"The authors propose MemReward, which uses a graph-based experience memory to propagate rewards to unlabeled rollouts during online policy optimization.","excerpt":"We propose MemReward, a graph-based experience memory that propagates rewards to unlabeled rollouts during online policy optimization.","support":"source-stated"},{"text":"The graph connects queries and rollouts, and a GNN trained on labeled rollouts predicts rewards for unlabeled ones.","excerpt":"warms up a GNN on labeled rollouts to propagate rewards over this graph.","support":"source-stated"},{"text":"The authors report evaluation on 13 benchmarks spanning mathematics, question answering, and code generation, as well as additional reward settings.","excerpt":"We evaluate MemReward on 13 benchmarks in mathematics, question answering, and code generation with verifiable rewards","support":"source-stated"},{"text":"With ground-truth rewards on 20% of rollouts, the authors report 96.6% and 97.3% of fully supervised training performance on Qwen2.5-1.5B and Qwen2.5-3B, respectively.","excerpt":"With ground-truth rewards on only 20% of rollouts, MemReward achieves 96.6% of the performance of fully supervised training (Oracle) on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv page’s abstract; it does not include benchmark-by-benchmark results or details needed to assess the reported performance across tasks.","The reported performance figures are the authors’ results; the supplied evidence does not independently verify them."],"mediaId":"64f4f258160aac730105071bdc7e4d97312663bf296c5e1b8891d7a9028f7e3b","correctionReason":null,"id":"84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e","publicationId":"9e22405f-142b-40b7-8987-b1cc283750b5","version":1,"sourceUrl":"https://arxiv.org/abs/2603.19310","sourceTitle":"MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-03-13","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:00:36.390Z","createdAt":"2026-11-09T12:46:42.118Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"64f4f258160aac730105071bdc7e4d97312663bf296c5e1b8891d7a9028f7e3b","kind":"article","url":"https://arxiv.org/abs/2603.19310","sourceUrl":"https://arxiv.org/abs/2603.19310","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T12:40:11.233Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2603.19310","checkedAt":"2026-11-09T13:50:10.946Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"64f4f258160aac730105071bdc7e4d97312663bf296c5e1b8891d7a9028f7e3b","confidence":0.81,"provider":"typesafe","model":"jev-1.13.0","inputHash":"348b75fccc2d590ad2b9d3b8bc8a653bcfb59bb2e6590616b19e8e3ad067658f","decidedAt":"2026-11-09T12:40:11.937Z"},"provenance":{"textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observationId":"3f54e9dac1e8715f98dce20e4f5ed436772d60dd09d02bd0ae42e2fba2be8128","retrievedAt":"2026-11-09T12:40:11.144Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"fb934db8-2e6a-4679-8c86-1782afa6a4b3"},"reviewer":{"model":"gpt-6-luna","runId":"28ca80b7-73c0-4b93-84dd-504896a212b2"}},"audit":{"status":"passed","reviewedAt":"2026-11-09T12:46:42.118Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Luo, Tianyang","url":null,"kind":"unknown"},{"name":"Feng, Tao","url":null,"kind":"unknown"},{"name":"Hua, Zhigang","url":null,"kind":"unknown"},{"name":"Xie, Yan","url":null,"kind":"unknown"},{"name":"Yang, Shuang","url":null,"kind":"unknown"},{"name":"Liu, Ge","url":null,"kind":"unknown"},{"name":"You, Jiaxuan","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Luo, Tianyang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luo, Tianyang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Feng, Tao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Feng, Tao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Hua, Zhigang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hua, Zhigang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Xie, Yan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xie, Yan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Yang, Shuang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Shuang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"Liu, Ge","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Ge","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}},{"actor":{"name":"You, Jiaxuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"You, Jiaxuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2603.19310","locator":"head/meta[name=author|citation_author]"},"assertionId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T13:50:10.946Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","sourceUrl":"https://arxiv.org/abs/2603.19310","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2603.19310","snapshotId":"36ba71d8843a655ce52f9ec7cd0a7ad89a1e28ce8e9d70a22040ba20d5ca8c0e","contentSha256":"34834b76e3cdaf902abdc6eacf3933244b061bb997621d3e9258ed10a0fea5a4","textHash":"9361f1d4f3a0d0ab834296f9542019ae9d833af73913de7c1f0a00ef4d683334","observedAt":"2026-11-09T13:50:10.946Z","basis":"publisher-page-metadata","sourceLinks":52,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/84f2ac4f0077b086ffa521a4da12a8a86a6af5d512853a5518a8dc413bcf2c5e/source"}}}],"audits":[{"publicationId":"9e22405f-142b-40b7-8987-b1cc283750b5","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes MemReward, its method and its reported results to Tianyang Luo and coauthors. This matches the arXiv author list. Performance figures are framed as the authors' reports rather than established facts.","excerpt":"Authors:Tianyang Luo, Tao Feng, Zhigang Hua, Yan Xie, Shuang Yang, Ge Liu, Jiaxuan You"},{"dimension":"summary","status":"pass","explanation":"The reading accurately conveys the problem, the method and the headline results, and explains rollout, RL fine-tuning and GNN in plain language. Two small repairs would help readers: 'near-Oracle' should be tied to the earlier 'fully supervised training', which the source calls Oracle. The 13 benchmarks are the verifiable-reward tasks, and the LLM-judge, human-preference and continuous rewards are further evaluations, not part of the 13. The omitted comparison against label propagation and FixMatch-style self-training is a minor loss. The question follows from the evidence.","excerpt":"We evaluate MemReward on 13 benchmarks in mathematics, question answering, and code generation with verifiable rewards, and further with LLM-judge, human-preference, and continuous rewards."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-03-13, which matches the v1 submission date. It is not the later v5 revision or the feed date.","excerpt":"[Submitted on 13 Mar 2026 (v1), last revised 4 Nov 2026 (this version, v5)]"},{"dimension":"support","status":"pass","explanation":"Each claim matches the abstract verbatim. The closing inference is hedged with 'suggest', and it notes that the abstract lacks benchmark-level detail. The limitations correctly state that the results are unverified author reports. No unsupported benefits or costs are introduced.","excerpt":"With ground-truth rewards on only 20% of rollouts, MemReward achieves 96.6% of the performance of fully supervised training (Oracle) on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page. It is the most direct way to access the source.","excerpt":null}],"createdAt":"2026-11-09T12:46:42.118Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}