MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels

MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels

In their paper, Tianyang Luo and coauthors propose MemReward, a graph-based experience memory for using unlabeled rollouts in reinforcement learning (RL) fine-tuning. A rollout is a model’s generated reasoning process and answer; the authors connect queries and rollouts in a heterogeneous graph, then use a graph neural network (GNN) to predict rewards for rollouts without labels. Their approach addresses a specific limitation: when reward labels cover only some rollouts, the others are left out of policy optimization, while existing semi-supervised methods either use fixed similarity weights or ignore relations among queries, reasoning and answers. The authors report evaluations across 13 benchmarks, including tasks with verifiable rewards and evaluations using LLM-judge, human-preference and continuous rewards. With ground-truth rewards on only 20% of rollouts, MemReward reaches 96.6% of fully supervised training performance on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B; the authors also report transfer to Llama backbones up to 8B and near-Oracle results on out-of-domain tasks. These results suggest a way to make more use of limited reward labels during training, but the abstract does not give benchmark-by-benchmark results or enough detail to assess how broadly the reported performance holds.

Publication: 9e22405f-142b-40b7-8987-b1cc283750b5

attribution — pass: The reading attributes MemReward, its method and its reported results to Tianyang Luo and coauthors. This matches the arXiv author list. Performance figures are framed as the authors' reports rather than established facts.

Authors:Tianyang Luo, Tao Feng, Zhigang Hua, Yan Xie, Shuang Yang, Ge Liu, Jiaxuan You

summary — pass: The reading accurately conveys the problem, the method and the headline results, and explains rollout, RL fine-tuning and GNN in plain language. Two small repairs would help readers: 'near-Oracle' should be tied to the earlier 'fully supervised training', which the source calls Oracle. The 13 benchmarks are the verifiable-reward tasks, and the LLM-judge, human-preference and continuous rewards are further evaluations, not part of the 13. The omitted comparison against label propagation and FixMatch-style self-training is a minor loss. The question follows from the evidence.

We evaluate MemReward on 13 benchmarks in mathematics, question answering, and code generation with verifiable rewards, and further with LLM-judge, human-preference, and continuous rewards.

date — pass: originalPublished is 2026-03-13, which matches the v1 submission date. It is not the later v5 revision or the feed date.

[Submitted on 13 Mar 2026 (v1), last revised 4 Nov 2026 (this version, v5)]

support — pass: Each claim matches the abstract verbatim. The closing inference is hedged with 'suggest', and it notes that the abstract lacks benchmark-level detail. The limitations correctly state that the results are unverified author reports. No unsupported benefits or costs are introduced.

With ground-truth rewards on only 20% of rollouts, MemReward achieves 96.6% of the performance of fully supervised training (Oracle) on Qwen2.5-1.5B and 97.3% on Qwen2.5-3B

media — pass: The selected media is the verified direct link to the original arXiv abstract page. It is the most direct way to access the source.