arXiv cs.AI: Learning Noise Optimization for Reward Alignment in Few-Step Generative Models

arXiv cs.AI: Learning Noise Optimization for Reward Alignment in Few-Step Generative Models

Hongeun Kim, Jeongsol Kim and Jong Chul Ye introduce LeNO, a method for aligning few-step generative models with rewards without iterative optimization at inference. Reward alignment for diffusion- and flow-based models is often framed over stochastic trajectories, which the authors say is less suitable for few-step deterministic generators. Their alternative optimizes initial noise, then learns an anchor-conditioned distribution of noise corrections through a residual transport, while keeping the pretrained generator frozen. Local GRPO and scalar rewards train the correction; for Gaussian initial noise with norm-preserving projection, the authors derive an upper bound on KL divergence from the original prior and use it as a regularization objective. They evaluate LeNO on text-to-image and protein structure generation, including tasks with expensive black-box designability rewards. The authors report that one noise correction at inference improves target rewards across both domains while largely preserving generation quality and diversity. The abstract does not give the size of those gains or quantify the changes in quality and diversity, so the reported results cannot be compared across tasks from this evidence alone.

Publication: 4e08189a-62a7-442c-839e-0d9a07ad4ef5

attribution — pass: The draft correctly credits Hongeun Kim, Jeongsol Kim and Jong Chul Ye. Method claims and results are framed as the authors' own statements, not as independently established facts.

Authors:Hongeun Kim, Jeongsol Kim, Jong Chul Ye

summary — uncertain: Central terms are left unexplained for non-specialists: diffusion- and flow-based models, stochastic trajectories, residual transport, anchor-conditioned, Local GRPO, KL divergence and designability. The draft also omits the paper's stated motivation that existing gradient-free noise methods need costly per-sample search at inference, so the practical point is unclear. 'Optimizes initial noise, then learns' suggests two steps; the abstract says LeNO amortizes that optimization into a learned transport. Smallest repair: add plain-language glosses or tooltips, state the costly-search problem, and say the learned correction replaces per-sample optimization.

existing gradient-free methods require costly per-sample search at inference

date — pass: originalPublished is 2026-11-04, which matches the v1 submission date. The later feed date was not used.

[Submitted on 4 Nov 2026]

support — pass: Every claim is backed by abstract text. The limitations correctly note that the abstract gives no quantitative gains. The question follows from that gap and is not leading. No unsupported benefits or costs are implied.

Across both domains, LeNO improves target rewards with a single noise correction at inference while largely preserving generation quality and diversity.

media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source.