Tang, Wenwu, Wang, Dong, Cheng, Yun, Saukh, Olga · Oct 27, 2026
Wenwu Tang and coauthors: A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation
Markdown · JSON · Original source
Authors Wenwu Tang, Dong Wang, Yun Cheng and Olga Saukh argue that distillation can reduce the variety of images produced from different initial noise samples, and that this loss is greatest at the first, high-noise step. They introduce ReSeed, a post-distillation method that adds a seed-response bias adapter to that first step and aligns its velocity with the teacher's first-interval secant velocity. The authors report evaluations on text-to-image and text-to-video models; for a distilled FLUX.1-dev model, they report improved DINO diversity while maintaining competitive visual quality. They say the method leaves student weights and later denoising steps unchanged and uses no target images, perceptual losses, or adversarial losses. The results suggest a possible way to recover output diversity without the full student model, but the supplied evidence is an abstract and does not provide the full evaluation details. How well does ReSeed's reported diversity improvement hold across prompts and evaluation settings beyond those described in the abstract?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-04T23:01:32.637Z. First observed: 2026-10-28T08:46:32.077Z. Retrieved: 2026-10-30T00:46:44.900Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the abstract, not the full paper; it does not provide detailed methods, evaluation settings, or results beyond the abstract's summary.
Open question
How well does ReSeed's reported diversity improvement hold across prompts and evaluation settings beyond those described in the abstract?
Source-stated
The authors report that the largest measured loss of response to different initial noises occurs during the first, high-noise step.
The largest measured reduction in the response to different initial noises occurs during the first, high-noise step.
Source-stated
ReSeed adds a seed-response bias adapter only to the first student step and aligns its velocity with the teacher's first-interval secant velocity.
adds a seed-response bias adapter only to the first student step and aligns its velocity with the teacher's first-interval secant velocity.
Source-stated
The authors evaluated ReSeed on three text-to-image models and one text-to-video model.
We evaluate ReSeed on three text-to-image models (SD3-Medium, SD3.5-Medium, and FLUX.1-dev) and one text-to-video model (Wan2.1-T2V-1.3B).
Source-stated
For a distilled 4-step FLUX.1-dev model, the authors report a 48.3% improvement in DINO diversity while maintaining competitive visual quality.
It improves DINO diversity by $48.3\%$ while maintaining competitive visual quality.
Source-stated
The authors say ReSeed does not change student weights or later denoising steps and does not require target images, perceptual losses, or adversarial losses.
All student weights and later denoising steps remain unchanged.
Source and contributor context
Tang, Wenwu, Wang, Dong, Cheng, Yun, Saukh, Olga · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Diffusion model
A kind of generative AI that makes images, audio or video by starting from random noise and refining it step by step.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b1eb25ca-55f5-4d30-8f6b-1f0511e3fc89.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-04T23:01:32.637Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 4e874bb620841acd39994ef325a11d6751783ff8268d8f5a780f9e811a969882. Source text hash: 8a5e1028ffe844dd8665fcab908989ac0c3fded450b822d368a6f791f47e9682.
Revision and source-check history
Version 1 · 2026-11-04T23:01:32.637Z
Wenwu Tang and coauthors: A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation
Authors Wenwu Tang, Dong Wang, Yun Cheng and Olga Saukh argue that distillation can reduce the variety of images produced from different initial noise samples, and that this loss is greatest at the first, high-noise step. They introduce ReSeed, a post-distillation method that adds a seed-response bias adapter to that first step and aligns its velocity with the teacher's first-interval secant velocity. The authors report evaluations on text-to-image and text-to-video models; for a distilled FLUX.1-dev model, they report improved DINO diversity while maintaining competitive visual quality. They say the method leaves student weights and later denoising steps unchanged and uses no target images, perceptual losses, or adversarial losses. The results suggest a possible way to recover output diversity without the full student model, but the supplied evidence is an abstract and does not provide the full evaluation details. How well does ReSeed's reported diversity improvement hold across prompts and evaluation settings beyond those described in the abstract?
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-04T23:01:32.637Z. First observed: 2026-10-28T08:46:32.077Z. Retrieved: 2026-10-30T00:46:44.900Z.
Limits
The supplied evidence is the abstract, not the full paper; it does not provide detailed methods, evaluation settings, or results beyond the abstract's summary.
Open question
How well does ReSeed's reported diversity improvement hold across prompts and evaluation settings beyond those described in the abstract?
Source-stated
The authors report that the largest measured loss of response to different initial noises occurs during the first, high-noise step.
The largest measured reduction in the response to different initial noises occurs during the first, high-noise step.
Source-stated
ReSeed adds a seed-response bias adapter only to the first student step and aligns its velocity with the teacher's first-interval secant velocity.
adds a seed-response bias adapter only to the first student step and aligns its velocity with the teacher's first-interval secant velocity.
Source-stated
The authors evaluated ReSeed on three text-to-image models and one text-to-video model.
We evaluate ReSeed on three text-to-image models (SD3-Medium, SD3.5-Medium, and FLUX.1-dev) and one text-to-video model (Wan2.1-T2V-1.3B).
Source-stated
For a distilled 4-step FLUX.1-dev model, the authors report a 48.3% improvement in DINO diversity while maintaining competitive visual quality.
It improves DINO diversity by $48.3\%$ while maintaining competitive visual quality.
Source-stated
The authors say ReSeed does not change student weights or later denoising steps and does not require target images, perceptual losses, or adversarial losses.
All student weights and later denoising steps remain unchanged.
Source and contributor context
Tang, Wenwu, Wang, Dong, Cheng, Yun, Saukh, Olga · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Diffusion model
A kind of generative AI that makes images, audio or video by starting from random noise and refining it step by step.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b1eb25ca-55f5-4d30-8f6b-1f0511e3fc89.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-04T23:01:32.637Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 4e874bb620841acd39994ef325a11d6751783ff8268d8f5a780f9e811a969882. Source text hash: 8a5e1028ffe844dd8665fcab908989ac0c3fded450b822d368a6f791f47e9682.
uncertain · 2026-11-04T23:01:32.637Z · anthropic/claude-opus-5.5
Publication: b1eb25ca-55f5-4d30-8f6b-1f0511e3fc89
attribution — pass: The reading correctly credits the paper's authors (Wenwu Tang, Dong Wang, Yun Cheng, Olga Saukh) and frames the findings as their reports, not as established fact.
Authors:Wenwu Tang, Dong Wang, Yun Cheng, Olga Saukh
summary — uncertain: The core terms are never explained in plain language: diffusion model distillation, teacher versus student, initial noise or seed, and DINO diversity. The reading also omits why distillation matters (high-quality images in only a few denoising steps). It drops the key diagnostic evidence: swapping in the teacher's first step restores most diversity. It drops the lightweight scale (0.00196% added parameters, 8,000 prompt-noise pairs). Smallest repair: add one plain sentence on distillation's purpose and the teacher-swap experiment, and add tooltips for these terms. The question is reasonable.
When we replace the first step with the teacher's corresponding trajectory and leave the final three student steps unchanged, most of the output diversity returns.
date — pass: originalPublished 2026-10-27 matches the arXiv v1 submission date. The later feed date was correctly not used.
[Submitted on 27 Oct 2026]
support — pass: Every claim is directly supported by the abstract, including the 48.3% DINO diversity gain for the 4-step FLUX.1-dev model, the unchanged student weights, and the absence of target images or perceptual and adversarial losses. The implied significance is appropriately hedged, and the limitation about abstract-only evidence is accurate.
keeping the distilled student fully frozen and requiring no target images, perceptual losses, or adversarial losses
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A kind of generative AI that makes images, audio or video by starting from random noise and refining it step by step.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA kind of generative AI that makes images, audio or video by starting from random noise and refining it step by step.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv