Collins, Jeremy A., Lee, Seungjae, Wachakorn, Rachanon, Srinivasan, Krishnan, Garg, Animesh, Guizilini, Vitor Campagnolo, Shah, Paarth · Oct 27, 2026
TAPESTRY: A Single Backbone for Video Generation and Robot Control
Markdown · JSON · Original source
The authors of TAPESTRY describe a way to use a video diffusion transformer for robot control without generating video at inference. They represent robot actions as tokens in the video model’s latent space, allowing video and action information to use the same transformer backbone. The model is trained with video diffusion, joint video+action diffusion, and action-only diffusion objectives. At test , it generates action sequences in a single DDIM step, which the authors say produces smooth action chunks at 84 ms per chunk on consumer hardware. They report strong performance in simulation and real-robot experiments, with lower inference cost than comparable joint video–action policies. If the reported performance and speed hold in deployment, this approach could reduce the cost of using video-pretrained models for robot control; the abstract does not establish performance across other tasks or settings. What performance and does TAPESTRY achieve on specific benchmarks and real-robot tasks?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-07T00:01:31.436Z. First observed: 2026-10-28T05:48:36.212Z. Retrieved: 2026-10-29T23:32:48.837Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract does not name the simulation or real-robot tasks, or provide specific performance results. The abstract reports inference speed on consumer hardware but does not identify the hardware or provide a comparison of specific . The reported results do not establish performance across tasks or deployment settings beyond those evaluated.
Open question
What performance and does TAPESTRY achieve on specific and real-robot tasks?
Source-stated
The authors introduce TAPESTRY, which repurposes a pretrained video diffusion transformer as a robotic policy without video generation at inference.
repurposes a pretrained video diffusion transformer into a robotic policy without requiring video generation at inference
Source-stated
TAPESTRY represents robot actions as single-patch tokens in the video model’s latent space and processes them with the same transformer backbone used for video.
treat robot actions as single-patch tokens in the video model's latent space
Source-stated
At test time, the model generates action tokens in one DDIM step; the authors report 84 ms per action chunk on consumer hardware.
generates a compact action token sequence in a single DDIM step from pure noise, producing smooth action chunks at 84 ms per action chunk on consumer hardware
Source-stated
The authors report strong performance across simulation benchmarks and real-robot experiments, with substantially lower inference cost than comparable joint video–action policies.
Across simulation benchmarks and real-robot experiments, TAPESTRY achieves strong performance with substantially lower inference cost than comparable joint video--action policies.
Source and contributor context
Collins, Jeremy A., Lee, Seungjae, Wachakorn, Rachanon, Srinivasan, Krishnan, Garg, Animesh, Guizilini, Vitor Campagnolo, Shah, Paarth · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Pre-training
A model's initial training on a large dataset, before further adaptation for particular uses.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Model inference
Using a trained model to produce a prediction or response to new input.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 0ac2fe74-2ea9-4afd-837c-d5c1cae8a21f.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-07T00:01:31.436Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b1a7660b1759036b51d4507fa026d87300de50a627d7f803d9eb0a0029890637. Source text hash: 78484a3b2d5f4bcfd21c93ae84f031a5f74d692f44e820ba873452e019e64f88.
Revision and source-check history
Version 1 · 2026-11-07T00:01:31.436Z
TAPESTRY: A Single Backbone for Video Generation and Robot Control
The authors of TAPESTRY describe a way to use a video diffusion transformer for robot control without generating video at inference. They represent robot actions as tokens in the video model’s latent space, allowing video and action information to use the same transformer backbone. The model is trained with video diffusion, joint video+action diffusion, and action-only diffusion objectives. At test , it generates action sequences in a single DDIM step, which the authors say produces smooth action chunks at 84 ms per chunk on consumer hardware. They report strong performance in simulation and real-robot experiments, with lower inference cost than comparable joint video–action policies. If the reported performance and speed hold in deployment, this approach could reduce the cost of using video-pretrained models for robot control; the abstract does not establish performance across other tasks or settings. What performance and does TAPESTRY achieve on specific benchmarks and real-robot tasks?
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-07T00:01:31.436Z. First observed: 2026-10-28T05:48:36.212Z. Retrieved: 2026-10-29T23:32:48.837Z.
Limits
The abstract does not name the simulation or real-robot tasks, or provide specific performance results. The abstract reports inference speed on consumer hardware but does not identify the hardware or provide a comparison of specific . The reported results do not establish performance across tasks or deployment settings beyond those evaluated.
Open question
What performance and does TAPESTRY achieve on specific and real-robot tasks?
Source-stated
The authors introduce TAPESTRY, which repurposes a pretrained video diffusion transformer as a robotic policy without video generation at inference.
repurposes a pretrained video diffusion transformer into a robotic policy without requiring video generation at inference
Source-stated
TAPESTRY represents robot actions as single-patch tokens in the video model’s latent space and processes them with the same transformer backbone used for video.
treat robot actions as single-patch tokens in the video model's latent space
Source-stated
At test time, the model generates action tokens in one DDIM step; the authors report 84 ms per action chunk on consumer hardware.
generates a compact action token sequence in a single DDIM step from pure noise, producing smooth action chunks at 84 ms per action chunk on consumer hardware
Source-stated
The authors report strong performance across simulation benchmarks and real-robot experiments, with substantially lower inference cost than comparable joint video–action policies.
Across simulation benchmarks and real-robot experiments, TAPESTRY achieves strong performance with substantially lower inference cost than comparable joint video--action policies.
Source and contributor context
Collins, Jeremy A., Lee, Seungjae, Wachakorn, Rachanon, Srinivasan, Krishnan, Garg, Animesh, Guizilini, Vitor Campagnolo, Shah, Paarth · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Pre-training
A model's initial training on a large dataset, before further adaptation for particular uses.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Model inference
Using a trained model to produce a prediction or response to new input.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 0ac2fe74-2ea9-4afd-837c-d5c1cae8a21f.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-07T00:01:31.436Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: b1a7660b1759036b51d4507fa026d87300de50a627d7f803d9eb0a0029890637. Source text hash: 78484a3b2d5f4bcfd21c93ae84f031a5f74d692f44e820ba873452e019e64f88.
uncertain · 2026-11-07T00:01:31.436Z · anthropic/claude-opus-5.5
Publication: 0ac2fe74-2ea9-4afd-837c-d5c1cae8a21f
attribution — pass: The reading attributes the method, the speed figure and the performance claims to the paper's authors, using phrasing such as 'the authors say' and 'They report'. This matches the arXiv abstract and author list. No claim is presented as Escape Meridian's own finding.
We introduce Tokenizing Actions as Video Patches in a Shared Transformer Policy (TAPESTRY)
summary — uncertain: The mechanism is described faithfully, but the reading leaves out the abstract's motivation, so a non-specialist cannot see why a video model is useful for robots. The abstract says video models encode physical-interaction priors and that existing methods either generate video at test time, which is costly, or train action heads from scratch. Smallest repair: add one attributed sentence stating that motivation. Central terms such as diffusion transformer, latent space, DDIM and inference need term-library tooltips. The question is also repeated verbatim at the end of the summary.
Pretrained video generation models encode rich priors about physical interactions, but leveraging them for robot control typically requires either resource-intensive video generation at test time or action heads trained from scratch
date — pass: The originalPublished value, 2026-10-27, matches the v1 submission date on arXiv. The feed date of 2026-10-29 is later and correctly not used.
[Submitted on 27 Oct 2026]
support — pass: Each claim is backed by an exact abstract excerpt. The cost implication is explicitly conditional: 'If the reported performance and speed hold in deployment'. The limitations correctly note that the benchmarks are not named, the hardware is unspecified and the results are not quantified. The question follows from those gaps and does not lead the reader.
producing smooth action chunks at 84 ms per action chunk on consumer hardware and avoiding the cost of full video generation
media — pass: The selected media is the verified direct link to the original arXiv abstract page, which matches the source URL. No other source media was found.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A model's initial training on a large dataset, before further adaptation for particular uses.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model's initial training on a large dataset, before further adaptation for particular uses.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referenceUsing a trained model to produce a prediction or response to new input.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference