# RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines [Canonical reading](https://escapemeridian.com/readings/rm-vlm-from-natural-language-task-descriptions-to-vision-5d8cc507) The authors propose RM-VLM, a method for reinforcement learning in tasks where rewards depend on the order or history of events. Reward machines represent reward functions as automata, making a task’s temporal structure explicit; the authors say these systems have relied on a manually engineered labelling function to connect observations with propositions. RM-VLM uses a large language model to generate a reward machine from a natural-language task description, while a vision-language model grounds its propositions in image-based observations during policy learning. A vision-language model-based value function estimator assesses progress toward transitions and derives a dense reward signal. The authors evaluate the approach in grid-world navigation, robotic manipulation and 3D object collection, reporting higher task success rates than existing vision-language model-based reward methods across all tested domains. The abstract does not give the size of those gains or details of the comparisons, so it leaves open how much the results depend on the tested tasks and methods. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 5, 2026 (2026-11-05). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T21:38:03.089Z. First observed: 2026-11-06T05:06:53.463Z. Retrieved: 2026-11-08T21:33:28.570Z. - [RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines](https://arxiv.org/abs/2611.08308) ## Limits The available source text is the paper’s abstract; it does not provide numerical task-success results or details of the comparison methods. The reported results therefore cannot be assessed in detail from this evidence alone. ## Open question How large were the task-success gains over the comparison methods in each of the three domains? ## Source-stated The authors propose using a large language model to generate a reward machine from a natural-language task description, and a vision-language model to ground its propositions in image-based observations. > a large language model (LLM) generates an RM from a natural-language task description, and a vision-language model (VLM) acts as the labelling function, grounding the RM's propositions over image-based observations - [Supporting source](https://arxiv.org/abs/2611.08308) ## Source-stated The authors say their approach was evaluated in three domains and achieved higher task success rates than existing vision-language model-based reward methods in all tested domains. > We evaluate RM-VLM on three domains covering non-Markovian grid-world navigation, robotic manipulation, and 3D object collection. Our experimental results show that RM-VLM consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains. - [Supporting source](https://arxiv.org/abs/2611.08308) ## Escape Meridian interpretation The abstract does not report the size of the task-success gains or give details of the comparison methods. > consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains. - [Supporting source](https://arxiv.org/abs/2611.08308) ## Source and contributor context Parac, Roko, Ziakas, Christos, Ardon, Leo, Russo, Alessandra · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Reinforcement learning Training by trial and error, in which a system is rewarded for the outcomes its designers want. - [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 0a325fdc-288a-4a07-8ca1-ccd6d00879ff. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-08T21:38:03.089Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 5b3f2745144cf1405c21a15d8096a08d65c1f40d7f854fb04c00138a15814e39. Source text hash: 11a460fcc0cb1f56fb1b5c6820741affc95f048403b4f261534b726c270b4da1. ## Revision and source-check history ### Version 1 — 2026-11-08T21:38:03.089Z The authors propose RM-VLM, a method for reinforcement learning in tasks where rewards depend on the order or history of events. Reward machines represent reward functions as automata, making a task’s temporal structure explicit; the authors say these systems have relied on a manually engineered labelling function to connect observations with propositions. RM-VLM uses a large language model to generate a reward machine from a natural-language task description, while a vision-language model grounds its propositions in image-based observations during policy learning. A vision-language model-based value function estimator assesses progress toward transitions and derives a dense reward signal. The authors evaluate the approach in grid-world navigation, robotic manipulation and 3D object collection, reporting higher task success rates than existing vision-language model-based reward methods across all tested domains. The abstract does not give the size of those gains or details of the comparisons, so it leaves open how much the results depend on the tested tasks and methods. ## Original source arXiv cs.AI · Article Original source date: Nov 5, 2026 (2026-11-05). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T21:38:03.089Z. First observed: 2026-11-06T05:06:53.463Z. Retrieved: 2026-11-08T21:33:28.570Z. - [RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines](https://arxiv.org/abs/2611.08308) ## Limits The available source text is the paper’s abstract; it does not provide numerical task-success results or details of the comparison methods. The reported results therefore cannot be assessed in detail from this evidence alone. ## Open question How large were the task-success gains over the comparison methods in each of the three domains? ## Source-stated The authors propose using a large language model to generate a reward machine from a natural-language task description, and a vision-language model to ground its propositions in image-based observations. > a large language model (LLM) generates an RM from a natural-language task description, and a vision-language model (VLM) acts as the labelling function, grounding the RM's propositions over image-based observations - [Supporting source](https://arxiv.org/abs/2611.08308) ## Source-stated The authors say their approach was evaluated in three domains and achieved higher task success rates than existing vision-language model-based reward methods in all tested domains. > We evaluate RM-VLM on three domains covering non-Markovian grid-world navigation, robotic manipulation, and 3D object collection. Our experimental results show that RM-VLM consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains. - [Supporting source](https://arxiv.org/abs/2611.08308) ## Escape Meridian interpretation The abstract does not report the size of the task-success gains or give details of the comparison methods. > consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains. - [Supporting source](https://arxiv.org/abs/2611.08308) ## Source and contributor context Parac, Roko, Ziakas, Christos, Ardon, Leo, Russo, Alessandra · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Reinforcement learning Training by trial and error, in which a system is rewarded for the outcomes its designers want. - [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 0a325fdc-288a-4a07-8ca1-ccd6d00879ff. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-08T21:38:03.089Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 5b3f2745144cf1405c21a15d8096a08d65c1f40d7f854fb04c00138a15814e39. Source text hash: 11a460fcc0cb1f56fb1b5c6820741affc95f048403b4f261534b726c270b4da1. ### passed — 2026-11-08T21:38:03.089Z — anthropic/claude-opus-5.5 Publication: 0a325fdc-288a-4a07-8ca1-ccd6d00879ff attribution — pass: The reading consistently attributes the method, the labelling-function limitation and the results to the paper's authors (Parac, Ziakas, Ardon, Russo). It presents them as the authors' claims rather than established fact. > Authors:Roko Parac, Christos Ziakas, Leo Ardon, Alessandra Russo summary — pass: The title matches the source. The summary faithfully states the problem: reward machines need a hand-built labelling function for each environment. It then gives the proposed fix (an LLM writes the RM, a VLM labels images), the dense-reward estimator and the three test domains. A non-specialist can follow the central idea. Terms such as automata, propositions and dense reward signal are kept exactly, suited to tooltips. Optional repair: state that the stated aim is wider applicability. The question is a genuine open issue the abstract leaves unanswered. > which must be manually engineered for each environment, limiting their wider applicability date — pass: The originalPublished date of 2026-11-05 matches the arXiv v1 submission date. It is not the later feed date of 2026-11-06. > \[Submitted on 5 Nov 2026\] support — pass: All claims are grounded in the abstract. The reading correctly marks the gap that the abstract gives no effect sizes and no comparison details. It does not stretch the reported success rates into a claim of real-world reliability. > Our experimental results show that RM-VLM consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains. media — pass: The selected media is the verified arXiv abstract page, which is the direct original source. It is the most relevant link available. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/5d8cc5070941d48cff955a319da9a390540aa34da0d91f63315a8f93c4d6e795/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/rm-vlm-from-natural-language-task-descriptions-to-vision-5d8cc507.json)