{"id":"efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a","title":"Ai2 introduces ToyBlocks, a controlled world for robot learning and evaluation","argument":"Ai2 introduces ToyBlocks, a simulated tabletop world where researchers can vary robot-training data, repeat tests and investigate why models succeed or fail. The authors argue that this deliberately narrow setting makes it possible to combine realistic physics and appearance, automatic task and demonstration generation, challenges involving reasoning and dexterity, and broad coverage within one domain.\n\nToyBlocks generates robot demonstrations without requiring a person to remotely control the robot, and lets researchers test performance on tasks kept out of training. Ai2 reports that an agent in one search-and-spell experiment solved tasks or correctly declared them infeasible 63% of the time; errors included confusing similar-looking letters and choosing the wrong letter orientation.\n\nThe authors also report tests on physical robots using models trained only in ToyBlocks simulation. The hierarchical Molmo 2 + ManiFlow system succeeded in 50.0% of real-world episodes and 55.5% of simulated episodes. Ai2 says further trials are needed to establish statistical confidence across tasks. The release includes simulation assets, data, models, training code and evaluation code, giving other researchers materials to investigate robot-learning questions and failure modes in this controlled setting.","topicIds":["intelligence-and-discovery","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"ToyBlocks is a deliberately constrained simulated domain; its results do not by themselves establish performance across open-ended environments. The reported real-world hierarchical-system evaluation covered 30 scene configurations, and Ai2 says further trials are needed to establish statistical confidence across tasks. The source describes tests of action-conditioned world models but the supplied text does not report their comparative results.","speaker":{"name":"Ai2","roleAtSource":"Publisher"},"source":{"title":"ToyBlocks: A controlled world for robot learning and evaluation | Ai2","publisher":"Ai2","format":"Article","url":"https://allenai.org/blog/toyblocks","publishedAt":"2026-11-06T08:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Ai2 presents ToyBlocks as a controlled simulated environment for studying robot learning, with tools to generate tasks and demonstrations and measure behavior.","excerpt":"Each pairs a controlled, simulated world targeted at studying robotics in a specific environment with tools for automatically generating tasks and training demonstrations, as well as metrics for measuring a robot's behavior over time.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"The authors say the narrow domain makes it possible to meet ToyBlocks’ four design requirements, including realistic physics and appearance and broad task coverage within the domain.","excerpt":"ToyBlocks’ narrow scope, on the other hand, makes it possible to fulfill them all.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"ToyBlocks can generate demonstrations automatically, allowing researchers to vary training experiences and evaluate on tasks withheld from training.","excerpt":"ToyBlocks automatically generates demonstrations, which are recordings of a robot completing tasks in simulation, without requiring time-consuming teleoperation","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"In the search-and-spell experiment, Ai2 reports that the agent solved tasks or correctly declared them infeasible 63% of the time.","excerpt":"We found that in this experiment, the agent was able to successfully solve or correctly declare infeasible 63% of the tasks.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"In reported sim-to-real tests, the hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes; Ai2 says further trials are needed for statistical confidence across tasks.","excerpt":"The hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes (111/200) and 50.0% of real-world episodes (15/30); further trials are needed to establish statistical confidence across tasks.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"Ai2 says ToyBlocks provides simulated ground truth for controlled tests of how action-conditioned world models predict physical interactions.","excerpt":"ToyBlocks provides this ground truth – the actual simulated scene state – enabling highly controlled tests of how recently developed world models perform in low-level object manipulation.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"Ai2 is releasing simulation assets, task and demonstration generators, demonstrations, trained models, training code, evaluation code and validation sets.","excerpt":"We’re releasing simulation assets, procedural task and demonstration generators, automatically generated demonstrations, two trained models showcasing what the framework can do","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"}],"sourceMedia":{"status":"found","sourceUrl":"https://allenai.org/blog/toyblocks","checkedAt":"2026-11-08T01:51:12.014Z","items":[{"type":"image","url":"https://allenai.org/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqERuwxeB0K8%2Fmaxresdefault.jpg&w=1200&q=75","sourceUrl":"https://allenai.org/blog/toyblocks","attribution":"Ai2","basis":"publisher-embed","availability":"verified","verifiedAt":"2026-11-08T01:51:13.486Z"}],"note":"Media declared by the original publisher; availability checked separately from the editorial reading."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How accurately do the tested action-conditioned world models predict block motion and skill outcomes, particularly over full trajectories?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Ai2 introduces ToyBlocks, a controlled world for robot learning and evaluation","summary":"Ai2 introduces ToyBlocks, a simulated tabletop world where researchers can vary robot-training data, repeat tests and investigate why models succeed or fail. The authors argue that this deliberately narrow setting makes it possible to combine realistic physics and appearance, automatic task and demonstration generation, challenges involving reasoning and dexterity, and broad coverage within one domain.\n\nToyBlocks generates robot demonstrations without requiring a person to remotely control the robot, and lets researchers test performance on tasks kept out of training. Ai2 reports that an agent in one search-and-spell experiment solved tasks or correctly declared them infeasible 63% of the time; errors included confusing similar-looking letters and choosing the wrong letter orientation.\n\nThe authors also report tests on physical robots using models trained only in ToyBlocks simulation. The hierarchical Molmo 2 + ManiFlow system succeeded in 50.0% of real-world episodes and 55.5% of simulated episodes. Ai2 says further trials are needed to establish statistical confidence across tasks. The release includes simulation assets, data, models, training code and evaluation code, giving other researchers materials to investigate robot-learning questions and failure modes in this controlled setting.","question":"How accurately do the tested action-conditioned world models predict block motion and skill outcomes, particularly over full trajectories?","claims":[{"text":"Ai2 presents ToyBlocks as a controlled simulated environment for studying robot learning, with tools to generate tasks and demonstrations and measure behavior.","excerpt":"Each pairs a controlled, simulated world targeted at studying robotics in a specific environment with tools for automatically generating tasks and training demonstrations, as well as metrics for measuring a robot's behavior over time.","support":"source-stated"},{"text":"The authors say the narrow domain makes it possible to meet ToyBlocks’ four design requirements, including realistic physics and appearance and broad task coverage within the domain.","excerpt":"ToyBlocks’ narrow scope, on the other hand, makes it possible to fulfill them all.","support":"source-stated"},{"text":"ToyBlocks can generate demonstrations automatically, allowing researchers to vary training experiences and evaluate on tasks withheld from training.","excerpt":"ToyBlocks automatically generates demonstrations, which are recordings of a robot completing tasks in simulation, without requiring time-consuming teleoperation","support":"source-stated"},{"text":"In the search-and-spell experiment, Ai2 reports that the agent solved tasks or correctly declared them infeasible 63% of the time.","excerpt":"We found that in this experiment, the agent was able to successfully solve or correctly declare infeasible 63% of the tasks.","support":"source-stated"},{"text":"In reported sim-to-real tests, the hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes; Ai2 says further trials are needed for statistical confidence across tasks.","excerpt":"The hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes (111/200) and 50.0% of real-world episodes (15/30); further trials are needed to establish statistical confidence across tasks.","support":"source-stated"},{"text":"Ai2 says ToyBlocks provides simulated ground truth for controlled tests of how action-conditioned world models predict physical interactions.","excerpt":"ToyBlocks provides this ground truth – the actual simulated scene state – enabling highly controlled tests of how recently developed world models perform in low-level object manipulation.","support":"source-stated"},{"text":"Ai2 is releasing simulation assets, task and demonstration generators, demonstrations, trained models, training code, evaluation code and validation sets.","excerpt":"We’re releasing simulation assets, procedural task and demonstration generators, automatically generated demonstrations, two trained models showcasing what the framework can do","support":"source-stated"}],"limitations":["ToyBlocks is a deliberately constrained simulated domain; its results do not by themselves establish performance across open-ended environments.","The reported real-world hierarchical-system evaluation covered 30 scene configurations, and Ai2 says further trials are needed to establish statistical confidence across tasks.","The source describes tests of action-conditioned world models but the supplied text does not report their comparative results."],"mediaId":"921b78122bbdc803b5bb54292ac0cd957132148010c818f34384f2f04584e3f4","correctionReason":null,"id":"efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a","publicationId":"7b04ca93-2a58-4c09-8b2e-88667a315f42","version":1,"sourceUrl":"https://allenai.org/blog/toyblocks","sourceTitle":"ToyBlocks: A controlled world for robot learning and evaluation | Ai2","publisher":"Ai2","publishedAt":"2026-11-06T08:00:00.000Z","originalPublished":{"value":"2026-11-06T08:00:00.000Z","precision":"instant","basis":"feed_entry"},"firstObservedAt":"2026-11-06T14:03:33.228Z","createdAt":"2026-11-08T02:16:22.134Z","historical":false,"entityIds":["ai2"],"topicIds":["intelligence-and-discovery","work-and-agency"],"media":[{"id":"921b78122bbdc803b5bb54292ac0cd957132148010c818f34384f2f04584e3f4","kind":"article","url":"https://allenai.org/blog/toyblocks","sourceUrl":"https://allenai.org/blog/toyblocks","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-08T01:51:13.486Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."},{"id":"056521bcd399666cdc521931d5bf9f056f3df6ced485157321a16fded42c2a53","kind":"image","url":"https://allenai.org/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqERuwxeB0K8%2Fmaxresdefault.jpg&w=1200&q=75","sourceUrl":"https://allenai.org/blog/toyblocks","label":"View source image","provenance":"publisher_page","verification":{"status":"verified","checkedAt":"2026-11-08T01:51:13.486Z","method":"publisher_association_and_head","contentType":"image/jpeg","cacheUntil":1794102973486},"recommendationReason":"Observed on the acquired publisher source; availability checked."}],"sourceMedia":{"status":"found","sourceUrl":"https://allenai.org/blog/toyblocks","checkedAt":"2026-11-08T01:51:12.014Z","items":[{"type":"image","url":"https://allenai.org/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqERuwxeB0K8%2Fmaxresdefault.jpg&w=1200&q=75","sourceUrl":"https://allenai.org/blog/toyblocks","attribution":"Ai2","basis":"publisher-embed","availability":"verified","verifiedAt":"2026-11-08T01:51:13.486Z"}],"note":"Media declared by the original publisher; availability checked separately from the editorial reading."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"921b78122bbdc803b5bb54292ac0cd957132148010c818f34384f2f04584e3f4","confidence":0.98,"provider":"typesafe","model":"jev-1.13.0","inputHash":"a30a26b0ccf88edeff6473c0cb4fa273bd9a70fd349cd454edf4783754a63854","decidedAt":"2026-11-08T01:51:17.771Z"},"provenance":{"textHash":"c0d8d4c7e20d5bbadaf5bac5935f4a020d5c1d01ef74f81a7388b8b2524501df","observationId":"d8207efd6a64ae1f94f6759638285bf160cac46b981e162c2e6856250f5f446d","retrievedAt":"2026-11-08T01:51:12.014Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"b5398c23-87f0-4ba9-8df5-27f3cf27f269"},"reviewer":{"model":"gpt-6-luna","runId":"1d73549a-724b-44ac-88b4-1ef2a00bde43"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-08T02:16:22.134Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[],"authorRole":"article-author","hosts":[],"status":"missing","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[],"participationCoverage":{"total":0,"returned":0,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://allenai.org/blog/toyblocks","snapshotId":"e7ed423a08642200f87f821c33ac0d6673f2bf1d5f444153508e764e3863da15","contentSha256":"3298c68ed4d6025b111585e2ebba525796ea68ef2351f2528d589823109c042c","textHash":"c0d8d4c7e20d5bbadaf5bac5935f4a020d5c1d01ef74f81a7388b8b2524501df","observedAt":"2026-11-08T01:51:12.014Z","basis":"publisher-page-metadata","sourceLinks":19,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a/source"}},"canonicalUrl":"https://escapemeridian.com/readings/ai2-introduces-toyblocks-a-controlled-world-for-robot-learning-efdba0ba","formats":{"html":"/readings/ai2-introduces-toyblocks-a-controlled-world-for-robot-learning-efdba0ba","markdown":"/readings/ai2-introduces-toyblocks-a-controlled-world-for-robot-learning-efdba0ba.md","json":"/readings/ai2-introduces-toyblocks-a-controlled-world-for-robot-learning-efdba0ba.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-world-model","name":"World model","explanation":"An AI system's internal model of how an environment behaves, used to predict what happens next or to plan.","referenceUrl":"https://arxiv.org/abs/1803.10122"}],"entities":[{"id":"ai2","name":"Ai2","kind":"organization","context":"Ai2 develops AI research tools and models, including the openly documented Olmo family.","referenceUrls":["https://allenai.org/","https://allenai.org/olmo"],"path":"/entities/ai2","json":"/entities/ai2.json","markdown":"/entities/ai2.md"},{"id":"time","name":"TIME","kind":"organization","context":"TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.","referenceUrls":["https://time.com/"],"path":"/entities/time","json":"/entities/time.json","markdown":"/entities/time.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a","revision":"7b04ca93-2a58-4c09-8b2e-88667a315f42","revisionKind":"publication-id","title":"Ai2 introduces ToyBlocks, a controlled world for robot learning and evaluation","url":"https://escapemeridian.com/readings/ai2-introduces-toyblocks-a-controlled-world-for-robot-learning-efdba0ba"},"source":{"type":"source","id":"https://allenai.org/blog/toyblocks","revision":"c0d8d4c7e20d5bbadaf5bac5935f4a020d5c1d01ef74f81a7388b8b2524501df","revisionKind":"sha256-source-text","title":"ToyBlocks: A controlled world for robot learning and evaluation | Ai2","url":"https://allenai.org/blog/toyblocks"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a","title":"Ai2 introduces ToyBlocks, a controlled world for robot learning and evaluation","argument":"Ai2 introduces ToyBlocks, a simulated tabletop world where researchers can vary robot-training data, repeat tests and investigate why models succeed or fail. The authors argue that this deliberately narrow setting makes it possible to combine realistic physics and appearance, automatic task and demonstration generation, challenges involving reasoning and dexterity, and broad coverage within one domain.\n\nToyBlocks generates robot demonstrations without requiring a person to remotely control the robot, and lets researchers test performance on tasks kept out of training. Ai2 reports that an agent in one search-and-spell experiment solved tasks or correctly declared them infeasible 63% of the time; errors included confusing similar-looking letters and choosing the wrong letter orientation.\n\nThe authors also report tests on physical robots using models trained only in ToyBlocks simulation. The hierarchical Molmo 2 + ManiFlow system succeeded in 50.0% of real-world episodes and 55.5% of simulated episodes. Ai2 says further trials are needed to establish statistical confidence across tasks. The release includes simulation assets, data, models, training code and evaluation code, giving other researchers materials to investigate robot-learning questions and failure modes in this controlled setting.","topicIds":["intelligence-and-discovery","work-and-agency"],"domain":"work","claimType":"attributed reading","limits":"ToyBlocks is a deliberately constrained simulated domain; its results do not by themselves establish performance across open-ended environments. The reported real-world hierarchical-system evaluation covered 30 scene configurations, and Ai2 says further trials are needed to establish statistical confidence across tasks. The source describes tests of action-conditioned world models but the supplied text does not report their comparative results.","speaker":{"name":"Ai2","roleAtSource":"Publisher"},"source":{"title":"ToyBlocks: A controlled world for robot learning and evaluation | Ai2","publisher":"Ai2","format":"Article","url":"https://allenai.org/blog/toyblocks","publishedAt":"2026-11-06T08:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Ai2 presents ToyBlocks as a controlled simulated environment for studying robot learning, with tools to generate tasks and demonstrations and measure behavior.","excerpt":"Each pairs a controlled, simulated world targeted at studying robotics in a specific environment with tools for automatically generating tasks and training demonstrations, as well as metrics for measuring a robot's behavior over time.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"The authors say the narrow domain makes it possible to meet ToyBlocks’ four design requirements, including realistic physics and appearance and broad task coverage within the domain.","excerpt":"ToyBlocks’ narrow scope, on the other hand, makes it possible to fulfill them all.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"ToyBlocks can generate demonstrations automatically, allowing researchers to vary training experiences and evaluate on tasks withheld from training.","excerpt":"ToyBlocks automatically generates demonstrations, which are recordings of a robot completing tasks in simulation, without requiring time-consuming teleoperation","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"In the search-and-spell experiment, Ai2 reports that the agent solved tasks or correctly declared them infeasible 63% of the time.","excerpt":"We found that in this experiment, the agent was able to successfully solve or correctly declare infeasible 63% of the tasks.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"In reported sim-to-real tests, the hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes; Ai2 says further trials are needed for statistical confidence across tasks.","excerpt":"The hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes (111/200) and 50.0% of real-world episodes (15/30); further trials are needed to establish statistical confidence across tasks.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"Ai2 says ToyBlocks provides simulated ground truth for controlled tests of how action-conditioned world models predict physical interactions.","excerpt":"ToyBlocks provides this ground truth – the actual simulated scene state – enabling highly controlled tests of how recently developed world models perform in low-level object manipulation.","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"},{"text":"Ai2 is releasing simulation assets, task and demonstration generators, demonstrations, trained models, training code, evaluation code and validation sets.","excerpt":"We’re releasing simulation assets, procedural task and demonstration generators, automatically generated demonstrations, two trained models showcasing what the framework can do","url":"https://allenai.org/blog/toyblocks","relationship":"Source-stated"}],"sourceMedia":{"status":"found","sourceUrl":"https://allenai.org/blog/toyblocks","checkedAt":"2026-11-08T01:51:12.014Z","items":[{"type":"image","url":"https://allenai.org/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqERuwxeB0K8%2Fmaxresdefault.jpg&w=1200&q=75","sourceUrl":"https://allenai.org/blog/toyblocks","attribution":"Ai2","basis":"publisher-embed","availability":"verified","verifiedAt":"2026-11-08T01:51:13.486Z"}],"note":"Media declared by the original publisher; availability checked separately from the editorial reading."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How accurately do the tested action-conditioned world models predict block motion and skill outcomes, particularly over full trajectories?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Ai2 introduces ToyBlocks, a controlled world for robot learning and evaluation","summary":"Ai2 introduces ToyBlocks, a simulated tabletop world where researchers can vary robot-training data, repeat tests and investigate why models succeed or fail. The authors argue that this deliberately narrow setting makes it possible to combine realistic physics and appearance, automatic task and demonstration generation, challenges involving reasoning and dexterity, and broad coverage within one domain.\n\nToyBlocks generates robot demonstrations without requiring a person to remotely control the robot, and lets researchers test performance on tasks kept out of training. Ai2 reports that an agent in one search-and-spell experiment solved tasks or correctly declared them infeasible 63% of the time; errors included confusing similar-looking letters and choosing the wrong letter orientation.\n\nThe authors also report tests on physical robots using models trained only in ToyBlocks simulation. The hierarchical Molmo 2 + ManiFlow system succeeded in 50.0% of real-world episodes and 55.5% of simulated episodes. Ai2 says further trials are needed to establish statistical confidence across tasks. The release includes simulation assets, data, models, training code and evaluation code, giving other researchers materials to investigate robot-learning questions and failure modes in this controlled setting.","question":"How accurately do the tested action-conditioned world models predict block motion and skill outcomes, particularly over full trajectories?","claims":[{"text":"Ai2 presents ToyBlocks as a controlled simulated environment for studying robot learning, with tools to generate tasks and demonstrations and measure behavior.","excerpt":"Each pairs a controlled, simulated world targeted at studying robotics in a specific environment with tools for automatically generating tasks and training demonstrations, as well as metrics for measuring a robot's behavior over time.","support":"source-stated"},{"text":"The authors say the narrow domain makes it possible to meet ToyBlocks’ four design requirements, including realistic physics and appearance and broad task coverage within the domain.","excerpt":"ToyBlocks’ narrow scope, on the other hand, makes it possible to fulfill them all.","support":"source-stated"},{"text":"ToyBlocks can generate demonstrations automatically, allowing researchers to vary training experiences and evaluate on tasks withheld from training.","excerpt":"ToyBlocks automatically generates demonstrations, which are recordings of a robot completing tasks in simulation, without requiring time-consuming teleoperation","support":"source-stated"},{"text":"In the search-and-spell experiment, Ai2 reports that the agent solved tasks or correctly declared them infeasible 63% of the time.","excerpt":"We found that in this experiment, the agent was able to successfully solve or correctly declare infeasible 63% of the tasks.","support":"source-stated"},{"text":"In reported sim-to-real tests, the hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes; Ai2 says further trials are needed for statistical confidence across tasks.","excerpt":"The hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated episodes (111/200) and 50.0% of real-world episodes (15/30); further trials are needed to establish statistical confidence across tasks.","support":"source-stated"},{"text":"Ai2 says ToyBlocks provides simulated ground truth for controlled tests of how action-conditioned world models predict physical interactions.","excerpt":"ToyBlocks provides this ground truth – the actual simulated scene state – enabling highly controlled tests of how recently developed world models perform in low-level object manipulation.","support":"source-stated"},{"text":"Ai2 is releasing simulation assets, task and demonstration generators, demonstrations, trained models, training code, evaluation code and validation sets.","excerpt":"We’re releasing simulation assets, procedural task and demonstration generators, automatically generated demonstrations, two trained models showcasing what the framework can do","support":"source-stated"}],"limitations":["ToyBlocks is a deliberately constrained simulated domain; its results do not by themselves establish performance across open-ended environments.","The reported real-world hierarchical-system evaluation covered 30 scene configurations, and Ai2 says further trials are needed to establish statistical confidence across tasks.","The source describes tests of action-conditioned world models but the supplied text does not report their comparative results."],"mediaId":"921b78122bbdc803b5bb54292ac0cd957132148010c818f34384f2f04584e3f4","correctionReason":null,"id":"efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a","publicationId":"7b04ca93-2a58-4c09-8b2e-88667a315f42","version":1,"sourceUrl":"https://allenai.org/blog/toyblocks","sourceTitle":"ToyBlocks: A controlled world for robot learning and evaluation | Ai2","publisher":"Ai2","publishedAt":"2026-11-06T08:00:00.000Z","originalPublished":{"value":"2026-11-06T08:00:00.000Z","precision":"instant","basis":"feed_entry"},"firstObservedAt":"2026-11-06T14:03:33.228Z","createdAt":"2026-11-08T02:16:22.134Z","historical":false,"entityIds":["ai2"],"topicIds":["intelligence-and-discovery","work-and-agency"],"media":[{"id":"921b78122bbdc803b5bb54292ac0cd957132148010c818f34384f2f04584e3f4","kind":"article","url":"https://allenai.org/blog/toyblocks","sourceUrl":"https://allenai.org/blog/toyblocks","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-08T01:51:13.486Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."},{"id":"056521bcd399666cdc521931d5bf9f056f3df6ced485157321a16fded42c2a53","kind":"image","url":"https://allenai.org/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqERuwxeB0K8%2Fmaxresdefault.jpg&w=1200&q=75","sourceUrl":"https://allenai.org/blog/toyblocks","label":"View source image","provenance":"publisher_page","verification":{"status":"verified","checkedAt":"2026-11-08T01:51:13.486Z","method":"publisher_association_and_head","contentType":"image/jpeg","cacheUntil":1794102973486},"recommendationReason":"Observed on the acquired publisher source; availability checked."}],"sourceMedia":{"status":"found","sourceUrl":"https://allenai.org/blog/toyblocks","checkedAt":"2026-11-08T01:51:12.014Z","items":[{"type":"image","url":"https://allenai.org/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqERuwxeB0K8%2Fmaxresdefault.jpg&w=1200&q=75","sourceUrl":"https://allenai.org/blog/toyblocks","attribution":"Ai2","basis":"publisher-embed","availability":"verified","verifiedAt":"2026-11-08T01:51:13.486Z"}],"note":"Media declared by the original publisher; availability checked separately from the editorial reading."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"921b78122bbdc803b5bb54292ac0cd957132148010c818f34384f2f04584e3f4","confidence":0.98,"provider":"typesafe","model":"jev-1.13.0","inputHash":"a30a26b0ccf88edeff6473c0cb4fa273bd9a70fd349cd454edf4783754a63854","decidedAt":"2026-11-08T01:51:17.771Z"},"provenance":{"textHash":"c0d8d4c7e20d5bbadaf5bac5935f4a020d5c1d01ef74f81a7388b8b2524501df","observationId":"d8207efd6a64ae1f94f6759638285bf160cac46b981e162c2e6856250f5f446d","retrievedAt":"2026-11-08T01:51:12.014Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"b5398c23-87f0-4ba9-8df5-27f3cf27f269"},"reviewer":{"model":"gpt-6-luna","runId":"1d73549a-724b-44ac-88b4-1ef2a00bde43"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-08T02:16:22.134Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[],"authorRole":"article-author","hosts":[],"status":"missing","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[],"participationCoverage":{"total":0,"returned":0,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://allenai.org/blog/toyblocks","snapshotId":"e7ed423a08642200f87f821c33ac0d6673f2bf1d5f444153508e764e3863da15","contentSha256":"3298c68ed4d6025b111585e2ebba525796ea68ef2351f2528d589823109c042c","textHash":"c0d8d4c7e20d5bbadaf5bac5935f4a020d5c1d01ef74f81a7388b8b2524501df","observedAt":"2026-11-08T01:51:12.014Z","basis":"publisher-page-metadata","sourceLinks":19,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/efdba0baf80ec39a2a94b3f89d9dad52aa5bef04c814856047c2b07b3934024a/source"}}}],"audits":[{"publicationId":"7b04ca93-2a58-4c09-8b2e-88667a315f42","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the framework, design rationale, experiments, results and release to Ai2 and its authors. Reported numbers are framed as Ai2's reports, not established facts.","excerpt":"Our first foundation world is ToyBlocks : an open simulation framework built around a robot arm and blocks on a tabletop."},{"dimension":"summary","status":"uncertain","explanation":"The reading is mostly faithful, but two key results are hard to read alone. First, 'hierarchical Molmo 2 + ManiFlow system' is unexplained. Repair: add a short gloss that Molmo 2 is a high-level vision-language planner choosing skills and blocks, and ManiFlow is the low-level model controlling the arm. Second, the 63% search-and-spell result omits that half the scenes were deliberately infeasible and that the agent was Claude Opus 5.5. Without this, readers cannot judge the figure. Repair: add one clause stating both points.","excerpt":"Importantly, 50% of the scenes did not have blocks with all the required letters, rendering the task infeasible."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-06 matches the date shown on the source page. No later revision is indicated.","excerpt":"ToyBlocks: A controlled world for robot learning and evaluation\nNovember 6, 2026"},{"dimension":"support","status":"pass","explanation":"Every claim is backed by an exact source excerpt. The sim-to-real figures keep Ai2's caveat about statistical confidence. The release list matches the source. The closing benefit sentence follows the source's own statement. The world-model question and the matching limitation correctly note that the supplied text does not report results.","excerpt":"This allows other teams to build their own training mixtures and investigate failure modes beyond those covered in our initial experiments."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original Ai2 blog post. This is the most direct way for readers to reach the source.","excerpt":null}],"createdAt":"2026-11-08T02:16:22.134Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}