# MIT Technology Review reports Covariant’s robot model learns from data, but still struggles

[Canonical reading](https://escapemeridian.com/readings/mit-technology-review-reports-covariant-s-robot-model-learns-93d864b8)

MIT Technology Review’s article, published March 11, 2024, reports that Covariant introduced RFM-1, a model trained on data from its robots as well as internet text and videos. In a demonstration, the model accepted text, images, video, robot instructions and measurements, and could carry out a picking task or generate images and video of a task. Covariant cofounder Peter Chen called the approach a leap forward from task-specific code, while outside researcher Lerrel Pinto said deploying a robot communicating in many modes at scale was an impressive feat. Pinto also said success outside the demonstration would depend on access to enough robot data. The report describes a failure: when asked to return a banana to a tote, the robot picked up several other items before completing the task. Chen said it might not work well where training data is poor. Covariant hopes deployment will make the model more capable and efficient; that outcome is not demonstrated here. The article also raises concerns about using creators’ videos without payment and about bias, and says errors could compound if the model trains on videos it creates. Whether more training data can make the system reliable in unfamiliar work settings, without compounding errors or other harms, remains unknown.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

MIT Technology Review · Article

Original source date: Mar 11, 2024 (2024-03-11T13:00:00Z).

Date basis: Date from the publisher’s own records, through its API.

Added to Escape Meridian: 2026-11-21T03:43:09.691Z. First observed: 2026-10-28T17:20:14.278Z. Retrieved: 2026-10-28T17:22:01.290Z.

- [An OpenAI spinoff has built an AI model that helps robots learn tasks like humans](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Limits

Text as retrieved on October 28, 2026; originally published March 11, 2024. The report describes a demonstration, not measured performance across warehouse floors or loading docks. The article gives no measured rate of task success or failure in unfamiliar settings. The article raises possible concerns about video use, bias and compounding errors; it does not report measured effects of these risks.


## Open question

When tested on warehouse floors and loading docks, how often does RFM-1 fail on unfamiliar tasks, and can errors compound if it trains on videos it creates?


## Source-stated

MIT Technology Review reports that Covariant trained RFM-1 on data from its robot fleet, along with internet text and videos.

> The new model, called RFM-1, was trained on years of data collected from Covariant’s small fleet of item-picking robots

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

The report says RFM-1 can take several kinds of input and, in a demonstration, could perform a picking task or generate images and video of a task.

> users can prompt the model using five different types of input: text, images, video, robot instructions, and measurements.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

MIT Technology Review reports that the robot struggled to return a banana to a tote, picking up other items before completing the task.

> it struggled with retracing its steps, leading it to pick up a sponge, then an apple, then a host of other items before it finally accomplished the banana task.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

The report says Lerrel Pinto considered deployment at scale with communication across many modes an impressive feat, while warning that useful performance in the wild would require enough data.

> To outpace its competitors, Covariant will have to get its hands on enough data for the robot to become useful in the wild, Pinto told me.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

Covariant hopes deployment will improve RFM-1, but the report does not demonstrate that outcome.

> The company hopes the system will become more capable and efficient as it’s deployed in the real world.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

The article raises concerns about unpaid use of video creators’ work and about errors compounding if the model trains on videos it creates.

> the type of meta-learning that not only makes my head spin but also sparks concern about what will happen if errors made by the model compound themselves.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source and contributor context

MIT Technology Review · Publisher

The publisher is identified here. No individual author has been inferred.


## Source media

Media retrieval status: unchecked.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Editorial provenance

Version 1. Publication ID: 2aeb338d-263a-0d6f-2bba-cfae84f32c94.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-21T03:38:32.123Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical.

Observation: c0fd2b979c0d7ec2b3f3d5672baae195e713c1a6c9d679da8de89fe99ac01171. Source text hash: 723e72004cea5e43a4e7922b6f8d34f1e7358bcc1271b441e3573115aa2d1cd4.


## Revision and source-check history

### Version 1 — 2026-11-21T03:43:09.691Z

MIT Technology Review’s article, published March 11, 2024, reports that Covariant introduced RFM-1, a model trained on data from its robots as well as internet text and videos. In a demonstration, the model accepted text, images, video, robot instructions and measurements, and could carry out a picking task or generate images and video of a task. Covariant cofounder Peter Chen called the approach a leap forward from task-specific code, while outside researcher Lerrel Pinto said deploying a robot communicating in many modes at scale was an impressive feat. Pinto also said success outside the demonstration would depend on access to enough robot data. The report describes a failure: when asked to return a banana to a tote, the robot picked up several other items before completing the task. Chen said it might not work well where training data is poor. Covariant hopes deployment will make the model more capable and efficient; that outcome is not demonstrated here. The article also raises concerns about using creators’ videos without payment and about bias, and says errors could compound if the model trains on videos it creates. Whether more training data can make the system reliable in unfamiliar work settings, without compounding errors or other harms, remains unknown.

## Original source

MIT Technology Review · Article

Original source date: Mar 11, 2024 (2024-03-11T13:00:00Z).

Date basis: Date from the publisher’s own records, through its API.

Added to Escape Meridian: 2026-11-21T03:43:09.691Z. First observed: 2026-10-28T17:20:14.278Z. Retrieved: 2026-10-28T17:22:01.290Z.

- [An OpenAI spinoff has built an AI model that helps robots learn tasks like humans](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Limits

Text as retrieved on October 28, 2026; originally published March 11, 2024. The report describes a demonstration, not measured performance across warehouse floors or loading docks. The article gives no measured rate of task success or failure in unfamiliar settings. The article raises possible concerns about video use, bias and compounding errors; it does not report measured effects of these risks.


## Open question

When tested on warehouse floors and loading docks, how often does RFM-1 fail on unfamiliar tasks, and can errors compound if it trains on videos it creates?


## Source-stated

MIT Technology Review reports that Covariant trained RFM-1 on data from its robot fleet, along with internet text and videos.

> The new model, called RFM-1, was trained on years of data collected from Covariant’s small fleet of item-picking robots

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

The report says RFM-1 can take several kinds of input and, in a demonstration, could perform a picking task or generate images and video of a task.

> users can prompt the model using five different types of input: text, images, video, robot instructions, and measurements.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

MIT Technology Review reports that the robot struggled to return a banana to a tote, picking up other items before completing the task.

> it struggled with retracing its steps, leading it to pick up a sponge, then an apple, then a host of other items before it finally accomplished the banana task.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

The report says Lerrel Pinto considered deployment at scale with communication across many modes an impressive feat, while warning that useful performance in the wild would require enough data.

> To outpace its competitors, Covariant will have to get its hands on enough data for the robot to become useful in the wild, Pinto told me.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

Covariant hopes deployment will improve RFM-1, but the report does not demonstrate that outcome.

> The company hopes the system will become more capable and efficient as it’s deployed in the real world.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source-stated

The article raises concerns about unpaid use of video creators’ work and about errors compounding if the model trains on videos it creates.

> the type of meta-learning that not only makes my head spin but also sparks concern about what will happen if errors made by the model compound themselves.

- [Supporting source](https://www.technologyreview.com/2024/03/11/1089653/an-openai-spinoff-has-built-an-ai-model-that-helps-robots-learn-tasks-like-humans/)

## Source and contributor context

MIT Technology Review · Publisher

The publisher is identified here. No individual author has been inferred.


## Source media

Media retrieval status: unchecked.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Editorial provenance

Version 1. Publication ID: 2aeb338d-263a-0d6f-2bba-cfae84f32c94.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-21T03:38:32.123Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical.

Observation: c0fd2b979c0d7ec2b3f3d5672baae195e713c1a6c9d679da8de89fe99ac01171. Source text hash: 723e72004cea5e43a4e7922b6f8d34f1e7358bcc1271b441e3573115aa2d1cd4.


### passed — 2026-11-21T03:38:32.123Z — anthropic/claude-opus-5.5

Publication: 2aeb338d-263a-0d6f-2bba-cfae84f32c94

attribution — pass: This is a single MIT Technology Review article, not a listing. The reading attributes its claims to the report, Covariant, Peter Chen and Lerrel Pinto as the source does, and identifies Pinto as having no ties to Covariant. The related-story blurbs at the end of the text are not used.

> Lerrel Pinto, a researcher who runs the general-purpose robotics and AI lab at New York University and has no ties to Covariant

summary — pass: The reading clearly presents Covariant's RFM-1, its five input types, Chen's claim of a leap beyond task-specific code, Pinto's praise and his data caveat, the banana failure, and the concerns about creator pay, bias and compounding errors. A general reader can follow it. It omits the article's work-related aside about managers issuing instructions without regard to human labor limits; that is a possible enrichment, not a fidelity failure. The question follows from the evidence.

> “It doesn’t understand the new concept,” Chen said by way of explanation, “but it’s a good example—it might not work well yet in the places where you don’t have good training data.”

date — pass: The reading's date of March 11, 2024 matches the UTC calendar day of originalPublished, and the evidence kind is a publisher page, so no archive disclosure is needed. The modifiedAt value precedes publication. The sidebar items, such as the AlphaGo ten-year piece, are likely later additions, and the reading does not use them. Covariant's hopes and Abbeel's prediction are not presented as outcomes.

> The company hopes the system will become more capable and efficient as it’s deployed in the real world.

support — pass: Each claim matches its excerpt. The demonstration capabilities, the banana failure, Pinto's comments, the hoped-for improvement and the compounding-error concern are all source-stated. The limitations accurately note that the source gives no measured success rates. No benefits or harms are invented.

> it struggled with retracing its steps, leading it to pick up a sponge, then an apple, then a host of other items before it finally accomplished the banana task.

media — pass: No media was selected, and no verified candidate media exists in the evidence, so leaving it empty is appropriate.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/93d864b84233688f14871bc550f451bb872c7cb51512cecef07e237cf3efb549/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/mit-technology-review-reports-covariant-s-robot-model-learns-93d864b8.json)
