Anthropic explains how its performance engineering take-home changed as Claude improved

Anthropic explains how its performance engineering take-home changed as Claude improved

Anthropic’s article was published on January 21, 2026. It describes how Claude models came to match or exceed human candidates on the company’s performance engineering take-home, a timed coding test, and how Anthropic redesigned it rather than banning AI assistance. The original test asked candidates to optimize code for a simulated accelerator. Anthropic says the test helped it identify strong engineers, including candidates with limited experience on paper, but Claude Opus 4.5 eventually matched the best human performance within the test’s time limit. The author tried a different optimization problem, then shifted to puzzles with a tiny, constrained instruction set. That replacement sacrifices some resemblance to the job: the author says the original worked because it resembled real work, while the replacement works because it simulates novel work. The article’s conclusion is that Anthropic still needs strong engineers and is seeking increasingly creative ways to find them. It also releases the original test as an open challenge, arguing that human experts retain an advantage over current models when given enough time. Anthropic says early results for the replacement are promising, but its long-term usefulness—and how long it will resist models—remains unknown.

Publication: a919ed96-c27c-28e3-a8ad-d272ec7845eb

attribution — pass: This is a single first-person Anthropic research post, not a listing. The reading attributes the account to Anthropic and to an unnamed author, and correctly notes that the text does not name the author. Claims remain framed as what Anthropic or the author says.

I've now iterated through three versions of our take-home in an attempt to ensure it still carries signal.

summary — pass: The summary captures the arc: Claude models defeated successive versions of the take-home, AI assistance was not banned, a first attempt failed, and the author moved to constrained puzzles. It preserves the realism trade-off, the open challenge and the closing uncertainty. It explains take-home as a timed coding test. Terms like simulated accelerator and instruction set are not explained, but the central development remains readable. The question follows directly from the source's own closing line.

We're curious how long it lasts.

date — pass: January 21, 2026 matches the originalPublished UTC day. The page was edited after publication (dateModified October 2026). However, the reading relies only on core narrative passages and does not use later-looking material such as the improved-harness benchmark. There is no hindsight framing. The retrieval date is disclosed. The kind is publisher-page, so no archive disclosure is needed.

support — pass: Each claim's excerpt is an exact quote from the source and supports the claim as attributed. These include Opus 4.5 matching the best 2-hour human score, the constrained-instruction-set puzzles, the shift from real to novel work, the continuing need for engineers, the long-horizon human advantage, and the promising early results. The cited passages are integral to the original narrative rather than plausible later edits. No benefits or costs are asserted beyond the source.

Early results are promising: scores correlate well with the caliber of candidates' past work

media — pass: No media is selected and no verified media candidates were supplied, so leaving the media empty is appropriate.