{"id":"97151f4d6392215d976846618eb33c2e8c538e67458d530cba11b5646647b178","title":"Anthropic explains how its performance engineering take-home changed as Claude improved","argument":"Anthropic’s article was published on January 21, 2026. It describes how Claude models came to match or exceed human candidates on the company’s performance engineering take-home, a timed coding test, and how Anthropic redesigned it rather than banning AI assistance. The original test asked candidates to optimize code for a simulated accelerator. Anthropic says the test helped it identify strong engineers, including candidates with limited experience on paper, but Claude Opus 4.5 eventually matched the best human performance within the test’s time limit. The author tried a different optimization problem, then shifted to puzzles with a tiny, constrained instruction set. That replacement sacrifices some resemblance to the job: the author says the original worked because it resembled real work, while the replacement works because it simulates novel work. The article’s conclusion is that Anthropic still needs strong engineers and is seeking increasingly creative ways to find them. It also releases the original test as an open challenge, arguing that human experts retain an advantage over current models when given enough time. Anthropic says early results for the replacement are promising, but its long-term usefulness—and how long it will resist models—remains unknown.","topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation","artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 28, 2026; originally published January 21, 2026. The article does not name its author, so this reading attributes its account to Anthropic. The article describes early results for the replacement test but gives no evidence of its long-term performance.","speaker":{"name":"Anthropic","roleAtSource":"Publisher"},"source":{"title":"Designing AI resistant technical evaluations","publisher":"Anthropic","format":"Article","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","publishedAt":"2026-01-21T00:00:00.000Z","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"Anthropic says Claude Opus 4.5 matched the best human performance on its take-home within the test’s time limit.","excerpt":"By the 2-hour mark, its score matched the best human performance within that time limit","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"The author replaced a realistic optimization task with puzzles using a tiny, heavily constrained instruction set.","excerpt":"I designed a new take-home consisting of puzzles using a tiny, heavily constrained instruction set","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"The author says the new take-home sacrifices realism, while its unfamiliarity is part of what makes it work.","excerpt":"The original worked because it resembled real work. The replacement works because it simulates novel work.","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"Anthropic says it still needs strong engineers and is seeking creative ways to identify them.","excerpt":"we still need more strong engineers—just increasingly creative ways to find them.","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"Anthropic invites people to try the original take-home and says human experts retain an advantage over current models with enough time.","excerpt":"Human experts retain an advantage over current models at sufficiently long time horizons.","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"Anthropic says early results for the replacement are promising, while wondering how long it will last.","excerpt":"Early results are promising: scores correlate well with the caliber of candidates' past work","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How long will the replacement take-home remain useful as Claude models improve?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Anthropic explains how its performance engineering take-home changed as Claude improved","summary":"Anthropic’s article was published on January 21, 2026. It describes how Claude models came to match or exceed human candidates on the company’s performance engineering take-home, a timed coding test, and how Anthropic redesigned it rather than banning AI assistance. The original test asked candidates to optimize code for a simulated accelerator. Anthropic says the test helped it identify strong engineers, including candidates with limited experience on paper, but Claude Opus 4.5 eventually matched the best human performance within the test’s time limit. The author tried a different optimization problem, then shifted to puzzles with a tiny, constrained instruction set. That replacement sacrifices some resemblance to the job: the author says the original worked because it resembled real work, while the replacement works because it simulates novel work. The article’s conclusion is that Anthropic still needs strong engineers and is seeking increasingly creative ways to find them. It also releases the original test as an open challenge, arguing that human experts retain an advantage over current models when given enough time. Anthropic says early results for the replacement are promising, but its long-term usefulness—and how long it will resist models—remains unknown.","question":"How long will the replacement take-home remain useful as Claude models improve?","claims":[{"text":"Anthropic says Claude Opus 4.5 matched the best human performance on its take-home within the test’s time limit.","excerpt":"By the 2-hour mark, its score matched the best human performance within that time limit","support":"source-stated"},{"text":"The author replaced a realistic optimization task with puzzles using a tiny, heavily constrained instruction set.","excerpt":"I designed a new take-home consisting of puzzles using a tiny, heavily constrained instruction set","support":"source-stated"},{"text":"The author says the new take-home sacrifices realism, while its unfamiliarity is part of what makes it work.","excerpt":"The original worked because it resembled real work. The replacement works because it simulates novel work.","support":"source-stated"},{"text":"Anthropic says it still needs strong engineers and is seeking creative ways to identify them.","excerpt":"we still need more strong engineers—just increasingly creative ways to find them.","support":"source-stated"},{"text":"Anthropic invites people to try the original take-home and says human experts retain an advantage over current models with enough time.","excerpt":"Human experts retain an advantage over current models at sufficiently long time horizons.","support":"source-stated"},{"text":"Anthropic says early results for the replacement are promising, while wondering how long it will last.","excerpt":"Early results are promising: scores correlate well with the caliber of candidates' past work","support":"source-stated"}],"limitations":["Text as retrieved on October 28, 2026; originally published January 21, 2026.","The article does not name its author, so this reading attributes its account to Anthropic.","The article describes early results for the replacement test but gives no evidence of its long-term performance."],"mediaId":null,"correctionReason":null,"id":"97151f4d6392215d976846618eb33c2e8c538e67458d530cba11b5646647b178","publicationId":"a919ed96-c27c-28e3-a8ad-d272ec7845eb","version":1,"sourceUrl":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","sourceTitle":"Designing AI resistant technical evaluations","publisher":"Anthropic","publishedAt":"2026-01-21T00:00:00.000Z","originalPublished":{"value":"2026-01-21T00:00:00.000Z","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-10-21T10:41:20.739Z","createdAt":"2026-11-09T01:03:30.964Z","historical":true,"entityIds":["anthropic"],"topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation","artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"8e75dd78d4cf29692c12cf88744d1cc638540414bd86dc4502303b6c2c9609b2","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-28T04:24:28.281Z"},"provenance":{"textHash":"676c8db1122128fd7d5d70e2907b582a1262823fbdd6d2c2a4c74032572fcb69","observationId":"96c91918ae916100a4666e47b19be8d35c13273b077e064c26d85e5e73c5f5a5","retrievedAt":"2026-10-28T04:25:53.704Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"858b1b3b-5f16-6354-d583-ee5d5bb96bf0"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"8a751209-d191-a11e-5f66-0f5e07bc654f"}},"audit":{"status":"passed","reviewedAt":"2026-11-09T00:59:29.317Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/anthropic-explains-how-its-performance-engineering-take-home-97151f4d","formats":{"html":"/readings/anthropic-explains-how-its-performance-engineering-take-home-97151f4d","markdown":"/readings/anthropic-explains-how-its-performance-engineering-take-home-97151f4d.md","json":"/readings/anthropic-explains-how-its-performance-engineering-take-home-97151f4d.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"anthropic","name":"Anthropic","kind":"organization","context":"Anthropic develops Claude and publishes research about AI capability, safety and how people use its systems.","referenceUrls":["https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/anthropic","json":"/entities/anthropic.json","markdown":"/entities/anthropic.md"},{"id":"claude","name":"Claude","kind":"model","context":"Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.","referenceUrls":["https://claude.com/product/overview","https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/claude","json":"/entities/claude.json","markdown":"/entities/claude.md"},{"id":"time","name":"TIME","kind":"organization","context":"TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.","referenceUrls":["https://time.com/"],"path":"/entities/time","json":"/entities/time.json","markdown":"/entities/time.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"97151f4d6392215d976846618eb33c2e8c538e67458d530cba11b5646647b178","revision":"a919ed96-c27c-28e3-a8ad-d272ec7845eb","revisionKind":"publication-id","title":"Anthropic explains how its performance engineering take-home changed as Claude improved","url":"https://escapemeridian.com/readings/anthropic-explains-how-its-performance-engineering-take-home-97151f4d"},"source":{"type":"source","id":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","revision":"676c8db1122128fd7d5d70e2907b582a1262823fbdd6d2c2a4c74032572fcb69","revisionKind":"sha256-source-text","title":"Designing AI resistant technical evaluations","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"97151f4d6392215d976846618eb33c2e8c538e67458d530cba11b5646647b178","title":"Anthropic explains how its performance engineering take-home changed as Claude improved","argument":"Anthropic’s article was published on January 21, 2026. It describes how Claude models came to match or exceed human candidates on the company’s performance engineering take-home, a timed coding test, and how Anthropic redesigned it rather than banning AI assistance. The original test asked candidates to optimize code for a simulated accelerator. Anthropic says the test helped it identify strong engineers, including candidates with limited experience on paper, but Claude Opus 4.5 eventually matched the best human performance within the test’s time limit. The author tried a different optimization problem, then shifted to puzzles with a tiny, constrained instruction set. That replacement sacrifices some resemblance to the job: the author says the original worked because it resembled real work, while the replacement works because it simulates novel work. The article’s conclusion is that Anthropic still needs strong engineers and is seeking increasingly creative ways to find them. It also releases the original test as an open challenge, arguing that human experts retain an advantage over current models when given enough time. Anthropic says early results for the replacement are promising, but its long-term usefulness—and how long it will resist models—remains unknown.","topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation","artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 28, 2026; originally published January 21, 2026. The article does not name its author, so this reading attributes its account to Anthropic. The article describes early results for the replacement test but gives no evidence of its long-term performance.","speaker":{"name":"Anthropic","roleAtSource":"Publisher"},"source":{"title":"Designing AI resistant technical evaluations","publisher":"Anthropic","format":"Article","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","publishedAt":"2026-01-21T00:00:00.000Z","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"Anthropic says Claude Opus 4.5 matched the best human performance on its take-home within the test’s time limit.","excerpt":"By the 2-hour mark, its score matched the best human performance within that time limit","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"The author replaced a realistic optimization task with puzzles using a tiny, heavily constrained instruction set.","excerpt":"I designed a new take-home consisting of puzzles using a tiny, heavily constrained instruction set","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"The author says the new take-home sacrifices realism, while its unfamiliarity is part of what makes it work.","excerpt":"The original worked because it resembled real work. The replacement works because it simulates novel work.","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"Anthropic says it still needs strong engineers and is seeking creative ways to identify them.","excerpt":"we still need more strong engineers—just increasingly creative ways to find them.","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"Anthropic invites people to try the original take-home and says human experts retain an advantage over current models with enough time.","excerpt":"Human experts retain an advantage over current models at sufficiently long time horizons.","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"},{"text":"Anthropic says early results for the replacement are promising, while wondering how long it will last.","excerpt":"Early results are promising: scores correlate well with the caliber of candidates' past work","url":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How long will the replacement take-home remain useful as Claude models improve?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Anthropic explains how its performance engineering take-home changed as Claude improved","summary":"Anthropic’s article was published on January 21, 2026. It describes how Claude models came to match or exceed human candidates on the company’s performance engineering take-home, a timed coding test, and how Anthropic redesigned it rather than banning AI assistance. The original test asked candidates to optimize code for a simulated accelerator. Anthropic says the test helped it identify strong engineers, including candidates with limited experience on paper, but Claude Opus 4.5 eventually matched the best human performance within the test’s time limit. The author tried a different optimization problem, then shifted to puzzles with a tiny, constrained instruction set. That replacement sacrifices some resemblance to the job: the author says the original worked because it resembled real work, while the replacement works because it simulates novel work. The article’s conclusion is that Anthropic still needs strong engineers and is seeking increasingly creative ways to find them. It also releases the original test as an open challenge, arguing that human experts retain an advantage over current models when given enough time. Anthropic says early results for the replacement are promising, but its long-term usefulness—and how long it will resist models—remains unknown.","question":"How long will the replacement take-home remain useful as Claude models improve?","claims":[{"text":"Anthropic says Claude Opus 4.5 matched the best human performance on its take-home within the test’s time limit.","excerpt":"By the 2-hour mark, its score matched the best human performance within that time limit","support":"source-stated"},{"text":"The author replaced a realistic optimization task with puzzles using a tiny, heavily constrained instruction set.","excerpt":"I designed a new take-home consisting of puzzles using a tiny, heavily constrained instruction set","support":"source-stated"},{"text":"The author says the new take-home sacrifices realism, while its unfamiliarity is part of what makes it work.","excerpt":"The original worked because it resembled real work. The replacement works because it simulates novel work.","support":"source-stated"},{"text":"Anthropic says it still needs strong engineers and is seeking creative ways to identify them.","excerpt":"we still need more strong engineers—just increasingly creative ways to find them.","support":"source-stated"},{"text":"Anthropic invites people to try the original take-home and says human experts retain an advantage over current models with enough time.","excerpt":"Human experts retain an advantage over current models at sufficiently long time horizons.","support":"source-stated"},{"text":"Anthropic says early results for the replacement are promising, while wondering how long it will last.","excerpt":"Early results are promising: scores correlate well with the caliber of candidates' past work","support":"source-stated"}],"limitations":["Text as retrieved on October 28, 2026; originally published January 21, 2026.","The article does not name its author, so this reading attributes its account to Anthropic.","The article describes early results for the replacement test but gives no evidence of its long-term performance."],"mediaId":null,"correctionReason":null,"id":"97151f4d6392215d976846618eb33c2e8c538e67458d530cba11b5646647b178","publicationId":"a919ed96-c27c-28e3-a8ad-d272ec7845eb","version":1,"sourceUrl":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","sourceTitle":"Designing AI resistant technical evaluations","publisher":"Anthropic","publishedAt":"2026-01-21T00:00:00.000Z","originalPublished":{"value":"2026-01-21T00:00:00.000Z","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-10-21T10:41:20.739Z","createdAt":"2026-11-09T01:03:30.964Z","historical":true,"entityIds":["anthropic"],"topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation","artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.anthropic.com/research/AI-resistant-technical-evaluations","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"8e75dd78d4cf29692c12cf88744d1cc638540414bd86dc4502303b6c2c9609b2","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-28T04:24:28.281Z"},"provenance":{"textHash":"676c8db1122128fd7d5d70e2907b582a1262823fbdd6d2c2a4c74032572fcb69","observationId":"96c91918ae916100a4666e47b19be8d35c13273b077e064c26d85e5e73c5f5a5","retrievedAt":"2026-10-28T04:25:53.704Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"858b1b3b-5f16-6354-d583-ee5d5bb96bf0"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"8a751209-d191-a11e-5f66-0f5e07bc654f"}},"audit":{"status":"passed","reviewedAt":"2026-11-09T00:59:29.317Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"a919ed96-c27c-28e3-a8ad-d272ec7845eb","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"This is a single first-person Anthropic research post, not a listing. The reading attributes the account to Anthropic and to an unnamed author, and correctly notes that the text does not name the author. Claims remain framed as what Anthropic or the author says.","excerpt":"I've now iterated through three versions of our take-home in an attempt to ensure it still carries signal."},{"dimension":"summary","status":"pass","explanation":"The summary captures the arc: Claude models defeated successive versions of the take-home, AI assistance was not banned, a first attempt failed, and the author moved to constrained puzzles. It preserves the realism trade-off, the open challenge and the closing uncertainty. It explains take-home as a timed coding test. Terms like simulated accelerator and instruction set are not explained, but the central development remains readable. The question follows directly from the source's own closing line.","excerpt":"We're curious how long it lasts."},{"dimension":"date","status":"pass","explanation":"January 21, 2026 matches the originalPublished UTC day. The page was edited after publication (dateModified October 2026). However, the reading relies only on core narrative passages and does not use later-looking material such as the improved-harness benchmark. There is no hindsight framing. The retrieval date is disclosed. The kind is publisher-page, so no archive disclosure is needed.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim's excerpt is an exact quote from the source and supports the claim as attributed. These include Opus 4.5 matching the best 2-hour human score, the constrained-instruction-set puzzles, the shift from real to novel work, the continuing need for engineers, the long-horizon human advantage, and the promising early results. The cited passages are integral to the original narrative rather than plausible later edits. No benefits or costs are asserted beyond the source.","excerpt":"Early results are promising: scores correlate well with the caliber of candidates' past work"},{"dimension":"media","status":"pass","explanation":"No media is selected and no verified media candidates were supplied, so leaving the media empty is appropriate.","excerpt":null}],"createdAt":"2026-11-09T00:59:29.317Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}