{"id":"fdfbe7fd4e4602851d730030a220e6e3db01c14ccca0897b105b6d7dc7780d03","title":"Epoch AI introduces MirrorCode for testing long-horizon coding tasks","argument":"Epoch AI’s page, published June 26, 2026, presents MirrorCode, a benchmark co-developed with METR to test whether AI models can reimplement entire programs without access to their original source code. The models’ solutions must match the programs’ outputs on end-to-end tests, including held-out tests. Epoch AI says Claude Opus 4.7 reimplemented gotree, a bioinformatics toolkit, in 14 hours for $251; the best implementations passed 2000/2001 tests but missed one edge case, so Epoch AI calls the result near-perfect rather than a complete solution. The example illustrates the kind of long task MirrorCode measures, not a general measure of software reliability. Epoch AI says open-source targets may have appeared in models’ pre-training, which could inflate scores. Its memorization screen results suggest performance was not dominated by memorization, but Epoch AI cannot rule out a contribution from memorization. It expects MirrorCode capabilities to generalize to unseen codebases. Whether they do remains unknown; the page describes this as an expectation, not a demonstrated result. The source’s central point is that AI can already perform some difficult, extended coding tasks, while the benchmark’s results still carry a data-contamination caveat.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 24, 2026; originally published June 26, 2026. Epoch AI says it cannot rule out that memorization contributes to performance; the page does not establish how much it contributes.","speaker":{"name":"Epoch AI","roleAtSource":"Publisher"},"source":{"title":"MirrorCode: What's the largest software project AI can complete on its own?","publisher":"Epoch AI","format":"Article","url":"https://epoch.ai/MirrorCode","publishedAt":"2026-06-26T00:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Epoch AI says MirrorCode tests AI models by asking them to reimplement entire programs without access to the original source code, with solutions assessed against end-to-end tests.","excerpt":"In a MirrorCode task, AI models are tasked with reimplementing an entire program end-to-end, without access to the original source code. AI-generated solutions must match the original program’s output exactly on end-to-end tests, including held-out tests.","url":"https://epoch.ai/MirrorCode","relationship":"Source-stated"},{"text":"Epoch AI reports that Claude Opus 4.7 reimplemented gotree in 14 hours for $251, and characterizes the result as near-perfect despite one failed edge-case test.","excerpt":"Opus 4.7 solved it in 14 hours, costing $251.","url":"https://epoch.ai/MirrorCode","relationship":"Source-stated"},{"text":"Epoch AI says possible data contamination could inflate performance, but it cannot rule out that memorization contributes to results; it expects the measured capabilities to generalize to unseen codebases.","excerpt":"This suggests that the results were not dominated by memorization, but we cannot rule out the possibility that memorization contributes to AI performance. Overall, we expect that the capabilities measured by MirrorCode would generalize to an unseen codebase.","url":"https://epoch.ai/MirrorCode","relationship":"Source-stated"},{"text":"Escape Meridian reads the reported gotree result as an example of a long coding task completed nearly, but not fully, to the benchmark’s standard.","excerpt":"The best-scoring AI gotree implementations passed 2000/2001 tests, but failed a single edge-case test for a niche command to manipulate date annotations. Consequently, they do not strictly solve the task to 100% completion, but we consider the reimplementation near-perfect, covering essentially all","url":"https://epoch.ai/MirrorCode","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/MirrorCode","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would MirrorCode performance generalize to codebases that were not in a model’s pre-training?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Epoch AI introduces MirrorCode for testing long-horizon coding tasks","summary":"Epoch AI’s page, published June 26, 2026, presents MirrorCode, a benchmark co-developed with METR to test whether AI models can reimplement entire programs without access to their original source code. The models’ solutions must match the programs’ outputs on end-to-end tests, including held-out tests. Epoch AI says Claude Opus 4.7 reimplemented gotree, a bioinformatics toolkit, in 14 hours for $251; the best implementations passed 2000/2001 tests but missed one edge case, so Epoch AI calls the result near-perfect rather than a complete solution. The example illustrates the kind of long task MirrorCode measures, not a general measure of software reliability. Epoch AI says open-source targets may have appeared in models’ pre-training, which could inflate scores. Its memorization screen results suggest performance was not dominated by memorization, but Epoch AI cannot rule out a contribution from memorization. It expects MirrorCode capabilities to generalize to unseen codebases. Whether they do remains unknown; the page describes this as an expectation, not a demonstrated result. The source’s central point is that AI can already perform some difficult, extended coding tasks, while the benchmark’s results still carry a data-contamination caveat.","question":"How well would MirrorCode performance generalize to codebases that were not in a model’s pre-training?","claims":[{"text":"Epoch AI says MirrorCode tests AI models by asking them to reimplement entire programs without access to the original source code, with solutions assessed against end-to-end tests.","excerpt":"In a MirrorCode task, AI models are tasked with reimplementing an entire program end-to-end, without access to the original source code. AI-generated solutions must match the original program’s output exactly on end-to-end tests, including held-out tests.","support":"source-stated"},{"text":"Epoch AI reports that Claude Opus 4.7 reimplemented gotree in 14 hours for $251, and characterizes the result as near-perfect despite one failed edge-case test.","excerpt":"Opus 4.7 solved it in 14 hours, costing $251.","support":"source-stated"},{"text":"Epoch AI says possible data contamination could inflate performance, but it cannot rule out that memorization contributes to results; it expects the measured capabilities to generalize to unseen codebases.","excerpt":"This suggests that the results were not dominated by memorization, but we cannot rule out the possibility that memorization contributes to AI performance. Overall, we expect that the capabilities measured by MirrorCode would generalize to an unseen codebase.","support":"source-stated"},{"text":"Escape Meridian reads the reported gotree result as an example of a long coding task completed nearly, but not fully, to the benchmark’s standard.","excerpt":"The best-scoring AI gotree implementations passed 2000/2001 tests, but failed a single edge-case test for a niche command to manipulate date annotations. Consequently, they do not strictly solve the task to 100% completion, but we consider the reimplementation near-perfect, covering essentially all","support":"interpretation"}],"limitations":["Text as retrieved on October 24, 2026; originally published June 26, 2026.","Epoch AI says it cannot rule out that memorization contributes to performance; the page does not establish how much it contributes."],"mediaId":null,"correctionReason":null,"id":"fdfbe7fd4e4602851d730030a220e6e3db01c14ccca0897b105b6d7dc7780d03","publicationId":"9c95fbd9-c3d3-4491-586a-e8a675e73b72","version":1,"sourceUrl":"https://epoch.ai/MirrorCode","sourceTitle":"MirrorCode: What's the largest software project AI can complete on its own?","publisher":"Epoch AI","publishedAt":"2026-06-26T00:00:00.000Z","originalPublished":{"value":"2026-06-26T00:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:13:22.796Z","createdAt":"2026-11-06T11:33:23.471Z","historical":true,"entityIds":["epoch-ai"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/MirrorCode","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"e0a14cef87c810978e6f2de684ee20507b15adff55b84541747d91b5c8b77ed2","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-24T05:09:55.394Z"},"provenance":{"textHash":"7a0ed674c0e8e674ede6dd31eaa915b00803b82d1c62dc3f69dcc20a220c7e95","observationId":"5d07d8826cde2ba93d545bb1ee7b70ec72d2fa7a69a316b32065b33e44b60154","retrievedAt":"2026-10-24T05:11:37.128Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"12fb8282-a7a2-b678-7490-720c257d3bcf"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"fe450b69-a5d6-f0d6-7884-d684a2f55d57"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T11:28:39.893Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/epoch-ai-introduces-mirrorcode-for-testing-long-horizon-coding-fdfbe7fd","formats":{"html":"/readings/epoch-ai-introduces-mirrorcode-for-testing-long-horizon-coding-fdfbe7fd","markdown":"/readings/epoch-ai-introduces-mirrorcode-for-testing-long-horizon-coding-fdfbe7fd.md","json":"/readings/epoch-ai-introduces-mirrorcode-for-testing-long-horizon-coding-fdfbe7fd.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-pre-training","name":"Pre-training","explanation":"A model's initial training on a large dataset, before further adaptation for particular uses.","referenceUrl":"https://developers.google.com/machine-learning/glossary#pre-training"}],"entities":[{"id":"epoch-ai","name":"Epoch AI","kind":"organization","context":"Epoch AI studies trends in AI capability, training resources and the infrastructure behind them.","referenceUrls":["https://epoch.ai/","https://epoch.ai/data"],"path":"/entities/epoch-ai","json":"/entities/epoch-ai.json","markdown":"/entities/epoch-ai.md"},{"id":"metr","name":"METR","kind":"organization","context":"METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.","referenceUrls":["https://metr.org/"],"path":"/entities/metr","json":"/entities/metr.json","markdown":"/entities/metr.md"},{"id":"claude","name":"Claude","kind":"model","context":"Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.","referenceUrls":["https://claude.com/product/overview","https://www.anthropic.com/news","https://www.anthropic.com/research/team/economics"],"path":"/entities/claude","json":"/entities/claude.json","markdown":"/entities/claude.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"fdfbe7fd4e4602851d730030a220e6e3db01c14ccca0897b105b6d7dc7780d03","revision":"9c95fbd9-c3d3-4491-586a-e8a675e73b72","revisionKind":"publication-id","title":"Epoch AI introduces MirrorCode for testing long-horizon coding tasks","url":"https://escapemeridian.com/readings/epoch-ai-introduces-mirrorcode-for-testing-long-horizon-coding-fdfbe7fd"},"source":{"type":"source","id":"https://epoch.ai/MirrorCode","revision":"7a0ed674c0e8e674ede6dd31eaa915b00803b82d1c62dc3f69dcc20a220c7e95","revisionKind":"sha256-source-text","title":"MirrorCode: What's the largest software project AI can complete on its own?","url":"https://epoch.ai/MirrorCode"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"fdfbe7fd4e4602851d730030a220e6e3db01c14ccca0897b105b6d7dc7780d03","title":"Epoch AI introduces MirrorCode for testing long-horizon coding tasks","argument":"Epoch AI’s page, published June 26, 2026, presents MirrorCode, a benchmark co-developed with METR to test whether AI models can reimplement entire programs without access to their original source code. The models’ solutions must match the programs’ outputs on end-to-end tests, including held-out tests. Epoch AI says Claude Opus 4.7 reimplemented gotree, a bioinformatics toolkit, in 14 hours for $251; the best implementations passed 2000/2001 tests but missed one edge case, so Epoch AI calls the result near-perfect rather than a complete solution. The example illustrates the kind of long task MirrorCode measures, not a general measure of software reliability. Epoch AI says open-source targets may have appeared in models’ pre-training, which could inflate scores. Its memorization screen results suggest performance was not dominated by memorization, but Epoch AI cannot rule out a contribution from memorization. It expects MirrorCode capabilities to generalize to unseen codebases. Whether they do remains unknown; the page describes this as an expectation, not a demonstrated result. The source’s central point is that AI can already perform some difficult, extended coding tasks, while the benchmark’s results still carry a data-contamination caveat.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 24, 2026; originally published June 26, 2026. Epoch AI says it cannot rule out that memorization contributes to performance; the page does not establish how much it contributes.","speaker":{"name":"Epoch AI","roleAtSource":"Publisher"},"source":{"title":"MirrorCode: What's the largest software project AI can complete on its own?","publisher":"Epoch AI","format":"Article","url":"https://epoch.ai/MirrorCode","publishedAt":"2026-06-26T00:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Epoch AI says MirrorCode tests AI models by asking them to reimplement entire programs without access to the original source code, with solutions assessed against end-to-end tests.","excerpt":"In a MirrorCode task, AI models are tasked with reimplementing an entire program end-to-end, without access to the original source code. AI-generated solutions must match the original program’s output exactly on end-to-end tests, including held-out tests.","url":"https://epoch.ai/MirrorCode","relationship":"Source-stated"},{"text":"Epoch AI reports that Claude Opus 4.7 reimplemented gotree in 14 hours for $251, and characterizes the result as near-perfect despite one failed edge-case test.","excerpt":"Opus 4.7 solved it in 14 hours, costing $251.","url":"https://epoch.ai/MirrorCode","relationship":"Source-stated"},{"text":"Epoch AI says possible data contamination could inflate performance, but it cannot rule out that memorization contributes to results; it expects the measured capabilities to generalize to unseen codebases.","excerpt":"This suggests that the results were not dominated by memorization, but we cannot rule out the possibility that memorization contributes to AI performance. Overall, we expect that the capabilities measured by MirrorCode would generalize to an unseen codebase.","url":"https://epoch.ai/MirrorCode","relationship":"Source-stated"},{"text":"Escape Meridian reads the reported gotree result as an example of a long coding task completed nearly, but not fully, to the benchmark’s standard.","excerpt":"The best-scoring AI gotree implementations passed 2000/2001 tests, but failed a single edge-case test for a niche command to manipulate date annotations. Consequently, they do not strictly solve the task to 100% completion, but we consider the reimplementation near-perfect, covering essentially all","url":"https://epoch.ai/MirrorCode","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/MirrorCode","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would MirrorCode performance generalize to codebases that were not in a model’s pre-training?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Epoch AI introduces MirrorCode for testing long-horizon coding tasks","summary":"Epoch AI’s page, published June 26, 2026, presents MirrorCode, a benchmark co-developed with METR to test whether AI models can reimplement entire programs without access to their original source code. The models’ solutions must match the programs’ outputs on end-to-end tests, including held-out tests. Epoch AI says Claude Opus 4.7 reimplemented gotree, a bioinformatics toolkit, in 14 hours for $251; the best implementations passed 2000/2001 tests but missed one edge case, so Epoch AI calls the result near-perfect rather than a complete solution. The example illustrates the kind of long task MirrorCode measures, not a general measure of software reliability. Epoch AI says open-source targets may have appeared in models’ pre-training, which could inflate scores. Its memorization screen results suggest performance was not dominated by memorization, but Epoch AI cannot rule out a contribution from memorization. It expects MirrorCode capabilities to generalize to unseen codebases. Whether they do remains unknown; the page describes this as an expectation, not a demonstrated result. The source’s central point is that AI can already perform some difficult, extended coding tasks, while the benchmark’s results still carry a data-contamination caveat.","question":"How well would MirrorCode performance generalize to codebases that were not in a model’s pre-training?","claims":[{"text":"Epoch AI says MirrorCode tests AI models by asking them to reimplement entire programs without access to the original source code, with solutions assessed against end-to-end tests.","excerpt":"In a MirrorCode task, AI models are tasked with reimplementing an entire program end-to-end, without access to the original source code. AI-generated solutions must match the original program’s output exactly on end-to-end tests, including held-out tests.","support":"source-stated"},{"text":"Epoch AI reports that Claude Opus 4.7 reimplemented gotree in 14 hours for $251, and characterizes the result as near-perfect despite one failed edge-case test.","excerpt":"Opus 4.7 solved it in 14 hours, costing $251.","support":"source-stated"},{"text":"Epoch AI says possible data contamination could inflate performance, but it cannot rule out that memorization contributes to results; it expects the measured capabilities to generalize to unseen codebases.","excerpt":"This suggests that the results were not dominated by memorization, but we cannot rule out the possibility that memorization contributes to AI performance. Overall, we expect that the capabilities measured by MirrorCode would generalize to an unseen codebase.","support":"source-stated"},{"text":"Escape Meridian reads the reported gotree result as an example of a long coding task completed nearly, but not fully, to the benchmark’s standard.","excerpt":"The best-scoring AI gotree implementations passed 2000/2001 tests, but failed a single edge-case test for a niche command to manipulate date annotations. Consequently, they do not strictly solve the task to 100% completion, but we consider the reimplementation near-perfect, covering essentially all","support":"interpretation"}],"limitations":["Text as retrieved on October 24, 2026; originally published June 26, 2026.","Epoch AI says it cannot rule out that memorization contributes to performance; the page does not establish how much it contributes."],"mediaId":null,"correctionReason":null,"id":"fdfbe7fd4e4602851d730030a220e6e3db01c14ccca0897b105b6d7dc7780d03","publicationId":"9c95fbd9-c3d3-4491-586a-e8a675e73b72","version":1,"sourceUrl":"https://epoch.ai/MirrorCode","sourceTitle":"MirrorCode: What's the largest software project AI can complete on its own?","publisher":"Epoch AI","publishedAt":"2026-06-26T00:00:00.000Z","originalPublished":{"value":"2026-06-26T00:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:13:22.796Z","createdAt":"2026-11-06T11:33:23.471Z","historical":true,"entityIds":["epoch-ai"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://epoch.ai/MirrorCode","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"e0a14cef87c810978e6f2de684ee20507b15adff55b84541747d91b5c8b77ed2","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-24T05:09:55.394Z"},"provenance":{"textHash":"7a0ed674c0e8e674ede6dd31eaa915b00803b82d1c62dc3f69dcc20a220c7e95","observationId":"5d07d8826cde2ba93d545bb1ee7b70ec72d2fa7a69a316b32065b33e44b60154","retrievedAt":"2026-10-24T05:11:37.128Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"12fb8282-a7a2-b678-7490-720c257d3bcf"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"fe450b69-a5d6-f0d6-7884-d684a2f55d57"}},"audit":{"status":"passed","reviewedAt":"2026-11-06T11:28:39.893Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"9c95fbd9-c3d3-4491-586a-e8a675e73b72","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the benchmark, the gotree result, the contamination caveat and the generalization expectation to Epoch AI, and correctly notes co-development with METR. The evidence is a single publisher page with its own argument, not a listing.","excerpt":"MirrorCode is our benchmark, co-developed with METR, to test AI models on long-horizon coding tasks."},{"dimension":"summary","status":"pass","explanation":"The summary leads with the actor and the development: a benchmark that asks AI to reimplement whole programs without the source code. It explains the gotree example in plain terms and keeps the near-perfect versus complete distinction. It also preserves the contamination caveat and the expectation, which is not a demonstrated result. Benchmark and pre-training are defined beside the reading. Long-horizon is conveyed as long tasks. The question follows from the source's stated uncertainty.","excerpt":"Because MirrorCode tasks involve reimplementing open-source programs, AI models are likely to have seen the original codebases in pretraining."},{"dimension":"date","status":"pass","explanation":"The reading's June 26, 2026 matches the UTC day of evidence.originalPublished. The leaderboard note naming newer models (Claude Opus 5.5 and others) may be a later page addition, but the reading neither uses nor relies on it. There is no hindsight framing. The generalization claim is presented as an expectation. The evidence kind is publisher-page, so no archive disclosure is needed.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim matches the source text. The source supports the 14 hours and $251, the 2000/2001 tests with one edge-case failure called near-perfect, and the memorization screen results. It also supports the expectation of generalization to unseen codebases. The interpretation claim is labeled and carries its excerpt. The caution that the example is not a general reliability measure adds no unsupported benefit or cost.","excerpt":"Consequently, they do not strictly solve the task to 100% completion, but we consider the reimplementation near-perfect, covering essentially all scoped functionality."},{"dimension":"media","status":"pass","explanation":"No media was selected and no verified media candidates were supplied, so there is no media identity or relevance issue.","excerpt":null}],"createdAt":"2026-11-06T11:28:39.893Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}