{"id":"2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee","title":"arXiv cs.AI: Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","argument":"The authors of “Who Can Evaluate AI Independently?” argue that taking part in an AI evaluation does not itself give participants authority over its design, execution or supporting resources. They introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms of access, whether it suits the evaluation’s purpose and what autonomy it supports. Applying the framework to African AI evaluation projects and joint testing exercises by an international network of AI Safety Institutes, the authors describe African actors contributing data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks. In the joint exercises, the institutes that led each exercise also ran the models; Kenya moved from annotation to task design, with no documented model execution. The authors’ central argument is that participation alone does not confer evaluative authority. They argue that credible governance requires access alongside control over priorities, methods, publication and continuity.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"This reading is based on the arXiv abstract; the supplied evidence does not include the paper’s detailed methods or case analyses. The abstract reports no documented model execution by Kenya in the joint testing exercises; that does not establish that no execution occurred.","speaker":{"name":"Adan, Sumaya Nur, Wiaterek, Joanna, Abdelrhim, Nourhan, Reis, Sydney, Trager, Robert F., Rosman, Benjamin, Jirotka, Marina","roleAtSource":"Article author"},"source":{"title":"Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.08134","publishedAt":"2026-11-05","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors argue that participation in an AI evaluation does not itself grant authority over its design, execution or supporting resources.","excerpt":"taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"},{"text":"The authors introduce PAAI to examine control of evaluation infrastructure, access terms, adequacy for purpose and supported autonomy.","excerpt":"we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"},{"text":"In the cases described, African actors often contribute evaluation inputs while external institutions retain authority over infrastructure and evaluation frameworks.","excerpt":"African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"},{"text":"The authors report that, in the joint testing exercises, the leading institutes also ran the models, while Kenya had no documented model execution.","excerpt":"the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08134","checkedAt":"2026-11-09T01:56:15.060Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence do the authors use in the full paper to distinguish partial independence from authority over evaluation infrastructure?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","summary":"The authors of “Who Can Evaluate AI Independently?” argue that taking part in an AI evaluation does not itself give participants authority over its design, execution or supporting resources. They introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms of access, whether it suits the evaluation’s purpose and what autonomy it supports. Applying the framework to African AI evaluation projects and joint testing exercises by an international network of AI Safety Institutes, the authors describe African actors contributing data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks. In the joint exercises, the institutes that led each exercise also ran the models; Kenya moved from annotation to task design, with no documented model execution. The authors’ central argument is that participation alone does not confer evaluative authority. They argue that credible governance requires access alongside control over priorities, methods, publication and continuity.","question":"What evidence do the authors use in the full paper to distinguish partial independence from authority over evaluation infrastructure?","claims":[{"text":"The authors argue that participation in an AI evaluation does not itself grant authority over its design, execution or supporting resources.","excerpt":"taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on","support":"source-stated"},{"text":"The authors introduce PAAI to examine control of evaluation infrastructure, access terms, adequacy for purpose and supported autonomy.","excerpt":"we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure","support":"source-stated"},{"text":"In the cases described, African actors often contribute evaluation inputs while external institutions retain authority over infrastructure and evaluation frameworks.","excerpt":"African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks","support":"source-stated"},{"text":"The authors report that, in the joint testing exercises, the leading institutes also ran the models, while Kenya had no documented model execution.","excerpt":"the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution","support":"source-stated"}],"limitations":["This reading is based on the arXiv abstract; the supplied evidence does not include the paper’s detailed methods or case analyses.","The abstract reports no documented model execution by Kenya in the joint testing exercises; that does not establish that no execution occurred."],"mediaId":"29e0a9cad31f3554581c259065cd748b75722a81c6ad3a86261ff1111b522a6d","correctionReason":null,"id":"2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee","publicationId":"9a9fe201-4d0a-4329-818b-4c4af520e91e","version":1,"sourceUrl":"https://arxiv.org/abs/2611.08134","sourceTitle":"Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-05","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:03:31.273Z","createdAt":"2026-11-09T01:40:19.619Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"29e0a9cad31f3554581c259065cd748b75722a81c6ad3a86261ff1111b522a6d","kind":"article","url":"https://arxiv.org/abs/2611.08134","sourceUrl":"https://arxiv.org/abs/2611.08134","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T01:39:28.085Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08134","checkedAt":"2026-11-09T01:56:15.060Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v5","mediaId":"29e0a9cad31f3554581c259065cd748b75722a81c6ad3a86261ff1111b522a6d","confidence":0.9100000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"a88890cc5feceff4e91da6af69aa7328a590aa8adb11588db8f9cf3a82f3a7a9","decidedAt":"2026-11-09T01:39:29.813Z"},"provenance":{"textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observationId":"39d4a06c5e3514d0e8e40e884958a31e3a3df782d13f1c5da660f25a5c27fe59","retrievedAt":"2026-11-09T01:39:27.788Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"e583cca4-1e22-488c-8b1f-b334039bd47d"},"reviewer":{"model":"gpt-6-luna","runId":"7f3f7857-23e2-4f97-8032-ec8c3b64309b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T01:40:19.619Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Adan, Sumaya Nur","url":null,"kind":"unknown"},{"name":"Wiaterek, Joanna","url":null,"kind":"unknown"},{"name":"Abdelrhim, Nourhan","url":null,"kind":"unknown"},{"name":"Reis, Sydney","url":null,"kind":"unknown"},{"name":"Trager, Robert F.","url":null,"kind":"unknown"},{"name":"Rosman, Benjamin","url":null,"kind":"unknown"},{"name":"Jirotka, Marina","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Adan, Sumaya Nur","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Adan, Sumaya Nur","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Wiaterek, Joanna","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wiaterek, Joanna","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Abdelrhim, Nourhan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Abdelrhim, Nourhan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Reis, Sydney","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Reis, Sydney","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Trager, Robert F.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Trager, Robert F.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Rosman, Benjamin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rosman, Benjamin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Jirotka, Marina","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jirotka, Marina","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.08134","snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-who-can-evaluate-ai-independently-a-framework-for-2fd1bfc3","formats":{"html":"/readings/arxiv-cs-ai-who-can-evaluate-ai-independently-a-framework-for-2fd1bfc3","markdown":"/readings/arxiv-cs-ai-who-can-evaluate-ai-independently-a-framework-for-2fd1bfc3.md","json":"/readings/arxiv-cs-ai-who-can-evaluate-ai-independently-a-framework-for-2fd1bfc3.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee","revision":"9a9fe201-4d0a-4329-818b-4c4af520e91e","revisionKind":"publication-id","title":"arXiv cs.AI: Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","url":"https://escapemeridian.com/readings/arxiv-cs-ai-who-can-evaluate-ai-independently-a-framework-for-2fd1bfc3"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.08134","revision":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","revisionKind":"sha256-source-text","title":"Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","url":"https://arxiv.org/abs/2611.08134"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee","title":"arXiv cs.AI: Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","argument":"The authors of “Who Can Evaluate AI Independently?” argue that taking part in an AI evaluation does not itself give participants authority over its design, execution or supporting resources. They introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms of access, whether it suits the evaluation’s purpose and what autonomy it supports. Applying the framework to African AI evaluation projects and joint testing exercises by an international network of AI Safety Institutes, the authors describe African actors contributing data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks. In the joint exercises, the institutes that led each exercise also ran the models; Kenya moved from annotation to task design, with no documented model execution. The authors’ central argument is that participation alone does not confer evaluative authority. They argue that credible governance requires access alongside control over priorities, methods, publication and continuity.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"This reading is based on the arXiv abstract; the supplied evidence does not include the paper’s detailed methods or case analyses. The abstract reports no documented model execution by Kenya in the joint testing exercises; that does not establish that no execution occurred.","speaker":{"name":"Adan, Sumaya Nur, Wiaterek, Joanna, Abdelrhim, Nourhan, Reis, Sydney, Trager, Robert F., Rosman, Benjamin, Jirotka, Marina","roleAtSource":"Article author"},"source":{"title":"Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.08134","publishedAt":"2026-11-05","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors argue that participation in an AI evaluation does not itself grant authority over its design, execution or supporting resources.","excerpt":"taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"},{"text":"The authors introduce PAAI to examine control of evaluation infrastructure, access terms, adequacy for purpose and supported autonomy.","excerpt":"we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"},{"text":"In the cases described, African actors often contribute evaluation inputs while external institutions retain authority over infrastructure and evaluation frameworks.","excerpt":"African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"},{"text":"The authors report that, in the joint testing exercises, the leading institutes also ran the models, while Kenya had no documented model execution.","excerpt":"the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution","url":"https://arxiv.org/abs/2611.08134","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08134","checkedAt":"2026-11-09T01:56:15.060Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evidence do the authors use in the full paper to distinguish partial independence from authority over evaluation infrastructure?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","summary":"The authors of “Who Can Evaluate AI Independently?” argue that taking part in an AI evaluation does not itself give participants authority over its design, execution or supporting resources. They introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms of access, whether it suits the evaluation’s purpose and what autonomy it supports. Applying the framework to African AI evaluation projects and joint testing exercises by an international network of AI Safety Institutes, the authors describe African actors contributing data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks. In the joint exercises, the institutes that led each exercise also ran the models; Kenya moved from annotation to task design, with no documented model execution. The authors’ central argument is that participation alone does not confer evaluative authority. They argue that credible governance requires access alongside control over priorities, methods, publication and continuity.","question":"What evidence do the authors use in the full paper to distinguish partial independence from authority over evaluation infrastructure?","claims":[{"text":"The authors argue that participation in an AI evaluation does not itself grant authority over its design, execution or supporting resources.","excerpt":"taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on","support":"source-stated"},{"text":"The authors introduce PAAI to examine control of evaluation infrastructure, access terms, adequacy for purpose and supported autonomy.","excerpt":"we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure","support":"source-stated"},{"text":"In the cases described, African actors often contribute evaluation inputs while external institutions retain authority over infrastructure and evaluation frameworks.","excerpt":"African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks","support":"source-stated"},{"text":"The authors report that, in the joint testing exercises, the leading institutes also ran the models, while Kenya had no documented model execution.","excerpt":"the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution","support":"source-stated"}],"limitations":["This reading is based on the arXiv abstract; the supplied evidence does not include the paper’s detailed methods or case analyses.","The abstract reports no documented model execution by Kenya in the joint testing exercises; that does not establish that no execution occurred."],"mediaId":"29e0a9cad31f3554581c259065cd748b75722a81c6ad3a86261ff1111b522a6d","correctionReason":null,"id":"2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee","publicationId":"9a9fe201-4d0a-4329-818b-4c4af520e91e","version":1,"sourceUrl":"https://arxiv.org/abs/2611.08134","sourceTitle":"Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-05","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:03:31.273Z","createdAt":"2026-11-09T01:40:19.619Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"29e0a9cad31f3554581c259065cd748b75722a81c6ad3a86261ff1111b522a6d","kind":"article","url":"https://arxiv.org/abs/2611.08134","sourceUrl":"https://arxiv.org/abs/2611.08134","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T01:39:28.085Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08134","checkedAt":"2026-11-09T01:56:15.060Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v5","mediaId":"29e0a9cad31f3554581c259065cd748b75722a81c6ad3a86261ff1111b522a6d","confidence":0.9100000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"a88890cc5feceff4e91da6af69aa7328a590aa8adb11588db8f9cf3a82f3a7a9","decidedAt":"2026-11-09T01:39:29.813Z"},"provenance":{"textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observationId":"39d4a06c5e3514d0e8e40e884958a31e3a3df782d13f1c5da660f25a5c27fe59","retrievedAt":"2026-11-09T01:39:27.788Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"e583cca4-1e22-488c-8b1f-b334039bd47d"},"reviewer":{"model":"gpt-6-luna","runId":"7f3f7857-23e2-4f97-8032-ec8c3b64309b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T01:40:19.619Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Adan, Sumaya Nur","url":null,"kind":"unknown"},{"name":"Wiaterek, Joanna","url":null,"kind":"unknown"},{"name":"Abdelrhim, Nourhan","url":null,"kind":"unknown"},{"name":"Reis, Sydney","url":null,"kind":"unknown"},{"name":"Trager, Robert F.","url":null,"kind":"unknown"},{"name":"Rosman, Benjamin","url":null,"kind":"unknown"},{"name":"Jirotka, Marina","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Adan, Sumaya Nur","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Adan, Sumaya Nur","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Wiaterek, Joanna","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wiaterek, Joanna","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Abdelrhim, Nourhan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Abdelrhim, Nourhan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Reis, Sydney","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Reis, Sydney","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Trager, Robert F.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Trager, Robert F.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Rosman, Benjamin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rosman, Benjamin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}},{"actor":{"name":"Jirotka, Marina","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jirotka, Marina","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08134","locator":"head/meta[name=author|citation_author]"},"assertionId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T01:56:15.060Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","sourceUrl":"https://arxiv.org/abs/2611.08134","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.08134","snapshotId":"e50e9d482a7bf7eb855007924fecf0eee51c18a76efb9480bc7c600343785d16","contentSha256":"eb1245d681b31677e651ad2a84503a90222eb1eba13e8c0f87728f5aaa899cc9","textHash":"449bbf4c0a713c44c02bd19c6006195d63ed98acf95bdb73f139c6bdafcfe291","observedAt":"2026-11-09T01:56:15.060Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/2fd1bfc31bbdf76efaebc128aaeaf6d45eaccbccf68e513d43306a6f9e6f7cee/source"}}}],"audits":[{"publicationId":"9a9fe201-4d0a-4329-818b-4c4af520e91e","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft consistently attributes the arguments and findings to the paper's authors. The arXiv title and publisher label match the evidence. The paper's primary category is cs.CY, but it is cross-listed in cs.AI, so the label is acceptable.","excerpt":"Subjects:\nComputers and Society (cs.CY); Artificial Intelligence (cs.AI)"},{"dimension":"summary","status":"uncertain","explanation":"The draft omits the abstract's sharpest finding: design authority without infrastructure authority never yielded more than partial independence. It also leaves out the three named configurations. The question then relies on 'partial independence', which a reader cannot follow without that finding. Scope details are missing: twelve projects (2022-2025), ten institutes, mostly high-income members, and Kenya as the network's only African member, which links the two case studies. Evaluation infrastructure and AI Safety Institutes are not explained in plain terms. Smallest repair: add the partial-independence finding, the three configurations and Kenya's status.","excerpt":"design authority without infrastructure authority never yielded more than partial independence"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-05. This matches the v1 submission date shown on the arXiv page. The feed date of Nov 6 is later and is correctly not used.","excerpt":"[Submitted on 5 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim's excerpt is verbatim and supports it. The governance conclusion stays attributed to the authors as their argument. The limitation about undocumented execution properly avoids overclaiming. No unsupported benefits, costs or forecasts are added.","excerpt":"For evaluation to serve as a credible governance mechanism, access must come with control over priorities, methods, publication and continuity."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-09T01:40:19.619Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}