Afolabi, Zainab, Kozodoi, Nikita, Butler, Jack, Khalidson, Babs · Nov 4, 2026
arXiv cs.AI: NaijaFacts: A Framework for Disentangling Language from Knowledge in Facts about Nigeria
Markdown · JSON · Original source
Authors Zainab Afolabi, Nikita Kozodoi, Jack Butler and Babs Khalidson introduce NaijaFacts, a of factual and statistical questions about Nigeria. They say native speakers wrote the questions, which are grounded in official statistics and available in English, Nigerian Pidgin, Yoruba, Igbo and Hausa. Similar questions about Nigeria, the UK and the US in different languages let the authors distinguish a language effect—when a model knows a fact but answers less well in a low-resource language—from a knowledge effect, when the fact is absent across languages. Across eight frontier and Nigeria-adapted models, the authors report that the knowledge effect dominates: English-query accuracy is 20% for Nigeria, compared with 49–50% for the UK and US. They report a smaller language effect that tracks web representation; accuracy falls from English to Yoruba by 9 points on Nigeria questions and up to 24 points on UK and US questions. The authors also report that scale does not close the gap: the strongest models score 66–79% on UK and US questions, but none exceed 31% on Nigerian statistics. In an extension covering all 54 African countries with their best-performing , they report 96% accuracy on culture and 98% on politics, against 26% on statistics. The results frame missing country-specific knowledge—not only language—as a major gap in the tested models.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T09:17:16.643Z. First observed: 2026-11-05T05:06:30.927Z. Retrieved: 2026-11-09T09:10:59.512Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the page and abstract, which do not identify the eight models or provide question counts or detailed evaluation methods.
Open question
How much do the reported accuracy gaps vary across the questions and models used in NaijaFacts?
Source-stated
The authors introduce a Nigeria-focused benchmark with questions in five languages, written by native speakers and grounded in official statistics.
a benchmark of factual and statistical questions about Nigeria, authored by native speakers and grounded in authoritative official statistics, with every question available in English, Nigerian Pidgin, \yoruba, Igbo, and Hausa
Source-stated
The authors use comparable questions about Nigeria, the UK and the US to distinguish a language effect from a knowledge effect.
yields a country by language matrix that disentangles a language effect (the model knows the fact but degrades in a low-resource language) from a knowledge effect (the fact is absent in every language).
Source-stated
Across eight models, the authors report that English-query accuracy is 20% for Nigeria and 49–50% for the UK and US, with the knowledge effect dominating.
Across eight frontier and Nigeria-adapted models, the knowledge effect dominates: English-query accuracy is 49--50\% for the US/UK but 20\% for Nigeria.
Source-stated
The authors report a smaller language effect that tracks web representation, with accuracy falling from English to Yoruba by 9 points on Nigeria questions and up to 24 points on UK and US questions.
The language effect is smaller and tracks web representation, degrading from English to \yoruba{} (9 points on Nigeria, up to 24 on US/UK questions).
Source-stated
The authors report that none of the tested models exceed 31% on Nigerian statistics, while the strongest score 66–79% on UK and US questions.
the strongest models reach 66--79\% on the UK and the US but none exceed 31\% on Nigerian statistics.
Source-stated
In an extension covering 54 African countries, the authors report much higher accuracy on culture and politics than on statistics.
Extending to all 54 African countries with our best performing frontier model shows the same profile continent-wide, 96\% on culture and 98\% on politics against 26\% on statistics
Source and contributor context
Afolabi, Zainab, Kozodoi, Nikita, Butler, Jack, Khalidson, Babs · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 8ac43c37-eb16-4ead-8d45-5ce7d8e18c15.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-09T09:17:16.643Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 974669458fd80c8a26f25d109deb2ac0647fdc28c7f04530f2c6e09784520ba1. Source text hash: 34027fdf45aede90343f144e175ee79c79b03dd4fdb7ce9aa7b6f5334279d45e.
Revision and source-check history
Version 1 · 2026-11-09T09:17:16.643Z
arXiv cs.AI: NaijaFacts: A Framework for Disentangling Language from Knowledge in Facts about Nigeria
Authors Zainab Afolabi, Nikita Kozodoi, Jack Butler and Babs Khalidson introduce NaijaFacts, a of factual and statistical questions about Nigeria. They say native speakers wrote the questions, which are grounded in official statistics and available in English, Nigerian Pidgin, Yoruba, Igbo and Hausa. Similar questions about Nigeria, the UK and the US in different languages let the authors distinguish a language effect—when a model knows a fact but answers less well in a low-resource language—from a knowledge effect, when the fact is absent across languages. Across eight frontier and Nigeria-adapted models, the authors report that the knowledge effect dominates: English-query accuracy is 20% for Nigeria, compared with 49–50% for the UK and US. They report a smaller language effect that tracks web representation; accuracy falls from English to Yoruba by 9 points on Nigeria questions and up to 24 points on UK and US questions. The authors also report that scale does not close the gap: the strongest models score 66–79% on UK and US questions, but none exceed 31% on Nigerian statistics. In an extension covering all 54 African countries with their best-performing , they report 96% accuracy on culture and 98% on politics, against 26% on statistics. The results frame missing country-specific knowledge—not only language—as a major gap in the tested models.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T09:17:16.643Z. First observed: 2026-11-05T05:06:30.927Z. Retrieved: 2026-11-09T09:10:59.512Z.
Limits
The supplied evidence is the page and abstract, which do not identify the eight models or provide question counts or detailed evaluation methods.
Open question
How much do the reported accuracy gaps vary across the questions and models used in NaijaFacts?
Source-stated
The authors introduce a Nigeria-focused benchmark with questions in five languages, written by native speakers and grounded in official statistics.
a benchmark of factual and statistical questions about Nigeria, authored by native speakers and grounded in authoritative official statistics, with every question available in English, Nigerian Pidgin, \yoruba, Igbo, and Hausa
Source-stated
The authors use comparable questions about Nigeria, the UK and the US to distinguish a language effect from a knowledge effect.
yields a country by language matrix that disentangles a language effect (the model knows the fact but degrades in a low-resource language) from a knowledge effect (the fact is absent in every language).
Source-stated
Across eight models, the authors report that English-query accuracy is 20% for Nigeria and 49–50% for the UK and US, with the knowledge effect dominating.
Across eight frontier and Nigeria-adapted models, the knowledge effect dominates: English-query accuracy is 49--50\% for the US/UK but 20\% for Nigeria.
Source-stated
The authors report a smaller language effect that tracks web representation, with accuracy falling from English to Yoruba by 9 points on Nigeria questions and up to 24 points on UK and US questions.
The language effect is smaller and tracks web representation, degrading from English to \yoruba{} (9 points on Nigeria, up to 24 on US/UK questions).
Source-stated
The authors report that none of the tested models exceed 31% on Nigerian statistics, while the strongest score 66–79% on UK and US questions.
the strongest models reach 66--79\% on the UK and the US but none exceed 31\% on Nigerian statistics.
Source-stated
In an extension covering 54 African countries, the authors report much higher accuracy on culture and politics than on statistics.
Extending to all 54 African countries with our best performing frontier model shows the same profile continent-wide, 96\% on culture and 98\% on politics against 26\% on statistics
Source and contributor context
Afolabi, Zainab, Kozodoi, Nikita, Butler, Jack, Khalidson, Babs · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 8ac43c37-eb16-4ead-8d45-5ce7d8e18c15.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-09T09:17:16.643Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 974669458fd80c8a26f25d109deb2ac0647fdc28c7f04530f2c6e09784520ba1. Source text hash: 34027fdf45aede90343f144e175ee79c79b03dd4fdb7ce9aa7b6f5334279d45e.
passed · 2026-11-09T09:17:16.643Z · anthropic/claude-opus-5.5
Publication: 8ac43c37-eb16-4ead-8d45-5ce7d8e18c15
attribution — pass: The reading names the four listed authors and attributes the benchmark design, findings and interpretation to them throughout (introduce, say, report). It does not present the results as established facts or as Escape Meridian's view.
Authors:Zainab Afolabi, Nikita Kozodoi, Jack Butler, Babs Khalidson
summary — pass: The reading leads with the actors and the new benchmark. It explains the language-versus-knowledge distinction in plain terms and accurately conveys the main figures, the 'scale does not help' finding and the continent-wide extension. Terms such as frontier models and web representation are left for tooltips, but they do not block understanding. The closing framing matches the abstract's 'knowledge effect dominates.' The question about variance across questions and models is a fair open issue. A minor optional repair would add the authors' explicit conclusion that the gap is not specific to Nigeria.
Across eight frontier and Nigeria-adapted models, the knowledge effect dominates
date — pass: originalPublished is 2026-11-04. This matches the arXiv submission date for v1, not the later feed date of 2026-11-05.
[Submitted on 4 Nov 2026]
support — pass: Every claim's excerpt is a verbatim match to the abstract, and the numbers are reproduced correctly: 20% vs 49–50%, 9 and up to 24 points, 66–79% vs 31%, and 96%/98% vs 26%. The reading invents no benefits, costs or forecasts. The limitation correctly notes that the abstract does not identify the models, the question counts or the detailed methods.
the strongest models reach 66--79\% on the UK and the US but none exceed 31\% on Nigerian statistics.
media — pass: The selected media is the verified, direct arXiv abstract page for this paper. It matches the source URL and is the most direct way to reach the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv