# One Confidence Threshold, Unequal Coverage: Language Gaps in Typed Decision Interfaces and How Translating Inputs Reduces Them [Canonical reading](https://escapemeridian.com/readings/one-confidence-threshold-unequal-coverage-language-gaps-in-bc9d871e) The authors argue that one confidence threshold can leave requests in languages other than English less likely to be handled automatically, even when the classifier answers them correctly. Typed decision interfaces return a probability for each option in a fixed multiple-choice question; requests below the threshold go to a person. In a confirmatory study, three vendors classified 1,000 MASSIVE items in 51 languages at a threshold of 0.9. Among requests answered correctly in both English and another language, English requests cleared it more often by an average of 8.6–11.7 percentage points. Observed coverage was lower in every other language. The authors say the requests handled automatically were rarely wrong, but matching languages for accuracy did not remove the coverage gap, and the standard calibration metric ECE did not detect it on one interface. Exploratory analyses found English still had higher coverage with language-specific thresholds fitted to the same 5% error target. Translating inputs into English nearly closed the gap at 0.9, but not at that error target. The authors recommend reporting per-language coverage at the operating threshold. The abstract does not establish whether these findings extend beyond the studied interfaces and items. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 17, 2026 (2026-10-17). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T03:20:17.016Z. First observed: 2026-10-30T06:41:22.806Z. Retrieved: 2026-10-31T10:04:28.966Z. - [One Confidence Threshold, Unequal Coverage: Language Gaps in Typed Decision Interfaces and How Translating Inputs Reduces Them](https://arxiv.org/abs/2610.37688) ## Limits The supplied evidence is the abstract, which describes three vendors' interfaces and MASSIVE items; it does not establish whether the findings generalize to other classifiers or decision tasks. The abstract labels the analyses using language-specific thresholds as exploratory, rather than part of the confirmatory study. ## Open question Would the reported coverage gaps persist in other classifiers or decision tasks? ## Source-stated The authors report that English requests cleared the confidence threshold more often than other-language requests, even when both versions were answered correctly. > Even when a request is answered correctly in both English and another language, the English version clears the threshold more often, by 8.6-11.7 percentage points on average. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated In the confirmatory study, three vendors classified 1,000 MASSIVE items in 51 languages using a threshold of 0.9. > In a confirmatory study with a written analysis plan, three vendors' interfaces classified 1,000 MASSIVE items, each in 51 languages, with a threshold of 0.9. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated Observed coverage—the share of requests handled automatically—was lower in every other language than in English; the authors say requests handled by the gate were rarely wrong. > Observed coverage, the share of requests handled automatically, is lower than English's in every other language, while the requests the gate handles are rarely wrong. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated The authors report that matching languages for accuracy did not remove the coverage gap, and ECE did not detect it on one interface. > The gap remains when every language is reweighted to the same accuracy, and a standard calibration metric (ECE) does not detect it on one interface. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated The authors report that translating inputs into English nearly closed the gap at the shared threshold, but English remained ahead under the specified error target. > Translating inputs into English nearly closes the gap at 0.9 but leaves English ahead at that error target. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Escape Meridian interpretation The abstract’s study scope leaves open whether the reported pattern extends to other classifiers or decision tasks. > three vendors' interfaces classified 1,000 MASSIVE items, each in 51 languages - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source and contributor context Yao, Junwei, Zhou, Yongxi, Yang, Zike, Wang, Zhijun, Liu, Jianan, Chen, Yanxian, Ye, Wenbo · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: db592510-0eb7-4e8c-8ede-aaf1a95f140a. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T03:20:17.016Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 52aa09f3f16e73e2448dff6b03b1b8696f83fb6a0d5522e9432715c230853079. Source text hash: c3c1511d4e638a3dcfa3e176c532a0c10a0c9816c0c5f175ff8afd03541954cf. ## Revision and source-check history ### Version 1 — 2026-11-13T03:20:17.016Z The authors argue that one confidence threshold can leave requests in languages other than English less likely to be handled automatically, even when the classifier answers them correctly. Typed decision interfaces return a probability for each option in a fixed multiple-choice question; requests below the threshold go to a person. In a confirmatory study, three vendors classified 1,000 MASSIVE items in 51 languages at a threshold of 0.9. Among requests answered correctly in both English and another language, English requests cleared it more often by an average of 8.6–11.7 percentage points. Observed coverage was lower in every other language. The authors say the requests handled automatically were rarely wrong, but matching languages for accuracy did not remove the coverage gap, and the standard calibration metric ECE did not detect it on one interface. Exploratory analyses found English still had higher coverage with language-specific thresholds fitted to the same 5% error target. Translating inputs into English nearly closed the gap at 0.9, but not at that error target. The authors recommend reporting per-language coverage at the operating threshold. The abstract does not establish whether these findings extend beyond the studied interfaces and items. ## Original source arXiv cs.AI · Article Original source date: Oct 17, 2026 (2026-10-17). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-13T03:20:17.016Z. First observed: 2026-10-30T06:41:22.806Z. Retrieved: 2026-10-31T10:04:28.966Z. - [One Confidence Threshold, Unequal Coverage: Language Gaps in Typed Decision Interfaces and How Translating Inputs Reduces Them](https://arxiv.org/abs/2610.37688) ## Limits The supplied evidence is the abstract, which describes three vendors' interfaces and MASSIVE items; it does not establish whether the findings generalize to other classifiers or decision tasks. The abstract labels the analyses using language-specific thresholds as exploratory, rather than part of the confirmatory study. ## Open question Would the reported coverage gaps persist in other classifiers or decision tasks? ## Source-stated The authors report that English requests cleared the confidence threshold more often than other-language requests, even when both versions were answered correctly. > Even when a request is answered correctly in both English and another language, the English version clears the threshold more often, by 8.6-11.7 percentage points on average. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated In the confirmatory study, three vendors classified 1,000 MASSIVE items in 51 languages using a threshold of 0.9. > In a confirmatory study with a written analysis plan, three vendors' interfaces classified 1,000 MASSIVE items, each in 51 languages, with a threshold of 0.9. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated Observed coverage—the share of requests handled automatically—was lower in every other language than in English; the authors say requests handled by the gate were rarely wrong. > Observed coverage, the share of requests handled automatically, is lower than English's in every other language, while the requests the gate handles are rarely wrong. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated The authors report that matching languages for accuracy did not remove the coverage gap, and ECE did not detect it on one interface. > The gap remains when every language is reweighted to the same accuracy, and a standard calibration metric (ECE) does not detect it on one interface. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source-stated The authors report that translating inputs into English nearly closed the gap at the shared threshold, but English remained ahead under the specified error target. > Translating inputs into English nearly closes the gap at 0.9 but leaves English ahead at that error target. - [Supporting source](https://arxiv.org/abs/2610.37688) ## Escape Meridian interpretation The abstract’s study scope leaves open whether the reported pattern extends to other classifiers or decision tasks. > three vendors' interfaces classified 1,000 MASSIVE items, each in 51 languages - [Supporting source](https://arxiv.org/abs/2610.37688) ## Source and contributor context Yao, Junwei, Zhou, Yongxi, Yang, Zike, Wang, Zhijun, Liu, Jianan, Chen, Yanxian, Ye, Wenbo · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: db592510-0eb7-4e8c-8ede-aaf1a95f140a. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: passed by anthropic/claude-opus-5.5 on 2026-11-13T03:20:17.016Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 52aa09f3f16e73e2448dff6b03b1b8696f83fb6a0d5522e9432715c230853079. Source text hash: c3c1511d4e638a3dcfa3e176c532a0c10a0c9816c0c5f175ff8afd03541954cf. ### passed — 2026-11-13T03:20:17.016Z — anthropic/claude-opus-5.5 Publication: db592510-0eb7-4e8c-8ede-aaf1a95f140a attribution — pass: The draft attributes the study, findings and recommendation to the paper's authors on arXiv, as shown on the page. It does not present the source's claims as established fact or as Escape Meridian's opinion. > Authors:Junwei Yao, Yongxi Zhou, Zike Yang, Zhijun Wang, Jianan Liu, Yanxian Chen, Wenbo Ye summary — pass: The title matches the source. The summary explains the central terms in plain language: the confidence threshold, typed decision interfaces, routing to a person, and coverage. It keeps the 0.9 threshold, the confirmatory versus exploratory labels and the recommendation. MASSIVE and ECE are named but not expanded, and the source does not explain them either. Leaving out the African-language result does not distort the meaning. The question follows from the stated scope without leading the reader. > We recommend reporting per-language coverage at the operating threshold. date — pass: The originalPublished value of 2026-10-17 matches the v1 submission date on arXiv. The later feed date is not used. > \[Submitted on 17 Oct 2026\] support — pass: Each claim tracks the abstract closely. The draft says matching languages for accuracy did not remove the gap, which fairly paraphrases reweighting to the same accuracy. The translation result keeps its condition about the error target. The generalization point is labeled as an interpretation and limited to the stated scope. The draft adds no stakeholder benefits or costs beyond what the source supports. > Translating inputs into English nearly closes the gap at 0.9 but leaves English ahead at that error target. media — pass: The selected media is the verified link to the original arXiv abstract page. It is the most direct way to reach the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/bc9d871e7e886225bf9714c8ebb74b13c569036cf0454d484e60c50bbc9bc39b/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/one-confidence-threshold-unequal-coverage-language-gaps-in-bc9d871e.json)