{"id":"f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1","title":"Khan and Misri test how many LLM-labelled rows a decision encoder needs","argument":"Mohammed Nayeem Khan and Aliyas Ahmad Misri test how training-set size and label errors affect Laya, a decision encoder that selects among named options without generating text. In 19 fine-tunes, accuracy on held-out items from the same generator rose from 69.4% before fine-tuning to 86.9% using all 2,818 training rows, but a saturating fit with bootstrap intervals could not pin down the curve’s plateau. The authors report that a single run with 10% symmetric label noise showed no detectable change, while 20% and 40% cost 3.4 and 8.7 points. Confusable-pair noise at 20% cost 7.4 points, and under noise the model shifted from over-confident to under-confident. The comparison with a TF-IDF classifier was much closer at full size than with 5% of the rows; at full size, Laya’s 0.9-point lead had an interval that included zero. These results describe a synthetic setup, with LLM-labelled data, one backbone, one trainer and mostly one seed; the held-out items were in-distribution, and no out-of-family test was run.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation used synthetic, LLM-labelled data, one backbone and one trainer, with mostly one seed; the abstract does not establish how results transfer beyond this setup. All 1,065 held-out items were in-distribution, and the authors report that no out-of-family test was run. The reported 10% symmetric label noise result comes from a single run. The source describes the plateau as unresolved: its saturating fit with bootstrap intervals could not pin it down.","speaker":{"name":"Khan, Mohammed Nayeem, Misri, Aliyas Ahmad","roleAtSource":"Article author"},"source":{"title":"How many LLM-labelled rows does a decision encoder need, and how wrong can the labels be?","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.16498","publishedAt":"2026-11-12","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors describe Laya as a decision encoder that selects a named option from a question and text without generating text.","excerpt":"A decision encoder such as Laya reads a question with named options (for example, which department should handle a ticket) and a text, and picks an option.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"Across the reported fine-tunes, accuracy on held-out items from the same generator increased with training-set size, from 69.4% before fine-tuning to 86.9% using all 2,818 rows.","excerpt":"We ran 19 fine-tunes of Laya (ModernBERT-large) on nested subsets of a 2,818-row pool and scored them on 1,065 held-out items from the same generator, so every test item is in-distribution and no out-of-family test was run. Accuracy rises from 69.4% before fine-tuning to 74.3% with 5% of the rows, 8","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"The authors say their saturating fit with bootstrap intervals could not determine the accuracy curve’s plateau.","excerpt":"The curve flattens, and a saturating fit with bootstrap intervals cannot pin down its plateau.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"In a single run, 10% symmetric label noise showed no detectable change; the authors report lower accuracy with 20% and 40% noise.","excerpt":"A single run with 10% symmetric label noise shows no detectable change (-0.2 points, 95% interval -1.7 to +1.4); 20% costs 3.4 points and 40% costs 8.7 points.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"The authors report that 20% confusable-pair noise cost 7.4 points and that the model shifted from over-confident to under-confident under noise.","excerpt":"Confusable-pair noise at 20% costs 7.4 points, and under noise the model moves from over-confident to under-confident.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"At full size, Laya’s reported 0.9-point lead over the TF-IDF classifier had an interval including zero; at 5% of the rows, the reported lead was 21.1 points.","excerpt":"Laya is +0.9 points ahead of a TF-IDF classifier trained on the same rows at full size (the interval includes zero) and +21.1 points ahead at 5% of the rows.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"The authors limit the study’s scope to synthetic, LLM-labelled data and one backbone and trainer, with mostly one seed.","excerpt":"Everything is synthetic, LLM-labelled and from one backbone, one trainer and mostly one seed.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.16498","checkedAt":"2026-11-13T10:34:10.666Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What additional data or runs, if any, would let the authors pin down the accuracy curve’s plateau?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Khan and Misri test how many LLM-labelled rows a decision encoder needs","summary":"Mohammed Nayeem Khan and Aliyas Ahmad Misri test how training-set size and label errors affect Laya, a decision encoder that selects among named options without generating text. In 19 fine-tunes, accuracy on held-out items from the same generator rose from 69.4% before fine-tuning to 86.9% using all 2,818 training rows, but a saturating fit with bootstrap intervals could not pin down the curve’s plateau. The authors report that a single run with 10% symmetric label noise showed no detectable change, while 20% and 40% cost 3.4 and 8.7 points. Confusable-pair noise at 20% cost 7.4 points, and under noise the model shifted from over-confident to under-confident. The comparison with a TF-IDF classifier was much closer at full size than with 5% of the rows; at full size, Laya’s 0.9-point lead had an interval that included zero. These results describe a synthetic setup, with LLM-labelled data, one backbone, one trainer and mostly one seed; the held-out items were in-distribution, and no out-of-family test was run.","question":"What additional data or runs, if any, would let the authors pin down the accuracy curve’s plateau?","claims":[{"text":"The authors describe Laya as a decision encoder that selects a named option from a question and text without generating text.","excerpt":"A decision encoder such as Laya reads a question with named options (for example, which department should handle a ticket) and a text, and picks an option.","support":"source-stated"},{"text":"Across the reported fine-tunes, accuracy on held-out items from the same generator increased with training-set size, from 69.4% before fine-tuning to 86.9% using all 2,818 rows.","excerpt":"We ran 19 fine-tunes of Laya (ModernBERT-large) on nested subsets of a 2,818-row pool and scored them on 1,065 held-out items from the same generator, so every test item is in-distribution and no out-of-family test was run. Accuracy rises from 69.4% before fine-tuning to 74.3% with 5% of the rows, 8","support":"source-stated"},{"text":"The authors say their saturating fit with bootstrap intervals could not determine the accuracy curve’s plateau.","excerpt":"The curve flattens, and a saturating fit with bootstrap intervals cannot pin down its plateau.","support":"source-stated"},{"text":"In a single run, 10% symmetric label noise showed no detectable change; the authors report lower accuracy with 20% and 40% noise.","excerpt":"A single run with 10% symmetric label noise shows no detectable change (-0.2 points, 95% interval -1.7 to +1.4); 20% costs 3.4 points and 40% costs 8.7 points.","support":"source-stated"},{"text":"The authors report that 20% confusable-pair noise cost 7.4 points and that the model shifted from over-confident to under-confident under noise.","excerpt":"Confusable-pair noise at 20% costs 7.4 points, and under noise the model moves from over-confident to under-confident.","support":"source-stated"},{"text":"At full size, Laya’s reported 0.9-point lead over the TF-IDF classifier had an interval including zero; at 5% of the rows, the reported lead was 21.1 points.","excerpt":"Laya is +0.9 points ahead of a TF-IDF classifier trained on the same rows at full size (the interval includes zero) and +21.1 points ahead at 5% of the rows.","support":"source-stated"},{"text":"The authors limit the study’s scope to synthetic, LLM-labelled data and one backbone and trainer, with mostly one seed.","excerpt":"Everything is synthetic, LLM-labelled and from one backbone, one trainer and mostly one seed.","support":"source-stated"}],"limitations":["The evaluation used synthetic, LLM-labelled data, one backbone and one trainer, with mostly one seed; the abstract does not establish how results transfer beyond this setup.","All 1,065 held-out items were in-distribution, and the authors report that no out-of-family test was run.","The reported 10% symmetric label noise result comes from a single run.","The source describes the plateau as unresolved: its saturating fit with bootstrap intervals could not pin it down."],"mediaId":"945104ff4db5895a179f2462eb38b0917d60021f652505e1e8eb8fee98d1ba35","correctionReason":null,"id":"f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1","publicationId":"5473cb5e-c05a-4d5f-8e8c-c5337455fef7","version":1,"sourceUrl":"https://arxiv.org/abs/2611.16498","sourceTitle":"How many LLM-labelled rows does a decision encoder need, and how wrong can the labels be?","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-12","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-13T07:43:09.674Z","createdAt":"2026-11-13T09:31:36.609Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"945104ff4db5895a179f2462eb38b0917d60021f652505e1e8eb8fee98d1ba35","kind":"article","url":"https://arxiv.org/abs/2611.16498","sourceUrl":"https://arxiv.org/abs/2611.16498","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-13T09:19:05.019Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.16498","checkedAt":"2026-11-13T10:34:10.666Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"945104ff4db5895a179f2462eb38b0917d60021f652505e1e8eb8fee98d1ba35","confidence":0.52,"provider":"typesafe","model":"jev-1.13.0","inputHash":"187ff3c58a526b2c5c0879aa2cd4d18a350497885b3af6167b1738b64886bbec","decidedAt":"2026-11-13T09:19:06.883Z"},"provenance":{"textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observationId":"ab204b9fe732910360019a274525ca34ce5feb82bac6e55d61fe04629989d702","retrievedAt":"2026-11-13T09:19:04.753Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"fccdee34-6509-4a48-89c4-01d5b31805cb"},"reviewer":{"model":"gpt-6-luna","runId":"d75c1464-f221-4765-861e-fb7ce3089735"}},"audit":{"status":"passed","reviewedAt":"2026-11-13T09:31:36.609Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Khan, Mohammed Nayeem","url":null,"kind":"unknown"},{"name":"Misri, Aliyas Ahmad","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Khan, Mohammed Nayeem","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Khan, Mohammed Nayeem","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.16498","locator":"head/meta[name=author|citation_author]"},"assertionId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T10:34:10.666Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31","sourceUrl":"https://arxiv.org/abs/2611.16498","contentSha256":"d44f9d47d71f9720baac7c78a2b21451801ccdeb8f00726d60287d38d7d86cb7","textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observedAt":"2026-11-13T10:34:10.666Z"}},{"actor":{"name":"Misri, Aliyas Ahmad","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Misri, Aliyas Ahmad","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.16498","locator":"head/meta[name=author|citation_author]"},"assertionId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T10:34:10.666Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31","sourceUrl":"https://arxiv.org/abs/2611.16498","contentSha256":"d44f9d47d71f9720baac7c78a2b21451801ccdeb8f00726d60287d38d7d86cb7","textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observedAt":"2026-11-13T10:34:10.666Z"}}],"participationCoverage":{"total":2,"returned":2,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.16498","snapshotId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31","contentSha256":"d44f9d47d71f9720baac7c78a2b21451801ccdeb8f00726d60287d38d7d86cb7","textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observedAt":"2026-11-13T10:34:10.666Z","basis":"publisher-page-metadata","sourceLinks":37,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1/source"}},"canonicalUrl":"https://escapemeridian.com/readings/khan-and-misri-test-how-many-llm-labelled-rows-a-decision-f5973fe0","formats":{"html":"/readings/khan-and-misri-test-how-many-llm-labelled-rows-a-decision-f5973fe0","markdown":"/readings/khan-and-misri-test-how-many-llm-labelled-rows-a-decision-f5973fe0.md","json":"/readings/khan-and-misri-test-how-many-llm-labelled-rows-a-decision-f5973fe0.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-fine-tuning","name":"Fine-tuning","explanation":"Further training of an existing model to adapt it to a particular task or use.","referenceUrl":"https://developers.google.com/machine-learning/glossary#fine-tuning"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1","revision":"5473cb5e-c05a-4d5f-8e8c-c5337455fef7","revisionKind":"publication-id","title":"Khan and Misri test how many LLM-labelled rows a decision encoder needs","url":"https://escapemeridian.com/readings/khan-and-misri-test-how-many-llm-labelled-rows-a-decision-f5973fe0"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.16498","revision":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","revisionKind":"sha256-source-text","title":"How many LLM-labelled rows does a decision encoder need, and how wrong can the labels be?","url":"https://arxiv.org/abs/2611.16498"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1","title":"Khan and Misri test how many LLM-labelled rows a decision encoder needs","argument":"Mohammed Nayeem Khan and Aliyas Ahmad Misri test how training-set size and label errors affect Laya, a decision encoder that selects among named options without generating text. In 19 fine-tunes, accuracy on held-out items from the same generator rose from 69.4% before fine-tuning to 86.9% using all 2,818 training rows, but a saturating fit with bootstrap intervals could not pin down the curve’s plateau. The authors report that a single run with 10% symmetric label noise showed no detectable change, while 20% and 40% cost 3.4 and 8.7 points. Confusable-pair noise at 20% cost 7.4 points, and under noise the model shifted from over-confident to under-confident. The comparison with a TF-IDF classifier was much closer at full size than with 5% of the rows; at full size, Laya’s 0.9-point lead had an interval that included zero. These results describe a synthetic setup, with LLM-labelled data, one backbone, one trainer and mostly one seed; the held-out items were in-distribution, and no out-of-family test was run.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation used synthetic, LLM-labelled data, one backbone and one trainer, with mostly one seed; the abstract does not establish how results transfer beyond this setup. All 1,065 held-out items were in-distribution, and the authors report that no out-of-family test was run. The reported 10% symmetric label noise result comes from a single run. The source describes the plateau as unresolved: its saturating fit with bootstrap intervals could not pin it down.","speaker":{"name":"Khan, Mohammed Nayeem, Misri, Aliyas Ahmad","roleAtSource":"Article author"},"source":{"title":"How many LLM-labelled rows does a decision encoder need, and how wrong can the labels be?","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.16498","publishedAt":"2026-11-12","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors describe Laya as a decision encoder that selects a named option from a question and text without generating text.","excerpt":"A decision encoder such as Laya reads a question with named options (for example, which department should handle a ticket) and a text, and picks an option.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"Across the reported fine-tunes, accuracy on held-out items from the same generator increased with training-set size, from 69.4% before fine-tuning to 86.9% using all 2,818 rows.","excerpt":"We ran 19 fine-tunes of Laya (ModernBERT-large) on nested subsets of a 2,818-row pool and scored them on 1,065 held-out items from the same generator, so every test item is in-distribution and no out-of-family test was run. Accuracy rises from 69.4% before fine-tuning to 74.3% with 5% of the rows, 8","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"The authors say their saturating fit with bootstrap intervals could not determine the accuracy curve’s plateau.","excerpt":"The curve flattens, and a saturating fit with bootstrap intervals cannot pin down its plateau.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"In a single run, 10% symmetric label noise showed no detectable change; the authors report lower accuracy with 20% and 40% noise.","excerpt":"A single run with 10% symmetric label noise shows no detectable change (-0.2 points, 95% interval -1.7 to +1.4); 20% costs 3.4 points and 40% costs 8.7 points.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"The authors report that 20% confusable-pair noise cost 7.4 points and that the model shifted from over-confident to under-confident under noise.","excerpt":"Confusable-pair noise at 20% costs 7.4 points, and under noise the model moves from over-confident to under-confident.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"At full size, Laya’s reported 0.9-point lead over the TF-IDF classifier had an interval including zero; at 5% of the rows, the reported lead was 21.1 points.","excerpt":"Laya is +0.9 points ahead of a TF-IDF classifier trained on the same rows at full size (the interval includes zero) and +21.1 points ahead at 5% of the rows.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"},{"text":"The authors limit the study’s scope to synthetic, LLM-labelled data and one backbone and trainer, with mostly one seed.","excerpt":"Everything is synthetic, LLM-labelled and from one backbone, one trainer and mostly one seed.","url":"https://arxiv.org/abs/2611.16498","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.16498","checkedAt":"2026-11-13T10:34:10.666Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What additional data or runs, if any, would let the authors pin down the accuracy curve’s plateau?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Khan and Misri test how many LLM-labelled rows a decision encoder needs","summary":"Mohammed Nayeem Khan and Aliyas Ahmad Misri test how training-set size and label errors affect Laya, a decision encoder that selects among named options without generating text. In 19 fine-tunes, accuracy on held-out items from the same generator rose from 69.4% before fine-tuning to 86.9% using all 2,818 training rows, but a saturating fit with bootstrap intervals could not pin down the curve’s plateau. The authors report that a single run with 10% symmetric label noise showed no detectable change, while 20% and 40% cost 3.4 and 8.7 points. Confusable-pair noise at 20% cost 7.4 points, and under noise the model shifted from over-confident to under-confident. The comparison with a TF-IDF classifier was much closer at full size than with 5% of the rows; at full size, Laya’s 0.9-point lead had an interval that included zero. These results describe a synthetic setup, with LLM-labelled data, one backbone, one trainer and mostly one seed; the held-out items were in-distribution, and no out-of-family test was run.","question":"What additional data or runs, if any, would let the authors pin down the accuracy curve’s plateau?","claims":[{"text":"The authors describe Laya as a decision encoder that selects a named option from a question and text without generating text.","excerpt":"A decision encoder such as Laya reads a question with named options (for example, which department should handle a ticket) and a text, and picks an option.","support":"source-stated"},{"text":"Across the reported fine-tunes, accuracy on held-out items from the same generator increased with training-set size, from 69.4% before fine-tuning to 86.9% using all 2,818 rows.","excerpt":"We ran 19 fine-tunes of Laya (ModernBERT-large) on nested subsets of a 2,818-row pool and scored them on 1,065 held-out items from the same generator, so every test item is in-distribution and no out-of-family test was run. Accuracy rises from 69.4% before fine-tuning to 74.3% with 5% of the rows, 8","support":"source-stated"},{"text":"The authors say their saturating fit with bootstrap intervals could not determine the accuracy curve’s plateau.","excerpt":"The curve flattens, and a saturating fit with bootstrap intervals cannot pin down its plateau.","support":"source-stated"},{"text":"In a single run, 10% symmetric label noise showed no detectable change; the authors report lower accuracy with 20% and 40% noise.","excerpt":"A single run with 10% symmetric label noise shows no detectable change (-0.2 points, 95% interval -1.7 to +1.4); 20% costs 3.4 points and 40% costs 8.7 points.","support":"source-stated"},{"text":"The authors report that 20% confusable-pair noise cost 7.4 points and that the model shifted from over-confident to under-confident under noise.","excerpt":"Confusable-pair noise at 20% costs 7.4 points, and under noise the model moves from over-confident to under-confident.","support":"source-stated"},{"text":"At full size, Laya’s reported 0.9-point lead over the TF-IDF classifier had an interval including zero; at 5% of the rows, the reported lead was 21.1 points.","excerpt":"Laya is +0.9 points ahead of a TF-IDF classifier trained on the same rows at full size (the interval includes zero) and +21.1 points ahead at 5% of the rows.","support":"source-stated"},{"text":"The authors limit the study’s scope to synthetic, LLM-labelled data and one backbone and trainer, with mostly one seed.","excerpt":"Everything is synthetic, LLM-labelled and from one backbone, one trainer and mostly one seed.","support":"source-stated"}],"limitations":["The evaluation used synthetic, LLM-labelled data, one backbone and one trainer, with mostly one seed; the abstract does not establish how results transfer beyond this setup.","All 1,065 held-out items were in-distribution, and the authors report that no out-of-family test was run.","The reported 10% symmetric label noise result comes from a single run.","The source describes the plateau as unresolved: its saturating fit with bootstrap intervals could not pin it down."],"mediaId":"945104ff4db5895a179f2462eb38b0917d60021f652505e1e8eb8fee98d1ba35","correctionReason":null,"id":"f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1","publicationId":"5473cb5e-c05a-4d5f-8e8c-c5337455fef7","version":1,"sourceUrl":"https://arxiv.org/abs/2611.16498","sourceTitle":"How many LLM-labelled rows does a decision encoder need, and how wrong can the labels be?","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-12","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-13T07:43:09.674Z","createdAt":"2026-11-13T09:31:36.609Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"945104ff4db5895a179f2462eb38b0917d60021f652505e1e8eb8fee98d1ba35","kind":"article","url":"https://arxiv.org/abs/2611.16498","sourceUrl":"https://arxiv.org/abs/2611.16498","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-13T09:19:05.019Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.16498","checkedAt":"2026-11-13T10:34:10.666Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"945104ff4db5895a179f2462eb38b0917d60021f652505e1e8eb8fee98d1ba35","confidence":0.52,"provider":"typesafe","model":"jev-1.13.0","inputHash":"187ff3c58a526b2c5c0879aa2cd4d18a350497885b3af6167b1738b64886bbec","decidedAt":"2026-11-13T09:19:06.883Z"},"provenance":{"textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observationId":"ab204b9fe732910360019a274525ca34ce5feb82bac6e55d61fe04629989d702","retrievedAt":"2026-11-13T09:19:04.753Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"fccdee34-6509-4a48-89c4-01d5b31805cb"},"reviewer":{"model":"gpt-6-luna","runId":"d75c1464-f221-4765-861e-fb7ce3089735"}},"audit":{"status":"passed","reviewedAt":"2026-11-13T09:31:36.609Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Khan, Mohammed Nayeem","url":null,"kind":"unknown"},{"name":"Misri, Aliyas Ahmad","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Khan, Mohammed Nayeem","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Khan, Mohammed Nayeem","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.16498","locator":"head/meta[name=author|citation_author]"},"assertionId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T10:34:10.666Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31","sourceUrl":"https://arxiv.org/abs/2611.16498","contentSha256":"d44f9d47d71f9720baac7c78a2b21451801ccdeb8f00726d60287d38d7d86cb7","textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observedAt":"2026-11-13T10:34:10.666Z"}},{"actor":{"name":"Misri, Aliyas Ahmad","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Misri, Aliyas Ahmad","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.16498","locator":"head/meta[name=author|citation_author]"},"assertionId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T10:34:10.666Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31","sourceUrl":"https://arxiv.org/abs/2611.16498","contentSha256":"d44f9d47d71f9720baac7c78a2b21451801ccdeb8f00726d60287d38d7d86cb7","textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observedAt":"2026-11-13T10:34:10.666Z"}}],"participationCoverage":{"total":2,"returned":2,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.16498","snapshotId":"d8fae5aa4cbfd907e68d31f97e12c705315d982638d4ec783ab8d84ae8a79e31","contentSha256":"d44f9d47d71f9720baac7c78a2b21451801ccdeb8f00726d60287d38d7d86cb7","textHash":"6c1e3ea362f4e73a81e92ac38cf8bf5fd370e6ce8585fdb2a63ca8c9630ef009","observedAt":"2026-11-13T10:34:10.666Z","basis":"publisher-page-metadata","sourceLinks":37,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/f5973fe04840960beff72921112a333c5098f2211e15d3509fc463a3b7461fb1/source"}}}],"audits":[{"publicationId":"5473cb5e-c05a-4d5f-8e8c-c5337455fef7","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly names Mohammed Nayeem Khan and Aliyas Ahmad Misri as the authors and keeps the results attributed to them with phrases such as 'the authors report'. It does not present them as established facts.","excerpt":"Authors:Mohammed Nayeem Khan, Aliyas Ahmad Misri"},{"dimension":"summary","status":"pass","explanation":"The title, summary and question match the abstract: Laya, data-size scaling, the unresolved plateau, the noise effects, the confidence shift, the TF-IDF comparison and the scope limits. 'Decision encoder' is explained in plain language. Terms such as TF-IDF, symmetric and confusable-pair noise, and seed are technical terms suited to tooltips. Optional, non-blocking omissions: the seed spread (s.d. 1.6) exceeding some size steps, and the independent-labeller result. The plateau question follows directly from the evidence.","excerpt":"Seed spread at 20% of the rows (s.d. 1.6 points) exceeds some of the steps between sizes."},{"dimension":"date","status":"pass","explanation":"The originalPublished date, 2026-11-12, matches the arXiv v1 submission date. The feed date of 2026-11-13 is later and is correctly not used.","excerpt":"[Submitted on 12 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Every figure is grounded verbatim in the abstract: 19 fine-tunes, 2,818 rows, 69.4% to 86.9%, the 10/20/40% noise results, 7.4 points, +0.9 with an interval including zero, +21.1 at 5% of rows, and the in-distribution and single-backbone limits. The reading does not overstate the results as real-world reliability, and it introduces no unsupported benefits or costs.","excerpt":"Everything is synthetic, LLM-labelled and from one backbone, one trainer and mostly one seed."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the most direct way to reach the source.","excerpt":null}],"createdAt":"2026-11-13T09:31:36.609Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}