{"id":"f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2","title":"arXiv cs.AI: Researchers propose system for extracting scores from scanned assessments","argument":"Authors Enshuo Hsu and colleagues propose a modular system using multimodal LLMs to extract fields and scores from scanned cognitive and behavioral assessments. A form classifier first identifies the assessment; a corresponding score extractor then organizes its information. In their design, dual-model validation sends pages for human review when two LLMs disagree. The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% for the NICHQ Vanderbilt Assessment. With human review, cognitive assessment accuracy rose to 97.5%, while manual work fell by 63.8%. They also report processing was 10 to 20 times faster and costs were over 12 times lower than with manual data extraction. For Mini-Cog Clock Drawing, their qualitative analysis attributed 62.3% of extraction errors to ambiguous drawings, which the authors say the LLM read reasonably. The findings suggest a possible way to reduce routine data entry while keeping people involved in disputed cases. The abstract does not give evaluation sample sizes or describe the data in enough detail to judge how broadly the reported results apply.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not give evaluation sample sizes or describe the data and evaluation conditions in enough detail to assess how broadly the reported results apply. The reported accuracy and efficiency figures are claims from the paper abstract; the supplied evidence does not include the full paper's methods or results.","speaker":{"name":"Hsu, Enshuo, Nguyen, Linh, Fu, Sunyang, Goodrum, Heath, Stansberry, Sandra, Roberts, Kirk, Liu, Hongfang, Le, Yen-Chi L.","roleAtSource":"Article author"},"source":{"title":"A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.22252","publishedAt":"2026-11-17","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose a modular system that classifies scanned assessment pages and extracts fields and scores, with human review when two LLMs disagree.","excerpt":"A human-in-the-loop design utilizing dual-model validation engages human annotators when two LLMs disagree","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% overall accuracy for the NICHQ Vanderbilt Assessment.","excerpt":"End-to-end accuracy for cognitive assessments was 88%, with most form types exceeding 90%. For the NICHQ Vanderbilt, overall accuracy reached 97.5%","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"The authors report that human review raised cognitive assessment accuracy to 97.5% and reduced manual work by 63.8%.","excerpt":"The human-in-the-loop design further improved cognitive assessment accuracy to 97.5% while reducing manual work by 63.8%.","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"The authors report processing times 10 to 20 times faster and costs over 12 times lower than manual data extraction.","excerpt":"reduces processing time by 10 to 20 times and costs by over 12 times.","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"For Mini-Cog Clock Drawing, the authors say 62.3% of extraction errors involved ambiguous drawings, which the LLM read reasonably.","excerpt":"62.3% of all extraction errors stemmed from ambiguous drawing, in which cases the LLM provided reasonable readings.","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.22252","checkedAt":"2026-11-18T17:48:12.957Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evaluation data and conditions produced the reported accuracy and efficiency results?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Researchers propose system for extracting scores from scanned assessments","summary":"Authors Enshuo Hsu and colleagues propose a modular system using multimodal LLMs to extract fields and scores from scanned cognitive and behavioral assessments. A form classifier first identifies the assessment; a corresponding score extractor then organizes its information. In their design, dual-model validation sends pages for human review when two LLMs disagree. The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% for the NICHQ Vanderbilt Assessment. With human review, cognitive assessment accuracy rose to 97.5%, while manual work fell by 63.8%. They also report processing was 10 to 20 times faster and costs were over 12 times lower than with manual data extraction. For Mini-Cog Clock Drawing, their qualitative analysis attributed 62.3% of extraction errors to ambiguous drawings, which the authors say the LLM read reasonably. The findings suggest a possible way to reduce routine data entry while keeping people involved in disputed cases. The abstract does not give evaluation sample sizes or describe the data in enough detail to judge how broadly the reported results apply.","question":"What evaluation data and conditions produced the reported accuracy and efficiency results?","claims":[{"text":"The authors propose a modular system that classifies scanned assessment pages and extracts fields and scores, with human review when two LLMs disagree.","excerpt":"A human-in-the-loop design utilizing dual-model validation engages human annotators when two LLMs disagree","support":"source-stated"},{"text":"The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% overall accuracy for the NICHQ Vanderbilt Assessment.","excerpt":"End-to-end accuracy for cognitive assessments was 88%, with most form types exceeding 90%. For the NICHQ Vanderbilt, overall accuracy reached 97.5%","support":"source-stated"},{"text":"The authors report that human review raised cognitive assessment accuracy to 97.5% and reduced manual work by 63.8%.","excerpt":"The human-in-the-loop design further improved cognitive assessment accuracy to 97.5% while reducing manual work by 63.8%.","support":"source-stated"},{"text":"The authors report processing times 10 to 20 times faster and costs over 12 times lower than manual data extraction.","excerpt":"reduces processing time by 10 to 20 times and costs by over 12 times.","support":"source-stated"},{"text":"For Mini-Cog Clock Drawing, the authors say 62.3% of extraction errors involved ambiguous drawings, which the LLM read reasonably.","excerpt":"62.3% of all extraction errors stemmed from ambiguous drawing, in which cases the LLM provided reasonable readings.","support":"source-stated"}],"limitations":["The abstract does not give evaluation sample sizes or describe the data and evaluation conditions in enough detail to assess how broadly the reported results apply.","The reported accuracy and efficiency figures are claims from the paper abstract; the supplied evidence does not include the full paper's methods or results."],"mediaId":"0b00acc2ef479fcb1cbb6aacecddc6c871beb3bc8b8161d43464ecc921394757","correctionReason":null,"id":"f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2","publicationId":"9a3f2fd9-4594-4b9b-80da-d4652b3202a5","version":1,"sourceUrl":"https://arxiv.org/abs/2611.22252","sourceTitle":"A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-17","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-18T05:04:20.871Z","createdAt":"2026-11-18T17:32:42.136Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"0b00acc2ef479fcb1cbb6aacecddc6c871beb3bc8b8161d43464ecc921394757","kind":"article","url":"https://arxiv.org/abs/2611.22252","sourceUrl":"https://arxiv.org/abs/2611.22252","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-18T17:19:29.055Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.22252","checkedAt":"2026-11-18T17:48:12.957Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"0b00acc2ef479fcb1cbb6aacecddc6c871beb3bc8b8161d43464ecc921394757","confidence":0.83,"provider":"typesafe","model":"jev-1.13.0","inputHash":"cef1e4e5aa200a7df0f5a85d2f1f46de02e335ac22b165b452935a43944d48c4","decidedAt":"2026-11-18T17:19:29.887Z"},"provenance":{"textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observationId":"379a24215e8b3ca9c553150680b6a5678fb5ad9175bebb49eff54a1320e1737f","retrievedAt":"2026-11-18T17:19:28.936Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"c8751574-4cc3-4b73-81bd-fcc572099cc4"},"reviewer":{"model":"gpt-6-luna","runId":"40b976e7-3e85-4db3-8491-79dde886cb28"}},"audit":{"status":"passed","reviewedAt":"2026-11-18T17:32:42.136Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Hsu, Enshuo","url":null,"kind":"unknown"},{"name":"Nguyen, Linh","url":null,"kind":"unknown"},{"name":"Fu, Sunyang","url":null,"kind":"unknown"},{"name":"Goodrum, Heath","url":null,"kind":"unknown"},{"name":"Stansberry, Sandra","url":null,"kind":"unknown"},{"name":"Roberts, Kirk","url":null,"kind":"unknown"},{"name":"Liu, Hongfang","url":null,"kind":"unknown"},{"name":"Le, Yen-Chi L.","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Hsu, Enshuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hsu, Enshuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Nguyen, Linh","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Nguyen, Linh","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Fu, Sunyang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Fu, Sunyang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Goodrum, Heath","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Goodrum, Heath","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Stansberry, Sandra","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Stansberry, Sandra","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Roberts, Kirk","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Roberts, Kirk","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Liu, Hongfang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Hongfang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Le, Yen-Chi L.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Le, Yen-Chi L.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.22252","snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-researchers-propose-system-for-extracting-scores-f95b2931","formats":{"html":"/readings/arxiv-cs-ai-researchers-propose-system-for-extracting-scores-f95b2931","markdown":"/readings/arxiv-cs-ai-researchers-propose-system-for-extracting-scores-f95b2931.md","json":"/readings/arxiv-cs-ai-researchers-propose-system-for-extracting-scores-f95b2931.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-multimodal-model","name":"Multimodal model","explanation":"A model that works with more than one kind of input or output, such as text, images and audio.","referenceUrl":"https://developers.google.com/machine-learning/glossary#multimodal-model"},{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2","revision":"9a3f2fd9-4594-4b9b-80da-d4652b3202a5","revisionKind":"publication-id","title":"arXiv cs.AI: Researchers propose system for extracting scores from scanned assessments","url":"https://escapemeridian.com/readings/arxiv-cs-ai-researchers-propose-system-for-extracting-scores-f95b2931"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.22252","revision":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","revisionKind":"sha256-source-text","title":"A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments","url":"https://arxiv.org/abs/2611.22252"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2","title":"arXiv cs.AI: Researchers propose system for extracting scores from scanned assessments","argument":"Authors Enshuo Hsu and colleagues propose a modular system using multimodal LLMs to extract fields and scores from scanned cognitive and behavioral assessments. A form classifier first identifies the assessment; a corresponding score extractor then organizes its information. In their design, dual-model validation sends pages for human review when two LLMs disagree. The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% for the NICHQ Vanderbilt Assessment. With human review, cognitive assessment accuracy rose to 97.5%, while manual work fell by 63.8%. They also report processing was 10 to 20 times faster and costs were over 12 times lower than with manual data extraction. For Mini-Cog Clock Drawing, their qualitative analysis attributed 62.3% of extraction errors to ambiguous drawings, which the authors say the LLM read reasonably. The findings suggest a possible way to reduce routine data entry while keeping people involved in disputed cases. The abstract does not give evaluation sample sizes or describe the data in enough detail to judge how broadly the reported results apply.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not give evaluation sample sizes or describe the data and evaluation conditions in enough detail to assess how broadly the reported results apply. The reported accuracy and efficiency figures are claims from the paper abstract; the supplied evidence does not include the full paper's methods or results.","speaker":{"name":"Hsu, Enshuo, Nguyen, Linh, Fu, Sunyang, Goodrum, Heath, Stansberry, Sandra, Roberts, Kirk, Liu, Hongfang, Le, Yen-Chi L.","roleAtSource":"Article author"},"source":{"title":"A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.22252","publishedAt":"2026-11-17","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose a modular system that classifies scanned assessment pages and extracts fields and scores, with human review when two LLMs disagree.","excerpt":"A human-in-the-loop design utilizing dual-model validation engages human annotators when two LLMs disagree","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% overall accuracy for the NICHQ Vanderbilt Assessment.","excerpt":"End-to-end accuracy for cognitive assessments was 88%, with most form types exceeding 90%. For the NICHQ Vanderbilt, overall accuracy reached 97.5%","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"The authors report that human review raised cognitive assessment accuracy to 97.5% and reduced manual work by 63.8%.","excerpt":"The human-in-the-loop design further improved cognitive assessment accuracy to 97.5% while reducing manual work by 63.8%.","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"The authors report processing times 10 to 20 times faster and costs over 12 times lower than manual data extraction.","excerpt":"reduces processing time by 10 to 20 times and costs by over 12 times.","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"},{"text":"For Mini-Cog Clock Drawing, the authors say 62.3% of extraction errors involved ambiguous drawings, which the LLM read reasonably.","excerpt":"62.3% of all extraction errors stemmed from ambiguous drawing, in which cases the LLM provided reasonable readings.","url":"https://arxiv.org/abs/2611.22252","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.22252","checkedAt":"2026-11-18T17:48:12.957Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What evaluation data and conditions produced the reported accuracy and efficiency results?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Researchers propose system for extracting scores from scanned assessments","summary":"Authors Enshuo Hsu and colleagues propose a modular system using multimodal LLMs to extract fields and scores from scanned cognitive and behavioral assessments. A form classifier first identifies the assessment; a corresponding score extractor then organizes its information. In their design, dual-model validation sends pages for human review when two LLMs disagree. The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% for the NICHQ Vanderbilt Assessment. With human review, cognitive assessment accuracy rose to 97.5%, while manual work fell by 63.8%. They also report processing was 10 to 20 times faster and costs were over 12 times lower than with manual data extraction. For Mini-Cog Clock Drawing, their qualitative analysis attributed 62.3% of extraction errors to ambiguous drawings, which the authors say the LLM read reasonably. The findings suggest a possible way to reduce routine data entry while keeping people involved in disputed cases. The abstract does not give evaluation sample sizes or describe the data in enough detail to judge how broadly the reported results apply.","question":"What evaluation data and conditions produced the reported accuracy and efficiency results?","claims":[{"text":"The authors propose a modular system that classifies scanned assessment pages and extracts fields and scores, with human review when two LLMs disagree.","excerpt":"A human-in-the-loop design utilizing dual-model validation engages human annotators when two LLMs disagree","support":"source-stated"},{"text":"The authors report 88% end-to-end accuracy for cognitive assessments and 97.5% overall accuracy for the NICHQ Vanderbilt Assessment.","excerpt":"End-to-end accuracy for cognitive assessments was 88%, with most form types exceeding 90%. For the NICHQ Vanderbilt, overall accuracy reached 97.5%","support":"source-stated"},{"text":"The authors report that human review raised cognitive assessment accuracy to 97.5% and reduced manual work by 63.8%.","excerpt":"The human-in-the-loop design further improved cognitive assessment accuracy to 97.5% while reducing manual work by 63.8%.","support":"source-stated"},{"text":"The authors report processing times 10 to 20 times faster and costs over 12 times lower than manual data extraction.","excerpt":"reduces processing time by 10 to 20 times and costs by over 12 times.","support":"source-stated"},{"text":"For Mini-Cog Clock Drawing, the authors say 62.3% of extraction errors involved ambiguous drawings, which the LLM read reasonably.","excerpt":"62.3% of all extraction errors stemmed from ambiguous drawing, in which cases the LLM provided reasonable readings.","support":"source-stated"}],"limitations":["The abstract does not give evaluation sample sizes or describe the data and evaluation conditions in enough detail to assess how broadly the reported results apply.","The reported accuracy and efficiency figures are claims from the paper abstract; the supplied evidence does not include the full paper's methods or results."],"mediaId":"0b00acc2ef479fcb1cbb6aacecddc6c871beb3bc8b8161d43464ecc921394757","correctionReason":null,"id":"f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2","publicationId":"9a3f2fd9-4594-4b9b-80da-d4652b3202a5","version":1,"sourceUrl":"https://arxiv.org/abs/2611.22252","sourceTitle":"A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-17","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-18T05:04:20.871Z","createdAt":"2026-11-18T17:32:42.136Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"0b00acc2ef479fcb1cbb6aacecddc6c871beb3bc8b8161d43464ecc921394757","kind":"article","url":"https://arxiv.org/abs/2611.22252","sourceUrl":"https://arxiv.org/abs/2611.22252","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-18T17:19:29.055Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.22252","checkedAt":"2026-11-18T17:48:12.957Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"0b00acc2ef479fcb1cbb6aacecddc6c871beb3bc8b8161d43464ecc921394757","confidence":0.83,"provider":"typesafe","model":"jev-1.13.0","inputHash":"cef1e4e5aa200a7df0f5a85d2f1f46de02e335ac22b165b452935a43944d48c4","decidedAt":"2026-11-18T17:19:29.887Z"},"provenance":{"textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observationId":"379a24215e8b3ca9c553150680b6a5678fb5ad9175bebb49eff54a1320e1737f","retrievedAt":"2026-11-18T17:19:28.936Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"c8751574-4cc3-4b73-81bd-fcc572099cc4"},"reviewer":{"model":"gpt-6-luna","runId":"40b976e7-3e85-4db3-8491-79dde886cb28"}},"audit":{"status":"passed","reviewedAt":"2026-11-18T17:32:42.136Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Hsu, Enshuo","url":null,"kind":"unknown"},{"name":"Nguyen, Linh","url":null,"kind":"unknown"},{"name":"Fu, Sunyang","url":null,"kind":"unknown"},{"name":"Goodrum, Heath","url":null,"kind":"unknown"},{"name":"Stansberry, Sandra","url":null,"kind":"unknown"},{"name":"Roberts, Kirk","url":null,"kind":"unknown"},{"name":"Liu, Hongfang","url":null,"kind":"unknown"},{"name":"Le, Yen-Chi L.","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Hsu, Enshuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hsu, Enshuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Nguyen, Linh","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Nguyen, Linh","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Fu, Sunyang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Fu, Sunyang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Goodrum, Heath","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Goodrum, Heath","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Stansberry, Sandra","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Stansberry, Sandra","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Roberts, Kirk","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Roberts, Kirk","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Liu, Hongfang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Hongfang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}},{"actor":{"name":"Le, Yen-Chi L.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Le, Yen-Chi L.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.22252","locator":"head/meta[name=author|citation_author]"},"assertionId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-18T17:48:12.957Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","sourceUrl":"https://arxiv.org/abs/2611.22252","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.22252","snapshotId":"5049a03d945d36f85d15a0f3b228a61cb6dd7b027c6d7776ad7f68da1c9d02c9","contentSha256":"173172efa5f06a715afe40f08b1809acbf2ddeec97e952ea54b2cb075a5f74ea","textHash":"ec55a16fa8f47f690747b5c0964011a94d5902f43400b2df677fc7fbc73034e6","observedAt":"2026-11-18T17:48:12.957Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/f95b29318c1ad162342ab9f3d499b89fdf737a590ff96fb7d3389eba374c70b2/source"}}}],"audits":[{"publicationId":"9a3f2fd9-4594-4b9b-80da-d4652b3202a5","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading credits Enshuo Hsu and colleagues on arXiv cs.AI, matching the author list. Accuracy, efficiency and error-analysis figures stay framed as the authors' reports rather than established facts.","excerpt":"Authors:Enshuo Hsu, Linh Nguyen, Sunyang Fu, Heath Goodrum, Sandra Stansberry, Kirk Roberts, Hongfang Liu, Yen-Chi L. Le"},{"dimension":"summary","status":"pass","explanation":"The title, summary and question faithfully reflect the abstract: a two-module system (form classifier, score extractor), dual-model validation that sends disagreements to humans, and the reported results. Clinical form names such as MMSE, MoCA and NICHQ Vanderbilt are not explained. The draft also omits the patient-care, handwritten-form context. A short note that these are paper-based patient-care forms would help general readers. The development remains understandable, and the question addresses a genuine gap.","excerpt":"Paper-based assessments are common in patient care, but handwriting and scanning artifacts make these forms difficult to process at scale."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-17 matches the v1 submission date. It is not the later feed date.","excerpt":"[Submitted on 17 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Every numeric claim matches the abstract: 88%, 97.5%, 63.8%, 10 to 20 times faster, over 12 times cheaper, and 62.3% for Mini-Cog Clock Drawing. The hedged line about reducing routine data entry while keeping people in disputed cases follows from the reported manual-work reduction and human-in-the-loop design. The limitations accurately say that sample sizes and data details are absent from the abstract.","excerpt":"The human-in-the-loop design further improved cognitive assessment accuracy to 97.5% while reducing manual work by 63.8%."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page, which is the source itself.","excerpt":null}],"createdAt":"2026-11-18T17:32:42.136Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}