{"id":"620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9","title":"arXiv cs.AI: Bio-VQA tests how finely vision-language models see biology","argument":"Authors S M Rayeed, Qian Ma, Charles V. Stewart and Yao Ma introduce Bio-VQA, a benchmark for testing vision-language models (VLMs) on biological images. It spans hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning and individual re-identification: identifying an organism, its traits and location, and whether it has appeared before. The benchmark contains 1.86M questions over 217K images from 11 public datasets, and uses graded scoring so near-misses count for more than distant errors.\n\nIn evaluations of 18 open-weight models, the authors report that general visual capabilities do not transfer well to biological organisms. The best model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts. Taxonomic accuracy fell from 0.72 at phylum to 0.12 at scientific name, while scaling helped mostly at coarse ranks. On trait size measurement, 15 models did no better than an image-blind baseline; under adversarial conditions, 17 fell below chance at re-identification. The authors hope Bio-VQA becomes a standard test of how well VLM capabilities transfer to biology. The abstract reports benchmark results; how they relate to performance in biodiversity monitoring remains unclear.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The reported evaluations cover 18 open-weight models ranging from 3.75B to 38.4B parameters; the abstract does not report results for other models. The source gives benchmark results, including re-identification under adversarial conditions, but does not report performance in biodiversity monitoring settings.","speaker":{"name":"Rayeed, S M, Ma, Qian, Stewart, Charles V., Ma, Yao","roleAtSource":"Article author"},"source":{"title":"How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05559","publishedAt":"2026-11-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present Bio-VQA as a benchmark spanning four aspects of biological visual understanding, with 1.86M questions over 217K images from 11 public datasets.","excerpt":"Bio-VQA contains 1.86M questions over 217K images from 11 public datasets","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The authors report that the best evaluated model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts.","excerpt":"the best model counts animals exactly on 41.7% of images with 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The authors report taxonomic accuracy falling from 0.72 at phylum to 0.12 at scientific name, and say scaling helps mostly at coarse ranks.","excerpt":"Taxonomic accuracy falls to one sixth, from 0.72 at phylum to 0.12 at scientific name, and scaling helps mostly at coarse ranks.","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The authors report that 15 of 18 models did no better than an image-blind baseline on trait size measurement, and 17 fell below chance at re-identification under adversarial conditions.","excerpt":"On trait size measurement, 15 of the 18 models do no better than an image-blind baseline that predicts one fixed number for every specimen, and on re-identification, 17 of the 18 models fall below chance under adversarial conditions.","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The abstract reports benchmark results but does not provide a comparison with performance in biodiversity monitoring settings.","excerpt":"In biodiversity monitoring, a single image raises several questions: which taxon, how many, which trait, and whether the individual has been seen before.","url":"https://arxiv.org/abs/2611.05559","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05559","checkedAt":"2026-11-10T02:56:08.842Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do the models’ Bio-VQA results compare with their performance in biodiversity monitoring?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Bio-VQA tests how finely vision-language models see biology","summary":"Authors S M Rayeed, Qian Ma, Charles V. Stewart and Yao Ma introduce Bio-VQA, a benchmark for testing vision-language models (VLMs) on biological images. It spans hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning and individual re-identification: identifying an organism, its traits and location, and whether it has appeared before. The benchmark contains 1.86M questions over 217K images from 11 public datasets, and uses graded scoring so near-misses count for more than distant errors.\n\nIn evaluations of 18 open-weight models, the authors report that general visual capabilities do not transfer well to biological organisms. The best model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts. Taxonomic accuracy fell from 0.72 at phylum to 0.12 at scientific name, while scaling helped mostly at coarse ranks. On trait size measurement, 15 models did no better than an image-blind baseline; under adversarial conditions, 17 fell below chance at re-identification. The authors hope Bio-VQA becomes a standard test of how well VLM capabilities transfer to biology. The abstract reports benchmark results; how they relate to performance in biodiversity monitoring remains unclear.","question":"How do the models’ Bio-VQA results compare with their performance in biodiversity monitoring?","claims":[{"text":"The authors present Bio-VQA as a benchmark spanning four aspects of biological visual understanding, with 1.86M questions over 217K images from 11 public datasets.","excerpt":"Bio-VQA contains 1.86M questions over 217K images from 11 public datasets","support":"source-stated"},{"text":"The authors report that the best evaluated model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts.","excerpt":"the best model counts animals exactly on 41.7% of images with 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts","support":"source-stated"},{"text":"The authors report taxonomic accuracy falling from 0.72 at phylum to 0.12 at scientific name, and say scaling helps mostly at coarse ranks.","excerpt":"Taxonomic accuracy falls to one sixth, from 0.72 at phylum to 0.12 at scientific name, and scaling helps mostly at coarse ranks.","support":"source-stated"},{"text":"The authors report that 15 of 18 models did no better than an image-blind baseline on trait size measurement, and 17 fell below chance at re-identification under adversarial conditions.","excerpt":"On trait size measurement, 15 of the 18 models do no better than an image-blind baseline that predicts one fixed number for every specimen, and on re-identification, 17 of the 18 models fall below chance under adversarial conditions.","support":"source-stated"},{"text":"The abstract reports benchmark results but does not provide a comparison with performance in biodiversity monitoring settings.","excerpt":"In biodiversity monitoring, a single image raises several questions: which taxon, how many, which trait, and whether the individual has been seen before.","support":"interpretation"}],"limitations":["The reported evaluations cover 18 open-weight models ranging from 3.75B to 38.4B parameters; the abstract does not report results for other models.","The source gives benchmark results, including re-identification under adversarial conditions, but does not report performance in biodiversity monitoring settings."],"mediaId":"00c200ef9629a45d82743841cac47aab7549763b63f80b260c12ff227c39c867","correctionReason":null,"id":"620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9","publicationId":"9f05e99e-1587-4343-8185-287ebaeeaa69","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05559","sourceTitle":"How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:32.942Z","createdAt":"2026-11-10T01:53:54.691Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"00c200ef9629a45d82743841cac47aab7549763b63f80b260c12ff227c39c867","kind":"article","url":"https://arxiv.org/abs/2611.05559","sourceUrl":"https://arxiv.org/abs/2611.05559","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T01:53:09.443Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05559","checkedAt":"2026-11-10T02:56:08.842Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"00c200ef9629a45d82743841cac47aab7549763b63f80b260c12ff227c39c867","confidence":0.91,"provider":"typesafe","model":"jev-1.13.0","inputHash":"7fda1f262bbae7d4a1793a4305242a1d9e13b87439e332cb1f3f9dc068d890d1","decidedAt":"2026-11-10T01:53:10.750Z"},"provenance":{"textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observationId":"a087b8ffe9a2c2223abcf10e0449086d18203ef0a4082d7c096fb94199d88c73","retrievedAt":"2026-11-10T01:53:09.242Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"b57e9e3a-816a-4433-83d3-1bb18f68df0f"},"reviewer":{"model":"gpt-6-luna","runId":"e1788670-873d-45b6-8b67-1f54a95a9f36"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T01:53:54.691Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Rayeed, S M","url":null,"kind":"unknown"},{"name":"Ma, Qian","url":null,"kind":"unknown"},{"name":"Stewart, Charles V.","url":null,"kind":"unknown"},{"name":"Ma, Yao","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Rayeed, S M","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rayeed, S M","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}},{"actor":{"name":"Ma, Qian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Qian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}},{"actor":{"name":"Stewart, Charles V.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Stewart, Charles V.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}},{"actor":{"name":"Ma, Yao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Yao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05559","snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-bio-vqa-tests-how-finely-vision-language-models-see-620ef1ad","formats":{"html":"/readings/arxiv-cs-ai-bio-vqa-tests-how-finely-vision-language-models-see-620ef1ad","markdown":"/readings/arxiv-cs-ai-bio-vqa-tests-how-finely-vision-language-models-see-620ef1ad.md","json":"/readings/arxiv-cs-ai-bio-vqa-tests-how-finely-vision-language-models-see-620ef1ad.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-open-weights","name":"Open weights","explanation":"A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.","referenceUrl":"https://opensource.org/ai/open-weights"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9","revision":"9f05e99e-1587-4343-8185-287ebaeeaa69","revisionKind":"publication-id","title":"arXiv cs.AI: Bio-VQA tests how finely vision-language models see biology","url":"https://escapemeridian.com/readings/arxiv-cs-ai-bio-vqa-tests-how-finely-vision-language-models-see-620ef1ad"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.05559","revision":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","revisionKind":"sha256-source-text","title":"How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning","url":"https://arxiv.org/abs/2611.05559"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9","title":"arXiv cs.AI: Bio-VQA tests how finely vision-language models see biology","argument":"Authors S M Rayeed, Qian Ma, Charles V. Stewart and Yao Ma introduce Bio-VQA, a benchmark for testing vision-language models (VLMs) on biological images. It spans hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning and individual re-identification: identifying an organism, its traits and location, and whether it has appeared before. The benchmark contains 1.86M questions over 217K images from 11 public datasets, and uses graded scoring so near-misses count for more than distant errors.\n\nIn evaluations of 18 open-weight models, the authors report that general visual capabilities do not transfer well to biological organisms. The best model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts. Taxonomic accuracy fell from 0.72 at phylum to 0.12 at scientific name, while scaling helped mostly at coarse ranks. On trait size measurement, 15 models did no better than an image-blind baseline; under adversarial conditions, 17 fell below chance at re-identification. The authors hope Bio-VQA becomes a standard test of how well VLM capabilities transfer to biology. The abstract reports benchmark results; how they relate to performance in biodiversity monitoring remains unclear.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The reported evaluations cover 18 open-weight models ranging from 3.75B to 38.4B parameters; the abstract does not report results for other models. The source gives benchmark results, including re-identification under adversarial conditions, but does not report performance in biodiversity monitoring settings.","speaker":{"name":"Rayeed, S M, Ma, Qian, Stewart, Charles V., Ma, Yao","roleAtSource":"Article author"},"source":{"title":"How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05559","publishedAt":"2026-11-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present Bio-VQA as a benchmark spanning four aspects of biological visual understanding, with 1.86M questions over 217K images from 11 public datasets.","excerpt":"Bio-VQA contains 1.86M questions over 217K images from 11 public datasets","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The authors report that the best evaluated model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts.","excerpt":"the best model counts animals exactly on 41.7% of images with 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The authors report taxonomic accuracy falling from 0.72 at phylum to 0.12 at scientific name, and say scaling helps mostly at coarse ranks.","excerpt":"Taxonomic accuracy falls to one sixth, from 0.72 at phylum to 0.12 at scientific name, and scaling helps mostly at coarse ranks.","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The authors report that 15 of 18 models did no better than an image-blind baseline on trait size measurement, and 17 fell below chance at re-identification under adversarial conditions.","excerpt":"On trait size measurement, 15 of the 18 models do no better than an image-blind baseline that predicts one fixed number for every specimen, and on re-identification, 17 of the 18 models fall below chance under adversarial conditions.","url":"https://arxiv.org/abs/2611.05559","relationship":"Source-stated"},{"text":"The abstract reports benchmark results but does not provide a comparison with performance in biodiversity monitoring settings.","excerpt":"In biodiversity monitoring, a single image raises several questions: which taxon, how many, which trait, and whether the individual has been seen before.","url":"https://arxiv.org/abs/2611.05559","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05559","checkedAt":"2026-11-10T02:56:08.842Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do the models’ Bio-VQA results compare with their performance in biodiversity monitoring?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Bio-VQA tests how finely vision-language models see biology","summary":"Authors S M Rayeed, Qian Ma, Charles V. Stewart and Yao Ma introduce Bio-VQA, a benchmark for testing vision-language models (VLMs) on biological images. It spans hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning and individual re-identification: identifying an organism, its traits and location, and whether it has appeared before. The benchmark contains 1.86M questions over 217K images from 11 public datasets, and uses graded scoring so near-misses count for more than distant errors.\n\nIn evaluations of 18 open-weight models, the authors report that general visual capabilities do not transfer well to biological organisms. The best model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts. Taxonomic accuracy fell from 0.72 at phylum to 0.12 at scientific name, while scaling helped mostly at coarse ranks. On trait size measurement, 15 models did no better than an image-blind baseline; under adversarial conditions, 17 fell below chance at re-identification. The authors hope Bio-VQA becomes a standard test of how well VLM capabilities transfer to biology. The abstract reports benchmark results; how they relate to performance in biodiversity monitoring remains unclear.","question":"How do the models’ Bio-VQA results compare with their performance in biodiversity monitoring?","claims":[{"text":"The authors present Bio-VQA as a benchmark spanning four aspects of biological visual understanding, with 1.86M questions over 217K images from 11 public datasets.","excerpt":"Bio-VQA contains 1.86M questions over 217K images from 11 public datasets","support":"source-stated"},{"text":"The authors report that the best evaluated model counted animals exactly in 41.7% of images containing 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts.","excerpt":"the best model counts animals exactly on 41.7% of images with 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts","support":"source-stated"},{"text":"The authors report taxonomic accuracy falling from 0.72 at phylum to 0.12 at scientific name, and say scaling helps mostly at coarse ranks.","excerpt":"Taxonomic accuracy falls to one sixth, from 0.72 at phylum to 0.12 at scientific name, and scaling helps mostly at coarse ranks.","support":"source-stated"},{"text":"The authors report that 15 of 18 models did no better than an image-blind baseline on trait size measurement, and 17 fell below chance at re-identification under adversarial conditions.","excerpt":"On trait size measurement, 15 of the 18 models do no better than an image-blind baseline that predicts one fixed number for every specimen, and on re-identification, 17 of the 18 models fall below chance under adversarial conditions.","support":"source-stated"},{"text":"The abstract reports benchmark results but does not provide a comparison with performance in biodiversity monitoring settings.","excerpt":"In biodiversity monitoring, a single image raises several questions: which taxon, how many, which trait, and whether the individual has been seen before.","support":"interpretation"}],"limitations":["The reported evaluations cover 18 open-weight models ranging from 3.75B to 38.4B parameters; the abstract does not report results for other models.","The source gives benchmark results, including re-identification under adversarial conditions, but does not report performance in biodiversity monitoring settings."],"mediaId":"00c200ef9629a45d82743841cac47aab7549763b63f80b260c12ff227c39c867","correctionReason":null,"id":"620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9","publicationId":"9f05e99e-1587-4343-8185-287ebaeeaa69","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05559","sourceTitle":"How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:32.942Z","createdAt":"2026-11-10T01:53:54.691Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"00c200ef9629a45d82743841cac47aab7549763b63f80b260c12ff227c39c867","kind":"article","url":"https://arxiv.org/abs/2611.05559","sourceUrl":"https://arxiv.org/abs/2611.05559","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T01:53:09.443Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05559","checkedAt":"2026-11-10T02:56:08.842Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"00c200ef9629a45d82743841cac47aab7549763b63f80b260c12ff227c39c867","confidence":0.91,"provider":"typesafe","model":"jev-1.13.0","inputHash":"7fda1f262bbae7d4a1793a4305242a1d9e13b87439e332cb1f3f9dc068d890d1","decidedAt":"2026-11-10T01:53:10.750Z"},"provenance":{"textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observationId":"a087b8ffe9a2c2223abcf10e0449086d18203ef0a4082d7c096fb94199d88c73","retrievedAt":"2026-11-10T01:53:09.242Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"b57e9e3a-816a-4433-83d3-1bb18f68df0f"},"reviewer":{"model":"gpt-6-luna","runId":"e1788670-873d-45b6-8b67-1f54a95a9f36"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T01:53:54.691Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Rayeed, S M","url":null,"kind":"unknown"},{"name":"Ma, Qian","url":null,"kind":"unknown"},{"name":"Stewart, Charles V.","url":null,"kind":"unknown"},{"name":"Ma, Yao","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Rayeed, S M","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rayeed, S M","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}},{"actor":{"name":"Ma, Qian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Qian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}},{"actor":{"name":"Stewart, Charles V.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Stewart, Charles V.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}},{"actor":{"name":"Ma, Yao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ma, Yao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05559","locator":"head/meta[name=author|citation_author]"},"assertionId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T02:56:08.842Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","sourceUrl":"https://arxiv.org/abs/2611.05559","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05559","snapshotId":"927ef29059b5ac052b7b46b8ce2729a8eaea9db207aa6f8024ef9157fe1cf421","contentSha256":"2eaad259e2e4ba37f3996a80efb14ff4f71072021116cddb2a3111ff16c6cc09","textHash":"141fec236cc4bfcc5e0a59a203968ea75fc06d65019b2223d3b3e1d569eb2c3d","observedAt":"2026-11-10T02:56:08.842Z","basis":"publisher-page-metadata","sourceLinks":43,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/620ef1add377c008087e1efc0d94e9a5e8c8babb0cfcca907db9c0c8ac4495f9/source"}}}],"audits":[{"publicationId":"9f05e99e-1587-4343-8185-287ebaeeaa69","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The benchmark and findings are attributed to the named authors (S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma) and framed as their reports. The paper is primarily cs.CV and cross-listed to cs.AI, so the arXiv cs.AI label is acceptable.","excerpt":"Authors:S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma"},{"dimension":"summary","status":"uncertain","explanation":"Mostly faithful, but it weakens the authors' force. (1) The source says capabilities \"do not transfer\"; the draft says \"do not transfer well\". Smallest repair: drop \"well\". (2) It omits two central results: the failures persist across a tenfold range of model size, and the widening gap to the specialist BioCLIP-2 (1.2x at class, 2x at scientific name). Smallest repair: add one sentence on each. (3) \"Image-blind baseline\" is not explained; the source defines it as predicting one fixed number for every specimen. Smallest repair: add that definition. (4) The gloss \"location\" for spatial reasoning drops the source's counting example (\"how many\").","excerpt":"These failures persist across a tenfold range of model size."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-03 matches the v1 submission date. The 2026-11-05 feed date is correctly not used.","excerpt":"[Submitted on 3 Nov 2026]"},{"dimension":"support","status":"uncertain","explanation":"The numeric claims match the abstract exactly, and the interpretation claim correctly notes that no real-world monitoring comparison is reported. However, the plain-language gloss of the four aspects as \"its traits and location\" is not supported. The source's own example lists counting (\"how many\") for the spatial aspect, not location. Smallest repair: change \"location\" to \"how many there are\".","excerpt":"which taxon, how many, which trait, and whether the individual has been seen before"},{"dimension":"media","status":"pass","explanation":"The selected media item is the verified direct link to the arXiv abstract page of this paper, so it is the original source.","excerpt":null}],"createdAt":"2026-11-10T01:53:54.691Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}