{"id":"84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87","title":"arXiv q-bio.NC: CogArena tests multimodal agents in interactive behavioral experiments","argument":"In their paper, Kianté Fernandez and coauthors introduce CogArena, a benchmark that tests AI agents on ten experiments from the social and behavioral sciences. Rather than presenting the tasks as text prompts, the benchmark has agents use a web browser with the visual stimuli and response interfaces used by online participants. Its scoring checks whether an agent finishes a task, how accurately it performs, and whether its data show behavioral signatures—the patterns each experiment reliably produces in people. The authors report that, across four multimodal models tested ten times per task, many sessions did not finish; completed sessions reproduced some signatures reliably and others not at all. They argue that task completion and reproducing human behavioral patterns are different capabilities, relevant to concerns about AI-generated data entering online research datasets. The paper provides the tasks, scoring code, and a public leaderboard. The abstract does not identify which experiments or signatures were reproduced reliably, or quantify how often online datasets are affected by AI agents.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not specify which experimental paradigms or behavioral signatures were reproduced reliably, or provide rates of completion and reproduction. The abstract raises concerns about contamination of online research datasets but does not quantify its prevalence or establish the extent of any contamination. The reported evaluation covers four multimodal models; the abstract does not name them or establish how results generalize to other models.","speaker":{"name":"Fernandez, Kianté, Chen, Caitlin, Zhou, Jialu, Sadowski, Bartek, Miceli, Anthony C., Krajbich, Ian","roleAtSource":"Article author"},"source":{"title":"CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments","publisher":"arXiv q-bio.NC","format":"Article","url":"https://arxiv.org/abs/2610.36484","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"The paper introduces CogArena, a benchmark of ten experiments from the social and behavioral sciences, conducted through a browser interface.","excerpt":"We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"CogArena presents the experiments through visual stimuli and response interfaces like those used by online participants, rather than reducing them to text prompts.","excerpt":"Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The scoring procedure separately assesses task completion, accuracy, and whether the agent's data show behavioral signatures found in people.","excerpt":"A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The authors report that many sessions did not finish; completed sessions reproduced some behavioral signatures reliably and others not at all.","excerpt":"Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The evaluation covered four multimodal models, with each run ten times per task.","excerpt":"Across four multimodal models, each run ten times per task","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The authors say completing an experiment and reproducing its behavioral signatures are different things.","excerpt":"finishing an experiment and reproducing its signatures are different things.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The authors cite concerns that AI agents completing online behavioral tasks could contaminate datasets in social and behavioral sciences.","excerpt":"AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36484","checkedAt":"2026-10-29T20:18:37.152Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which experiments and behavioral signatures did the tested agents reproduce reliably, and how often do AI agents contribute data to online research datasets?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv q-bio.NC: CogArena tests multimodal agents in interactive behavioral experiments","summary":"In their paper, Kianté Fernandez and coauthors introduce CogArena, a benchmark that tests AI agents on ten experiments from the social and behavioral sciences. Rather than presenting the tasks as text prompts, the benchmark has agents use a web browser with the visual stimuli and response interfaces used by online participants. Its scoring checks whether an agent finishes a task, how accurately it performs, and whether its data show behavioral signatures—the patterns each experiment reliably produces in people. The authors report that, across four multimodal models tested ten times per task, many sessions did not finish; completed sessions reproduced some signatures reliably and others not at all. They argue that task completion and reproducing human behavioral patterns are different capabilities, relevant to concerns about AI-generated data entering online research datasets. The paper provides the tasks, scoring code, and a public leaderboard. The abstract does not identify which experiments or signatures were reproduced reliably, or quantify how often online datasets are affected by AI agents.","question":"Which experiments and behavioral signatures did the tested agents reproduce reliably, and how often do AI agents contribute data to online research datasets?","claims":[{"text":"The paper introduces CogArena, a benchmark of ten experiments from the social and behavioral sciences, conducted through a browser interface.","excerpt":"We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences.","support":"source-stated"},{"text":"CogArena presents the experiments through visual stimuli and response interfaces like those used by online participants, rather than reducing them to text prompts.","excerpt":"Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use.","support":"source-stated"},{"text":"The scoring procedure separately assesses task completion, accuracy, and whether the agent's data show behavioral signatures found in people.","excerpt":"A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people.","support":"source-stated"},{"text":"The authors report that many sessions did not finish; completed sessions reproduced some behavioral signatures reliably and others not at all.","excerpt":"Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all.","support":"source-stated"},{"text":"The evaluation covered four multimodal models, with each run ten times per task.","excerpt":"Across four multimodal models, each run ten times per task","support":"source-stated"},{"text":"The authors say completing an experiment and reproducing its behavioral signatures are different things.","excerpt":"finishing an experiment and reproducing its signatures are different things.","support":"source-stated"},{"text":"The authors cite concerns that AI agents completing online behavioral tasks could contaminate datasets in social and behavioral sciences.","excerpt":"AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination.","support":"source-stated"}],"limitations":["The abstract does not specify which experimental paradigms or behavioral signatures were reproduced reliably, or provide rates of completion and reproduction.","The abstract raises concerns about contamination of online research datasets but does not quantify its prevalence or establish the extent of any contamination.","The reported evaluation covers four multimodal models; the abstract does not name them or establish how results generalize to other models."],"mediaId":"5f3dc98e7cb7ecb8532ceff637e7d6b8ddd07d5f62337e08920030734958eb6f","correctionReason":null,"id":"84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87","publicationId":"81253fe6-93e9-4f21-8f54-d63af3bb0893","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36484","sourceTitle":"CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments","publisher":"arXiv q-bio.NC","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T04:12:43.668Z","createdAt":"2026-10-29T16:02:51.260Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"5f3dc98e7cb7ecb8532ceff637e7d6b8ddd07d5f62337e08920030734958eb6f","kind":"article","url":"https://arxiv.org/abs/2610.36484","sourceUrl":"https://arxiv.org/abs/2610.36484","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-29T16:02:02.758Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36484","checkedAt":"2026-10-29T20:18:37.152Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"5f3dc98e7cb7ecb8532ceff637e7d6b8ddd07d5f62337e08920030734958eb6f","confidence":0.8200000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"af6f057c521956ec3e319ec225b645b7bde3df882cc837be87e837814c7a1bd7","decidedAt":"2026-10-29T16:02:07.345Z"},"provenance":{"textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observationId":"2b774935e83722b4d2b2e717850a15fa7f3e840ae4e7079ba9d6fbd9b8672e76","retrievedAt":"2026-10-29T16:02:02.456Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"db6f2bdf-da54-45b4-868a-cee6c84f288a"},"reviewer":{"model":"gpt-6-luna","runId":"8300ef4c-1227-42b0-8707-bca1a9682a92"}},"audit":{"status":"passed","reviewedAt":"2026-10-29T16:02:51.260Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Fernandez, Kianté","url":null,"kind":"unknown"},{"name":"Chen, Caitlin","url":null,"kind":"unknown"},{"name":"Zhou, Jialu","url":null,"kind":"unknown"},{"name":"Sadowski, Bartek","url":null,"kind":"unknown"},{"name":"Miceli, Anthony C.","url":null,"kind":"unknown"},{"name":"Krajbich, Ian","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Fernandez, Kianté","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Fernandez, Kianté","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Chen, Caitlin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Caitlin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Zhou, Jialu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhou, Jialu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Sadowski, Bartek","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sadowski, Bartek","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Miceli, Anthony C.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Miceli, Anthony C.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Krajbich, Ian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Krajbich, Ian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36484","snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z","basis":"publisher-page-metadata","sourceLinks":48,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-q-bio-nc-cogarena-tests-multimodal-agents-in-interactive-84a0b82b","formats":{"html":"/readings/arxiv-q-bio-nc-cogarena-tests-multimodal-agents-in-interactive-84a0b82b","markdown":"/readings/arxiv-q-bio-nc-cogarena-tests-multimodal-agents-in-interactive-84a0b82b.md","json":"/readings/arxiv-q-bio-nc-cogarena-tests-multimodal-agents-in-interactive-84a0b82b.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-multimodal-model","name":"Multimodal model","explanation":"A model that works with more than one kind of input or output, such as text, images and audio.","referenceUrl":"https://developers.google.com/machine-learning/glossary#multimodal-model"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87","revision":"81253fe6-93e9-4f21-8f54-d63af3bb0893","revisionKind":"publication-id","title":"arXiv q-bio.NC: CogArena tests multimodal agents in interactive behavioral experiments","url":"https://escapemeridian.com/readings/arxiv-q-bio-nc-cogarena-tests-multimodal-agents-in-interactive-84a0b82b"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.36484","revision":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","revisionKind":"sha256-source-text","title":"CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments","url":"https://arxiv.org/abs/2610.36484"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87","title":"arXiv q-bio.NC: CogArena tests multimodal agents in interactive behavioral experiments","argument":"In their paper, Kianté Fernandez and coauthors introduce CogArena, a benchmark that tests AI agents on ten experiments from the social and behavioral sciences. Rather than presenting the tasks as text prompts, the benchmark has agents use a web browser with the visual stimuli and response interfaces used by online participants. Its scoring checks whether an agent finishes a task, how accurately it performs, and whether its data show behavioral signatures—the patterns each experiment reliably produces in people. The authors report that, across four multimodal models tested ten times per task, many sessions did not finish; completed sessions reproduced some signatures reliably and others not at all. They argue that task completion and reproducing human behavioral patterns are different capabilities, relevant to concerns about AI-generated data entering online research datasets. The paper provides the tasks, scoring code, and a public leaderboard. The abstract does not identify which experiments or signatures were reproduced reliably, or quantify how often online datasets are affected by AI agents.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not specify which experimental paradigms or behavioral signatures were reproduced reliably, or provide rates of completion and reproduction. The abstract raises concerns about contamination of online research datasets but does not quantify its prevalence or establish the extent of any contamination. The reported evaluation covers four multimodal models; the abstract does not name them or establish how results generalize to other models.","speaker":{"name":"Fernandez, Kianté, Chen, Caitlin, Zhou, Jialu, Sadowski, Bartek, Miceli, Anthony C., Krajbich, Ian","roleAtSource":"Article author"},"source":{"title":"CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments","publisher":"arXiv q-bio.NC","format":"Article","url":"https://arxiv.org/abs/2610.36484","publishedAt":"2026-10-27","dateBasis":"Date basis not provided."},"evidence":[{"text":"The paper introduces CogArena, a benchmark of ten experiments from the social and behavioral sciences, conducted through a browser interface.","excerpt":"We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"CogArena presents the experiments through visual stimuli and response interfaces like those used by online participants, rather than reducing them to text prompts.","excerpt":"Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The scoring procedure separately assesses task completion, accuracy, and whether the agent's data show behavioral signatures found in people.","excerpt":"A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The authors report that many sessions did not finish; completed sessions reproduced some behavioral signatures reliably and others not at all.","excerpt":"Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The evaluation covered four multimodal models, with each run ten times per task.","excerpt":"Across four multimodal models, each run ten times per task","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The authors say completing an experiment and reproducing its behavioral signatures are different things.","excerpt":"finishing an experiment and reproducing its signatures are different things.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"},{"text":"The authors cite concerns that AI agents completing online behavioral tasks could contaminate datasets in social and behavioral sciences.","excerpt":"AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination.","url":"https://arxiv.org/abs/2610.36484","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36484","checkedAt":"2026-10-29T20:18:37.152Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which experiments and behavioral signatures did the tested agents reproduce reliably, and how often do AI agents contribute data to online research datasets?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv q-bio.NC: CogArena tests multimodal agents in interactive behavioral experiments","summary":"In their paper, Kianté Fernandez and coauthors introduce CogArena, a benchmark that tests AI agents on ten experiments from the social and behavioral sciences. Rather than presenting the tasks as text prompts, the benchmark has agents use a web browser with the visual stimuli and response interfaces used by online participants. Its scoring checks whether an agent finishes a task, how accurately it performs, and whether its data show behavioral signatures—the patterns each experiment reliably produces in people. The authors report that, across four multimodal models tested ten times per task, many sessions did not finish; completed sessions reproduced some signatures reliably and others not at all. They argue that task completion and reproducing human behavioral patterns are different capabilities, relevant to concerns about AI-generated data entering online research datasets. The paper provides the tasks, scoring code, and a public leaderboard. The abstract does not identify which experiments or signatures were reproduced reliably, or quantify how often online datasets are affected by AI agents.","question":"Which experiments and behavioral signatures did the tested agents reproduce reliably, and how often do AI agents contribute data to online research datasets?","claims":[{"text":"The paper introduces CogArena, a benchmark of ten experiments from the social and behavioral sciences, conducted through a browser interface.","excerpt":"We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences.","support":"source-stated"},{"text":"CogArena presents the experiments through visual stimuli and response interfaces like those used by online participants, rather than reducing them to text prompts.","excerpt":"Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use.","support":"source-stated"},{"text":"The scoring procedure separately assesses task completion, accuracy, and whether the agent's data show behavioral signatures found in people.","excerpt":"A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people.","support":"source-stated"},{"text":"The authors report that many sessions did not finish; completed sessions reproduced some behavioral signatures reliably and others not at all.","excerpt":"Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all.","support":"source-stated"},{"text":"The evaluation covered four multimodal models, with each run ten times per task.","excerpt":"Across four multimodal models, each run ten times per task","support":"source-stated"},{"text":"The authors say completing an experiment and reproducing its behavioral signatures are different things.","excerpt":"finishing an experiment and reproducing its signatures are different things.","support":"source-stated"},{"text":"The authors cite concerns that AI agents completing online behavioral tasks could contaminate datasets in social and behavioral sciences.","excerpt":"AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination.","support":"source-stated"}],"limitations":["The abstract does not specify which experimental paradigms or behavioral signatures were reproduced reliably, or provide rates of completion and reproduction.","The abstract raises concerns about contamination of online research datasets but does not quantify its prevalence or establish the extent of any contamination.","The reported evaluation covers four multimodal models; the abstract does not name them or establish how results generalize to other models."],"mediaId":"5f3dc98e7cb7ecb8532ceff637e7d6b8ddd07d5f62337e08920030734958eb6f","correctionReason":null,"id":"84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87","publicationId":"81253fe6-93e9-4f21-8f54-d63af3bb0893","version":1,"sourceUrl":"https://arxiv.org/abs/2610.36484","sourceTitle":"CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments","publisher":"arXiv q-bio.NC","publishedAt":null,"originalPublished":{"value":"2026-10-27","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-28T04:12:43.668Z","createdAt":"2026-10-29T16:02:51.260Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"5f3dc98e7cb7ecb8532ceff637e7d6b8ddd07d5f62337e08920030734958eb6f","kind":"article","url":"https://arxiv.org/abs/2610.36484","sourceUrl":"https://arxiv.org/abs/2610.36484","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-29T16:02:02.758Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.36484","checkedAt":"2026-10-29T20:18:37.152Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"5f3dc98e7cb7ecb8532ceff637e7d6b8ddd07d5f62337e08920030734958eb6f","confidence":0.8200000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"af6f057c521956ec3e319ec225b645b7bde3df882cc837be87e837814c7a1bd7","decidedAt":"2026-10-29T16:02:07.345Z"},"provenance":{"textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observationId":"2b774935e83722b4d2b2e717850a15fa7f3e840ae4e7079ba9d6fbd9b8672e76","retrievedAt":"2026-10-29T16:02:02.456Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"db6f2bdf-da54-45b4-868a-cee6c84f288a"},"reviewer":{"model":"gpt-6-luna","runId":"8300ef4c-1227-42b0-8707-bca1a9682a92"}},"audit":{"status":"passed","reviewedAt":"2026-10-29T16:02:51.260Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Fernandez, Kianté","url":null,"kind":"unknown"},{"name":"Chen, Caitlin","url":null,"kind":"unknown"},{"name":"Zhou, Jialu","url":null,"kind":"unknown"},{"name":"Sadowski, Bartek","url":null,"kind":"unknown"},{"name":"Miceli, Anthony C.","url":null,"kind":"unknown"},{"name":"Krajbich, Ian","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Fernandez, Kianté","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Fernandez, Kianté","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Chen, Caitlin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Caitlin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Zhou, Jialu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhou, Jialu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Sadowski, Bartek","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sadowski, Bartek","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Miceli, Anthony C.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Miceli, Anthony C.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}},{"actor":{"name":"Krajbich, Ian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Krajbich, Ian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.36484","locator":"head/meta[name=author|citation_author]"},"assertionId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-10-29T20:18:37.152Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","sourceUrl":"https://arxiv.org/abs/2610.36484","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.36484","snapshotId":"ade92d7dea0801830d1f8457a558e52b1b55dfb480866e36b67959762ffa88a2","contentSha256":"546c55f3a8dd0f63213bbd9b63a2b795baa40382f06d0c0444d2faaae5cba9a5","textHash":"72ef4958775535c8165b796e3ada361bbc0435bdf59365eec9c531b608e35cd4","observedAt":"2026-10-29T20:18:37.152Z","basis":"publisher-page-metadata","sourceLinks":48,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/84a0b82bda73fc3c011615e23154c1a5832bd592c78d0e216d5876336c44ed87/source"}}}],"audits":[{"publicationId":"81253fe6-93e9-4f21-8f54-d63af3bb0893","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The paper and its claims are correctly attributed to Kianté Fernandez and coauthors on arXiv. The page lists cs.AI as primary and q-bio.NC as a cross-list, so the publisher label is acceptable.","excerpt":"Authors:Kianté Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C. Miceli, Ian Krajbich"},{"dimension":"summary","status":"pass","explanation":"The summary explains the benchmark, the browser-based setup and the three-level scoring, and defines behavioral signatures in plain language. It reports the findings faithfully, ties them to the contamination concern stated in the source and notes what the abstract leaves unknown. The question follows from those gaps.","excerpt":"we find that finishing an experiment and reproducing its signatures are different things."},{"dimension":"date","status":"pass","explanation":"The originalPublished date, 2026-10-27, matches the v1 submission date. The feed date of 2026-10-29 is later and was correctly not used.","excerpt":"[Submitted on 27 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim is supported by an exact abstract excerpt. The contamination concern stays attributed and unquantified, and no real-world reliability or benefit is overstated.","excerpt":"Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the arXiv abstract page, which is the original source.","excerpt":null}],"createdAt":"2026-10-29T16:02:51.260Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}