{"id":"631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65","title":"arXiv cs.AI: Do large language models rank moral values? A transitivity test of moral competence","argument":"The authors argue that strong performance on morality benchmarks does not show that a large language model can weigh competing moral values in a decision. They test a minimal condition for that ability: whether the model’s moral choices can be represented by an ordinal ranking, an ordering of values. Such a ranking requires transitivity: if one value is preferred to a second, and the second to a third, the first must also be preferred to the third.\n\nThe authors presented models with pairwise trade-offs between moral value violations in vignettes and derived choice probabilities from first-token distributions. They report that most choice sets admitted weak or unstable orderings when compared with three stochastic transitive choice models. They also report that aggregating responses—a common practice in LLM moral evaluation—produced transitive structure that matched no individual choice set.\n\nThe distinction matters for how moral evaluations are interpreted: according to the authors, producing appropriate benchmark answers is not enough to establish that choices are structured for weighing values. They argue that evaluations should establish this structure before asking whether choices are aligned. The supplied abstract does not identify the models or describe the choice sets in detail, so it does not show how the reported results vary across models or scenarios.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied source text contains the abstract but does not identify the tested models or provide details of the vignette choice sets, limiting assessment of how the findings vary across models or scenarios.","speaker":{"name":"Wasson, Katelyn A., Song, Kiwon, Jennings III, James M., Currie, Brittney, Sokratous, Konstantina, Stober, Clintin P.","roleAtSource":"Article author"},"source":{"title":"Do large language models rank moral values? A transitivity test of moral competence","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.10265","publishedAt":"2026-11-06","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors distinguish moral performance on benchmarks from moral competence in weighing competing values; benchmark performance alone does not establish competence.","excerpt":"producing appropriate outputs on these benchmarks (moral performance) does not establish that an LLM weighs competing moral values at stake in a decision","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors test whether an ordinal ranking can represent LLM moral choices, a minimal necessary condition for weighing values, and note that such a ranking requires transitivity.","excerpt":"Such a ranking requires transitivity.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted choice probabilities from first-token distributions.","excerpt":"we presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted resulting choice probabilities from first-token distributions","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"Compared with three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","excerpt":"Tested against three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors report that aggregating responses produced transitive structure matching no individual choice set.","excerpt":"Aggregating an LLM's responses, a common practice in LLM moral evaluation, produced transitive structure that matched no individual choice set.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors argue that evaluations should establish whether choices are structured before assessing whether they are aligned.","excerpt":"Evaluations of AI morality should establish that choices are structured before asking whether they are aligned.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10265","checkedAt":"2026-11-12T10:34:14.917Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which model-specific and choice-set results underlie the finding that most choice sets admitted weak or unstable orderings?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Do large language models rank moral values? A transitivity test of moral competence","summary":"The authors argue that strong performance on morality benchmarks does not show that a large language model can weigh competing moral values in a decision. They test a minimal condition for that ability: whether the model’s moral choices can be represented by an ordinal ranking, an ordering of values. Such a ranking requires transitivity: if one value is preferred to a second, and the second to a third, the first must also be preferred to the third.\n\nThe authors presented models with pairwise trade-offs between moral value violations in vignettes and derived choice probabilities from first-token distributions. They report that most choice sets admitted weak or unstable orderings when compared with three stochastic transitive choice models. They also report that aggregating responses—a common practice in LLM moral evaluation—produced transitive structure that matched no individual choice set.\n\nThe distinction matters for how moral evaluations are interpreted: according to the authors, producing appropriate benchmark answers is not enough to establish that choices are structured for weighing values. They argue that evaluations should establish this structure before asking whether choices are aligned. The supplied abstract does not identify the models or describe the choice sets in detail, so it does not show how the reported results vary across models or scenarios.","question":"Which model-specific and choice-set results underlie the finding that most choice sets admitted weak or unstable orderings?","claims":[{"text":"The authors distinguish moral performance on benchmarks from moral competence in weighing competing values; benchmark performance alone does not establish competence.","excerpt":"producing appropriate outputs on these benchmarks (moral performance) does not establish that an LLM weighs competing moral values at stake in a decision","support":"source-stated"},{"text":"The authors test whether an ordinal ranking can represent LLM moral choices, a minimal necessary condition for weighing values, and note that such a ranking requires transitivity.","excerpt":"Such a ranking requires transitivity.","support":"source-stated"},{"text":"The authors presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted choice probabilities from first-token distributions.","excerpt":"we presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted resulting choice probabilities from first-token distributions","support":"source-stated"},{"text":"Compared with three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","excerpt":"Tested against three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","support":"source-stated"},{"text":"The authors report that aggregating responses produced transitive structure matching no individual choice set.","excerpt":"Aggregating an LLM's responses, a common practice in LLM moral evaluation, produced transitive structure that matched no individual choice set.","support":"source-stated"},{"text":"The authors argue that evaluations should establish whether choices are structured before assessing whether they are aligned.","excerpt":"Evaluations of AI morality should establish that choices are structured before asking whether they are aligned.","support":"source-stated"}],"limitations":["The supplied source text contains the abstract but does not identify the tested models or provide details of the vignette choice sets, limiting assessment of how the findings vary across models or scenarios."],"mediaId":"5e2831a3482515ca49143718359989e26979837111cbd79e7957aee2c0725298","correctionReason":null,"id":"631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65","publicationId":"dc3edc06-54a7-45fd-8675-36766e9600c7","version":1,"sourceUrl":"https://arxiv.org/abs/2611.10265","sourceTitle":"Do large language models rank moral values? A transitivity test of moral competence","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-06","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-10T08:33:49.835Z","createdAt":"2026-11-12T10:02:58.590Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"5e2831a3482515ca49143718359989e26979837111cbd79e7957aee2c0725298","kind":"article","url":"https://arxiv.org/abs/2611.10265","sourceUrl":"https://arxiv.org/abs/2611.10265","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T10:02:06.806Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10265","checkedAt":"2026-11-12T10:34:14.917Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"5e2831a3482515ca49143718359989e26979837111cbd79e7957aee2c0725298","confidence":0.55,"provider":"typesafe","model":"jev-1.13.0","inputHash":"12b302f1f9af3b10aa12478cb2ee70296b25cf0d20ac37ff5aae68a46e87b9d3","decidedAt":"2026-11-12T10:02:07.602Z"},"provenance":{"textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observationId":"050b07d5fdb2bf8d3d73ccdf9ce7c07472e7726d28f25869d4d5f8297a71dc90","retrievedAt":"2026-11-12T10:02:06.706Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d4fac482-b24f-403b-8342-a48b8ff75c30"},"reviewer":{"model":"gpt-6-luna","runId":"269d2826-4e6f-4b25-8a43-6a92db5e680f"}},"audit":{"status":"passed","reviewedAt":"2026-11-12T10:02:58.590Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wasson, Katelyn A.","url":null,"kind":"unknown"},{"name":"Song, Kiwon","url":null,"kind":"unknown"},{"name":"Jennings III, James M.","url":null,"kind":"unknown"},{"name":"Currie, Brittney","url":null,"kind":"unknown"},{"name":"Sokratous, Konstantina","url":null,"kind":"unknown"},{"name":"Stober, Clintin P.","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wasson, Katelyn A.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wasson, Katelyn A.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Song, Kiwon","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Song, Kiwon","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Jennings III, James M.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jennings III, James M.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Currie, Brittney","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Currie, Brittney","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Sokratous, Konstantina","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sokratous, Konstantina","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Stober, Clintin P.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Stober, Clintin P.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.10265","snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-do-large-language-models-rank-moral-values-a-631b69ad","formats":{"html":"/readings/arxiv-cs-ai-do-large-language-models-rank-moral-values-a-631b69ad","markdown":"/readings/arxiv-cs-ai-do-large-language-models-rank-moral-values-a-631b69ad.md","json":"/readings/arxiv-cs-ai-do-large-language-models-rank-moral-values-a-631b69ad.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65","revision":"dc3edc06-54a7-45fd-8675-36766e9600c7","revisionKind":"publication-id","title":"arXiv cs.AI: Do large language models rank moral values? A transitivity test of moral competence","url":"https://escapemeridian.com/readings/arxiv-cs-ai-do-large-language-models-rank-moral-values-a-631b69ad"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.10265","revision":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","revisionKind":"sha256-source-text","title":"Do large language models rank moral values? A transitivity test of moral competence","url":"https://arxiv.org/abs/2611.10265"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65","title":"arXiv cs.AI: Do large language models rank moral values? A transitivity test of moral competence","argument":"The authors argue that strong performance on morality benchmarks does not show that a large language model can weigh competing moral values in a decision. They test a minimal condition for that ability: whether the model’s moral choices can be represented by an ordinal ranking, an ordering of values. Such a ranking requires transitivity: if one value is preferred to a second, and the second to a third, the first must also be preferred to the third.\n\nThe authors presented models with pairwise trade-offs between moral value violations in vignettes and derived choice probabilities from first-token distributions. They report that most choice sets admitted weak or unstable orderings when compared with three stochastic transitive choice models. They also report that aggregating responses—a common practice in LLM moral evaluation—produced transitive structure that matched no individual choice set.\n\nThe distinction matters for how moral evaluations are interpreted: according to the authors, producing appropriate benchmark answers is not enough to establish that choices are structured for weighing values. They argue that evaluations should establish this structure before asking whether choices are aligned. The supplied abstract does not identify the models or describe the choice sets in detail, so it does not show how the reported results vary across models or scenarios.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied source text contains the abstract but does not identify the tested models or provide details of the vignette choice sets, limiting assessment of how the findings vary across models or scenarios.","speaker":{"name":"Wasson, Katelyn A., Song, Kiwon, Jennings III, James M., Currie, Brittney, Sokratous, Konstantina, Stober, Clintin P.","roleAtSource":"Article author"},"source":{"title":"Do large language models rank moral values? A transitivity test of moral competence","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.10265","publishedAt":"2026-11-06","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors distinguish moral performance on benchmarks from moral competence in weighing competing values; benchmark performance alone does not establish competence.","excerpt":"producing appropriate outputs on these benchmarks (moral performance) does not establish that an LLM weighs competing moral values at stake in a decision","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors test whether an ordinal ranking can represent LLM moral choices, a minimal necessary condition for weighing values, and note that such a ranking requires transitivity.","excerpt":"Such a ranking requires transitivity.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted choice probabilities from first-token distributions.","excerpt":"we presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted resulting choice probabilities from first-token distributions","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"Compared with three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","excerpt":"Tested against three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors report that aggregating responses produced transitive structure matching no individual choice set.","excerpt":"Aggregating an LLM's responses, a common practice in LLM moral evaluation, produced transitive structure that matched no individual choice set.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"},{"text":"The authors argue that evaluations should establish whether choices are structured before assessing whether they are aligned.","excerpt":"Evaluations of AI morality should establish that choices are structured before asking whether they are aligned.","url":"https://arxiv.org/abs/2611.10265","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10265","checkedAt":"2026-11-12T10:34:14.917Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Which model-specific and choice-set results underlie the finding that most choice sets admitted weak or unstable orderings?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Do large language models rank moral values? A transitivity test of moral competence","summary":"The authors argue that strong performance on morality benchmarks does not show that a large language model can weigh competing moral values in a decision. They test a minimal condition for that ability: whether the model’s moral choices can be represented by an ordinal ranking, an ordering of values. Such a ranking requires transitivity: if one value is preferred to a second, and the second to a third, the first must also be preferred to the third.\n\nThe authors presented models with pairwise trade-offs between moral value violations in vignettes and derived choice probabilities from first-token distributions. They report that most choice sets admitted weak or unstable orderings when compared with three stochastic transitive choice models. They also report that aggregating responses—a common practice in LLM moral evaluation—produced transitive structure that matched no individual choice set.\n\nThe distinction matters for how moral evaluations are interpreted: according to the authors, producing appropriate benchmark answers is not enough to establish that choices are structured for weighing values. They argue that evaluations should establish this structure before asking whether choices are aligned. The supplied abstract does not identify the models or describe the choice sets in detail, so it does not show how the reported results vary across models or scenarios.","question":"Which model-specific and choice-set results underlie the finding that most choice sets admitted weak or unstable orderings?","claims":[{"text":"The authors distinguish moral performance on benchmarks from moral competence in weighing competing values; benchmark performance alone does not establish competence.","excerpt":"producing appropriate outputs on these benchmarks (moral performance) does not establish that an LLM weighs competing moral values at stake in a decision","support":"source-stated"},{"text":"The authors test whether an ordinal ranking can represent LLM moral choices, a minimal necessary condition for weighing values, and note that such a ranking requires transitivity.","excerpt":"Such a ranking requires transitivity.","support":"source-stated"},{"text":"The authors presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted choice probabilities from first-token distributions.","excerpt":"we presented LLMs with pairwise trade-offs among moral value violations in vignette choice sets and extracted resulting choice probabilities from first-token distributions","support":"source-stated"},{"text":"Compared with three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","excerpt":"Tested against three stochastic transitive choice models, most choice sets admitted weak or unstable orderings.","support":"source-stated"},{"text":"The authors report that aggregating responses produced transitive structure matching no individual choice set.","excerpt":"Aggregating an LLM's responses, a common practice in LLM moral evaluation, produced transitive structure that matched no individual choice set.","support":"source-stated"},{"text":"The authors argue that evaluations should establish whether choices are structured before assessing whether they are aligned.","excerpt":"Evaluations of AI morality should establish that choices are structured before asking whether they are aligned.","support":"source-stated"}],"limitations":["The supplied source text contains the abstract but does not identify the tested models or provide details of the vignette choice sets, limiting assessment of how the findings vary across models or scenarios."],"mediaId":"5e2831a3482515ca49143718359989e26979837111cbd79e7957aee2c0725298","correctionReason":null,"id":"631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65","publicationId":"dc3edc06-54a7-45fd-8675-36766e9600c7","version":1,"sourceUrl":"https://arxiv.org/abs/2611.10265","sourceTitle":"Do large language models rank moral values? A transitivity test of moral competence","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-06","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-10T08:33:49.835Z","createdAt":"2026-11-12T10:02:58.590Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"5e2831a3482515ca49143718359989e26979837111cbd79e7957aee2c0725298","kind":"article","url":"https://arxiv.org/abs/2611.10265","sourceUrl":"https://arxiv.org/abs/2611.10265","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T10:02:06.806Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.10265","checkedAt":"2026-11-12T10:34:14.917Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"5e2831a3482515ca49143718359989e26979837111cbd79e7957aee2c0725298","confidence":0.55,"provider":"typesafe","model":"jev-1.13.0","inputHash":"12b302f1f9af3b10aa12478cb2ee70296b25cf0d20ac37ff5aae68a46e87b9d3","decidedAt":"2026-11-12T10:02:07.602Z"},"provenance":{"textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observationId":"050b07d5fdb2bf8d3d73ccdf9ce7c07472e7726d28f25869d4d5f8297a71dc90","retrievedAt":"2026-11-12T10:02:06.706Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d4fac482-b24f-403b-8342-a48b8ff75c30"},"reviewer":{"model":"gpt-6-luna","runId":"269d2826-4e6f-4b25-8a43-6a92db5e680f"}},"audit":{"status":"passed","reviewedAt":"2026-11-12T10:02:58.590Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wasson, Katelyn A.","url":null,"kind":"unknown"},{"name":"Song, Kiwon","url":null,"kind":"unknown"},{"name":"Jennings III, James M.","url":null,"kind":"unknown"},{"name":"Currie, Brittney","url":null,"kind":"unknown"},{"name":"Sokratous, Konstantina","url":null,"kind":"unknown"},{"name":"Stober, Clintin P.","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wasson, Katelyn A.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wasson, Katelyn A.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Song, Kiwon","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Song, Kiwon","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Jennings III, James M.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jennings III, James M.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Currie, Brittney","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Currie, Brittney","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Sokratous, Konstantina","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sokratous, Konstantina","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}},{"actor":{"name":"Stober, Clintin P.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Stober, Clintin P.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.10265","locator":"head/meta[name=author|citation_author]"},"assertionId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T10:34:14.917Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","sourceUrl":"https://arxiv.org/abs/2611.10265","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.10265","snapshotId":"55ae40b8940f1da572eb15ea965c4559fbdb45e74687f81210f4a19ba7e040a7","contentSha256":"8842f742cc48e0719f6f62cf1419c1ca3b0ec8ecf0ed9e93839c01cc3a670ccf","textHash":"05cfd2b60366d639087f5f00949b8f60240fade1da604102209a5962a6f5faaa","observedAt":"2026-11-12T10:34:14.917Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/631b69ad79e5f65c05ed41279e9f53305d17422126963f1b47e718cb92a34d65/source"}}}],"audits":[{"publicationId":"dc3edc06-54a7-45fd-8675-36766e9600c7","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading consistently attributes the argument, method and findings to 'the authors' of the arXiv paper. It keeps their conclusions as their claims rather than as established facts or Escape Meridian's views.","excerpt":"Authors:Katelyn A. Wasson, Kiwon Song, James M. Jennings III, Brittney Currie, Konstantina Sokratous, Clintin P. Stober"},{"dimension":"summary","status":"pass","explanation":"The reading faithfully covers the abstract: performance versus competence, the transitivity test, the method, both findings and the call to check structure before alignment. It explains ordinal ranking and transitivity plainly. Optional tooltips: stochastic transitive choice models, first-token distributions, vignette, alignment. Smallest repair: add the abstract's opening stake that LLMs are increasingly used to support decisions requiring moral consideration. The question addresses a real gap.","excerpt":"LLMs are increasingly used to support decisions that require moral consideration."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-06 matches the arXiv v1 submission date. The feed date of November 10 is correctly not used.","excerpt":"[Submitted on 6 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Every claim is quoted almost verbatim from the abstract and attributed to the authors. The stated limitation is accurate, because the abstract does not name the tested models. No benefits, costs or forecasts are invented.","excerpt":"Aggregating an LLM's responses, a common practice in LLM moral evaluation, produced transitive structure that matched no individual choice set."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the arXiv abstract page, which is the original source.","excerpt":null}],"createdAt":"2026-11-12T10:02:58.590Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}