# Researchers test detection of AI-agent commits without authorship signatures [Canonical reading](https://escapemeridian.com/readings/researchers-test-detection-of-ai-agent-commits-without-88331289) Researchers Kosei Horikawa and coauthors investigate whether commits made by AI coding agents can be distinguished from human commits after authorship signatures are removed. Their study analyzes 7,837,946 commits from 1,226 repositories, covering five coding agents and human commits. The researchers report that detectors ranked agent commits above human commits with a ROC AUC of up to 0.859, a measure of how well a detector separates two groups. Most of the signal came from commit-message verbosity and change size, rather than the code itself. But at the observed share of agent commits, the best detector achieved a precision of only 0.332: most commits it flagged were not agent commits. Detection also transferred poorly to held-out Claude Code and Copilot commits. The authors say signature-free detection may help prioritize commits for review, but cannot replace recorded attribution. For studies using signatures to identify agent commits, they argue that those labels should be treated as a lower bound on agent activity. An open question is whether these detection signals remain useful as agents and their commit practices change. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-05T16:31:33.678Z. First observed: 2026-10-28T07:02:43.816Z. Retrieved: 2026-10-30T03:32:07.429Z. - [Agent or Human? Detecting Commits by AI Coding Agents in the Absence of Authorship Signatures](https://arxiv.org/abs/2610.36229) ## Limits The evidence is the paper's abstract; it does not provide details of the study design or results beyond those summarized there. The authors report limited transfer to held-out Claude Code and Copilot commits; the abstract does not establish how well detection works for other agents or repositories. ## Open question How well will signature-free detectors identify agent commits as agents and their commit practices change? ## Source-stated The study analyzes 7,837,946 commits from 1,226 repositories, covering five coding agents and human commits. > using 7,837,946 commits from 1,226 repositories, covering five coding agents and Human commits. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated After authorship signatures were masked, detectors ranked agent commits above human commits with a ROC AUC of up to 0.859; most of the signal came from message verbosity and change size, not code. > After signatures are masked, detectors rank Agent commits above Human commits with a ROC AUC of up to 0.859, and most of this signal comes from message verbosity and change size rather than from the code itself. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated At the observed share of agent commits, the best detector achieved a precision of 0.332 on the full test set. > At the observed Agent share, however, the best detector on the full test set reaches a precision of only 0.332. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated Detection transferred poorly to held-out Claude Code and Copilot commits; TF-IDF reached an AUC of 0.524 for Claude Code. > Detection transfers poorly to held-out Claude Code and Copilot, with TF-IDF reaching an AUC of only 0.524 for Claude Code. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated The authors say signature-free detection can prioritize commits for review but cannot replace recorded attribution; studies relying on signature-based labels should treat them as a lower bound on agent activity. > These results indicate that signature-free detection can prioritize commits for review but cannot replace recorded attribution, and that studies relying on signature-based labels should treat them as a lower bound on agent activity. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source and contributor context Horikawa, Kosei, Li, Hao, Kashiwa, Yutaro, Ouatiti, Youssef Esseddiq, Sayagh, Mohammed, Adams, Bram, Iida, Hajimu, Hassan, Ahmed E. · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: 017f907a-da12-4264-82f5-44cc127c2940. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-05T16:31:33.678Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 18d7a4b6d2e8bfe715b622370359cc86c7961c3f04af9b9b0cd302f628257812. Source text hash: 5c9f72e695786c433116957db8f9e0f2bf4f42e78ec2a7f423ed59ccf9302d99. ## Revision and source-check history ### Version 1 — 2026-11-05T16:31:33.678Z Researchers Kosei Horikawa and coauthors investigate whether commits made by AI coding agents can be distinguished from human commits after authorship signatures are removed. Their study analyzes 7,837,946 commits from 1,226 repositories, covering five coding agents and human commits. The researchers report that detectors ranked agent commits above human commits with a ROC AUC of up to 0.859, a measure of how well a detector separates two groups. Most of the signal came from commit-message verbosity and change size, rather than the code itself. But at the observed share of agent commits, the best detector achieved a precision of only 0.332: most commits it flagged were not agent commits. Detection also transferred poorly to held-out Claude Code and Copilot commits. The authors say signature-free detection may help prioritize commits for review, but cannot replace recorded attribution. For studies using signatures to identify agent commits, they argue that those labels should be treated as a lower bound on agent activity. An open question is whether these detection signals remain useful as agents and their commit practices change. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-05T16:31:33.678Z. First observed: 2026-10-28T07:02:43.816Z. Retrieved: 2026-10-30T03:32:07.429Z. - [Agent or Human? Detecting Commits by AI Coding Agents in the Absence of Authorship Signatures](https://arxiv.org/abs/2610.36229) ## Limits The evidence is the paper's abstract; it does not provide details of the study design or results beyond those summarized there. The authors report limited transfer to held-out Claude Code and Copilot commits; the abstract does not establish how well detection works for other agents or repositories. ## Open question How well will signature-free detectors identify agent commits as agents and their commit practices change? ## Source-stated The study analyzes 7,837,946 commits from 1,226 repositories, covering five coding agents and human commits. > using 7,837,946 commits from 1,226 repositories, covering five coding agents and Human commits. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated After authorship signatures were masked, detectors ranked agent commits above human commits with a ROC AUC of up to 0.859; most of the signal came from message verbosity and change size, not code. > After signatures are masked, detectors rank Agent commits above Human commits with a ROC AUC of up to 0.859, and most of this signal comes from message verbosity and change size rather than from the code itself. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated At the observed share of agent commits, the best detector achieved a precision of 0.332 on the full test set. > At the observed Agent share, however, the best detector on the full test set reaches a precision of only 0.332. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated Detection transferred poorly to held-out Claude Code and Copilot commits; TF-IDF reached an AUC of 0.524 for Claude Code. > Detection transfers poorly to held-out Claude Code and Copilot, with TF-IDF reaching an AUC of only 0.524 for Claude Code. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source-stated The authors say signature-free detection can prioritize commits for review but cannot replace recorded attribution; studies relying on signature-based labels should treat them as a lower bound on agent activity. > These results indicate that signature-free detection can prioritize commits for review but cannot replace recorded attribution, and that studies relying on signature-based labels should treat them as a lower bound on agent activity. - [Supporting source](https://arxiv.org/abs/2610.36229) ## Source and contributor context Horikawa, Kosei, Li, Hao, Kashiwa, Yutaro, Ouatiti, Youssef Esseddiq, Sayagh, Mohammed, Adams, Bram, Iida, Hajimu, Hassan, Ahmed E. · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Claude Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding. - [Explore this context](/entities/claude) - [Reference](https://claude.com/product/overview) - [Reference](https://www.anthropic.com/news) - [Reference](https://www.anthropic.com/research/team/economics) ## Editorial provenance Version 1. Publication ID: 017f907a-da12-4264-82f5-44cc127c2940. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-05T16:31:33.678Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 18d7a4b6d2e8bfe715b622370359cc86c7961c3f04af9b9b0cd302f628257812. Source text hash: 5c9f72e695786c433116957db8f9e0f2bf4f42e78ec2a7f423ed59ccf9302d99. ### uncertain — 2026-11-05T16:31:33.678Z — anthropic/claude-opus-5.5 Publication: 017f907a-da12-4264-82f5-44cc127c2940 attribution — pass: The draft attributes the study and its conclusions to Kosei Horikawa and coauthors, matching the arXiv author list. Reported results and recommendations are framed as the researchers' findings and arguments, not as established fact. > Authors:Kosei Horikawa, Hao Li, Yutaro Kashiwa, Youssef Esseddiq Ouatiti, Mohammed Sayagh, Bram Adams, Hajimu Iida, Ahmed E. Hassan summary — uncertain: Numbers and conclusions are accurate, and the draft explains ROC AUC and precision. However, the central term 'authorship signatures' is never explained. The draft also omits why the study matters: signatures such as bot accounts and Co-authored-by trailers are optional, removable and often attached to pull requests rather than commits. 'Commit' and 'held-out' are also unexplained for non-specialists. Smallest repair: add one sentence defining signatures and why they go missing, and define held-out as agents excluded from training. > These signatures are optional, can be removed, and are often attached to pull requests rather than to individual commits. date — pass: originalPublished 2026-10-26 matches the v1 submission date shown on the arXiv page. It is not the later feed date. > \[Submitted on 26 Oct 2026\] support — pass: Each claim matches the abstract verbatim. The inference that most flagged commits were not agent commits follows from a precision of 0.332. 'May help prioritize' slightly softens the source's 'can prioritize' without changing its meaning. The open question about changing agents follows from the reported temporal and transfer limits and is not a disguised claim. > At the observed Agent share, however, the best detector on the full test set reaches a precision of only 0.332. media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/88331289a09c5a8908460bc1e0ceafff780c756a19021738ec50ecacd7a26d4c/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/researchers-test-detection-of-ai-agent-commits-without-88331289.json)