{"id":"3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd","title":"arXiv cs.AI: TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","argument":"The authors present TACT, a procedure for using AI to analyze qualitative responses while leaving teams in control of interpretation and judgment. They frame it as an alternative to either leaving large collections of responses unanalyzed or accepting a large language model’s interpretation without that control. TACT-Lite uses the core AI council procedure; TACT-Full adds adversaries—models from other model families that critique each model’s output—and a decision model that surfaces high-uncertainty code assignments. In an evaluation on 256 Stack Overflow threads coded by other researchers, TACT’s candidate codebook achieved an F1 of 0.75–0.77, and its final coding achieved an F1 of 0.71–0.72 against those labels. On a held-out sample, the authors’ manual coding and TACT’s coding had a Cohen’s κ of 0.79–0.85. The authors also report model-call costs of $13.77 for TACT-Lite and $33.59 for TACT-Full per 100 threads. They say the paper identifies pitfalls and safeguards, but the abstract does not describe those safeguards in detail.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation described in the abstract uses an independent dataset of 256 Stack Overflow threads; it does not establish how TACT performs on other qualitative datasets. The abstract reports that the authors identify pitfalls and safeguards, but does not describe them in detail. The abstract does not provide further details about the held-out sample or the evaluation procedure.","speaker":{"name":"Choudhuri, Rudrajit, Bird, Christian, Badea, Carmen, Sarma, Anita","roleAtSource":"Article author"},"source":{"title":"TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05702","publishedAt":"2026-11-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present TACT as a procedure for analyzing qualitative responses at scale while keeping teams in control of interpretation and judgment.","excerpt":"a procedure that enables such analysis at scale while keeping teams in control of interpretation and judgment","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"},{"text":"TACT-Full adds models from other model families that critique each model’s output, and a decision model that surfaces high-uncertainty code assignments.","excerpt":"adds adversaries (models from other model families that critique each model's output) and a decision model that surfaces high-uncertainty code assignments","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"},{"text":"On an independent dataset of 256 Stack Overflow threads, the authors report candidate codebook F1 of 0.75–0.77 and final coding F1 of 0.71–0.72 against other researchers’ labels.","excerpt":"Against their labels, TACT's candidate codebook achieved an F1 of 0.75-0.77 and its final coding an F1 of 0.71-0.72.","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"},{"text":"The authors report Cohen’s κ of 0.79–0.85 for agreement between their own manual coding and TACT’s coding on a held-out sample, and model-call costs of $13.77 to $33.59 per 100 threads.","excerpt":"On a held-out sample, agreement between our own manual coding and TACT's coding was Cohen's $\\kappa$ of 0.79-0.85. TACT is also economical: model calls cost \\$13.77 (TACT-Lite) to \\$33.59 (TACT-Full) per 100 threads.","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05702","checkedAt":"2026-11-09T07:48:06.732Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would TACT’s reported coding results compare on qualitative datasets beyond the 256 Stack Overflow threads used in this evaluation?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","summary":"The authors present TACT, a procedure for using AI to analyze qualitative responses while leaving teams in control of interpretation and judgment. They frame it as an alternative to either leaving large collections of responses unanalyzed or accepting a large language model’s interpretation without that control. TACT-Lite uses the core AI council procedure; TACT-Full adds adversaries—models from other model families that critique each model’s output—and a decision model that surfaces high-uncertainty code assignments. In an evaluation on 256 Stack Overflow threads coded by other researchers, TACT’s candidate codebook achieved an F1 of 0.75–0.77, and its final coding achieved an F1 of 0.71–0.72 against those labels. On a held-out sample, the authors’ manual coding and TACT’s coding had a Cohen’s κ of 0.79–0.85. The authors also report model-call costs of $13.77 for TACT-Lite and $33.59 for TACT-Full per 100 threads. They say the paper identifies pitfalls and safeguards, but the abstract does not describe those safeguards in detail.","question":"How would TACT’s reported coding results compare on qualitative datasets beyond the 256 Stack Overflow threads used in this evaluation?","claims":[{"text":"The authors present TACT as a procedure for analyzing qualitative responses at scale while keeping teams in control of interpretation and judgment.","excerpt":"a procedure that enables such analysis at scale while keeping teams in control of interpretation and judgment","support":"source-stated"},{"text":"TACT-Full adds models from other model families that critique each model’s output, and a decision model that surfaces high-uncertainty code assignments.","excerpt":"adds adversaries (models from other model families that critique each model's output) and a decision model that surfaces high-uncertainty code assignments","support":"source-stated"},{"text":"On an independent dataset of 256 Stack Overflow threads, the authors report candidate codebook F1 of 0.75–0.77 and final coding F1 of 0.71–0.72 against other researchers’ labels.","excerpt":"Against their labels, TACT's candidate codebook achieved an F1 of 0.75-0.77 and its final coding an F1 of 0.71-0.72.","support":"source-stated"},{"text":"The authors report Cohen’s κ of 0.79–0.85 for agreement between their own manual coding and TACT’s coding on a held-out sample, and model-call costs of $13.77 to $33.59 per 100 threads.","excerpt":"On a held-out sample, agreement between our own manual coding and TACT's coding was Cohen's $\\kappa$ of 0.79-0.85. TACT is also economical: model calls cost \\$13.77 (TACT-Lite) to \\$33.59 (TACT-Full) per 100 threads.","support":"source-stated"}],"limitations":["The evaluation described in the abstract uses an independent dataset of 256 Stack Overflow threads; it does not establish how TACT performs on other qualitative datasets.","The abstract reports that the authors identify pitfalls and safeguards, but does not describe them in detail.","The abstract does not provide further details about the held-out sample or the evaluation procedure."],"mediaId":"23299e52ee3e1c1e8c042d9e7ed5777d7fc196c3b0bcad65074e66283cf5b696","correctionReason":null,"id":"3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd","publicationId":"704aefb4-ab56-4cc2-8e12-80ee0d8d33b4","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05702","sourceTitle":"TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:03:28.124Z","createdAt":"2026-11-09T07:32:14.445Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"23299e52ee3e1c1e8c042d9e7ed5777d7fc196c3b0bcad65074e66283cf5b696","kind":"article","url":"https://arxiv.org/abs/2611.05702","sourceUrl":"https://arxiv.org/abs/2611.05702","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T07:25:22.587Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05702","checkedAt":"2026-11-09T07:48:06.732Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"23299e52ee3e1c1e8c042d9e7ed5777d7fc196c3b0bcad65074e66283cf5b696","confidence":0.78,"provider":"typesafe","model":"jev-1.13.0","inputHash":"10e83e4979157b4034f87ce68d6ef05ed55f4acae1f4b599f671fb273fe843e5","decidedAt":"2026-11-09T07:25:24.195Z"},"provenance":{"textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observationId":"47bbbc7cf47a176ebe10699adc871df1c9d226d417596cb6882a6fd6816f05d2","retrievedAt":"2026-11-09T07:25:22.326Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"819a306d-06cb-4f2f-8b45-f555ff63553a"},"reviewer":{"model":"gpt-6-luna","runId":"2b27af8e-769f-47ef-8c1f-b30ce7094eab"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T07:32:14.445Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Choudhuri, Rudrajit","url":null,"kind":"unknown"},{"name":"Bird, Christian","url":null,"kind":"unknown"},{"name":"Badea, Carmen","url":null,"kind":"unknown"},{"name":"Sarma, Anita","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Choudhuri, Rudrajit","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Choudhuri, Rudrajit","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}},{"actor":{"name":"Bird, Christian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bird, Christian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}},{"actor":{"name":"Badea, Carmen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Badea, Carmen","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}},{"actor":{"name":"Sarma, Anita","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sarma, Anita","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05702","snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-tact-a-pragmatic-approach-to-qualitative-analysis-3e3b3276","formats":{"html":"/readings/arxiv-cs-ai-tact-a-pragmatic-approach-to-qualitative-analysis-3e3b3276","markdown":"/readings/arxiv-cs-ai-tact-a-pragmatic-approach-to-qualitative-analysis-3e3b3276.md","json":"/readings/arxiv-cs-ai-tact-a-pragmatic-approach-to-qualitative-analysis-3e3b3276.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd","revision":"704aefb4-ab56-4cc2-8e12-80ee0d8d33b4","revisionKind":"publication-id","title":"arXiv cs.AI: TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","url":"https://escapemeridian.com/readings/arxiv-cs-ai-tact-a-pragmatic-approach-to-qualitative-analysis-3e3b3276"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.05702","revision":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","revisionKind":"sha256-source-text","title":"TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","url":"https://arxiv.org/abs/2611.05702"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd","title":"arXiv cs.AI: TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","argument":"The authors present TACT, a procedure for using AI to analyze qualitative responses while leaving teams in control of interpretation and judgment. They frame it as an alternative to either leaving large collections of responses unanalyzed or accepting a large language model’s interpretation without that control. TACT-Lite uses the core AI council procedure; TACT-Full adds adversaries—models from other model families that critique each model’s output—and a decision model that surfaces high-uncertainty code assignments. In an evaluation on 256 Stack Overflow threads coded by other researchers, TACT’s candidate codebook achieved an F1 of 0.75–0.77, and its final coding achieved an F1 of 0.71–0.72 against those labels. On a held-out sample, the authors’ manual coding and TACT’s coding had a Cohen’s κ of 0.79–0.85. The authors also report model-call costs of $13.77 for TACT-Lite and $33.59 for TACT-Full per 100 threads. They say the paper identifies pitfalls and safeguards, but the abstract does not describe those safeguards in detail.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evaluation described in the abstract uses an independent dataset of 256 Stack Overflow threads; it does not establish how TACT performs on other qualitative datasets. The abstract reports that the authors identify pitfalls and safeguards, but does not describe them in detail. The abstract does not provide further details about the held-out sample or the evaluation procedure.","speaker":{"name":"Choudhuri, Rudrajit, Bird, Christian, Badea, Carmen, Sarma, Anita","roleAtSource":"Article author"},"source":{"title":"TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05702","publishedAt":"2026-11-03","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present TACT as a procedure for analyzing qualitative responses at scale while keeping teams in control of interpretation and judgment.","excerpt":"a procedure that enables such analysis at scale while keeping teams in control of interpretation and judgment","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"},{"text":"TACT-Full adds models from other model families that critique each model’s output, and a decision model that surfaces high-uncertainty code assignments.","excerpt":"adds adversaries (models from other model families that critique each model's output) and a decision model that surfaces high-uncertainty code assignments","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"},{"text":"On an independent dataset of 256 Stack Overflow threads, the authors report candidate codebook F1 of 0.75–0.77 and final coding F1 of 0.71–0.72 against other researchers’ labels.","excerpt":"Against their labels, TACT's candidate codebook achieved an F1 of 0.75-0.77 and its final coding an F1 of 0.71-0.72.","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"},{"text":"The authors report Cohen’s κ of 0.79–0.85 for agreement between their own manual coding and TACT’s coding on a held-out sample, and model-call costs of $13.77 to $33.59 per 100 threads.","excerpt":"On a held-out sample, agreement between our own manual coding and TACT's coding was Cohen's $\\kappa$ of 0.79-0.85. TACT is also economical: model calls cost \\$13.77 (TACT-Lite) to \\$33.59 (TACT-Full) per 100 threads.","url":"https://arxiv.org/abs/2611.05702","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05702","checkedAt":"2026-11-09T07:48:06.732Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would TACT’s reported coding results compare on qualitative datasets beyond the 256 Stack Overflow threads used in this evaluation?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","summary":"The authors present TACT, a procedure for using AI to analyze qualitative responses while leaving teams in control of interpretation and judgment. They frame it as an alternative to either leaving large collections of responses unanalyzed or accepting a large language model’s interpretation without that control. TACT-Lite uses the core AI council procedure; TACT-Full adds adversaries—models from other model families that critique each model’s output—and a decision model that surfaces high-uncertainty code assignments. In an evaluation on 256 Stack Overflow threads coded by other researchers, TACT’s candidate codebook achieved an F1 of 0.75–0.77, and its final coding achieved an F1 of 0.71–0.72 against those labels. On a held-out sample, the authors’ manual coding and TACT’s coding had a Cohen’s κ of 0.79–0.85. The authors also report model-call costs of $13.77 for TACT-Lite and $33.59 for TACT-Full per 100 threads. They say the paper identifies pitfalls and safeguards, but the abstract does not describe those safeguards in detail.","question":"How would TACT’s reported coding results compare on qualitative datasets beyond the 256 Stack Overflow threads used in this evaluation?","claims":[{"text":"The authors present TACT as a procedure for analyzing qualitative responses at scale while keeping teams in control of interpretation and judgment.","excerpt":"a procedure that enables such analysis at scale while keeping teams in control of interpretation and judgment","support":"source-stated"},{"text":"TACT-Full adds models from other model families that critique each model’s output, and a decision model that surfaces high-uncertainty code assignments.","excerpt":"adds adversaries (models from other model families that critique each model's output) and a decision model that surfaces high-uncertainty code assignments","support":"source-stated"},{"text":"On an independent dataset of 256 Stack Overflow threads, the authors report candidate codebook F1 of 0.75–0.77 and final coding F1 of 0.71–0.72 against other researchers’ labels.","excerpt":"Against their labels, TACT's candidate codebook achieved an F1 of 0.75-0.77 and its final coding an F1 of 0.71-0.72.","support":"source-stated"},{"text":"The authors report Cohen’s κ of 0.79–0.85 for agreement between their own manual coding and TACT’s coding on a held-out sample, and model-call costs of $13.77 to $33.59 per 100 threads.","excerpt":"On a held-out sample, agreement between our own manual coding and TACT's coding was Cohen's $\\kappa$ of 0.79-0.85. TACT is also economical: model calls cost \\$13.77 (TACT-Lite) to \\$33.59 (TACT-Full) per 100 threads.","support":"source-stated"}],"limitations":["The evaluation described in the abstract uses an independent dataset of 256 Stack Overflow threads; it does not establish how TACT performs on other qualitative datasets.","The abstract reports that the authors identify pitfalls and safeguards, but does not describe them in detail.","The abstract does not provide further details about the held-out sample or the evaluation procedure."],"mediaId":"23299e52ee3e1c1e8c042d9e7ed5777d7fc196c3b0bcad65074e66283cf5b696","correctionReason":null,"id":"3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd","publicationId":"704aefb4-ab56-4cc2-8e12-80ee0d8d33b4","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05702","sourceTitle":"TACT: A Pragmatic Approach to Qualitative Analysis with an AI Council","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-03","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:03:28.124Z","createdAt":"2026-11-09T07:32:14.445Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"23299e52ee3e1c1e8c042d9e7ed5777d7fc196c3b0bcad65074e66283cf5b696","kind":"article","url":"https://arxiv.org/abs/2611.05702","sourceUrl":"https://arxiv.org/abs/2611.05702","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T07:25:22.587Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05702","checkedAt":"2026-11-09T07:48:06.732Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"23299e52ee3e1c1e8c042d9e7ed5777d7fc196c3b0bcad65074e66283cf5b696","confidence":0.78,"provider":"typesafe","model":"jev-1.13.0","inputHash":"10e83e4979157b4034f87ce68d6ef05ed55f4acae1f4b599f671fb273fe843e5","decidedAt":"2026-11-09T07:25:24.195Z"},"provenance":{"textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observationId":"47bbbc7cf47a176ebe10699adc871df1c9d226d417596cb6882a6fd6816f05d2","retrievedAt":"2026-11-09T07:25:22.326Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"819a306d-06cb-4f2f-8b45-f555ff63553a"},"reviewer":{"model":"gpt-6-luna","runId":"2b27af8e-769f-47ef-8c1f-b30ce7094eab"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T07:32:14.445Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Choudhuri, Rudrajit","url":null,"kind":"unknown"},{"name":"Bird, Christian","url":null,"kind":"unknown"},{"name":"Badea, Carmen","url":null,"kind":"unknown"},{"name":"Sarma, Anita","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Choudhuri, Rudrajit","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Choudhuri, Rudrajit","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}},{"actor":{"name":"Bird, Christian","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bird, Christian","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}},{"actor":{"name":"Badea, Carmen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Badea, Carmen","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}},{"actor":{"name":"Sarma, Anita","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sarma, Anita","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05702","locator":"head/meta[name=author|citation_author]"},"assertionId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T07:48:06.732Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","sourceUrl":"https://arxiv.org/abs/2611.05702","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z"}}],"participationCoverage":{"total":4,"returned":4,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05702","snapshotId":"1b86f3ef74dd66642b8ef08e620ff39c9a7fb004549a1368be6a0bcb2fc8b0a4","contentSha256":"dac695a3f86ae561ea15aa3aa782a750b941a9b17ed081521a9b18852a177eb6","textHash":"c0c580fd4de0c6c3ec5c7f995b474d308ff2871a47ce866c26b2cfa74f5ec94d","observedAt":"2026-11-09T07:48:06.732Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/3e3b32765508244a582da9fde94e16cea55e211f1d86446641837876726fa7bd/source"}}}],"audits":[{"publicationId":"704aefb4-ab56-4cc2-8e12-80ee0d8d33b4","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes TACT, its framing, the evaluation results and the cost figures to the paper's authors (Choudhuri, Bird, Badea, Sarma). Results stay as the authors' reports, not established facts. The 'arXiv cs.AI' label in the title matches the publisher field, though the paper's primary listing is cs.SE.","excerpt":"Authors:Rudrajit Choudhuri, Christian Bird, Carmen Badea, Anita Sarma"},{"dimension":"summary","status":"uncertain","explanation":"The reading is accurate but hard for a non-specialist to follow. Central terms are never explained in plain language or flagged for tooltips: coding and codebook (qualitative labeling categories), code assignments, F1, Cohen's κ, and what the 'AI council' procedure is. It also drops the source's motivation that rigorous analysis takes expert time few teams can afford or justify, and that the data tells software teams what stakeholders need. Smallest repair: add one sentence restoring that stake, and give brief explanations or tooltips for coding, F1 and κ.","excerpt":"Qualitative data can tell software teams what their stakeholders need, yet rigorously analyzing thousands of qualitative responses takes expert time that few teams can afford or justify."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-03, which matches the arXiv v1 submission date. The feed date of Nov 5 is later and was correctly not used.","excerpt":"[Submitted on 3 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Every claim matches the abstract: the TACT variants, F1 0.75-0.77 and 0.71-0.72, κ 0.79-0.85, and costs of $13.77 to $33.59 per 100 threads. The limitations and the question about generalizing beyond 256 Stack Overflow threads follow from the evidence without overclaiming real-world reliability.","excerpt":"We evaluated TACT on an independent dataset of 256 Stack Overflow threads qualitatively coded by other researchers."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-09T07:32:14.445Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}