# MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge [Canonical reading](https://escapemeridian.com/readings/marq-a-modular-llm-agnostic-requirement-quality-assessor-8185c111) Jannatul Shefa, Alejandro Salado, Paul Wach and Taylan G. Topcu present MARQ, a prototype that adds expert-derived systems engineering knowledge to large language models (LLMs) assessing the quality of requirements. The authors evaluated it against off-the-shelf baselines on an expert-labeled NASA requirement set, using three models and 100 executions per model. They report higher median F1 scores with MARQ for all three models, but say the gains differed: some models detected more defects while also producing more false alarms or remaining variable across executions. The reported gains were concentrated in Correctness and Conformance, and the authors say human-in-the-loop review remains necessary. The results concern this evaluation; the evidence does not establish how well MARQ would perform on other requirement sets or in operational systems. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-06T11:01:16.745Z. First observed: 2026-10-28T07:52:39.778Z. Retrieved: 2026-10-30T03:02:34.516Z. - [MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge](https://arxiv.org/abs/2610.36234) ## Limits The reported evaluation used one expert-labeled NASA requirement set; the supplied evidence does not establish performance on other requirement sets or in operational systems. The authors report model-dependent results, including more false alarms for GPT-5.6 Sol and high variability for Gemma 3 12B. ## Open question Would MARQ show similar results on requirement sets beyond the expert-labeled NASA set used in this evaluation? ## Source-stated The authors present MARQ as a modular prototype that supplements LLMs with expert-derived systems engineering knowledge for requirement quality assessment. > a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The evaluation used an expert-labeled NASA requirement set, three named models, and 100 executions per model. > evaluated against off-the-shelf baselines on an expert-labeled NASA requirement set using Claude Opus 5, GPT-5.6 Sol, and the open-weight Gemma 3 12B, with 100 executions per model - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The authors report that MARQ achieved higher median F1 scores than off-the-shelf baselines for each evaluated model. > MARQ achieves median F1 scores of 0.80 for Claude Opus 5, 0.62 for GPT-5.6 Sol, and 0.53 for Gemma 3 12B, compared with off-the-shelf baselines of 0.55, 0.52, and 0.18, respectively. - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The authors report that the gains varied by model, including trade-offs involving defect detection, false alarms, and execution-level consistency. > the nature of these gains is model-dependent. Claude Opus 5 improves defect detection and false-alarm control; GPT-5.6 Sol gains detection and better execution-level consistency at the cost of more false alarms - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The authors say the evaluation's gains were concentrated in Correctness and Conformance and call for human-in-the-loop review. > Criterion-level analysis shows gains concentrated in Correctness and Conformance, trade-offs between misses and false alarms within individual criteria - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source and contributor context Shefa, Jannatul, Salado, Alejandro, Wach, Paul, Topcu, Taylan G. · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## NASA NASA is the US agency for civilian space exploration, aeronautics and related scientific research. - [Explore this context](/entities/nasa) - [Reference](https://www.nasa.gov/) - [Reference](https://www.earthdata.nasa.gov/) ## Editorial provenance Version 1. Publication ID: 7d25ac92-0bc3-4db3-8db6-142825420485. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-06T11:01:16.745Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 8f370abba6a70900773f6aaf1e9e9fea546c06c2294375e86de439f6c1e6713c. Source text hash: 362edce06bec4f03d0c165a43c72fe4676a4ca5ae76fc8cbe5340fefded59037. ## Revision and source-check history ### Version 1 — 2026-11-06T11:01:16.745Z Jannatul Shefa, Alejandro Salado, Paul Wach and Taylan G. Topcu present MARQ, a prototype that adds expert-derived systems engineering knowledge to large language models (LLMs) assessing the quality of requirements. The authors evaluated it against off-the-shelf baselines on an expert-labeled NASA requirement set, using three models and 100 executions per model. They report higher median F1 scores with MARQ for all three models, but say the gains differed: some models detected more defects while also producing more false alarms or remaining variable across executions. The reported gains were concentrated in Correctness and Conformance, and the authors say human-in-the-loop review remains necessary. The results concern this evaluation; the evidence does not establish how well MARQ would perform on other requirement sets or in operational systems. ## Original source arXiv cs.AI · Article Original source date: Oct 26, 2026 (2026-10-26). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-06T11:01:16.745Z. First observed: 2026-10-28T07:52:39.778Z. Retrieved: 2026-10-30T03:02:34.516Z. - [MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge](https://arxiv.org/abs/2610.36234) ## Limits The reported evaluation used one expert-labeled NASA requirement set; the supplied evidence does not establish performance on other requirement sets or in operational systems. The authors report model-dependent results, including more false alarms for GPT-5.6 Sol and high variability for Gemma 3 12B. ## Open question Would MARQ show similar results on requirement sets beyond the expert-labeled NASA set used in this evaluation? ## Source-stated The authors present MARQ as a modular prototype that supplements LLMs with expert-derived systems engineering knowledge for requirement quality assessment. > a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The evaluation used an expert-labeled NASA requirement set, three named models, and 100 executions per model. > evaluated against off-the-shelf baselines on an expert-labeled NASA requirement set using Claude Opus 5, GPT-5.6 Sol, and the open-weight Gemma 3 12B, with 100 executions per model - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The authors report that MARQ achieved higher median F1 scores than off-the-shelf baselines for each evaluated model. > MARQ achieves median F1 scores of 0.80 for Claude Opus 5, 0.62 for GPT-5.6 Sol, and 0.53 for Gemma 3 12B, compared with off-the-shelf baselines of 0.55, 0.52, and 0.18, respectively. - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The authors report that the gains varied by model, including trade-offs involving defect detection, false alarms, and execution-level consistency. > the nature of these gains is model-dependent. Claude Opus 5 improves defect detection and false-alarm control; GPT-5.6 Sol gains detection and better execution-level consistency at the cost of more false alarms - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source-stated The authors say the evaluation's gains were concentrated in Correctness and Conformance and call for human-in-the-loop review. > Criterion-level analysis shows gains concentrated in Correctness and Conformance, trade-offs between misses and false alarms within individual criteria - [Supporting source](https://arxiv.org/abs/2610.36234) ## Source and contributor context Shefa, Jannatul, Salado, Alejandro, Wach, Paul, Topcu, Taylan G. · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## NASA NASA is the US agency for civilian space exploration, aeronautics and related scientific research. - [Explore this context](/entities/nasa) - [Reference](https://www.nasa.gov/) - [Reference](https://www.earthdata.nasa.gov/) ## Editorial provenance Version 1. Publication ID: 7d25ac92-0bc3-4db3-8db6-142825420485. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-06T11:01:16.745Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 8f370abba6a70900773f6aaf1e9e9fea546c06c2294375e86de439f6c1e6713c. Source text hash: 362edce06bec4f03d0c165a43c72fe4676a4ca5ae76fc8cbe5340fefded59037. ### uncertain — 2026-11-06T11:01:16.745Z — anthropic/claude-opus-5.5 Publication: 7d25ac92-0bc3-4db3-8db6-142825420485 attribution — pass: The reading correctly names the four authors listed on the arXiv page and keeps the results and recommendations attributed to them, not presented as Escape Meridian's view or as established fact. > Authors:Jannatul Shefa, Alejandro Salado, Paul Wach, Taylan G. Topcu summary — uncertain: The summary is faithful, but a non-specialist would struggle to follow it. F1 score, the central metric, is never explained. Correctness and Conformance are INCOSE criteria and are unexplained, and INCOSE is not named. The draft also omits why requirement quality matters: the source says defects propagate downstream and harm development outcomes. Smallest repair: add tooltips for F1, requirements and the INCOSE criteria, and one sentence on downstream defect propagation. The draft also drops the source's point that Gemma 3 12B with MARQ reaches the level of off-the-shelf proprietary models. > requirement defects can propagate downstream and adversely affect development outcomes date — pass: The originalPublished date of 2026-10-26 matches the v1 submission date on the arXiv page. It is not the later feed date. > \[Submitted on 26 Oct 2026\] support — pass: The source abstract supports all claims and limitations, including the scores, model-dependent trade-offs, criterion concentration and the call for human-in-the-loop review. The question about other requirement sets is a fair, non-leading gap. A minor issue: claim 5's excerpt does not contain the human-in-the-loop phrase, though the source states it elsewhere. Do not read this as independent verification of the reported scores. > underscoring the need for criterion-specific interventions and human-in-the-loop review media — pass: The selected media is the verified direct arXiv abstract page for this paper, which gives readers the easiest access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/8185c11123378ec95202bf72929420b6ee067519a88a1a90e666210c3d5f9844/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/marq-a-modular-llm-agnostic-requirement-quality-assessor-8185c111.json)