# arXiv cs.AI: Model Merging Does Not Require Uniform LoRA Ranks [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-model-merging-does-not-require-uniform-lora-ranks-78faaf61) In an arXiv cs.AI abstract, the authors of “Model Merging Does Not Require Uniform LoRA Ranks” argue that model merging need not force separately fine-tuned experts to use the same LoRA rank. They report experiments with 36 experts across mathematics, instruction following and multilingual tasks, using four ranks, three model scales and six merging methods. The authors say averaging and utility-based selection gained up to 1.8 percentage points in multi-domain accuracy when each domain used its own best rank, while conflict-resolution methods were inconsistent and lost up to 4 points. They attribute the difference to how merging methods select spectral components: methods selecting by size can discard components that would reduce loss and nearly drop the instruction adapter, which they link to a 16-to-25-point fall on IFEval. By contrast, selecting components by their effect on loss kept all three experts and left accuracy nearly flat across an 8× rank range in the reported comparison. The authors conclude that rank can be chosen separately for each expert when the merge operator does not select by magnitude. Their results make the choice of merging method—not a shared rank—the key condition in their argument; how broadly that condition holds beyond the reported experiments remains open. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T08:23:33.799Z. First observed: 2026-11-05T05:06:22.287Z. Retrieved: 2026-11-09T08:17:09.331Z. - [Model Merging Does Not Require Uniform LoRA Ranks](https://arxiv.org/abs/2611.06486) ## Limits This reading is based on the arXiv abstract, which reports the experimental setup and headline results but does not provide the detailed methods or results needed to assess them independently. The reported experiments cover 36 experts, three domains, four ranks, three model scales and six merging methods; the abstract does not establish how the results generalize beyond those tested combinations. ## Open question How do these results vary across domains, model scales and merging methods beyond the combinations tested in the paper? ## Source-stated The authors argue that requiring every expert to use the same LoRA rank is a property of the merge method, not a requirement of model merging. > We show this coupling is a property of the merge method, not of merging. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors report that allowing domain-specific ranks cost averaging and utility-based selection nothing and improved multi-domain accuracy by up to 1.8 percentage points; conflict-resolution methods were inconsistent and lost up to 4 points. > Giving experts different ranks costs averaging and utility-based selection nothing, and gains them up to 1.8 pp of multi-domain accuracy, while the conflict-resolution methods are inconsistent and can lose up to 4 pp. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors attribute the difference to selection by size discarding components that would have reduced loss; they report that the instruction adapter was nearly discarded and IFEval fell by 16 to 25 percentage points. > methods that choose them by size discard many that would have reduced loss. They also tend to discard the same expert: the instruction adapter has the smallest singular values, so it is dropped almost entirely and IFEval falls by 16 to 25 pp. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors say selecting components by their effect on loss kept all three experts and left accuracy nearly flat across an 8× rank range, compared with a 6.7-point change for TIES at 4B scale. > Choosing components by their effect on loss instead keeps all three experts, and leaves accuracy nearly flat across an 8$\\times$ rank range (0.7 pp, against 6.7 pp for TIES at 4B scale). - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors conclude that rank can be chosen separately for each expert when the merge operator does not select by magnitude. > Rank can therefore be chosen per expert whenever the merge operator does not select by magnitude. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source and contributor context Xie, Zirong, Afolabi, Zainab, Kozodoi, Nikita, Butler, Jack · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Fine-tuning Further training of an existing model to adapt it to a particular task or use. - [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 51673607-eb97-4086-828a-91206e73ba10. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T08:23:33.799Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 690ab8bd07cef4fd072ba6682b0ab980c9a854fe82f40987bc9d4f9de8513737. Source text hash: 53b93303ce85b40a4e8536e5f106ad68c0e1dcb53cd6ecc29c03c0667831d64d. ## Revision and source-check history ### Version 1 — 2026-11-09T08:23:33.799Z In an arXiv cs.AI abstract, the authors of “Model Merging Does Not Require Uniform LoRA Ranks” argue that model merging need not force separately fine-tuned experts to use the same LoRA rank. They report experiments with 36 experts across mathematics, instruction following and multilingual tasks, using four ranks, three model scales and six merging methods. The authors say averaging and utility-based selection gained up to 1.8 percentage points in multi-domain accuracy when each domain used its own best rank, while conflict-resolution methods were inconsistent and lost up to 4 points. They attribute the difference to how merging methods select spectral components: methods selecting by size can discard components that would reduce loss and nearly drop the instruction adapter, which they link to a 16-to-25-point fall on IFEval. By contrast, selecting components by their effect on loss kept all three experts and left accuracy nearly flat across an 8× rank range in the reported comparison. The authors conclude that rank can be chosen separately for each expert when the merge operator does not select by magnitude. Their results make the choice of merging method—not a shared rank—the key condition in their argument; how broadly that condition holds beyond the reported experiments remains open. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T08:23:33.799Z. First observed: 2026-11-05T05:06:22.287Z. Retrieved: 2026-11-09T08:17:09.331Z. - [Model Merging Does Not Require Uniform LoRA Ranks](https://arxiv.org/abs/2611.06486) ## Limits This reading is based on the arXiv abstract, which reports the experimental setup and headline results but does not provide the detailed methods or results needed to assess them independently. The reported experiments cover 36 experts, three domains, four ranks, three model scales and six merging methods; the abstract does not establish how the results generalize beyond those tested combinations. ## Open question How do these results vary across domains, model scales and merging methods beyond the combinations tested in the paper? ## Source-stated The authors argue that requiring every expert to use the same LoRA rank is a property of the merge method, not a requirement of model merging. > We show this coupling is a property of the merge method, not of merging. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors report that allowing domain-specific ranks cost averaging and utility-based selection nothing and improved multi-domain accuracy by up to 1.8 percentage points; conflict-resolution methods were inconsistent and lost up to 4 points. > Giving experts different ranks costs averaging and utility-based selection nothing, and gains them up to 1.8 pp of multi-domain accuracy, while the conflict-resolution methods are inconsistent and can lose up to 4 pp. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors attribute the difference to selection by size discarding components that would have reduced loss; they report that the instruction adapter was nearly discarded and IFEval fell by 16 to 25 percentage points. > methods that choose them by size discard many that would have reduced loss. They also tend to discard the same expert: the instruction adapter has the smallest singular values, so it is dropped almost entirely and IFEval falls by 16 to 25 pp. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors say selecting components by their effect on loss kept all three experts and left accuracy nearly flat across an 8× rank range, compared with a 6.7-point change for TIES at 4B scale. > Choosing components by their effect on loss instead keeps all three experts, and leaves accuracy nearly flat across an 8$\\times$ rank range (0.7 pp, against 6.7 pp for TIES at 4B scale). - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source-stated The authors conclude that rank can be chosen separately for each expert when the merge operator does not select by magnitude. > Rank can therefore be chosen per expert whenever the merge operator does not select by magnitude. - [Supporting source](https://arxiv.org/abs/2611.06486) ## Source and contributor context Xie, Zirong, Afolabi, Zainab, Kozodoi, Nikita, Butler, Jack · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Fine-tuning Further training of an existing model to adapt it to a particular task or use. - [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 51673607-eb97-4086-828a-91206e73ba10. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T08:23:33.799Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 690ab8bd07cef4fd072ba6682b0ab980c9a854fe82f40987bc9d4f9de8513737. Source text hash: 53b93303ce85b40a4e8536e5f106ad68c0e1dcb53cd6ecc29c03c0667831d64d. ### uncertain — 2026-11-09T08:23:33.799Z — anthropic/claude-opus-5.5 Publication: 51673607-eb97-4086-828a-91206e73ba10 attribution — pass: The reading consistently attributes the claims to the paper's authors and presents them as reported findings rather than established facts. The arXiv listing is primarily cs.LG, cross-listed to cs.AI. The 'arXiv cs.AI' label matches the feed publisher and does not misattribute the work. > Authors:Zirong Xie, Zainab Afolabi, Nikita Kozodoi, Jack Butler summary — uncertain: The figures and conditions are faithful to the abstract. However, central terms are not explained for a non-specialist reader: LoRA rank, spectral components, singular values or magnitude selection, IFEval, TIES and merge operator. The draft also omits the abstract's plain definition that merging combines fine-tuned experts without retraining. A reader cannot follow why rank or selection by size matters. Smallest repair: add the abstract's definition of merging, and add term-library tooltips for LoRA rank, IFEval and TIES, without paraphrasing those terms. > Model merging combines separately fine-tuned experts into a single model without retraining. date — pass: The originalPublished date of 2026-11-04 matches the v1 submission date. It is not the later feed entry date. > \[Submitted on 4 Nov 2026\] support — pass: Every claim's excerpt is verbatim and supports its claim, including 1.8 pp, up to 4 pp, 16 to 25 pp on IFEval, and 0.7 pp against 6.7 pp for TIES at 4B scale. The conditional conclusion is preserved. The closing sentence and the question are appropriately framed as open rather than asserting generalization. > Rank can therefore be chosen per expert whenever the merge operator does not select by magnitude. media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/78faaf61937b85c1666a0dbbc75cd2495dcfcf8bf1808289790ea4896c7aadcb/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-model-merging-does-not-require-uniform-lora-ranks-78faaf61.json)