# arXiv cs.AI: Compositional Unalignment: Auditing Safety in Large Vision-Language Models [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-compositional-unalignment-auditing-safety-in-large-42ad36d7) In their arXiv abstract, Hong kyu Lee, Kai Shu and Li Xiong argue that strong scores on compositional safety benchmarks do not show how robustly safety is integrated into a model. They describe compositional harm as arising when an image and prompt that are benign separately become harmful through cross-modal reasoning. The authors report that minimal supervised fine-tuning can reach an LLM-judge score comparable to a multi-stage reinforcement-learning pipeline, and caution that similar scores alone cannot establish robustness. They propose CMaG, a training-free un-alignment method that uses a refusal-versus-compliance log-probability margin and internal model activations to identify a direction for steering responses. In experiments on Qwen2.5-VL-7B and LLaVA-1.5-7B, they report that steering along this direction reduces compositional safety scores by up to 65 points while over 98% of outputs remain coherent. Their central argument is that behavior that looks safe on a benchmark may remain vulnerable to targeted internal intervention. These are the authors’ reported results; the abstract alone does not establish how broadly they apply beyond the tested models. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T20:38:30.091Z. First observed: 2026-11-06T05:10:46.077Z. Retrieved: 2026-11-08T20:33:20.571Z. - [Compositional Unalignment: Auditing Safety in Large Vision-Language Models](https://arxiv.org/abs/2611.07216) ## Limits The available evidence is the arXiv abstract, not the full paper; detailed methods and supporting experimental evidence are therefore not available here. The reported score changes and coherence rates are the authors’ results, not independently validated by this source record. The abstract names Qwen2.5-VL-7B and LLaVA-1.5-7B for the reported results; broader applicability is not established by the supplied evidence. ## Open question Escape Meridian interpretation: How broadly does the reported CMaG effect hold beyond Qwen2.5-VL-7B and LLaVA-1.5-7B? ## Source-stated The authors argue that benchmark scores for compositional safety do not, by themselves, establish how robustly safety is integrated into a model. > similar benchmark scores cannot establish robustness by themselves - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated The authors describe compositional harm as arising when benign image and prompt inputs become harmful through cross-modal reasoning. > an image and a prompt that are benign in isolation become harmful through the model's cross-modal reasoning - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated The authors report that minimal supervised fine-tuning reaches an LLM-judge score comparable to a multi-stage reinforcement-learning pipeline. > minimal supervised fine-tuning reaches an LLM-judge score comparable to a multi-stage reinforcement-learning pipeline - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated The authors propose CMaG, a training-free un-alignment method that uses a refusal-versus-compliance log-probability margin and mid-layer activations to isolate a cross-modal interaction term. > CMaG measures refusal intention with a refusal-versus-compliance log-probability margin, takes its gradient with respect to mid-layer activations - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated On Qwen2.5-VL-7B and LLaVA-1.5-7B, the authors report safety-score reductions of up to 65 points while over 98% of outputs remain coherent. > On Qwen2.5-VL-7B and LLaVA-1.5-7B, CMaG reduces compositional safety scores by up to 65 points while over 98% of outputs remain coherent - [Supporting source](https://arxiv.org/abs/2611.07216) ## Escape Meridian interpretation The abstract reports experiments on two named models; whether the reported effect extends more broadly remains an open question. > On Qwen2.5-VL-7B and LLaVA-1.5-7B - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source and contributor context Lee, Hong kyu, Shu, Kai, Xiong, Li · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Fine-tuning Further training of an existing model to adapt it to a particular task or use. - [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## Reinforcement learning Training by trial and error, in which a system is rewarded for the outcomes its designers want. - [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 0a06d411-e51f-4a6b-81b6-118c6f42d678. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T20:38:30.091Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: b7f48e25871a9a2abcf04aa9507299438ffb3a41849ecb234a176c3c35694c5e. Source text hash: ec422c62acb9510298e7e2a4d822528e456713e856d29fa6a5775ad175034014. ## Revision and source-check history ### Version 1 — 2026-11-08T20:38:30.091Z In their arXiv abstract, Hong kyu Lee, Kai Shu and Li Xiong argue that strong scores on compositional safety benchmarks do not show how robustly safety is integrated into a model. They describe compositional harm as arising when an image and prompt that are benign separately become harmful through cross-modal reasoning. The authors report that minimal supervised fine-tuning can reach an LLM-judge score comparable to a multi-stage reinforcement-learning pipeline, and caution that similar scores alone cannot establish robustness. They propose CMaG, a training-free un-alignment method that uses a refusal-versus-compliance log-probability margin and internal model activations to identify a direction for steering responses. In experiments on Qwen2.5-VL-7B and LLaVA-1.5-7B, they report that steering along this direction reduces compositional safety scores by up to 65 points while over 98% of outputs remain coherent. Their central argument is that behavior that looks safe on a benchmark may remain vulnerable to targeted internal intervention. These are the authors’ reported results; the abstract alone does not establish how broadly they apply beyond the tested models. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T20:38:30.091Z. First observed: 2026-11-06T05:10:46.077Z. Retrieved: 2026-11-08T20:33:20.571Z. - [Compositional Unalignment: Auditing Safety in Large Vision-Language Models](https://arxiv.org/abs/2611.07216) ## Limits The available evidence is the arXiv abstract, not the full paper; detailed methods and supporting experimental evidence are therefore not available here. The reported score changes and coherence rates are the authors’ results, not independently validated by this source record. The abstract names Qwen2.5-VL-7B and LLaVA-1.5-7B for the reported results; broader applicability is not established by the supplied evidence. ## Open question Escape Meridian interpretation: How broadly does the reported CMaG effect hold beyond Qwen2.5-VL-7B and LLaVA-1.5-7B? ## Source-stated The authors argue that benchmark scores for compositional safety do not, by themselves, establish how robustly safety is integrated into a model. > similar benchmark scores cannot establish robustness by themselves - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated The authors describe compositional harm as arising when benign image and prompt inputs become harmful through cross-modal reasoning. > an image and a prompt that are benign in isolation become harmful through the model's cross-modal reasoning - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated The authors report that minimal supervised fine-tuning reaches an LLM-judge score comparable to a multi-stage reinforcement-learning pipeline. > minimal supervised fine-tuning reaches an LLM-judge score comparable to a multi-stage reinforcement-learning pipeline - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated The authors propose CMaG, a training-free un-alignment method that uses a refusal-versus-compliance log-probability margin and mid-layer activations to isolate a cross-modal interaction term. > CMaG measures refusal intention with a refusal-versus-compliance log-probability margin, takes its gradient with respect to mid-layer activations - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source-stated On Qwen2.5-VL-7B and LLaVA-1.5-7B, the authors report safety-score reductions of up to 65 points while over 98% of outputs remain coherent. > On Qwen2.5-VL-7B and LLaVA-1.5-7B, CMaG reduces compositional safety scores by up to 65 points while over 98% of outputs remain coherent - [Supporting source](https://arxiv.org/abs/2611.07216) ## Escape Meridian interpretation The abstract reports experiments on two named models; whether the reported effect extends more broadly remains an open question. > On Qwen2.5-VL-7B and LLaVA-1.5-7B - [Supporting source](https://arxiv.org/abs/2611.07216) ## Source and contributor context Lee, Hong kyu, Shu, Kai, Xiong, Li · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## Fine-tuning Further training of an existing model to adapt it to a particular task or use. - [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## Reinforcement learning Training by trial and error, in which a system is rewarded for the outcomes its designers want. - [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 0a06d411-e51f-4a6b-81b6-118c6f42d678. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T20:38:30.091Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: b7f48e25871a9a2abcf04aa9507299438ffb3a41849ecb234a176c3c35694c5e. Source text hash: ec422c62acb9510298e7e2a4d822528e456713e856d29fa6a5775ad175034014. ### uncertain — 2026-11-08T20:38:30.091Z — anthropic/claude-opus-5.5 Publication: 0a06d411-e51f-4a6b-81b6-118c6f42d678 attribution — pass: The reading correctly credits Hong kyu Lee, Kai Shu and Li Xiong and frames the results as their abstract's claims. The arXiv cs.AI label matches the feed publisher, even though the page lists cs.LG as the primary category. > Authors:Hong kyu Lee, Kai Shu, Li Xiong summary — uncertain: The reading is faithful but leaves out key parts of the argument. It never explains un-alignment, meaning deliberately undoing safety behavior, as the authors' proposed audit. It also skips the finding that recognition and refusal are decoupled. It says only that CMaG steers responses; the abstract says it converts refusals into compliance using one backward pass over 80 examples. It omits transfer to text-explicit harm and to defensively hardened models. Smallest repair: one sentence each on un-alignment as an audit, conversion of refusal to compliance at low cost, and the transfer claims. LLM-judge and log-probability margin need tooltips. > A single backward pass over 80 examples yields a direction along which steering monotonically converts refusal into compliance. date — pass: The originalPublished date of 2026-11-04 matches the v1 submission date. The 2026-11-06 feed date is correctly not used. > \[Submitted on 4 Nov 2026\] support — pass: Each claim matches the abstract, including the cross-modal interaction term, which the abstract states. The question is labeled as interpretation and stays within the evidence, because the results were reported on two named models. That question does not contradict the abstract's separate claims of transfer to other harm types and to hardened models, though the summary should mention those claims. > contrasts it against partially benign pairs to isolate the cross-modal interaction term media — pass: The selected media is the verified direct link to the original arXiv abstract page, which is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/42ad36d76eb39ff8a315292ecf70390fd05abb7eec758172b7e914b26ab63b43/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-compositional-unalignment-auditing-safety-in-large-42ad36d7.json)