arXiv: Wen and coauthors propose InfMasking for multimodal models

arXiv: Wen and coauthors propose InfMasking for multimodal models

In their arXiv paper, Liangjian Wen and coauthors argue that multimodal models can leave useful cross-modal evidence unused: a fusion module may meet its training objective by relying on one dominant modality or information shared across modalities. They propose InfMasking, which randomly masks modality features before fusion. Each mask gives the model a different partial combination of evidence, and the resulting fused representations must stay consistent with a reference target: an unmasked fused view in unsupervised training, or a class proxy when labels are available. The authors say this consistency discourages reliance on a dominant modality or shared information alone, encouraging interactions that remain meaningful under partial observations. To optimize an objective over infinitely many masks, they model masked fused representations as Gaussian and use closed-form variational bounds. On evaluated benchmarks, they report improved predictive performance, higher mean PID estimates of task-label synergy in learned modality-specific representations, and higher InterSHAP interaction attribution in trained predictors. These results are benchmark findings, not evidence in this abstract about performance in other settings. The paper says it was accepted for publication in IEEE Transactions on Pattern Analysis and Machine Intelligence.

Publication: 4496992e-da58-42a2-8ea9-9a8e0d54f555

attribution — pass: The reading attributes the argument, method and results to Liangjian Wen and coauthors on arXiv, matching the listed authors. Results are framed as author reports. The TPAMI acceptance is correctly attributed to the paper's own page.

Authors:Liangjian Wen, Qun Dai, Jiang Duan, Yong Dai, Jianzhuang Liu, Dongkai Wang, Jun Wang, Fuyuan Cao, Zenglin Xu, Zhao Kang

summary — uncertain: The summary is faithful, but its central terms are not readable without specialist background. Multimodal, modality and fusion module are never explained in plain language (for example, models that combine inputs such as images and text, and the component that merges them). Synergy is paraphrased as useful cross-modal evidence without the source's definition: information available only when modalities are observed together. PID, InterSHAP, class proxy and variational bounds also appear without gloss or a tooltip note. Smallest repair: add the source's synergy definition and one plain clause on what multimodal and fusion mean. The question is substantive and non-leading.

synergistic information, that is, information available only when multiple modalities are observed together

date — pass: The originalPublished date of 2026-11-05 matches the arXiv v1 submission date. The feed date of 2026-11-06 was correctly not used.

[Submitted on 5 Nov 2026]

support — pass: Each claim's excerpt matches the abstract. The benchmark-scope caveat and the limitations (no named benchmarks, no numbers, transfer unknown) are supported and do not overreach. The reading implies no unsupported benefits or costs. The Gaussian limitation is a fair framing of the source's modeling assumption.

we model the masked fused representations as Gaussian and optimize closed-form variational bounds

media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.