{"id":"8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42","title":"arXiv cs.AI: When Does Sharing Work?","argument":"In their paper, Xiaofei Sun and coauthors present Omni-Encoder, a multimodal encoder that processes image, video, audio, and audio-visual inputs using a single SigLIP-2 Vision Transformer with fully shared parameters. They argue that, rather than training encoders independently and aligning them afterward, the system jointly optimizes all modalities end-to-end. Its design maps each signal into a shared embedding space, places visual frames and audio segments in a shared timeline using timestamps, and interleaves batches from different modalities during training. The authors identify this alternating-modality training as central to stable joint optimization. They report competitive results across benchmarks for all four modalities, and say joint training improves each modality over its separately trained counterpart. The authors say they will publicly release the code and models.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied source is the paper’s abstract; it gives no benchmark names, individual scores, or evaluation details for assessing the reported results. The abstract says code and models will be publicly released, but does not provide them or specify when they will be available.","speaker":{"name":"Sun, Xiaofei, Wang, Peng, Liu, Huayao, Xu, Xiaoxiao, Song, Jun, Li, Deqing, Qu, Lin","roleAtSource":"Article author"},"source":{"title":"When Does Sharing Work? Scaling a Shared SigLIP2 Across Image, Video, Audio, and Audio-Visual Representations","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05406","publishedAt":"2026-10-20","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present Omni-Encoder, which processes four kinds of input in one SigLIP-2 Vision Transformer with shared parameters.","excerpt":"a general multimodal encoder that processes image, video, audio, and audio-visual inputs within a single SigLIP-2 Vision Transformer with fully shared parameters","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The authors jointly optimize the modalities end-to-end rather than relying on separate encoders followed by post-hoc alignment.","excerpt":"all modalities are optimized jointly end-to-end","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The system maps each signal into a shared embedding space and uses timestamps to place visual and audio elements in a shared spatiotemporal coordinate system.","excerpt":"A modality-native input stage first maps each signal into the shared embedding space","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The authors identify alternating-modality training as central to stable joint optimization.","excerpt":"which we find central to stable joint optimization","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The authors report competitive results across benchmarks for all four modalities and improvements over separately trained counterparts.","excerpt":"Omni-Encoder achieves competitive results on 16 benchmarks across all four modalities with one shared parameter set, and joint training improves each modality over its separately trained counterpart.","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"If the reported gains hold beyond the evaluations, shared parameters could reduce the need for separate modality-specific encoders while preserving performance across inputs.","excerpt":"joint training improves each modality over its separately trained counterpart","url":"https://arxiv.org/abs/2611.05406","relationship":"Escape Meridian interpretation"},{"text":"The authors say code and models will be publicly released.","excerpt":"Code and models will be publicly released.","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05406","checkedAt":"2026-11-10T03:40:09.135Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What results across the reported benchmarks support the authors’ claim that joint training improves each modality over its separately trained counterpart?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: When Does Sharing Work?","summary":"In their paper, Xiaofei Sun and coauthors present Omni-Encoder, a multimodal encoder that processes image, video, audio, and audio-visual inputs using a single SigLIP-2 Vision Transformer with fully shared parameters. They argue that, rather than training encoders independently and aligning them afterward, the system jointly optimizes all modalities end-to-end. Its design maps each signal into a shared embedding space, places visual frames and audio segments in a shared timeline using timestamps, and interleaves batches from different modalities during training. The authors identify this alternating-modality training as central to stable joint optimization. They report competitive results across benchmarks for all four modalities, and say joint training improves each modality over its separately trained counterpart. The authors say they will publicly release the code and models.","question":"What results across the reported benchmarks support the authors’ claim that joint training improves each modality over its separately trained counterpart?","claims":[{"text":"The authors present Omni-Encoder, which processes four kinds of input in one SigLIP-2 Vision Transformer with shared parameters.","excerpt":"a general multimodal encoder that processes image, video, audio, and audio-visual inputs within a single SigLIP-2 Vision Transformer with fully shared parameters","support":"source-stated"},{"text":"The authors jointly optimize the modalities end-to-end rather than relying on separate encoders followed by post-hoc alignment.","excerpt":"all modalities are optimized jointly end-to-end","support":"source-stated"},{"text":"The system maps each signal into a shared embedding space and uses timestamps to place visual and audio elements in a shared spatiotemporal coordinate system.","excerpt":"A modality-native input stage first maps each signal into the shared embedding space","support":"source-stated"},{"text":"The authors identify alternating-modality training as central to stable joint optimization.","excerpt":"which we find central to stable joint optimization","support":"source-stated"},{"text":"The authors report competitive results across benchmarks for all four modalities and improvements over separately trained counterparts.","excerpt":"Omni-Encoder achieves competitive results on 16 benchmarks across all four modalities with one shared parameter set, and joint training improves each modality over its separately trained counterpart.","support":"source-stated"},{"text":"If the reported gains hold beyond the evaluations, shared parameters could reduce the need for separate modality-specific encoders while preserving performance across inputs.","excerpt":"joint training improves each modality over its separately trained counterpart","support":"interpretation"},{"text":"The authors say code and models will be publicly released.","excerpt":"Code and models will be publicly released.","support":"source-stated"}],"limitations":["The supplied source is the paper’s abstract; it gives no benchmark names, individual scores, or evaluation details for assessing the reported results.","The abstract says code and models will be publicly released, but does not provide them or specify when they will be available."],"mediaId":"c960c69d82190b7c58347afea8ce77e8b27fb4746dec25fe75702f521d8d86d9","correctionReason":null,"id":"8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42","publicationId":"90a7fa87-274d-4ab0-8b08-dd4cf92fd4cc","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05406","sourceTitle":"When Does Sharing Work? Scaling a Shared SigLIP2 Across Image, Video, Audio, and Audio-Visual Representations","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-20","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:00:30.526Z","createdAt":"2026-11-10T03:09:09.812Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"c960c69d82190b7c58347afea8ce77e8b27fb4746dec25fe75702f521d8d86d9","kind":"article","url":"https://arxiv.org/abs/2611.05406","sourceUrl":"https://arxiv.org/abs/2611.05406","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T03:04:09.065Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05406","checkedAt":"2026-11-10T03:40:09.135Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"c960c69d82190b7c58347afea8ce77e8b27fb4746dec25fe75702f521d8d86d9","confidence":0.95,"provider":"typesafe","model":"jev-1.13.0","inputHash":"47f521ca28360b27b42630ccbcb8475e3aba222d98952d57e760e17cab5413ba","decidedAt":"2026-11-10T03:04:11.267Z"},"provenance":{"textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observationId":"225bac46eee476bdb339ec90e0da9bf71d633f021de09734047c4c538925668c","retrievedAt":"2026-11-10T03:04:08.694Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"038cc54e-3fab-4c38-8d37-d861f6023b12"},"reviewer":{"model":"gpt-6-luna","runId":"174ac1f4-315d-4eb6-82a2-0e57a971a825"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T03:09:09.812Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Sun, Xiaofei","url":null,"kind":"unknown"},{"name":"Wang, Peng","url":null,"kind":"unknown"},{"name":"Liu, Huayao","url":null,"kind":"unknown"},{"name":"Xu, Xiaoxiao","url":null,"kind":"unknown"},{"name":"Song, Jun","url":null,"kind":"unknown"},{"name":"Li, Deqing","url":null,"kind":"unknown"},{"name":"Qu, Lin","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Sun, Xiaofei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sun, Xiaofei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Wang, Peng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Peng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Liu, Huayao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Huayao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Xu, Xiaoxiao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xu, Xiaoxiao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Song, Jun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Song, Jun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Li, Deqing","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Deqing","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Qu, Lin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qu, Lin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05406","snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-when-does-sharing-work-8c47f146","formats":{"html":"/readings/arxiv-cs-ai-when-does-sharing-work-8c47f146","markdown":"/readings/arxiv-cs-ai-when-does-sharing-work-8c47f146.md","json":"/readings/arxiv-cs-ai-when-does-sharing-work-8c47f146.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-multimodal-model","name":"Multimodal model","explanation":"A model that works with more than one kind of input or output, such as text, images and audio.","referenceUrl":"https://developers.google.com/machine-learning/glossary#multimodal-model"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42","revision":"90a7fa87-274d-4ab0-8b08-dd4cf92fd4cc","revisionKind":"publication-id","title":"arXiv cs.AI: When Does Sharing Work?","url":"https://escapemeridian.com/readings/arxiv-cs-ai-when-does-sharing-work-8c47f146"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.05406","revision":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","revisionKind":"sha256-source-text","title":"When Does Sharing Work? Scaling a Shared SigLIP2 Across Image, Video, Audio, and Audio-Visual Representations","url":"https://arxiv.org/abs/2611.05406"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42","title":"arXiv cs.AI: When Does Sharing Work?","argument":"In their paper, Xiaofei Sun and coauthors present Omni-Encoder, a multimodal encoder that processes image, video, audio, and audio-visual inputs using a single SigLIP-2 Vision Transformer with fully shared parameters. They argue that, rather than training encoders independently and aligning them afterward, the system jointly optimizes all modalities end-to-end. Its design maps each signal into a shared embedding space, places visual frames and audio segments in a shared timeline using timestamps, and interleaves batches from different modalities during training. The authors identify this alternating-modality training as central to stable joint optimization. They report competitive results across benchmarks for all four modalities, and say joint training improves each modality over its separately trained counterpart. The authors say they will publicly release the code and models.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied source is the paper’s abstract; it gives no benchmark names, individual scores, or evaluation details for assessing the reported results. The abstract says code and models will be publicly released, but does not provide them or specify when they will be available.","speaker":{"name":"Sun, Xiaofei, Wang, Peng, Liu, Huayao, Xu, Xiaoxiao, Song, Jun, Li, Deqing, Qu, Lin","roleAtSource":"Article author"},"source":{"title":"When Does Sharing Work? Scaling a Shared SigLIP2 Across Image, Video, Audio, and Audio-Visual Representations","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.05406","publishedAt":"2026-10-20","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present Omni-Encoder, which processes four kinds of input in one SigLIP-2 Vision Transformer with shared parameters.","excerpt":"a general multimodal encoder that processes image, video, audio, and audio-visual inputs within a single SigLIP-2 Vision Transformer with fully shared parameters","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The authors jointly optimize the modalities end-to-end rather than relying on separate encoders followed by post-hoc alignment.","excerpt":"all modalities are optimized jointly end-to-end","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The system maps each signal into a shared embedding space and uses timestamps to place visual and audio elements in a shared spatiotemporal coordinate system.","excerpt":"A modality-native input stage first maps each signal into the shared embedding space","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The authors identify alternating-modality training as central to stable joint optimization.","excerpt":"which we find central to stable joint optimization","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"The authors report competitive results across benchmarks for all four modalities and improvements over separately trained counterparts.","excerpt":"Omni-Encoder achieves competitive results on 16 benchmarks across all four modalities with one shared parameter set, and joint training improves each modality over its separately trained counterpart.","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"},{"text":"If the reported gains hold beyond the evaluations, shared parameters could reduce the need for separate modality-specific encoders while preserving performance across inputs.","excerpt":"joint training improves each modality over its separately trained counterpart","url":"https://arxiv.org/abs/2611.05406","relationship":"Escape Meridian interpretation"},{"text":"The authors say code and models will be publicly released.","excerpt":"Code and models will be publicly released.","url":"https://arxiv.org/abs/2611.05406","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05406","checkedAt":"2026-11-10T03:40:09.135Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What results across the reported benchmarks support the authors’ claim that joint training improves each modality over its separately trained counterpart?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: When Does Sharing Work?","summary":"In their paper, Xiaofei Sun and coauthors present Omni-Encoder, a multimodal encoder that processes image, video, audio, and audio-visual inputs using a single SigLIP-2 Vision Transformer with fully shared parameters. They argue that, rather than training encoders independently and aligning them afterward, the system jointly optimizes all modalities end-to-end. Its design maps each signal into a shared embedding space, places visual frames and audio segments in a shared timeline using timestamps, and interleaves batches from different modalities during training. The authors identify this alternating-modality training as central to stable joint optimization. They report competitive results across benchmarks for all four modalities, and say joint training improves each modality over its separately trained counterpart. The authors say they will publicly release the code and models.","question":"What results across the reported benchmarks support the authors’ claim that joint training improves each modality over its separately trained counterpart?","claims":[{"text":"The authors present Omni-Encoder, which processes four kinds of input in one SigLIP-2 Vision Transformer with shared parameters.","excerpt":"a general multimodal encoder that processes image, video, audio, and audio-visual inputs within a single SigLIP-2 Vision Transformer with fully shared parameters","support":"source-stated"},{"text":"The authors jointly optimize the modalities end-to-end rather than relying on separate encoders followed by post-hoc alignment.","excerpt":"all modalities are optimized jointly end-to-end","support":"source-stated"},{"text":"The system maps each signal into a shared embedding space and uses timestamps to place visual and audio elements in a shared spatiotemporal coordinate system.","excerpt":"A modality-native input stage first maps each signal into the shared embedding space","support":"source-stated"},{"text":"The authors identify alternating-modality training as central to stable joint optimization.","excerpt":"which we find central to stable joint optimization","support":"source-stated"},{"text":"The authors report competitive results across benchmarks for all four modalities and improvements over separately trained counterparts.","excerpt":"Omni-Encoder achieves competitive results on 16 benchmarks across all four modalities with one shared parameter set, and joint training improves each modality over its separately trained counterpart.","support":"source-stated"},{"text":"If the reported gains hold beyond the evaluations, shared parameters could reduce the need for separate modality-specific encoders while preserving performance across inputs.","excerpt":"joint training improves each modality over its separately trained counterpart","support":"interpretation"},{"text":"The authors say code and models will be publicly released.","excerpt":"Code and models will be publicly released.","support":"source-stated"}],"limitations":["The supplied source is the paper’s abstract; it gives no benchmark names, individual scores, or evaluation details for assessing the reported results.","The abstract says code and models will be publicly released, but does not provide them or specify when they will be available."],"mediaId":"c960c69d82190b7c58347afea8ce77e8b27fb4746dec25fe75702f521d8d86d9","correctionReason":null,"id":"8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42","publicationId":"90a7fa87-274d-4ab0-8b08-dd4cf92fd4cc","version":1,"sourceUrl":"https://arxiv.org/abs/2611.05406","sourceTitle":"When Does Sharing Work? Scaling a Shared SigLIP2 Across Image, Video, Audio, and Audio-Visual Representations","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-20","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:00:30.526Z","createdAt":"2026-11-10T03:09:09.812Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"c960c69d82190b7c58347afea8ce77e8b27fb4746dec25fe75702f521d8d86d9","kind":"article","url":"https://arxiv.org/abs/2611.05406","sourceUrl":"https://arxiv.org/abs/2611.05406","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T03:04:09.065Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.05406","checkedAt":"2026-11-10T03:40:09.135Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"c960c69d82190b7c58347afea8ce77e8b27fb4746dec25fe75702f521d8d86d9","confidence":0.95,"provider":"typesafe","model":"jev-1.13.0","inputHash":"47f521ca28360b27b42630ccbcb8475e3aba222d98952d57e760e17cab5413ba","decidedAt":"2026-11-10T03:04:11.267Z"},"provenance":{"textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observationId":"225bac46eee476bdb339ec90e0da9bf71d633f021de09734047c4c538925668c","retrievedAt":"2026-11-10T03:04:08.694Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"038cc54e-3fab-4c38-8d37-d861f6023b12"},"reviewer":{"model":"gpt-6-luna","runId":"174ac1f4-315d-4eb6-82a2-0e57a971a825"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T03:09:09.812Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Sun, Xiaofei","url":null,"kind":"unknown"},{"name":"Wang, Peng","url":null,"kind":"unknown"},{"name":"Liu, Huayao","url":null,"kind":"unknown"},{"name":"Xu, Xiaoxiao","url":null,"kind":"unknown"},{"name":"Song, Jun","url":null,"kind":"unknown"},{"name":"Li, Deqing","url":null,"kind":"unknown"},{"name":"Qu, Lin","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Sun, Xiaofei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sun, Xiaofei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Wang, Peng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Peng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Liu, Huayao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Huayao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Xu, Xiaoxiao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Xu, Xiaoxiao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Song, Jun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Song, Jun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Li, Deqing","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Deqing","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}},{"actor":{"name":"Qu, Lin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qu, Lin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.05406","locator":"head/meta[name=author|citation_author]"},"assertionId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T03:40:09.135Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","sourceUrl":"https://arxiv.org/abs/2611.05406","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.05406","snapshotId":"4e0309eaff1f4b2fc29bd50d2e6ea7cfdc59d4f25ee52872e035815def4218e2","contentSha256":"10fbb18b061a4e4c2ff8ec2c9f8df11bf7c4c3bdcd1fbfd030370f47d689da5e","textHash":"ac6a4181f8960dc6c9bd23112f52a35ea73de48cd2bdeb695310aa4e77017f1d","observedAt":"2026-11-10T03:40:09.135Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/8c47f1460346f80cab6aafaa72708c278c1a904cf84fc375637b60e8085ddf42/source"}}}],"audits":[{"publicationId":"90a7fa87-274d-4ab0-8b08-dd4cf92fd4cc","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading consistently attributes the design, the finding about alternating-modality training, the benchmark results and the release plan to Xiaofei Sun and coauthors. It does not present these source claims as established fact.","excerpt":"Authors:Xiaofei Sun, Peng Wang, Huayao Liu, Xiaoxiao Xu, Jun Song, Deqing Li, Lin Qu"},{"dimension":"summary","status":"uncertain","explanation":"The summary is faithful, but central terms are not explained for a general reader: encoder, SigLIP-2 Vision Transformer, shared parameters, embedding space and end-to-end. Nothing says what an encoder does or why one model for four input types matters; that significance appears only in the claims list. 'Shared timeline using timestamps' paraphrases the source term 'multimodal rotary position embedding'. The count of 16 benchmarks is dropped. Smallest repair: add one plain sentence on what an encoder is and why one shared model matters, keep the exact technical term (with a tooltip), and restore '16 benchmarks'.","excerpt":"A multimodal rotary position embedding parameterized by absolute timestamps then places visual frames and audio segments in a unified spatiotemporal coordinate system."},{"dimension":"date","status":"pass","explanation":"The displayed original publication date, 2026-10-20, matches the v1 submission date. The later feed date of 2026-11-05 is correctly not used.","excerpt":"[Submitted on 20 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"The source-stated claims match the abstract. The interpretation is explicitly conditional ('If the reported gains hold'), is labeled as interpretation, and follows from the source's claim of one shared parameter set with per-modality gains. The limitations are accurate. The question asks for missing evidence rather than assuming a result.","excerpt":"with one shared parameter set, and joint training improves each modality over its separately trained counterpart"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified arXiv abstract page for this paper, which is the direct original source.","excerpt":null}],"createdAt":"2026-11-10T03:09:09.812Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}