ICML 2026

Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in Modern Transformers

Controlled experiments on small transformers reveal an asymmetry in multimodal in-context learning: with a high-diversity primary modality, surprisingly low data complexity in the secondary modality is enough for ICL to emerge.

Yiran Huang, Karsten Roth, Quentin Bouniot, Wenjia Xu, Zeynep Akata