Integrating measurements across modalities and scales into unified quantitative frameworks of cellular function is a central goal in computational biology. Although powerful unimodal foundation models exist, the scarcity of large-scale paired data limits joint multimodal training. This constraint favours compositional foundation models, which fuse frozen unimodal experts through a learned interface. However, it remains unclear when such fusion provides task-relevant information beyond the strongest unimodal representation rather than merely aggregating redundant signal. Here we introduce Synergistic Information Score, a metric grounded in partial information decomposition that quantifies information gain arising only from cross-modal interactions. Extending theory from self-supervised learning, we show that standard alignment-based objectives with frozen encoders collapse to detect linear redundancies, which constrains their ability to capture nonlinear synergistic states. This distinction is especially important for tasks that connect tissue morphology with gene expression. Benchmarking ten fusion methods on spatial transcriptomics datasets, we demonstrate that tasks dominated by linear redundancies are well served by unimodal baselines, whereas complex niche definitions benefit from synergy-aware integration. Finally, we perform a scaling analysis and show that fine-tuning a dominant unimodal expert is the most sample-efficient strategy for standard tasks. Together, these results suggest that progress in multimodal modelling of cellular state requires shifting from alignment-based objectives to synergy-driven integration that exploits complementary cross-modal signal.
Advances in self-supervised learning and the increasing scale of biological datasets have produced powerful encoders for individual modalities, including transcriptomics1,2,3,4, histopathology5,6,7 and spatial transcriptomics8,9,10. Recent multimodal efforts extend these representations to chromatin accessibility11,12, proteomics13 and text-based biological metadata14,15,16,17. However, multimodal data remain fundamentally asymmetric: millions of unimodal samples exist in isolation, whereas paired measurements capturing multiple modalities in the same cell or tissue context are much less available. This asymmetry favours compositional foundation models (CFMs), which reuse robust unimodal encoders as frozen experts and combine their outputs through learned interfaces18. These interfaces enable scalable multimodal modelling without requiring end-to-end retraining and have become central to efforts aimed at constructing integrated representations of cellular identity and function across biological scales19,20,21 (see discussion on related work in Supplementary Information section 1). Still, a fundamental question remains: When does multimodal fusion truly add information beyond the strongest unimodal expert, and when does it merely repackage redundant signal?
Standard alignment-based fusion interfaces, which learn mappings between frozen unimodal representations into a shared embedding space (see the Methods for a formal definition), are optimized to maximize agreement between modalities. This is typically done by bringing representations of paired samples closer while separating unrelated samples through contrastive or regression-based objectives22,23. These objectives are effective for retrieval and matching tasks, where the goal is to collapse modalities onto a shared structure (for example, CLIP22). However, we argue that many scientific objectives require the opposite regime where discovery arises from synthesizing non-overlapping sets of information. For example, defining spatially varying cell states, niches or microenvironmental interactions may benefit from fusing tissue morphology and gene expression (GEX). In these regimes, we hypothesize that alignment-based fusion may reach a spectral ceiling defined here as a limit imposed…
Source: Nature Machine Intelligence — Read the full article on the original site ↗