Excited to share CellBridge, our new preprint led by
@sevdoche in collaboration with
@pafrossard and
@DorinaThanou at
@EPFL_en! 🔬
Looking at an H&E slide, we can learn a remarkable amount about a tissue: its architecture, the cells within it, and how they interact. But identifying exactly which cells are present can be harder. A lymphocyte may be recognisable, while telling whether it is a T cell or a B cell often requires additional markers.
Spatial transcriptomics provides that molecular detail by measuring gene expression in individual cells. But it is much harder to obtain routinely than an H&E slide.
Could we use these molecular measurements to teach an image model more about cell identity and help it recognise when appearance alone is not enough?
This is the idea behind CellBridge.
We trained CellBridge on millions of cells with paired H&E images and Xenium spatial transcriptomic profiles. During training, the model learns to connect each cell’s appearance with its molecular identity. Once trained, it predicts cell types from H&E images alone.
Across our benchmarks, CellBridge improved fine-grained cell classification over leading pathology models, with gains of up to 16 percentage points in macro F1.
But better classification is only part of the story. Some biologically important distinctions remain difficult to resolve from an image.
Our Cell-Ontology-Fallback (COF) module lets the model adjust how specific its answer should be. For example, when the evidence does not support distinguishing a T cell from a B cell, it can report “lymphocyte” rather than commit to an uncertain subtype.
COF combines the biological relationships in the Cell Ontology with branch-calibrated conformal prediction to make these decisions.
What excites me is the possibility of extracting more cellular detail from routinely available slides while making clearer where additional molecular measurements are still needed.
🎥 Our 1-minute video walks through the idea.
Jointly supervised by
@pafrossard,
@DorinaThanou, and me. A huge thank you to our co-authors:
@arpitdm,
@AmirhVahidi, Dabin Jeong,
@SebastianBirk_, Adam Boxall,
@Adib_m_, Marta Rosa Sallese, and Yinshui Chang.
📄 Preprint:
biorxiv.org/content/10.64898…
💻 Code:
github.com/Lotfollahi-lab/ce…
📓 Tutorial:
github.com/Lotfollahi-lab/ce…
#ComputationalPathology #SpatialOmics #CellTypeAnnotation #MultimodalLearnin