Virtual patient representations from single-cell transcriptomics

Transformer models that summarize the single-cell data of a patient as one representation, as a basis for in silico perturbation and for integration with electronic health records.

mcBERT overview: data collection across blood, heart, kidney and lung datasets; patient embedding from 1,023 cells per donor; transformer architecture; self-supervised pre-training with teacher and student models; and fine-tuning with supervised contrastive learning.
von Querfurth B, et al. mcBERT: Patient-Level Single-cell Transcriptomics Data Representation. bioRxiv (2024).

Background

Single-cell RNA sequencing (scRNA-seq) has improved our understanding of cellular heterogeneity in health and disease. Most analyses, however, remain confined to single cells or distinct cell populations, which limits their clinical applicability.

To translate single-cell data into a patient-level understanding of disease, we introduced mcBERT, a method that uses scRNA-seq data and a transformer-based model to generate integrative patient representations. A self-supervised learning phase is followed by contrastive learning, which refines the representations.

Questions we are working on

  • How can mcBERT and related approaches be used to build virtual representations of patients?
  • Can in silico perturbations that shift the representation of a patient predict changes in phenotype?
  • Can these representations be extended to include data from electronic health records?

Key literature

  1. von Querfurth B, et al. mcBERT: Patient-Level Single-cell Transcriptomics Data Representation. bioRxiv (2024). [Hayat Lab]
  2. He B, et al. CloudPred: Predicting Patient Phenotypes From Single-cell RNA-seq. Pac Symp Biocomput (2022).
  3. Zhang Y, Venkatesh MS, Theodoris CV. Discovery of candidate therapeutic targets with Geneformer. Nat Protoc (2026).
  4. Liu T, et al. Learning multi-cellular representations of single-cell transcriptomics data enables characterization of patient-level disease states. Cell Syst (2026).