scGraPhT: merging transformers and graph neural networks for single-cell annotation

Abstract

The invention of single-cell RNA sequencing (scRNA-seq) has enabled transcriptomic examination of cells on an individual basis, uncovering cell-to-cell phenotypic heterogeneity within isogenic cell populations and tissues. Inevitably, cell type annotation has emerged as a fundamental, albeit challenging task in scRNA-seq data analysis, which involves identifying and characterizing cells based on their unique molecular profiles. Recently, deep learning techniques with their data-driven priors have shown significant promise in this task. On the one hand, task-agnostic transformer networks pre-trained on large-scale biological databases can capture generalizable data representations to serve as foundation models despite their ineffectiveness in characterizing intricate relationships between biological entities such as cells or genes. Contrarily, task-specific graph neural networks (GNNs) can be trained on target datasets to sensitively characterize entity relationships, but they can suffer from relatively poor generalizability. Furthermore, existing GNNs focus exclusively on either homogeneous or heterogeneous relationships, limiting their ability to offer a complete picture of the diverse inner structure of cells. In this study, we propose a novel merged transformer-graph model, scGraPhT, that integrates a pre-trained transformer to extract rich representations of scRNA-seq data with a multi-layered GNN to capture cell-cell, cell-gene, and gene-gene relationships. Different from previous GNNs, scGraPhT examines both homogeneous and heterogeneous relationships through subgraph layers to offer a more comprehensive assessment. Since the graph construction in scGraPhT relies on representations from a pre-trained transformer model, our approach does not require costly training procedures. Moreover, scGraPhT can also be adapted to leverage any transformer-based single-cell annotation method, such as scGPT or scBERT, that produces suitable embedding representations as input for GNNs. Demonstrations on three benchmark scRNA-seq datasets indicate that scGraPhT outperforms state-of-the-art annotation methods without compromising efficiency. We offer insights into performance improvements by employing Grad-CAM, a visual explainability method that elucidates the complementary nature of the GNN- and transformer-based components of scGraPhT to improve its predictive performance.