單細胞RNA定序技術已成為理解細胞多樣性與探索新細胞族群的重要工具。在單細胞資料分析中，細胞類型的分類是一項基本且關鍵的步驟，其目的在於透過具有已知細胞類型資訊的參考資料，判定新資料集中每個細胞的身分。然而，當新資料集中出現參考資料未涵蓋的細胞類型時，現有方法往往會面臨限制。部分方法採用強制分類的方式，即使沒有合適的已知標籤，仍會將每個細胞指派到某一個已知細胞類型，因而容易造成錯誤註解。另一些方法雖然加入額外的未知或未指派類別，但這類廣義類別在通常缺乏清楚定義。為了解決上述問題，本研究將無法被已知參考細胞類型適當分類的細胞重新定義為兩個概念類別：未知細胞類型與未指派細胞。未知細胞類型是指參考資料中尚未記錄的新生物族群；未指派細胞則是指缺乏足夠分類信心的細胞，其原因可能來自資料品質不佳、技術性雜訊，或尚未明確定義的細胞次族群。基於此定義，本研究提出BaySCA，一種半參考式貝氏細胞類型註解方法。此方法以負二項分布建立模型，並透過階層式先驗結構整合參考資料資訊，以進行標記基因發掘與細胞類型的分類。BaySCA的主要特色在於，它不會將所有無法對應至已知細胞類型的細胞合併為單一未知類別；相反地，BaySCA將潛在的未知細胞類型視為不同群體，使模型能夠辨識多個未知的細胞族群。此外，BaySCA 也利用後驗指派機率偵測未指派細胞，藉此將低信心觀測值與具有明確結構的已知或未知細胞類型區分開來。為了因應缺乏可靠參考資料的情境，本研究進一步將此架構延伸為BaySCA-c，使其能直接根據新資料集推論細胞群體。本研究透過模擬實驗與真實資料分析，將BaySCA與多種現有方法進行比較。結果顯示，BaySCA在未知細胞類型的偵測與分類上具有較穩健的表現，特別是在參考資料不完整或與新資料集不完全相符的情境下，能有效降低未知細胞被錯誤歸入已知細胞類型的問題。

Single-cell RNA sequencing technology has become a vital tool for understanding cellular diversity and discovering new types of cells in complex biological systems. A fundamental step in this process is cell type annotation. This procedure involves identifying the identities of cells in a new dataset, known as the query dataset, by comparing them to a reference that provides known cell type information. However, a major challenge arises when the query dataset contains cells that are not represented in the reference. Some methods rely on forced classification, assigning every query cell to one of the known cell types even when no suitable label exists. Other methods introduce an additional unknown or unassigned category, but the biological and statistical meaning of this broad category is often not clearly defined. To address these limitations, we redefine cells that cannot be properly categorized by the known reference cell types into two conceptual categories: unknown cell types and unassigned cells. Unknown cell types represent novel biological populations that are not yet recorded in the reference, whereas unassigned cells refer to cells with insufficient assignment confidence, which may arise from low data quality, technical artifacts, or undefined substructures. Based on this definition, we propose BaySCA, a semi-reference Bayesian approach that models single-cell count data using a negative binomial distribution and incorporates reference information through a hierarchical prior structure for marker gene discovery and cell-type annotation. A key feature of BaySCA is that it does not force all the cells not represented in the known cell types into a single unknown category. Instead, it treats potential unknown cell types as distinct groups, allowing multiple novel cell populations. In addition, BaySCA uses posterior assignment probabilities to detect unassigned cells, thereby separating low-confidence observations from structured known or unknown cell types. To accommodate broader scenarios, we further extend the framework to BaySCA-c, a reference-free clustering model that infers cell groups directly from the query dataset. Through simulation studies and real data analysis, we compare BaySCA with several benchmarking methods. Our results demonstrate that BaySCA outperforms existing tools, particularly in the robust detection and classification of unknown cell types.

Abdelaal, T., Michielsen, L., Cats, D., Hoogduin, D., Mei, H., Reinders, M. J., and Mahfouz, A. (2019). A comparison of automatic cell identification methods for single-cell rna sequencing data. Genome biology, 20(1):194.
Aran, D., Looney, A. P., Liu, L., Wu, E., Fong, V., Hsu, A., Chak, S., Naikawadi, R. P., Wolters, P. J., Abate, A. R., et al. (2019). Reference-based analysis of lung single-cell sequencing reveals a transitional profibrotic macrophage. Nature immunology, 20(2):163–172.
Bianconi, E., Piovesan, A., Facchin, F., Beraudi, A., Casadei, R., Frabetti, F., Vitale, L., Pelleri, M. C., Tassani, S., Piva, F., et al. (2013). An estimation of the number of cells in the human body. Annals of human biology, 40(6):463–471.
Clarke, Z. A., Andrews, T. S., Atif, J., Pouyabahar, D., Innes, B. T., MacParland, S. A., and Bader, G. D. (2021). Tutorial: guidelines for annotating single-cell transcriptomic maps using automated and manual methods. Nature protocols, 16(6):2749–2764.
Fonseca, S., Pereira, V., Lau, C., Teixeira, M. d. A., Bini-Antunes, M., and Lima, M. (2020). Human peripheral blood gamma delta t cells: Report on a series of healthy caucasian portuguese adults and comprehensive review of the literature. Cells, 9(3).
Fu, Q., Dong, C., Liu, Y., Xia, X., Liu, G., Zhong, F., and Liu, L. (2024). A comparison of scrna-seq annotation methods based on experimentally labeled immune cell subtype dataset. Briefings in Bioinformatics, 25(5):bbae392.
Guo, H. and Li, J. (2021). scsorter: assigning cells to known cell types according to marker genes. Genome biology, 22(1):69.
Huang, Q., Liu, Y., Du, Y., and Garmire, L. X. (2021). Evaluation of cell type annotation r packages on single-cell rna-seq data. Genomics, proteomics & bioinformatics, 19(2):267–281.
Hwang, B., Lee, J. H., and Bang, D. (2018). Single-cell rna sequencing technologies and bioinformatics pipelines. Experimental & molecular medicine, 50(8):1–14.
Jaccard, P. (1901). Étude comparative de la distribution florale dans une portion des alpes et des jura. Bull Soc Vaudoise Sci Nat, 37:547–579.
Jovic, D., Liang, X., Zeng, H., Lin, L., Xu, F., and Luo, Y. (2022). Single-cell rna sequencing technologies and applications: a brief overview. Clinical and translational medicine, 12(3):e694.
Kiselev, V. Y., Yiu, A., and Hemberg, M. (2018). scmap: projection of single-cell rna-seq data across data sets. Nature methods, 15(5):359–362.
Lieberman, Y., Rokach, L., and Shay, T. (2018). Castle–classification of single cells by transfer learning: harnessing the power of publicly available single cell rna sequencing experiments to annotate new experiments. PloS one, 13(10):e0205499.
Pasquini, G., Arias, J. E. R., Schäfer, P., and Busskamp, V. (2021). Automated methods for cell type annotation on scrna-seq data. Computational and Structural Biotechnology Journal, 19:961–969.
Polson, N. G., Scott, J. G., and Windle, J. (2013). Bayesian inference for logistic models using pólya–gamma latent variables. Journal of the American statistical Association, 108(504):1339–1349.
Sanz, M., Mann, B. T., Ryan, P. L., Bosque, A., Pennington, D. J., Hackstein, H., and Soriano-Sarabia, N. (2023). Deep characterization of human γδ t cell subsets defines shared and lineage-specific traits. Frontiers in Immunology, Volume 14- 2023.
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., and Satija, R. (2019). Comprehensive integration of single-cell data. cell, 177(7):1888–1902.
Tang, F. et al.(2009). mrna-seq whole-transcriptome analysis of a single cell. Nature methods, 6(5):377–382.
Yuan, Y., Xu, Q., Wani, A., Dahrendorff, J., Wang, C., Shen, A., Donglasan, J., Burgan, S., Graham, Z., Uddin, M., et al. (2024). Differentially expressed heterogeneous overdispersion genes testing for count data. Plos one, 19(7):e0300565.
Zeisel, A., Muñoz-Manchado, A. B., Codeluppi, S., Lönnerberg, P., La Manno, G., Juréus, A., Marques, S., Munguba, H., He, L., Betsholtz, C., et al. (2015). Cell types in the mouse cortex and hippocampus revealed by single-cell rna-seq. Science, 347(6226):1138–1142.
Zhang, Z., Luo, D., Zhong, X., Choi, J. H., Ma, Y., Wang, S., Mahrt, E., Guo, W., Stawiski, E. W., Modrusan, Z., et al. (2019). Scina: a semi-supervised subtyping algorithm of single cells and bulk samples. Genes, 10(7):531.
Zhao, X., Wu, S., Fang, N., Sun, X., and Fan, J. (2020). Evaluation of single-cell classifiers for single-cell rna sequencing data sets. Briefings in bioinformatics, 21(5):1581–1595.