類別無關的姿態估計（Category-Agnostic Pose Estimation，簡稱CAPE）目的是利用來自同一類別少量帶有標註資料的支援影像，預測查詢影像中的關鍵點位置。其核心理念是建立支援影像中的關鍵點特徵與查詢影像中視覺特徵之間的對應關係。然而，現有的CAPE 方法仍面臨兩大挑戰：1）在少樣本設定下，支援影像資訊有限，導致其關鍵點特徵表現薄弱且泛化能力不足；2）支援與查詢影像之間存在顯著的姿態差異，使得關鍵點定位困難。為了解決這些問題，我們提出SuPACape，一種融合超類別先驗與姿態適應性的圖神經網路方法。具體而言，我們設計了兩個關鍵模組：（i）超類別內子圖關鍵點特徵融合模組（Intra Super-Category Subgraph Fusion module），用於從同一超類別中語意相關的其他類別聚合關鍵點特徵，提升支援特徵表現；（ii）查詢自適應鄰接矩陣生成器（Query-Adaptive Adjacency Matrix Generator），根據支援關鍵點與查詢圖像之間的特徵相似性，構建查詢專屬的關鍵點關係圖。這兩個模組共同整合至基於Transformer 的傳統CAPE 架構中，有效提升對於姿態變化的穩定性與對未知類別的泛化能力。我們在MP-100 基準資料集上進行了大量實驗，結果顯示SuPACape 在1-shot 設定下達到最先進表現，在5-shot 設定下亦表現接近最先進方法。

Category-agnostic pose estimation (CAPE) aims to predict the keypoints of a query image based on a limited number of annotated support images from the same category. The core idea is to establish the correspondence between support keypoint features and visual features in the query image. However, existing CAPE methods face two major challenges: 1) Limited support information, especially under few-shot settings, leads to weak and less generalizable support representations; and 2) Large pose differences between the support and query images often make it difficult to accurately locate keypoints. To address these issues, we propose SuPACape, a novel graph-based method that leverages Super-category prior and Pose Adaptivity (SuPA). Specifically, we introduce (i) an Intra Super-Category Subgraph Fusion module that aggregates keypoint features from semantically related categories within the same super-category to enhance support representations, and (ii) a Query-Adaptive Adjacency Matrix Generator that constructs a query-specific inter-keypoint relation graph based on feature similarity between the support and a query. These two components are integrated into a unified graph framework to improve robustness against pose variation and enhance generalization to unseen categories. Extensive experiments on the MP-100 benchmark show that our method achieves state-of-the-art performance in the 1-shot setting and performs comparably to the best methods in the 5-shot setting.

[1] Arthur Allshire, Mayank MittaI, Varun Lodaya, Viktor Makoviychuk, Denys Makoviichuk, Felix Widmaier, Manuel Wüthrich, Stefan Bauer, Ankur Handa, and Animesh Garg. Transferring dexterous manipulation from gpu simulation to a remote real-world trifinger. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 11802–11809, 2022.
[2] Nihar Bendre, Hugo Terashima Marín, and Peyman Najafirad. Learning from few samples: A survey. arXiv preprint arXiv:2007.15484, 2020.
[3] Yuanhao Cai, Zhicheng Wang, Zhengxiong Luo, Binyi Yin, Angang Du, Haoqian Wang, Xiangyu Zhang, Xinyu Zhou, Erjin Zhou, and Jian Sun. Learning delicate local representations for multi-person pose estimation. In Proceedings of European Conference on Computer Vision, pages 455–472, 2020.
[4] Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7291–7299, 2017.
[5] Junjie Chen, Jiebin Yan, Yuming Fang, and Li Niu. Meta-point learning and refining for category-agnostic pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23534–23543, 2024.
[6] Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cascaded pyramid network for multi-person pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7103– 7112, 2018.
[7] Yuxin Chen, Ziqi Zhang, Chunfeng Yuan, Bing Li, Ying Deng, and Weiming Hu. Channel-wise topology refinement graph convolution for skeleton-based action recognition. In Proceedings of IEEE/CVF International Conference on Computer Vision, pages 13359–13368, 2021.
[8] Xiao Chu, Wei Yang, Wanli Ouyang, Cheng Ma, Alan L Yuille, and Xiaogang Wang. Multi-context attention for human pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1831 1840, 2017.
[9] MMPose Contributors. Openmmlab pose estimation toolbox and benchmark, 2020.
[10] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of International Conference on Machine Learning, pages 1126–1135, 2017.
[11] Yuying Ge, Ruimao Zhang, Xiaogang Wang, Xiaoou Tang, and Ping Luo. Deepfashion2: A versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5337–5345, 2019.
[12] Zigang Geng, Ke Sun, Bin Xiao, Zhaoxiang Zhang, and Jingdong Wang. Bottom-up human pose estimation via disentangled keypoint regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14676–14686, 2021.
[13] Jacob M Graving, Daniel Chae, Hemal Naik, Liang Li, Benjamin Koger, Blair R Costelloe, and Iain D Couzin. Deepposekit, a software toolkit for fast and robust animal pose estimation using deep learning. Elife, 8:e47994, 2019.
[14] Or Hirschorn and Shai Avidan. Edge weight prediction for category-agnostic pose estimation. arXiv preprint arXiv:2411.16665, 2024.
[15] Or Hirschorn and Shai Avidan. A graph-based approach for category-agnostic pose estimation. In Proceedings of European Conference on Computer Vision, pages 469–485, 2024.
[16] Bo Jiang, Ziyan Zhang, Doudou Lin, Jin Tang, and Bin Luo. Semi-supervised learning with graph learning-convolutional networks. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11313–11320, 2019.
[17] Sheng Jin, Wentao Liu, Enze Xie, Wenhai Wang, Chen Qian, Wanli Ouyang, and Ping Luo. Differentiable hierarchical graph grouping for multi-person pose estimation. In Proceedings of European Conference on Computer Vision, pages 718–734, 2020.
[18] Xuan Ju, Ailing Zeng, Jianan Wang, Qiang Xu, and Lei Zhang. Human-art: A versatile human-centric dataset bridging natural and artificial scenes. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 618–629, 2023.
[19] Muhammad Haris Khan, John McDonagh, Salman Khan, Muhammad Shahabuddin, Aditya Arora, Fahad Shahbaz Khan, Ling Shao, and Georgios Tzimiropoulos. Animalweb: A large-scale hierarchical dataset of annotated animal faces. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6939–6948, 2020.
[20] Martin Koestinger, Paul Wohlhart, Peter M Roth, and Horst Bischof. Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In Proceedings of IEEE/CVF International Conference on Computer Vision Workshop, pages 2144–2151, 2011.
[21] Rollyn Labuguen, Jumpei Matsumoto, Salvador Blanco Negrete, Hiroshi Nishimaru, Hisao Nishijo, Masahiko Takada, Yasuhiro Go, Ken-ichi Inoue, and Tomohiro Shibata. Macaquepose: a novel“in the wild＂macaque monkey pose dataset for markerless motion capture. Frontiers in Behavioral Neuroscience, 14:581154, 2021.
[22] Jiefeng Li, Siyuan Bian, Ailing Zeng, Can Wang, Bo Pang, Wentao Liu, and Cewu Lu. Human pose regression with residual log-likelihood estimation. In Proceedings of IEEE/CVF International Conference on Computer Vision, pages 11025–11034, 2021.
[23] Yujia Liang, Zixuan Ye, Wenze Liu, and Hao Lu. Scape: A simple and strong category-agnostic pose estimator. In Proceedings of European Conference on Computer Vision, pages 478–494, 2024.
[24] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proceedings of European Conference on Computer Vision, pages 740 755, 2014.
[25] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12009–12019, 2022.
[26] Ziyu Liu, Hongwen Zhang, Zhenghao Chen, Zhiyong Wang, and Wanli Ouyang. Disentangling and unifying graph convolutions for skeleton-based action recognition. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 143–152, 2020.
[27] Alexander Mathis, Pranav Mamidanna, Kevin M Cury, Taiga Abe, Venkatesh N Murthy, Mackenzie Weygandt Mathis, and Matthias Bethge. Deeplabcut: markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 21(9):1281–1289, 2018.
[28] Khoi Duc Nguyen, Chen Li, and Gim Hee Lee. Escape: Encoding super-keypoints for category-agnostic pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23491–23500, 2024.
[29] Xuecheng Nie, Jiashi Feng, Jianfeng Zhang, and Shuicheng Yan. Single-stage multiperson pose machines. In Proceedings of IEEE/CVF International Conference on Computer Vision, pages 6951–6960, 2019.
[30] Duo Peng, Zhengbo Zhang, Ping Hu, Qiuhong Ke, David KY Yau, and Jun Liu. Harnessing text-to-image diffusion models for category-agnostic pose estimation. In Proceedings of European Conference on Computer Vision, pages 342–360, 2024.
[31] Talmo D Pereira, Diego E Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S-H Wang, Mala Murthy, and Joshua W Shaevitz. Fast animal pose estimation using deep neural networks. Nature Methods, 16:117–125, 2019.
[32] N Dinesh Reddy, Minh Vo, and Srinivasa G Narasimhan. Carfusion: Combining point tracking and part detection for dynamic 3d reconstruction of vehicles. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1906–1915, 2018.
[33] Pengfei Ren, Yuanyuan Gao, Haifeng Sun, Qi Qi, Jingyu Wang, and Jianxin Liao. Dynamic support information mining for category-agnostic pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921–1930, 2024.
[34] Christos Sagonas, Epameinondas Antonakos, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: Database and results. Image and Vision Computing, 47:3–18, 2016.
[35] Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu. Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12026 12035, 2019.
[36] Min Shi, Zihao Huang, Xianzheng Ma, Xiaowei Hu, and Zhiguo Cao. Matching is not enough: A two-stage framework for category-agnostic pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7308–7317, 2023.
[37] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few shot learning. In Proceedings of Advances in Neural Information Processing Systems, 2017.
[38] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5693–5703, 2019.
[39] Xiao Sun, Jiaxiang Shang, Shuang Liang, and Yichen Wei. Compositional human pose regression. In Proceedings of IEEE/CVF International Conference on Computer Vision, pages 2602–2611, 2017.
[40] Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei. Integral human pose regression. In Proceedings of European Conference on Computer Vision, pages 529–545, 2018.
[41] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1199–1208, 2018.
[42] Yansong Tang, Yi Tian, Jiwen Lu, Peiyang Li, and Jie Zhou. Deep progressive reinforcement learning for skeleton-based action recognition. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5323 5332, 2018.
[43] Alexander Toshev and Christian Szegedy. Deeppose: Human pose estimation via deep neural networks. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1653–1660, 2014.
[44] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In Proceedings of International Conference on Learning Representations, 2018.
[45] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Proceedings of Advances in Neural Information Processing Systems, 2016.
[46] Yangang Wang, Cong Peng, and Yebin Liu. Mask-pose cascaded cnn for 2d hand pose estimation from single color image. IEEE Transactions on Circuits and Systems for Video Technology, 29:3258–3268, 2018.
[47] Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010.
[48] Jiajun Wu, Tianfan Xue, Joseph J Lim, Yuandong Tian, Joshua B Tenenbaum, Antonio Torralba, and William T Freeman. Single image 3d interpreter network. In Proceedings of European Conference on Computer Vision, pages 365–382, 2016.
[49] Lumin Xu, Sheng Jin, Wang Zeng, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, and Xiaogang Wang. Pose for everything: Towards category-agnostic pose estimation. In Proceedings of European Conference on Computer Vision, pages 398–416, 2022.
[50] Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of AAAI Conference on Artificial Intelligence, volume 32, 2018.
[51] Jie Yang, Ailing Zeng, Ruimao Zhang, and Lei Zhang. X-pose: Detecting any keypoints. In Proceedings of European Conference on Computer Vision, pages 249–268, 2024.
[52] Fanfan Ye, Shiliang Pu, Qiaoyong Zhong, Chao Li, Di Xie, and Huiming Tang. Dynamic gcn: Context-enriched topology learning for skeleton-based action recognition. In Proceedings of ACM International Conference on Multimedia, pages 55–63, 2020.
[53] Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. Ap-10k: A benchmark for animal pose estimation in the wild. arXiv preprint arXiv:2108.12617, 2021.