TY - GEN
T1 - Relational Graph Convolutional Network with BERT Embeddings for Ontology Relationship Classification
AU - Bashar, T. M.Rubaith
AU - Geller, James
AU - Xu, Mengjia
N1 - Publisher Copyright:
© 2026 The Authors.
PY - 2026/5/21
Y1 - 2026/5/21
N2 - SNOMED CT is one of the largest and most widely used medical ontologies. It is a collection of medical concepts with a large number of relationships between concept pairs. One of the most laborious tasks for SNOMED CT curators is placing concepts in the correct hierarchical positions with accurate relationships. We present a method that utilizes domain-specific BERT models integrated with a Relational Graph Convolutional Network (RGCN) for classifying the relationships between SNOMED CT concept pairs. We generate node embeddings using five domain-specific BERT models, BioBERT, ClinicalBERT, SapBERT, SciBERT, and BioMedBERT, and feed them individually into an RGCN model for relationship classification. We apply a three-layer RGCN to capture and utilize the graph-structured dependencies among concepts, enabling the model to propagate relational information across connected nodes before performing classification. We demonstrate that each BERT embedding integrated with RGCN outperforms both the Message-Passing Neural Network (MPNN) and Neural Network (NN) baselines. Our work shows that SapBERT-RGCN achieves the best performance across all tested encoders, with an accuracy of 0.9394 and an F1-weighted score of 0.9412. Compared to the best-performing domain-specific BERT-NN baseline, this is a 4% accuracy gain with a notable improvement of F1-macro.
AB - SNOMED CT is one of the largest and most widely used medical ontologies. It is a collection of medical concepts with a large number of relationships between concept pairs. One of the most laborious tasks for SNOMED CT curators is placing concepts in the correct hierarchical positions with accurate relationships. We present a method that utilizes domain-specific BERT models integrated with a Relational Graph Convolutional Network (RGCN) for classifying the relationships between SNOMED CT concept pairs. We generate node embeddings using five domain-specific BERT models, BioBERT, ClinicalBERT, SapBERT, SciBERT, and BioMedBERT, and feed them individually into an RGCN model for relationship classification. We apply a three-layer RGCN to capture and utilize the graph-structured dependencies among concepts, enabling the model to propagate relational information across connected nodes before performing classification. We demonstrate that each BERT embedding integrated with RGCN outperforms both the Message-Passing Neural Network (MPNN) and Neural Network (NN) baselines. Our work shows that SapBERT-RGCN achieves the best performance across all tested encoders, with an accuracy of 0.9394 and an F1-weighted score of 0.9412. Compared to the best-performing domain-specific BERT-NN baseline, this is a 4% accuracy gain with a notable improvement of F1-macro.
KW - Domain-Specific BERTs
KW - Relational Graph Neural Network
KW - SNOMED CT
UR - https://www.scopus.com/pages/publications/105039957143
UR - https://www.scopus.com/pages/publications/105039957143#tab=citedBy
U2 - 10.3233/SHTI260706
DO - 10.3233/SHTI260706
M3 - Conference contribution
C2 - 42175373
AN - SCOPUS:105039957143
T3 - Studies in Health Technology and Informatics
SP - 2430
EP - 2434
BT - Opening the Personal Gate between Technology and Health Care - Proceedings of MIE 2026
A2 - Hagglund, Maria
A2 - Lindskold, Lars
A2 - Lhotska, Lenka
A2 - Marceglia, Sara
A2 - Parimbelli, Enea
A2 - Sacchi, Lucia
A2 - Soda, Paolo
A2 - Stoicu-Tivadar, Lacramioara
A2 - Veltri, Pierangelo
A2 - Vizza, Patrizia
A2 - Giacomini, Mauro
A2 - Delgado, Jaime
A2 - Arvanitis, Theodoros N.
A2 - Andrikopoulou, Elisavet
A2 - Benis, Arriel
A2 - Balestra, Gabriella
A2 - Bellazzi, Riccardo
A2 - Gallos, Parisis G.
A2 - Gatta, Roberto
A2 - Giacobbe, Daniele Roberto
A2 - Giordano, Noemi
PB - IOS Press BV
T2 - 36th Medical Informatics Europe Conference, MIE 2026
Y2 - 25 May 2026 through 28 May 2026
ER -