Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies

Christopher Ochs, James Geller, Yehoshua Perl, Yan Chen, Junchuan Xu, Hua Min, James T. Case, Zhi Wei

Research output: Contribution to journalArticlepeer-review

40 Scopus citations


Objective Standards terminologies may be large and complex, making their quality assurance challenging. Some terminology quality assurance (TQA) methodologies are based on abstraction networks (AbNs), compact terminology summaries. We have tested AbNs and the performance of related TQA methodologies on small terminology hierarchies. However, some standards terminologies, for example, SNOMED, are composed of very large hierarchies. Scaling AbN TQA techniques to such hierarchies poses a significant challenge. We present a scalable subject-based approach for AbN TQA. Methods An innovative technique is presented for scaling TQA by creating a new kind of subject-based AbN called a subtaxonomy for large hierarchies. New hypotheses about concentrations of erroneous concepts within the AbN are introduced to guide scalable TQA. Results We test the TQA methodology for a subject-based subtaxonomy for the Bleeding subhierarchy in SNOMED's large Clinical finding hierarchy. To test the error concentration hypotheses, three domain experts reviewed a sample of 300 concepts. A consensus-based evaluation identified 87 erroneous concepts. The subtaxonomy-based TQA methodology was shown to uncover statistically significantly more erroneous concepts when compared to a control sample. Discussion The scalability of TQA methodologies is a challenge for large standards systems like SNOMED. We demonstrated innovative subject-based TQA techniques by identifying groups of concepts with a higher likelihood of having errors within the subtaxonomy. Scalability is achieved by reviewing a large hierarchy by subject. Conclusions An innovative methodology for scaling the derivation of AbNs and a TQA methodology was shown to perform successfully for the largest hierarchy of SNOMED.

Original languageEnglish (US)
Pages (from-to)507-518
Number of pages12
JournalJournal of the American Medical Informatics Association
Issue number3
StatePublished - 2015

All Science Journal Classification (ASJC) codes

  • Health Informatics


  • Abstraction network
  • Scalable quality assurance
  • Standards quality assurance
  • Subject-based terminology quality assurance
  • Terminology quality assurance


Dive into the research topics of 'Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies'. Together they form a unique fingerprint.

Cite this