A review of auditing techniques for the unified medical language system

Ling Zheng, Zhe He, Duo Wei, Vipina Keloth, Jung Wei Fan, Luke Lindemann, Xinxin Zhu, James J. Cimino, Yehoshua Perl

Research output: Contribution to journalReview articlepeer-review

7 Scopus citations


Objective: The study sought to describe the literature related to the development of methods for auditing the Unified Medical Language System (UMLS), with particular attention to identifying errors and inconsistencies of attributes of the concepts in the UMLS Metathesaurus. Materials and Methods: We applied the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) approach by searching the MEDLINE database and Google Scholar for studies referencing the UMLS and any of several terms related to auditing, error detection, and quality assurance. A qualitative analysis and summarization of articles that met inclusion criteria were performed. Results: Eighty-three studies were reviewed in detail. We first categorized techniques based on various aspects including concepts, concept names, and synonymy (n ¼ 37), semantic type assignments (n ¼ 36), hierarchical relationships (n ¼ 24), lateral relationships (n ¼ 12), ontology enrichment (n ¼ 8), and ontology alignment (n ¼ 18). We also categorized the methods according to their level of automation (ie, automated systematic, automated heuristic, or manual) and the type of knowledge used (ie, intrinsic or extrinsic knowledge). Conclusions: This study is a comprehensive review of the published methods for auditing the various conceptual aspects of the UMLS. Categorizing the auditing techniques according to the various aspects will enable the curators of the UMLS as well as researchers comprehensive easy access to this wealth of knowledge (eg, for auditing lateral relationships in the UMLS). We also reviewed ontology enrichment and alignment techniques due to their critical use of and impact on the UMLS.

Original languageEnglish (US)
Pages (from-to)1625-1638
Number of pages14
JournalJournal of the American Medical Informatics Association
Issue number10
StatePublished - Oct 1 2020
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Health Informatics


  • Auditing
  • Quality assurance
  • Review
  • Unified medical language system


Dive into the research topics of 'A review of auditing techniques for the unified medical language system'. Together they form a unique fingerprint.

Cite this