Quality assurance of complex ChEBI concepts based on number of relationship types

The Chemical Entities of Biological Interest (ChEBI) ontology is an important reference for applications dealing with chemical annotations and data mining. Modeling errors and inconsistencies in the large and complex ChEBI ontology are unavoidable. The errors can adversely affect applications dependent on it. We present a quality assurance (QA) methodology based on the correspondence between a concept's number of errors and its number of distinct relationship types - an intuitive measure of complexity. Specifically, we hypothesize that concepts with more relationship types tend to concentrate more errors. A study is carried out to assess the hypothesis. Two domain experts reviewed the correctness of a random sample of ChEBI concepts and formed a QA consensus report, which was then reviewed by a ChEBI curator. A two-tailed Fisher's exact test is performed on the consensus report and the curator's report to test the hypothesis. Various kinds of errors, including errors of both a relationship and non-relationship nature, were discovered and reported to the ChEBI's curator, who confirmed and corrected 65.8% of them. Our hypothesis was confirmed with statistical significance for both the domain experts' and the curator's reviews. Thus, ChEBI curators should employ a QA methodology concentrating on concepts with many relationship types.