We hope this guide will encourage research communities to annotate gene products of their interest to enhance the corpus of GO annotations available to all.
The detection of mentions of entities of the A natomy and M olecule categories can be performed at broadly comparable accuracy on this corpus containing balanced numbers of annotations of the two, suggesting that fine-grained anatomical entity detection is no more difficult than established molecular level entity detection tasks.
With a similar scope, the ADE corpus contains annotations of drug-related adverse effects, covering chemicals/drugs in a therapeutic context for 3,000 abstracts.
> To evaluate the consistency of the corpus annotations, we measure inter-annotator agreement (IAA) over the articles annotated by both annotators.
As a result of these changes, the final numbers reflect a difference in the total number of annotations per corpus, when compared to the original publications.
The performance of these tools has typically been evaluated intrinsically, that is, with respect to a gold standard set of annotations over a corpus of documents.
> -wrap-foot> Recall is particularly low for genes/proteins in the development data set of the CF-Kidney corpus owing to a high number of annotations from a few genes/proteins, which have been missed by the system: 'Gata3' (155), 'Ret' (97) and 'EpCAM' (83).
We choose tmChem [24] and ChemSpot [13], two state-of-the-art tools for chemical NER from scientific articles, and evaluate their performance (without retraining) on all four freely available gold standard patent corpora with annotations of chemical mentions.
However, to search for reported biomarkers, (e.g. genes, proteins, or genetic variations) in a text corpus, additional annotation of these entities and their normalization to relevant databases is required.
The GM corpus (e.g., [ 34]) includes mention annotations but not gene identifiers of the target document; the GN corpus contains annotations for the gene identifiers but not their associated mentions.
