Sentence examples for trained human annotators from inspiring English sources

Indeed, the accuracy of these matches in AutoTutor is almost as reliable as trained human annotators (Cai et al. 2011).

This indicates that temporal annotation was less confusing for human annotators than location annotation, and that our schemes for temporal annotation were reliable.

False positive (FP) is the number of recognized chemical mentions that were not annotated by human annotators.

The human annotator annotated the same sentences and we compared the results.

To check reproducibility, we conducted annotation experiments with two human annotators who were familiar with the corpus domain and scientific writing.

Each abstract was annotated by two human annotators for disease mentions, as well as their corresponding concept identifiers in MEDIC (inter-annotator agreement: 87.5%).

Previously, some studies discovered gene regulations by using information from the biomedical literature and most of them require effort of human annotators to build the training dataset.

It is even difficult for multiple human annotators to get the consistent annotation result.

Disagreements between human annotators implicitly indicate hard cases for automatic annotation.

Trained annotators manually annotated each sentence as to whether it included an inclusive pronoun (per the above definition) or not and cross-checked their results to ensure reliability.

In order to study the reliability of the annotation scheme, we analyzed the inter-annotator agreements on a group of human annotators for over 1000 reported events.