Whitespace characters must be retained unaltered as, for some Fortran applications, the exact amount, form and location of such characters can be significant.
Finally, an automatic revision of annotations was done to cross check the mention boundaries, trimming whitespace characters, and ensuring their technical coherence with the annotation rules.
The interpreter reads a line of input from the user input device, which is then parsed for a word using spaces as a delimiter; some systems recognise additional whitespace characters.
These documents had at least 100 but less than 800 non-whitespace characters.
More than 90%% of all reports were covered by a filter that requires at least 800 non-whitespace characters and assumes less than 5 list elements.
We arrived at the filtering settings displayed in Table 4. Less than 5%% of all reports stemmed from one of three sites that were excluded or had less than 100 non-whitespace characters and were mostly defective.
Whitespace, including tabulation characters, multiple spaces and spaces at the start and end of lines should be preserved unaltered unless it is known that the application generating the CML document is insensitive to such changes.
Tokens are split on all whitespace and most punctuation characters, with exceptions for brackets, colons, and other symbols in certain situations to preserve entities such as chemical names as a single token.
For example, the tokeniser implemented in the ChemDataExtractor package [22] overrides some rules in the Penn Treebank policy to better handle chemical entities: Tokens are split on all whitespace and most punctuation characters, with exceptions for brackets, colons, and other symbols in certain situations to preserve entities such as chemical names as a single token.
Only those letters which occur in the English alphabet are affected; numbers, symbols, whitespace, and all other characters are left unchanged.
