Sentence examples for token sequence from inspiring English sources

Will people go for this new proposition to buy a token, sequence their own data and then sell it directly?

Table 11 Example of a token sequence tagged with matches against our affix lists.

But with this tool only the token sequence is taken for evaluation.

Table 10 Example of a token sequence tagged with matches against chemical dictionaries.

In this paper, we propose a new similarity detection technique not only based on token sequence matching but also on the factorization of the function call graphs.

Then, general checks (like the match of numbers of opening and closing brackets or the pairwise occurrence of ring closures) are performed and finally the token sequence is analysed in a consecutive manner for its syntactic validity.

Supervised methods classify word (token) sequences by assigning them to one of a set of predefined entity classes.

In the matching phase, the dictionary tagging tool performs the same conversion step on input text and then captures longest possible matches between the normalised token sequences and dictionary entries.

On the other hand, even though the widely used n-gram analysis can bild a full spectrum of token sequences present in the text, it may also produce a great level of noise, making it difficult to use them.

A 'terminology spectrum' of a natural language publication may be defined as an indexed list of tagged token sequences with calculated weights, such as recognized general scientific notions, terms linked to existing thesauri, names of chemical entities and 'term-like phrases'.

In tagging named entities in input abstracts, NERsuite generates the same set of features and assigns begin-inside-outside labels to the token sequences using the trained models.