Although there are other lexical diversity indices, including those described by Yu (2009) that measure diversity of language in passages of text, the application of Shannon diversity index for comparison of the diversity of words and phrases in this context seemed more appropriate for comparison of single words or phrases in association with a single stimulus word over time.
When we analyzed word frequency and word diversity together, there were approximately the same number of words but a higher diversity of words in the postunit as compared with the preunit, indicating a greater evenness or greater consistency in words and phrases revealed through word association among our student population at the end of the unit.
Furthermore, 8-mers are long enough that there is enough diversity of word choices (~64,000) to reduce false positive results, while retaining sufficient word counts to be statistically informative.
Although the features in each model were different, the most common features across models are rate of negative emotion, lexical diversity, rate of words over six letters, and word count.
In addition, our database is the first to include information about the contextual diversity of the words and to provide good frequency estimates for multi-character words and the different syntactic roles in which the words are used.
The first of these findings is thought to reflect the higher 'contextual diversity' of frequent words: high frequency items occur in a wider range of contexts and are therefore more easily associated with the distracter words in a synonym judgement task, increasing control demands (Hoffman et al., 2011 a ; Almaghyuli et al., 2012).
"We urge them to remember that this beautiful country will be stronger and happier when, and if, it appreciates pluralism, diversity and the freedom of words," they write.
We urge them to remember that this beautiful country will be stronger and happier when, and if, it appreciates pluralism, diversity and the freedom of words.
But that argument ignores the Whitman administration's odd history of racial politics, which is one in which the governor and her staff worked hard to say the right words about diversity issues, words of affirmative action and inclusion, and yet often got the music wrong.
We analyzed word diversity on dimensions of word richness (total number of different words) and evenness (the extent to which the words appear in equal frequency) using a Shannon diversity index.
Apart from the raw frequencies for the word forms, SUBTLEX-PL offers frequencies transformed to the Zipf logarithmic scale, measures of contextual diversity, part-of-speech-specific word frequencies, and frequencies of associated lemmas and word bigrams, providing researchers with necessary tools for conducting psycholinguistic research in Polish.
