We did not perform language-specific preprocessing techniques such as word stemming, stop word removal, or case folding, since other researchers found that such techniques tend to hurt spam-filtering accuracy [22, 38, 51].
After the stop words and small words removal steps, words such as "between" and "and" will be removed and the entire corpus is transformed to lowercase.
Higher order n-gram features, stemming or removal of stop words did not improve performance.
For simple syntax-based formats such as XML, parsing of desired data is possible, where for non-standardized formats, such as raw text, pre-processing involves tasks derived from text mining, such as word stemming, stop word removal, and generation of document-term matrix (DTM) [ 11].
The pre-processing techniques applied are: Handle Synonyms; Stop-words removal; Word validation using a dictionary and Stemming.
Another advantage of our proposed method over the work at [19] is that no stop-word removal or other pre-processing of the tags is performed in this paper.
Thus, the scores we present in this article are limited to the F-scores of the ROUGE-1, ROUGE-2 and ROUGE-SU4 without stemming or stop word removal.
It must be noted that sialidase treatment of LNCaP and C4-2B cells; or in other words, the removal of the cell surface sialic acid residues, leads to two cell lines that show a similar tendency to adhere to AsGM1.
In other words, top removal is out.
Hierarchical clustering was used to highlight the structure, and candidate words for removal were chosen, reducing the list to 25 words.
