For every pair of correlated features (Pearson ρ correlation coefficient larger than 0.8 at a p value smaller than 0.05), the least frequent was discarded, and its frequency was added to the most frequent feature.
For the detection of correlated features between the two datasets, x exp and x seq are first mapped to Hilbert space, H, by function φ.
For each binary combination of correlated features, one was eliminated.
Third, there is additional information in call activity at other times of day that might help distinguish between commercial and residential areas, so the model will need to able look for those signals among hundreds of correlated features.
Remove one of each pair of correlated features (r ≥ 0.98); 13 feature columns removed.
Because importance of correlated features is biased towards their selection order in tree building [ 5], only one of the highly-correlated features, based on Pearson (for linear relations) and Spearman (for non-linear relations) correlation metrics, is used.
Kernel CCA looks for maximally correlated features between x exp and x seq : where λ exp and λ seq are regularization parameters, var means a variance and cov is a covariance between two variables.
~Omics data also needs to be preprocessed for highly correlated features, as the contribution of such features in classification would be wrongly estimated in tree-based classifiers [ 5], and for features with homogeneous values across all observations as they decrease classification accuracy [ 6].
CCA is a classical multivariate statistical method for finding linearly correlated features from a pair of datasets [ 36].
We show the best correlated features for the high-knowledge students, for which we had the most data, in Table 4. Two new features, compared to the features in Table 3, seem to be specific for this group in the fully-adaptive condition: fg15 (percent of articles such as an and the) and fg16percentnt of pronouns that are not references such as you and they).
