I am working right now in R with correlation coefficients. I know that we can achieve approximately a Pearson coefficient r=0 and Spearman p=1, because Pearson is sensitive to outliers. However, I was wondering if we could achieve the reverse as well? i.e. r=1, p=0
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ Pedantry corner: I don't recommend the notation p for Spearman correlation. Some people use the Greek letter rho, which perhaps you intend, i,e, $\rho$. For a sample Spearman correlation I recommend $r_S$. But there's no notation that everyone likes. $\endgroup$– Nick CoxCommented Oct 27, 2023 at 16:20
-
1$\begingroup$ Analysts work with fallible measures of dependence like Pearson and Spearman for the simple reason that they don't know anything different and, even when they do know something different, their audiences typically don't, forcing reversion to known fallibilities. There are better alternatives to evaluating dependence in the presence of extreme outliers and/or nonlinearity. A boatload of them are reviewed in Clark's Comparison of Correlation Measures https://m clark.github.io/docs/CorrelationComparison.pdf Of particular interest is Szekely's distance correlation. $\endgroup$– user78229Commented Nov 2, 2023 at 11:51
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Take data for two variables $x, y$ that aren't correlated and add one massively different outlier with much larger $x$, much larger $y$ in the far top right (or equally the far bottom left) of the corresponding scatter plot. Then Pearson correlation will be close to 1 and Spearman correlation will be close to 0.
(This is easier for me to imagine, and closer to common experience with real data, than the opposite case where Spearman is close to 1 and Pearson is close to 0. But take a cluster of points related monotonically and then add four outliers in each corner of the plot. The outliers dominate Pearson, but the Spearman should be higher.)
answered Oct 27, 2023 at 16:18