11
$\begingroup$

I posted a question earlier but failed miserably in trying to explain what I am looking for (thanks to those who tried to help me anyway). Will try again, starting with a clean sheet.

Standard deviations are sensitive to scale. Since I am trying to perform a statistical test where the best result is predicted by the lowest standard deviation amongst different data sets, is there a way to "normalize" it for scale, or use a different standard-deviation-type test altogether?

Unfortunately dividing the resulting standard deviation by the mean in my case does not work, as the mean is almost always close to zero.

Thanks

Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ I think you'll get more useful help if you take a few steps back, and ask us what scientific question you are trying to answer. Why are you looking for the smallest normalized SD? $\endgroup$
    – Harvey Motulsky
    Commented Nov 26, 2010 at 16:42
  • $\begingroup$ I agree with Harvey. $\endgroup$
    – NPE
    Commented Nov 26, 2010 at 17:04
  • 4
    $\begingroup$ It's less that the standard deviation is sensitive to scale, more that the standard deviation is a measure of the scale. $\endgroup$
    – onestop
    Commented Nov 26, 2010 at 19:37

3 Answers 3

Reset to default
4
$\begingroup$

If all your measurements are using the same units, then you've already addressed the scale problem; what's bugging you is degrees of freedom and precision of your estimates of standard deviation. If you recast your problem as comparing variances, then there are plenty of standard tests available.

For two independent samples, you can use the F test; its null distribution follows the (surprise) F distribution which is indexed by degrees of freedom, so it implicitly adjusts for what you're calling a scale problem. If you're comparing more than two samples, either Bartlett's or Levene's test might be suitable. Of course, these have the same problem as one-way ANOVA, they don't tell you which variances differ significantly. However, if, say, Bartlett's test did identify inhomogeneous variances, you could do follow-up pairwise comparisons with the F test and make a Bonferroni adjustment to maintain your experimentwise Type I error (alpha).

You can get details for all of this stuff in the NIST/SEMATECH e-Handbook of Statistical Methods.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ In the earlier question, he pointed out that the goal was to compare the SD of raw data with the SD of smoothed data with the SD of highly smoothed data. I don't think the F test (or Bartlett's or Levene's) tests will do any more than complete the circle. The whole point of smoothing is to reduce the variation, so tests that compare variation will tell you smoothed data are smoother. I don't see how this circular logic will get you anywhere. $\endgroup$
    – Harvey Motulsky
    Commented Nov 27, 2010 at 21:22
  • $\begingroup$ Using the same units does not address the scale problem - unless you use different units for every data set, which only recasts the problem as "how should I normalize the units". $\endgroup$
    – einpoklum
    Commented Mar 19, 2017 at 9:03
1
$\begingroup$

How about using the mean of the absolute value, i.e., $$\sigma_\textrm{normalized}(X) = \frac{\sigma(X)}{\mathbb{E}[|X|]}$$?

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for looking at this old question. But considering this formula would throw away all information about standard deviations when all group distributions have the same shape, it doesn't look like an effective suggestion. $\endgroup$
    – whuber ♦
    Commented Mar 9, 2017 at 22:04
0
$\begingroup$

Harvey:

You're absolutely right that the F and Bartlett's tests won't work to compare raw data with smoothed data! Once the data has been smoothed, there's all manner of autocorrelation in there, and the testing becomes much more complicated. Better to compare separate--and hopefully independent--sequences.

Improve this answer
$\endgroup$
0

Not the answer you're looking for? Browse other questions tagged or ask your own question.