Posted: Wed Dec 01, 2010 11:47 pm
I liked this article a lot. Some of it seems like common sense, even for quantification-phobes like myself. I think it does have relevance for astrology.
For example, oftentimes scientific studies cannot be safely generalized beyond the limits of the study parameters, yet this happens all the time. For example, a medical study might show that young white males eating guavas show a statistically significant drop in harmful blood cholesterol levels. Yet when you read the study, actually the researchers looked at only a very small sample of men (like maybe 2 dozen) for a very short period of time (like maybe 2 weeks.) Nonetheless, we're out there buying guavas, in the hope of a miracle cure!
Another problem, is that a statistical study itself only shows a correlation between variables at a level higher than chance. If two variables constantly co-varied by the identical amount, you'd have a candidate for a scientific law, not a hypothesis in a statistical test. You normally get all kinds of data points that don't fit the curve or model. Yet the study design really cannot explain the data that don't fit snugly into the correlation. The scientist might attempt to do so, but only by conjecture or referal to other studies that may or may not be a good match with her research. In the above hypothetical example, it would not explain why some of the men in the guava study did not show a drop in triglyceride levels. This is similar to the "well-fed, barking dog" in the article's example.
Some of us have talked a bit about Geoffrey Cornelius, The Moment of Astrology. He argues against wide-spread statistical studies, partially on the grounds that some big studies in the past showed no significant results. A major example was a study of suicides in New York, where neither statistical tests of horoscope placements, nor astrologers' delineations, produced results beyond random chance. He makes the point that a horoscope, like a life, is a complex and highly individualistic thing.
So this is why I think one might possibly get astrology to the point where a really large, sophisticated statistical study of horoscopes might produce meaningful results. But then the problem would be, how do you explain the outliers in the data? If 80% of the data behave as expected, the study is probably still no good in explaining why the remaining 20% got out of bounds, even where the 80% correlation is highly significant.
This might not be such a problem for simple personality delineation. But it would get huge with crunch-time death-clock predictions or financial adivising in astrology.
To quote the author:?Determining the best treatment for a particular patient is fundamentally different from determining which treatment is best on average,? physicians David Kent and Rodney Hayward wrote in American Scientist in 2007. ?Reporting a single number gives the misleading impression that the treatment-effect is a property of the drug rather than of the interaction between the drug and the complex risk-benefit profile of a particular group of patients.?
This is another good argument for astrology's focus on the individual horoscope. To put it differently I might suggest that a woman with Venus in Scorpio exhibits jealousy--except when she doesn't. And when and why she doesn't should be indicated by additional horoscope factors that (hopefully) I am smart enough to detect. Maybe the computer can do a better job of this than I could, but so far, so bad.
Insofar as I understand the "Bayesian" approach to statistics, this wouldn't fare much better, as it opens the door to a lot of pre-trial biases.
This is why one needs a lot of studies to affirm or reject a "scientific fact." A single study can be prone to all kinds of design problems.
For example, oftentimes scientific studies cannot be safely generalized beyond the limits of the study parameters, yet this happens all the time. For example, a medical study might show that young white males eating guavas show a statistically significant drop in harmful blood cholesterol levels. Yet when you read the study, actually the researchers looked at only a very small sample of men (like maybe 2 dozen) for a very short period of time (like maybe 2 weeks.) Nonetheless, we're out there buying guavas, in the hope of a miracle cure!
Another problem, is that a statistical study itself only shows a correlation between variables at a level higher than chance. If two variables constantly co-varied by the identical amount, you'd have a candidate for a scientific law, not a hypothesis in a statistical test. You normally get all kinds of data points that don't fit the curve or model. Yet the study design really cannot explain the data that don't fit snugly into the correlation. The scientist might attempt to do so, but only by conjecture or referal to other studies that may or may not be a good match with her research. In the above hypothetical example, it would not explain why some of the men in the guava study did not show a drop in triglyceride levels. This is similar to the "well-fed, barking dog" in the article's example.
Some of us have talked a bit about Geoffrey Cornelius, The Moment of Astrology. He argues against wide-spread statistical studies, partially on the grounds that some big studies in the past showed no significant results. A major example was a study of suicides in New York, where neither statistical tests of horoscope placements, nor astrologers' delineations, produced results beyond random chance. He makes the point that a horoscope, like a life, is a complex and highly individualistic thing.
So this is why I think one might possibly get astrology to the point where a really large, sophisticated statistical study of horoscopes might produce meaningful results. But then the problem would be, how do you explain the outliers in the data? If 80% of the data behave as expected, the study is probably still no good in explaining why the remaining 20% got out of bounds, even where the 80% correlation is highly significant.
This might not be such a problem for simple personality delineation. But it would get huge with crunch-time death-clock predictions or financial adivising in astrology.
To quote the author:?Determining the best treatment for a particular patient is fundamentally different from determining which treatment is best on average,? physicians David Kent and Rodney Hayward wrote in American Scientist in 2007. ?Reporting a single number gives the misleading impression that the treatment-effect is a property of the drug rather than of the interaction between the drug and the complex risk-benefit profile of a particular group of patients.?
This is another good argument for astrology's focus on the individual horoscope. To put it differently I might suggest that a woman with Venus in Scorpio exhibits jealousy--except when she doesn't. And when and why she doesn't should be indicated by additional horoscope factors that (hopefully) I am smart enough to detect. Maybe the computer can do a better job of this than I could, but so far, so bad.
Insofar as I understand the "Bayesian" approach to statistics, this wouldn't fare much better, as it opens the door to a lot of pre-trial biases.
This is why one needs a lot of studies to affirm or reject a "scientific fact." A single study can be prone to all kinds of design problems.