Rather than pointing out the shortcomings of statistics the article points out the incorrect use of statistics in science, a problem that professional statisticians themselves have been warning for.
the article wrote:How could so many studies be wrong? Because their conclusions relied on ?statistical significance,? a concept at the heart of the mathematical analysis of modern scientific experiments.
& wrote:Another common error equates statistical significance to ?significance? in the ordinary use of the word. Because of the way statistical formulas work, a study with a very large sample can detect ?statistical significance? for a small effect that is meaningless in practical terms.
This was also concluded from an observation of the only Gauquelin study that (for astrology) yielded positive results.
http://www.astrology-and-science.com/g-hist2.htm under ? "Use of effect sizes"
Geoffrey Dean wrote:Effect sizes are usually expressed as a correlation where 0 = no effect, as between one coin toss and another, and 1 = perfect correlation. Thus if all soldiers had the same sun sign, or if all children had the same aspect as their parents, the effect size in each case would be 1. An effect size involves the whole sample and thus gives a good idea of what is happening, whereas looking at just part of the sample can be misleading. Thus the typical planetary surplus or deficit in key sectors of 10% to 25% may seem like a lot, but the corresponding effect size (which for 40 years nobody bothered to calculate) is only 0.02 to 0.05. Planetary effects are tinier than they might seem.
& wrote:Unfortunately Gauquelin never used effect sizes. Instead he generally presented his findings in terms of statistical significance, which varies roughly as N2 where N = sample size. As N increases, even the most trivial effect will eventually reach astonishing significance.
I'll give a 'gauquelian-like' example: If we want to test the significant house position of Mars from charts of 1200 sports champions, then according to chance 100 champions will have Mars in I, 100 in II and so on. This is a chance of 8,33%. If studies would point out that 122 champions have Mars in I and 98 champions in each of the other houses then the statistical significance of Mars in I is the relation of 122 to the chance value of 100. The statistical significance or the chance that a sports champion has Mars in I will be 22%. This sounds quite impressive but in terns of distribution over the houses this means that 122/1200= 10,17% of the sports champions have Mars in I and 98/1200= 8,17% of the sports champions will have Mars in each of the other houses. This is hardly a significant difference. Although there's an effect it doesn't mean a lot in practical use.
Tom Siegfried concludes his article with the suggestion of the Bayesian approach of the use of previous knowledge included in the statistics. His example ('Box 4: Bayesian Reasoning') on the last page of the article about the drug tests of steroids users in baseball is interesting. The question is how this could be applied in astrology. First the premise will be problematic. In the example a premise of 95% of the accuracy of the drug test is used. Let's imagine other accuracy premises.
- In an extreme way one could imagine a 100% accuracy if the to be tested baseball players are never left alone or permanently left under camera survey. In case of a 100% accuracy the chance that a positive test is really correct is also correct is also 100%. This is alwasy the case; the prior probability will never make any difference.
- One could also consider that the chance that the test is correct is 50%, or the chance of tossing a coin. In this case the chance that a positive test is correct will always be equal to the used prior probability.
- In another extreme way one could think of a 5% accuracy. Imagine that the premise is that everybody whose name is John uses steroids is accurate for 5%. A ridiculous example perhaps but when applied to the example in the article it will lead to the conclusion that the chance that baseball players who are called John use steroids is 1 : 361, fortunately very small.
What with astrology then? Which premise should be used. No book on astrology ever states that the chance that a planet X in position Y means Z is ...%. The indications are usually stated in a way that the accuracy is 100%. Still this leads to the problem how we came to that idea in the first place. The steroid drug test example uses a 95% accuracy. However in the first place som laboratory tests must have been made to develop this test method, for example by using certain substances that will colour blue when exposed to a sample of steroid drugs. The lesser accuracy than laboratory circumstances which is due to the digestion in the human body will lead to the premise that the test will be somewhat less accurate. Anyhow, some previous testing must have taken place to come to the premise in the first place. If this previous (laboratory) research never took place, the premise will be based on nothing and won't provide better than chance or even worse as in the 'John' example.
No previous tests of the astrology characteristics of planets are known of; the meanings of planets and signs were mainly based upon anology and later became generally accepted as fixed rules. However these astrological rules are considered as 100% accurate. The good news is that because of this astrology can be tested through statistics. The bad news is that all tests turned out to be negative.
There are three main possible reactions to these negative results.
- one can conclude that astrology is incorrect and quit the practice;
- one can conclude that statistics are an improper way to test astrology, or;
- one can conclude that characteristics of planets/signs can't or should not be set in rules.
*Apart from a possible personal drama of disenchantment, the first reaction is not a real practical problem.
*The second reaction is a bit problematic in my opinion be cause I tend to believe in statistics. If astrological features are presented as facts then they can be tested. Complete denial could lead to an ostrich attitude. Most often the denial is based upon the meaning that is ascribed to the great merit of astrology. However this invariably leads to the old objective versus subjective discussion or fact versus meaning. The problem with this is that people will be discussing different things and the discussion never ends. Another problem is that different schools can ascribe different meanings to astrological issues, tropical versus sidereal for example. The disbelief in testability leads to the problem that two totally opposite systems can coexist (e.g. Saturn as a benefic and Saturn as a malefic), which is illogic.
*The last reaction could be that all dogma's and schools should be abandoned and just a handful somewhat flexible criteria could be used as premises. Not the premise that planet X in position Y definitely means Z woud count but rather the premis that planet X in position Y could mean A in this individual or B in the other or C under certain circumstances. The problem of this view is that it necessarily leads to vagueness and testing astrology will be very difficult, it becomes nonfalsifiable, and in a far advanced stage astrological technique will become useless from a practical point of view.