Posted: Thu Jan 23, 2014 11:10 am
Haha, I was gonna suggest what Waybread said (Method 2), but I guess that is no longer necessary 
It is interesting that you suggested a blind experiment, I forgot about that and did not really think it was necessary...but it is good to be careful not to introduce systematic bias if possible.
About the Astro-Databank sample though, can we claim that it is a random sample? I think not. The charts in the database are often those people who are prominent or infamous in some way, and if we want to gather sufficient events to rectify the charts, this may also make it less random (i.e. people whose live events are more known will be more likely to be selected).
Furthermore, we would prefer to work with AA charts for this study, so that further de-randomises the sample.
Also, is the sample representative? That's hard to answer, because the demographic profile of the sample is very heterogenous.
If we cannot have a sufficiently random and representative sample, we might be inclined to strive for a large sample instead. The problem with this is, it may exaggerate small effects.
Which brings us to the fundamental question: how should we construct the hypothesis test? Or do we really need one?
It is interesting that you suggested a blind experiment, I forgot about that and did not really think it was necessary...but it is good to be careful not to introduce systematic bias if possible.
About the Astro-Databank sample though, can we claim that it is a random sample? I think not. The charts in the database are often those people who are prominent or infamous in some way, and if we want to gather sufficient events to rectify the charts, this may also make it less random (i.e. people whose live events are more known will be more likely to be selected).
Furthermore, we would prefer to work with AA charts for this study, so that further de-randomises the sample.
Also, is the sample representative? That's hard to answer, because the demographic profile of the sample is very heterogenous.
If we cannot have a sufficiently random and representative sample, we might be inclined to strive for a large sample instead. The problem with this is, it may exaggerate small effects.
Which brings us to the fundamental question: how should we construct the hypothesis test? Or do we really need one?