http://norvig.com/experiment-design.html
When an experimental study states "The group with treatment X had significantly less disease (p = 1%)", many people interpret this statement as being equivalent to "there is a 99% chance that if I do treatment X it will prevent disease." This essay explains why these statements are not equivalent. F…
Hasnain says:
"Prof. Michael Wigler has a more pessimistic way of putting it (quoted by Natalie Angier): "Most of the time, when you get an amazing, counterintuitive result, it means you screwed up the experiment.""
Worth reading. Norvig concludes with
"By now you should see that much can go wrong between the simple statement of "this result is significant at p=1%." and the conclusion about what that really means. As Darell Huff said, "it is easy to lie with statistics," but as Frederick Mosteller said, "it is easier to lie without them." By scrutinizing experiments against the checklist provided here, you have a better chance of separating truth from fiction."
Posted on 2014-04-16T21:36:54+0000