Misuse of the P value — a common test for judging the strength of scientific evidence — is contributing to the number of research findings that cannot be reproduced, the American Statistical Association (ASA) warns in a statement released today1. The group has taken the unusual step of issuing principles to guide use of the P value, which it says cannot determine whether a hypothesis is true or whether results are important.In the medical and social sciences, p-values rule. Every paper cites them. The p-value determines whether the paper is publishable or not.
This is the first time that the 177-year-old ASA has made explicit recommendations on such a foundational matter in statistics, says executive director Ron Wasserstein. The society’s members had become increasingly concerned that the P value was being misapplied in ways that cast doubt on statistics generally, he adds.
In its statement, the ASA advises researchers to avoid drawing scientific conclusions or making policy decisions based on P values alone. ...
The statement’s six principles, many of which address misconceptions and misuse of the p-value, are the following:
1. P-values can indicate how incompatible the data are with a specified statistical model.
2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
4. Proper inference requires full reporting and transparency.
5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.
Paper do not get retracted for bogus use of p-values, but people raise a storm when the word "Creator" sneaks into a paper. It appears to be just a mistranslation, as the author intended "nature" or something similar.